A Scholar

You must be wondering why I am writing about Google Scholar, now that all relevant people possible know about the service. The reason is that I met the architect of Google scholar today, Anurag Acharya, and an IIT Kharagpur alumnus. He was in the institute and agreed to deliver a talk to the students.

He first introduced us to Google search and the technology behind it, the challenges in various stages such as URL redirection, dynamic pages, data replication and load balancing during crawling, the subsequent generation of indices, the serving of the query by thousands of computers connected in a cluster in strategically located data centres all over the world. He rightfully claimed that it was perhaps the largest distributed data system in operation.

He went on to demonstrate the usefulness of Goolge Scholar, which is mainly aimed at researchers. Acharya describes it as payback to the universities from which Google initially emerged. He also anticipates another class of users who might be seeking expert knowledge in fields such as medicine.

He also elaborated on the difficulties faced during the creation of Google Scholar, convincing publishers to part with data about research papers, crawling them and mining useful information out of them, indexing them on the basis of relevance and the number of citations of each paper, and subsequently using google's search system to display results. There were several challenges involved since Scholar has to throw up very specific results, and there should not be any duplication of papers on the results page. The challenges involved identifying author, keywords, citations etc. on each page, which is no mean task since scientists tend to be sloppy at times and also because the same paper may be published in many places.

However, I really admired the magnificient set up Google has created. It is based on normal personal computers running on Linux, and is extensible, fault-tolerant, and really really fast. It is amazing to even come to know the number of processes which run in parallel behind the scenes. A very inspiring lecture.

---

I broke my resolution today because of the lecture.

Current Mood: Inspired
Current Music: Hum hain iss pal yahan, from Kisna

Visit Google scholar at http://scholar.google.com