LinkedIn just announced a new search architecture that it has already implemented, moving away from Lucene, upon which its early search engines were built, to Galene.
LinkedIn’s Sriram Sankar and Asif Makhani explain, “Around a year ago, we decided to completely redesign our platform given our growth needs and our direction towards realizing the world’s first economic graph. The result was Galene, our new search architecture, which has since been implemented and successfully powering multiple search products at LinkedIn. Galene has helped us improve our development culture and forced us to incorporate new development processes. For example, the ability to build new indices every week with changes in the offline algorithms requires us to adopt a more agile testing and release process. Galene has also helped us clearly separate infrastructure tasks from relevance tasks. For example, relevance engineers no longer have to worry about writing multi-threaded code, perform RPCs, or worry about scaling the system.”
Search is obviously an important component of LinkedIn, so the better its search capabilities get, the better the service will be able to help people find the right jobs, companies, groups, people, and content with it opening up its publisher platform.
The search platform is being used to power People Search, Job Search, and LinkedIn Recruiter, as well as some of the company’s internal applications like customer service and ad support.
Sankar and Makhani get into the technical aspects of migrating to the new architecture in a lengthy blog post on the company’s Engineering blog.
“Search is one of the most intensely studied problems in software engineering,” the two write. “It brings together information retrieval, machine learning, distributed systems, and other fundamental areas of computer science. And search is core to LinkedIn.”
LinkedIn recently announced that it has over 300 million members.
Image via LinkedIn