Developer of the General Search System

We are building the Search of the future.

Our team is starting to build a large-scale project that will define the technological development of Yandex Search for the coming years.

As strange as it may seem, the company currently does not have a general search system. Web search (Yandex's main search, available at yandex.ru/search) differs in its implementation from image and video search. Search in Yandex Eda, Music, or Market is also developed largely independently. The main development vector has always been set by web search, with the others only catching up. Over time, it has pulled far ahead: improvements fail to reach other services in time, and the technology gap continues to grow.

The company has already tackled the task of creating a unified search. The internal SaaS project (Search-as-a-Service) provides a convenient interface for solving simple search scenarios and is used in hundreds of the company's services. However, key search projects are still developed independently, and SaaS is not flexible enough and remains in the role of catching up.

Given the experience accumulated in the company, today we understand the search task so well that we are ready to formulate a general solution. Our goal is to develop a new search system that will allow us to generalize and unify all search and recommendation scenarios. We are making it so that both web search and other searches within the company develop in a single environment on top of a common technological base, rather than in parallel with each other.

The task is ambitious and complex. We are currently working on the design of the next-generation SaaS and implementing an MVP. Now is the very moment to join the project at its start and influence its further development as a whole.

What do we consider a good solution?

• Conceptually similar to external products, for example Elasticsearch, but on a larger scale and with greater modularity.
• Supporting various index formats: full-text, key-value, trie, N-gram, kNN, and others.
• Allowing a search query to be described in a relatively simple form, for example in an SQL-like language.
• With a low entry barrier, with the ability to launch a search on your own data practically without writing code.

We view search systems as a type of database, but with weakened consistency guarantees, which allows for significantly higher performance.

What tasks await you

Research and Design You will get acquainted with existing search projects and participate in designing the architecture of a new distributed fault-tolerant search system.

Low-Level Development You will be engaged in developing and implementing various search indexes, compression codecs, and compact data structures.

Development of Efficient Runtime and Indexing Our installations store tens of petabytes of data, consume hundreds of thousands of CPU cores, and process hundreds of millions of queries per second.

More about backend at Yandex — in the channel Yandex for Backend

We expect you

• Have a good command of the C++ language and want to write in it
• Know classical algorithms and data structures
• Have a deep understanding of multithreaded programming
• Have used and developed distributed systems
• Understand how search systems work and how search indexes are structured internally (will be an advantage)