Analyst-Developer of Offline Metrics at Yandex Pictures

The primary quality metric for pictures, Corsa, is built from assessor ratings on individual signals: relevance, source authority, and others. It sets the target for search algorithms ranking pictures by query in Search. We are constantly improving the key metrics of Search, and the team currently faces many challenges.

What tasks await you

Improving offline metrics You will need to understand the structure of offline metrics and help improve their quality across the entire pipeline: from assessor ratings of individual signals to the form of the final metric. You will investigate, formulate, and digitize problems in the metric and the final ranking, think up and implement ways to fix them.

Automation You will research and integrate models, such as Visual Language Models (VLM), Deep Semantic Similarity Model (DSSM), and others, to automate the data labeling process, which is traditionally performed by assessors. You need to understand the current labeling process, you will search for suitable models to replace manual work, and also work on the economic efficiency of the process. Be prepared to study, identify, and solve problems arising from model implementation, find optimal solutions, and implement them, thereby increasing the accuracy and speed of data processing.

Evaluating metric quality and its changes Together with the team, you will search for answers to questions on how to correctly measure improvements in the metric or its individual components, as well as how to assess how far the current metric is from the ideal.

More about analytics at Yandex — in the channel Yandex for Analytics

We expect you to

• Have analyzed data with Python or SQL
• Have a grasp of probability theory and mathematical statistics basics
• Be able to solve non-standard mathematical problems
• Be capable of calmly and reasonably defending your point of view and listening to others

Will be a plus

• Love mathematics
• Have participated and won in mathematics, physics, programming Olympiads
• Have worked with statistical programs
• Read and understand specialized texts in English
• Possess basic knowledge in machine learning