RocketHuntSmart job search
Reach out directly about this role
By company
3 years
Experience
Full-time
Employment
Data Analytics
Specialization
IT & Tech
Industry
Corporation
Company Type
Analyst for the Offline Metrics Team of Yandex Images
Yandex Images is a service used by millions of people daily to search for visual ideas, products, and to solve everyday tasks. The heart of our search quality is the Image Proxima metric. It is built on assessments from real people and machine predictions (relevance, image quality, aesthetics, etc.) and serves as a 'beacon' for all ML ranking algorithms.
Our team has a wide scope of responsibility and many ideas on how to become better. We are looking for a strong analyst who will help bring these ideas to life and make Yandex Images even more useful.
Improving offline metrics You will be responsible for the entire 'measuring instrument' of Search. You will need to understand the structure of offline metrics and help improve their quality across the entire pipeline — from assessor ratings of individual signals to the final metric form. On this path, you will not only work with assessor ratings but also implement automatic labeling using modern LLM and VLM models. You will research, formulate, and quantify problems in the metric and final ranking, devise ways to fix them, and implement those fixes.
Problem discovery You will need to analyze what users are searching for, identify problematic segments where our responses are poor, and devise ways to improve them. Your insights will directly drive the development of the service.
Assessing metric quality and its changes Together with the team, you will assess how sensitive and stable our metrics are. You will need to devise methodologies for measuring improvements where it's difficult to do so and explore the boundaries of applicability of current models.
More about analytics at Yandex — in the channel Yandex for Analytics