Offline Ranking Metrics Analyst-Developer in Search

The primary Proxima page quality metric is built from assessor ratings on individual signals — relevance, source authority, and others. Proxima sets the target for search algorithms ranking documents for a query. We are constantly improving the key metrics of Search, and the offline metrics team currently faces numerous challenges.

What tasks await you

Improving offline metrics You will need to understand the structure of offline metrics and help improve their quality across the entire pipeline — from assessor ratings of individual signals to the form of the final metric. You will research, formulate, and quantify problems in the metric and the final ranking, devise ways to fix them, and implement those fixes.

Optimizing offline metrics on important product slices You will research what users need and how the Search product should look on specific slices (for example, on product queries), and adapt metrics to improve search results for users.

Evaluating the quality of the metric and its changes Together with the team, you will seek answers to questions on how to correctly measure improvements in the metric or its individual components, as well as how to assess how far the current metric is from ideal.

We expect that you

• Have analyzed data using Python or SQL
• Possess the fundamentals of probability theory and mathematical statistics
• Can solve non-standard mathematical problems
• Are capable of calmly and logically defending your point of view and listening to others

Will be a plus

• Love mathematics
• Have participated and won in olympiads in mathematics, physics, programming
• Have applied mathematical statistics in work
• Can read and understand specialized texts in English
• Possess basic knowledge in machine learning