Multimodal Models Analytics Developer

The Visual Generative Models Analytics team, at the intersection of NLP and computer vision, is creating a visual-text generative model that will solve user tasks involving images. We develop tools for comparing models, create datasets for benchmarks, and for training models.

We are looking for talented analytics specialists who are ready to join our team.

What tasks await you

Creating tools for assessing model answer quality Our primary task is to learn how to assess the quality of our model's performance on various classes of tasks. For this, we adapt open-source benchmarks, create our own, and utilize the help of performers who handle data labeling.

Researching growth points To improve quality, it is necessary to find the model's weak spots: classes of requests or properties of requests where we are not performing well enough.

Creating new and improving existing datasets used to train the model Having good tools for quality assessment and understanding what exactly needs improvement, we gather data for quality growth. This can include data for both SFT and RL.

Learn more about analytics at Yandex — in the channel Yandex for Analytics

We expect that you

• Have been involved in analytics for over a year
• Are proficient in Python and SQL, know the main libraries
• Know the basics of machine learning
• Are ready to design labeling tasks for remote performers

It will be a plus

• Are familiar with Toloka or other crowdsourcing platforms
• Have created data labeling processes using remote performers
• Have worked with products in the field of computer vision