Senior Analyst-Developer for the AI Productivity team (LLM, agents, RAG)

We are creating top-notch solutions for the AI Productivity of the data analyst profession, which includes an AI IDE with multi-tools interaction / AI YQL-codegen, a testing bench (we are building something similar to LangFuse / [mlflow/kubeflow/clearml], but focused on evaluating code generation quality).

We are looking for a strong colleague to join our team for the position of Senior Analyst-Developer with experience in LLM, AI agents, RAG.

It's great with us because: * we work at the intersection of AI and big data — these are the two most in-demand areas at the moment; * our products are immediately seen by thousands of users inside Yandex, they directly influence business metrics; * we provide the opportunity to work with a modern tech stack and influence architectural decisions; * the team has strong specialists in ML and data engineering from whom you can learn a lot; * your work will influence the development of key products of Yandex Search and Advertising; * we are creating AI tools for AI products — a meta-task of the highest level of complexity.

What tasks await you

Working with business users It is necessary to understand the business context and real needs of people.

Developing AI agents for working with data You will create and fine-tune LLM models for understanding the subject area and generating correct analytical queries. This includes working with YQL, CHYT, and other internal Yandex technologies.

Creating systems for evaluating AI generation quality and enriching the existing dataset We are developing complex benchmarks and metrics for assessing the accuracy of generated queries, checking their syntactic correctness and semantic adequacy.

Integrating AI into products of the Search and Advertising block Your solutions will directly influence the effectiveness of working with advertising data, the quality of analytics, and the speed of decision-making in key business areas.

Working with large volumes of data labeling You will participate in creating datasets for training and validating models.

Experiments with advanced neural network architectures We have ambitions to train some non-standard neural network architectures from scratch. Of course, there is also a part with fine-tuning more standard models.

Mentoring and teamwork Management of junior colleagues, creating strategic partnerships at the middle management level.

We expect that you

• Are meticulous and structured in writing code, write code like a senior backend developer
• Have a deep understanding of machine learning, have practical experience with LLM and know evaluation methods
• Can program in Python and have experience with ML frameworks (PyTorch, TensorFlow)
• Know SQL and are ready to understand the specifics of YQL and internal Yandex technologies
• Possess analytical thinking and can formalize complex business requirements
• Are independent and ready to take responsibility for the full cycle of AI solution development
• Can work in a team and convey complex technical concepts to colleagues

Will be a plus

• Are well-versed in NLP
• Have had experience in AI-specific MLOps
• Have worked with the guidance library or similar
• Have engaged in prompt engineering and fine-tuning of LLM, incl. tools calling
• Have worked on finetuning LLM with proven quality improvement
• Have experience with distributed computing and big data
• Have participated in developing product AI solutions