We are building the foundation for the safe and efficient use of AI in the Bank. Our team is developing a multi-agent system for autonomous monitoring of all Bank AI agents in industrial operation. This is not just dashboards and alerts – it's an intelligent platform that must understand how and why agents make decisions, predict failures before they occur, and automatically localize root causes.
We offer the opportunity to play a key role in developing a unique system from scratch, upon which the reliability of all the Bank's AI services for millions of people depends.
Responsibilities
- Development, testing, and implementation of AI agents (including LLM-as-a-Judge) and classic ML models for quality assessment, anomaly detection, degradation prediction, and automatic localization of failure causes
- Research and implementation of new approaches in LLM fine-tuning, LLM-as-a-Judge, and RAG to make monitoring more accurate, stable, and understandable.
- Project management throughout the full cycle: from idea and prototype to a working solution in production, its testing, and support
- Designing data (agent traces) and model training pipelines, integrating them into our MLOps ecosystem
- Collaboration with AI agent development and validation teams, and MLOps for solution integration and best practice formation.
Requirements
- Deep knowledge of mathematical statistics, classic ML algorithms, and neural network architectures
- 5+ years of experience in a Data Science / Machine Learning role with full development cycle experience – from research and prototyping to production deployment and monitoring
- Experience in conducting analytical research (R&D): ability to independently study a problem domain, formulate and test hypotheses, select and adapt state-of-the-art methods for project tasks
- Confident proficiency with the technology stack for analysis, experiments, and development: NumPy, Pandas, Polars, Scikit-learn, XGBoost, LightGBM, CatBoost, Optuna, Hyperopt
- Skill in writing clean, modular Python code, understanding of SOLID principles, and experience with Git.
Will be an advantage:
- Experience developing AI agents / agent systems, understanding their operational principles, communication, and orchestration
- Experience working with AI agent traces and metadata (OpenAI, Arize Phoenix, LangSmith)
- Experience with vector databases
- Knowledge of observability stack (OpenTelemetry, Prometheus, Grafana) for ML system monitoring
- Publications or significant contributions to open-source in the field of ML/NLP/LLM.
Conditions
- Comfortable modern office: Moscow, Kutuzovsky Prospekt metro station
- Work format – office-based; hybrid format is possible for a portion of tasks upon agreement with the manager
- Annual salary review, annual bonus
- Corporate gym and relaxation areas
- Over 400 educational programs from SberUniversity for professional and career development
- Adaptation program and manager support at the start
- Extended voluntary medical insurance (DMS), preferential insurance for family members
- Employee mortgage program
- Free SberPrime+ subscription, discounts on products from partner companies
- Referral bonus for recommending friends to the Sber team.
