Description
About the Product
B2B Digital Twin - Sber's AI service that helps companies make data-driven decisions. The service analyzes financial metrics, geo-context, and the client's market environment to assess the impact of business initiatives on the client company's key metrics.
Tech Stack: Python, FastAPI, GigaChat API, PyTorch. The product is in production, active development phase.
Responsibilities
- development and enhancement of the LLM agent: prompting, context handling strategies, branching logic and multi-step scenarios
- integration of new data sources into the agent pipeline (schemas, validations, passing into context)
- building and labeling golden datasets for answer quality assessment; regression testing
- designing the architecture of multi-step LLM calls (tool use / retrieval / post-processing)
- writing tests, debugging prompts and scenarios.
Requirements
- mandatory - student, currently enrolled in full-time study at a university (bachelor's or master's degree), technical major
- python - proficient, OOP; neat, readable code
- git + basic engineering discipline (branches, PR, review)
- practical understanding of how LLMs work (context/tokens, generation parameters, typical errors and how to reduce them)
- RAG experience (end-to-end): chunking, embeddings, vector search, context gathering and answer generation; basic quality assessment of retrieval/answers
- experience working with data: pandas and/or SQL, REST API (understanding client-server interaction)
- gitHub is mandatory: link in the application; presence of pet-projects with code, willingness to delve into business logic, not just code.
Will be a plus
- fine-tuning language models on PyTorch
- reranking, hybrid search, selection and comparison of embedding models
- LLM evaluation tools: golden set, auto-checks, LLM-as-a-judge (as an approach), pairwise comparisons
- Docker/CI, production experience with FastAPI.
Conditions
- paid internship
- internship duration 3 months (20-40 hours per week)
- comfortable, modern office: Moscow, Kutuzovsky Prospekt 32, bldg.1
- in-office internship format, Mon-Fri from 9:00 to 18:00, flexible schedule.
