At SimbirSoft, over 1500 developers tackle interesting challenges daily that make the lives of millions of users more comfortable.
If you're interested in complex technical tasks, working in a team of professionals, and the opportunity to enhance your expertise on large domestic projects from various business sectors – join us!
We are looking for a talented Prompt Engineer who will work on interesting, high-load projects.
What you will do:
- Prototype, test, and manage the quality of LLM-based solutions;
- Configure model behavior (without fine-tuning) — experiment with prompts (CoT, Tree-of-thoughts, few-shot, structured outputs, tool calling, etc.), context, model parameters, and LLM pipeline architecture (single prompt, prompt chain, etc.);
- Participate in dialogue scenario design with product managers;
- Test and evaluate the quality of LLM solutions at all stages — collect and regularly update golden sets, validation samples, create metrics and benchmarks, configure LLM-as-a-Judge;
- Develop instructions for annotators, participate in annotation (including independently annotating golden sets);
- Analyze logs, traces, and feedback on the actual performance of assistants in production and improve their quality based on identified insights;
- Integrate LLMs into products with developers.
Requirements:
- 1-3 years of experience in solving real business problems using LLMs (prompt chains, structured outputs, conversational assistants);
- Good Python skills, experience with analytical tools (Jupyter, pandas, matplotlib);
- Experience with SQL, data handling skills, ability to collect datasets and perform analytics;
- Experience with OpenAI API and its alternatives;
- Prompt testing and regression check skills, understanding of approaches to evaluating LLM systems (offline metrics, human evaluation, LLM-as-a-Judge, A/B testing).
Will be a plus:
- Experience using automatic prompt optimization algorithms (GEPA, etc.);
- Experience organizing the annotation process, building golden datasets, and automating evaluations;
- Experience using validation frameworks and observability tools (DeepEval, LangFuse, MlFlow, LangSmith, etc.);
- Experience with LLM orchestration frameworks (LangGraph, LangChain, or similar);
- Experience building complex agent systems, RAG.
We offer:
- Work in a team of experts
- Wide technology stack, hundreds of projects.
- Flexible work schedule that allows you to get enough sleep and dedicate time to hobbies.
- Opportunity to develop IT solutions for various industries, try new technologies.
- We have established remote work processes, while also having spacious offices with co-working areas where you can comfortably socialize, play board games, or quietly contemplate work tasks.
- Opportunity to grow in all desired directions: become a team lead, architect, or full-stack developer.
- Developed mentorship system, company-sponsored certification, conference participation, English language learning.
- Active knowledge sharing, internal and external development meetups, hackathons, presentations on hard and soft skill development.
- 50+ interest clubs and professional communities within the company.
- Corporate knowledge base and social network for easy communication with each of the 1500+ employees.
