Inference Engineer
We are looking for an Inference Engineer at Tochka Bank. We are a fintech company, creating an online bank and an ecosystem of services for businesses. Our team has over 6,000 employees, and our services are used by over 800,000 clients.
About the Team
You will be working in the internal LLM platform development team. Our focus is on technical infrastructure and core tools. We are developing an LLM gateway, internal LLM inference, an Information Retrieval system, and guardrails. These components form the basis for working with neural networks.
Essentially, we are building the technological foundation for LLM automation within the bank. Our task is to ensure everything runs stably and with high quality. Therefore, the tasks will be technical, with minimal interaction with the product.
We work closely with inference engines: not just running them, but configuring them for high load and patching the source code when necessary.
Responsibilities
- Develop a high-performance inference layer. Package ML models into optimal services: configure tokenization, dynamic batching, GIL and asynchronous operation, optimize GPU and CPU resource utilization.
- Work with deployment. Configure CI/CD, work with Kubernetes. Also, develop solutions for deploying services that work with ML models, including LLMs.
- Maintain service stability. Work with technical debt, conduct code reviews, set up detailed monitoring and alerting, maintain architectural documentation.
- Drive R&D. The LLM industry is constantly changing, so it's necessary to research new optimization methods and tools to ensure the platform always uses best practices.
Qualifications
- 4+ years of experience with Python and asyncio, as well as aiohttp and FastAPI.
- Enjoy working at a low level: you are interested in understanding how tools work under the hood and optimizing their performance.
- Comfortable with the fact that code may be frequently changed or discarded in the pursuit of the best solution.
- Comfortable working in conditions of uncertainty and rapid experimentation.
- Ability to take responsibility for results and communication.
- Independently organize your work and determine the direction based on transparent team goals.
- Experience with ML infrastructure and ML services. Understand the basic principles of LLM operation, their limitations, and capabilities.
Will be an advantage:
- Experience with vLLM or SGLang as an engineer, not just a user. For example, understanding the principles of KV-cache operation and the difference between the prefill phase and the decoding phase.
- Experience with GPU optimization. For example, writing GPU Kernels in CUDA or Tritonlang/Tilelang.
What awaits you at Tochka Bank
- Official salary up to 700,000 ₽. The exact amount depends on your skills and expectations – we will discuss them at the interview. This is the amount before taxes. Remember that from 2025, the personal income tax rate changes depending on the total income for the year.
- Five-day work week with flexible start and end times.
- Remote work or any office in our presence cities. Our offices are well-designed open spaces with comfortable workstations and relaxation areas. You can choose where to work independently – coming to the office is not mandatory.
- A clear system for development and growth by grades.
- Opportunity to influence and make changes. If you have a suggestion or an idea, feel free to share it with the team – you will be heard.
- Free training: you can attend IT conferences, meetups, and hackathons, and take courses at the company's expense. You can also use our library and online course platform.
- Comfort not only at work but also in life. After the probationary period: voluntary medical insurance with dental care, a psychologist, health check-ups, and 4 additional days off per year.
- Partner bonuses: Skillbox, Skyeng, Yandex Practicum, Yasno, and others.
- Online employee portal where you can order a certificate, request vacation or day off, order equipment, or view payments.
- Active corporate life: we organize movie screenings, sports marathons, and corporate events that are remembered for a long time.
