City and office address: Kaliningrad

Work format: remote, hybrid, office

Employment: full-time

Salary range: from 170,000 gross to 230,000 gross

Hiring specifics:

• remote work only from cities in the Russian Federation
• 2 interview stages

Vacancy description:

Responsibilities:

• Deployment, maintenance, and updating of LLM inference platforms (vLLM, SGLang, Ollama);
• Operation and development of AI platform in Kubernetes, including scaling and GPU resource management;
• Configuration of AI Gateway (LiteLLM): request routing, API keys, quotas, limits, and access policies;
• Optimization of inference performance: batching, KV Cache, semantic cache, reducing latency and inference cost;
• Configuration of monitoring, logging, and tracing (OpenTelemetry, Langfuse, SigNoz, Prometheus/Grafana);
• Automation of CI/CD processes for infrastructure and models, support for GitOps approach;
• Integration and maintenance of memory services, vector databases, and RAG/Agentic AI components;
• Diagnosis and resolution of incidents, ensuring stability and high availability of the LLM platform.

What we expect from a new team member:

• IMPORTANT! 4+ years of experience in a DevOps role;
• Experience in Kubernetes administration in production, Docker, Helm, understanding of GitOps approaches;
• Experience operating LLM inference servers (vLLM, SGLang, Ollama, or TGI) and understanding their architecture;
• Good understanding of LLM principles: tokens, KV Cache, batching, streaming, quantization, RAG;
• Practical experience with GPUs (NVIDIA/CUDA), resource allocation, and performance optimization;
• Experience configuring monitoring and observability (Prometheus, Grafana, OpenTelemetry, SigNoz/Langfuse);
• Solid knowledge of Linux, Docker, network communication, and infrastructure problem diagnosis;
• Automation skills in Python and experience with CI/CD (GitLab CI/GitHub Actions);
• Experience with Redis, PostgreSQL, and API gateways;
• Knowledge of LiteLLM, KEDA, Qdrant/Milvus will be an advantage.