Developer for the YQL Team

Our group is part of Platform Engineering at Yandex. We develop YQL, a SQL-like language for data processing. YQL is the foundation of the YQL service: the entry point for analysts and data transformation processes across all Yandex data storage and processing systems, such as YTsaurus and YDB. YQL enables efficient processing of huge volumes of data, providing a convenient syntax for complex queries, including support for PostgreSQL-compatible constructs. And thanks to integration with Apache Arrow and JIT compilation, YQL significantly speeds up computations. YQL is a key part of the infrastructure that enables data analysis at Yandex.

Our primary language is C++. It is used for the YQL core, including parsing with ANTLR and compilation via LLVM. We use Java to develop the service's REST API and JDBC client, ensuring integration with external systems. We use Python for tests and API client libraries to automate functionality verification.

Working on YQL will require you to understand formal grammars, relational algebra, and graph algorithms, as well as have backend development experience. You will improve the codebase, which is partially open-sourced in the YDB and YTsaurus projects, accelerating computations and enhancing service stability. You don't need to know all the technologies from the start—but the interest in learning them is very important. Join us if you want to work with cutting-edge tools and contribute to big data processing infrastructure.

What tasks await you

Development and Support You will improve query parsing and processing, optimize the functional language's virtual machine, including implementing JIT compilation via LLVM to accelerate code execution. The tasks also include integration with Apache Arrow to improve the performance of runtime computations on large data volumes. This is work on the system core, where every optimization directly impacts the service's speed and stability.

You will also develop and support complex algorithms related to relational algebra and graphs, ensuring queries are executed efficiently even under high load. Among the tasks are configuring and improving the YQL backend, developing client libraries and APIs. You will need to analyze performance, identify bottlenecks, and propose solutions to eliminate them. This is an excellent opportunity to dive deep into systems programming and working with big data infrastructure.

Testing and Debugging Write and maintain unit and integration tests for existing and new code, covering key functional blocks with tests (pytest). Diagnose and fix bugs identified during testing and in the production environment. Optimize the testing infrastructure to speed up the testing process and improve product quality.

More about Backend at Yandex — in the channel Yandex for Backend

We expect that you

• Are proficient in C++ and are ready to develop high-performance systems (e.g., the YQL core) using LLVM
• Understand the basics of systems programming and are interested in code optimization to increase query and computation speed
• Have a basic understanding of formal grammars, relational algebra, or graph algorithms or are ready to master these areas
• Are not afraid of complex tasks and strive to deeply understand technologies such as Apache Arrow
• Can work with different parts of the stack: from C++ backend to Java APIs and Python tests, adapting to the project
• Are ready to analyze performance, identify bottlenecks, and propose how to eliminate them
• Are interested in big data and infrastructure services, even if you don't yet have experience in this area
• Are proactive, capable of independently finding approaches to solving non-trivial tasks
• Are ready to contribute to our open-source platforms YDB and YTsaurus and maintain the stability of a critical service
• Can work with feedback, respond quickly to questions, and participate in on-call support duties
• Don't know everything right away—but strive to grow and deepen your experience in compilers, optimizations, and data processing