Python Developer at Yandex DataLens

DataLens is a cloud service for data visualization and analysis. It is widely used both within Yandex and externally as part of the Yandex Cloud platform.

The service allows connecting to many different data sources, builds abstractions on top of them in the form of datasets, and provides tools to display this data in the required format. Input: raw data, output: beautiful and convenient dashboards.

The service core is developed in open source under an open license.

The DataLens backend solves the following tasks: * Working with multiple data sources (databases, external APIs, online tables, uploaded files) and providing a unified interface for retrieving heterogeneous data * Generating complex SQL queries on the fly: we provide an easy-to-use formula description language, which behind the scenes is transformed into optimal queries considering the specifics of a particular data source * Background processing of data from files and online tables using task queues * Ensuring data and infrastructure security: individual operations are isolated in specialized execution environments

To learn more about the service, you can watch a short promo video or a more detailed demo.

We are looking for experienced Python developers. We welcome those who are proficient in other languages and are ready to switch to Python. We have a lively and friendly team, we formulate goals and make decisions together. Join us!

About the team

Subscribe to the Telegram channel Inside Yandex Cloud to learn more about our team and technologies!

What tasks await you

Build a flexible API for solving data analysis tasks, optimize data model construction Datasets are a way to combine large amounts of data into a model that, regardless of the data source, allows querying it thanks to a universal API. Datasets must work quickly with any number of fields: this is achieved through various optimizations and several levels of caching. In addition to developing the dataset API, we also enrich the service with other features that improve the user experience (these include, for example, exports, alerting, public API).

Develop connectors for connecting to new sources We are constantly expanding the set of available connectors to data sources to cover more analytics scenarios. The task of connecting each new source is unique in its own way: it is necessary not only to understand the intricacies of its operation but also to design the interaction with the connector from the user's perspective.

Work on scaling, stability, and service performance Our team is responsible for all stages of the service lifecycle, from design and development to deployment and maintaining the operability of all environments. We ourselves support both DataLens inside Yandex—one of the largest installations among similar tools—and DataLens in Yandex Cloud—the largest cloud analytics and visualization service in Russia. Optimizing CI/CD pipelines, improving monitoring tools, developing diagnostic tools without direct access to the installation and user data—there is always room for relevant infrastructure tasks in our plans.

Develop the service core in open source to open new possibilities for its extension So that anyone can add a new connector or other functionality to the service, assemble and configure the system according to their requirements, it is necessary to maintain and develop the modular code architecture: identify common parts, providing extension points.

More about backend at Yandex — in the channel Yandex for Backend

We expect you

• Have developed in Python or are ready to switch to it from another language
• Have worked with relational databases
• Have built and maintained fault-tolerant systems

Will be a plus

• Have worked with Flask, AIOHTTP, SQLAlchemy
• Are familiar with Docker, Terraform, Kubernetes
• Have worked with other clouds
• Are interested in data analysis and visualization and have worked with other BI systems