Company:** Sber. IT
Location: Moscow, RU
Employment type: FULL_TIME
We are a team responsible for the uninterrupted delivery of data to an analytical platform based on Greenplum and ClickHouse. Our work is the foundation on which analysts and data scientists build their models and make key business decisions. We are looking for a colleague to help us maintain and develop this data pipeline, automate processes, and monitor information quality.
Responsibilities
- Ensure data delivery: load new data marts into Greenplum and ClickHouse Lab using our internal tools. Create your own tools for data delivery.
- Monitor and optimize: track the status of ETL processes (in Airflow and proprietary solutions), respond to failures, identify and eliminate "bottlenecks" in data pipelines to speed up loading.
- Work with Data Quality: develop and implement quality checks at all stages, analyze the causes of errors and fix them.
- Automate routine tasks: write Python scripts to automate repetitive data processing tasks.
- Maintain documentation: develop and maintain technical documentation and a knowledge base for our processes.
- Participate in R&D: test new technologies and tools to improve our platform.
Requirements
- SQL: confident proficiency: complex queries, window functions, CTEs, ability to read execution plans for optimization.
- Python: programming skills, experience with the Pandas library for data processing, writing scripts for routine automation.
- Theoretical foundation: understanding of relational DBMS principles, DWH/data mart architecture, and ETL/ELT processes.
- Tools: experience using task schedulers (Airflow or similar).
- Proactivity: desire to develop in the field of data engineering, learn new things, and solve complex problems independently.
- Communication and teamwork: ability to not only solve tasks independently but also interact effectively with other teams.
- English: fluent reading of technical documentation.
- Skills in working with generative AI models; experience
- creating AI agents and using them in work will be an advantage.
- Experience using GigaChat, Kandinsky, and similar tools in products, skills in creating and using AI agents.
- Instrumental use of AI for analysis, generation, and automation.
Conditions
- Office format (Poklonnaya 3k1)
- Annual salary review and annual bonus
- Extended voluntary medical insurance and preferential insurance for family
- Sber's unique training system for professional and career development
- Favorable mortgage for employees
- Prime subscription with the possibility of shared use by three close individuals
- Reward for recommending friends to the Sber team
- Corporate pension program.
