One of the largest companies developing technological solutions for the entertainment industry is looking for a strong Data Engineer to develop a modern data platform and high-load data processing systems.
Responsibilities
- Design, develop, and maintain reliable data pipelines and DAG processes;
- Prepare data for analytical, product, and business tasks;
- Work with large volumes of data, optimize SQL queries and storage performance;
- Develop ETL/ELT processes and participate in the design of DWH, data models, and marts;
- Integrate with external sources and services via API, FTP, and other data transfer channels;
- Design and develop Python services, libraries, and tools for data processing;
- Participate in the development of the data platform architecture and independently propose technical solutions to improve reliability, performance, and scalability.
Requirements
- Strong knowledge of SQL: complex queries, working with large volumes of data, performance analysis and optimization;
- Confident proficiency in Python: experience writing production code, libraries, services, and tools for working with data;
- Understanding of DWH construction principles: data layers, models, marts, ETL/ELT, and data quality control;
- Practical experience in building and supporting data pipelines;
- Independence in finding and selecting technical solutions;
- Ability to understand the business context of tasks and effectively interact with the team.
Experience with the following will be an advantage:
- Airflow — DAG development and support, scheduling, retries, backfill, monitoring;
- dbt or similar data transformation tools;
- Spark and distributed data processing;
- analytical DBMS and OLAP/MPP systems: ClickHouse, Greenplum, Vertica, and others;
- Kafka / RabbitMQ;
- Iceberg, Trino;
- Docker, Kubernetes, CI/CD;
- Data Mesh approaches;
- open-source projects or own technical projects.
Conditions
- Work with high-load products, terabytes of data, and an audience of 100M+ users worldwide;
- Flexible work format: remote, hybrid, or from the office;
- Extended relocation package, including relocation for the specialist and their family members;
- Voluntary Medical Insurance with dental coverage after the probationary period;
- Compensation for sports, English language, and sessions with a psychologist;
- Compensation for specialized training and the opportunity to share expertise within the team;
- Corporate library, team events, masterclasses, and other internal activities;
- High level of autonomy and the opportunity to directly influence technical solutions and the development of the data platform.
