Data Engineer (Middle / Senior)
REMOTE GLOBAL
We are developing an IT product that includes a large-scale WEB platform and an interactive Telegram bot / Mini App.
Our stack: Python, Airflow 2+, dbt Core, ClickHouse, PostgreSQL, Docker, GitLab.
The product is growing rapidly, and we are now looking for a Data Engineer to help us scale the analytical infrastructure and make data processing fast and reliable.
Your main task is to develop and maintain ETL/ELT processes, integrate new data sources, develop the ClickHouse storage, and provide analysts with quality data for decision-making.
What you will do:
- Develop and maintain ETL/ELT processes based on Airflow and dbt Core
- Integrate new data sources: PostgreSQL, external APIs, and services
- Develop the existing ClickHouse data warehouse and maintain analytical marts
- Optimize SQL queries and pipeline performance
- Support and develop the analytical platform infrastructure (Docker, Airflow, dbt)
- Interact with analysts to implement new marts, metrics, and data sources
- Participate in incident investigations related to data quality and pipeline performance
- Maintain technical documentation and participate in knowledge sharing within the team
Our requirements:
- At least three years of experience as a Data Engineer
- Solid knowledge of SQL and experience working with analytical DBMS
- Practical experience with ClickHouse: table design, engine selection, partitioning, query optimization
- Experience in developing and maintaining ETL/ELT processes: ability to independently choose a data source integration method, design data loading structure, storage, and updates, considering business requirements and technical constraints
- Experience with Apache Airflow 2+: development, maintenance, and debugging of DAGs, understanding of scheduling mechanisms, dependencies, and pipeline monitoring
- Experience with dbt Core
- Good knowledge of Python: ability to understand existing code, modify and develop new components for integrations and pipelines
- Experience with PostgreSQL and understanding of incremental data extraction specifics
- Experience integrating external REST APIs
- Understanding of containerization principles and experience with Docker / Docker Compose
- Ability to independently decompose tasks and bring them to completion
Will be a plus:
- Experience with Airflow 3+
- Experience with BigQuery
- Experience with Google Analytics / GA4
- Experience with cloud platforms (GCP, Yandex Cloud, AWS)
- Understanding of basic blockchain network principles and transactional data
- Experience using AI tools for code development and maintenance
Work format: remote
Time zone: MSK
Send your resume to: katyacheresh
