Middle/Middle+ Data Engineer (Python / PySpark / Airflow / dbt / Data Vault 2.0)

Vacancy ID: 92467_SS Rate: to be discussed. Grade: Middle / Middle+. Workload: Full-time. Engagement Term: 24 months with the possibility of extension. Format: Remote. Time Zone: UTC+3 (Moscow time). Location/Citizenship: Russian Federation. ⏳ 🚨 URGENT: Vacancy is active until 10.27 inclusive until 17:00! 🚨 📌 Important: candidates with at least 85% relevant experience are considered (outsourcing position). 🏢 About the Project: a major retail chain. Building, developing, and supporting modern ETL pipelines and a corporate data warehouse. 🧩 About the Role: we are looking for a confident Data Engineer who will become a reliable link in building robust and scalable data flows. The role involves working with a modern stack (PySpark, Airflow, dbt), supporting the Data Vault 2.0 methodology, and ensuring high data quality. 🚀 Key Tasks: — Creation and development of ETL pipelines using Python and PySpark. — Support and enhancement of DAGs in Apache Airflow: log analysis, typical failure troubleshooting, backfill execution. — Development of dbt models, as well as accompanying tests and documentation. — Maintenance of Data Vault 2.0 entities (in terms of Satellites). — Interaction with S3 storage (via boto3) and relational/columnar DBMS. — Performing basic data quality checks: NULL value control, key uniqueness verification, and type correctness. — Recording technical metadata in DataHub and tracking pipeline status in monitoring systems. 🎯 Mandatory Requirements (Hard Skills): Experience and Languages (CRITICAL!): — At least 2 years of experience as a Data Engineer. — Confident proficiency in Python: syntax, OOP fundamentals, writing ETL scripts using ready-made libraries, debugging skills. — Knowledge of PySpark: working with DataFrames API, understanding the difference between Actions and Transformations, basic understanding of Spark architecture. Databases and SQL (CRITICAL!): — SQL skills: building JOINs, using aggregations and window functions, basic understanding of index operation. — Experience with DBMS: PostgreSQL, ClickHouse, Greenplum, Oracle, MySQL (understanding of relational model and ACID principles), basic NoSQL knowledge. — Understanding the differences between OLTP and OLAP, specifics of row-based and columnar storage, as well as key principles of Data Vault 2.0. Tools and Orchestration (CRITICAL!): — Experience with Airflow: DAG configuration, schedules, operators and sensors, working with Connections and Variables, configuring retries and alerts. — Proficiency in dbt: creating models, sources, using refs and Jinja, configuring materializations, seeds, and tests. — Working with Git and GitLab CI: branch management, creating MRs, participating in code reviews, reading .gitlab-ci.yml configurations. — Basic Docker skills: creating Dockerfiles, local build and container launch. Monitoring and Quality: — Ability to work with monitoring tools: reading dashboards in Grafana, Sentry, Kibana. 💡 Will be a significant plus: — Experience with Data Vault 2.0 in practice (especially Hub, Link, Satellite). — Understanding the specifics of the retail domain and large volumes of transactional data.

📲 Contact for application: recruiter contact (⚠️ DEADLINE: 10.27 until 17:00! When applying, please immediately send your current resume with an emphasis on experience with PySpark, Airflow, dbt, and understanding of Data Vault. Please mention the vacancy ID: 92467_SS)