☁️Position: Data Engineer (ETL Migration) (Senior)
🏙Company: «HuntTech»
💪Experience: from 5 years
💰Salary: Range: 230 - 280k RUB
📅Work Format: Remote
📍Location/Citizenship: Russian Federation
📅Employment: Full-time
☎️Contact: recruiter contact
📌Project Description:
- Migration of an telecom operator's ETL system from Oracle/Informatica to Apache stack (S3, Airflow, NiFi, Kafka, Greenplum);
- Transfer of business logic from Informatica to high-performance SQL code;
- Writing and optimizing SQL for data marts and ETL processes;
- Developing DAGs in Airflow;
- Configuring pipelines in NiFi (sources: Oracle, PostgreSQL; sinks: S3, file archives);
- Developing Spark jobs (PySpark) for large volumes;
- Integration with Kafka for stream processing;
- Configuring monitoring (Grafana/Prometheus) and tracking metrics;
- Analyzing data quality incidents and loading errors;
- Code review.
🔺Requirements:
- Experience in ETL system migration (preferably from Oracle/Informatica to Apache stack);
- Deep SQL: window functions, complex joins, hierarchical queries;
- SQL optimization for highload (partitioning, indexes, explain plans);
- Oracle (PL/SQL) and PostgreSQL;
- Greenplum and/or Clickhouse;
- Designing architectures on Apache Spark (PySpark), NiFi, Airflow;
- Hadoop (HDFS) and Hive;
- ETL monitoring: Grafana/Prometheus or Victoria Metrics;
- Experience with billing systems (preferably telecom domain).
➕Will be a plus:
- Stream processing: Kafka Streams / Spark Streaming;
- Informatica PowerCenter (understanding of the migrated system);
- Binary storage formats in Hadoop;
- Differences: Hive vs Impala, Greenplum vs PostgreSQL, Clickhouse vs PostgreSQL;
- Patroni for PostgreSQL/Greenplum clustering;
- S3 storage;
- CryptoPro or secure data transmission channels.
🔥If you like everything - write to recruiter contact
