#vacancy #job #Senior #Data #engineer #BigData/#Lakehouse #Java #Python #Spark
❤️Senior Data engineer
salary: 280k - 300k RUB net
Format: Remote work from RF, RB.
Company: Omega Solutions
Project: Retail
☀️Requirements:
- General understanding of the Lakehouse technology stack;
- Understanding the difference in operation between BigData/Lakehouse and data of normal size;
- Knowledge of SQL (indexes, functions, optimization, performance profiling);
- Knowledge of programming languages (JAVA, Python);
- Experience with relational databases (Oracle, Postgres, MySQL, MsSQL, etc.);
- Ability to work with Git (knowledge of git pull/commit/push commands);
- Experience with DBT, Cosmos, Ni-Fi;
- Experience developing on Spark;
- Skills in using Hadoop ecosystem components: Yarn, Ranger, Zookeeper, Hive metastore;
- Understanding of Trino specifics;
- Understanding of data formats Iceberg, Parquet, Avro;
- Understanding how to work with Minio or any other S3-based storage;
- Experience using project management and documentation systems;
- Experience developing non-standard integrations (including SAP systems);
- Experience developing near-realtime streams (Flink, Debezium);
- Experience optimizing high-load streams (billions of incremental records) using Observability tools (grafana, victoria metrics, zabbix).
👁Tasks:
- Development, implementation, and support of integration flows, as well as data mart assembly flows using the technology stack adopted by the team: Trino, Iceberg, S3, Spark, Apache Airflow, Kafka, Cosmos, Flink;
- Development of non-standard integrations (including SAP);
- System development of data processing tools (reusable templates);
- Development of system design for complex solutions (C2/DD);
- Prompt response to information about problems in the area of responsibility, execution of standard tasks within the established timeframe;
- Keeping documentation of standard integration solutions for the big data platform up-to-date;
- Providing activity reports to the manager.
🤩Conditions
Employment via sole proprietor registration
Friendly atmosphere within the company
Development through real tasks, not formal courses
Opportunity to propose ideas and see them implemented
Flexible work schedule and remote work option
Payment for actual time worked
Participation in interesting projects without unnecessary micromanagement
💼Contract: @CodeVal
