Data Engineer (RecSys team)

We are looking for a Data Engineer for the recommendations domain who will build pipelines and develop the data platform together with us. In our team, colleagues grow quickly and gain experience in various tasks — from classic ETL for analytical data marts to real-time streaming processing for ML.

Data-driven processes are well-developed at Okko: a large number of A/B experiments, sources, and data marts make the Data Engineer's work especially important. Your contribution is truly visible here: colleagues value the results of your work, and your decisions directly impact the quality of analytics and the effectiveness of ML models.

There are currently two Data Engineers in the recommendations team, and we are looking for colleagues to maintain the current development pace, as well as to develop data streaming processing and feature store functionality.

What you will do:

• support and development of batch ETL pipelines on Airflow for analytical data marts and ML;
• streaming data processing using Spark Streaming, Kafka, MongoDB;
• development of feature store as a tool, populating it with features and integrating data for ML models;
• documentation, optimization, and monitoring of pipelines and data streams.

Our stack:

• ClickHouse
• Airflow
• Spark (Spark Streaming, PySpark, Spark SQL)
• Kafka
• MongoDB
• Hive
• S3
• DataHub

Requirements:

• confident knowledge of Python and SQL;
• experience in building ETL processes using the listed stack;
• will be a plus: experience with CI/CD, Docker, Grafana, basic knowledge in the field of ML (model training and inference, working with feature store).

Conditions:

• Necessary equipment and software for work.
• Official employment;
• Voluntary medical insurance with dental care, an on-site doctor, sick leave compensation, corporate discounts;
• Preferential mortgage terms within the salary project;
• Free subscription to partner services:
• Corporate sports;
• Rich corporate life;
• Electronic library of the MYTH publishing house, which includes almost 2,000 content items on business, self-development, healthy lifestyles, and other relevant topics.