Data Engineer

Remote | Alfa-Bank

Company: Alfa-Bank

🔹 What you will do

• Implementation of high-load data processing pipelines to ensure reliable and uninterrupted data replication from the Bank's IT systems;

• Implementation of complex tasks for data preparation in target analytical storages (DataLake, SandBox, FeatureStore) to build features required for machine learning model development;

• Development and maintenance of up-to-date documentation for implemented functionality;

• Timely reflection of task execution status in Jira;

• Checking code quality (code review) written by data engineers and junior data engineers.

🔹 Our expectations from candidates

-Python - confident knowledge of data structures and algorithms, effective application of OOP and FP (Functional Programming) principles, experience writing unit and integration tests, knowledge and experience using data processing and analysis libraries - numpy, pandas;

• Experience in developing and deploying into production services for loading and processing unstructured and semi-structured data (text, xml, json) from external sources;

• Ability to understand data provider APIs using available documentation;

-SQL - ability to create complex queries using analytical window functions and use profiling tools to optimize their performance, experience working with Oracle, Postgres, Greenplum databases;

• Confident knowledge and experience with development, scheduling, and monitoring tools (workflow engines) for batch data processing - Airflow;

• Experience in developing complex, high-load data processing applications based on PySpark, confident knowledge of Spark settings and their impact on Spark application performance.

Contacts: job.alfabank.ru/vacancies/moskva/remote-job/inzhener-dannyikh_31892