Hybrid
IT & Data Development
Position
We are looking for a Senior Data Engineer with a strong Python and SQL background, who can design reliable data pipelines, work with large volumes of data, and make independent technical decisions.
Responsibilities
- Develop and maintain data pipelines and DAG processes in Airflow
- Prepare data for analytical and business tasks
- Work with ClickHouse and MySQL, optimize queries and analyze their execution plans
- Integrate with partner services and external data sources via API and FTP
- Design and develop microservices for data processing
- Work with high-load systems processing terabytes of data
- Participate in architecture development and independently propose technical solutions to improve performance and scalability
Requirements
- 5+ years of commercial Python development experience
- SQL knowledge and query optimization experience
- Experience with ClickHouse and/or MySQL
- Experience in designing Python frameworks and libraries, profiling and performance optimization
- Experience in developing and working with OLAP systems
- Practical experience with Airflow, Pandas, Kafka and/or RabbitMQ
- Experience in obtaining and processing data from APIs and FTP
- Experience working with Docker and Kubernetes
- Independence in finding and choosing technical solutions
- Ability to understand the business context of tasks and effectively interact with the team
Benefits
- Highload products processing tens of thousands of requests per second and serving an audience of 100M+ users worldwide
- Medical insurance in the countries where the company operates
- Annual budget of $2,000 after the probation period for medical insurance, language learning, and sports
- Partial compensation for professional training, courses, and participation in industry conferences
- Access to the corporate e-library
- Weekly online consultations with a psychologist at the company's expense
- Opportunity for relocation to countries where company offices are located, with relocation support after the probation period
