Data Engineer
Our analytics infrastructure team helps analysts do their job. We ensure data delivery to a unified data warehouse for analytics and support tools for its processing and presentation.
There are many self-sufficient teams in the company. Each team has its own services, its own developers and analysts. So we have enough data streams of the most diverse formats and volumes. This includes booking data that the entire company looks at, and each record is checked almost manually, as well as ticket data with a volume of over a terabyte per day. This requires different approaches from our side.
- Our platform is based on LakeHouse architecture principles. Our stack: Spark, Trino, AWS S3, DeltaLake format, DBT, Kafka, Airbyte, Apache Airflow, DataHub.
- Testing and building are done on a CI server (Jenkins). Terraform is also present. Deployment to Kubernetes (EKS in AWS). We do not handle server configuration and database deployment. A separate infrastructure team does this.
Our friendly team is looking for a junior data engineer who will be focused on tasks related to maintaining the operability of the data platform and the company's main product in the near future.
What needs to be done
- actively participate in the team chat: track requests, promptly respond to questions and tasks during working hours;
- monitor systems and resolve emerging incidents together with on-duty data engineers;
- optimize current data processing processes;
- modernize the current data platform together with the Core data engineering team.
What we expect from you
- data engineer experience of 1 year or more;
- knowledge of programming languages: Python/SQL (required), experience with other languages will be a plus.
- understanding of ETL/ELT process principles and experience with tools
- understanding of how distributed computing works
- experience with our stack: Apache Spark, Apache Airflow, DBT, Trino will be a plus;
- independence and responsibility, ability to communicate with different teams and people
- desire to develop and build processes around yourself.
How we work
- from anywhere in the world: we are not tied to a location, we pay in dollars, we love to travel;
- no bureaucracy: convenient, healthy processes, horizontal and open communication, quick discussion of ideas and decision-making;
- compensation: voluntary medical insurance, psychotherapy or foreign language courses, sports activities, and sick leave;
- on a paradise island: you can work from Phuket with an ocean view.
