Description
At SberData, we are building a data platform for the entire Sber. This encompasses over 350 data sources and 100+ Petabytes of information, data ordering and delivery within 15 minutes, and a modern technology stack for data operations, including proprietary DBMS builds based on Hadoop and Greenplum. Our team develops tools and implements the separated Compute & Storage technology within the Lakehouse concept. Our goal is to design and implement a reliable, secure solution for high-load data mart calculations on modern Compute (Spark, Flink) and Storage (S3) technologies, operating on 10,000+ servers.
Responsibilities
Tasks:
- Development and evolution of the architecture of a corporate Lakehouse-platform based on OpenShift for Spark and Flink technologies, taking into account information security and reliability requirements.
- Participation in the design (specifications) of server hardware together with the infrastructure unit
- Collection and systematization of workload requirements, profiling of usage scenarios - batch, streaming
- Collection and analysis of technical requirements, integration interactions, regulatory requirements (DCB).
Requirements
Requirements:
- Development/design experience: 7+ years in Data / Platform / Distributed Systems in large information systems (1000+ servers)
- 5+ years with Kubernetes / OpenShift in production
- spark in k8s or similar.
Experience with the following will be a plus:
- Lakehouse
- Analytical OSS DBMS ClickHouse / StarRocks / Trino / Greenplum
- Iceberg or similar
- S3 storage, HDFS, Hive Metastore
- Skills in working with generative AI models; experience in creating AI agents and using them in work.
Conditions
We offer:
- Comfortable modern office: Danilovsky Fort Business Center, near M.Tulskaya, Verkhnie Kotly
- Opportunity to choose a convenient schedule – office/hybrid
- Annual salary review, annual bonus
- Corporate gym and recreation areas
- More than 400 educational programs from SberUniversity for professional and career development
- Extended voluntary health insurance (VHI), preferential insurance for family and corporate pension program
- Flexible mortgage discount, equal to 1/3 of the Central Bank's key rate
- Free subscription to SberPrime+, discounts on products from partner companies
- Reward for recommending friends to the Sber team.
