Hadoop Developer

IT Company GNIVC is a partner of state-owned companies and leaders of Russian business, a developer and system integrator of the largest state information systems, as well as commercial solutions for tax monitoring.

• The company is among the TOP-100 best employers in the country and ranks 9th in the "IT companies" category for 2025 according to the hh.ru employer rating among large companies;
• We are in the top 25% for happiness level among companies in the IT industry and Russia for 2025 according to Happy Job;
• We have an AI sandbox — an environment for experiments and real projects on modern open-source large language models. Here you can apply AI to optimize your work, automate processes, and implement your own ideas from hypothesis to result;
• We are an accredited IT company

We are looking for an experienced developer for big data processing projects with the potential to grow into a team lead in Big Data.

The tasks include development and support of distributed data processing processes based on Apache Spark (Java/Scala) with integration into the Hadoop ecosystem (HDFS, Hive, HBase), as well as interaction with Oracle/Postgres systems and streaming data via Kafka. Work is carried out in the ADH cluster.

• Development and support of Big Data on a large project;
• Development and maintenance of ETL processes and streaming pipelines on Spark (Java/Scala);
• Integration with data sources and storage: HDFS, Hive, HBase, Oracle, Postgres, Kafka, ClickHouse;
• Writing SQL queries;
• Optimization of Spark job performance and loading/unloading processes;
• Participation in architecture design and implementation of best practices for Big Data development;
• Performance optimization at the cluster level (Spark tuning, partitioning, broadcasting)

• Commercial development experience in Java and Scala: 5+ years of development experience, including 2–3 years in Big Data;
• Practical experience with Apache Spark;
• Knowledge of the Hadoop ecosystem (HDFS, Hive, HBase);
• Experience with Kafka;
• Good knowledge of SQL;
• Experience with relational DBMS;
• Confident Linux skills;
• Deep understanding of Spark internals (RDD/DataFrame API, shuffle, Catalyst Optimizer);
• Performance optimization experience;
• Understanding of distributed systems principles (HDFS, Zookeeper, YARN);
• Experience interacting with business customers/analysts (ability to translate requirements into solution architecture)

• Flexible work formats: option to work in the office, on a hybrid schedule, or fully remotely within the Russian Federation;
• Working hours: five-day working week (Mon–Thu from 09:00 to 18:00, Fri from 09:00 to 16:45);
• Decent remuneration: competitive salary based on interview results, as well as bonuses for effective work and achieved results;
• Official employment: full compliance with the Labor Code of the Russian Federation, including paid leave (with an additional payment of 50% of the salary after 11 months of work in the Company) and salary payment twice a month;
• Health care:
  • Compensation for sick leave up to 7 days with full pay equivalent to a working day,
  • Voluntary medical insurance (VMI) upon completion of the probation period, with a wide range of leading medical institutions, including quality dentistry,
  • Reimbursement of up to 50% of sports expenses;
• Development and training:
  • Professional training and certification at the company's expense,
  • Organization of internal and external meetups, hackathons, conferences, seminars, and training sessions,
  • Partnership programs for learning foreign languages and developing professional skills from Skyeng and Skillbox,
  • Access to the corporate library on the Alpina Digital platform;
• Additional days off: option to take 5 additional paid days off (resource days) during the calendar year (from January 1st to December 31st) for employees who have worked in the company for more than 11 months