System Analyst for Sber's "Data Insight" service to build Data Lineage. The task is to conduct a PoC of alternative solutions for Apache Atlas, collect requirements and describe the transformation of messages into Openlineage format.
Tasks:
- Analysis of the current solution (Apache Atlas) and collection of requirements for the new one
- Description of the logic for transforming messages from Apache Atlas format to Openlineage format
- Validation and verification of the correctness of Data Lineage collection on the new solution
- Interaction with development teams and Big Data architects
Requirements:
- At least 3 years of experience in system analysis
- Excellent knowledge of SQL, ability to write complex queries for data analysis
- Knowledge of the Hadoop stack (understanding of HDFS, Spark, Hive operation)
- Experience with data formats: json, xml, xsd, wsdl
- Understanding of integration principles: REST / SOAP
- Knowledge and practical application of UML diagrams, EPC and BPMN notations
Conditions:
- Rate: up to 210,000 rubles net per month
- Grade: Middle+ / Senior
- Load: Full-time
- Format: Remote
- Location/Citizenship: RF / RF
- Outstaffing position, candidates with experience relevance of 85% or more are considered
Skills:
- SQL
- Hadoop
- HDFS
- Spark
- Hive
- Apache Atlas
- Openlineage
- Data Lineage
- json
- xml
- xsd
- wsdl
- REST
- SOAP
- UML
- EPC
- BPMN
- Oracle
- PostgreSQL
- Greenplum
- Data Governance
