ML Developer for Autonomous Vehicle Sensor Simulation Team

In autonomous vehicle development, safety is paramount. Making a car stay in its lane under normal conditions is not difficult. The challenge is ensuring it behaves correctly in a wide variety of rare or dangerous situations: for example, with unpredictable behavior from other road users, in poor weather or lighting conditions, and while accounting for equipment performance characteristics. Reproducing such situations in reality is expensive and dangerous, which is why it's important to be able to model them using simulation technologies.

Within the sensor simulation workstream, there are two major tasks: sensor simulation as part of an end-to-end testing simulator for software, and generating synthetic data with rare edge cases for fine-tuning the core algorithms of autonomous vehicles. Both tasks involve generating data from various sensor types: cameras, lidars, radars. The tasks include research elements: reading papers on generative models and 3D reconstruction, reimplementing and improving methods, conducting experiments. In all tasks, we use industrial-grade machine learning: building ML pipelines and processes, collecting datasets and metrics that best reflect the business objective.

What tasks await you

Improving generation models You will enhance the quality of models for 3D scene reconstruction (we are actively considering Gaussian Splatting) and neural networks used both as part of the reconstruction pipeline and as standalone methods for data generation. Among the latter are diffusion models, VAEs, and networks for object segmentation and detection.

To improve quality, you will need to implement methods from academic papers, refine architecture, optimize training and inference, as well as build quality assessment pipelines and improve the dataset: collecting, filtering, and preprocessing data.

Integrating models into the simulator You will need to integrate models into the production simulator. It's necessary to build the process so that models receive the correct data, their inference works on a large cluster, and the simulations are useful for all teams working on autonomous vehicle development. A related team assists with the backend part, but you will need to oversee the overall process and ensure that the actual inference metrics of the models do not differ from the expected metrics obtained during training.

More about ML at Yandex — in the channel Yandex for ML

We expect you

• Have a good understanding of methods for solving computer vision tasks using neural networks
• Are proficient with classical algorithms and data structures
• Can program in Python
• Have experience with PyTorch
• Are capable of teamwork
• Are responsible and organized

Will be a plus

• Have practical experience training diffusion models or 3D reconstructions using 3DGS, NeRF, etc., and have built production processes involving these technologies
• Have been involved in 3D graphics, 3D rendering, or actively pursue this as a hobby
• Have programmed in C++
• Have participated in ML or programming competitions: ACM ICPC, Codeforces, Topcoder, Kaggle, etc.