Developer in Arc VCS

Our group is part of Platform Engineering at Yandex.

The vast majority of Yandex's code resides in a huge monorepository. To work with it comfortably, our team created the internal version control system Arc VCS. It is similar to the well-known Git, but Arc is far more scalable and capable of working with repositories whose sizes exceed tens of terabytes. To display such volumes of code, we use file system virtualization mechanisms. To correctly and quickly show the history of file changes, we apply complex algorithms and indexes. For daily handling of a load of 200k RPS — our own easily scalable in all directions microservices. Therefore, to develop and maintain Arc, you need to be a very broad-based specialist.

What tasks await you

Service stabilization When writing Arc, we consciously cut corners in many places. Now Arc is no longer a young project, but a very important service for the company. It's time to raise clusters for different consumers, implement rate limiters, improve caching, etc.

Implementation of complex algorithms and indexes Many basic Git algorithms have long been implemented in our system as well, but their improvement is a continuous process. For example, a recently implemented history index allowed us to speed up the analog of git log in Arc by three times — and there are still many candidates for improvement.

Development of a virtual file system You will have to get your hands dirty with inodes, figure out cache invalidation in fuse, and learn how to make a checkout in a 300 GB working copy in seconds.

Development of a custom workflow for users Git set the standard for working with commits, branches, and PRs, but we find it overly complex. We want to preserve the familiar Git-like flow — while giving users an alternative that will significantly simplify development.

More about backend at Yandex — in the channel Yandex for Backend

We expect you to have

• Developed in C++ and love this language
• Understand good service development practices

Will be a plus

• Understand the principles of building modern version control systems
• Engaged in system programming for Linux, Windows, macOS
• Developed high-load distributed systems