Inference Engineer

Responsibilities — Develop and optimize the core inference stack, including low-level compute kernels. — Add support for new model architectures, modalities, as well as various hardware/software backends. — Develop advanced inference capabilities such as KV cache paging, continuous batching, quantization. — Create tooling around the inference stack to make the system easier to test, understand, and reliably deploy to production.

Requirements — Strong experience in low-level systems engineering — you should understand how computers work at a level below application-level programming. — Proficient in one of the systems languages: Rust, C, C++, or similar.

Will be a plus — High-quality open-source contributions. — Experience writing high-performance GPU kernels or working with Metal, Vulkan, CUDA, and similar technologies.

Contact - recruiter contact

Mirai is an AI lab developing a foundational model and inference infrastructure for on-device AI, enabling LLMs to run locally with ultra-low latency. 📱 Today, the AI economy scales poorly through cloud computing and GPU inference. Over time, a significant portion of AI workloads will move from servers directly to user devices. 🔝 The company was founded by the founders of Prisma labs (one of the pioneers of on-device ML) and Reface (100+ million users), and has also raised $10 million in a seed round. Fully remote, distributed team (from London to Tokyo), English is the primary working language.