ABOUT THE COMPANY
Higgsfield AI is the fastest-growing AI company in history:
- $1B annual revenue (ARR),
- 35M+ users worldwide in 18 months after launch.
Our goal is not just to be successful, but to grow faster than anyone else, to be the first to create and adopt the best solutions and playbooks.
Only winners and self-sufficient specialists feel comfortable in our team. If you want to take on a lot of responsibility and see tasks through from start to finish – join us.
📍 Almaty · Office-based work · Relocation (if needed)
3+ years of commercial development. Remote work is not provided.
ABOUT THE TEAM
The Data Engineering team at Higgsfield AI is responsible for collecting, processing, and structuring large-scale datasets to create new, unique products for the world. We work with content from numerous external sources and process hundreds of millions of media objects and petabytes of video, images, and audio, closely collaborating with Infrastructure and ML teams.
RESPONSIBILITIES
- Lead data collection and processing projects: source integration, media transformation, data pipelines, and large-scale distributed processing.
- Design and maintain high-load media and metadata ingestion systems with reliable scheduling, prioritization, and fault tolerance when working with multiple independent external sources.
- Develop flexible integrations with sources, extract structured metadata from diverse and semi-structured content, and quickly detect format changes on the source side and recover from them.
- Expand source coverage: connect new sources and evaluate their volume, quality, and relevance.
- Deploy distributed systems at scale to process petabytes of data.
- Design and build production services in Python for concurrent high-load processing with separation of CPU-bound, I/O-bound, and GPU-bound pipeline stages.
- Build fault-tolerant pipelines with retries and backoff, checkpoints, and correct recovery after failures or degradation of individual stages.
- Ensure pipeline reusability and idempotency through checkpoints, content-based deduplication, and recovery without full reprocessing.
- Develop and maintain backend services for storing datasets, intermediate artifacts, and metadata, including large-scale object storage and relational metadata stores under load.
- Deploy solutions in Kubernetes with auto-scaling based on queue depth and correct operation on spot instances, conduct regular system checks.
- Configure monitoring and instrumentation of services using Prometheus, Grafana, and VictoriaLogs, analyze performance, identify and eliminate bottlenecks.
- Design systems with efficient use of compute resources, storage, and network, optimize loads for throughput and cost.
- Collaborate with ML and product teams on the composition, coverage, and quality of datasets.
🎯 YOUR TASK
Design and maintain high-load systems for distributed collection, processing, and structuring of large-scale datasets. Real responsibility for results.
🧠 WHO WE ARE LOOKING FOR
3+ years of commercial development. Python/Go, distributed systems, high loads, and Kubernetes. Middle/Senior level professional who is not afraid to challenge corporations and top AI labs.
📌 WE OFFER
- Competitive salary in USD + stock options
- A lot of compute power
- The strongest ML team in the CIS
- Tasks that are being solved in the best labs in the world.
REQUIREMENTS
- 3+ years of commercial development experience.
- Proficient in Python and experience in building production services that operate under constant load.
- Deep understanding of distributed systems and large-scale data processing, including message queues, at-least-once semantics, idempotency, and backpressure.
- Experience in building systems for large-scale data ingestion from external sources and APIs, including working with changing data formats.
- Experience in processing large-scale multimodal media datasets.
- Understanding of high-performance data transfer principles and network loads.
- Proficient in Kubernetes and understanding of Infrastructure-as-Code concepts.
- Willingness and interest in new approaches and technologies.
- Ability to work on multiple tasks simultaneously and adapt to changing priorities.
WILL BE A PLUS
- Experience with large-scale data pipelines, self-hosted storage, or media processing pipelines.
- Experience working with an ML team on preparing and processing media datasets.
WE OFFER
- ⚡ Abundant compute resources – more than anyone in the CIS and Europe can offer.
- 🔬 Opportunity to solve revolutionary problems with the strongest ML team in the CIS.
- 💵 Competitive base salary in USD, considering experience, skills, and role level.
- 📈 Participation in the company's stock option program.
- ✈️ Relocation support to Almaty for candidates from other cities and countries.
- 🎯 Real responsibility for results.
- 🚀 Zero bureaucracy in the workflow.
💬 Telegram — Yerzat Dulat (Co-founder, I will respond personally)
