Bitmanager.ai - An AI startup in one of the fastest-growing categories of conversational AI agents.
We are building a platform for voice and text AI agents in customer communications: sales, support, consulting, and debt collection.
Our agents work with data, access internal company systems, invoke tools, and conduct dialogues within defined business scenarios.
We are developing multilingual voice solutions and independently fine-tuning models to expand the number of supported languages and improve product performance.
We are currently looking for someone to take on one of our key areas - the entire data preparation system for AI/ML models.
The main outcome of this work will be consistently high-quality data and a scalable methodology for its preparation for different models, languages, and projects.
This is not a classic project manager role. We need someone who understands how data composition, volume, and quality affect model performance, can independently design processes, and is responsible for the final outcome.
What you will be doing
- Responsible for data quality and methodology for AI/ML projects;
- Establish a full data lifecycle: collection, sourcing, procurement, preparation, annotation, verification, and acceptance;
- Collaborate with ML engineers to define requirements for the composition, volume, and quality of datasets;
- Select the optimal operating model: in-house team, contractors, crowdsourcing, freelance, or a hybrid approach;
- Build a team, attract annotators, linguistic, and subject matter experts;
- Assign tasks to external contractors and monitor deadlines, quality, and results;
- Develop instructions, annotation standards, evaluation criteria, and data acceptance rules;
- Implement quality control mechanisms and continuously improve them;
- Verify datasets acquired and received from partners before using them for model training;
- Analyze data errors and model performance, identify systemic causes of problems, and adjust the approach;
- Automate manual stages of annotation, verification, and acceptance;
- Coordinate work across different languages and domains;
- Transfer successful processes between projects and set priorities considering quality, deadlines, and available resources;
- Work closely with ML engineers, linguistic specialists, and product teams.
What is important to us
- Experience in building or significantly re-architecting data preparation and annotation processes for AI/ML projects;
- Experience with multiple stages of the data lifecycle: collection, procurement, annotation, verification, and dataset acceptance;
- Ability to independently design processes and choose an operating model considering quality, speed, cost, and available resources;
- Practical understanding of how data composition, volume, and quality affect the training and performance of AI/ML models;
- Experience in building quality control systems, developing instructions, and training contractors;
- Experience managing annotators, contractors, experts, or distributed teams;
- Ability to identify systemic causes of problems and improve processes, not just fix individual errors;
- Ability to independently lead an area, make decisions under uncertainty, and be responsible for the final outcome;
- Ability to work at the intersection of data operations, ML, and people management;
- Familiarity with Label Studio, Toloka, Scale AI, or other annotation platforms;
- Good Russian and English language skills for working with the team, documentation, and contractors;
- Willingness to work from our office in Moscow.
Experience with a specific language or speech data is not required. The ability to build processes, understand methodology in depth, and quickly master new data types is more important to us.
Bonus points
- Experience preparing speech and audio data for ASR, TTS, and other Speech AI tasks;
- Experience working with data for LLMs, conversational AI, and AI agents;
- Experience preparing multilingual datasets;
- Experience procuring and accepting external data, selecting contractors, and managing their work;
- Experience automating annotation and building human-in-the-loop or model-assisted labeling processes;
- Understanding of annotation quality metrics, including inter-annotator agreement and acceptance rate;
- Experience with golden sets, honeypot tasks, double annotation, and spot checks;
- Experience implementing modern data preparation and quality control tools.
If you enjoy speed, short cycles, and the opportunity to directly impact the product, you will feel comfortable with us.
