Job Title: QA Lead (AI-First)
Title: OLX Uzbekistan Company Vacancies
Description: OLX Uzbekistan is a large online classifieds platform used by millions of people. Recently, our project has entered a new stage of development, and we are actively expanding the team. We are currently building our own independent local platform. We are migrating some services and designing and writing some tasks...
We are building AI-first product development: AI agents write code and conduct reviews, and engineers design the environment (harness), rules, and checks within which agents safely deliver changes to production. We need someone who will make this safe and measurable.
This is not a classic QA. You will not be running test cases manually and creating tickets after development. Your task is to design a quality system within which an autonomous pipeline (AI Flow) physically cannot release a change to production below a given threshold. Your product will be quality gates, autotests as guardrails, eval-harness for checking agent output, and metrics by which the quality of agentic delivery becomes measurable.
You are the last line of defense for quality in a team where agents write code, not people.
Responsibilities
What you will be doing:
- Quality Harness. Developing the test and verification layer of AI Harness: autotests, linters, type checks, contract and integration tests, security and regression checks – everything that stands between "the agent wrote the code" and "merge". These checks are the primary mechanism for combating hallucinations.
- Agent Output Validation. Designing checks for behavior and contracts, not specific implementations (the agent can generate different code with each run). Golden datasets, regression sets, AI review quality assessment.
- Quality Gates in AI Flow. Mastering the "human gate": what is automated, what remains for humans, what criteria block a merge. The goal is to minimize human involvement without compromising reliability.
- Quality Metrics. Making quality measurable: change failure rate, escaped defects, percentage of changes without escalation to a human, regression stability, AI review false-positive rate. Consolidating them into a common dashboard alongside Time-to-Market and Uptime.
- Quality on Legacy. Establishing basic test coverage on inherited code so that agents can refactor and make changes to it.
- Team Management. Growing and developing the QA team: hiring, onboarding, goal setting, developing competencies towards AI-first. Building processes and a quality culture where control is built into the pipeline, rather than relying on the heroism of individuals.
Requirements
What we expect from you:
- Technical background in QA automation: ability to read code and work with autotests, understanding the pipeline at an engineer's level (Python / JS / TS – welcome, but a writing-heavy role is not assumed).
- People management: experience leading a QA team – hiring, development, goal setting, process building.
- Deep understanding of the test pyramid, contract and integration testing, regression strategies.
- Practical experience with AI assistants/agents in development (Claude / Codex) and understanding their typical failures: hallucinations, "confidently wrong" code, implementation drift, only happy path tests.
- Eval-driven mindset: ability to formalize "what good means" into verifiable criteria and datasets.
- T-shape: ability to understand architecture and module boundaries, not just tests.
Will be a plus:
- SDET / Quality Engineer experience in a product team with a fast release cycle.
- Performance and security testing.
- Experience in the marketplace / classifieds domain.
- Experience with migrations and working with unowned legacy.
- Experience working with ML teams
Conditions
What we offer:
- Participation in the project development from the start.
- Direct impact on the final product.
- Prospects for professional and career growth.
- Development within a team of experienced colleagues.
- Minimal bureaucracy and quick decision-making.
- Individual discussion of the work schedule and format with each employee.
