If you want to lead the build of a production AI evaluation platform from the first line of code to the first paying clients, this role is for you. You’ll be the founding technical lead, owning architecture, engineering standards, and delivery across a small AI-focused team.

👤 Requirements:

---

• 5+ years of engineering experience
• 2+ years leading teams of 3–8 engineers through a full product build cycle
• Strong production-grade Python backend: async, FastAPI, APIs, integrations, PostgreSQL, Redis
• Deep understanding of LLM evaluation: LLM-as-judge, calibration, golden datasets, regression testing, hallucination detection
• Production RAG experience: ingestion, chunking, embeddings, retrieval, re-ranking, vector databases
• Experience with AI agents and agentic workflows: ReAct, Plan-and-Execute, supervisor/sub-agent patterns
• LLM observability and distributed tracing experience
• Ability to make and justify architecture and build-vs-integrate decisions
• Strong written technical English
• Comfortable working in an ambiguous, early-stage product environment

⭐ Nice to have:

• OpenAI, Anthropic Claude and open-weight model experience
• MLflow or other experiment/dataset versioning tools
• EU AI Act / NIST AI RMF knowledge
• GDPR / PII handling in AI pipelines
• AI red teaming, jailbreak and prompt-injection testing

📌 Responsibilities:

• Lead architecture and a team of 5–7 engineers
• Build production AI evaluation, RAG quality & monitoring systems
• Own LLM observability, regression testing and cost/latency tracking
• Set engineering standards and work with pilot clients