Remote · outside RF and RB · conversational English__**

We are Aspirity Solution, a development studio with over 10 years of experience in creating web and mobile applications, no-code solutions, and intelligent AI agents for clients worldwide.

We are currently looking for a QA Engineer who will be responsible for the quality, stability, and predictability of AI agents and LLM systems.

Responsibilities:

• Testing and evaluating the performance of AI agents (dialogue and cognitive systems).

• Developing and conducting eval tests to verify the correctness, robustness, and stability of agent behavior.

• Preparing test cases and scenarios for functional and non-functional testing.

• Analyzing agent behavior with non-standard input data, identifying logical errors and problems in prompts.

• Closely interacting with developers and prompt engineers to improve models and test pipelines.

• Supporting and developing test automation and QA processes within the AI team.

• Maintaining technical documentation and describing best practices.

Requirements

• QA experience (manual or automation) of 2+ years.

• Practical experience testing AI agents or chatbots.

• Understanding of AI agents, LLM systems, and prompt engineering principles.

• Experience with eval tools (OpenAI Evals, DeepEval, LangChain Evals, or equivalents).

• Experience with automated testing (preferably for AI agents).

• Proficient in Python for writing tests and utility scripts.

• Skills in working with Git and bug tracking systems.

• English language proficiency from B2 for working in an international team.

• Residency outside of RF and RB.

• Willingness to work in the PT time zone: from 7:00 to 16:00 PT. (America time, Washington)

Will be a plus

• Experience with LangChain, vapi, Synthflow, or other AI agent creation tools.

• Experience analyzing LLM outputs based on quality metrics: accuracy, coherence, relevance.

• Understanding of CI/CD processes and basic monitoring tools.

• Participation in building test architecture for AI systems.

We offer

• Fully remote work format.

• Work in a strong team of developers, ML engineers, and AI specialists.

• Opportunity for professional growth in QA for LLM and AI agent systems.

• Payment based on qualification — from 20 USD/hour.