Agentic Test Engineer

Agentic Test Engineer
About Us
SimplAI is transforming how enterprises harness the power of AI. Our platform makes it incredibly simple
and fast to build, deploy, and monitor intelligent AI applications—without the typical complexity. We empower
businesses to innovate with confidence through solutions that are scalable, secure, and production-ready
from day one. At SimplAI, we don’t just make AI possible—we make it practical.
About the Role
We are seeking a curious and hands-on QA/Automation Engineer to join our team and help test
next-generation AI-driven products. This role goes beyond traditional QA—it involves validating AI Agents,
GPT/LLMs, and context-driven workflows, ensuring they are reliable, scalable, and deliver seamless user
experiences.
As an Agentic Test Engineer, you will be responsible for designing and executing manual, API, and
automation tests, while also evaluating the reliability and accuracy of AI-powered workflows. If you’re excited
about exploring how AI agents think, interact, and evolve, this opportunity is for you.
What you’ll do
● Design, develop, and execute test plans, test cases, and automation scripts across web, API, and
AI-agent workflows.
● Validate agentic behavior including context retention, decision-making, and multi-step execution
accuracy.
● Test integrations with GPT/LLMs, APIs, and vector stores for correctness, scalability, and stability.
● Collaborate closely with developers, product managers, and AI researchers to define test strategies.
● Debug, log, and track issues in AI-powered workflows.
● Continuously improve testing frameworks for automation, scalability, and AI edge cases.
● Stay updated with AI evaluation techniques, test harness frameworks, and emerging tools.
What We’re Looking For
● 1–3 years of QA/Automation experience (Manual + API + Automation using Selenium, Pytest,
Playwright, or similar).
● Strong understanding of REST APIs, JSON, and database queries (SQL/MongoDB preferred).
● Exposure to AI concepts such as GPT/LLMs, AI Agents, embeddings, context windows, and prompt
engineering.
● Ability to design creative test cases for ambiguous or non-deterministic AI outputs.
● Strong problem-solving mindset with attention to detail.
Bonus Skills:
● Experience with LangChain, RAG pipelines, vector databases, or ML model evaluation.
Website:https://simplai.ai