logo[мetahunt]
> DOU
senior

QA Engineer

Windmill Digital
format:Remotecompany:Product
manual testingtest planningplaywrightcypressseleniumadobe illustratorllmautomation qa
alertingobservabilityprompt engineeringai agents
experience3+ years
domainAI
> full description

We’re looking for a quality-focused engineer who combines a solid manual testing foundation with hands-on automation experience and a growing interest in AI evaluation. You care deeply about how systems behave,
you’re not satisfied with surface-level testing, and you’re excited about what modern QA looks like in an AI-driven world.

WHAT YOU’LL DO

• Design and execute evaluation strategies for AI-powered features, including LLM-based flows,
recommendation engines, and decision logic.
• Validate AI outputs for correctness, consistency, bias, hallucination risk, and edge cases — defining clear
qualitative and quantitative acceptance criteria.
• Perform deep manual, exploratory, and scenario-based testing across both AI and non-AI features.
• Build and maintain automated evaluation and regression pipelines for AI-enabled systems.
• Identify non-determinism and reliability risks specific to AI systems and propose practical mitigations.
• Partner closely with engineers and product teams to improve testability, observability, and overall release
quality.
• Review and verify fixes, prevent regressions, and own quality assurance end-to-end before releases ship.

WHAT WE’RE LOOKING FOR

• Minimum 3 years of hands-on manual testing experience, including exploratory testing, test planning, and
writing acceptance criteria for complex or enterprise systems.
• Minimum 3 years of test automation experience using frameworks such as Playwright, Cypress, Selenium,
or comparable tools — covering UI and/or API layers.
• Demonstrated ability to own a full test strategy, not just execute test cases.
• Strong understanding of AI system behavior, including non-determinism, prompt sensitivity, and the unique
challenges of validating model outputs.
• Sharp analytical thinking and attention to detail — you notice what others miss.

NICE TO HAVE

• Experience with AI evaluation frameworks, prompt testing, or model validation techniques.
• Familiarity with agentic QA concepts and AI-assisted testing workflows.
• Exposure to observability tooling and how it supports quality in production systems.

WHAT WE OFFER

• A fast-paced, product-first environment where your
work has direct impact.
• Flexible working practices.
• Competitive salary. • Training program allowance to keep your skills
sharp.
• Real challenges and meaningful personal growth. • A respectful, inclusive team culture with regular
team-building activities.

Відгукнутись на вакансію