Automation QA Engineer

Innovecs is a global digital services company with a presence in the US, the UK, the EU, Israel, Australia, and Ukraine. Specializing in software solutions, the Innovecs team has experience in Supply Chain, Healthtech, Collaboration Tech, and Gaming.

For the fifth year in a row, Innovecs is included in the Inc. 5000, the list of fastest-growing private companies in the US, and a ranking of the best global outsourcing service providers by IAOP. Recently, Innovecs was honored with the prestigious Global Good Awards for the Employee Engagement & Wellbeing, won gold at the Employer Brand Management Awards, and was included in the Global Top 100 Inspiring Workplaces Ranking.

Our value to you:

• Flexible hours and remote-first mode,
• Competitive compensation,
• Complete Hardware/Software setup — anything you need for work,
• Open-door culture, transparent communication, and top management at a handshake distance,
• Health insurance, vacation, sick leaves, holidays, paid maternity/paternity leave,
• Access to our learning & development center: workshops, webinars, training platform, and edutainment events,
• Virtual team buildings and social activities.

If you feel like you’re the perfect match for this role, drop us your CV!

There are no limitations, no barriers when the right people are on your way — apply for the vacancy and succeed with us!

Innovecs is an equal opportunity employer. All hiring decisions are based on professional qualifications, skills, and experience. We are committed to a transparent, merit-based recruitment process that prevents discrimination and ensures equal opportunities for all candidates. Reasonable accommodations are available upon request throughout the recruitment process to support accessibility and inclusion.

Requirements

• 5+ years of experience in Quality Assurance and Test Automation.
• 2+ years of experience testing AI-enabled applications, LLM solutions, or agentic systems.
• Strong Python/ Java/ Javascript skills.
• Experience with Playwright, Selenium, Cypress, or similar automation frameworks.
• Solid understanding of software testing methodologies and test design techniques.
• Experience with API, integration, and database testing.
• Experience with CI/CD pipelines and DevOps practices.
• Knowledge of AI evaluation frameworks.
• Understanding of AI-specific failure modes: hallucinations, prompt injection, jailbreak attacks, model drift, bias and fairness issues, RAG failures.
• Experience working in cloud-native environments (Azure or AWS).
• Understanding of MCP architecture and agent-to-tool integrations.

Will be a plus: Background in Supply Chain, Logistics, WMS, TMS, OMS, or ERP platforms. Experience with LangSmith, Braintrust, MLflow, Arize, or Evidently AI. Familiarity with EU AI Act and AI governance frameworks.

Strong plus: ISTQB CT-AI certification or equivalent AI testing credentials. Contributions to AI testing frameworks or open-source projects. Leadership experience driving AI quality strategy across multiple teams. Knowledge of warehouse operations and fulfillment processes.

Responsibilities

Focus: application-layer LLM QA, and enterprise/AISDLC quality engineering.

AI Quality Engineering: * Design and execute quality strategies for AI-powered warehouse assistants, copilots, and autonomous agents. * Build evaluation frameworks for LLM-based warehouse workflows and agentic systems. * Define and maintain golden datasets and benchmark scenarios for warehouse operations. * Validate AI-generated recommendations, decisions, and workflow actions. * Perform hallucination, factuality, grounding, and response-quality testing. * Validate RAG implementations and retrieval accuracy. * Design agent trajectory tests for multi-step autonomous workflows. * Conduct adversarial testing, prompt injection testing, and jailbreak assessments. * Monitor model drift, output quality degradation, and AI performance trends.

Test Automation & Enterprise Quality: * Create and maintain scalable automation frameworks for functional, API, integration, and end-to-end testing. * Design CI/CD quality gates for AI and non-AI releases. * Validate enterprise integrations across WMS, ERP, TMS, OMS, WCS, and third-party platforms. * Implement automated regression suites for business-critical warehouse processes. * Ensure release readiness through risk-based testing and quality metrics. * Perform root-cause analysis of complex system-level defects. * Drive quality best practices across engineering teams.

Governance & Compliance * Define AI quality standards and testing methodologies. * Support compliance with EU AI Act, ISO 42001, and responsible AI principles. * Maintain audit-ready quality evidence and evaluation reports. * Establish measurable KPIs for AI reliability, safety, and business accuracy. * Contribute to AI quality engineering standards and Centers of Excellence.

Cross-Functional Collaboration * Partner with AI Product Engineer, Solution Architect, and UX Lead. * Participate in AI solution design reviews. * Manage warehouse business requirements and maintain validation strategies.

Відгукнутись на вакансію