> DOU
senior
Windmill DigitalQA Engineer
format:Remotecompany:Outsource
reactangulartypescriptjavascriptjavanode.jsrubyplaywrightcypressseleniumsqlmongodbdockerci/cdawsazuregoogle cloud
pythonfastapidjangospringnext.jskubernetesllmprompt engineering
semantic dedup
Reposted 2× across DOU — deduped to one listing.
metahunt matched 2 postings of this exact role by semantic similarity, so you see it once instead of 2 times.
> full description
We are looking for a senior full-stack engineer who owns quality and verification in an AI-native delivery environment — combining deep engineering judgment with AI tools to review, validate, and test code and AI-driven behavior across the full stack, from generated code to production output.
Key Responsibilities
- Review and verify code produced by AI coding agents — assessing correctness, architecture, performance, and security to the standard of a senior engineer, not surface-level testing.
- Own quality end-to-end before releases ship: deep manual, exploratory, and scenario-based testing across both AI and non-AI features, plus fix verification and regression prevention.
- Evaluate AI-powered features — LLM-based flows, recommendation engines, decision logic — validating outputs for correctness, consistency, bias, hallucination risk, and edge cases, with clear qualitative and quantitative acceptance criteria.
- Identify non-determinism, prompt sensitivity, and reliability risks specific to AI systems, and propose practical mitigations.
- Build and maintain automated evaluation and regression pipelines for AI-enabled systems — using AI agents (Cursor, Claude Code) in your own verification workflow.
- Partner with engineers and product to improve testability, observability, and overall release quality, and to raise the bar on AI-assisted delivery practices.
Required Skills
- AI-Native Development: Cursor or Claude Code (or equivalent agentic tools); prompt engineering, applied to both building and verifying
- Engineering depth: 8+ years across the full stack, strong enough to review and validate another engineer’s (or AI’s) code
- Front-End: React or Angular, with TypeScript / JavaScript
- Back-End: Java, Node.js, or Ruby
- AI system behavior: solid understanding of non-determinism, prompt sensitivity, and the challenges of validating model outputs
- Test automation: Playwright, Cypress, Selenium, or comparable (UI and/or API)
- Data: SQL and NoSQL (e.g. MongoDB)
- Infrastructure: Docker, CI/CD, and a major cloud (AWS, Azure, or GCP)
- Proven ability to own a full verification and test strategy, not just execute test cases
- Upper-intermediate English (B2+)
Nice to Have
- AI evaluation frameworks, prompt testing, or model validation techniques
- Agentic QA concepts and AI-assisted testing workflows
- Observability tooling and its role in production quality
- Python (FastAPI / Django), Spring Boot, Next.js
- Kubernetes, serverless & event-driven architectures
- High-growth startup or product-led company background
Personal Attributes
- Sharp analytical thinking and attention to detail — notices what others miss
- Autonomous & proactive — owns quality without being told what to check
- Strong engineering judgment and a reviewer’s mindset
- Strong team player
- Excellent communication
- Eager to learn & adapt fast
What We Offer
- Competitive compensation and benefits package.
- Remote-first work with a flexible schedule.
- Opportunities for professional growth, training, and certifications.
- A dynamic environment where innovation, security, operational excellence, and cutting-edge ML technologies are highly valued.
- Projects that span cutting-edge AI, UX, fintech, and healthtech domains.