The RoleComfyUI has 4 million users, 60,000+ community-built nodes, and a canvas where every workflow is different. Nobody can write enough tests to cover that by hand. We want someone to build the systems that do it instead.
This is not a testing role in the traditional sense. You won't run standups, manage a manual test team, or own a defect queue. You'll build the infrastructure that decides whether a release is safe to ship - and make that decision fast enough that we can ship every day.
We're moving from a release cycle measured in weeks to one measured in days. That only works if quality is proven by machines, not negotiated in meetings. That's your mandate.
What you'll do- Own what "correct" means for ComfyUI, and build the systems that prove it. Release gates, coverage of real user journeys, and the automation that makes shipping daily safe rather than scary.
- Solve the combinatorial problem. Users combine 60,000+ community nodes freely. Design property-based and generative testing that finds breakage in workflows nobody has written yet.
- Solve the visual correctness problem. Canvas rendering, node layout, and link routing break in ways pixel diffs can't catch without drowning you in false positives. Build perceptual comparison that engineers actually trust.
- Solve the non-determinism problem. Model output shifts with seeds, hardware, and drivers. Build the evaluation harness that separates a real regression from acceptable variance.
- Build custom node CI. Our ecosystem is our moat and our largest untested surface. Make third-party node compatibility something we verify continuously, not discover from bug reports.
- Use AI agents aggressively, and verify their output. Agent-driven exploratory testing, generated test cases, self-repairing suites - all of it, gated behind checks that reject the majority of what the agents produce.
- Make quality legible. Publish the metrics that tell us whether we're getting better, and own them in front of the team.
You'll also help us answer a harder question: as more of our code gets written with AI assistance, how do we know it's actually good? Building evaluation for our own engineering loop is part of this role, once product quality is on solid ground.
You might be a good fit if you- Have built testing or verification infrastructure that other engineers depended on. You write production code - this is an engineering role, not a process role.
- Have worked on a system where scripted test coverage was impossible, and did something smarter about it: property-based testing, fuzzing, simulation, generative testing, or evaluation harnesses.
- Have shipped something that gates releases, and can tell us what happened the first time it blocked a launch.
- Are strong in TypeScript or Python, and comfortable in CI systems and build graphs.
- Have opinions about flakiness. You treat a flaky test as a defect with a root cause, not weather.
- Have a startup mindset: you thrive in fast-moving environments, take ownership, and love solving problems at scale.
Bonus if you have- Tested graphics, canvas, or rendering surfaces - WebGL, shaders, or visual diffing at scale.
- Built evaluation systems for non-deterministic output: golden datasets, judge calibration, scored regression tracking.
- Maintained an open-source project, especially one where you had to keep a plugin or extension ecosystem working across versions.
- Used ComfyUI, or contributed to it.
What this role is not- Managing a manual QA team.
- Owning a Jira defect workflow.
- Writing Playwright scripts to someone else's specification.
- We have Playwright, Vitest, Storybook, and Codecov in CI today. That's the floor, not the job.
Compensation- Location: San Francisco preferred
- Compensation: $180,000-$220,000 + Equity + Benefits
