Code-Data Eval Author — Software Test Engineer / SDET (Pilot)

Code-Data Eval Author — Software Test Engineer / SDET (Mercor · remote contract)

Mercor partners with frontier AI labs to build the evaluations their coding models are trained and measured against. You’ll design the verifiers, correctness rubrics, and adversarial test cases that decide whether an AI agent’s code actually works.

What you’ll do

• Design verifiers and correctness rubrics for coding tasks

• Enumerate edge cases and build adversarial test cases for agent/model evaluation

• Grade agent trajectories and improve test/rubric quality through review

You are

• ~5+ years as an SDET / software test engineer at a real product organization

• Write code _and_ tests: automation frameworks (pytest, Playwright, Cypress), CI/CD (SDET preferred over manual-only QA)

• Clear written communication; familiarity with AI tools / evals is a plus

Engagement & pay

• Remote contract, flexible 30+ hrs/week

• Hourly rate set to your local market (e.g., US/Canada $75–100/hr; Europe and LatAm scaled to region)

Hiring process — paid
A short Mercor Technical Screen, a live Code Review Session, and a Domain Expert Interview. You’re paid $200 for completing all three, regardless of outcome.

Compensation

• Pay: $30 – $100/hour
• Type: Hourly contract
• Location: Remote — Americas & Europe