About the role
ABOUT THE ROLE
Lead the data quality team at an early-stage AI company building infrastructure for reinforcement learning and model evaluation. You will shape how training data is assessed and improved, helping research teams create more reliable and useful AI agents.
WHAT YOU'LL DO
- Lead development of systems that evaluate tasks across reinforcement learning environments, synthetic data, benchmarks, and domain-specific workflows.
- Set data quality strategy, establish standards, and design experiments to assess agent outputs.
- Develop scalable methods for validating synthetic data, including failure analysis, task mutation checks, and trajectory audits.
- Work with research engineers, domain experts, and data vendors to diagnose quality issues and improve data generation.
- Turn research insights into production tools, dashboards, validation pipelines, and feedback loops.
- Mentor research engineers and promote technical rigor, clear communication, and strong execution.
WHAT WE'RE LOOKING FOR
- At least 5 years of research or engineering experience focused on AI/ML data evaluation or data quality systems.
- Experience leading technical projects or teams through ambiguous problem definition, implementation, and iteration.
- Advanced proficiency in Python, Docker, and Linux.
- Background in AI evaluations, post-training, benchmarks, synthetic data pipelines, or model evaluation infrastructure.
- Strong judgment about what makes agent training data realistic, learnable, diverse, reliable, and useful.
- Experience designing metrics, experiments, and quality processes, and translating research findings into production systems.
- Ability to work with subject-matter experts and convert domain judgment into scalable review or generation workflows.
- Strong written communication, attention to detail, and experience in an early-stage startup environment.
COMPENSATION & BENEFITS
Base salary: $130,000 to $225,000 annually. Visa sponsorship is available.
LOCATION
On-site in the United States. The primary city was not specified.