Software Engineer L5/L6 — Model Evaluations & Data Curation (MEDC)

About this role

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Engineer L5/L6 — Model Evaluations & Data Curation (MEDC) based in the United States.

The Software Engineer will build foundational infrastructure that accelerates how AI teams create, evaluate, and improve high-quality datasets.
You will transform ad-hoc data curation workflows into reusable, scalable platforms that researchers and engineers can rely on.
The role combines strong software engineering with hands-on work in LLM-driven data generation, evaluation, sampling, and quality control.
You will work closely with researchers and data scientists to understand how curation decisions influence model behavior and performance.
Your work will help make datasets more discoverable, reproducible, versioned, and production-ready across multiple AI initiatives.
The environment is highly collaborative and technically ambitious, with significant autonomy and opportunities to influence engineering practices.
At the L6 level, the role additionally calls for technical leadership and the ability to establish direction across data and evaluation infrastructure.

### Accountabilities

• Design and build reusable data curation infrastructure, including shared libraries, components, and workflows that replace fragmented notebook-based processes.
• Develop scalable LLM-powered pipelines that transform raw catalog, metadata, and other data sources into training and evaluation datasets such as question-answer pairs and synthetic scenarios.
• Implement large-scale batch inference workflows while balancing data quality, computational efficiency, token usage, and cost.
• Develop sampling strategies that optimize coverage, diversity, difficulty, and representation across relevant content and member segments.
• Create data-quality and filtering systems using techniques such as LLM-as-judge scoring, evaluation-model-based ranking, dedupl...