Machine Learning Engineer, Evaluation and Quality, Creativity Apps

Responsibilities

You will define and implement evaluation strategies, metrics, and infrastructure to ensure the quality of AI features within creativity applications. This involves collaborating with cross-functional teams to translate product use cases into rigorous testing methodologies and reporting quality risks to stakeholders.

Requirements

Candidates must have at least 5 years of industry experience in machine learning or software engineering with a focus on model evaluation. Proficiency in Python or Swift and a strong understanding of the ML development cycle, including LLMs and data curation, are required.

Skills

• Machine Learning
• LLM
• Evaluation Methodology
• Data Curation
• Python
• Swift
• Quality Assurance
• Agentic Workflows
• Infrastructure Design
• Metrics Definition
• Generative AI
• Software Engineering
• Data Science
• Cross-functional Collaboration
• Model Development
• Technical Reporting

About this role

Imagine what you could do here. At Apple, new ideas have a way of becoming
outstanding products, services, and customer experiences very quickly. Bring
passion and dedication to your job and there's no telling what you could
accomplish. The Creativity Apps team is looking for a Machine Learning Engineer
focused on the quality and evaluation of ML models and AI features in our
creativity applications. You will define how we measure whether the product and
AI features are good enough to ship: designing the evaluation strategy and
metrics, building the datasets and infrastructure behind them, and driving
end-to-end quality from model development through to the feature in customers'
hands. You will have the unique and rewarding opportunity to help shape upcoming
products that will delight and inspire millions of creators every day.

DESCRIPTION

In this role, you will own the evaluation and quality of AI features across
Creativity Apps. You will work closely with Machine Learning engineering,
infrastructure, product, and design teams to understand each model and use case,
and translate them into evaluation methodology, testing strategy, and quality
metrics that reflect what creators actually experience. You will: * Define
evaluation strategy and metrics for ML models, LLMs, and AI features, tailored
to the model, the product, and its use cases. * Curate and create evaluation
datasets, measure and improve dataset quality, and run iterative improvement
loops so that datasets stay representative of each product. * Design, build,
maintain, and optimize evaluation infrastructure at scale, with reproducible
runs, tracked baselines, and comparisons between model versions. * Apply ML,
LLMs, and agentic workflows to the evaluation itself, including automated
judging, test generation, and analysis, to make evaluation more efficient, more
accurate, and more scalable. * Drive end-to-end evaluation and quality of AI
features, from model checkpoints to integrated, shipping features, including
triaging issues and driving them to resolution. * Report quality clearly,
turning evaluation results into ship/no-ship risk and recommendations for both
technical and non-technical audiences. * Partner with and drive cross-functional
teams, comfortable with ambiguity and making educated, critical judgments. Your
curiosity about the latest developments in ML and generative AI, and your drive
to apply new technology to practical problems, will set you apart.

MINIMUM QUALIFICATIONS

5+ years of industry experience in machine learning or software engineering,
with a focus on ML/AI model evaluation and quality. Experience with evaluation
methodology for ML models, LLMs, and AI products or features. Strong
understanding of the ML development cycle, from data and training to evaluation,
deployment, and monitoring. Solid machine learning fundamentals and LLM
knowledge, including agentic workflows, and you keep up with the latest
developments in the field. Track record of applying LLMs, or agentic workflows
to ML evaluation and quality. Experience in data curation, dataset creation, and
dataset quality, including iterative improvement workflows that deliver
high-quality datasets tailored to specific products. Experience designing,
implementing, maintaining, or optimizing ML evaluation infrastructure at scale.
Experience designing evaluation metrics based on the model, the product, and its
use cases. Experience driving end-to-end evaluation and quality for AI products.
Fluency in quality reporting, presenting quality data to both technical and
non-technical audiences. Strong communication skills and experience working with
and driving cross-functional partners. Proficient in Python and/or Swift or
comparable languages. Bachelor's degree in Computer Science, Data Science,
Machine Learning, or a related field, or equivalent experience.

PREFERRED QUALIFICATIONS

Knowledge of the software development life cycle and software quality assurance
methodologies. Experience in objective image and video quality evaluation and
metrics. Knowledge of color science and/or experience with subjective image or
video quality review. Passion for creative applications; experience as a creator
(photo, video, music, or design) is a plus. Excitement about learning new
technology and applying it to practical problems. Familiarity with iOS and
macOS.

About Apple

Computers and Electronics Manufacturing · 10,001+ employees · headquartered in Cupertino, California · founded 1976