Staff Data Scientist - GenAI

Responsibilities

Design, build, evaluate, and deploy agentic AI systems for complex business workflows, establishing architecture, controls, evaluation methods, and measurable production outcomes. Guide engineers through technical influence, develop reusable AI practices and tools, and identify high-impact opportunities for automation across the business.

Requirements

Requires 8+ years of data science or machine learning experience, including 2–3 years building and operating production LLM-based or agentic systems, plus strong Python engineering and deep expertise in LLM evaluation. A relevant bachelor's degree is required; postgraduate education or academic achievement is desirable, along with a record of technical leadership and mentoring.

Skills

• Generative AI
• Agentic AI Systems
• Large Language Models
• LLM Evaluation
• Python
• Prompt Engineering
• AI Architecture
• Evaluation Dataset Design
• Statistical Reasoning
• Error Analysis
• Tool Integration
• Workflow Orchestration
• Cloud AI Platforms
• Technical Leadership
• Mentoring
• Cross-Functional Collaboration

About this role

The Company  

Cover Genius is the global infrastructure for embedded protection. Active in
over 60 countries and all 50 US States, we protect the customers of the world’s
largest digital companies, including Klarna, Revolut, Stripe, Priceline, Agoda,
Booking.com [http://Booking.com], Turkish Airlines, Tongcheng Travel, eBay, and
Uber, with seamless, end-to-end experiences. Cover Genius has protected more
than 73M customers globally across 240M policies with USD $3.2BN in gross
written sales. 

Coming off a stellar year with 40% YoY revenue growth and a recent $100M capital
raise
[https://covergenius.com/company/news/cover-genius-raises-100m-backed-by-vista-credit-partners/],
putting our valuation at $1.9BN, we are accelerating into our next phase of
growth. As part of our team, you’ll help drive our AI-first roadmap, developing
hyper-personalization engines, agentic distribution, and automated claims
infrastructure, while building the scalable technology powering the fast-growing
$70B embedded protection market.

Our people are: Accountable, customer-obsessed, collaborative, driven

Our people are not: Passive, defensive, siloed, hesitant

About the Role:

As a Staff Data Scientist for Generative AI on our Customer Ops Team, you will
own the design and delivery of agentic AI systems that take on complex,
judgement-heavy work across the business, from automated claims decisions to
customer operations and partner-facing/customer-facing products. You will go
where the highest-impact problem is, take it from idea to production, and leave
behind systems and teams that can keep improving without you.

To drive success in this role, you will have deep hands-on experience taking
LLM-based and agentic systems from prototype to production, a rigorous approach
to evaluation, and the credibility to shape engineering decisions without formal
authority. As the Staff Data Scientist, you will set the architecture and
evaluation strategy for our agentic systems, build and ship core components
yourself, and raise the AI engineering capability of the teams you work
with. Regular collaboration with Engineering, Product, Operations, Compliance,
Analytics team will be key in ensuring that the systems we build are accurate,
auditable and safe to scale.

NOTE: We are actively considering candidates at both Senior and Staff levels for
this role. Final leveling and compensation will be determined based on overall
experience and interview performance, so we encourage you to apply regardless of
which title best fits you today!

Key Responsibilities:

* Own the design of agentic AI systems. Decide where models add value and where
deterministic code should do the work, define agent boundaries, tools and
contracts, and design the controls that determine when a system may act on
its own.

* Build the evaluation foundation. Create ground-truth datasets, labelling
rubrics with measured inter-rater agreement, and eval harnesses that gate
every prompt and model change, measuring both component-level and end-to-end
performance against a defined production bar.

* Ship production systems hands-on. Build prompts, agents, tool integrations,
evaluation pipelines and LLM-as-judge scorers, and lead the error analysis
that turns failures into specific, prioritised fixes.

* Lead engineers through influence. Run design reviews, pair on implementation,
and establish the patterns teams use to build, test and release AI
components, so engineers can independently ship eval-gated changes.

* Raise AI capability across the organisation. Turn each engagement into
reusable guidance, templates, shared tooling and hands-on sessions that other
engineering and product teams can adopt.

* Own the outcomes, not just the models. Define and track the metrics that
matter for each system, such as automation rate, decision accuracy,
unsafe-action rate and cost per task, and report to leadership with a clear
account of what is moving them and why.

* Shape where we invest next. Assess new opportunities for agentic AI across
the business, recommend what to build, what to buy and what not to automate,
and run fast, falsifiable proofs of concept to decide.

What you will bring:

* 8+ years in data science or machine learning, including at least 2 to 3 years
building and shipping LLM-based or agentic systems into production, where you
owned the outcome after launch and not only the prototype.

* Deep expertise in LLM evaluation: ground-truth dataset and rubric design,
LLM-as-judge methods and how to validate them, statistical reasoning with
small samples, and error analysis of multi-step or multi-agent systems.

* Strong Python engineering, with practical experience of agent and tool-use
patterns (e.g. MCP, function calling), workflow orchestration (e.g. Temporal
or similar), and a major cloud AI platform (e.g. AWS Bedrock).

* A track record of influencing technical direction beyond your own team:
shaping architecture, influencing engineering standards, and mentoring
engineers who were new to AI/ML.

* Bachelor's degree in a relevant field. A postgraduate degree and/or record of
academic achievement is also desirable.

Proficiencies and Attributes:

* Leadership without authority. You bring data scientists and engineers along
by making them better, through clear reasoning, pairing and patient
explanation rather than title.

* Judgement about when not to use AI. You are as comfortable arguing that a
component should be plain code as you are designing an agent.

* Range and fast ramp-up. You can walk into an unfamiliar domain, learn how the
work is really done from the people who do it, and turn that into a system
design quickly.

* Clarity with non-technical audiences. You can explain to an operator, a
compliance lead or an executive what a system does, how confident we should
be in it, and why.

* Accountable and delivery-focus...

About Cover Genius

Technology, Information and Internet · 501-1,000 employees · headquartered in New York, NY · founded 2014