AI Engineer

Responsibilities

You will design and build intelligent AI agents for claim review, including multi-step reasoning and tool use. Additionally, you will develop RAG pipelines and fine-tune LLMs to improve accuracy and performance using reinforcement learning techniques.

Requirements

The role requires 3+ years of experience in applied ML or AI engineering with a strong foundation in Python. Candidates must have hands-on experience with LLM fine-tuning, RAG systems, and agentic workflows.

Skills

• Python
• LLM fine-tuning
• RAG development
• Reinforcement learning
• Agentic workflows
• PyTorch
• Hugging Face
• Multi-step reasoning
• Vector databases
• RLHF
• DPO
• Machine learning
• Healthcare data
• Model evaluation
• System architecture

Benefits

• Medical insurance
• Dental insurance
• Vision insurance
• Flexible paid vacation policy

About this role

About Alaffia & Our Mission

Every year, U.S. health plans lose billions to improper payments and
administrative waste. That wasted spending ultimately trickles down across the
healthcare ecosystem, driving up costs for plans, providers, and patients alike.
We’re here to change that paradigm.

Alaffia is a new kind of claims operations partner for health plans. Using
expert clinicians and transparent AI, we deliver deeper insights, smarter
automation, and consistently better outcomes across the entire lifecycle of
claims. With Alaffia, health plans can cut wasted spending more effectively than
ever — and provide their members the most affordable care.

We’re a high-growth, venture-backed Series B healthtech startup based in NYC and
are actively scaling our company. Join us in helping to build a healthcare
system that works better for everyone.

*This position requires current authorization to work in the United States.
Unfortunately, we are not in a position to sponsor work visas at this time.

About the Role & What You’ll Be Doing

Alaffia Health is a healthcare AI startup revolutionizing medical billing and
data automation. Our AI-driven platform leverages state-of-the-art generative AI
and machine learning technologies to enhance accuracy, efficiency, and
compliance in medical billing and auditing

* As we scale, we’re looking for an AI Engineer to build the intelligence
behind Alaffia’s AI agents. Our agentic systems are already in production
reviewing real claims. As our customers’ needs grow more complex, we need
someone to make our agents smarter, more accurate, and better grounded in the
policies, contracts, and clinical evidence they reason over.

* This is a model-focused role spanning four areas. You’ll design and build the
agents themselves, including their planning, tool use, and multi-step
reasoning across claims and medical records. You’ll develop and fine-tune the
LLMs that power them on domain-specific healthcare data. You’ll build the RAG
systems that keep their outputs accurate and traceable to source documents.
And you’ll apply reinforcement learning techniques such as RLHF and DPO,
turning feedback from our clinical auditors into measurable improvements in
agent performance.

* This role is for you if you like working at the frontier of applied LLM
research and seeing your work land in production. You’ll take ideas from
early experiments through to models that make real decisions on real claims,
and you’ll have a direct hand in shaping how a Series B healthtech builds,
trains, and improves its AI. You’ll work closely with our AI engineers, MLOps
team, and clinical subject-matter experts.

Your Responsibilities

* AI Agent Development: Design and build single- and multi-agent systems for
claim review. This includes planning, tool use, memory, and multi-step
reasoning across claims data, medical records, and coding guidelines.

* RAG Development: Build and optimize retrieval-augmented generation pipelines
over policies, contracts, clinical guidelines, and medical records. This
covers chunking strategies, embeddings, hybrid search, re-ranking, and
grounding and citation of outputs.

* LLM Development & Fine-Tuning: Fine-tune open-source and proprietary LLMs
(e.g., SFT, LoRA/QLoRA, instruction tuning) on domain-specific healthcare and
claims data. Manage dataset curation, training runs, and model versioning.

* Reinforcement Learning: Develop RL-based training and alignment methods
(e.g., RLHF, DPO, reward modeling) that use human-in-the-loop feedback from
clinical auditors. Use them to improve agent accuracy, reasoning, and
decision quality.

* Evaluation & Experimentation: Define evaluation frameworks, benchmarks, and
metrics for agents, fine-tuned models, and RAG systems. These include
hallucination, faithfulness, retrieval quality, and task-level accuracy. Run
and track experiments rigorously.

* Production Integration (in collaboration with the Software and DevOps teams):
Take models and agents from prototype to production, optimizing for latency,
cost, and reliability at scale.

* Compliance & Safety: Ensure models and agents handle PHI securely and comply
with HIPAA. Build guardrails that keep outputs accurate, explainable, and
auditable.

What We’re Looking For

* 3+ years of experience in applied ML or AI engineering, ideally with
experience developing agentic systems, fine-tuning LLMs, and building RAG
pipelines in production.

* Strong software engineering fundamentals and proficiency in Python.

* Highly experienced in developing AI agents and agentic workflows, including
tool calling, orchestration, and multi-step reasoning

* Hands-on experience with LLM fine-tuning (SFT, PEFT/LoRA) and training
pipelines using frameworks such as PyTorch and Hugging Face

* Proven experience building production RAG systems, including vector
databases, embedding models, and retrieval optimization

* Practical experience applying reinforcement learning to LLMs (e.g., RLHF,
DPO, PPO, reward modeling)

* Strong understanding of LLM evaluation methodologies and experiment tracking

Nice to have

* Experience with agent frameworks such as LangGraph, LangChain, LlamaIndex, or
similar

* Familiarity with LLM provider APIs and self-hosted model serving (e.g., vLLM,
TGI)

* Experience with distributed training and GPU infrastructure

* Working knowledge of AI tooling such as MLFlow, LangFuse, and Weights &
Biases

* Experience with healthcare data (claims, EHRs, medical coding such as
ICD-10/CPT) or other regulated domains

Our Culture

Alaffia was born out of our founders’ personal connection to the inefficiency of
the U.S. healthcare system. We are deeply mission-driven, with an abiding belief
that technology can help create a better future for everyone — and we’re looking
for others who share our passi...

About Alaffia Health

Hospitals and Health Care · 51-200 employees · headquartered in New York, New York · founded 2020