Applied AI Engineer — Global Health & AI (REMOTE ROLE)

Company: ICF

Locations: Rockville, Maryland, United States; Reston, Virginia, United States

Salary: $98.6K – $167.6K a year

Work arrangement: Remote possible

Employment type: Full-time

Experience: 2-5 years

Education: bachelor degree

Posted: 2026-11-16

Responsibilities

Design and build LLM pipelines, agents, and evaluation harnesses to support global health survey workflows and data analysis. Collaborate with domain experts to define acceptance criteria and ensure AI tools meet rigorous performance standards.

Requirements

Requires a bachelor's degree in a technical field and at least 4 years of professional software development experience, specifically with LLM-powered systems. U.S. citizenship is mandatory due to federal government contract requirements.

Skills

  • Python
  • LLM Pipelines
  • Agentic Coding Tools
  • Frontier-model APIs
  • Retrieval-augmented generation
  • Cloud Platforms
  • AI Evaluation
  • Data Analysis
  • Natural Language Processing
  • Software Development
  • Prompt Engineering
  • Model Governance
  • Statistical Analysis
  • Multilingual NLP
  • Technical Writing

About this role

Applied AI Engineer — Global Health & AI

ICF's Global Health and Development (GHD) line of business, part of the Health, People, and Human Services (HPHS) Group, is seeking an Applied AI Engineer to build frontier-model systems for various global health and survey programs. This role will support the development of AI-enabled tools that draft survey reports, let researchers and policymakers analyze data in natural language, translate questionnaires into low-resource languages, and support interviewers and data quality in the field. No tool is adopted unless it matches or beats the expert-led process it replaces. You will build the prompting, agent, and pipeline layer across the program. You will also build the evaluation harnesses that show whether each tool actually works, judged against ground truth produced by some of the most experienced survey methodologists in the world.

This position can be remote within the United States, with occasional travel.

It is contingent upon award of program funding.

Key Responsibilities
Design and build LLM pipelines and agents for survey workflows, including:
an AI-assisted data-to-report pipeline
natural-language analysis of survey data
multilingual questionnaire translation
field support tools for interviewers and supervisors
Build evaluation harnesses as first-class deliverables. Work with the Technical Lead and survey experts to define acceptance criteria before the work is judged. Build test sets from expert-produced ground truth and run the non-inferiority comparisons that decide whether a tool moves forward.
Route work across models based on reasoning complexity, volume, cost, latency, and data-residency requirements.
Manage inference cost by tracking token usage, context limits, prompt caching, and batch processing, and account for these operating costs in model-selection decisions.
Connect models to data services and to the applications built by full stack developers.
Use agentic coding tools to produce software from clear specifications, then review, test, and own the resulting code.
Monitor production systems, conduct disciplined error analysis, and report positive and negative results.
Document methods and results for public release, including negative results. Help transfer tools to partner-country institutions so they can run them on their own.

Minimum Required Experience
Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent years experience)
Minimum 4 years of professional software development experience, including building LLM-powered systems that real users relied on in production
U.S. citizenship is required by federal government contract

Additional Required Skills and Experience
Proficiency in Python or another modern programming language (e.g., JavaScript/TypeScript, Go) for back-end and AI application development
Hands-on experience building with frontier-model APIs (from any provider, or through any cloud platform), including tool use and retrieval-augmented generation
Demonstrated experience designing LLM evaluations, including test-set construction, automated and human grading, regression testing, and ship/no-ship decisions.
Working knowledge of LLM cost, latency, and model-selection trade-offs
Experience building and deploying on a major cloud platform (e.g., AWS, Azure, or Google Cloud)
Demonstrated experience using agentic coding tools, such as Claude Code, OpenAI Codex, Cursor, Antigravity, or GitHub Copilot agent mode, to specify, review, test, and take ownership of production code.
Experience with agent frameworks and AI evaluation tooling.

Preferred Skills
Experience with tasks where ground truth came from subject matter experts (for example, statisticians, clinicians, scientists, or methodologists)
Experience with grounded generation over structured or tabular data, or with LLM-assisted statistical analysis
Experience with multilingual or low-resource-language NLP, translation, or speech
Experience deploying open-weight models
Experience with sensitive or regulated data, such as health data or human-subjects research
Familiarity with model governance, monitoring, and responsible AI practice
Experience in global health, international development, or low- and middle-income country settings
Public work such as open-source contributions, published evaluations, or technical writing about systems you built undefined

Professional Skills
Explains AI trade-offs clearly to technical and non-technical audiences, including funders, survey experts, and partner-government staff
Collaborates effectively across disciplines, cultures, time zones, and organizational levels while treating domain experts as the source of truth
Communicates uncertainty, limitations, and negative results candidly.Self-motivated
Finds ways to add value without a lot of direct supervision

#Indeed
#LI-CCI
#GHDAI
Working at ICF
ICF is a global advisory and technology services provider, but we’re not your typical consultants. We combine unmatched expertise with cutting-edge technology to help clients solve their most complex challenges, navigate change, and shape the future.
We can only solve the world's toughest challenges by building a workplace that allows everyone to thrive. We are an equal opportunity employer. Together, our employees are empowered to share their expertise and collaborate with others to achieve personal and professional goals. For more information, please read our EEO policy.

We will consider for employment qualified applicants with arrest and conviction records.

Work Authorization and Sponsorship
All candidates must be authorized to work in the United States. This position is not eligible for new H-1B visa sponsorship by ICF. Candidates who currently hold valid work authorization may be considered for a change of employer petition, subject to company policy and applicable immigration requirements.
Reasonable Accommodations
Reasonable Accommodations are available, including, but not limited to, for ...

About ICF

Marketing Services · 1,001-5,000 employees · headquartered in Reston, Virginia

See all my matches — 2 min, free

More Applied AI Engineer Jobs

10 new Applied AI Engineer jobs were posted this week.

Get new Applied AI Engineer jobs by email — start free

Other Jobs in Rockville

Before you apply, check your resume against ATS filters for free.