Applied AI Engineer

Responsibilities

The Applied AI Engineer will own the quality, safety, and performance of AI-powered consumer features by managing system prompts, evaluation frameworks, and guardrails. This role involves building tools for analysts, monitoring model reliability, and partnering with cross-functional teams to ensure compliance and fairness.

Requirements

Candidates must have a Bachelor's or Master's degree in Computer Science or a related field with professional software engineering experience in Python and Java. Proven experience in building and operating LLM-powered applications in production, including evaluation and security testing, is required.

Skills

• Python
• Java
• Spring Boot
• LLM
• Prompt Engineering
• Machine Learning
• SQL
• AWS
• Amazon Bedrock
• Data Analysis
• CI/CD
• Agile
• Scrum
• Evaluation Frameworks
• Guardrails
• Model Monitoring

Benefits

• Competitive compensation
• Rewards programs
• Work/life balance
• Employee resource groups
• Social events

About this role

FICO (NYSE: FICO) is a leading global analytics software company, helping businesses in 100+ countries make better decisions. Join our world-class team today and fulfill your career potential!
The Opportunity
The myFICO Scores B2C Software Engineering team is hiring an Applied AI engineer to own the quality, safety, and performance of our AI-powered consumer features. As myFICO expands its AI-powered product features, this role combines Python and Java backend engineering with hands-on LLM experience to own how our assistants are prompted, evaluated, guarded, and improved. You will treat AI quality as something we measure and build the tooling our analysts and QA engineers rely on.
What You’ll Contribute
Own the system prompts, tool descriptions, and evaluation prompts behind Personalized Credit Insights and the AI chat assistant, improving them using production evidence.
Build and grow the evaluation framework: turn production failures into test cases, run regression suites before every prompt or model change, and report results the business can trust.
Measure and improve LLM-based evaluator reliability by comparing against human-labeled data and tracking precision, recall, and agreement over time.
Design, tune, and monitor guardrails, including input and output classifiers, PII redaction, prompt-injection defenses, and financial-advice compliance rules defined with Legal and Compliance.
Lead model upgrades by re-running evaluations, comparing quality, cost, and latency, and shipping safely through A/B tests.
Reduce cost and latency with prompt caching, model routing, and context management.
Build tools for Conversation Analysts and QA engineers, such as transcript viewers, labeling workflows, and dashboards.
Partner with Analytics, Legal, and Compliance on fairness and bias evaluation of AI outputs.
Leverage AI coding assistants and agentic workflows (Claude Code, Cursor, GitHub Copilot) to accelerate delivery, with the judgment to review and refine their output.

What We’re Seeking
Bachelor's or Master's in Computer Science, Engineering, or a related field, with demonstrated professional software engineering experience, strong Python, and the ability to read and contribute to Java (Spring Boot) microservices.
Proven experience building and operating LLM-powered applications in production, including prompt design, tool calling, structured outputs, and streaming.
Demonstrated experience building evaluations for LLM systems: test sets, regression suites, and LLM-as-judge approaches, including measuring evaluator quality against human labels.
Working knowledge of Amazon Bedrock or a comparable managed LLM platform, and AWS services such as S3, Lambda, and DynamoDB.
Proficient SQL and comfort analyzing production data in a data warehouse such as Amazon Redshift.
Production ownership, including code review, CI/CD, release management, and on-call support, within Agile/Scrum teams.
Strong written and verbal communication, with the ability to explain evaluation results and tradeoffs to product, legal, and executive stakeholders.
Experience with guardrail tooling, LLM security testing, RAG and retrieval evaluation, fairness and bias evaluation, model migrations, observability tools such as Datadog, BI tools such as Tableau, or regulated domains (FCRA, ECOA, UDAAP, CFPB guidance) is a plus.
Our Offer to You

An inclusive culture strongly reflecting our core values: Act Like an Owner, Delight Our Customers and Earn the Respect of Others.
The opportunity to make an impact and develop professionally by leveraging your unique strengths and participating in valuable learning experiences.
Highly competitive compensation, benefits and rewards programs that encourage you to bring your best every day and be recognized for doing so.
An engaging, people-first work environment offering work/life balance, employee resource groups, and social events to promote interaction and camaraderie.
The targeted base pay range for this role is: $105,000 to $165,000 with this range reflecting differences in candidate knowledge, skills and experience.
Why Make a Move to FICO?
At FICO, you can develop your career with a leading organization in one of the fastest-growing fields in technology today – Big Data analytics. You’ll play a part in our commitment to help businesses use data to improve every choice they make, using advances in artificial intelligence, machine learning, optimization, and much more.

FICO makes a real difference in the way businesses operate worldwide:
• Credit Scoring — FICO® Scores are used by 90 of the top 100 US lenders.
• Fraud Detection and Security — 4 billion payment cards globally are protected by FICO fraud systems.
• Lending — 3/4 of US mortgages are approved using the FICO Score.
Global trends toward digital transformation have created tremendous demand for FICO’s solutions, placing us among the world’s top 100 software companies by revenue. We help many of the world’s largest banks, insurers, retailers, telecommunications providers and other firms reach a new level of success. Our success is dependent on really talented people – just like you – who thrive on the collaboration and innovation that’s nurtured by a diverse and inclusive environment. We’ll provide the support you need, while ensuring you have the freedom to develop your skills and grow your career. Join FICO and help change the way business thinks!
Learn more about how you can fulfil your potential at www.fico.com/Careers
FICO promotes a culture of inclusion and seeks to attract a diverse set of candidates for each job opportunity. We are an equal employment opportunity employer and we’re proud to offer employment and advancement opportunities to all candidates without regard to race, color, ancestry, religion, sex, national origin, pregnancy, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. Research has shown that women and candidates from underrepresented communities may no...

About FICO

Software Development · 1,001-5,000 employees · headquartered in Bozeman, Montana · founded 1956