Applied AI Engineer

Responsibilities

Build and maintain production applications and features that integrate approved AI models and services into clinical documentation, coding, and revenue cycle workflows. Design user interactions, prompts, safeguards, and evaluations, while owning reliability, monitoring, security, compliance, and ongoing production support.

Requirements

A bachelor’s degree in computer science, software engineering, human-computer interaction, or a related technical field—or equivalent practical experience—is required, along with at least three years of relevant production software or application engineering experience. Candidates need experience building AI-powered applications using APIs, designing prompts, integrating AI services, and addressing reliability, latency, cost, testing, and observability; healthcare data experience is also listed.

Skills

• Application Engineering
• AI Integration
• Large Language Models
• Prompt Design
• AI Evaluation
• Software Testing
• Production Monitoring
• User Experience Design
• Healthcare Data
• Security and Compliance
• HIPAA
• HITRUST
• API Integration
• Incident Troubleshooting
• Technical Documentation
• Cross-Functional Collaboration

About this role

Applied AI Engineer
Department: Technology
Location: Remote
FLSA Status: Exempt
People Leader: No
Travel: Less than 10%
Summary
Company Summary
Accuity partners with hospitals and health systems through a technology-enabled, physician-led model that improves clinical documentation integrity, coding accuracy, reimbursement optimization, and quality outcomes.
Job Summary
The Applied AI Engineer designs, prototypes, and builds production applications and features that apply existing AI models and services to clinical documentation and revenue cycle workflows. Reporting to the Vice President, Data Science and AI, this role owns the application layer, including user experience, service integration, prompt design, safeguards, testing, and production performance.
The engineer partners with AI Engineering, Data Science, Product, Design, and software engineering teams to turn validated AI capabilities into reliable tools that clinicians, clinical coders, and revenue cycle teams can use in their daily work. This includes embedded features within existing products and standalone internal applications.
Success in this role is measured by the ability to deliver reliable, user-centered AI applications that produce measurable improvements in the effectiveness, efficiency, and quality of clinical, coding, and revenue cycle workflows across the Accuity ecosystem.
Responsibilities
AI Application Development
Design, build, test, and maintain applications, features, and internal tools that use AI models, services, and agents provided by internal teams or approved vendors.
Translate user and business problems into defined AI application requirements, including expected behavior, prohibited behavior, failure handling, and human review.
Build embedded features within existing products and standalone applications when a dedicated tool better serves the workflow.
Integrate AI services into application interfaces, workflows, and data systems without duplicating existing platforms or infrastructure.
Develop prototypes to determine whether an AI capability solves the intended problem before committing to full production development.
Design clear interfaces between applications and AI services, including request formats, responses, error handling, latency requirements, and usage constraints.
Write clean, secure, maintainable, and reviewable production code.
Participate in code reviews, technical design discussions, testing, and release activities.
Prompt Interaction and Experience Design
Design and refine prompts, examples, structured outputs, and interaction patterns for specific application use cases.
Design how AI-generated content, supporting evidence, citations, uncertainty, and limitations are presented to users.
Provide users with appropriate review, editing, confirmation, and recovery options when AI output is incomplete or incorrect.
Partner with Product and Design to create interaction patterns suited to AI-assisted clinical and operational work.
Determine when an AI feature can act automatically, when it should request confirmation, and when it should defer to a person.
Build feedback mechanisms that capture user corrections, overrides, and other signals for product and model improvement.
Apply accessibility and usability standards to AI-powered interfaces.
AI Evaluation and Application Quality
Define measurable product-level acceptance criteria for AI-powered features.
Build repeatable tests that evaluate application behavior across expected, unusual, and high-risk scenarios.
Create and maintain regression tests for prompts, service integrations, and model or configuration changes.
Evaluate the quality of generated content, retrieval results, citations, structured outputs, and tool calls.
Test application behavior when AI services return incorrect, incomplete, delayed, or unavailable responses.
Analyze common failure patterns and work with AI Engineering and Data Science to resolve or mitigate them.
Document known limitations and ensure that product behavior reflects those limitations.
Escalate when a product requirement exceeds what an available model or service can reliably support.
Software Engineering and Production Ownership
Own AI-powered application features from technical design through production release and ongoing maintenance.
Develop automated tests appropriate to the application and its AI integrations.
Follow established source control, code review, continuous integration, and deployment practices.
Implement logging, telemetry, and monitoring to understand application performance and user behavior.
Monitor production reliability, latency, usage, and cost.
Investigate and resolve production issues involving application logic, integrations, prompts, or AI service behavior.
Maintain technical documentation covering architecture, dependencies, configuration, support procedures, and known limitations.
Work with platform and engineering teams to meet security, performance, availability, and support requirements.
Vendor and Tool Evaluation
Evaluate third-party AI products, APIs, and tools against defined product requirements, cost, reliability, security, and data-handling constraints.
Run structured trials using representative workflows and documented evaluation criteria.
Compare vendor claims with observed performance and document material limitations.
Recommend whether Accuity should build, buy, or integrate a capability in partnership with AI Engineering and Product.
Monitor developments in relevant AI tools and services and identify capabilities that may address established product needs.
Support the technical integration and ongoing evaluation of selected vendor products.
Security, Compliance, and Responsible AI
Build AI-powered applications in accordance with HIPAA, HITRUST, PHI protection requirements, and Accuity policy.
Use only approved models, services, vendors, data routes, and technical environments.
Follow required controls and contractual requirements when applications process prote...

About Accuity

Hospitals and Health Care · 501-1,000 employees · headquartered in Mount Laurel, New Jersey · founded 2016