Senior Data Engineer

About this role

InnoVet Health, a small and growing business that provides health IT professional services to the Department of Veterans Affairs (VA) is looking for an experienced Sr. Azure Data Engineer proficient in Databricks, Delta Lake, and medallion/lakehouse architectures to support healthcare analytics, interoperability, and data integration needs. You will build scalable data pipelines, ensure data quality, and support advanced analytics across clinical, operational, and regulatory datasets. Your work will directly impact VA healthcare delivery by building an analytical infrastructure that enables AI/ML and advanced analytics to deliver short-term and longitudinal insights to healthcare professionals serving Veterans. You will work in an agile environment alongside VA and contractor stakeholders. It is flexible, full-time, and does not require relocation (work-from-home). The pay, benefits, and growth potential are competitive.

Responsibilities

• Gather and translate business, technical, and functional requirements into data architecture and pipeline design decisions. Design and develop Azure Data Factory and Databricks-based ETL/ELT pipelines using PySpark, Delta Lake, and medallion/lakehouse architecture.
• Ingest and transform healthcare data (clinical, claims, FHIR, HL7, EHR, ADT, PGHD) from diverse sources.
• Build secure, scalable solutions using Azure Data Lake Storage, ADF, and Event Hubs, and related services, with attention to latency and reliability requirements.
• Implement data quality, lineage, and governance using Microsoft Purview.
• Optimize Databricks jobs (performance tuning, cluster sizing, Z-ordering, partitioning).
• Enforce HIPAA-aligned security practices: RBAC, Key Vault, private endpoints, PHI protection.
• Collaborate with data scientists, analysts, and clinical informatics teams.
• Stay up to date with emerging technologies and trends in data engineering and healthcare data management.
• Present and discuss results with IT and business stakeho...