Senior Data Engineer

About this role

Description
*Position at Healthmap Solutions*
Company Background
Healthmap Solutions is the future of specialty health management that focuses on progressive diseases, with a particular expertise in kidney health populations. Healthmap Solutions uses clinical big data resources and high-powered analytics to power complex specialty health management programs. Healthmap Solutions is a diverse, growing company committed to our clients and our employees. We are champions for better health, for those who need us most.


*Position Summary*
The Senior Data Engineer drives the design, development, and operational excellence of our data platform built on Databricks and the Delta Lakehouse Architecture. This role requires deep expertise in scalable ETL/ELT, Spark optimization, and modern data governance using Unity Catalog. The Senior Data Engineer serves as a technical leader, establishing best practices for code quality, performance, and automated deployments across all environments.


Responsibilities

• Design and implement scalable data pipelines using Delta Lake and manage enterprise-wide data access, security, and lineage using Unity Catalog
• Optimize large-scale Spark jobs (PySpark/SQL) and cluster configurations (Photon) to meet stringent SLA and cost performance targets across all workflows
• Build resilient data scheduling via Databricks Workflows (Jobs) and establish automated CI/CD pipelines for reliable code promotion across Dev, Staging, and Prod workspaces
• Migrate data and models from relational databases to Databricks
• Ensure best practices for development using industry standard development patterns
• Monitor data pipelines performance
• Partner with Data Management and Full Stack development engineers to operationalize models with current applications and processes
• Support existing data pipelines to ensure business continuity
• Stay updated with the latest trends and technologies in data engineering and cloud computing
• Perfor...