Principal Data Engineer

About this role

Become a part of our caring community
The Principal Data Engineer owns the design and delivery of the NBA platform's data pipelines on the Databricks Lakehouse. This is a hands-on engineering role first - you are expected to be a significant individual contributor building production pipelines every sprint. You also lead a small team of data engineers and contractors, and are accountable for the quality, reliability, and timeliness of the data that feeds the platform's AI models and decisioning system.


**What You'll Own**

• Pipeline delivery - design, build, and own the Bronze/Silver/Gold lakehouse pipelines that power the NBA platform: member profiles, clinical signals, behavioral data, web clickstream, and socioeconomic features
• Hands-on development - actively write and ship production SQL, Python, and Spark code alongside your team; this is not a supervisory role
• Data modeling - own the Gold layer schema that serves as the canonical member feature set for model training and batch scoring; make design decisions that balance flexibility, performance, and governance
• Data quality - build and maintain data quality checks, null/sparsity monitoring, and pipeline alerting so the team knows before anyone else when something is wrong
• External data integration - onboard new data sources (clinical feeds, government datasets, third-party signals) from raw ingestion through production-ready features
• Contractor management - manage data engineering contractors within your pod; set expectations, review work, and hold the team to quality standards
• Cross-functional partnership - work closely with the Principal Decision Intelligence Engineer on feature requirements, and with software engineering pods on how data flows into the decisioning system

**What You'll Need**

• Sponsorship is not available for this role
• Bachelor's degree in computer science or relevant field
• 10+ years of data engineering experience with at least 1-2 years in a tech lead or...