Senior Data Engineer

Responsibilities

You will lead the technical migration of data pipelines to meet strict user-data lifecycle requirements including encryption and retention. Additionally, you will establish engineering processes such as testing standards, release checklists, and incident post-mortems to ensure long-term pipeline reliability.

Requirements

The role requires deep hands-on experience with Airflow, Spark, dbt, BigQuery, and Python, along with a proven track record of leading pipeline migrations. Candidates must also demonstrate experience in introducing engineering processes to a team and providing effective technical mentorship.

Skills

• Airflow
• Spark
• Dbt
• BigQuery
• Python
• Data Engineering
• Pipeline Migration
• Encryption
• Data Retention
• Data Deletion
• Testing Standards
• Incident Management
• Mentoring
• Architecture Design
• GCP KMS
• Kafka

About this role

Role: Senior Data Engineer
Job Ref Id: JD-1292
Competence level: 5+ years
Start date: 2026-11-09
End date: 2027-04-30
Extension option: Yes
Scope: 100%
Location: Stockholm
Remote work: On-site; remote considered
Language: English
Application deadline: 2026-11-01
Number of consultants: 1
Security clearance: Not stated
About the assignment
We are looking for a Senior Data Engineer for an interim position in our Trust Intelligence & Calling department. Note: We prefer on-site but we're considering remote candidate as well. We are re-architecting our data pipelines to meet stricter requirements on user-data lifecycle: encryption, retention, and deletion across historical and derived data. You lead the technical migration and establish the engineering processes that keep the pipelines reliable after it.
Responsibilities
Propose and drive the architecture for user-data handling across Airflow, Spark, dbt, and BigQuery: encryption points, derived tables, backfills, partitions.
Migrate existing pipelines one at a time without changing their output, behind a regression harness you build.
Design the processes the team keeps after the migration: pipeline change review, testing standards, incident post-mortems, release checklist.
Remove duplicated steps and copy-pasted logic as you pass through each pipeline.
Build verification jobs that prove the data-lifecycle rules hold end to end.
Pair with and mentor a data engineer on the team, with the goal that the team owns the migrated pipelines independently.
Share pipeline maintenance and incident handling; fix root causes.
Document decisions as short architecture decision records.
Requirements
Mandatory
Deep hands-on experience with Airflow, Spark, dbt, BigQuery, and Python.
At least one pipeline migration you led technically from start to finish.
Experience refactoring a live pipeline while proving its output is unchanged.
Experience introducing engineering processes (testing, review, incident handling) to a team and making them stick.
Mentoring experience with a concrete outcome.
Nice-to-have
A prior project on encryption at rest, data retention, or deletion compliance (for example GDPR).
Envelope encryption with a key manager (for example GCP KMS), BigQuery AEAD functions, or Tink.
BigQuery column-level security, policy tags, Data Catalog.
Config Connector (KCC) for managing GCP resources from Kubernetes.
Events with Kafka.

About Aqilea (formerly Soltia)

IT Services and IT Consulting · 51-200 employees · headquartered in Liljeholmen, Stockholm County · founded 2012