Senior Data Engineer

Responsibilities

The Senior Data Engineer will design, build, and maintain scalable data pipelines and cloud-native architectures to support AI and LLM-driven applications. They will also lead technical workstreams, mentor junior engineers, and engage directly with clients to translate business needs into robust data solutions.

Requirements

Candidates must have a bachelor's degree and at least 5 years of hands-on data engineering experience, specifically with AWS data services. Proficiency in designing production-grade ETL/ELT pipelines and experience supporting AI or LLM-adjacent data workflows are required.

Skills

• Data Engineering
• AWS
• ETL/ELT
• Data Pipelines
• LLM
• AI
• Vector Stores
• Cloud Architecture
• Data Warehousing
• Streaming Architectures
• Data Governance
• Infrastructure-as-Code
• Mentorship
• Client Engagement
• Jira
• Python

About this role

POSITION SUMMARY

The Senior Data Engineer is a technically deep, independently operating practitioner who designs, builds, and maintains the data foundations that power RS21's analytics, AI, and LLM-driven applications across both product and professional services work. This role focuses on data readiness, cloud-native data architecture, and AI enablement, ensuring that LLM-powered systems are supported by reliable, scalable, and high-quality data pipelines.

At the Senior level, this person does not require day-to-day direction. They lead pipeline architecture decisions, establish data contracts for their workstreams, mentor junior and core-level engineers, and serve as a reliable technical contributor in client-facing engagements. They translate business and product requirements into well-structured, maintainable technical solutions, and they understand how their work connects to the broader delivery plan.

This role bridges hands-on execution with team-level technical leadership. The Senior Data Engineer is the person a project team depends on to own the data layer end to end, flag problems before they become blockers, and bring enough communication skill to engage directly with clients and cross-functional partners without requiring a translator.

KEY RESPONSIBILITIES

DATA PIPELINES & PLATFORMS

* Lead pipeline architecture decisions for assigned workstreams, including batch and real-time ingestion, transformation, and delivery to analytics and AI layers.
* Design, build, and maintain scalable ETL/ELT workflows with strong data quality, lineage, monitoring, and observability built in from the start.
* Establish data contracts and pipeline standards for the projects you own, and ensure those standards are followed by other contributors on the workstream.
* Ensure data reliability, performance, and scalability across platforms; proactively surface bottlenecks and recommend improvements before they affect delivery.
* Support both batch and streaming ingestion patterns, including real-time data pipeline design and implementation.

LLM ENABLEMENT & AI DATA FOUNDATIONS

* Design and implement data pipelines that support LLM and AI use cases, including:

* Document and unstructured data ingestion
* Data preprocessing, enrichment, and embedding generation
* Vector store integration and retrieval-optimized data structures

* Lead embedding pipeline architecture and vector store configuration in collaboration with developers building LLM features.
* Ensure data freshness, lineage, and governance for AI-powered systems.
* Optimize data structures and retrieval patterns to support efficient LLM context usage.
* Contribute to RAG pipeline design and maintain awareness of foundation model data requirements across active engagements.

CLOUD ARCHITECTURE (AWS)

* Architect and provision AWS services to support data and AI workloads, including S3, Glue, Glue Catalog, Redshift, Athena, EMR, Kinesis, MSK, Lambda, Step Functions, and EventBridge.
* Lead FedRAMP-compliant architecture design for data environments where required; apply security and access control patterns using Lake Formation and IAM.
* Contribute to reusable reference architectures for data lakes, warehouses, streaming systems, and AI-ready platforms.
* Partner with platform and DevOps teams to ensure secure, cost-effective, and scalable cloud deployments.
* Apply infrastructure-as-code and automation practices to data platform provisioning and maintenance.

CLIENT & STAKEHOLDER ENGAGEMENT

* Participate in client discovery and requirements-gathering sessions, translating operational needs into concrete data architecture recommendations.
* Communicate clearly with both technical and non-technical stakeholders, adapting depth and language to the audience without losing precision.
* Assess and document client data readiness for analytics and AI adoption; identify gaps and propose remediation paths to the lead architect or engagement manager.
* Support the technical narrative during delivery, contributing to solution design documents, architecture diagrams, and client-facing documentation.
* Build working trust with client counterparts through consistency, follow-through, and clear expectation-setting.

TECHNICAL DELIVERY LEADERSHIP

* Own the data engineering workstream within a project delivery plan, including task decomposition, estimation, and sequencing in Jira.
* Understand how individual tickets connect to the larger delivery arc, and surface dependencies or risks before they block progress.
* Facilitate technical planning and review ceremonies for the data workstream; provide clear updates to project managers and technical leads.
* Partner with project managers and product owners to keep the data engineering track aligned with contractual and delivery constraints.
* Coordinate across engineering, platform, analytics, and ML teams to ensure data pipelines meet downstream requirements.

PRODUCT & DATA READINESS SUPPORT

* Support the evolving data architecture behind RS21 product capabilities, including predictive and real-time ML systems.
* Assess and improve internal and client data readiness for analytics and AI adoption.
* Translate business, product, and client needs into scalable data architectures that can be maintained and extended by the broader team.
* Document data architectures, pipeline designs, and integration patterns to support transparency and reuse across engagements.

STAFF DEVELOPMENT & KNOWLEDGE SHARING

* Mentor junior and core-level data engineers through code reviews, architecture critiques, and hands-on guidance on active projects.
* Identify skill gaps in team members and work with technical leadership to address them through structured coaching or pairing.
* Contribute to internal playbooks, onboarding materials, and engineering standards that reduce tribal knowledge and improve team consistency.
* Help team members understand the business conte...