QA Engineer (SDET) — AI, Data & Platform Quality

About this role

*Programmatic QA • Testing for LLMs & Agents • Data Quality • Platform Reliability*

About Finalytics.ai

Finalytics.ai is the leading provider of personalization for the financial industry. Our platform combines data integrations, machine learning, and real-time technology to make digital experiences more relevant and higher-converting for credit unions and banks. We're a growing startup led by industry veterans, building the next generation of AI-driven personalization.

Why This Role Is Different

QA at Finalytics goes well beyond clicking through a UI. Our platform makes model-driven decisions, runs LLMs and agents that generate content and answer questions, and depends on data pipelines that feed those models every day — and all of it has to be tested programmatically.

We're looking for an engineering-minded QA team contributor to help build quality across three areas: our core personalization features, our LLM and agentic capabilities, and the data that powers them. This is a coding role, embedded in the same repo and release flow as our engineers that will report directly to the CTO. You won't just find bugs — you'll build the automated tests, evals, and data checks that let a small team ship trustworthy AI every sprint.

Our stack is Python/Django with a JavaScript personalization tag, backed by MySQL, Celery, BigQuery, and AWS.

What You'll Do

1. Programmatic QA of Core Features

• **Extend our scenario test runner** — a proprietary harness that captures real production personalization requests and replays them across environments, asserting on expected algorithms and content selection. Grow it into automated regression across every client.
• **Write automated tests in Python with pytest** across our tiers — unit, integration, HTTP, and end-to-end.
• **Build headless Playwright end-to-end tests** to verify how personalized content and tracking render on real client pages.
• **Harden the pre-deploy quality gate** and pre-commit checks that ...