AI Evals Engineer — Evaluation Datasets & Ground Truth

About this role

Prophetic Software is not able to sponsor employment visas now or in the future. Candidates must be authorized to work in the United States without current or future sponsorship to be considered for this role.
About Prophetic
Real estate development is a multi-billion-dollar industry that has run on fragmented data, manual processes, and gut instinct for decades. Prophetic is changing that. We're building the AI-native platform that enables homebuilders, developers, and investors to find, analyze, and act on land opportunities from a single system — powered by proprietary technologies that process billions of data points across all 50 states. We are the market leader in our space, and our customers don't just use the product — they love it. We're not making teams more efficient. We're changing how they operate.


Prophetic is scaling fast — demand is outpacing our ability to hire, and we're just getting started. This is a once-in-a-lifetime team: sharp, low ego, deeply collaborative, and obsessed with building the best product in the industry. Come disrupt an industry with us.


Why this role exists
We iterate on pipeline components constantly: prompts, models, classifier thresholds, retrieval logic, harness design. We are hiring a full-time engineer to produce, maintain, and defend the evaluation datasets that let every meaningful component in our stack be measured. You will own ground truth at Prophetic.


This is not an evals-infrastructure role and it is not a labeling-operations role, although you’ll touch both. Your deliverable is
trusted data
: for a given module, a versioned set of inputs and expected outputs, plus a written definition of what “correct” means and how confident we should be in the labels.


What You’ll Do
Decide what needs to be measured, and how

• Read our pipelines and system architecture, sit with product and engineering, and decompose each system into evaluable modules with explicit input → expecte...