Software Engineer

Company: Haystack

Location: New York City Metropolitan Area

Type: Full-time

Level: mid

Posted: 2026-03-02

About this role

AI Engineer | New York | Hybrid

We're working with a Fortune 500 financial powerhouse and leader in the life insurance sector on this exciting opportunity.

Shape the future of an industry by building sophisticated Agentic AI and Generative AI systems from the ground up. This isn't just about prompt engineering; you’ll be working alongside PhDs in physics and math to deploy production-grade LLM applications, RAG pipelines, and multi-step agentic workflows that drive multi-billion dollar strategic decisions.

The Role

• Lead the end-to-end development of scalable AI solutions, from initial ideation and prototyping to full-scale production deployment using Python and cloud-native architectures.

• Design and implement advanced Agentic AI systems involving tool-use, multi-step workflows, and autonomous behavior evaluation across various foundation models.

• Collaborate with cross-functional teams to build robust, production-grade AI pipelines and APIs while establishing enterprise-wide best practices for Responsible AI.

• Architect and optimize data strategies using vector databases, semantic search, and complex SQL database design to power LLM-based applications.

• Prototype impactful AI-powered applications, including interactive web dashboards and sophisticated visualizations, to translate complex data into business intelligence.

What You'll Need

• 5+ years of a strong foundation in Python, SQL, and statistical modeling, with a Master’s degree in a quantitative field like Computer Science, Math, or Physics.

• 3+ years of dedicated experience in AI Engineering or Data Science, with a proven track record of shipping end-to-end ML/AI solutions in an enterprise environment.

• 2+ years of hands-on expertise with LLM orchestration frameworks such as LangChain, AgentCore, MCP, or Strands, specifically focusing on RAG and prompt management.

• Deep technical experience with containerization and cloud-native technologies, specifically Docker and Kubernet...

Create Your Job Alert

Other Software Jobs

Other Jobs in New York City Metropolitan Area