Software Engineer
Company: Haystack
Location: New York City Metropolitan Area
Type: Full-time
Level: mid
Posted: 2026-03-02
About this role
AI Engineer | New York | Hybrid
We're working with a Fortune 500 financial powerhouse and leader in the life insurance sector on this exciting opportunity.
Shape the future of an industry by building sophisticated Agentic AI and Generative AI systems from the ground up. This isn't just about prompt engineering; you’ll be working alongside PhDs in physics and math to deploy production-grade LLM applications, RAG pipelines, and multi-step agentic workflows that drive multi-billion dollar strategic decisions.
The Role
• Lead the end-to-end development of scalable AI solutions, from initial ideation and prototyping to full-scale production deployment using Python and cloud-native architectures.
• Design and implement advanced Agentic AI systems involving tool-use, multi-step workflows, and autonomous behavior evaluation across various foundation models.
• Collaborate with cross-functional teams to build robust, production-grade AI pipelines and APIs while establishing enterprise-wide best practices for Responsible AI.
• Architect and optimize data strategies using vector databases, semantic search, and complex SQL database design to power LLM-based applications.
• Prototype impactful AI-powered applications, including interactive web dashboards and sophisticated visualizations, to translate complex data into business intelligence.
What You'll Need
• 5+ years of a strong foundation in Python, SQL, and statistical modeling, with a Master’s degree in a quantitative field like Computer Science, Math, or Physics.
• 3+ years of dedicated experience in AI Engineering or Data Science, with a proven track record of shipping end-to-end ML/AI solutions in an enterprise environment.
• 2+ years of hands-on expertise with LLM orchestration frameworks such as LangChain, AgentCore, MCP, or Strands, specifically focusing on RAG and prompt management.
• Deep technical experience with containerization and cloud-native technologies, specifically Docker and Kubernet...