Sr. AI Engineer 4B

Responsibilities

The role involves developing and executing automated test suites for traditional application layers and AI-powered components like LLMs and RAG pipelines. You will also integrate these tests into CI/CD pipelines and collaborate with developers to manage and resolve defects.

Requirements

Candidates must have at least 3 years of experience in QA automation with strong programming skills in Python or Java. Proficiency in automation frameworks like Selenium or Cypress and a foundational understanding of AI/LLM testing methodologies are mandatory.

Skills

• Test Automation
• Python
• Java
• Selenium
• Cypress
• Playwright
• API Testing
• REST APIs
• CI/CD
• Jenkins
• GitHub Actions
• Azure DevOps
• Jira
• LLM Testing
• Cloud Testing
• Git

About this role

Sr. AI Engineer
Ready to turn bold ideas into real-world impact?
At Genpact, we don’t just adapt to change, we lead it. AI and digital innovation are transforming the way businesses work, and we’re at the forefront of it. Genpact’s AI Gigafactory, our industry-first accelerator, exemplifies how we scale advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. Whether tackling complex challenges through large-scale models or agentic AI, our breakthrough solutions tackle companies’ most complex challenges.

If you thrive in a fast-moving, innovation-driven environment, love building and deploying cutting-edge AI solutions, and want to push the boundaries of what’s possible, this is your moment.

Genpact (NYSE: G) is an agentic and advanced technology solutions company. We leverage process intelligence and artificial intelligence to deliver measurable outcomes. With a strong partner ecosystem and decades of client trust, we provide innovative solutions that transform how businesses run. Powered by a team with an active learning mindset and client centricity at its core, we deliver lasting value for the world’s leading enterprises.
Get to know us at genpact.com and on LinkedIn, YouTube, X, and Facebook.
Job Description
Role Overview
We are looking for a skilled QA Tester to join our Advanced AI Practice within the AI Application Engineering team. This is a hands-on individual contributor role for someone who enjoys building test automation and wants to grow into testing the next generation of Generative AI and Agentic AI applications. You will build and execute automated test suites for both traditional application layers (APIs, UI, integrations) and AI-powered components (LLM responses, RAG pipelines, agentic workflows), working under the guidance of senior AI engineers and QA/test leads.
Key Responsibilities
Test automation development: Build, maintain, and execute automated test scripts for functional, integration, and regression testing using frameworks such as Selenium, Cypress, Jmeter or Playwright.
API testing: Design and execute REST API test cases for application services and AI/model endpoints (request/response validation, contract testing, negative testing).
AI/LLM output testing: Support testing of GenAI and agentic application behavior including validating LLM responses against expected outcomes, checking for consistency, and flagging hallucinations or incorrect outputs using defined test datasets and evaluation criteria.
End-to-end suite development: Contribute to building and extending end-to-end automation suites covering UI, API, and data layers of AI-powered applications.
CI/CD integration: Integrate automated test suites into CI/CD pipelines (Jenkins, GitHub Actions, or Azure DevOps) to enable continuous testing.
Cloud testing: Execute and validate test cases for applications deployed on AWS or Azure environments.
Defect management: Log, track, and triage defects using Jira/HP ALM; collaborate with developers and DevOps engineers to drive issues to closure.
Test documentation & reporting: Prepare test cases, test data, execution reports, and coverage metrics for functional and AI-specific test cycles.
Test strategy support: Assist senior QA engineers and leads in defining test plans and strategies for new AI/GenAI features under their guidance.
Continuous learning: Stay current with evolving AI/LLM testing methodologies, tools, and industry practices, and bring improvement ideas to the team.
Must-Have Skills & Experience (Mandatory)
3+ years of hands-on experience in QA/Test Automation Engineering especially in AI application development.
Strong programming skills in Java or Python for test script development.
Hands-on experience with key automation frameworks: Selenium, Cypress, Jmeter or Playwright.
Solid experience in API testing (REST APIs) using tools such as Postman or RestAssured.
Practical experience in Functional, Integration, and Regression testing.
Experience integrating automated tests into CI/CD pipelines (Jenkins, GitHub Actions, or Azure DevOps).
Working knowledge of Git/GitHub for version control.
Understanding of cloud testing concepts on AWS and Azure.
Foundational understanding of AI concepts, LLMs, and AI-specific testing considerations (e.g., prompt-response validation, output consistency checks).
Familiarity with test/defect management tools such as Jira or HPALM.
Good-to-Have Skills
Exposure to AI/LLM evaluation frameworks and tools (e.g., Ragas, DeepEval, Promptfoo, LangSmith/Langfuse) for automated GenAI output evaluation.
Exposure to testing RAG pipelines, agentic workflows, or conversational AI/chatbot applications.
Performance/load testing experience using JMeter.
DevOps exposure: containerization (Docker) and awareness of Kubernetes for test environment setup.
Awareness of responsible AI testing concepts: bias, fairness, safety, and guardrail validation.
Exposure to GCP in addition to AWS/Azure.
Exposure to Agile/Scrum ways of working.
Qualifications
Education: Minimum Bachelor's degree – B.E./B.Tech/B.Sc. in Computer Science, Information Technology, Electronics, or a related field. Master's degree (M.Tech/MCA/M.Sc.) is a plus.
Certifications (preferred): ISTQB Foundation Level, AWS/Azure/GCP cloud certifications, Agile/Scrum certifications.
Bonus: Personal or academic projects involving test automation, API testing, or AI/LLM application testing.
Qualifications
Bachelors - Computer Science, Bachelors - Mathematics, Bachelors - Software Engineering, Masters - Artificial Intelligence, Masters - Statistics
Certifications
BUILD - GENERATIVE AI - Genpact, Certification in Python and Machine Learning - Decodr Technology, Deep Learning Specialization - Coursera, Generative AI & Large Language Models - Databricks Academy, IBM Machine Learning Specialization - IBM
Required Skills
API Testing, Application Integrations, Perform Testing, Software Application Testing, Software Quality Assurance (SQA) Test...

About BrightClaim

Business Consulting and Services · 10,001+ employees · headquartered in New York, NY