Senior QA Analyst - AI/ML

Job Title: Senior QA Analyst - AI/ML Applications (AI/ML QA Engineer)

Experience Required: 8-10 Years

Job Summary

We are seeking a Senior QA Analyst - AI/ML Applications with strong expertise in software quality assurance, API testing, AI/ML systems, and data validation. The ideal candidate will be responsible for validating AI/ML applications, machine learning models, data pipelines, and Generative AI solutions while ensuring high-quality, scalable, and reliable software delivery.

This role requires collaboration with Data Scientists, ML Engineers, DevOps, and Product teams to validate machine learning models, AI workflows, APIs, and enterprise applications throughout the Software Development Lifecycle (SDLC).

Key Responsibilities

AI/ML Application Testing

• Design and execute comprehensive test strategies for AI/ML applications and machine learning workflows.
• Validate AI model outputs, prediction accuracy, and overall system performance.
• Perform functional, integration, regression, and end-to-end testing of AI-powered applications.
• Test Large Language Model (LLM) applications, prompt responses, and Generative AI workflows.
• Validate model behavior across edge cases, bias detection, drift analysis, and performance benchmarking.

Data Validation & ETL Testing

• Validate data pipelines, ETL workflows, and preprocessing logic.
• Perform data quality testing and database validation.
• Verify data consistency, completeness, and integrity across multiple systems.
• Test data transformations and ML feature engineering pipelines.

API & Backend Testing

• Perform REST API testing using Postman or similar tools.
• Validate model inference APIs and integration endpoints.
• Test backend services supporting AI/ML applications.
• Verify API responses, authentication, and performance.

Automation Testing

• Develop and maintain automated test scripts for AI/ML workflows.
• Execute automation using Selenium, PyTest, TestComplete, or similar frameworks.
• Integrate automated testing into CI/CD pipelines.

Defect Management & Release Support

• Prepare test plans, test cases, and QA documentation.
• Participate in UAT, production validation, and release verification.
• Track defects using JIRA, Rally, or similar tools.
• Collaborate with development teams to resolve issues efficiently.

Collaboration

• Work closely with Data Scientists, ML Engineers, DevOps, Business Analysts, and Product Owners.
• Participate in Agile ceremonies including Sprint Planning, Daily Stand-ups, Reviews, and Retrospectives.
• Provide quality recommendations throughout the AI/ML development lifecycle.

Required Skills

QA & Testing

• Functional Testing
• Regression Testing
• Integration Testing
• End-to-End Testing
• UAT
• Defect Management

API Testing

• REST APIs
• Postman
• API Validation
• Backend Testing

AI/ML

• AI/ML Fundamentals
• Machine Learning Model Validation
• Prediction Validation
• Model Evaluation
• AI Workflow Testing
• AI Model Performance Testing

Data & Database

• SQL
• Database Testing
• ETL Testing
• Data Validation
• Data Quality

Programming

• Python (Basic to Intermediate)
• Automation Scripting

Tools

• Selenium
• TestComplete
• JIRA
• Rally
• PyTest

Preferred Skills

• Generative AI (GenAI)
• Large Language Models (LLMs)
• Prompt Validation
• Prompt Engineering
• Output Verification
• AI Model Drift Analysis
• Bias Testing
• ML Lifecycle
• Databricks
• Apache Spark
• Apache Airflow
• Healthcare Domain (Claims, Enrollment, Payer Systems)

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 8-10 years of experience in Software QA and Testing.
• Hands-on experience with AI/ML application testing.
• Strong knowledge of API testing, SQL, ETL, and automation frameworks.
• Experience working in Agile/Scrum environments.

Preferred Certifications

• ISTQB Certified Tester
• Certified Selenium Professional
• Microsoft AI Fundamentals (AI-900)
• Azure AI Engineer Associate
• AWS Certified Machine Learning (Preferred)