Test Lead (AI Validation & Integration) - remote job

Digital Forms is hiring a Test Lead (AI Validation & Integration) - a lead role, which means responsibility for a team or a direction. The role sits in IT and programming and is fully remote. The company places no restriction on where the candidate lives, so you can apply from anywhere.

The main tools named in the advert are: Salesforce, Agile / Scrum. Your CV should show concrete examples of exactly these skills.

The employer published the pay: 19k - 30k PLN a month. The advert sets no condition on working hours.

This vacancy passed an automated check: the list excludes any advert requiring a foreign work permit, visa sponsorship, a particular citizenship, or residence in a specific country.

We are seeking an experienced Test Lead to guide the Quality and Validation strategy for a high-impact project, particularly in a regulated industry that incorporates AI components and Salesforce integration. This role will oversee both the creation and implementation of the Testing & AI Validation Strategy. You will ensure that the output of the solution is accurately validated for correctness, reliability, and safety, connecting the development of new AI Agent components with the client’s Salesforce environment.

Responsibilities

- Strategy & Planning: - Develop and refine the QA Strategy, covering Functional Unit, E2E, Integration, Regression, and User Acceptance Testing (UAT). - Ensure alignment with non-functional requirements (NFRs), such as GDPR, HIPAA, and HITRUST. - Define validation protocols for AI components, establishing criteria for "correct" outputs and setting evaluation standards, including handling scenarios for hallucinations and grounding rules. - Outline detailed plans for Test Environments (Dev/Test/Prod) and establish a Test Data Strategy to cover standard and edge cases, as well as data ingestion issues.

- Execution & Reporting: - Conduct functional, regression, exploratory, performance, and load tests. - Coordinate "Joint End-to-End Validation" efforts, working closely with the client team to manage the testing of Salesforce integration. - Integrate testing activities into the CI/CD pipeline for ongoing quality assurance. - Manage Jira test case structures that align with user stories and acceptance criteria. - Provide regular updates, tracking QA metrics like defect ratios.

Project Context - This is a new project for a client we have previously collaborated with. We are now in the final stages of the Discovery Phase and moving towards the Delivery phase. - The project involves developing a data validation and automation engine that processes structured and unstructured data through a complex logic framework. AI models interpret user-submitted data, identifying inconsistencies and missing information before converting the results into a structured format. Key elements include automated data ingestion, rule-based validation, and integration with the existing enterprise architecture in a highly regulated environment.

Technical Requirements - Proficiency in English. - Experience in designing and implementing comprehensive end-to-end testing strategies for complex solutions. - Expertise in validating AI models and executing end-to-end tests for complex, integrated systems. - Strong background in testing non-functional requirements such as performance, reliability, auditability, and security. - In-depth knowledge of privacy and sensitive data handling, including minimizing PII/PHI and data masking/anonymization. - Hands-on experience in automation using at least one robust framework (e.g., Python/Java/JS + frameworks like pytest/JUnit). - Ability to monitor and report on QA metrics such as defect ratios. - Excellent communication skills, both written and verbal. - Proficiency in Jira.

Availability - Full-time availability, with preferred overlap during 14:00-18:00 to ensure smooth collaboration with the USA-based client.

Additional Desirable Skills - Familiarity with AWS platform. - ISTQB or similar certifications. - Previous experience in fast-paced consulting or client-facing projects.

Key Highlights - Lead the QA and AI validation process for a regulated AI + Salesforce project, from strategy formulation to hands-on testing, ensuring end-to-end validation and close collaboration with the client.

Originally posted on Himalayas