[Remote] Practice Architect - Test Lead
Note: The job is a remote job and is open to candidates in USA. TEKsystems Global Services (TGS) is an information technology services company focused on delivering solutions in Application Development, Quality Engineering, Data, AI, and managed services. The AI Test Lead Architect will define, build, and scale the AI Testing Practice, ensuring the quality and reliability of AI systems across various delivery models.
Responsibilities
• Define, build, and scale the AI Test Practice, including standards, playbooks, accelerators, and delivery models
• Establish a clear practice vision, roadmap, and KPIs aligned with business outcomes and client needs
• Act as the senior escalation point for AI quality, risk, and architecture decisions
• Identify internal process and technology gaps as well as provide R&D direction to address them
• Support hiring, mentoring, and career development across AI testing and quality engineering teams
• Design and implement AI runtime testing methodologies for deterministic and non deterministic systems (ML models, GenAI, LLMs)
• Define AI test architecture addressing:
• Data quality and lineage
• Model behavior and performance
• Bias, fairness, and explainability
• Drift, robustness, and security
• Develop and guide shift-left AI testing through early collaboration, strategic conversations, and shared standards that shape responsible, high-quality AI delivery from the start
• Establish AI quality gates embedded in CI/CD and MLOps workflows
• Create, evolve, and maintain AI testing accelerators across:
• SDLC integration
• Test automation
• Runtime monitoring and observability
• Risk and compliance reporting
• Standardize reusable frameworks for AI regression testing, GenAI prompt validation, drift detection, and reporting
• Maintain a big picture view of frameworks, dependencies, client goals, and technology direction
• Apply AI testing strategies across Agile, Waterfall, and hybrid delivery models
• Partner with engineering, data science, and product teams to embed quality early and continuously
• Act as a client facing AI testing advisor, engaging confidently with technical and executive stakeholders
• Lead client workshops, AI quality assessments, and roadmap definition efforts
• Support solution design, proposals, presentations, demos, and proofs of value
• Define and track AI quality KPIs, including model accuracy, drift, bias indicators, coverage, and production stability
• Establish AI testing governance aligned with client, regulatory, and risk requirements
• Build and deliver executive level reporting dashboards for AI quality, risk, and compliance
• Drive continuous improvement through retrospectives, metrics analysis, and feedback loops
Skills
• Proven experience designing and implementing AI runtime testing methodologies
• Strong background as a Test Architect for enterprise scale systems
• Demonstrated success delivering AI testing accelerators across SDLC, automation, and reporting
• Experience leading and scaling testing teams and practices
• Hands on experience with AI/ML and GenAI quality concepts (training, inference, drift, bias, explainability)
• Familiar with collecting and analysis of Precision, Recall, F1 Scoring, and Confusion Matrix
• Shift left testing and continuous quality engineering
• Agile, Waterfall, and hybrid delivery models
• CI/CD and MLOps integration
• Test automation frameworks and data validation strategies
• Monitoring, observability, and AI system reporting
• Excellent communication skills with both technical and non technical audiences
• Ability to influence decisions at engineering, product, risk, and executive levels
• Strong presentation and workshop facilitation skills
• Consultative mindset focused on outcomes, risk reduction, and business value
• Comfortable working in open, dynamic, and collaborative team environments
• Strong analytical, problem solving, and systems thinking abilities
• Self starter with a proactive leadership mindset
• Strong time management and organizational skills
• Passionate advocate for quality, trust, and responsible AI
• Desire to continuously improve AI quality processes and practices
• Minimum 12 years of experience in Quality Engineering, Testing, or Software Engineering
• Minimum 8 years in senior technical or architecture roles
• 3–5 years of experience leading teams or practices
• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
Benefits
• Medical, Dental, and Vision
• Critical Illness, Accident, and Hospital
• 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
• Life Insurance (Voluntary Life and AD&D for employee and dependents)
• Short and Long-Term Disability
• Health Spending Account (HSA)
• Transportation Benefits
• Employee Assistance Program
• Time Off/Leave (PTO, Vacation or Sick Leave)
Company Overview
• At TEKsystems, they understand people. Every year they deploy over 80,000 IT professionals at 6,000 client sites across North America, It was founded in 1994, and is headquartered in Hanover, Maryland, USA, with a workforce of 10001+ employees. Its website is http://www.teksystems.com.
Company H1B Sponsorship
• TEKsystems has a track record of offering H1B sponsorships, with 46 in 2025, 52 in 2024, 33 in 2023, 22 in 2022, 36 in 2021, 53 in 2020. Please note that this does not guarantee sponsorship for this specific role.
Apply tot his job
Apply To this Job
Responsibilities
• Define, build, and scale the AI Test Practice, including standards, playbooks, accelerators, and delivery models
• Establish a clear practice vision, roadmap, and KPIs aligned with business outcomes and client needs
• Act as the senior escalation point for AI quality, risk, and architecture decisions
• Identify internal process and technology gaps as well as provide R&D direction to address them
• Support hiring, mentoring, and career development across AI testing and quality engineering teams
• Design and implement AI runtime testing methodologies for deterministic and non deterministic systems (ML models, GenAI, LLMs)
• Define AI test architecture addressing:
• Data quality and lineage
• Model behavior and performance
• Bias, fairness, and explainability
• Drift, robustness, and security
• Develop and guide shift-left AI testing through early collaboration, strategic conversations, and shared standards that shape responsible, high-quality AI delivery from the start
• Establish AI quality gates embedded in CI/CD and MLOps workflows
• Create, evolve, and maintain AI testing accelerators across:
• SDLC integration
• Test automation
• Runtime monitoring and observability
• Risk and compliance reporting
• Standardize reusable frameworks for AI regression testing, GenAI prompt validation, drift detection, and reporting
• Maintain a big picture view of frameworks, dependencies, client goals, and technology direction
• Apply AI testing strategies across Agile, Waterfall, and hybrid delivery models
• Partner with engineering, data science, and product teams to embed quality early and continuously
• Act as a client facing AI testing advisor, engaging confidently with technical and executive stakeholders
• Lead client workshops, AI quality assessments, and roadmap definition efforts
• Support solution design, proposals, presentations, demos, and proofs of value
• Define and track AI quality KPIs, including model accuracy, drift, bias indicators, coverage, and production stability
• Establish AI testing governance aligned with client, regulatory, and risk requirements
• Build and deliver executive level reporting dashboards for AI quality, risk, and compliance
• Drive continuous improvement through retrospectives, metrics analysis, and feedback loops
Skills
• Proven experience designing and implementing AI runtime testing methodologies
• Strong background as a Test Architect for enterprise scale systems
• Demonstrated success delivering AI testing accelerators across SDLC, automation, and reporting
• Experience leading and scaling testing teams and practices
• Hands on experience with AI/ML and GenAI quality concepts (training, inference, drift, bias, explainability)
• Familiar with collecting and analysis of Precision, Recall, F1 Scoring, and Confusion Matrix
• Shift left testing and continuous quality engineering
• Agile, Waterfall, and hybrid delivery models
• CI/CD and MLOps integration
• Test automation frameworks and data validation strategies
• Monitoring, observability, and AI system reporting
• Excellent communication skills with both technical and non technical audiences
• Ability to influence decisions at engineering, product, risk, and executive levels
• Strong presentation and workshop facilitation skills
• Consultative mindset focused on outcomes, risk reduction, and business value
• Comfortable working in open, dynamic, and collaborative team environments
• Strong analytical, problem solving, and systems thinking abilities
• Self starter with a proactive leadership mindset
• Strong time management and organizational skills
• Passionate advocate for quality, trust, and responsible AI
• Desire to continuously improve AI quality processes and practices
• Minimum 12 years of experience in Quality Engineering, Testing, or Software Engineering
• Minimum 8 years in senior technical or architecture roles
• 3–5 years of experience leading teams or practices
• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
Benefits
• Medical, Dental, and Vision
• Critical Illness, Accident, and Hospital
• 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
• Life Insurance (Voluntary Life and AD&D for employee and dependents)
• Short and Long-Term Disability
• Health Spending Account (HSA)
• Transportation Benefits
• Employee Assistance Program
• Time Off/Leave (PTO, Vacation or Sick Leave)
Company Overview
• At TEKsystems, they understand people. Every year they deploy over 80,000 IT professionals at 6,000 client sites across North America, It was founded in 1994, and is headquartered in Hanover, Maryland, USA, with a workforce of 10001+ employees. Its website is http://www.teksystems.com.
Company H1B Sponsorship
• TEKsystems has a track record of offering H1B sponsorships, with 46 in 2025, 52 in 2024, 33 in 2023, 22 in 2022, 36 in 2021, 53 in 2020. Please note that this does not guarantee sponsorship for this specific role.
Apply tot his job
Apply To this Job