Customer Support Specialists, AI Response Evaluation
About mpathic.ai
Keeping the human in AI. mpathic is a trusted leader in advancing quality and safety in AI systems through expert-led evaluation and human data. We partner with leading technology companies to support red teaming, trust & safety, expert annotation, and model evaluation across high-stakes domains.
Our reviewers bring deep expertise in behavioral analysis, conversational design, mental health, and increasingly, financial and enterprise decision-making contexts.
About the Role
mpathic is seeking part-time Customer Support Specialists to support a red-teaming and quality assurance (QA) campaign focused on evaluating AI system behavior in customer-facing interactions.
In this role, you will review AI-generated responses and multi-turn conversations to assess quality, accuracy, tone, and safety—similar to evaluating customer support agents. You will identify issues such as incorrect information, poor customer experience, inappropriate tone, or failure to resolve user needs effectively.
This is not a traditional support role. Instead, it focuses on evaluating AI performance, applying your customer support expertise to assess how well AI systems handle real-world customer interactions.
What You’ll Be Working On
You will help identify, prevent, and characterize risks that emerge when users interact with AI-powered support systems.
Responsibilities may include:
Reviewing AI-generated customer support responses for accuracy, clarity, and helpfulness
Evaluating tone, empathy, and professionalism in customer interactions
Identifying poor support behaviors (e.g., dismissiveness, lack of resolution, incorrect guidance)
Assessing whether AI responses appropriately address customer intent and resolve issues
Evaluating how AI handles frustration, confusion, or escalating user behavior
Identifying gaps in responses, including missing steps, unclear instructions, or lack of follow-through
Detecting overconfident, misleading, or incorrect responses
Evaluating multi-turn conversations for consistency, escalation handling, and resolution quality
Identifying patterns of agreement without problem-solving (e.g., empty empathy or unhelpful reassurance)
Participating in or reviewing red teaming exercises to surface failure modes in support scenarios
Supporting quality assurance (QA) to ensure consistency across evaluations
Documenting edge cases, failure patterns, and customer experience risks
Providing structured written feedback to internal teams
Collaborating with interdisciplinary teams on AI quality, safety, and evaluation frameworks
Maintaining strict confidentiality and quality standards
This role requires strong judgment, attention to detail, and the ability to evaluate nuanced customer interactions.
What We’re Looking For
Successful candidates are detail-oriented, customer-focused, and experienced in evaluating or delivering high-quality support in fast-paced environments.
Professional experience in one or more of the following:
Customer support (chat, email, or phone)
Call centers or contact centers
Technical support or help desk roles
Online support environments (SaaS, e-commerce, fintech, etc.)
Strong understanding of:
High-quality customer experience and service standards
Effective communication in customer-facing interactions
Common support workflows (triaging, troubleshooting, escalation)
How customers interpret and respond to support interactions
Ability to identify:
Poor or incomplete resolutions
Incorrect or misleading information
Inappropriate tone (e.g., robotic, dismissive, overly casual)
Lack of empathy or ineffective de-escalation
Failure to follow support best practices
Strong written communication skills and ability to clearly explain reasoning
Experience with or interest in:
Evaluating conversations or QA for support teams
Working with AI tools, chatbots, or automated systems
Assessing how systems perform under real customer behavior
Comfort with:
Tech tools and platforms (Slack, spreadsheets, dashboards)
Evaluating AI-generated responses (no coding required, but must be tech-comfortable)
Ambiguity, iteration, and feedback-driven workflows
Willingness to:
Sign NDAs and work with sensitive content
Nice to Have (Not Required)
Experience in QA, coaching, or training within support teams
Background in trust & safety or content moderation
Experience with chatbot or AI-assisted support tools
Familiarity with CX metrics (CSAT, QA scoring, resolution rate)
Interest in AI, automation, or improving customer experience systems
Compensation
$30-40/hour, depending on experience and specific project tasks/difficulty
Apply Now