AI Member Experience Quality Analyst

We’re hiring an AI Member Experience Quality Analyst who is passionate about quality and eager to influence how AI‑powered member experiences are delivered across daily operations. In this role, success is defined by the impact of your quality insights and the improvements they drive across AI performance, compliance, and member trust. You’ll conduct rigorous quality evaluations grounded in real operational workflows, identify high‑risk issues, and translate findings into clear, actionable recommendations that strengthen AI responses at scale. You’ll partner closely with training, operations, product, and business stakeholders to ensure quality learnings directly influence how our AI learns, evolves, and performs in live member environments. By connecting quality data, trends, and operational realities, you’ll tell a compelling quality story that reduces risk, elevates the member experience, and helps set the standard for AI quality excellence. Our ideal candidate brings a strong foundation in quality, operations, or analytics and thrives in environments where precision, judgment, and collaboration matter. You’re comfortable working with data, spotting patterns and risks, and clearly communicating insights to diverse stakeholders. You understand how quality shows up in day‑to‑day operations and are motivated by the opportunity to influence AI‑driven experiences at scale. If you’re excited to apply your quality and operational expertise to the future of AI‑driven member experiences, apply today and help shape how trust, accuracy, and innovation come together at scale. Job Responsibilities Be the quality champion: conduct thorough reviews of AI conversations, starting with 100% coverage at launch and evolving to strategic sampling as we prove excellence Apply your expertise: evaluate AI interactions against approved quality and experience standards, spanning accuracy, completeness, compliance, and member impact Document with precision: capture quality findings, categorize issues by impact (critical, major, minor), and track trends that tell the quality story Drive AI improvement: identify training gaps and knowledge base opportunities, providing specific, actionable recommendations to appropriate stakeholders Bridge quality and training: collaborate with Business Requirements Analysts to transform your findings into AI enhancements Create quality narratives: prepare compelling reports and scorecards that highlight trends, celebrate wins, and focus attention on opportunities Ensure consistency: participate in calibration sessions that align quality evaluation across the team Protect compliance: review AI responses for regulatory adherence, ensuring required disclosures and catching prohibited content Sound the alarm: escalate critical quality issues immediately, protecting members and the organization from potential harm Investigate root causes: support deep-dive analysis when quality incidents or member complaints require thorough understanding Shape quality standards: contribute to developing and refining the evaluation criteria and scoring approaches that define AI excellence Build quality knowledge: maintain evaluation guides, job aids, and training materials that elevate quality practice across the team Job Qualifications Education Associate degree or equivalent work experience required. Equivalent experience is defined as 2 years of professional work experience in healthcare or related field. Experience 3+ years of experience in quality assurance, quality monitoring, or auditing with a track record of driving improvement 2+ years of experience in health insurance or healthcare operations with strong product and process knowledge SkillsCertifications Expertise in quality evaluation methodologies, scoring rubrics, and calibration processes Deep knowledge of health insurance products, terminology, and member service standards Eagle-eye attention to detail with ability to identify errors and inconsistencies others might miss Strong written communication skills; you can document findings clearly and deliver constructive feedback Number of Openings Available 1 Worker Type: Employee Company: BCBST BlueCross BlueShield of Tennessee, Inc. Applying for this job indicates your acknowledgement and understanding of the following statements: BCBST will recruit, hire, train and promote individuals in all job classifications without regard to race, religion, color, age, sex, national origin, citizenship, pregnancy, veteran status, sexual orientation, physical or mental disability, gender identity, or any other characteristic protected by applicable law. Further information regarding BCBST's EEO Policies/Notices may be found by reviewing the following page: BCBST's EEO Policies/Notices BlueCross BlueShield of Tennessee is not accepting unsolicited assistance from search firms for this employment opportunity. All resumes submitted by search firms to any employee at BlueCross BlueShield of Tennessee via-email, the Internet or any other method without a valid, written Direct Placement Agreement in place for this position from BlueCross BlueShield of Tennessee HR/Talent Acquisition will not be considered. No fee will be paid in the event the applicant is hired by BlueCross BlueShield of Tennessee as a result of the referral or through other means. As Tennessee's largest health benefit plan company, we've been helping Tennesseans find their own unique paths to good health since 1945. More than that, we're your neighbors and friends – fellow Tennesseans with deep roots of caring tradition, a focused approach to physical, financial and community good health for today, and a bright outlook for an even healthier tomorrow. At BCBST, we empower our employees to thrive both independently and collaboratively, creating a collective impact on the lives of our members. We seek talented individuals who excel in a team environment, share responsibility, and embrace accountability. We're also seeking candidates who are proficient in the Microsoft Office suite, including Microsoft Teams, organized, and capable of managing multiple assignments or projects simultaneously. Additional, strong interpersonal abilities along with strong oral and written communication skills are important across all roles at BCBST. We foster a culture where innovation is encouraged. That includes using AI enabled tools responsibly to support everyday work — guided by proven workflows, templates, and policies. As roles become more advanced, we expect employees to leverage AI more broadly to transform how we serve members. BCBST is a remote-first organization with many employees working primarily from their homes. Each position within the company is classified as either fully remote, partially remote, or office based. BCBST hires employees for remote positions from across the U.S. with the exception of the following states: California, Massachusetts, New Hampshire, New Jersey, and New York. Applicants living in these states may move to an approved state prior to starting a position with BCBST at their own expense. If the position requires the individual to reside in Chattanooga, TN, they may be eligible for relocation assistance.

Apply tot his job

Apply To this Job