Manager Digital Product - AI Quality Assurance and Testing

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

This AI Quality Assurance & Testing is the first of its kind and critical role for accelerating the AI agenda across UHC Consumer Operations. This position will report to the Director of AI Quality and Assurance testing.. This leader's mission is to drive 3 things a) AI Program Pre-Production Testing b) AI Program Postproduction Testing/Monitoring c) AI Operational readiness and go to market strategies

This leader's mission is to lead a team of approximately 5-7 individuals) to rigorously test seamless AI enabled consumer experiences, maximize AI agentic delivery and ensure UHC wide ambitious cost savings due to consumer self-service adoption. 

Primary Responsibilities:

• Establish and oversee AI/ML testing environments and frameworks for functionality, performance, fairness, bias detection and robustness
• Implement pre-launch validation and sign off processes for AI models and features
• Build QA playbook for AI products/features covering regression, stress testing, explainability and edge case validation
• Develop post launch QA playbook covering live shadow testing against ground truth datasets, A/B and multivariant testing to drive adoption of AI risk management practices
• Oversee integration of RHLF (reinforcement learning from human in the loop) pipelines - structured human oversight in model evaluation, labeling, QA to ensure that models align with ethical and business standards
• Build RLAIF frameworks (Reinforcement learning AI feedback loop) leveraging AI-assisted evaluation and reinforcement learning for continuous feedback loops at scale
• Define and run operational readiness reviews (ORR) before AI product launches
• Drive release readiness assessments across product, engineering, operations, technology
• Champion responsible AI practices ensuring fairness, transparency and accountability in AI Models
• Hire, mentor and scale a high performing AI delivery and business integration organization
• Define program KPI's and regularly report process to executive leadership
• Drive a culture of blameless postmortems to strength resilience and accountability
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• 10+ years in product and/or operations with 2+ years of AI/ML focused testing leadership roles
• Experience in establishing data integrity and governance frameworks at scale
• Solid understanding of machine learning, generative AI, data platforms and AI ethics
• Proven track record of delivering AI products to market with measurable impact
• Demonstrated intellectual capacity and sound judgment to lead in changing and ambiguous environments
• Proven track record of:
  • Working collaboratively in a highly matrixed organization
  • Building relationships with people from diverse areas of an organization
  • Bringing fresh ideas and new thinking to the team (with views that change opportunistically)
  • Being energized and passionate with the ability to persuasively sell ideas and concepts
  • Operating effectively at executive levels
  • Thinking broadly about overall business issues with both an "inside-out" and "outside-in" perspective
  • Managing, motivating and developing a team and attracting and retaining outstanding talent
  • Empowering team members and holding them accountable
  • Rapidly assessing situations and absorbing & applying new information to decision-making and direction setting
• Demonstrated confidence and perseverance in addressing challenges and overcoming obstacles
• Proven history of addressing difficult issues with candor, speaking the truth, expressing a point of view and taking a stand

Preferred Qualifications: 

• Experience partnering with AI forward companies like Amazon, Google, Microsoft
• Familiarity with global data protection and AI regulatory frameworks
• Proven well-rounded in AI literacy and upskilling programs

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.