Practicing Clinician for AI Model Evaluation

Role Overview

Help a leading AI research team improve how advanced AI models reason about real clinical work. In this hybrid, full-time role, you will apply practicing-clinician judgment to define high-quality clinical tasks, model answers, and evaluation standards alongside research and program management teams.

Key Responsibilities

• Review clinical knowledge tasks and model outputs for missing behaviors, weak reasoning, unsafe recommendations, departures from clinical guidelines, and plausible-sounding answers that do not meet clinical standards.
• Write detailed instruction specifications and gold-standard solutions for clinical problems.
• Create clinical tasks that reflect real-world medical practice.
• Design challenging benchmarks and evaluation sets to measure model improvement.
• Partner with researchers to develop medicine-specific capabilities and tools.
• Calibrate standards with researchers and adjacent-domain specialists, turning implicit clinical judgment into clear, teachable criteria.

Qualifications

• MD or DO from an accredited medical school, with completed residency training in a recognized specialty.
• At least 4 years of post-residency clinical practice. Residency and fellowship training do not count toward this requirement.
• Active, unrestricted medical license in at least one U.S. state and board certification in your specialty.
• Established expertise in a clinical specialty, such as internal medicine, oncology, radiology, emergency medicine, surgery, psychiatry, or a medical subspecialty.
• Senior clinical progression, such as Attending Physician, Medical Director, Division Chief, Associate or full Professor, or Chief Medical Officer, with meaningful ownership of clinical decisions.
• Hands-on professional use of large language models and the ability to distinguish sound clinical reasoning from convincing but incorrect answers.
• Excellent written communication and the ability to provide precise, structured feedback.
• Experience in utilization management, clinical informatics, or medical affairs is a plus.

Work Terms

• Full-time W-2 hourly employment, with an initial 6-month commitment and reliable availability for 40 hours per week.
• Hybrid role based in the Bay Area, California. You must live in the Bay Area and be available to work on-site with the client team multiple days per week when required.
• This is not a remote position. Candidates outside the Bay Area must relocate at their own expense before the engagement begins; relocation assistance is not provided.
• You will work within the client’s tools alongside internal research teams and receive client-issued accounts and equipment.
• The position is a structured, role-based placement within an enterprise team, not a freelance engagement.

Compensation

$70 to $110 per hour.

Application Process

Opportunities may be discovered through an online job platform. Employment, onboarding, payroll, benefits, and compliance are managed by the employer of record for the engagement.

Equal Opportunity

Equal employment opportunity is provided without discrimination based on any legally protected characteristic. Reasonable accommodations are available for qualified individuals with disabilities and disabled veterans throughout the application process.