Medical AI Evaluator (Remote, Hourly Contractor)

Position Summary

In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work directly improves how leading AI models handle healthcare information, making them more accurate, reliable, and safe. Tasks may include:

• Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications

• Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data

• Writing clear, precise feedback explaining corrections and reasoning gaps

• Developing prompts and test cases that probe AI accuracy across clinical scenarios

• Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness

• Fact-checking medical content against reliable sources using consistent reasoning

Profile Requirements:

• Bachelor's degree or higher in Medicine (MD/DO), Nursing, Public Health, Health Sciences, or Allied Health

• 5+ years of professional experience in a relevant healthcare discipline

• Strong clinical reasoning skills: differential diagnosis, risk stratification, red-flag recognition

• Solid grounding in disease processes, patient care, public health principles, and medical terminology

• Full professional English proficiency

• Exceptional attention to detail and ability to explain corrections clearly in writing

• Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow

Preferred Experience:

• Experience in clinical documentation review, utilization review, or healthcare editorial QA

• Prior experience with AI data training or annotation

About CNTXT AI

CNTXT AI builds artificial intelligence products and data solutions with a focus on making AI accurate, safe, and globally relevant for impact. Our work spans data services, custom AI solutions, and proprietary AI products, with deep expertise in Arabic-native and secure, sovereign solutions.