Healthcare Specialist – Rubric Design - Remote

Location: Remote – United States
Type: Contract
Compensation: Up to $192,000/year equivalent
Original Rate: Up to $100/hour
Interview: Required
Language: English
Status: Hot Opportunity

Annualized compensation is an illustrative equivalent based on $100/hour × 40 hours/week × 48 weeks/year. Actual earnings depend on approved work, hours, and project availability.

About the Role

We are seeking experienced Healthcare Subject Matter Experts (SMEs) to help evaluate and improve advanced AI systems through expert-level question development, clinical content evaluation, and detailed rubric design.

In this remote contractor role, you will create challenging healthcare questions that require genuine professional expertise, evaluate AI-generated clinical content, and develop rigorous scoring rubrics that define what an accurate, complete, safe, and professionally appropriate response should contain.

You will help identify where advanced AI models produce incomplete, inaccurate, poorly reasoned, overconfident, or clinically inadequate responses. Your expertise will be used to establish the standards AI systems must meet to deliver more reliable and clinically appropriate outputs.

Key Responsibilities

• Create expert healthcare questions: Develop challenging, specialized questions designed to test advanced AI systems and expose limitations in clinical reasoning.

• Design detailed evaluation rubrics: Define the facts, reasoning steps, clinical considerations, caveats, and standards required for a high-quality response.

• Identify critical criteria: Separate essential clinical reasoning and safety considerations from supplementary information.

• Evaluate AI-generated responses: Assess outputs for accuracy, completeness, reasoning quality, safety, professional appropriateness, and clinical relevance.

• Identify subtle failures: Detect unsupported assumptions, missing considerations, inappropriate certainty, clinically significant omissions, and reasoning gaps.

• Critique peer-created rubrics: Review rubrics developed by other healthcare professionals and identify ambiguity, redundancy, incorrect expectations, missing criteria, or weak scoring structures.

• Develop reference answers: Create expert examples, explanations, critiques, and gold-standard response content.

• Support AI training workflows: Produce questions, rubrics, evaluations, critiques, and reference-answer pairs supporting RL and SFT workflows.

• Maintain clinical quality: Ensure evaluation content uses appropriate healthcare terminology, professional reasoning, and evidence-aware judgment.

Required Qualifications

• Professional degree, license, credential, or recognized qualification in medicine, dentistry, pharmacy, nursing, or another healthcare discipline.

• 3+ years of professional or clinical experience.

• Demonstrated expertise in a clearly defined healthcare specialty or subspecialty.

• Strong professional proficiency in English (C1 minimum).

• Ability to develop challenging clinical or professional questions requiring genuine expertise and nuanced judgment.

• Strong written communication and the ability to develop detailed evaluation criteria.

• Ability to distinguish minor omissions from errors that could materially affect clinical quality or safety.

• Ability to evaluate AI-generated content for accuracy, completeness, reasoning quality, safety, calibration, and clinical relevance.

• Ability to independently produce consistent, high-quality written work.

Preferred Qualifications

• Senior-level or specialist healthcare experience.

• Experience with medical education, examination writing, competency assessment, clinical guidelines, teaching, peer review, or quality assurance.

• Experience with AI evaluation, RLHF, SFT, data annotation, model benchmarking, or rubric-based scoring.

• Strong understanding of clinical reasoning and evidence-based professional judgment.

Areas of Expertise

Relevant backgrounds may include:

• Medicine

• Dentistry

• Pharmacy

• Clinical Pharmacology

• Nursing

• Surgery

• Radiology

• Pathology

• Emergency Medicine

• Clinical Reasoning

• Medical Education

• Healthcare Quality & Assurance

• AI Evaluation

• Rubric Design

• Expert Review

What You'll Work On

Your work may involve evaluating AI responses across complex healthcare scenarios and determining whether an answer demonstrates the level of accuracy, completeness, reasoning, and professional judgment expected from a qualified healthcare expert.

You will help define what separates a superficial response from an expert-level healthcare answer by identifying required concepts, appropriate reasoning pathways, important caveats, acceptable alternatives, and critical errors.

Why Join This Opportunity?

Shape the Future of AI

Apply your healthcare expertise to help improve how advanced AI systems reason about and communicate healthcare information.

Work Remotely

Work remotely with flexibility around your schedule and professional commitments.

Use Your Professional Expertise

Your clinical or healthcare knowledge directly contributes to the development and evaluation of next-generation AI systems.

Weekly Payments

Receive payment for approved work on a weekly basis.

Community Rewards

Referral opportunities may provide additional bonus income.

Position Details

Role: Healthcare Subject Matter Expert – Rubric Design
Work Arrangement: Remote
Location: United States
Employment Type: Contract
Compensation: Up to $100/hour
Annualized Equivalent: Up to $192,000/year
Interview: Required
Language: English
Experience: 3+ years professional or clinical experience

If you are a healthcare professional with strong subject-matter expertise, analytical reasoning, and the ability to clearly define what constitutes an excellent clinical answer, this is an opportunity to apply your expertise to the evaluation and improvement of advanced AI systems.