Legal & Compliance Specialist – Rubric Design - Remote

Role Overview

• Job Type: Contract

• Work Arrangement: Fully Remote

• Eligible Location: United States

• Compensation: Up to $120 per hour

• Interview: Required

• Working Language: English (C1 or higher)

About the Role

We are seeking an experienced legal or compliance professional to develop challenging evaluation tasks, assess AI-generated legal and regulatory content, and design structured rubrics that define accurate, complete, and professionally appropriate responses.

You will apply your specialist knowledge to legal analysis, regulatory interpretation, compliance, risk assessment, contracts, privacy, employment matters, financial regulation, corporate compliance, or another relevant area of expertise.

The role focuses on translating complex professional judgment into clear evaluation criteria and helping improve the accuracy, reasoning, and reliability of AI-generated legal and compliance outputs.

Key Responsibilities

• Develop realistic, challenging legal and compliance scenarios requiring expert analysis and professional judgment.

• Design detailed rubrics covering relevant issues, applicable standards, reasoning steps, exceptions, jurisdictional considerations, risks, and critical omissions.

• Evaluate AI-generated legal and regulatory responses for accuracy, completeness, relevance, and analytical rigor.

• Assess whether responses identify material legal issues and correctly apply relevant rules or standards.

• Identify unsupported conclusions, faulty assumptions, missing exceptions, and incorrect interpretations.

• Review and critique rubrics developed by other experts for ambiguity, omissions, redundancy, subjectivity, and inaccurate expectations.

• Develop reference analyses, answer frameworks, critiques, and high-quality example responses.

• Produce evaluation questions, scoring rubrics, preference data, and reference-answer pairs for reinforcement learning and supervised fine-tuning workflows.

• Distinguish factual analysis, legal interpretation, professional judgment, and uncertainty in evaluation materials.

Required Qualifications

• JD, LLB, LLM, law degree, relevant compliance qualification, or equivalent professional background.

• At least 3 years of professional experience in law, regulatory compliance, legal research, policy, risk, privacy, financial compliance, contracts, or a related field.

• Demonstrated expertise in a clearly defined legal, regulatory, or compliance specialty.

• English proficiency at C1 level or higher.

• Ability to create challenging professional scenarios requiring issue spotting, interpretation, and analytical judgment.

• Strong rubric-writing skills, including the ability to define required issues, applicable standards, reasoning steps, exceptions, risks, and critical errors.

• Ability to distinguish legitimate alternative interpretations from materially deficient analysis.

• Excellent attention to detail when assessing terminology, jurisdiction, authority, assumptions, exceptions, materiality, and conclusions.

Preferred Qualifications

• Professional legal qualification or bar admission for attorney-focused work.

• Experience in legal education, examination writing, peer review, legal editing, quality assurance, policy drafting, compliance testing, or supervisory review.

• Previous experience with AI evaluation, legal AI, data annotation, reinforcement learning from human feedback (RLHF), model benchmarking, or rubric-based scoring.

Areas of Expertise

Relevant specializations may include:

• Legal analysis and regulatory interpretation

• Corporate law and corporate compliance

• Contracts and commercial law

• Privacy and data protection

• Financial regulation and compliance

• Employment law and workplace compliance

• Risk assessment and policy interpretation

Candidates should have demonstrable expertise in at least one relevant domain.

Compensation and Benefits

• Hourly rate: Up to $120, depending on the project and applicable terms.

• Payment frequency: Weekly, once work is approved.

• Flexible schedule: Manage your working hours remotely.

• Additional rewards: Referral bonuses may be available.

Interview and Work Arrangement

The selection process includes an interview. Successful candidates will work remotely on legal and compliance evaluation tasks, rubric development, and AI training projects.