Math Reasoning Evaluator - BS/MS/PhD in Mathematics (Top-100 Univ. preferred)

About OpenTrain

OpenTrain is a central job board for AI-training and data-labeling work. We aggregate roles from many AI companies and labeling platforms so contributors can find opportunities in one place.

Creating an OpenTrain account is free and applying takes only a few minutes.

• Aggregator of data-labeling and AI-training jobs from many sources
• Free account and fast application process

About AI training work

AI training (data labeling / RLHF / model evaluation) is the human work that helps models learn to reason, write, and judge correctness. Contributors annotate, review, and rate model outputs so systems become more accurate and reliable.

This role focuses on mathematical reasoning — evaluating proofs, derivations, calculations, and explanatory clarity to directly shape how math-capable AI behaves.

• Human reviewers provide examples, corrections, and ratings that guide model improvements
• Work is typically remote and flexible, and contributes to model safety and accuracy

The Role

We are hiring mathematically specialized evaluators to judge AI-generated math responses. This is a part-time contractor role paid per hour at USD 80/hour.

You must hold (or be pursuing) a BS, MS, or PhD in Mathematics, Mathematical Statistics, or Applied Math from a top-100 university. The role requires rigorous mathematical reasoning and advanced written explanations in C1+ English.

• Position type: Contractor, Part-time
• Pay: $80 per hour (PAY_PER_HOUR, USD)
• Weekly commitment: minimum 17–20 hrs/week (fits the project’s Less than 20 hours/week expectation)
• Preferred cadence: ~8 hrs/day during active sprints

What you'll do

You will review and rate AI-generated math solutions using detailed rubrics, identify errors, and produce exemplar solutions that model correct reasoning and presentation.

• Judge correctness, reasoning depth, and clarity of AI responses
• Identify subtle conceptual, methodological, and computational errors in proofs, derivations, and calculations
• Fact-check quantitative claims and cite reputable public sources when needed
• Author step-by-step exemplar solutions that model correct methods and clear exposition
• Rate and compare multiple responses using consistent, rubric-driven criteria

Requirements

All of the following qualifications are required or explicitly stated in the role description; we will verify as part of onboarding and qualification exams.

• BS, MS, or PhD (or in-progress) in Mathematics, Mathematical Statistics, or Applied Math from a top-100 university (no general STEM degrees)
• Mastery across core areas such as algebra, calculus, probability, and statistics; comfort with proofs and formal notation
• Exceptional mathematical writing: lucid, rigorous, step-wise explanations and error analysis in C1+ English
• Strong quantitative fact-checking using reputable public sources and precise citation of references when needed
• Consistent application of grading/rating rubrics and high attention to detail
• Availability for a minimum of 17–20 hours per week; preferred cadence ~8 hours/day during active sprints

Preferred and bonus qualifications

These are not required but make applications stronger and may affect assignment to specialized projects.

• Research experience, analytical writing, or competitive debate experience
• Programming literacy (e.g., Python) and facility with typesetting mathematics (e.g., LaTeX)
• Prior data labeling, RLHF, or AI model evaluation experience

How it works

This is a remote, worldwide contractor role managed through OpenTrain’s platform. Onboarding includes paid qualification steps to verify your skills and fit for the work.

You will complete a paid 1–2 hour qualification exam and a paid 1–2 hour project exam as part of onboarding. Assignments require strict adherence to provided rubrics and may be delivered in focused sprints.

• Employment type: Contractor, Part-time, remote (worldwide)
• Onboarding: paid 1–2 hour qualification exam and paid 1–2 hour project exam
• Compensation: hourly pay at $80/hr, paid per hour per project bookkeeping