Physics Reasoning Evaluator — BS/MS/PhD in Physics (Top-100 Univ. preferred)

Overview

We’re hiring physics specialists to evaluate AI-generated physics responses and author exemplar solutions. This is a contract, part-time role that focuses on advanced physics reasoning, correctness, and clear scientific explanation.

Work involves reviewing video-format AI outputs and assigning evaluation ratings using detailed rubrics; the labeling software is a custom/other platform provided by the project.

• Rate and compare AI responses using detailed rubrics (label type: EVALUATION_RATING).
• Data type: VIDEO; labeling software: OTHER.
• Employment: Contractor, Part-time; worldwide applicants accepted.

About OpenTrain

OpenTrain is a central job board for data-labeling and AI-training work. We aggregate projects from many AI companies and labeling platforms so you can find relevant roles in one place.

Creating an OpenTrain account is free and applying takes only a few minutes; OpenTrain lists project details, pay, and onboarding steps so you know what to expect before you apply.

• OpenTrain connects workers with short- and long-term AI evaluation projects.
• You’ll apply through the OpenTrain platform and complete paid qualification steps where required.

About AI Training Work (Physics Focus)

AI training is the human work behind model accuracy and safety. For physics tasks, evaluators check derivations, units, assumptions, and the depth of reasoning so models produce reliable scientific answers.

This role directly shapes how physics-focused AI systems reason and explain solutions; your judgments and exemplar solutions train models to be correct, clear, and scientifically rigorous.

• Human reviewers correct conceptual, methodological, and computational errors.
• Your writing and rubrics help models learn step-by-step scientific reasoning.

The Role — What You’ll Do

Evaluate AI-generated physics responses for correctness, reasoning depth, clarity, and presentation. Identify subtle errors in derivations, assumptions, dimensional analysis, units, and numerical computation.

Author model-quality exemplar solutions and explanations, fact-check physics claims using reputable public sources, and rate or compare multiple responses according to detailed rubrics.

• Judge correctness and logical flow of physics solutions and derivations.
• Spot unit, sign, approximation, and assumption errors; verify numerical work.
• Write clear, step-by-step exemplar solutions using correct notation (LaTeX proficiency preferred).
• Apply evaluation rubrics consistently and provide precise feedback and references when needed.

Requirements

This role is for physics specialists — not generalists. All substantive qualifications below are required or explicitly stated as preferred/bonus in the project brief.

• Must-have: BS, MS, or PhD (or in-progress) in Physics or a closely related physics field from a top-100 university.
• Mastery across core physics areas (classical mechanics, E&M, thermodynamics, quantum, statistical physics); familiarity with relativity is a plus.
• Strong quantitative reasoning: dimensional analysis, unit consistency, uncertainty/approximation awareness.
• C1+ level English writing ability; excellent scientific writing and step-wise explanations.
• LaTeX proficiency preferred for composing clear notation-rich solutions.
• Ability to follow detailed evaluation rubrics; high attention to detail and consistency.
• Effective fact-checking with precise referencing to reputable public sources.

Availability, Preferred Experience, and Onboarding

Time commitment: less than 20 hours/week overall, with a minimum availability of 17–20 hrs/week expected; during active sprints the preferred cadence is ~8 hrs/day.

Preferred background: research experience, analytical writing or debate, and programming literacy (e.g., Python/Matlab). Bonus: prior data-labeling, RLHF, or AI model evaluation experience.

• Onboarding includes a paid 1–2 hour qualification exam and a paid 1–2 hour project exam.
• Role type: contractor, part-time; worldwide applicants welcome.

Compensation, How to Apply, and Next Steps

Pay: USD $80 per hour (hourly contractor rate). Apply through OpenTrain to start the onboarding process and access the paid qualification steps.

After you submit an application via OpenTrain you will be invited to the paid qualification exam. Successful completion leads to the paid project exam and assignment to active sprints when available.

• Compensation: $80/hr, paid per hour as a contractor.
• Application: create a free OpenTrain account and apply; onboarding steps are paid and required.
• Work uses custom/other labeling tools and focuses on video-format AI responses (label type: EVALUATION_RATING).