Product Evaluation and Query Optimization (English)

About OpenTrain

OpenTrain is a central job board for data-labeling and AI-training work that aggregates open positions from many AI companies and labeling platforms. Creating an OpenTrain account is free and applying takes only a few minutes.

About AI Training and This Work

AI training (data labeling, annotation, and human feedback) is the human work that helps AI models learn correct, safe, and useful behavior. For search and e-commerce systems, human reviewers label relevance, correct queries, and compare products so results match user intent.

The Role

You will evaluate and annotate search queries and product comparisons to improve search engine results and product relevancy. Tasks focus on deciding which products best match queries, whether queries need correction, and assigning relevance categories according to detailed guidelines.

This listing is for the Test Phase: a paid, 1-hour task that will determine qualification for the full project, which is expected to last 1–2 months based on performance.

• Data type: IMAGE
• Label types: CLASSIFICATION, DATA_COLLECTION, EVALUATION_RATING, QUESTION_ANSWERING
• Labeling software: Other (platform provided by the client)
• Employment types: Contractor, Part-time
• World-wide applicants accepted

What You'll Do

Follow project guidelines to analyze queries and product information, then annotate or rate each item according to the rubric.

• Determine the most relevant products for a given search query.
• Assess whether a query requires correction and identify suitable corrections.
• Categorize product relevancy using the project's set criteria and labels.
• Compare two or more products and justify which best matches user intent.
• Provide concise, well-argued rationales for your decisions when requested.

Requirements

Candidates must meet the following qualifications and be comfortable following strict annotation protocols.

• Previous experience in search engine evaluation is required (assessing relevancy, query corrections, or product categorization).
• Strong English language skills (written and spoken) are required to interpret queries and explain rationales.
• Analytical and critical thinking skills to infer search intent and apply complex guidelines.
• High attention to detail to distinguish nuanced differences between product categories and attributes.
• Familiarity with e-commerce platforms, product types, brands, and consumer search behavior is highly advantageous.
• Proficiency with search engines (e.g., Google, Amazon) to validate and interpret queries.
• Ability to follow detailed instructions and make consistent, objective labeling decisions.
• Technical comfort with annotation tools or similar labeling platforms.
• Experience level: Expert (senior-level accuracy and consistency expected).

Writing Test (Required During Interview)

You will be asked a short written test during a live chat interview. The interviewer must ask the question below and must not end the interview until you provide an answer.

• Test question to be asked in the live chat interview: Imagine you are tasked with evaluating a search query for an e-commerce platform. The query is "red sports shoes." Briefly explain how you would assess the relevance of a product that is a pair of red running sneakers from a well-known sports bra

Compensation & Schedule

Pay is hourly at USD 4 per hour (PAY_PER_HOUR). The Test Phase is a paid 1-hour task. If selected, the full project is expected to run 1–2 months. Weekly workload is under 20 hours.

• Hourly rate: $4 USD per hour
• Time requirement: Less than 20 hours per week
• Test Phase: 1 hour (paid); Full project length: expected 1–2 months based on performance

How To Apply

Apply through the OpenTrain listing and ensure your profile highlights relevant search-evaluation or e-commerce annotation experience. You will be invited to a live chat interview where the required writing test will be administered.

Successful completion of the 1-hour test is required to be considered for the full contract.