Magi AI Content Quality Evaluator – En-US (Freelance, Temporary)
Job Description:
We are hiring 20 Freelance Evaluators to support a high-priority Magi AI Evaluation Pilot Project for Uber’s AI team. This role involves reviewing and rating AI-generated responses for quality, clarity, accuracy, and factual correctness across varying levels of complexity.
All candidates must meet Expertise Level 3 (L3) requirements—the highest evaluator standard—to qualify for this project. Your evaluations will directly shape how AI systems learn, improve, and interact with users.
What is Expertise Level 3 (L3)?
This role requires Expertise Level 3, meaning the evaluator must have:
Advanced analytical and research skills
The ability to handle highly complex content, such as technical or domain-specific materials (e.g., science, legal, medical, or data-heavy text)
Experience evaluating multi-modal data (charts, PDFs, screenshots, etc.)
Capability to conduct line-by-line fact-checking and justify responses with critical reasoning
Comfort working with structured rubrics and independently verifying factual accuracy
Compensation:
Pay Rate: $25 – $31 per hour (USD)
The assessment is paid if the evaluator passes and completes at least 4 hours of work on the project
Key Responsibilities:
Evaluate AI-generated responses across varying task types, using structured guidelines
Identify tone, style, factual, and product-specific issues in output
Perform detailed comparisons, fact-checks, and accuracy reviews
Submit ratings using tools such as dropdowns, screencasts, and feedback forms
Meet tight deadlines (all assigned work must be completed within 4 business days)
Task Complexity & Time Commitment:
Tasks will range from simple to complex, but all evaluators must qualify at the L3 expertise level
Average Handling Time: 75–145 minutes per task
Minimum expectation: 3+ hours per task cycle, with option to take on more work
Work is asynchronous, though task assignment may align with IST time zone
Requirements:
Professional and expert-level English (US) speaker (can reside in or outside of the U.S.)
5+ years of experience in linguistics, research, writing, content evaluation, or technical review
Master’s degree or PhD strongly preferred
High attention to detail, accuracy, and critical thinking
Secure internet connection and workspace
Must complete and pass a qualifying assessment at the Expertise Level 3 standard
Assessment Details:
Approx. 30 minutes to complete
Paid if passed and evaluator completes at least 4 hours of project work
Project Details:
Project Name: Magi AI Evaluation (Uber Special Project)
Work Type: Freelance, Temporary
Start Date: Within 48 hours of onboarding completion (target: April 28, 2025)
Schedule: Flexible, asynchronous
Location: Global (must be a native En-US speaker)
Duration: Initial pilot cycle, with possible future work based on performance and need
Job Types: Part-time, Temporary
Pay: $25.00 - $31.00 per hour
Apply Now
We are hiring 20 Freelance Evaluators to support a high-priority Magi AI Evaluation Pilot Project for Uber’s AI team. This role involves reviewing and rating AI-generated responses for quality, clarity, accuracy, and factual correctness across varying levels of complexity.
All candidates must meet Expertise Level 3 (L3) requirements—the highest evaluator standard—to qualify for this project. Your evaluations will directly shape how AI systems learn, improve, and interact with users.
What is Expertise Level 3 (L3)?
This role requires Expertise Level 3, meaning the evaluator must have:
Advanced analytical and research skills
The ability to handle highly complex content, such as technical or domain-specific materials (e.g., science, legal, medical, or data-heavy text)
Experience evaluating multi-modal data (charts, PDFs, screenshots, etc.)
Capability to conduct line-by-line fact-checking and justify responses with critical reasoning
Comfort working with structured rubrics and independently verifying factual accuracy
Compensation:
Pay Rate: $25 – $31 per hour (USD)
The assessment is paid if the evaluator passes and completes at least 4 hours of work on the project
Key Responsibilities:
Evaluate AI-generated responses across varying task types, using structured guidelines
Identify tone, style, factual, and product-specific issues in output
Perform detailed comparisons, fact-checks, and accuracy reviews
Submit ratings using tools such as dropdowns, screencasts, and feedback forms
Meet tight deadlines (all assigned work must be completed within 4 business days)
Task Complexity & Time Commitment:
Tasks will range from simple to complex, but all evaluators must qualify at the L3 expertise level
Average Handling Time: 75–145 minutes per task
Minimum expectation: 3+ hours per task cycle, with option to take on more work
Work is asynchronous, though task assignment may align with IST time zone
Requirements:
Professional and expert-level English (US) speaker (can reside in or outside of the U.S.)
5+ years of experience in linguistics, research, writing, content evaluation, or technical review
Master’s degree or PhD strongly preferred
High attention to detail, accuracy, and critical thinking
Secure internet connection and workspace
Must complete and pass a qualifying assessment at the Expertise Level 3 standard
Assessment Details:
Approx. 30 minutes to complete
Paid if passed and evaluator completes at least 4 hours of project work
Project Details:
Project Name: Magi AI Evaluation (Uber Special Project)
Work Type: Freelance, Temporary
Start Date: Within 48 hours of onboarding completion (target: April 28, 2025)
Schedule: Flexible, asynchronous
Location: Global (must be a native En-US speaker)
Duration: Initial pilot cycle, with possible future work based on performance and need
Job Types: Part-time, Temporary
Pay: $25.00 - $31.00 per hour
Apply Now