Position: AI Evaluation Specialist
Type: Hourly contract
Compensation: $47–$90/hour
Location: Remote
Commitment: 10–40 hours/week
Role Responsibilities:
- Annotate and evaluate AI-generated text against safety criteria such as bias, misinformation, unsafe reasoning, and disallowed content.
- Apply harm taxonomies and evolving safety guidelines consistently, including in ambiguous scenarios.
- Document clear reasoning to improve evaluation frameworks and internal guidelines.
- Identify subtle unsafe behaviors, biases, or inconsistencies that automated systems may miss.
- Contribute high-quality human data that supports AI safety research, model improvements, and risk audits.
- Collaborate asynchronously with research and safety teams in a fast-moving environment.
Requirements:
- Experience in model evaluation, structured annotation, applied research, or related analytical roles.
- Strong ability to detect bias, edge cases, and nuanced safety risks in AI outputs.
- Clear written communication and the ability to explain and defend evaluation decisions.
- Comfort working in experimental environments where methods evolve rapidly.
- High attention to detail and consistency across large volumes of text-based tasks.
- Ability to work independently in a remote, project-based setup.
Application Process:
- Upload resume
- Interview (15 min)
- Submit form