Machine Learning Evaluation Specialist (Remote)
Role Overview
This senior-level research role involves designing and evaluating complex machine learning problems that challenge state-of-the-art AI, requiring deep domain expertise. Day-to-day tasks include proposing original ML problems, creating specialized evaluation tasks, and assessing AI-generated solutions for correctness and rigor. The hire will work independently on intellectually demanding projects, impacting the frontier of AI research by identifying and documenting failure modes in advanced domains.
Perks & Benefits
The role offers fully remote work from anywhere with flexible hours of 10–40 per week, ideal for independent contractors. Compensation is high at $200–$400 per hour, depending on domain and seniority, and includes paid assessments if approved. It provides a project-based, freelance opportunity with no guaranteed hours, suitable for self-motivated professionals seeking intellectually challenging work in a research-focused environment.
Full Job Description
Machine Learning Evaluation Specialist (Remote)
List of accepted countries and locations
Important for US applicants: This is a 1099 independent contractor role and is not compatible with F-1 OPT, STEM OPT, or other visa statuses that require W-2 employment, guaranteed hours, or employer sponsorship. We are unable to provide offer letters or employment verification for this role.
Help design the hardest ML problems state-of-the-art AI hasn't solved yet.
We're hiring domain experts to build evaluation tasks that challenge the frontier of AI. This is not an ML engineering role — it's a research role. You'll use deep expertise in your field to create problems that general ML knowledge can't touch.
What you'll do
Propose and frame original, research-grade ML problems rooted in your domain
Design evaluation tasks that require specialized knowledge well beyond standard pipelines
Assess AI-generated solutions for correctness, creativity, and methodological rigor — and explain exactly where and why they fall short
Document problem difficulty, required domain knowledge, and expected failure modes
What you need
Graduate-level expertise (MS or PhD preferred) in a scientific or technical domain that intersects with ML
Strong working knowledge of ML methods — model selection, feature engineering, evaluation metrics
Deep familiarity with active research problems in your field — you know where general ML knowledge runs out
Excellent written communication — you can articulate complex problems clearly and precisely. This cannot be overstated.
Self-motivated and comfortable working independently on intellectually demanding tasks
What you don't need
No prior AI training or RLHF experience required
No software engineering background needed — domain expertise and research instincts are what matter
Domains we're especially looking for
Computational Biology / Bioinformatics
Genomics / Molecular Biology
Physics / Astrophysics / Signal Processing
Climate / Environmental Modeling
Healthcare / Medical Imaging
Neuroscience / Brain-Computer Interfaces
Materials Science / Chemistry
Finance / Quantitative Modeling
Robotics / Control Systems / Reinforcement Learning
Advanced NLP (specialized domains)
Mathematics / Statistics (applied)
Logistics
Fully remote — work from anywhere
$200–$400/hr depending on domain and seniority
10–40 hrs/week, hourly contract
Assessment required — paid if approved
Independent contractor (1099) — not compatible with F-1 OPT, STEM OPT, or visa statuses requiring W-2 employment or employer sponsorship
⚠️ This is a project-based, freelance opportunity with no guaranteed hours. We recommend keeping other work options open while waiting for project assignment.
Similar jobs
Found 6 similar jobs