- Create structured test cases that simulate complex human workflows
- Define gold-standard behavior and scoring logic to evaluate agent actions
- Analyze agent logs, failure modes, and decision paths
- Work with code repositories and test frameworks to validate your scenarios
- Iterate on prompts, instructions, and test cases to improve clarity and difficulty
- Ensure that scenarios are production-ready, easy to run, and reusable
- 3+ of software development experience with strong Python focus
- Experience with Git and code repositories
- Comfortable with structured formats like JSON/YAML for scenario description
- Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design
- Familiarity with Docker
- English proficiency - B2
- Paid contributions, with rates up to $80/hour*
- Fixed project rate or individual rates, depending on the project
- Some projects include incentive payments
-
Engineering Technician
1 week ago
Only for registered members PacificOperates the electro-mechanical test equipment for qualifying new engine components and new sources for engine components. Evaluates engine components on vehicles, procedures and specifications for sensor testing, modification of existing applications where appropriate, · Operate ...
-
Engineering Technician
1 month ago
Only for registered members PacificThe Engineering Technician operates the electro-mechanical test equipment for qualifying new engine components and new sources for engine components. · Other responsibilities include the evaluation of engine components on vehicles, procedures and specifications for sensor testing ...
-
Engineering Program Administrator
1 week ago
Only for registered members United States Full time $98,612 - $123,260 (USD)The Engineering Program Administrator provides overall management for the planning, budgeting, scheduling, design, and construction of public works projects. Be part of a team that's committed to service in Dallas. ...
-
deputy chief engineer
3 days ago
Only for registered members United States Full time $139,713 - $158,244 (USD)This position oversees the administration and operation of a major subdivisional transportation engineering program within the Department of Transportation: Design or Construction Management Sections. · To oversee the administration and operation of a major sub-divisional enginee ...
-
deputy chief engineer
17 hours ago
Only for registered members United States Full time $139,713 - $158,244 (USD)+To oversee the administration and operation of a major subdivisional transportation engineering program within the Department of Transportation: Design, or Construction Management Sections; to perform highly difficult and responsible administrative work of a professional enginee ...
-
Principal Research Engineer
4 days ago
Only for registered members United States Full timeJoin Cleveland Clinic's Main Campus where research and surgery are advanced, · technology is leading-edge, · patient care is world class and caregivers are family. · ...
-
Senior Machine Learning Engineer II
2 weeks ago
Only for registered members United States Full time $148,000 - $173,200 (USD)As a Machine Learning Engineer, you'll build the intelligence behind the next generation of agentic AI systems and related AI systems that reason over massive, heterogeneous log data. · ...
-
Senior Engineer, Dallas Water Utilities
4 days ago
Only for registered members United States Full time $89,440 - $111,800 (USD)Serves as a project manager for the design and construction of City projects including all aspects of planning, design, bid preparation cost estimation and monitoring construction compliance May manage a function as licensed engineer or supervise staff responsible infrastructure ...
-
Lead Human Factors Engineer
1 month ago
Only for registered members United StatesThe Lead Human Factors Engineer provides expert evaluation, design guidance, and research leadership to ensure products, processes, and systems align with human factors engineering (HFE) principles... · ...
-
Senior Engineer – AI/ML Strategy
3 days ago
Only for registered members USA Full timeWe are seeking a Senior AI/ML Engineer to lead the strategic implementation of generative AI and machine learning capabilities across enterprise platforms. · Design and deliver end‑to‑end generative AI solutions using cloud AI services such as Azure AI and Google AI. · Lead devel ...
-
AI Solutions Architect
2 days ago
Only for registered members United States Full timeCodeRoad is seeking a strategic AI Solutions Architect to serve as the technical bridge between our boutique consulting partners and our execution engines. · ...
-
Backend Engineering Specialist
3 days ago
Only for registered members United States of AmericaAre you an experienced Backend Engineer eager to shape the future of AI? · Ready to channel your backend engineering expertise into building the AI tools of tomorrow? · ...
-
Senior Engineer
1 day ago
Only for registered members United StatesWe are seeking a highly experienced Senior Pavement Engineer to lead and deliver innovative pavement design solutions for various industry sectors. The ideal candidate will have a proven track record in technical leadership and project management. · Lead safety efforts for team a ...
-
construction mgmt resident(dot
4 days ago
Only for registered members United States Full time $74,233 - $83,955 (USD)The construction management resident will oversee complex multi-year contract construction projects on assigned roads and/or bridges. · A thorough knowledge of basic engineering principles. · The ability to apply mathematical and statistical concepts. · ...
-
Research Engineer
1 week ago
Only for registered members United States Full time· Designing, building, and maintaining scalable data pipelines, data warehouses, and data lakes that power clinical and financial reporting. Ensuring seamless integration of structured, semi-structured, and unstructured data into Snowflake environments. Familiarity with REDCap d ...
-
Senior Product Manager
1 month ago
Only for registered members United StatesWe are hiring a Senior Product Manager to own Tracer and drive the systems, processes, and execution behind how we build, evaluate, and improve SuperDial's AI workflows. · Own Tracer end-to-end (internal tooling + systems) · Own the roadmap and execution for Tracer as an internal ...
-
Full Stack Engineering Specialist
3 days ago
Only for registered members United States of AmericaAre you an experienced Full Stack Engineer eager to shape the future of AI We re looking for a highly skilled Full Stack Engineering specialist who can bring technical depth coding expertise and architectural insight to training data You ll work with cutting-edge AI tools evaluat ...
-
Inside Sales Engineer
3 days ago
Only for registered members United StatesWe are looking for a knowledgeable Inside Sales Engineer who will work closely with the Sales Engineering and Operations team to support company growth by uncovering and securing project opportunities with new and current customers. · The key to success of an Inside Sales Enginee ...
-
Senior CNC Programmer
3 days ago
Only for registered members United States Full time $110,000 - $150,000 (USD)PPG's Aerospace Business is seeking a Senior CNC Programmer / Engineer to join our engineering team. · You will play a critical role in CNC programming, process planning, tooling, machine tool selection, and continuous improvement for multi-axis CNC machining operations. · ...
-
AVP / Lead AI Engineer
1 month ago
Only for registered members United StatesThe Lead AI Engineer will design and implement Retrieval-Augmented Generation pipelines to ground LLMs in enterprise or domain-specific data. They will make strategic decisions on chunking strategy, embedding models, and retrieval mechanisms to balance context precision, recall, ...
-
Engineer C
3 days ago
Only for registered members United States Full time $97,450 - $125,200 (USD)The Downtown Transportation Engineer will lead a team of engineers and planners in transportation development projects in the City of Austin for reviewing proposed developments' compliance with the city code, ordinances, criteria manuals, established engineering design manuals, a ...
Freelance Agent Evaluation Engineer - United States - Mind Rift
Description
Please submit your CV in English and indicate your level of English proficiency.
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
What this opportunity involves
While each project involves unique tasks, contributors may:
What we look for
This opportunity is a good fit for software engineers, open to part-time, non-permanent projects. Ideally, contributors will have:
How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Project time expectations
Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Payment
*Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
-
Engineering Technician
Only for registered members Pacific
-
Engineering Technician
Only for registered members Pacific
-
Engineering Program Administrator
Full time Only for registered members United States
-
deputy chief engineer
Full time Only for registered members United States
-
deputy chief engineer
Full time Only for registered members United States
-
Principal Research Engineer
Full time Only for registered members United States
-
Senior Machine Learning Engineer II
Full time Only for registered members United States
-
Senior Engineer, Dallas Water Utilities
Full time Only for registered members United States
-
Lead Human Factors Engineer
Only for registered members United States
-
Senior Engineer – AI/ML Strategy
Full time Only for registered members USA
-
AI Solutions Architect
Full time Only for registered members United States
-
Backend Engineering Specialist
Only for registered members United States of America
-
Senior Engineer
Only for registered members United States
-
construction mgmt resident(dot
Full time Only for registered members United States
-
Research Engineer
Full time Only for registered members United States
-
Senior Product Manager
Only for registered members United States
-
Full Stack Engineering Specialist
Only for registered members United States of America
-
Inside Sales Engineer
Only for registered members United States
-
Senior CNC Programmer
Full time Only for registered members United States
-
AVP / Lead AI Engineer
Only for registered members United States
-
Engineer C
Full time Only for registered members United States