Bilingual LLM Evaluator

Evaluate LLM outputs for accuracy relevance bias and safety Design test cases and evaluation benchmarks for AI models Analyze model behavior and document findings Collaborate with ML engineers and data scientists to improve models Provide structured feedback to enhance model perf ...

We're looking for an LLM Evaluation, Benchmarking & Experimentation Engineer to rigorously test our proprietary LLM API and build the infrastructure for systematic model improvement. · ...

· At LeoTech, we are passionate about building software that solves real-world problems in the Public Safety sector. Our software has been used to help the fight against continuing criminal enterprises, drug trafficking organizations, identifying financial fraud, disrupting sex ...

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . · ...

Mercor connects elite creative and technical talent with leading AI research labs. · Evaluate LLM-generated responses on their ability to effectively answer user queries. · ...

+Job summary · Evaluate LLM-generated responses on their ability to effectively answer user queries. · QualificationsBachelor's degree · Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in Hindi · ...

We are passionate about building software that solves real-world problems in the Public Safety sector. · ...

Build the evaluation layer in the ThirdLaw platform—the part of the system that decides whether an LLM prompt, response, tool call, or agent behavior is acceptable. · ...

We're partnering with a deep-tech AI company building autonomous agentic systems for complex physical and real-world environments. The team operates at the edge of what's possible today designing AI systems that plan act recover and improve over long horizons in high-stakes setti ...

Tetrad Digital Integrity (TDI) is a leading-edge cybersecurity firm with a mission to safeguard and protect our customers from increasing threats and vulnerabilities in this digital age. · We are seeking a highly skilled LLM Security Evaluation Expert to join our team. In this ro ...

Our Client is a well-funded nonprofit research organization focused on measuring frontier AI capabilities—especially agentic / autonomous capabilities and the ability of models to conduct AI R&D because those capabilities can create outsized societal and security risk if they sca ...

We are looking for an LLM-GenAI Model Evaluator position. · ...

We are seeking an LLM-GenAI Model Evaluator position for our team in Austin tx & Sunnyvale. The ideal candidate will have strong understanding of LLMs, generative AI, and transformer-based architectures. · Strong understanding of LLMs, generative AI, and transformer-based archite ...

You'd be building the UI that turns messy LLM evaluation outputs into clear, explorable artifacts that researchers can trust. · ...

Immediate need for a talented LLM-GenAI Model Evaluator. · Evaluate LLM-GenAI models. · ...

Mercor connects elite creative and technical talent with leading AI research labs. The company is headquartered in San Francisco and has investors including Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey. As a Bilingual LLM Evaluation Anal ...

SilverEdge Government Solutions is seeking a highly skilled LLM Security Evaluation Expert to join our team. · ...

We're partnering with a deep-tech AI company building autonomous agentic systems for complex physical environments. · Build eval harnesses for agentic LLM systems offline + in-workflow ...

SilverEdge Government Solutions is seeking a highly skilled LLM Security Evaluation Expert. In this role, you will be responsible for rigorously testing the security and integrity of Large Language Models (LLMs). · TS/SCI with Polygraph level Clearance · Familiarity with prominen ...

I'm seeking a technical mentor to help deepen my understanding of LLM evaluation and benchmarking, with particular attention to high-stakes applications · (e.g., mental health), while developing a generalizable framework for reasoning about model performance across domains. · ...