Economist – AI Model Evaluation

Candidate will be familiar with RAG and agentic systems, including end-to-end query-to-response workflowsExperience in fine-tuning language models, tracing, analysis, and debugging. · ...

We are seeking Model Evaluators with experience in fine-tuning language models, tracing analysis and debugging. Strong proficiency in Python and Jupyter Notebooks is required. · familiar with RAG and agentic systems including end-to-end query-to-response workflows · experience i ...

We are seeking Model Evaluators with experience in fine-tuning language models, tracing, analysis, and debugging. Strong proficiency in Python and Jupyter Notebooks is required. · ...

Evaluate LLM outputs for accuracy relevance bias and safety Design test cases and evaluation benchmarks for AI models Analyze model behavior and document findings Collaborate with ML engineers and data scientists to improve models Provide structured feedback to enhance model perf ...

· ...

We want AI to be safe and beneficial for our users and for society as a whole.Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. · Design novel evaluation methodologies to ...

About Anthropic's mission is to create reliable, interpretable and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. · ...

The Payment Model Evaluation role supports the Payment Model Evaluation process, which assesses merchant relationships in alignment with Global Merchant Acquisition policies within a rapidly evolving Fintech and payments environment. · Supporting the interpretation and applicatio ...

We are seeking an LLM-GenAI Model Evaluator position for our team in Austin tx & Sunnyvale. The ideal candidate will have strong understanding of LLMs, generative AI, and transformer-based architectures. · Strong understanding of LLMs, generative AI, and transformer-based archite ...

We are looking for someone with natural English writing ability who can evaluate AI model responses and grade responses on a Likert scale. · ...

As Model Evaluation QA Lead you'll ensure model quality assurance across Deepgram's AI pipeline. · Design automated evaluation frameworks · Integrate model quality gates into release pipelines · ...

Immediate need for a talented LLM-GenAI Model Evaluator. · Evaluate LLM-GenAI models. · ...

Design and implement evaluation pipelines for large generative AI models. · ...

Job summary · The Payment Model Evaluation role is an individual contributor position that supports the Payment Model Evaluation (PME) process assessing merchant relationships in alignment with Global Merchant Acquisition policies within a rapidly evolving Fintech and payments en ...

· Key Responsibilities: · Perform scoring and qualitative evaluations of · LLM-generated responses across multiple use cases. · Develop and maintain scoring guidelines and rubrics to · ensure consistency and objectivity. · Collaborate with data scientists, product managers, and ...

...

Company Overview · Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More than 200,000 developers and 1,300+ or ...

We are looking for an LLM-GenAI Model Evaluator position. · ...

We are a forward thinking company looking for a Remote Financial AnalystEvaluate the output of AI systems for accuracy and performanceTrain and enhance AI models used for financial analysisCollaborate with team members on project selection and goalsEvaluate the output of AI syste ...

The freelancer will work on finding if user responses are on or off topic and rehearsed or not. The scripts are provided but need optimization for better accuracy. · ...