- Develop Evaluation Architectures: Design and implement state-of-the-art evaluation pipelines for conversational agents using LLM-as-a-judge, and hybrid scoring frameworks.
- Prompt Engineering & Calibration: Develop high-precision prompts for evaluator models and rigorously test them against human judgment to ensure high inter-rater reliability.
- Model Distillation & Optimization: Lead the fine-tuning of smaller, cost-effective models to act as scalable "Judge" models, balancing trade-offs between accuracy, latency, and cost.
- Dataset Curation: Work with large-scale conversation logs to curate "Golden Set" datasets and design annotation instructions that standardize ground truth for subjective tasks.
- Cross-Functional Integration: Collaborate with Engineering teams to integrate quality signals into CI/CD pipelines, enabling automated regression testing and production monitoring.
- Failure Mode Analysis: Conduct deep-dive analyses on agent failures (hallucinations, tool misuse, safety violations) and define actionable feedback loops for the modeling team.
- Insight Discovery & Strategic Influence: Leverage evaluation data to discover systemic weaknesses and root causes, actively influencing sub-agent modeling teams and cross-functional partners to prioritize and drive targeted improvements in overall performance.
- Thought Leadership: Mentor senior data scientists, standardize best practices for evaluation across the org, and maintain world-class credentials through patents, publications, or conference presentations.
- Education: Advanced degree (Master's or PhD) in Computer Science, Statistics, Mathematics, Computational Linguistics, or a related field.
- Experience: 7+ years of experience in Data Science or Machine Learning with a focus on NLP, Deep Learning, or AI evaluation.
- Generative AI Expertise: Deep understanding of Large Language Models (LLMs), including prompt engineering, chain-of-thought reasoning, and instruction tuning.
- Technical Proficiency: Solid understanding of Python and expertise with core data science packages (NumPy, Pandas, PyTorch, Scikit-learn).
- Metric Design: Proven experience designing metrics for non-deterministic outputs (e.g., evaluating summarization, relevance, or helpfulness).
- Engineering Fundamentals: Experience building scalable data pipelines and familiarity with distributed training/inference frameworks.
- PhD in Machine Learning, NLP, or a related quantitative field.
- Experience with conversational AI, chatbots, summarization, retrieval-augmented generation, or recommendation evaluation in an e-commerce context.
- Knowledge of model distillation, LoRA, instruction tuning, or parameter-efficient adaptation techniques
- Familiarity with evaluating open-ended outputs where ground truth is subjective or contextual
- Publications, patents, or open-source contributions in LLM evaluation or applied AI
-
Data Scientist Everfit | Hybrid, San Francisco Bay Area · About Everfit · Everfit is a fitness technology company building an AI-powered coaching platform that serves 280,000+ coaches and millions of training clients globally. We're transforming how fitness professionals deliver ...
Hayward $85,000 - $150,000 (USD) per year1 day ago
-
Staff Data Scientist – Post Sales · Location: San Francisco (Hybrid) · Salary: $200–250k base + RSUs · This fast-growing Series E AI SaaS company is redefining how modern engineering teams build and deploy applications. We're expanding our data science organization to accelerate ...
Hayward $1,400,000 - $2,500,000 (USD) per year1 day ago
-
Position Summary... · What you'll do... · About Us: · The Revenue and Marketing Data Science Team at Walmart Global Tech is dedicated to incubating and developing data-driven, ML/AI solutions that help Walmart, suppliers, sellers, creators and influencers connect with followers a ...
Hayward $143,000 - $286,000 (USD) per year Full time15 hours ago
-
Position Summary... · What you'll do... · Cortex Team is Walmart's core A.I. conversational platform, powering the vision of delivering the worlds best personal assistants to Walmart's customers, accessible via natural voice commands, text messages, rich UI interactions, and a mi ...
Hayward $143,000 - $286,000 (USD) per year Full time15 hours ago
-
Position Summary... · What you'll do... · Immigration Sponsorship is not available in this role What you'll do... Cortex Team is Walmart's core A.I. conversational platform, powering the vision of delivering the worlds best personal assistants to Walmart's customers, accessible v ...
Hayward $143,000 - $286,000 (USD) per year Full time15 hours ago
-
Position Summary... · What you'll do... · Join Walmart and your work could help over 275 million global customers live better every week. Yes, we are the Fortune #1 company. But you'll quickly find we're a company who wants you to feel comfortable bringing your whole self to work ...
Hayward $140,000 - $220,000 (USD) per year Full time15 hours ago
-
Dice is the leading career destination for tech experts at every stage of their careers. · ...
Dublin2 weeks ago
-
Data Scientist at Donato Technologies Inc in Dublin, CA. Apply via Dice today. · Dice is the leading career destination for tech experts at every stage of their careers. · The role involves data science skills including Python/MATLAB expertise, familiarity with C++, PhD in comput ...
Dublin $85,000 - $150,000 (USD) per year1 week ago
-
We currently have an exciting opportunity for Data Scientist – Machine Learning, to join our Enterprise Analytics team. In this role, you will work with stakeholders across the organization, including Data Engineering teams. You will leverage your Technical, Analytical, and Busin ...
Dublin $80,700 - $110,000 (USD) Full time1 week ago
-
Data Scientist · Expert on Python/MATLAB, familiar with C++ · Ph.d in computer science stream is must. · 5+ years Algorithm development for commercial medical product · ...
Dublin2 weeks ago
-
Data Scientist position at ProCorp Systems Inc., requiring expertise in Python/MATLAB, C++, computer vision algorithms, image processing, and medical imaging analysis. · ...
Dublin $85,000 - $150,000 (USD) per year1 week ago
-
We are seeking a seasoned Data Scientist to join our team and lead initiatives in predictive modeling, personalization, and AI-driven financial recommendations. · Perform exploratory and statistical data analysis to identify opportunities for model development and business optimi ...
Dublin $160,000 - $185,000 (USD)1 month ago
-
We currently have an exciting opportunity for Data Scientist – Machine Learning to join our Enterprise Analytics team. · ...
Dublin, CA2 weeks ago
-
Data Scientist role in Dublin · Expert on Python/MATLAB, familiar with C++. · Ph.d in computer science stream is must. · 5+ years Algorithm development for commercial medical product · ...
Dublin2 weeks ago
-
Dice is the leading career destination for tech experts at every stage of their careers. Our client, ProCorp Systems Inc., is seeking the following. Apply via Dice today · Position: Data Scientist · Location: Dublin, CA · Top 5 Skills for Interview: · Expert on Python/MATLAB, fam ...
Dublin, CA $85,000 - $150,000 (USD) per year1 week ago
-
Data Scientist to lead initiatives in predictive modeling, personalization, and AI-driven financial recommendations at SavvyMoney. · SavvyMoney is a leading San Francisco East Bay fintech company providing integrated credit score and personal finance solutions to 1 ,500+ bank and ...
Dublin, CA1 month ago
-
The Data Scientist will support the Electric Operations Excellence Team by translating business needs into advanced analytics and machine learning solutions. This role focuses on strong execution and delivery of high-impact models, partnering closely with product owners, technica ...
San Ramon1 month ago
-
PG&E is seeking an experienced Data Scientist to join the Electric Operations Excellence team.This role focuses on translating complex business problems into advanced analytics and machine learning solutions that drive operational efficiency and strategic decision-making. · ...
San Ramon1 month ago
-
We are seeking an experienced Data Scientist who will provide strong execution and delivery of data science.Working as a part of the Electric Operations Excellence Team, this Data Scientist will translate business needs into advanced analytics and machine learning models. · ...
San Ramon1 month ago
-
We are seeking an experienced Data Scientist who will provide strong execution and delivery of data science. Working as a part of the Electric Operations Excellence Team, this Data Scientist will translate business needs into advanced analytics and machine learning models. · Lead ...
San Ramon1 month ago
Principal, Data Scientist - Hayward - Walmart
Description
Position Summary...
What you'll do...
OverviewWalmart's Next Gen Commerce team is building the future of conversational shopping with intelligent agents that reason, recommend, and proactively assist customers. As a Principal Data Scientist for Quality & LLM Judging Systems, you will serve as the technical lead for defining and measuring the success of these AI systems. You will be responsible for designing the "brain" that critiques our agents, utilizing a mix of LLM-as-a-judge frameworks, human benchmarks, and automated pipelines. In this high-impact, hands-on role, you will partner closely with engineering and product leaders to translate subjective quality goals into rigorous, actionable metrics that drive model improvement and safe deployment. Responsibilities
Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to a specific plan or program terms.
For information about benefits and eligibility, see One.Walmart.
The annual salary range for this position is $143, $286,000.00 Additional compensation includes annual or quarterly performance bonuses. Additional compensation for certain positions may also include :
- Stock
ㅤ
ㅤ
ㅤ
ㅤ
Minimum Qualifications...
Outlined below are the required minimum qualifications for this position. If none are listed, there are no minimum qualifications.
Option 1: Bachelors degree in Statistics, Economics, Analytics, Mathematics, Computer Science, Information Technology or related field and 5 years' experience in an analytics related field. Option 2: Masters degree in Statistics, Economics, Analytics, Mathematics, Computer Science, Information Technology or related field and 3 years' experience in an analytics related field. Option 3: 7 years' experience in an analytics or related fieldPreferred Qualifications...
Outlined below are the optional preferred qualifications for this position. If none are listed, there are no preferred qualifications.
Data science, machine learning, optimization models, PhD in Machine Learning, Computer Science, Information Technology, Operations Research, Statistics, Applied Mathematics, Econometrics, Publications or active peer reviewer in related journals or conference, Successful completion of one or more assessments in Python, Spark, Scala, or R, Using open source frameworks (for example, scikit learn, tensorflow, torch), We value candidates with a background in creating inclusive digital experiences, demonstrating knowledge in implementing Web Content Accessibility Guidelines (WCAG) 2.2 AA standards, assistive technologies, and integrating digital accessibility seamlessly. The ideal candidate would have knowledge of accessibility best practices and join us as we continue to create accessible products and services following Walmart's accessibility standards and guidelines for supporting an inclusive culture.Primary Location...
1375 Crossman Ave, Sunnyvale, CA , United States of AmericaWalmart and its subsidiaries are committed to maintaining a drug-free workplace and has a no tolerance policy regarding the use of illegal drugs and alcohol on the job. This policy applies to all employees and aims to create a safe and productive work environment.-
Data Scientist
Everfit- Hayward
-
Staff Data Scientist
Harnham- Hayward
-
Staff, Data Scientist
Full time Walmart- Hayward
-
Staff, Data Scientist
Full time Walmart- Hayward
-
Staff, Data Scientist
Full time Walmart- Hayward
-
Senior, Data Scientist
Full time Walmart- Hayward
-
Data Scientist
Only for registered members Dublin
-
Data Scientist
Only for registered members Dublin
-
Data Scientist
Full time Only for registered members Dublin
-
Data Scientist
Only for registered members Dublin
-
Data Scientist
Only for registered members Dublin
-
Data Scientist
Only for registered members Dublin
-
Data Scientist
Only for registered members Dublin, CA
-
Data Scientist
Only for registered members Dublin
-
Data Scientist
Only for registered members Dublin, CA
-
Data Scientist
Only for registered members Dublin, CA
-
Data Scientist
Only for registered members San Ramon
-
Data Scientist
Only for registered members San Ramon
-
Data Scientist
Only for registered members San Ramon
-
Data Scientist
Only for registered members San Ramon