Director of AI Foundations, Foundation Model Evaluation - Mountain View, CA, USA; San Francisco, CA, USA
46 minutes ago
Job description
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The World's Most Experienced Driver—to improve access to mobility while saving thousands of lives now lost to traffic crashes. The Waymo Driver powers Waymo's fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases. The Waymo Driver has provided over ten million rider-only trips, enabled by its experience autonomously driving over 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.
The mission of the Waymo AI Foundations team is to develop machine learning solutions addressing open problems in autonomous driving, towards the goal of safely operating Waymo vehicles in dozens of cities and under all driving conditions. As part of our work, we also initiate and foster collaborations with other research teams in Alphabet. AI Foundations areas that we are currently focusing on include reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation.
This role follows a hybrid work schedule and you will report to the Vice President, AI Foundations.
You will:
- Be accountable for the productive integration of the Foundation Model Teacher setups developed by the AIF and ML Infra teams into customer student team frameworks, and for maintaining these integrations
- Working with customers to understand their needs and working backwards from those needs
- Optimize the teacher / student model distillation process, model inference footprints and sampling techniques to fit compute constraints
- Define and implement metrics and evaluation infrastructure necessary for evaluating Waymo Foundation Models
- Develop frameworks and approaches that power data 'Data Flywheel' mining and annotation applications using Foundation Models training and evaluation, using Waymo data and also third party data
You have:
- Machine learning expertise, with a track record of contributing to large scale ML production systems and their development cycle
- Software engineering infrastructure experience. Technical ability to dive deep into complex systems, infrastructure, algorithms, and modeling techniques
- Experience with evaluating and optimizing large scale ML / AI systems
- Experience of leading engineering teams of 10+
- Hired and developed world class managers and scientists
- Strong communication and planning skills (e.g. building 12 month roadmaps)
- Strong cross-organization collaboration skills
We prefer:
- Direct experience developing large pre-trained foundational models (LLM / VLM / Video models)
- Experience across multiple ML applications (e.g. natural language processing and vision) in multiple settings (e.g. founded a successful startup and lead a significant team at a FAANG company)
- Has experience with the dev-ops side of ML models (e.g. model training latency, regression prevention)
The expected base salary range for this full-time position across US locations is listed below. Actual starting pay will be based on job-related factors, including exact work location, experience, relevant training and education, and skill level. Your recruiter can share more about the specific salary range for the role location or, if the role can be performed remote, the specific salary range for your preferred location, during the hiring process.
Waymo employees are also eligible to participate in Waymo's discretionary annual bonus program, equity incentive plan, and generous Company benefits program, subject to eligibility requirements.
Salary Range $332,000—$421,000 USDSimilar jobs
ML Engineer, Foundation Model Evaluation
1 month ago
The Waymo Driver powers Waymo's fully autonomous ride-hail service. As part of our work in the AI Foundations team we develop machine learning solutions addressing open problems in autonomous driving.We are looking for an ML Engineer to advance state-of-the-art methodologies for ...
ML Engineer, Foundation Model Evaluation
1 month ago
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. · Develop and extend cutting-edge research in robotics and machine learning to advance state-of-the-art methodologies for evaluating the quality, safety, and realism of embod ...
The Waymo Driver powers Waymo's fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases. · Be accountable for the productive integration of the Foundation Model Teacher setups developed by the AIF and ML Infra teams into cu ...
We are looking for a Director of AI Foundations to lead our team in developing machine learning solutions for autonomous driving. The successful candidate will be accountable for the productive integration of Foundation Model Teacher setups and will work with customers to underst ...
Member of Technical Staff, Model Evaluation
3 days ago
xAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. · ...
+Job summary · Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. · +Develop novel metrics and sampling techniques to measure the driving trajectories generated by ML models. · Employ creative simulation strategies to measure ...
Fullstack Engineer, GenAI Model Evaluation
1 month ago
The GenAI Model Evaluation team is the main line of defense in ensuring customer safety.We are looking for an experienced fullstack developer to take ownership of the tooling we use to determine the best model for release. · Create, maintain and expand internal tools for evaluati ...
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. · Expected base salary range for this full-time position across US locations is listed below: $170,000—$216,000 USD · ...
++Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. · + · +++BS/MS/PhD in Computer Science, Machine Learning, Robotics, Statistics, Physics, Math or another quantitative area · Proficiency in programming in Python or C++ · ...
Waymo is building state-of-the-art AI systems to improve access to mobility while saving thousands of lives now lost to traffic crashes. · ...
We are looking for a creative and rigorous Research Scientist to focus on tailoring world models for effective use in policy learning and evaluation for autonomous vehicles.To lead this transformative shift in mobility, we've built a world-class team advancing the state of the ar ...
We are looking for a creative and rigorous Research Scientist to focus on tailoring world models for effective use in policy learning and evaluation for autonomous vehicles. · Develop and refine world models that support realistic and diverse counterfactual reasoning, scenario ge ...
The Large Model Evaluation team is at the nexus of Waymo's AI ambition. With advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs), Waymo is building state-of-the-art AI systems that handle the full complexity of real-world driving. · The team is looking ...
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009... · ...
AI Evaluation Engineer, Siri Core Modeling
1 week ago
+We are seeking talented engineers to join our team and push the boundaries of evaluations for Siri AI Agents. · +Design, build and maintain auto-evaluators that measure the quality of Siri's core AI engine. · , Collaborate with scientists and engineers across software and ML tea ...
We are looking for exceptionally driven engineers and applied scientists with deep experience in generative modeling to help improve the quality of our world models. · Develop detailed, domain-specific benchmarks for evaluating world foundation models. · Use sophisticated prompt ...
As an AIML Engineering Program Manager (EPM) at Apple, you will drive complex and innovative GenAI-based features that are essential to the success of Apple's products. · ...
We seek candidates with exceptional emotional intelligence who can lead diverse cross-functional teams to deliver key features integrated across all Apple products and services. · ...
We are seeking a highly innovative and passionate Applied AI Scientist to develop cutting-edge AI/ML models for the automatic grading and quality assessment of our internal GenAI products.In this pivotal role, you will design and advance state-of-the-art autograder systems that e ...
We are seeking a highly innovative and passionate Applied AI Scientist to develop cutting-edge AI/ML models for the automatic grading and quality assessment of our internal GenAI products. · ...
+ We are seeking a highly innovative and passionate Applied AI Scientist to develop cutting-edge AI/ML models for the automatic grading and quality assessment of our internal GenAI products. · ...