Jobs
>
Palo Alto

    Senior Research Engineer, Language Model Evaluations - Palo Alto, United States - Hippocratic AI

    Default job background
    Description
    Hippocratic AI's mission is to develop the first safest focused Large Language Model (LLM) for healthcare.

    The company believes that a safe LLM can dramatically improve healthcare accessibility and health outcomes in the world by bringing deep healthcare expertise to every human.

    No other technology has the potential to have this level of global impact on health.

    The company was co-founded by CEO Munjal Shah, alongside a group of physicians, hospital administrators, healthcare professionals, and artificial intelligence researchers from El Camino Health, Johns Hopkins, Washington University in St.

    Louis, Stanford, Google, and Nvidia.

    Hippocratic AI has received a total of $120M in funding and is backed by leading investors, including General Catalyst, Andreessen Horowitz, Premji Invest, and SV Angel.

    About the role

    We are looking for a Research Engineer to lead evaluations for Hippocratic AI's 1 trillion+ parameters constellation of Large Language Models.

    Your job will be to design and implement evaluations that allow Hippocratic AI to evaluate the performance and safety of our models.

    As a Research Engineer focused on Evaluation, you'll work closely with our research and applied science teams to design experiments and build evaluation infrastructure.

    You'll help validate performance and safety across a wide range of important tasks.


    You'll help to assure that our LLMs are well-benchmarked with known performance and safety on a wide range of healthcare related tasks, allowing us to compare against human feedback.

    Requirements:

    5+ years Python programming experience / machine learning research

    Have experience using Large Language Models, preferably have trained or fine tuned large models in the past.

    Are comfortable writing code

    Want to learn more about machine learning research

    Care about patient safety

    You want to design and implement rigorous evaluations


    Preferred:
    Building user interfaces for data analysis

    Developing robust evaluation metrics for language models

    Handling textual dataset sourcing, curation, and processing tasks at scale

    Statistics


    Representative projects:
    Designing and running a new evaluation that tests our model's reasoning capabilities

    Leading the vision of what it takes to safely evaluate patient safety in the world of Generative AI

    Devise a consistent but representative evaluation suite for healthcare conversations

    Running experiments to determine how prompting techniques affect results on industry benchmarks

    Improving the tooling that researchers use to implement evaluations

    Explaining our evaluations and their results to internal decision makers and Stakeholders

    Collaborating with a research team to develop a robust evaluation for a new model capability they are developing

    #J-18808-Ljbffr

  • Stanford Health Care

    Clinical Nurse

    2 days ago


    Stanford Health Care Palo Alto, United States

    0.9 FTE Full time Day - 12 Hour R Onsite 60195 L4 ICU Nursing 300 Pasteur Dr,PALO ALTO,California · If you're ready to be part of our legacy of hope and innovation, we encourage you to take the first step and explore our current job openings. Your best is waiting to be discovered ...

  • SAP

    UX Researcher

    1 week ago


    SAP Palo Alto, United States

    **We help the world run better · **At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. How? We focus every day on building the foundation for tomorrow and creating a workplace that embrac ...

  • Momentum for Health

    Pd Registered Nurse

    1 week ago


    Momentum for Health Palo Alto, United States

    **This is on-call as needed position. Experienced LVNs are welcome to apply.** · **Pay rates for Union positions are set in the Collective Bargaining Agreement with SEIU Local 521. Each position has a total of six steps, Step 1 (entry level) through Step 6 (15 years of employment ...


  • The Primary School East Palo Alto, United States

    **ABOUT US**: · The Primary School is creating a new school model that brings together all of the adults in a child's life, including parents, educators, and medical and mental health providers, starting from a very early age. We recognize the entrenched racism that exists within ...


  • YMCA of Silicon Valley East Palo Alto, United States

    ***: · The YMCA Early Learning Readiness Program also known as Nana y Yo Facilitator will work directly with informal family, friend and neighbor caregivers and the children they care for in twice weekly two hour groups that are established to provide rich experiences for both ad ...


  • Meta Menlo Park, United States

    **Internal Audit Manager - Sales & Marketing and LATAM Responsibilities**: · - Manages quarterly vertical risk assessment with limited oversight · - Participates in annual enterprise risk assessment · - Assist in developing and executing a dynamic audit plan to address high risk ...

  • Stellar Innovations

    Civil engineer

    1 week ago


    Stellar Innovations University Park, United States

    About the Company · Energy conversion, that is the focus of our company. We are a medium-sized company based in Riesa and Dresden. Our core competence lies in the construction, maintenance, and modernization of substations or grid feed-ins - in the range of 10 to 380kV. · What do ...


  • MFS Talent Palo Alto, United States

    Job Details Shift: 40 hours per week, Monday- Friday · Work Location: Onsite, Palo Alto, CA · Duties and ResponsibilitiesCollaborates with nursing, interdisciplinary teams and community resources to define problems, develop · Plans and provide appropriate referrals. · Develops, ...


  • NovaWorks West Menlo Park, United States InternshipSHIP

    Internship in Creative Therapy · Full-time · 93449 Waldmünchen, Germany · No professional experience required · West Menlo Park location · Company Description: · We are a value-oriented psychosomatic and somatic clinic group based in Bad Kissingen. We operate 8 clinics at 5 locat ...

  • Microsoft

    Applied Scientist

    1 day ago


    Microsoft Mountain View, United States

    On our team, understanding the customer is just the beginning. We analyze customer data, work to understand customer problems, build models to assess these problems, and then work with partner teams to build and ship machine learning models as customer-facing features. A flexible ...


  • Waymo Mountain View, United States

    Waymo is an autonomous driving technology company with a mission to make it safe and easy for people and things to get where they're going. Since our start as the Google Self-Driving Car Project in 2009, Waymo has been focused on building the Waymo Driver—The World's Most Experie ...


  • Mitra Chem Mountain View, United States

    We have set up a state-of-the-art 15,000 sq ft industrial R&D center in Mountain View, CA. Join our team and be a part of the growing battery revolution. · We are looking for **Battery Materials Scientists **to design, develop and scale-up production of innovative materials, leve ...


  • Mitra Chem Mountain View, United States

    We have set up a state-of-the-art 15,000 sq ft industrial R&D center in Mountain View, CA. Join our team and be a part of the growing battery revolution. · We are looking for a **Cathode Precursor Scientist** to design, develop, and scale-up the production of innovative materials ...

  • El Camino Health

    Social Worker

    2 days ago


    El Camino Health Mountain View, United States

    El Camino Health is committed to hiring, retaining and growing the best and brightest professionals who will carry our mission and vision forward. We are proud of our reputation in the community: One built on compassion, innovation, collaboration and delivering high-quality care. ...

  • El Camino Health

    Social Worker

    2 days ago


    El Camino Health Mountain View, United States

    El Camino Health is committed to hiring, retaining and growing the best and brightest professionals who will carry our mission and vision forward. We are proud of our reputation in the community: One built on compassion, innovation, collaboration and delivering high-quality care. ...


  • NICE Systems Mountain View, United States

    At NICE, we don't limit our challenges. We challenge our limits. Constantly. We're relentless. We're ambitious. And we make an impact. Our NICErs bring their A game and spend each day turning it into an A+. And if you're like us, we can offer you the kind of challenge that will l ...


  • Lightmatter Mountain View, United States

    Lightmatter builds chips for artificial intelligence infrastructure that leverage the unique properties of light for efficient interconnect and computing. If you're a collaborative Product Management Leader who has a passion for innovation, solving challenging technical problems, ...


  • Khan Lab School Mountain View, United States

    **ASSISTANT DIRECTOR OF ADMISSIONS** · **POSITION** **DESCRIPTION** · Khan Lab School · Role: Assistant Director of Admissions · Program: Khan Lab School · Location: Mountain View, CA · Start Date: By July 1, 2024 · Reports To: Director of Admissions · **ABOUT KHAN LAB SCHOOL** · ...

  • Aurora Technologies

    Data Scientist

    1 week ago


    Aurora Technologies Bay View, United States

    As a Data Scientist (m/f/d/-) at formigas, you will be part of our co_labs and work with data from various industries. You will explore advanced algorithms and build initial prototypes, particularly in the areas of Machine Learning, Computer Vision, and Natural Language Processin ...

  • Caminar

    Program Director I

    1 day ago


    Caminar Redwood City, United States

    **Position Title**: · Program Director I · **Program/Dept**: · REACH / Case Management · **Reports to**: · Director of Case Management · **Classification**: · Regular, Full-Time, Exempt · **Compensation**: · Range starting at $100K+ DOEE + Full Benefits Package · **Agency Descrip ...