Engineering Manager, Data Platform - San Francisco, United States - Anthropic Limited

Description

We are looking for an Engineering Manager to lead the Tokens: Data Platform team, which owns the data platform responsible for creating the datasets used by our largest training runs.

Most of Anthropics research and product builds on top of the best pretrained models that we can produce, which in turn rely on the best pretraining data.

These datasets drive the quality of the model and are therefore critical to get right.

The teams responsibilities are as follows:1. Own our largest data pipelines required for turning raw data into datasets, in turn used to train our models2. Ensure that data processing happens seamlessly across petabyte scale and across a broad set of use cases3.

Innovate by creating data primitives, collaborating with our data research efforts, to create the composable building blocks of data transformations at AnthropicResponsibilities:
Grow, coach and support your reports in understanding, and pursuing, their professional growthPrioritize the teams work to make sure data processing is never the bottleneck in generating pretraining data required to launch new training runsDesign processes (e.g.

postmortem review, incident response, on-call rotations) that help the team operate effectivelyRun the teams recruiting efforts efficiently, ensuring we can grow as quickly as we need through a period of rapid growthYou may be a good fit if you:Believe that advanced AI systems could have a transformative effect on the world, and are interested in helping make sure that transformation goes wellAre interested in distributed data processing for building the best pretraining datasets, in turn used to build the best LLMsAre an experienced manager and actively enjoy people managementHave at least 1 year of prior engineering management or equivalent experience (e.g.

as a tech lead)

Strong candidates may also have:
Strong people management experience: Coaching, performance evaluation, mentorship, career developmentHave built a large scale data platform (100s of jobs running concurrently processing data at the petabyte scale)

Technical expertise:

Quickly understanding systems design tradeoffs, keeping track of rapidly evolving software systemsSolid project management skills: Prioritization, communicating across team/org boundariesExperience recruiting for your team: Predicting staffing needs, designing interview loops, evaluating candidates and closing themFamiliarity with PythonDeadline to apply: None.

Applications will be reviewed on a rolling basis.#J-18808-Ljbffr