- Operate and automate large GPU clusters including provisioning, imaging, and capacity planning.
- Write software that abstracts cluster management and presents a unified interface for training and inference.
- Extend scheduling/orchestration (Kubernetes, Slurm, or similar) for topology-aware placement, preemption, quotas, and fair-share multi-tenancy.
- Monitor and improve operational metrics of speed, reliability, and error recovery.
- Build reliable storage and artifact paths for datasets, checkpoints, and logs with clear retention and lineage.
- Partner with researchers to unblock scale runs and advise on parallelism and performance trade-offs.
- Bachelor's degree or equivalent experience in computer science, engineering, or similar.
- Proficiency in at least one backend language (we use Python or Rust).
- Experience operating large-scale clusters and container orchestration systems (e.g. Kubernetes or Slurm).
- Comfort operating across the stack and owning projects end-to-end.
- Thrive in a highly collaborative environment involving many, different cross-functional partners and subject matter experts.
- A bias for action with a mindset to take initiative to work across different stacks and different teams where you spot the opportunity to make sure something ships.
- Strong systems background: Linux, networking, and infrastructure-as-code.
- Familiarity with CUDA/NCCL and performance profiling for distributed training/inference.
- Prior work supporting large-scale model training or inference environments.
- Understanding of deep learning frameworks (e.g., PyTorch, TensorFlow, JAX) and their underlying system architectures.
- Track record of working in fast-paced environments balancing care with urgency.
- Location: This role is based in San Francisco, California.
- Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $350,000 - $475,000 USD.
- Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
- Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
-
We're building the company which will de-risk the largest infrastructure build-out in history. · When people finance GPU clusters, the datacenters housing them, and the infrastructure powering them, they need "offtake" - meaning someone has signed a contract to lease the cluster ...
San Francisco $170,000 - $300,000 (USD)3 weeks ago
-
We're building the company which will de-risk the largest infrastructure build-out in history. · You've managed at least one GPU training cluster in the past (ideally a cluster with >1k GPU's but not required) · You deeply understand Linux, networking fundamentals, CUDA, NCCL, an ...
San Francisco $170,000 - $300,000 (USD) Full time1 month ago
-
The Frontier Systems team at OpenAI builds, launches, and supports the largest supercomputers in the world that OpenAI uses for its most cutting edge model training. · We take data center designs, turn them into real, working systems and build any software needed for running larg ...
San Francisco $310,000 - $460,000 (USD)3 weeks ago
-
We take data center designs, turn them into real, working systems and build any software needed for running large-scale frontier model trainings. · ...
San Francisco6 days ago
-
The Frontier Systems team at OpenAI builds, launches and supports the largest supercomputers in the world that OpenAI uses for its most cutting edge model training. · We take data center designs turn them into real working systems and build any software needed for running large-s ...
San Francisco, CA1 month ago
-
The Frontier Systems team at OpenAI builds, launches, and supports the largest supercomputers in the world that OpenAI uses for its most cutting edge model training. · Own and improve the system health checks that keep our hyperscale supercomputers stable during model training. · ...
San Francisco6 days ago
-
The Frontier Systems team at OpenAI builds large supercomputers for cutting-edge AI research. As a Software Engineer on this team, you'll build critical infrastructure to keep these supercomputers running reliably. · Owning and improving system health checks to keep hyperscale su ...
San Francisco $250,000 - $445,000 (USD)2 weeks ago
-
The Data Visualization team at OpenAI is responsible for building and maintaining all the visualization tools used for analyzing various software and hardware aspects of our custom-built hyperscale supercomputers. · We maximize available supercomputing capacity for researchers an ...
San Francisco $255,000 - $405,000 (USD)1 month ago
-
The Data Visualization team at OpenAI is responsible for building and maintaining all the visualization tools used for analyzing various software and hardware aspects of our custom-built hyperscale supercomputers. · ...
San Francisco, CA3 weeks ago
-
The Platform Visualization team at OpenAI is responsible for building and maintaining all the visualization tools used for analyzing various software and hardware aspects of our custom-built hyperscale supercomputers. · ...
San Francisco1 month ago
-
The Data Visualization team at OpenAI is responsible for building and maintaining all the visualization tools used for analyzing various software and hardware aspects of our custom-built hyperscale supercomputers. · Develop and maintain full-stack visualization tools for hardware ...
San Francisco6 days ago
-
+The Compute Runtime team builds the low level framework components to power our ML training systems. · We work on building robust, scalable, high performance components to support our distributed training workloads. · About This Role · Work across our Python and Rust stack · ...
San Francisco, CA3 weeks ago
-
Magic's mission is to build safe AGI that accelerates humanity's progress on the world's most important problems. · ...
San Francisco, CA1 month ago
-
We're looking for people who love optimizing an end to end system, understanding high performance I/O to maximize local performance and distributed across our supercomputers. · ...
San Francisco6 days ago
-
The Compute Runtime team builds low level framework components to power ML training systems. As a Distributed Systems engineer you will work on delivering APIs orchestrating thousands of computers moving vast amounts of data. · ...
San Francisco Full time1 month ago
-
We are hiring for a fast-growing AI marketing analytics company that helps enterprises understand what truly drives business outcomes. · Work on GPU-native analytics and causal AI used by Fortune 100 companies. · ...
San Francisco1 week ago
-
We are hiring for a fast-growing AI marketing analytics company that helps enterprises understand what truly drives business outcomes. · ...
San Francisco, CA6 days ago
-
We are looking for people who have a background in low level performance critical software. Experience with collective communication is a bonus. · ...
San Francisco6 days ago
-
About The Team · The Workload Networking team is responsible for the collective communication stack used in our largest training jobs. Using a combination of C++ and CUDA we work on novel collective communication techniques that enable efficient training of our flagship models on ...
San Francisco, CA1 month ago
-
About FluidStack is the AI Cloud Platform. We build GPU supercomputers for top AI labs, governments, and enterprises. · We put our customers first in everything we do. · Perform design and equipment submittal review for new Data Centers in your region. · Troubleshoot, conduct Roo ...
San Francisco, CA1 month ago
-
The Workload Networking team is responsible for the collective communication stack used in our largest training jobs. · Using a combination of C++ and CUDA we work on novel collective communication techniques that enable efficient training of our flagship models on our largest cu ...
San Francisco $380,000 - $555,000 (USD)1 month ago
Software Engineer, Supercomputing - San Francisco - Thinking Machines Lab
Description
Thinking Machines Lab's mission is to empower humanity through advancing collaborative general intelligence. We're building a future where everyone has access to the knowledge and tools to make AI work for their unique needs and goals.
We are scientists, engineers, and builders who've created some of the most widely used AI products, including ChatGPT and , open-weights models like Mistral, as well as popular open source projects like PyTorch, OpenAI Gym, Fairseq, and Segment Anything.
About the Role
We're looking for an engineer to design, build, and operate the GPU supercomputing environment that powers large-scale training and inference. You will deliver high-performant, reliable, and cost-efficient compute so our users and researchers can move fast at scale.
Note: This is an "evergreen role" that we keep open on an on-going basis to express interest. We receive many applications, and there may not always be an immediate role that aligns perfectly with your experience and skills. Still, we encourage you to apply. We continuously review applications and reach out to applicants as new opportunities open. You are welcome to reapply if you get more experience, but please avoid applying more than once every 6 months. You may also find that we put up postings for singular roles for separate, project or team specific needs. In those cases, you're welcome to apply directly in addition to an evergreen role.
What You'll Do
Minimum qualifications:
-
Supercomputing Engineer
Only for registered members San Francisco
-
Supercomputing Engineer
Full time Only for registered members San Francisco
-
Software Engineer, Frontier Systems
Only for registered members San Francisco
-
Software Engineer, Frontier Systems
Only for registered members San Francisco
-
Software Engineer, Frontier Systems
Only for registered members San Francisco, CA
-
Software Engineer, Frontier Systems
Only for registered members San Francisco
-
Software Engineer, Frontier Systems
Only for registered members San Francisco
-
Software Engineer, Data Visualization
Only for registered members San Francisco
-
Software Engineer, Data Visualization
Only for registered members San Francisco, CA
-
Software Engineer, Data Visualization
Only for registered members San Francisco
-
Software Engineer, Data Visualization
Only for registered members San Francisco
-
Software Engineer, Distributed Systems
Only for registered members San Francisco, CA
-
Software Engineer
Only for registered members San Francisco, CA
-
Software Engineer, Distributed Systems
Only for registered members San Francisco
-
Software Engineer, Distributed Systems
Full time Only for registered members San Francisco
-
Senior SRE
Only for registered members San Francisco
-
Senior Site Reliability Engineer
Only for registered members San Francisco, CA
-
Software Engineer, Collective Communication
Only for registered members San Francisco
-
Software Engineer, Collective Communication
Only for registered members San Francisco, CA
-
Senior Electrical Engineer
Only for registered members San Francisco, CA
-
Software Engineer, Collective Communication
Only for registered members San Francisco