# Staff Software Engineer - AI Compute, Together Cloud at Together AI

> Set architecture and build the GPU virtualization, data center IaaS, and global scheduling platform powering Together's AI cloud.

- Company: Together AI (together.ai)
- Location: San Francisco
- Places: San Francisco, United States
- Posted: 2026-09-23
- Level: Staff
- Team: Together Cloud - AI Compute
- Apply: https://job-boards.greenhouse.io/togetherai/jobs/5224090007
- Page: https://ye.saranatour.dev/jobs/staff-software-engineer-ai-compute-together-cloud-together-ai-kn7ddddbe9es5dtg6jqy5dcq0x8hc22p

## The posting

Staff Software Engineer - AI Compute, Together Cloud
Together AI · San Francisco

### **About the Role**

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the \[Together GPU Clusters\](https://www.together.ai/gpu-clusters) flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products.

As a Staff Software Engineer focusing on AI Compute in the Together Cloud org, you will set technical direction for and build major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware: GB300s/VRs, BlueField DPUs, InfiniBand and dual/quad-plane RoCEv2 fabrics. That virtualized computing platform powers our own SaaS products – inference, RL, and fine-tuning – and serves external cloud customers through self-serve offerings such as on-demand/reserved Kubernetes/Slurm clusters, across dozens of data centers and hundreds of thousands of GPUs.

This is an architect-and-build role. Fully automated bootstrapping of GPU data centers, high-performance virtualization of GPU compute and DC networking without compromising isolation or portability, and fault-tolerant decentralized control planes — you'll set the architecture for these across our global and in-DC services, and be a key owner of the hardest parts, in the code as well as the design. Your designs will span the IaaS layer of a greenfield Vera Rubin data center up to the global management plane that schedules capacity across all of them. At this level the job is as much leverage as code: the standards you set and the engineers you grow decide how fast the rest of Together Cloud ships.

### **Responsibilities**

- **Own the GPU and network virtualization stack:**the hypervisor, kernel, and SDN work that keeps GPU compute and DC networking high-performance, portable, and strongly isolated across heterogeneous hardware.
- **Own the in-DC IaaS layer:** architect and roadmap the services, Kubernetes operators, and libraries that provision and manage compute, storage, and networks in our data centers — VMs, parallel filesystems, VPCs, and InfiniBand partitions; lead its build-out for a new Vera Rubin data center with thousands of GPUs, from hardware bring-up to customer-facing API.
- **Design the GPU scheduling and global management plane:** the distributed control plane behind on-demand and reserved clusters across dozens of data centers, including the systems that scale per-cluster limits and automate the onboarding of new capacity.
- **Architect monitoring and automated remediation for fault tolerance:**the strategy for automated detection, isolation, and recovery of failed nodes that keeps distributed pretraining and large-scale inference running through hardware failures.
- **Set technical direction across teams:** lead design reviews, resolve cross-cutting architectural disagreements, unblock cross-team dependencies and integration risks, and define the standards other engineers build against — measured in cluster reliability, time-to-first-GPU on new capacity, and quality at scale.
- **Grow the team:** mentor senior and junior engineers, deepen the team's expertise in v