[Remote] Principal Engineer, Compute Fleet Management
Note: The job is a remote job and is open to candidates in USA. Databricks is a data and AI company focused on enabling data teams to solve complex problems through their infrastructure platform. The Principal Engineer for Compute Fleet Management will lead efforts in optimizing compute resources across major cloud platforms, ensuring high availability and efficiency.
Responsibilities
• Pioneering Fleet Optimization: Provisioning and pooling of O(Billion)s of cloud resources to achieve peak workload performance, industry-leading efficiency, and robust resource isolation
• Delivering Hyper-Scale Resilience: Build the architecture that guarantees horizontal scaling and resilience against zonal or even cloud account-level failures, ensuring Databricks is always on
• Owning the Critical Path: Lead the development of the lowest-dependency systems required to bootstrap and manage our massive compute platform
• High Availability: Achieve and maintain 99.99% availability for all batch and serving workloads
• Stellar Efficiency: Drive utilization to 60% or higher—a crucial metric that requires balancing high efficiency with unwavering tolerance for cloud failures
• Best-in-Class Isolation: Architect and enforce strong security and performance isolation across a diverse range of customer workloads
• Leading Transformative Projects: Taking ownership of complex, cross-team, cross-layer, and multi-quarter strategic engineering initiatives from concept to execution
• Distributed Systems Mastery: Deep, hands-on experience developing and operating high-scale distributed systems on at least one major public cloud
• Influence Without Authority: Proven ability to drive consensus, establish technical direction, and lead large technical efforts across organizational boundaries
• Execution Discipline: Exceptional strength in planning, tracking project progress, and managing complex cross-organizational dependencies
Skills
• Leading Transformative Projects: Taking ownership of complex, cross-team, cross-layer, and multi-quarter strategic engineering initiatives from concept to execution
• Distributed Systems Mastery: Deep, hands-on experience developing and operating high-scale distributed systems on at least one major public cloud
• Influence Without Authority: Proven ability to drive consensus, establish technical direction, and lead large technical efforts across organizational boundaries
• Execution Discipline: Exceptional strength in planning, tracking project progress, and managing complex cross-organizational dependencies
• Experience managing and scaling a massive fleet of GPUs for AI/ML workloads
• Experience with developing and operating large-scale distributed systems across all major clouds (AWS, Azure, and GCP)
Benefits
• Eligibility for annual performance bonus
• Equity
• Comprehensive benefits and perks
Company Overview
• Databricks is a data and AI platform that unifies data engineering, analytics, and machine learning on a lakehouse architecture. It was founded in 2013, and is headquartered in San Francisco, California, USA, with a workforce of 5001-10000 employees. Its website is https://www.databricks.com.
Company H1B Sponsorship
• Databricks has a track record of offering H1B sponsorships, with 385 in 2025, 319 in 2024, 227 in 2023, 222 in 2022, 166 in 2021, 64 in 2020. Please note that this does not guarantee sponsorship for this specific role.
Apply tot his job
Apply To this Job
Responsibilities
• Pioneering Fleet Optimization: Provisioning and pooling of O(Billion)s of cloud resources to achieve peak workload performance, industry-leading efficiency, and robust resource isolation
• Delivering Hyper-Scale Resilience: Build the architecture that guarantees horizontal scaling and resilience against zonal or even cloud account-level failures, ensuring Databricks is always on
• Owning the Critical Path: Lead the development of the lowest-dependency systems required to bootstrap and manage our massive compute platform
• High Availability: Achieve and maintain 99.99% availability for all batch and serving workloads
• Stellar Efficiency: Drive utilization to 60% or higher—a crucial metric that requires balancing high efficiency with unwavering tolerance for cloud failures
• Best-in-Class Isolation: Architect and enforce strong security and performance isolation across a diverse range of customer workloads
• Leading Transformative Projects: Taking ownership of complex, cross-team, cross-layer, and multi-quarter strategic engineering initiatives from concept to execution
• Distributed Systems Mastery: Deep, hands-on experience developing and operating high-scale distributed systems on at least one major public cloud
• Influence Without Authority: Proven ability to drive consensus, establish technical direction, and lead large technical efforts across organizational boundaries
• Execution Discipline: Exceptional strength in planning, tracking project progress, and managing complex cross-organizational dependencies
Skills
• Leading Transformative Projects: Taking ownership of complex, cross-team, cross-layer, and multi-quarter strategic engineering initiatives from concept to execution
• Distributed Systems Mastery: Deep, hands-on experience developing and operating high-scale distributed systems on at least one major public cloud
• Influence Without Authority: Proven ability to drive consensus, establish technical direction, and lead large technical efforts across organizational boundaries
• Execution Discipline: Exceptional strength in planning, tracking project progress, and managing complex cross-organizational dependencies
• Experience managing and scaling a massive fleet of GPUs for AI/ML workloads
• Experience with developing and operating large-scale distributed systems across all major clouds (AWS, Azure, and GCP)
Benefits
• Eligibility for annual performance bonus
• Equity
• Comprehensive benefits and perks
Company Overview
• Databricks is a data and AI platform that unifies data engineering, analytics, and machine learning on a lakehouse architecture. It was founded in 2013, and is headquartered in San Francisco, California, USA, with a workforce of 5001-10000 employees. Its website is https://www.databricks.com.
Company H1B Sponsorship
• Databricks has a track record of offering H1B sponsorships, with 385 in 2025, 319 in 2024, 227 in 2023, 222 in 2022, 166 in 2021, 64 in 2020. Please note that this does not guarantee sponsorship for this specific role.
Apply tot his job
Apply To this Job