Staff Software Engineer, Replication Foundations

We’re hiring a Staff Software Engineer to join the Replication Foundations team, part of Temporal’s Cloud Global Services (CGS) organization.

Replication Foundations owns and evolves Temporal’s core replication stack in Temporal OSS—the distributed systems backbone behind key Temporal Cloud capabilities like High Availability namespaces, cross-cluster and cross-region failover, and migration products that let customers move workloads between self-hosted Temporal and Temporal Cloud. The team also drives fundamental scalability and reliability mechanisms that define how Temporal operates at large scale.

At the Staff level, you’ll operate as a technical leader in Temporal’s distributed core: shaping protocols and correctness guarantees, leading complex efforts end-to-end, raising engineering rigor across replication foundations, and partnering cross-functionally to ensure OSS primitives unlock current and future cloud products. This role is deeply technical and requires comfort reasoning about consistency, concurrency, failure modes, performance, and operational safety in correctness-critical systems.

Key outcomes you’ll drive:

• Evolve replication protocols and correctness mechanisms that make failures (and cloud outages) a non-event for customers.
• Deliver scalability primitives (e.g., multi-cell namespaces) that unlock the next phase of Temporal Cloud growth.
• Raise the bar on safety, observability, and operability of Temporal’s replication layer across OSS and cloud.

What You'll Do

• Lead the design and implementation of core components of Temporal’s OSS replication stack, from initial design through rollout and long-term operational ownership.
• Design and evolve replication protocols that power:
  
  
  • High Availability namespaces
    
    
  • Cross-cluster and cross-region replication
    
    
  • Migration between Temporal clusters (cloud ↔ self-hosted, cloud ↔ cloud)
    
    
• Build scalability and reliability capabilities such as:
  
  
  • Multi-cell namespaces
    
    
  • Protocols enabling a single namespace to span multiple clusters
    
    
  • Dynamic split/merge strategies based on usage, hot spots, and capacity needs
    
    
• Reason deeply about correctness: consistency models, ordering guarantees, idempotency, failure recovery, and safe rollouts of protocol changes.
• Drive cross-team alignment with Cloud Enablement and other CGS teams to ensure OSS foundations support current and future cloud products.
• Author high-quality design docs that clarify invariants, trade-offs, failure modes, and operational playbooks for complex changes.
• Raise engineering standards through reviews, mentorship, and technical leadership—improving correctness testing, fault injection, and incident readiness.
• Participate in on-call/incident response related to replication and core system behavior, helping build durable fixes and prevention mechanisms. 

What You'll Bring

• 10+ years building production systems, including significant experience with distributed systems and correctness-critical infrastructure.
• Strong experience with replication, consistency, fault tolerance, and failure recovery in distributed environments.
• Demonstrated ability to design and implement concurrent, correctness-critical systems with clear invariants and safety guarantees.
• Proven track record of leading complex technical projects across teams—setting direction, driving execution, and landing changes safely in production.
• Hands-on experience debugging complex production issues involving race conditions, data consistency, partial failures, and performance degradation.
• Proficiency writing production-quality concurrent code, preferably in Go (Java/C++ or similar systems languages also welcome).
• Solid understanding of distributed systems fundamentals such as replication, sharding/partitioning, backpressure, failure detection, and durability mechanisms.
• Ability to operate with high ownership and minimal oversight, balancing deep technical rigor with pragmatic delivery.
• Curiosity and rigor in understanding how systems behave under stress, failure, and scale.

Nice to Have

• Experience designing or maintaining replication protocols or data-plane infrastructure.
• Experience with multi-cluster or multi-region architectures (active-active / active-passive).
• Familiarity with database internals, log-based replication, or event-sourced systems.
• Prior contributions to large OSS projects or distributed systems infrastructure.

Compensation

• The estimated pay range for this role is $212,000 - $285,000
• Additionally, this role is eligible to participate in Temporal's equity plan.

• Unlimited PTO, 12 Holidays + 2 Floating Holidays
• 100% Premiums Coverage for Medical, Dental, and Vision
• AD&D, LT & ST Disability, and Life Insurance (Standard & Supplemental Available)
• Empower 401K Plan
• Additional Perks for Learning & Development, Lifestyle Spending, In-Home Office Setup, Professional Memberships, WFH Meals, Internet Stipend and more!

Paid Time Off (PTO) and Benefits outside the United States vary by country, and are issued in partnership with Remote.com.  Additionally, Temporal offers perks to all international employees for learning & career development, a lifestyle spending account, in-home office setup (in addition to company-issued hardware), professional memberships, work-from-home meals, and access to the Calm app for mental wellness.

Travel

Temporal is a globally distributed, collaborative team that values opportunities for in-person connection. Occasional travel may be required for company events, team offsites, and other meaningful moments that bring us together.