12+ years of engineering experience, with 7+ years as the primary architect for infrastructure resiliency and high-availability architecture.
Deep hands-on AWS production experience in large multi-account, multi-region environments.
Proven ownership of highly available architectures with credible judgment on active-active versus active-passive strategies.
Track record of end-to-end BC/DR strategy ownership, including genuine recovery exercises.
Expert command of the AWS Well-Architected Framework, particularly the Reliability pillar.
Production depth with AWS container platforms (ECS/Fargate, EKS).
Strong data-tier resiliency experience with Aurora or RDS and replication technologies.
Responsibilities
Own the resiliency strategy and reference architecture for the AWS estate.
Lead business continuity and disaster recovery end to end, including RTO/RPO target-setting and automated failover design.
Run AWS Well-Architected Reviews with a focus on Reliability and Operational Excellence.
Drive resiliency into the container platform and data tier, ensuring predictable recovery for stateful workloads.
Encode resilience in automation, including IaC-defined failover paths and self-healing mechanisms.
Raise the quality bar on incident response for availability events and lead postmortems.
Translate continuity and availability requirements from various stakeholders into concrete AWS designs.
Benefits
Hybrid working model with options for remote work.
Opportunity to work in a complex organizational context with diverse teams.
Engagement in cutting-edge AWS technologies and practices.
Focus on continuous improvement and architectural evolution.
Participation in genuine recovery exercises and operational excellence initiatives.
Full Job Description
Principal Engineer - Resiliency
The role is preferred as Hybrid working in Austin TX, Houston TX or Denver CO, we will consider virtual for the right candidate. Position responsibilities
Own the resiliency strategy and reference architecture for the AWS estate: set availability targets by service tier, define the standard patterns for multi-AZ and multi-region operation, and drive existing services toward them
Own the decision framework for active-active versus active-passive versus warm-standby and pilot-light topologies, including the real tradeoffs in cost, operational complexity, and achievable RTO/RPO, and the migration path for services that need to move between them
Lead business continuity and disaster recovery end to end: RTO/RPO target-setting, automated failover design, regular DR exercises and game days, recovery runbook ownership, and translating exercise findings into durable architectural change
Run AWS Well-Architected Reviews across service teams with emphasis on the Reliability and Operational Excellence pillars, and convert findings into a prioritized, tracked remediation backlog rather than a report that sits on a shelf
Drive resiliency into the container platform: ECS/Fargate and EKS workload placement, capacity headroom, failure-domain isolation, graceful degradation, and recovery behavior under partial-region impairment
Drive resiliency into the data tier so stateful workloads recover as predictably as stateless ones, covering Aurora and RDS multi-AZ and Global Database topologies, DynamoDB global tables, ElastiCache, and S3 cross-region replication, with clear expectations for failover time, replica lag, and data-loss boundaries
Encode resilience in automation: IaC-defined failover paths, self-healing and auto-remediation, drift detection, and continuous fault injection so recovery capability is exercised routinely instead of documented once
Own the reliability signal set for core infrastructure: SLOs and error budgets, health-check and failover-trigger design, dependency mapping, and blast-radius analysis across control-plane and data-plane signals
Raise the quality bar on incident response for availability events, lead resiliency-focused postmortems, and convert repeat failure modes into architectural fixes and platform guardrails
Operate effectively across a complex organizational context, translating continuity and availability requirements from engineering, product, and business stakeholders into concrete AWS design and validated recovery capability
Required skills
12+ years of engineering experience, with at least 7 as the primary architect or technical owner of infrastructure resiliency and high-availability architecture at scale
Deep hands-on AWS production experience in large multi-account, multi-region environments, including VPC and networking design, Route 53, ALB/NLB, Global Accelerator, and the IAM and account-boundary decisions that constrain failover
Demonstrated ownership of highly available architectures where multi-AZ is the baseline and multi-region is real for critical workloads, with credible judgment about when active-active is warranted and when active-passive or warm-standby is the better tradeoff
Track record owning BC/DR strategy end to end: setting RTO/RPO targets, designing and automating regional failover, and running genuine recovery exercises rather than tabletop-only reviews
Expert command of the AWS Well-Architected Framework, particularly the Reliability pillar, with experience leading formal reviews and driving the resulting remediation to completion
Production depth with container platforms on AWS (ECS/Fargate, EKS, or both): scaling behavior, placement and failure domains, deployment safety, and degradation characteristics under failure
Strong data-tier resiliency experience with Aurora or RDS (multi-AZ, Global Database, failover behavior, replica lag) and at least one of DynamoDB global tables, ElastiCache replication, or S3 cross-region replication
Deep production experience designing and operating IaC at scale, Terraform preferred (CDK acceptable), with strong opinions on module strategy, state management, and guardrail enforcement across many accounts and environments
Expert command of CI/CD for infrastructure: pipeline design, drift detection, plan/apply workflows, and self-service patterns that serve engineering teams safely at scale
Hands-on experience with chaos engineering and fault injection in production environments, including failure-injection tooling and game-day operations
Strong grounding in observability for infrastructure: SLO and error-budget design, health-check strategy, dependency mapping, and state-of-the-fleet visibility
Cross-functional leadership, comfortable as a peer with senior engineering, product, and finance leaders on availability, redundancy cost, and recovery-capability conversations
Comfortable with the coordination work of a recently combined company: divergent AWS estates, in-flight unification, and the political work that comes with consolidation
Preferred qualifications
Hands-on experience with AWS resiliency tooling such as Resilience Hub, Route 53 Application Recovery Controller, Elastic Disaster Recovery, or Fault Injection Service
AWS Solutions Architect Professional certification, or equivalent demonstrable depth
Hands-on experience with infrastructure orchestration tooling beyond raw Terraform (Terragrunt, Atlantis, Spacelift, env0, Crossplane, or similar)
Background in SRE or platform reliability with strong instincts for SLO design, error budget policy, and toil reduction
Experience modeling the cost of redundancy (standby capacity, cross-region data transfer, replication overhead) and defending those tradeoffs to business stakeholders
Experience post-M&A integrating infrastructure and resiliency practices across two or more legacy AWS estates
Experience in regulated industries, or other domains with mission-critical workloads and strict continuity requirement
Pay is based upon candidate experience and qualifications, as well market and business considerations: Summary Pay Range: $208,000.00 to $244,000.00
