5+ years in infrastructure, platform, or backend engineering roles with hands-on distributed systems experience
Strong understanding of distributed systems fundamentals including consensus and failure modes
Production experience with major cloud providers (AWS, GCP, or Azure) at scale
Hands-on experience running Kubernetes in production, including upgrades and security policy
Experience with distributed data stores, message queues, or event streaming systems
Strong background in infrastructure-as-code (Terraform preferred) and CI/CD pipelines
Proficiency in at least one programming language (Go, Python, or Rust) for building distributed tooling
Experience with production reliability, including on-call and incident response
Responsibilities
Design and operate distributed systems for AI agents, focusing on consistency and fault tolerance
Architect service-to-service communication and state management across regions
Own Kubernetes cluster architecture, scaling, and security hardening
Build and maintain Terraform modules and GitOps pipelines for infrastructure changes
Design for reliability and auditability required by regulated financial institutions
Diagnose and resolve distributed-systems failure modes under production pressure
Build observability tools to proactively identify issues
Collaborate with ML, platform, and security engineering on capacity planning and architecture reviews
Participate in an on-call rotation for production infrastructure
Mentor engineers on distributed systems design and review architecture changes
Benefits
Collaborative onsite work environment
Opportunity to work on cutting-edge AI technology in financial services
Mentorship opportunities within a skilled engineering team
Engagement with cross-functional teams including ML and security
Focus on building reliable systems for regulated industries
Full Job Description
What We're Looking For:
At Kastle our Infrastructure team is responsible for building the underlying systems that help thousands of people daily, make their largest and most important financial transactions.
We're looking for a Senior Infrastructure Engineer with deep distributed systems expertise to design, build, and operate the infrastructure that powers Kastle's AI agents in production at banks, lender and other financial institutions. You'll work on the hard problems of distributed systems - consistency, fault tolerance, horizontal scalability, and low-latency coordination - across AWS/GCP/Azure, Kubernetes, and Terraform, keeping compliance-critical systems reliable as we grow. This is an onsite role working closely with platform, ML, and security engineering.
What You'll Do:
Design and operate distributed systems that power AI agents in production, with an emphasis on consistency, fault tolerance, and horizontal scalability
Architect service-to-service communication, coordination, and state management for systems spanning multiple regions and availability zones
Own Kubernetes cluster architecture, scaling, and security hardening across environments
Build and maintain Terraform modules and GitOps pipelines so infrastructure changes are reviewed, versioned, and repeatable
Design for the reliability and auditability bar that regulated financial institutions require: replication, failover, backups, disaster recovery, and change logs
Diagnose and resolve distributed-systems failure modes - network partitions, cascading failures, split-brain scenarios, race conditions - under production pressure
Build observability (metrics, distributed tracing, logging, alerting) so the team can see problems before customers do
Partner with ML, platform, and security engineering on capacity planning, data consistency guarantees, and architecture reviews
Participate in an on-call rotation for production infrastructure
Mentor other engineers on distributed systems design and review their architecture and IaC changes
What We're Looking For:
5+ years in infrastructure, platform, or backend engineering roles, with substantial hands-on distributed systems experience
Strong understanding of distributed systems fundamentals: consensus, replication, partitioning, consistency models (CAP/PACELC), and failure modes
Production experience with at least one major cloud provider (AWS, GCP, or Azure) at scale
Hands-on experience running Kubernetes in production, including cluster upgrades, networking, and security policy
Experience designing or operating distributed data stores, message queues, or event streaming systems (e.g. Kafka, etcd, Cassandra, CockroachDB)
Strong background in infrastructure-as-code (Terraform preferred) and CI/CD pipelines
Proficiency scripting/programming in at least one language (Go, Python, or Rust) for building distributed tooling
Experience owning production reliability: on-call, incident response, postmortems
Comfortable working onsite and collaborating closely with product, ML, and security teams
