ResponsibilitiesPeraton is seeking an experienced Senior Kubernetes Engineer to support development, modernization, and migration of containerized and agentic workloads in a government multi-cloud environment with 70+ customer tenants and growing.
The applicant must have extensive engineering experience and deep Kubernetes and Azure expertise, including designing and leading implementation of container-platform requirements and architecture on Azure Kubernetes Service (AKS), as well as standing up and operating Agentic Runtime for Kubernetes (ARK) to host AI-agent workloads. The Senior Kubernetes Engineer will serve as a technical escalation point, drive root cause analysis on complex incidents, and provide mentorship and knowledge sharing across engineering teams. A passion for performance, strong desire for quality conformance, and attention to detail are prerequisites for this position.
Location: RemoteWork Hours: This position requires working online from 8:00 AM Eastern to 5:00 PM Eastern
Day to Day Roles and Responsibilities:
Incident Response and Troubleshooting
- Serve as the Tier 3 escalation point for complex incidents and outages across the Azure Kubernetes Service (AKS) and ARK agentic-runtime environment
- Lead investigation and resolution of issues across the Kubernetes stack including AKS clusters, the control plane, CNI networking, ingress, CSI storage, and workload scheduling, as well as the ARK control plane and its Custom Resources (Agents, Models, Teams, Tools, and Queries)
- Drive root cause analysis on high-severity incidents and own implementation of solutions to prevent recurrence
- Manage high-priority incident communications with stakeholders, leadership, and affected tenants
Platform Monitoring and Optimization
- Define and enforce observability standards across tenant clusters using Azure Monitor, Managed Prometheus, Container Insights, Azure Log Analytics, and OpenTelemetry, including the agent-level telemetry emitted by ARK
- Lead proactive identification and mitigation of cluster, node-pool, and workload risks before they impact tenants
- Analyze performance, cost, and capacity trends and architect optimizations to improve stability, efficiency, and security at scale
Kubernetes and ARK Platform Engineering
- Lead management and maintenance of Azure Kubernetes Service (AKS) clusters and the ARK agentic runtime across a 70+ tenant environment
- Deploy, upgrade, and operate ARK on Kubernetes — managing its CRDs, controllers, and model and tool integrations (including MCP servers) to host AI agents declaratively and provider-agnostically
- Ensure all tenant resources are secure, FedRAMP compliant, and continuously optimized, enforcing Kubernetes RBAC, network policy, and Azure workload identity
- Partner with the Architecture team to design solutions aligned with the Azure Well-Architected Framework and Kubernetes best practices, and own and enforce Change Management procedures across the team
Collaboration, Mentorship, and Knowledge Sharing
- Serve as technical lead for incident intake, triage prioritization, and team workload management
- Mentor and train Tier 2 engineers on Kubernetes and agentic-runtime operations; lead knowledge transfer sessions to build team capability
- Drive improvements in team metrics and service delivery in coordination with the Service Delivery Manager and Product Owner
- Maintain comprehensive documentation of incident response procedures, architecture decisions, and lessons learned
Qualifications
Basic Qualifications:
- Bachelors degree and 8 years of experience or a Masters degree and 6 years of experience or an Associates degree and 10 year of experience or a High School diploma/equivalent and 12 years of experience.
- Must be a U.S. Citizen with the ability to obtain and maintain a DHS Public Trust clearance.
- Engineering and systems analysis experience, with a deep hands-on focus on Kubernetes in production
- Extensive hands-on expertise operating Azure Kubernetes Service (AKS) and the broader Kubernetes stack — control plane, CNI networking, ingress, CSI storage, RBAC, Helm, and workload scheduling
- Hands-on experience deploying and operating Agentic Runtime for Kubernetes (ARK) or comparable Kubernetes-native agent or operator platforms built on the CRD-plus-controller pattern (e.g., Argo), including ARK Custom Resources (Agents, Models, Teams, Tools, and Queries) and MCP/A2A integration
- Proven experience leading incident management and root cause analysis in large-scale, multi-tenant government environments, with deep knowledge of FedRAMP compliance and Kubernetes and Azure security best practices
- Strong leadership, communication, and technical documentation skills; ability to communicate effectively with both technical teams and senior stakeholders
Preferred Qualifications:
- Certified Kubernetes Administrator (CKA) or Certified Kubernetes Application Developer (CKAD), plus Microsoft Azure Administrator Associate (AZ-104) or Microsoft Azure Solutions Architect Expert (AZ-305)
- Proficiency with GitOps and Infrastructure as Code (Terraform, Bicep, Helm, and Argo CD or Flux) and CI/CD pipelines (Azure DevOps or GitHub Actions)
- Advanced scripting with Python, PowerShell, or Go for automation and remediation, including use of the ARK Python SDK (ark-sdk)
- Experience integrating LLM providers (Azure OpenAI, Anthropic, Google, or local models) behind an agentic runtime
Target Salary Range$104,000 - $166,000. This represents the typical salary range for this position. Salary is determined by various factors, including but not limited to, the scope and responsibilities of the position, the individual’s experience, education, knowledge, skills, and competencies, as well as geographic location and business and contract considerations. Depending on the position, employees may be eligible for overtime, shift differential, and a discretionary bonus in addition to base pay.
