10+ years of software engineering experience with complex production systems
Expertise in backend and distributed systems design
Proven track record as a code and architecture gatekeeper
Strong fundamentals in computer science applied with sound judgment
Deep proficiency in Python and strong TypeScript or full-stack experience
Experience with LLM systems and rapid learning ability
Strong instincts for testing and observability
Familiarity with cloud infrastructure, containers, and CI/CD processes
Excellent communication skills with technical and non-technical stakeholders
Eligibility for US security clearance
Responsibilities
Serve as a quality gate for AI services, ensuring high standards in design and testing
Ensure safe and scalable integration of AI services with backend systems
Act as a trusted code reviewer for the AI team
Establish a scalable review culture with clear architecture guidelines
Prevent architectural drift among parallel engineering efforts
Own scalability targets and identify bottlenecks in systems
Write production code for complex AI-specific backend features
Lead design reviews and drive decisions on infrastructure changes
Make informed build-versus-buy decisions under deployment constraints
Unblock AI team access to necessary data sources and tooling
Mentor engineers and help set the hiring bar
Benefits
Opportunity to work on cutting-edge AI-driven logistics solutions
Collaborative environment with a focus on mentorship and growth
Chance to influence engineering practices and standards
Engagement in high-impact military operations
Access to advanced tools and technologies in AI and backend systems
Full Job Description
About the role
Gallatin AI builds AI-driven logistics planning and decision support for military operations. Our AI team is growing quickly, and we are hiring a Principal AI Engineer to be a technical anchor for it: someone who sets the engineering bar, owns architecture and scalability, and helps the team design, build, review, and ship its own work with high quality, in close partnership with our platform and backend teams.
We are looking for an exceptional engineer with broad range and sound judgment. You have designed systems that other engineers build on, you make the people around you better, and you care as much about how a team builds as what it builds. Deep AI experience is a plus. Deep engineering judgment is the requirement.
What you will do
Serve as a quality gate for our AI services: hold the bar on design, testing, and scalability, and review the changes that matter most before they ship.
Be accountable for how our AI services integrate with our event-driven backend (NATS-based messaging), data models, and APIs, working with platform and backend engineers to keep those integrations safe, scalable, and consistent.
Be a trusted code reviewer for the AI team, including changes into shared backend and data systems
Build a review culture that scales: clear architecture guidelines, common patterns to follow and avoid, design review templates, and risk-based review tiers, including AI-assisted review, so that quality holds as PR volume grows.
Prevent architectural drift. Make sure engineers working in parallel, with or without AI coding assistants, are building toward the same design instead of undoing each other's work.
Own scalability: set latency, throughput, concurrency, and cost targets, find bottlenecks, and fix them. Ask "what happens at 10x?" before things ship.
Write production code on difficult problems, including AI-specific backend features, agent and retrieval services, and data pipelines.
Lead design reviews and RFCs and drive them to decisions. Know when infrastructure or data model changes are safe and how to roll them out without breaking existing systems.
Make build-versus-buy and model-selection calls, including self-hosted versus cloud tradeoffs under data-handling and deployment constraints.
Unblock the AI team's access to data sources, infrastructure, and tooling so they can move on their own.
Mentor engineers, grow future technical leads, and help set the hiring bar.
What you bring
10+ years of software engineering, with a record of designing and scaling complex production systems through multiple orders of magnitude of growth.
Principal-level backend and distributed systems depth: service design, event-driven and message-based architectures, APIs, data modeling, concurrency, caching, and failure handling.
Demonstrated trust as a code and architecture gatekeeper for a team or platform, and a track record of spreading that capability across other engineers.
Strong computer science fundamentals, applied with judgment.
Deep Python fluency, plus strong TypeScript or comparable full-stack experience.
Production experience with, or the demonstrated ability to quickly ramp on, LLM systems such as RAG, agents, tool use, and evaluation.
Strong testing and observability instincts, including reliable tests around nondeterministic components.
Working fluency with cloud infrastructure, containers, and CI/CD. You can ship what you build.
Clear written and verbal communication with engineers, product leaders, and customers.
Eligibility for a US security clearance.
Bonus Points
Prior principal or staff-level experience on a platform other teams build on.
Experience scaling an engineering organization's practices, such as review, design standards, and onboarding, as headcount tripled.
Experience with data platforms, feature stores, or ML pipelines.
Experience with optimization, simulation, or logistics systems.
Experience in constrained, air-gapped, or regulated deployments.
Background in military logistics, planning doctrine, or defense programs.