Software Engineer - AI Systems (Go)

Software Engineers - AI Systems (Go) to design, build, and operate production AI systems that perform complex, multi-stage work reliably and at scale.

This is a hands-on engineering role focused on a challenging class of problems: long-running AI workflows that call models, tools, and external APIs; maintain state across extended executions; produce structured content and generated media; interact with human reviewers; recover from partial failures; and consistently deliver accurate results to users.

The core engineering challenges extend well beyond prompt development. These systems must account for provider failures and rate limits, interrupted workflows, changing state, non-deterministic model behavior, incorrect or unsupported outputs, variable latency and cost, and deployments occurring while work is in progress.

The successful candidate will combine strong software and distributed-systems engineering judgment with practical experience building and operating LLM-backed applications, AI agents, or agent-based systems in production.

You will join at a stage where significant architecture remains to be designed and built. Engineers in this role will have substantial ownership over the patterns, services, infrastructure, and engineering standards that shape the University's AI systems.

Remote Work Eligibility
This position is eligible for remote work for candidates residing in states where Stanbridge University is currently authorized to employ remote employees. Eligible states currently include: Arizona, Colorado, Illinois, Indiana, Kansas, Kentucky, Louisiana, Maryland, Michigan, Minnesota, Nevada, New Jersey, North Carolina, North Dakota, Ohio, Tennessee, Texas, and Wisconsin.

Candidates must reside in an eligible state at the time of employment. Remote-work eligibility is subject to University employment requirements and may change based on applicable state requirements.
Engineering Environment

The engineering problems addressed by this team include:

• Durable workflow orchestration: Long-running, multi-stage pipelines using persistent state, job queues, checkpoints, resumability, idempotent execution, and recovery from interrupted or orphaned work.
• Multi-provider model infrastructure: Routing across commercial model providers with model registries, token and cost controls, rate-limit handling, retries, circuit breakers, health monitoring, and provider failover.
• Agent and tool orchestration: Systems in which AI agents interact with tools, APIs, data sources, application services, and deterministic business logic to complete multi-step work.
• Prompt engineering infrastructure: Treating prompts as version-controlled production artifacts with review, testing, regression protection, and measurable behavior.
• Testing non-deterministic systems: Recorded and replayable provider interactions, deterministic fixtures, evaluation harnesses, baselines, and regression testing for AI behavior.
• Correctness and quality controls: Structured-output validation, automated evaluation, domain-specific requirements, evidence checking, and safeguards against confident but incorrect model output.
• Human-in-the-loop workflows: Review and approval stages within automated processes, including systems capable of safely responding when users modify state or inputs during execution.
• Generated media pipelines: Systems capable of producing and managing documents, audio, imagery, video, and other generated assets.
• Production service architecture: Go-based APIs and backend services, relational data stores, job infrastructure, observability, and integrations supporting user-facing applications.
• Secure ingestion: Processing user-supplied and potentially untrusted documents while maintaining appropriate security and authorization boundaries.

Two Engineering Emphases

Engineers will meet the same overall technical bar but may bring deeper expertise in one of two areas:
Platform and Pipeline

Focused on distributed systems, workflow orchestration, durable state, job queues, recovery, provider infrastructure, latency, throughput, scalability, reliability, and cost optimization.

This emphasis is particularly well suited for experienced systems engineers who view AI models as powerful components that introduce a new set of distributed-systems and reliability challenges.
Agents and Quality

Focused on agent and tool architecture, prompt systems, context management, evaluation frameworks, regression testing, judge models, evidence validation, and end-to-end output quality.

This emphasis is particularly well suited for engineers who approach AI behavior through experimentation, measurement, testing, and systematic improvement.

Candidates may indicate an area of preference; however, specialization in one area is not required.
Essential Functions

• Design, develop, test, deploy, and operate production-quality software and backend services, primarily using Go (Golang).
• Architect and build AI agents and agent-based systems capable of using tools, interacting with APIs, maintaining state and context, and executing complex multi-step workflows.
• Design durable workflows that can checkpoint, resume, retry, recover, and safely continue execution following partial failures or system interruptions.
• Determine how models, agents, tools, APIs, data sources, services, and deterministic application logic should divide responsibilities within an AI system, including recognizing when an AI model is not the appropriate solution.
• Design integrations across multiple AI model providers, including routing, failover, rate-limit management, health monitoring, and degradation strategies.
• Build safeguards including structured-output validation, error handling, retries, quality gates, evidence validation, and automated recovery mechanisms.
• Develop testing strategies for non-deterministic AI behavior using techniques such as deterministic fixtures, recorded and replayed interactions, regression suites, evaluation harnesses, and quality baselines.
• Develop observability capabilities including tracing, metrics, logging, evaluation data, and replayable execution histories to support production debugging and performance analysis.
• Design human-in-the-loop workflows incorporating review, approval, intervention, and modification of workflow state.
• Build and maintain APIs, backend services, relational data models, job-processing infrastructure, and supporting application components.
• Develop secure methods for ingesting and processing user-supplied documents and other external content.
• Optimize systems for reliability, latency, throughput, scalability, output quality, and cost per execution.
• Build reusable engineering patterns and shared components that simplify the addition of new agents, model providers, tools, workflows, and output types.
• Own technical problems from initial investigation and architecture through implementation, deployment, monitoring, troubleshooting, and ongoing production operation.
• Collaborate directly with product stakeholders and domain experts to translate qualitative requirements into measurable system behavior and technical solutions.
• Contribute to architectural decisions and engineering standards for AI-powered applications across the University.

Qualifications

Required

• Substantial professional experience developing and operating production software, with strong experience in Go (Golang) or demonstrated depth in another backend language with the ability to become productive in Go quickly.
• Hands-on experience shipping an LLM-backed application, AI agent, or agent-based system into production and supporting it after deployment.
• Strong understanding of AI application architecture, including how models, agents, tools, APIs, data sources, services, and application logic interact.
• Demonstrated distributed-systems engineering knowledge, including concurrency, asynchronous processing, queues, idempotency, retries, partial failure, state management, and recovery.
• Experience designing systems that remain reliable when individual services, providers, or workflow stages fail.
• Demonstrated testing discipline for systems involving non-deterministic behavior.
• Strong experience designing and consuming HTTP APIs.
• Experience with relational databases, SQL, and persistent application state.
• Experience building, deploying, monitoring, and troubleshooting backend services in production environments.
• Understanding of software architecture, testing, debugging, observability, and production engineering practices.
• Ability to independently own ambiguous technical problems from investigation through production implementation.
• Strong analytical judgment and the ability to balance reliability, quality, performance, complexity, and cost.

Preferred Qualifications

• Experience with AI agent frameworks, orchestration patterns, or custom agent architectures, including an understanding of when a framework may not be appropriate.
• Experience with retrieval-augmented generation (RAG), embeddings, vector databases, semantic search, or knowledge-retrieval architectures.
• Experience developing evaluation and observability systems for LLM applications, including tracing, regression suites, quality dashboards, or automated evaluation.
• Experience with judge models, structured-output validation, evidence checking, or other AI quality-control mechanisms.
• Experience designing human-in-the-loop workflows involving review, approval, intervention, or modification of active workflow state.
• Experience with document processing, headless-browser rendering, text-to-speech, image generation, video generation, or other media pipelines.
• Experience with event-driven architectures, durable job queues, and asynchronous processing at scale.
• Experience with containers, cloud infrastructure, CI/CD, and production deployment environments.
• Understanding of prompt injection, authorization boundaries, data isolation, and security considerations when untrusted content is processed by AI systems.
• Experience developing systems in environments where the accuracy of generated output carries significant operational, regulatory, compliance, or safety implications.

What Success Looks Like

An exceptional engineer in this role will:

• Build AI workflows that operate reliably in production without requiring routine human intervention.
• Design systems that recover gracefully from model-provider failures, interrupted execution, deployments, rate limits, and other partial failures.
• Make AI behavior increasingly measurable, reproducible, testable, and observable rather than relying on subjective evaluation.
• Improve output quality while systematically reducing latency and cost per execution.
• Create durable engineering patterns that make subsequent agents, providers, workflows, and output types easier and safer to introduce.
• Identify when deterministic software should replace or constrain model-driven behavior.
• Build systems whose failures can be diagnosed through instrumentation and replay rather than guesswork.
• Establish architecture and engineering practices that become foundational components of the University's broader AI capabilities.

Compensation

Compensation is based on education, experience, and qualifications and internal equity
Work Environment

• Standard professional and technology-focused work environment.
• Duties are typically performed while sitting at a desk or computer workstation.
• Position requires extensive interaction with computers, software development environments, cloud services, AI systems, and technical infrastructure.
• Subject to collaboration with cross-functional teams, changing technical requirements, and demanding project timelines.

Physical Demands

• Regularly sits for extended periods.
• Physical ability to perform department-related duties.
• Proficient in using electronic keyboards and office equipment.
• Effective verbal communication via phone, video conferencing, and in person.
• Ability to read fine print, operate com