Software Engineer (AI Infrastructure / Training / Inference)

SpreeAI

$120K — $160K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Degree in Computer Science, Engineering, or relevant experience.
  • Strong object-oriented programming skills in languages like Python, C++, Java, or Go.
  • Solid understanding of data structures and algorithms.
  • Experience in developing production-grade backend or distributed systems.
  • Familiarity with cloud infrastructure and containerization techniques.

Responsibilities

  • Design and build scalable infrastructure for training and inference workflows.
  • Develop high-performance APIs for AI model serving.
  • Optimize GPU utilization for multimodal workloads.
  • Build distributed systems for large-scale generative models.
  • Enhance the observability and reliability of AI systems.
  • Collaborate with Applied Science teams to implement research systems effectively.
  • Drive improvements in deployment workflows and platform automation.

Benefits

  • Opportunity to work at the intersection of systems engineering and AI.
  • Access to cutting-edge technologies in AI infrastructure development.
  • Collaborative environment with applied scientists to influence AI research outcomes.
  • Focus on performance optimization and cost efficiency in large-scale systems.
Full Job Description
About the Role

We are hiring Software Engineers focused on AI Infrastructure to build the systems that enable frontier multimodal AI to operate reliably at production scale. This role exists because modern generative and vision models require infrastructure beyond traditional backend engineering - including GPU orchestration, large-scale inference systems, performance optimization, and developer platforms that allow applied scientists to move fast without sacrificing reliability or cost efficiency.

You will work on:

  • Scalable model serving and inference pipelines.
  • Distributed GPU infrastructure.
  • Performance and cost optimization.
  • Reliability, observability, and production readiness.

You will operate at the boundary between systems engineering and machine learning - building the "paved roads" that allow advanced AI systems to scale safely and efficiently.

What you'll do

  • Design and build scalable infrastructure supporting training and inference workflows.
  • Develop high-performance APIs and backend services for AI model serving.
  • Optimize GPU utilization, latency, and throughput for multimodal workloads.
  • Build distributed systems supporting large-scale generative models.
  • Improve observability, monitoring, and reliability of AI systems.
  • Partner closely with Applied Science teams to productionize research systems.
  • Drive improvements in deployment workflows, automation, and platform usability.


Qualifications

  • Degree in Computer Science, Engineering, or comparable combination of education and practical experience.
  • Strong object-oriented programming skills (Python, C++, Java, Go, or similar).
  • Strong data structures and algorithms foundations.
  • Experience building production backend or distributed systems.
  • Understanding of cloud infrastructure concepts and containerized systems.

Preferred Qualifications

  • Experience with Kubernetes, Docker, or container orchestration.
  • Familiarity with GPU-based ML workloads or distributed training/inference systems.
  • Experience with model serving frameworks (vLLM, Triton, Ray Serve, or similar).
  • Experience with observability tools and performance debugging.
  • Familiarity with PyTorch or ML workflows.
  • Interest in optimizing systems for efficiency, scalability, and developer velocity.

Similar Jobs

More Jobs at SpreeAI

More Information Technology Jobs

Find similar Software Engineer (AI Infrastructure / Training / Inference) jobs: