Distinguished AI Engineer

Compunnel

• $160K — $200K *
Enterprise Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 10+ years in software engineering and related technical domains
  • Experience in building and supporting production-level platforms
  • Hands-on expertise in LLM platforms and AI orchestration tools
  • Strong foundations in distributed systems and cloud-native architecture
  • Proven ability to evaluate performance and security trade-offs
  • Experience with CI/CD processes and deployment patterns
  • Effective communication with diverse technical and non-technical stakeholders

Responsibilities

  • Lead design evolution of AI platform services
  • Solve complex engineering trade-offs
  • Lead technical spikes and prototype developments
  • Establish engineering expectations for platform performance
  • Create reusable engineering assets and documentation
  • Drive production excellence with observability and incident planning
  • Implement security controls as part of platform capabilities
  • Assess emerging AI technologies for practical adoption
  • Mentor senior engineers and promote AI platform patterns

Benefits

  • Opportunity to influence large-scale enterprise AI solutions
  • Support for professional development and mentoring opportunities
  • Collaborative environment with cross-functional teams
  • Exposure to cutting-edge AI technologies and practices
  • Potential for involvement in high-impact technical initiatives
Full Job Description
div:has([data-free-thinking-preview-answer=true])+:is(.text-message,.relative:has(>.text-message))]:-mt-2 grow">

Job Summary
The Distinguished AI Engineer will provide senior individual-contributor technical leadership for an enterprise AI platform, focusing on solving complex AI platform engineering challenges and developing reusable, secure, reliable, observable, and cost-efficient capabilities. The role will drive the technical evolution of shared AI platform capabilities, including AI gateways, model access and serving, Retrieval-Augmented Generation (RAG), agent and tool execution, orchestration, evaluation, observability, and LLMOps/MLOps. The position will lead through deep technical expertise, engineering assets, and influence rather than direct people management, while supporting enterprise AI and Generative AI solutions at scale.

Key Responsibilities
• Lead the design and engineering evolution of AI platform services for model access, routing, serving, RAG, agents, orchestration, evaluation, and observability.
• Solve complex engineering trade-offs across capability, latency, throughput, resiliency, security, data isolation, portability, and cost.
• Lead critical technical spikes, prototypes, reference implementations, deep design reviews, and resolution of major platform or production issues.
• Establish measurable engineering expectations for availability, recovery, performance, capacity, telemetry, release safety, evaluation coverage, and inference cost.
• Build or sponsor reusable engineering assets, including APIs, SDKs, Terraform/IaC modules, deployment patterns, CI/CD templates, dashboards, evaluation tooling, and developer guidance.
• Drive production excellence through observability, traceability, controlled releases, rollback capabilities, incident learning, capacity planning, and cost optimization.
• Engineer security and AI controls into platform capabilities, including identity and access controls, authorization-aware retrieval, secure tool execution, prompt-injection defenses, data protection, logging, and audit evidence.
• Partner with cybersecurity, risk, compliance, legal, audit, architecture, product, and business teams to translate enterprise requirements into usable technical controls and delivery patterns.
• Assess emerging AI technologies through practical technical evaluations and recommend adoption based on value, maturity, risk, operational fit, and total cost of ownership.
• Mentor senior engineers and drive adoption of enterprise AI platform patterns across teams that do not report directly to the role.

Required Qualifications
• 10+ years of progressive experience in software engineering, distributed systems, cloud/platform engineering, AI/ML infrastructure, or related technical domains, including substantial experience leading complex systems across multiple teams.
• Demonstrated experience building, scaling, transforming, or supporting production platforms or critical technical systems used by multiple engineering teams, products, or business domains.
• Deep hands-on expertise in several of the following areas: LLM platforms, model serving, inference optimization, AI gateways, model routing, RAG, embeddings/vector search, agent frameworks, tool execution, AI orchestration, LLMOps/MLOps, evaluation systems, AI observability, or AI security controls.
• Strong engineering foundation in distributed systems, API and platform design, cloud-native architecture, containers/Kubernetes, networking, IAM, and secrets management.
• Experience designing and engineering reliable, scalable, secure, and production-ready AI platform capabilities.
• Strong technical judgment and ability to evaluate complex trade-offs involving performance, resiliency, security, scalability, portability, and cost.
• Ability to lead complex technical initiatives through architecture, prototyping, implementation, production deployment, and operational support.
• Strong collaboration and communication skills with technical and non-technical stakeholders across cybersecurity, risk, compliance, legal, audit, architecture, product, and business functions.

Similar Jobs

More Jobs at Compunnel

More Enterprise Technology Jobs

Find similar Distinguished AI Engineer jobs: