Backend Engineer _ AI Gateway

Woongjin, Inc • $120K — $145K *
Englewood Cliffs, NJ 07632In-Person
Enterprise Technology
Less than 5 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 5-7 years of experience in network programming and async Python.
  • Proficient in managing bidirectional streaming and large-payload handling.
  • Strong knowledge of HTTP/2, HTTP/3, gRPC, and custom protocol parsing.
  • 3+ years of developing REST/Streaming APIs with FastAPI.
  • Experience with containerization using Docker and orchestration on Kubernetes.
  • Proficient in async messaging and event-driven design with tools like Redis and Kafka.
  • Familiar with cloud networking, security practices, and infrastructure as code (IaC).

Responsibilities

  • Design and operate a secure, high-performance AI Gateway.
  • Implement latency-sensitive, policy-driven traffic management between clients and AI services.
  • Handle massive concurrent streaming connections efficiently.
  • Develop and maintain production-grade REST/Streaming APIs using FastAPI.
  • Manage containerized applications and orchestrate deployments on Kubernetes.
  • Create and maintain observability and FinOps practices for monitoring and cost management.
  • Integrate security protocols and IAM solutions for enhanced enterprise security.

Benefits

  • Opportunity to work on cutting-edge AI technologies and frameworks.
  • Potential for hybrid work arrangements after initial on-site assignment.
  • Exposure to large-scale enterprise environments and high-stakes traffic management.
  • Chance to contribute to efficient, high-performance AI operations in a collaborative team.
  • Flexible relocation package to Dallas by October 2026 for the right candidate.
Full Job Description
Job Description

We are building an AI Gateway - a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

What we are looking for
  • Async Python & Network Programming (3 + yrs) - Strong hands-on experience with asyncio, aiohttp/anyio, low-level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection-pooling, keep-alive, and zero-copy buffering.
  • Streaming & Large-Payload Handling - Ability to manage chunked transfer, bidirectional streaming, back-pressure and high-throughput data paths for real-time LLM inference.
  • Protocol Knowledge - Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers.
  • FastAPI-Based Backend (3 + yrs) - Design, develop and test production-grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs.
  • Containerization & Kubernetes - Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service-mesh (e.g., Istio/Envoy) basics.
  • Async Messaging & Event-Driven Design - Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back-off logic.
  • PostgreSQL & Vault - Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management.
  • Cloud Networking & Security - L4/L7 load balancing, TLS termination, reverse-proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let's Encrypt).
  • IaC & CI/CD - Terraform modules for multi-env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue-green deployments.
  • Availability to work on-site in New Jersey for the initial assignment and relocate to Dallas by October 2026


Qualifications
  • Enterprise Security Gateways - Integration with CASB/DLP/WSS or similar edge-security platforms; centralized logging & audit trails.
  • IAM & SSO - Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning.
  • AI/LLM Pipelines - Building prompt-engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
  • API-Gateway Operations - Custom webhook/REST integrations, circuit-breaker patterns, rate-limiting, auto-retry and health-checking.
  • Observability & FinOps - End-to-end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost-optimisation and 24/7 incident response (SLA/SLO).


Additional Information

*****NO C2C******

About Woongjin, Inc

Woongjin, Inc Careers

There has never been a better time to join the global team at Woongjin, Inc—the forefront of innovation and professional growth.

Work You’ll Do

Join Woongjin, Inc's market-leading team to assist some of the most dynamic companies in mastering their business challenges. At Woongjin, Inc, the focus is on transforming businesses through leadership in industries and digital innovation. Lead from a unique position in the marketplace, leveraging technology, industry expertise, and digital innovation to make a significant impact. Engage with a global team of business and technology advisors to help clients navigate through their transformation journeys. Collaborate with a dedicated team of professionals, driving growth and innovation at every step.

Introducing the Woongjin, Inc Business Advisory

The team is dedicated to building a leading Advisory group to guide well-known companies through their digital transformation challenges using comprehensive strategies.

Do Innovative Work

Be part of Woongjin, Inc— a company where innovation meets expertise at the intersection of technology and industry knowledge.

Drive Innovative Solutions

Deliver targeted solutions through a depth of consulting experience and innovation that stands out in the industry.

Be Part of a Great Team

Engage in a wide range of technology projects and harness the capabilities of a team known for its global reach and solution development.

Future-proof Your Career

Woongjin, Inc offers nearly limitless opportunities for career advancement, supported by unmatched training, development, and certification programs.

Explore Job Opportunities and Growth

Woongjin, Inc is committed to employee growth, offering diverse job opportunities, from leadership roles to internships. The company fosters a culture of innovation and diversity, ensuring a vibrant career path for all team members.

The Woongjin, Inc Commitment

With a commitment to diversity and professional growth, Woongjin, Inc helps employees master their skills and achieve their career goals in a supportive environment.

Stay Connected

Join the Team

Search for open positions that match your skills and interests at Woongjin, Inc. The company seeks passionate, curious, creative, and solution-driven team players. SEARCH WOONGJIN JOBS

Keep Up to Date

Stay informed with career tips, insider perspectives, and industry-leading insights you can put to use today—all from the professionals at Woongjin, Inc.

READ CAREERS BLOG

Job Alert Emails

Customize your subscription to receive job alerts, the latest news, and insider tips tailored to your preferences. Discover the exciting and rewarding opportunities that await at Woongjin, Inc.
Learn more about Woongjin, Inc
Industry
Consumer Technology
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Woongjin, Inc

More Enterprise Technology Jobs

Find similar Backend Engineer _ AI Gateway jobs:

NationwideEnglewood Cliffs, NJ