USM Business Systems

JB059543 - Sr kafka Engineer

USM Business Systems • $130K — $180K *
Irvine, CA 92620In-Person
Information Technology
8 - 10 years of experience
More than 3 months ago
Job Overview by Ladders

Qualifications

  • 8+ years in platform engineering with 3+ years of hands-on experience with Apache Kafka.
  • Expertise with Confluent Platform components like Brokers and Schema Registry.
  • Experience deploying and managing Kafka on AWS (MSK or EC2 setups).
  • Solid understanding of Kubernetes, particularly EKS.
  • Proficiency in monitoring and alerting tools (Prometheus, Grafana, ELK, etc.).
  • Experience troubleshooting Kafka integration issues and custom client applications in Java, .NET, Python, Spring Boot.
  • Familiarity with Docker and Helm.
  • Kafka or Kubernetes certifications are a plus.

Responsibilities

  • Manage and enhance existing Apache Kafka and Confluent Platform on AWS.
  • Review existing implementations and recommend improvements.
  • Collaborate with internal teams to address user requirements and implement solutions.
  • Integrate new use cases and define streaming patterns with engineering teams.
  • Monitor and tune Kafka clusters using Confluent Control Center, Prometheus, and Grafana.
  • Automate Kafka infrastructure deployment using Kubernetes and Terraform.
  • Enforce best practices for Kafka usage and administration.

Benefits

  • Opportunities for professional development and certifications.
  • Flexible working environment with potential remote work options.
  • Access to cutting-edge technology stack and tools.
  • Collaboration with a skilled and diverse team.
Full Job Description
  • Start Date: Interview Types
  • Skills Kafka, Confluent Kaf.. Visa Types H1B, Green Card, US ..


    • We are seeking a Senior Kafka Engineer to manage, enhance, and scale an enterprise-grade Apache Kafka implementation deployed on AWS and the Confluent Platform . This person will be responsible for keeping the system reliable, improving it over time, and expanding it to support new applications.

    This role involves performing detailed architectural reviews, monitoring, performance tuning, optimizing existing Kafka pipelines, and partnering with application teams to deliver reliable, secure, and performant streaming solutions.

    Qualifications:

    • 8+ years in platform engineering with 3+ years of hands-on experience with Apache Kafka.
    • Expertise with Confluent Platform (Brokers, Schema Registry, Control Center, ksqlDB).
    • Experience deploying and managing Kafka on AWS (including MSK or self-managed EC2-based setups).
    • Solid understanding of Kubernetes, especially EKS, for microservices integration.
    • Familiarity with monitoring and alerting stacks: Prometheus, Grafana, ELK, or similar.
    • Management and support of Kafka Cloud and on-premise platforms.
    • Capacity management.
    • Good understanding of SRE principles and methodologies, and experience learning, adapting to, and automating new developments in the Kafka ecosystem.
    • Experience troubleshooting integration platform issues (e.g., connectivity, schema management, producer/consumer, etc.) for Kafka, API gateway, etc.
    • In-depth understanding of the Kafka producer and consumer client functionality.
    • Experience troubleshooting custom Kafka client applications written in Java, .NET, Python, and Spring Boot.
    • Hands-on experience with Kafka Connect, Kafka Streams, and Kafka Schema Registry components, understanding its underlying functionality and implementation.
    • Preferably, at least 3 years' experience working in AWS, specifically EKS, EC2, IAM, Route53 and Terraform.
    • Proficiency in deploying, scaling, and managing Kubernetes clusters, with a strong understanding of security best practices.
    • Familiarity with Docker and Helm.
    • Kafka, Kubernetes, Docker, or any Cloud certification.


    Responsibilities:

    • Manage and enhance existing Apache Kafka and Confluent Platform on AWS.
    • Review existing implementations and recommend improvements.
    • Collaborate with internal teams and respective stakeholders to understand user requirements and implement technical solutions.
    • Collaborate with engineering and product teams to integrate new use cases and define scalable streaming patterns.
    • Implement and maintain Kafka producers/consumers, Connectors, and Kafka Streams applications.
    • Enforce governance around topic design, schema evolution, partitioning, and data retention.
    • Monitor, troubleshoot, and tune Kafka clusters using Confluent Control Center, Prometheus, and Grafana.
    • Use Kubernetes and Terraform to automate Kafka infrastructure deployment and scaling.
    • Ensure high availability, security, and disaster recovery including participation in all DR exercises
    • Responsibility and ownership for lifecycle management which includes upgrades, maintenance, restart, and migration projects.
    • Analyze all platforms and ensure the environments are right-sized (i.e., capacity management), along with managing configurations, monitors, and alerts.
    • Create guidelines, procedures, standards, conventions, and best practices for Kafka usage and administration.
    • Develop, maintain, and troubleshoot Terraform IaC modules.
    • Automate repetitive tasks using Terraform scripts.
    • Guide and mentor team members on Terraform implementations.
    • Partner with team members deploying releases to production, and support solution teams
    • Coordinate with the Compute, Database, and other infrastructure teams to support weekend patching activities.
    • Coordinate vendor support.
    • Broad support with troubleshooting and resolving issues, and to be an escalation point for on-premise and cloud integration platform incidents.
    • Remediate security vulnerabilities reported by VMAST across application runtimes, integration services, and messaging platforms.
    • Support production incidents and outages for faster service restoration and provide required support to ensure application connectivity to API services.
    • Work on service requests submitted by application teams requesting application integration team services to integration products (Kafka and any other messaging platforms).
    • Create defects when services like tenant onboarding, resource provisioning, and other customer products are not functioning as per the documentation or working as intended.
    • Analyze SN INCs and REQs to identify trends and opportunities for improvement.
    • Consider implementing automation for manual and repetitive operations support functions.
    • Troubleshoot operational issues related to the AWS infrastructure.

    About USM Business Systems

    USM is a company which develops game apps for iPhone

    USM Careers

    Joining USM presents an unparalleled opportunity to become part of a leading team of professionals dedicated to pioneering innovations in their industry. USM stands as a beacon of excellence, offering a plethora of job opportunities that cater to a diverse range of skills and professional interests.

    Explore Career Opportunities

    USM is actively seeking talented individuals for various positions across the company. With a commitment to diversity and leadership development, USM provides an environment where every employee can thrive. The company values the growth of its team members and supports this with comprehensive benefits and professional development opportunities.

    Innovation and Leadership at USM

    At the core of USM's success is a culture steeped in innovation and leadership. The company is renowned for its dynamic approach to challenges and its ability to lead market trends. Professionals at USM are encouraged to lead projects that stretch their capabilities and foster significant industry advancements.

    USM's Commitment to Professional Growth

    Career advancement is a cornerstone of USM's ethos. The company offers extensive training programs, including leadership and diversity training, to ensure that all team members have the tools they need to succeed. USM's commitment to professional growth is evident in its robust internal promotion and career development pathways.

    Internship Programs and Entry-Level Positions

    For those starting their careers, USM provides a range of internship and entry-level job opportunities. These positions are designed to build foundational skills and integrate new professionals into the industry effectively. Interns at USM gain valuable hands-on experience, working alongside seasoned experts and contributing to impactful projects.

    Networking and Career Development

    USM encourages its employees to engage in networking within the professional community, enhancing their career prospects and industry knowledge. The company hosts regular events and seminars, fostering a network of motivated, forward-thinking professionals.

    Applying for a Position at USM

    To apply for a position at USM, candidates are encouraged to submit a resume that highlights relevant experience and skills. The hiring process at USM includes a comprehensive interview to assess fit both for the role and the company culture. This ensures that all team members are aligned with USM's values and goals.

    Stay Connected with USM Jobs and Careers

    Keep up to date with the latest from USM by following their careers page and signing up for job alert emails. These updates provide insights into new job opportunities, company news, and tips for applicants.

    Join USM

    Explore the exciting and rewarding career opportunities at USM. Search open positions that match your skills and interests. USM looks for passionate, curious, creative, and solution-driven team players.

    SEARCH USM JOBS

    Read Careers Blog

    Stay ahead with career tips, insider perspectives, and industry-leading insights you can put to use today—all from the professionals at USM.

    Job Alert Emails

    Personalize your subscription to receive job alerts, latest news, and insider tips tailored to your preferences. Discover the exciting and rewarding opportunities that await at USM.
    Learn more about USM Business Systems
    Industry
    Staffing
    * Ladders Estimates

    Similar Jobs

    Ad banner background

    Get Ready For Your
    Next Interview

    More Jobs at USM Business Systems

    More Information Technology Jobs

    Find similar JB059543 - Sr kafka Engineer jobs:

    NationwideIrvine, CA