Comcast

Sr. Platform Engineer, Kubernetes

Comcast$120K — $150K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's degree in computer science or related field, or equivalent experience (typically 7 years in DevOps or Systems Engineering)
  • Expertise in Apache Spark architecture and functionalities
  • Proficiency in Python, Pyspark, Scala/Java, and Linux Scripting (Bash)
  • Strong SQL writing capability
  • Experience with CI/CD tools and GitHub
  • Familiar with observability tools like Prometheus and Grafana
  • Hands-on experience with cloud providers (AWS, Azure, GCP) and container orchestration (Docker, Kubernetes)
  • Strong analytical and debugging skills for distributed systems.

Responsibilities

  • Architect and manage platforms for Spark workloads across Kubernetes clusters and AWS services
  • Package Spark workloads with Docker/Kubernetes and integrate with orchestration systems like Apache Flyte
  • Deploy infrastructure using Terraform/Ansible
  • Troubleshoot job failures, resource issues, and optimize Spark configurations for cost-efficiency
  • Build automation tools in Python, Java, or Scala, and enhance development team productivity
  • Implement systems for monitoring, logging, and alerting with Prometheus and Grafana
  • Collaborate with Data Stewards, Analysts, and Scientists to fulfill data needs.

Benefits

  • Comprehensive health plans
  • 401(k) matching
  • Generous paid time off
  • Professional development opportunities
  • Employee assistance programs
Full Job Description
Job Summary
As a Sr. Platform Engineer, you will be responsible for building, managing, and optimizing the underlying infrastructure and tools that enable efficient, scalable, and reliable execution of large-scale data processing workloads. Designing systems for collecting metrics (Prometheus) and visualizing data (Grafana) to provide deep insights into application/infrastructure performance. This role is a specialized subset of data platform engineering, ensuring the environment where data engineers and data scientists run their Spark jobs is robust and cost-efficient.

Job Description

RESPONSIBILITIES AND DUTIES:
  • Architecting and managing the platforms where Spark runs, such as Kubernetes clusters, or cloud services like AWS (EKS).
  • Packaging Spark workloads (often via Docker/Kubernetes) and integrating them with orchestration systems like Apache Flyte.
  • Deploying Infrastructure via Terraform/Ansible
  • Troubleshooting and resolving job failures, memory/resource issues, and execution anomalies. This includes optimizing Spark configurations to reduce cloud compute and storage costs.
  • Building automation and tools in languages like Python, Java, or Scala, Linux Scripting (Bash) to increase the productivity of development teams.
  • Write medium to complex SQL Queries as needed.
  • Implementing and maintaining systems for monitoring, logging, and alerting (e.g., Prometheus, Grafana) to ensure platform stability and reliability.
  • Develop and optimize the data catalog platform (e.g., Apache Iceberg, Unity Catalog) for authorization, search, and lineage.
  • Automate workflows, monitoring, and incident resolution.
  • Collaborate with Data Stewards, Analysts, and Scientists to address data needs and issues.
  • Promote best practices and assess emerging technologies.
  • Working closely with data engineers, data scientists, and other engineering teams to define requirements, advise on best practices, and ensure successful delivery of data objectives.
  • Engaging with open-source communities (like Apache Spark, Delta Lake, or Apache Iceberg) to discuss technical challenges and contribute improvements.
  • Create and maintain comprehensive documentation for Kubernetes infrastructure, processes, and procedures. Provide training and support to team members as needed.

QUALIFICATIONS:
  • Bachelor's degree in computer science or a related field, or equivalent experience, typically 7 years in a DevOps or Systems Engineering role.
  • Expertise in Apache Spark: Deep understanding of Spark architecture, including RDDs, DataFrames, execution hierarchy, lazy evaluation, shuffling, and fault tolerance.
  • Proficiency in languages used for Spark development and automation, such as Python, Pyspark and Scala/Java.
  • Proficient in Linux Scripting (Bash).
  • Proficient in writing SQL.
  • Experience in CI/CD tools, Github.
  • Experience in setting up and using observability tools like Prometheus, Grafana etc.,
  • Strong knowledge on Networking Protocols (TCP/IP, DNS, Load Balancer etc.,) and hardware components,
  • Automation via Terraform/Ansible
  • Hands-on experience with on-prem and major cloud providers (AWS, Azure, GCP) and container orchestration tools like Docker and Kubernetes.
  • Handson experience setting up IAM, VPC, EC2 etc.,
  • Familiarity with related technologies and formats like Delta Lake, Apache Iceberg, Apache Kafka, Hadoop, and various data storage systems (S3, HDFS, etc.).
  • Hands-on experience with Databricks, Snowflake, Apache Iceberg, Unity Catalog, or similar tools.
  • Solid understanding of data lakes and governance.
  • Experience setting up, maintaining caching layers like Alluxio.
  • Strong analytical skills for debugging complex distributed systems issues.
  • Strong communication and collaboration abilities.


Disclaimer: This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.

Skills
Adobe Spark, Amazon S3, Databricks Platform, Go Programming Language, Kubernetes, Linux, Terraform (Software)

Please visit the benefits summary on our careers site for more details.

Education
Bachelor's Degree

While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.

Certifications (if applicable)

Relevant Work Experience
7-10 Years

About Comcast

FreeWheel manages the economics of content for the enterprise class of entertainment, offering technical infrastructure for revenue rights management and business operations, wrapped in advisory services. They offer advertising sales rights, display, video, mobile ad management, contract management, professional content, ad serving, reporting, analytics, technology, services, advisory services, business operations, programmatic advertising, advertising marketplace, and linear, and online advertising.

Comcast Careers

Joining Comcast means becoming part of a dynamic team committed to shaping the future of media and technology. As a leading company in entertainment, communications, and technology, Comcast offers unparalleled job opportunities designed to foster professional growth and innovation. Work You’ll Do At Comcast, you’ll be part of a culture that cherishes diversity, leadership, and innovation. Our team is at the forefront of the industry, leading changes that redefine how people connect and experience content across the globe. Transform your career with a company that is committed to your professional development. Comcast offers a variety of positions across multiple fields, where your skills will be honed for leadership roles and your career aspirations supported by comprehensive training programs. Join Our Leading Team Be part of a team that values creativity and diversity. Comcast’s commitment to inclusive culture and diversity training ensures that every team member’s voice is heard and valued. This approach not only enhances our work environment but also drives our leadership in the competitive tech industry. Innovate with Us Engage in work that matters with a team of over 100,000 employees worldwide. At Comcast, innovation isn’t just another buzzword—it's at the core of everything we do. From developing groundbreaking technology to reimagining how our customers experience entertainment and communication, your work will influence millions. Internship and Employment Opportunities Start your career with a Comcast internship, where you’ll gain invaluable industry experience, network with professionals, and develop skills that will benefit you throughout your career. For those seeking full-time positions, Comcast’s hiring process is designed to find not just the right skills but the right fit for our culture of growth and innovation. Benefits and Growth Comcast is dedicated to the well-being and advancement of our team members. Enjoy a wealth of benefits that support both your professional and personal life, including health, wellness, and retirement plans. Our employees are encouraged to pursue continuous growth through professional development opportunities and leadership training. Stay Connected Join the Comcast Team Explore job opportunities that match your skills and interests. We are always looking for passionate, curious, and solution-driven team players. Search Comcast jobs today and find out where your career could take you. Keep Up to Date Stay ahead with career tips, insider perspectives, and industry-leading insights you can put to use today—all from the people who work here. Read Careers Blog Job Alert Emails Customize your subscription to receive job alerts, latest news, and insider tips tailored to your preferences. Discover the exciting and rewarding career opportunities that await at Comcast. By joining Comcast, you’re not just finding a job—you’re joining a community of innovators who are passionate about creating extraordinary experiences for our customers and for each other.
Learn more about Comcast
Size
31,000 employees
Market Cap
$149.9 billion
Industry
Net Income
$10.5 billion
Founded
1963
5 Year Trend
+7.6%
Revenue
$103.5 billion
NASDAQ

Similar Jobs

More Jobs at Comcast

More Information Technology Jobs

Find similar Sr. Platform Engineer, Kubernetes jobs: