You will:

• Design, build, and optimize scalable data processing pipelines using Spark, Airflow, and related big data technologies to support both batch and real-time use cases.

• Lead and collaborate with technical, application, and security stakeholders to deliver reliable, secure Big Data infrastructure leveraging tools and platforms such as Spark, Dataproc, Redpanda, and Temporal.

• Own and participate in on-call responsibilities for Big Data platforms, triaging and resolving incidents, responding to tickets, and ensuring systems consistently meet defined SLAs for availability, performance, and data quality.

• Design, develop, and automate large-scale infrastructure on Kubernetes using Terraform and other infrastructure-as-code patterns to support highperformance data processing, analytics, and AI/ML workloads.

• Define and implement monitoring, alerting, and runbooks to provide endtoend observability and drive continuous improvement in reliability and operational excellence using tools like Grafana.

• Onboard, train, and mentor vendor teams and external partners so they can effectively support, operate, and extend the solutions owned by the Big Data Infrastructure (BDI) team.

• Drive the technical roadmap for emerging data and infrastructure technologies by evaluating options, building proofs of concept (POCs), and authoring solution selection and design documents.

• Champion engineering best practices (code reviews, testing strategies, CI/CD, security and compliance standards) across the Big Data ecosystem to ensure maintainable, resilient, and costefficient systems.

Your team will:

• Build and operate the core Big Data Infrastructure (BDI) platform that powers largescale data processing, analytics, and AI/ML workloads across LiveRamp using tools like Dataproc, Redpanda, Airflow and Temporal.

• Provide secure, self-service, and costefficient data environments that enable product and data teams to experiment, ship, and scale data collaboration applications quickly.

• Partner closely with application, security, SRE, and platform teams to ensure data systems meet LiveRamps standards for privacy, compliance, reliability, and performance.

• Evolve common patterns, tooling, and reference architectures that reduce friction for teams adopting new big data, streaming, and AI/ML capabilities.

• Continuously improve platform reliability and developer experience by automating operations, simplifying workflows, and incorporating feedback from internal customers.

About you:

• 7+ years of experience in software engineering or data engineering, with a focus on large-scale data processing and distributed systems.

• Hands-on experience building and operating pipelines with Apache Spark (batch and/or streaming) and at least one orchestration framework such as Airflow or Temporal.

• Practical experience running Big Data workloads on cloud platforms (for example, using Dataproc or similar managed compute services).

• Proficiency with Kubernetes and infrastructure-as-code tools such as Terraform to provision, configure, and manage production services.

• Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns.

• Solid understanding of systems design, scalability, reliability, and observability for data-intensive workloads.

• Experience participating in or leading on-call rotations and incident management processes for production systems.

• Strong collaboration and communication skills, including working with cross-functional partners (security, application owners, vendors) and documenting designs and decisions clearly.

• Ability to evaluate new technologies, build POCs, and make data-informed recommendations that influence team and platform roadmaps.

• Bachelors degree in Computer Science, Engineering, Mathematics, or a related technical field, or equivalent practical experience.

Preferred Skills:

• Experience working on Big Data Infrastructure, data platform, or core infrastructure teams in a cloud-native environment.

• 3 years of Software Development experience in one of the languages - Python, Go, Java or Scala.

• 3 years of experience with Linux Operating Systems

• 3 years of experience in setting up, managing and optimizing Big Data ecosystems such as Kafka , airflow, Redpanda and temporal.

• 3 years of experience using cloud-based platforms such as AWS, GCP, Azure or similar.

• 3 years of experience with a Version Control System (GIT, Subversion, or similar)

• Experience in using AI tools, NLP and agent creation

• Exposure to security and compliance considerations for data platforms (for example, access control, encryption, data governance, and auditing).

• Prior experience mentoring engineers, leading technical initiatives, or coordinating work with vendors and external partners.

The approximate annual base compensation range is $130,00 to $196,500. The actual offer, reflecting the total compensation package and benefits, will be determined by a number of factors including the applicants experience, knowledge, skills, and abilities, geography, as well as internal equity among our team.

• People: Work with talented, collaborative, and friendly people who love what they do.
• Fun: We host in-person and virtual events such as game nights, happy hours, camping trips, and sports leagues.
• Work/Life Harmony: Flexible paid time off, paid holidays, options for working from home, and paid parental leave.
• Comprehensive Benefits Package: LiveRamp offers a comprehensive benefits package designed to help you be your best self in your personal and professional lives. Our benefits package offers medical, dental, vision, life and disability, an employee assistance program, voluntary benefits as well as perks programs for your healthy lifestyle, career growth and more.
• Savings: Our 401K matching plan1:1 match up to 6% of salary1 helps you plan ahead.