Data Infrastructure Engineer (Hadoop)New York City, NYHybrid Schedule (M/F remote, T/W/TH in-office)We are seeking a Data Platform Engineer (Hadoop) to join our Infrastructure Engineering team. This role is responsible for designing, building, maintaining, and modernizing our enterprise big data platforms that support large-scale data processing and analytics across the organization.
The ideal candidate has extensive experience with distributed data platforms such as Kafka, Hadoop/MapR, ultra-fast NoSQL databases (Redis, DynamoDB, Aerospike), and Linux-based infrastructure. This individual will work closely with Infrastructure, SRE, Database and Engineering teams to ensure our platforms are scalable, highly available, secure and performant.
This is a hands-on engineering role requiring strong operational experience supporting mission-critical production environments while driving platform modernization initiatives.
In this role you will: Cluster & Infrastructure Administration:- Hadoop Management: Provision, configure, monitor and maintain robust distributed Hadoop clusters (e.g. MapR, Cloudera, Hortonworks)
- Linux Systems Engineering: Manage, optimize, and troubleshoot enterprise Linux servers (RedHat, CentOS, Rocky Linux) hosting the data platform.
- Performance Tuning: Troubleshooting, root cause analysis for production environments.
- High Availability: Maintain cluster backup, disaster recovery, production support and on-call rotations
Cross-Functional DevOps & SRE Collaboration:- Infrastructure as Code (Iac)
- CI/CD Integration
Data Ecosystem & Pipeline Management:- Kafka Event Streaming: Deploy, configure, tune and scale multi-node Kafka clusters, topics, partitions and consumer groups.
- NoSQL Database Management: Administer, schema design, and performance-tune NoSQL databases (such as HBase, Aerospike, Cassandra, or MongoDB)
- Ecosystem Integration: Support and integrate core Hadoop ecosystem tools including Hive, Spark, and HDFS.
We're looking for someone with: - 4+ years of experience administering distributed Big Data Platforms.
- Messaging Systems: Proven experience deploying and managing production grade Kafka infrastructure.
- NoSQL Databases: Practical experience administering at least on major NoSQL database
- Scripting: Proficiency with shell scripting (Bash, Python, or similar).
- Experience with relational databases such as PostgreSQL or MySWL
- Linux Administration: Experience in Linux operating systems, including storage management, networking, security and performance troubleshooting.
- Experience with monitoring platforms such as Grafana, Prometheus, or similar
Preferred Qualifications:- Experience with Kubernetes and containerized workloads
- Experience with Ceph or other distributed storage platforms
- Experience with Infrastructure as Code tools such as Terraform or Spacelift
- Experience with configuration management tools such as Puppet or Ansible
- Experience with cloud platforms (AWS)
In New York, the base salary range for this role is $170,000 - $180,000..
Actual compensation will be determined based on location, experience, skills, and internal equity.
This role is also eligible for:
- Annual performance-based bonus
- Equity (NASDAQ: MGNI)
- Comprehensive benefits package
During the interview process, candidates will receive more specific information about the salary range applicable to their experience and location.
Perks and Benefits: - Comprehensive Healthcare Coverage from Day One
- Generous Time Off
- Holiday Breaks and Quarterly Wellness Days
- Equity and Employee Stock Purchase Plan
- Family-Focused Benefits and Parental Leave
- 401k Retirement Savings Plan with Employer Match
- Disability and Life Insurance
- Cell Phone Subsidy
- Monthly Fitness and Reimbursement
Career Growth:We support internal mobility, skill development, and leadership pathways through structured career frameworks and learning programs.
Magnite's culture is rooted in collaboration and connection, recognizing that both are essential to success. Our hybrid schedule includes two remote workdays (Monday and Friday) and three in-office days (Tuesday through Thursday), where employees can take advantage of catered lunches, beverages, snacks, training opportunities, and engaging office events that enhance the work experience. In addition, we support our team through meaningful benefits and programs, including community service and volunteer opportunities, company-matched charitable contributions, access to mental health resources, structured career development initiatives and growth frameworks, culture and inclusion programs, and our Bonusly peer-to-peer recognition platform.
Recruiting Agency NoticeMagnite does not accept agency resumes and is not responsible for any fees related to unsolicited resumes.
