Senior Big Data EngineerNew York, NYHybrid Schedule (M/F remote, T/W/TH in-office)U.S. RemoteAbout this team: We are seeking a Senior Big Data Engineer to join our Infrastructure Engineering team. This role is responsible for designing, building, maintaining, and modernizing our enterprise big data platforms that support large-scale data processing and analytics across the organization.
The ideal candidate has extensive experience with distributed data platforms such as Kafka, Hadoop/MapR, Aerospike, and Linux-based infrastructure. This individual will work closely with Infrastructure, SRE, Database, and Engineering teams to ensure our platforms are scalable, highly available, secure, and performant.
This is a hands-on engineering role requiring strong operational experience supporting mission-critical production environments while driving platform modernization initiatives.
In this role you will:- Design, deploy, administer, and optimize enterprise Big Data platforms.
- Manage and support distributed technologies including:
- Apache Kafka
- Hadoop / MapR
- Aerospike
- Build highly available, scalable, and resilient data infrastructure across multiple data centers.
- Perform capacity planning, performance tuning, troubleshooting, and root cause analysis for production environments.
- Automate infrastructure deployment, configuration management, and operational workflows using Infrastructure as Code and scripting.
- Collaborate with Software Engineering, Data Engineering, SRE, and Database teams to support new applications and platform requirements.
- Implement monitoring, alerting, and observability solutions to ensure platform health and availability.
- Participate in production support and on-call rotations for critical infrastructure.
We're looking for someone with:- 4+ years of experience supporting large-scale Linux infrastructure.
- 4+ years of experience administering distributed Big Data platforms.
- Strong experience with:
- Apache Kafka
- Hadoop (MapR, Cloudera, Apache Hadoop, or similar)
- Aerospike
- Strong Linux system administration experience (RHEL, Rocky Linux, CentOS).
- Experience with relational databases such as:
- Experience troubleshooting distributed systems in production environments.
- Strong knowledge of networking fundamentals including DNS, TCP/IP, load balancing, and security.
- Experience with shell scripting (Bash, Python, or similar).
- Experience with monitoring platforms such as Grafana, Prometheus, Graphite, or similar.
- Experience working in highly available, 24x7 production environments.
- Ability to work independently in a fast-paced production environment.
- Passion for automation, operational excellence, and continuous improvement.
Preferred Qualifications- Experience with Kubernetes and containerized workloads.
- Experience with Ceph or other distributed storage platforms.
- Experience with Infrastructure as Code tools such as Terraform or Spacelift.
- Experience with configuration management tools such as Puppet or Ansible.
- Experience with cloud platforms (AWS).
- Familiarity with CI/CD pipelines and DevOps practices.
Our Current Tech Stack:Big Data: Apache Kafka, Hadoop / MapR, Aerospike, SparkOperating Systems: Rocky Linux
Databases: PostgreSQL, MySQL, ScyllaDB
Automation: Bash, Python, Terraform, Ansible, PuppetMonitoring: Grafana, Prometheus, GraphiteInfrastructure: Kubernetes, Ceph, Networking (TCP/IP, DNS, Load Balancing)In NY, the base salary range for this role is $170,000 - $180,000.
Actual compensation will be determined based on location, experience, skills, and internal equity.
This role is also eligible for:
- Annual performance-based bonus
- Equity (NASDAQ: MGNI)
- Comprehensive benefits package
During the interview process, candidates will receive more specific information about the salary range applicable to their experience and location.
Perks and Benefits: - Comprehensive Healthcare Coverage from Day One
- Generous Time Off
- Holiday Breaks and Quarterly Wellness Days
- Equity and Employee Stock Purchase Plan
- Family-Focused Benefits and Parental Leave
- 401k Retirement Savings Plan with Employer Match
- Disability and Life Insurance
- Cell Phone Subsidy
- Monthly Fitness and Reimbursement
Career Growth:We support internal mobility, skill development, and leadership pathways through structured career frameworks and learning programs.
