Data Engineering Team Lead

New York, NY | Full Time

COMPENSATION RANGE: 300,000.00 - 400,000.00

We are looking for a highly skilled Data Engineering Team Lead with a deep background in big data architectures and a proven track record of leading high-performing technical teams. You should be passionate about extreme performance and excited by the challenge of managing massive data streams. Your success will be measured by your ability to scale the engineering organization, design robust data quality frameworks, and deliver the infrastructure required for advanced machine learning feature serving.

This is a full-time position based in Austin, TX, or New York, NY

What will you do at Smadex?

You will lead our core Data Engineering group, reporting directly to the Chief Data Officer. You will be the architect and leader for a collaborative group of engineers responsible for enabling complex data to be available in our real-time bidding platform for Machine Learning models. As the leader, you will oversee the formation and growth of two specialized sub-teams: Data Quality and Feature Engineering. We foster a fast-paced environment where autonomy is valued, and you will provide the platform for your team to solve extreme engineering challenges while growing their careers.

You will lead the strategic expansion of our data capabilities, managing the end-to-end lifecycle of our data pipelines. You will be responsible for building out two distinct sub-teams:

• Data Quality Team: Focused on building automated testing, observability, and validation frameworks to ensure our massive data ingestion is accurate and reliable.
• Feature Engineering Team: Focused on bridging the gap between raw data and ML inference, building the "Feature Store" and Spark pipelines that feed our real-time bidding models.

You will define the technical roadmap, mentor senior and junior engineers, and ensure operational excellence across our distributed systems.

Key Responsibilities

• Organizational Leadership: Lead and scale two specialized sub-teams (Data Quality and Feature Engineering), managing performance, career development, and hiring.
• System Architecture: Design scalable, highly available data pipelines bridging big data storage and real-time machine learning inference.
• Data Quality Frameworks: Establish rigorous standards for data observability, implementing automated monitoring and alerting for complex data ingestion services.
• Data Governance and Compliance: Define and enforce policies for data privacy, security, and lifecycle management, ensuring compliance with relevant regulations and company standards.
• Feature Store Ownership: Oversee the development and optimization of high-throughput features using Scala to aggregate data for our Redis Cluster.
• Pipeline Strategy Coordination: Coordinate with the infrastructure team ensuring the execution of complex data workflows and DAGs using Apache Airflow.
• Operational Excellence: Monitor global deployments and ensure the stability, reliability, and performance of production systems following a "you build it, you run it" philosophy.
• CDO Collaboration: Partner with the CDO to align technical strategies with business objectives and Machine Learning roadmaps.
• Executive Alignment: Collaborate closely with the CTO for technical solutions and strategy, and partner with the CPO on product integration and transversal projects.

What you need to have:

• 6+ years of experience in software or data engineering, with at least 2 years in a formal leadership or management role building high-throughput distributed systems.
• Deep Technical Mastery: Advanced knowledge of Spark and the ability to conduct deep code reviews.
• Big Data Expertise: Proven track record of designing and optimizing complex batch and streaming pipelines using Apache Spark.
• Infrastructure & SQL: Advanced SQL knowledge and experience with cloud environments (AWS), particularly EMR, EC2, Athena, and S3.
• Analytical Leadership: Strong ability to debug complex distributed systems and drive root-cause analysis for production issues.
• Communication: Excellent communication skills in English, with the ability to translate technical complexity for stakeholders.

Extra points for

• Previous experience in the AdTech industry or working with Real-Time Bidding (RTB) ecosystems, ideally Demand Side Platforms (DSP) side.
• Direct experience building Feature Stores or working closely with Data Science teams on the ML lifecycle.
• Experience with containerization and orchestration (Docker, Kubernetes) at scale.
• Master's degree in Computer Science, Engineering, or Applied Mathematics.

Our Tech Stack

• Languages: Python, Scala and Bash.
• Frameworks: Spark.
• Databases: MySQL, Redis.
• Cloud & DevOps: AWS, Jenkins, Docker, Airflow.
• Tools: Git, Jira, Notion.

What We Offer

• Be part of a leading, fast-growing, innovative company shaping the future of mobile advertising.
• Join a highly motivated and young team.
• Possibility of traveling to the Barcelona HQ for collaboration and team-building activities after your first year.
• Great compensation package tailored to the U.S. market.
• Medical, dental, and vision benefits plans.
• Exposure to leading global app publishers and media partners in the digital advertising industry.
• Learning and training opportunities to grow your career.

Please note that we do NOT provide VISA sponsorship for this role. Candidates without a legal permit to work in the location will not be considered.

POSITION TYPE/EXPECTED HOURS OF WORKThis is a Full Time position. Actual schedule and hours may vary.

SUPERVISORY RESPONSIBILITYReports directly to Smadex Chief Data Officer

Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities and activities may change at any time with or without notice.