Job SummaryWe are looking for a proactive and technically skilled Data Lake Studio Manager to oversee and expand our Data Lake Studio workspace. This platform allows our customers-data engineers, analysts, and business teams-to easily transfer data across diverse storage systems and coordinate intricate data pipelines from start to finish.
In this role, you will be responsible for the operational success, feature adoption, and reliability of our Studio platform. You will ensure users can easily set up connectors for cross-storage data transfers. You will build and schedule automated ETL/ELT workflows, monitor job execution, and provide transparency into data movement across cloud, on-premises, and object storage systems.
Key Responsibilities- Data Transfer & Storage Coordination
- Direct the Studio's data movement suite, maintaining high efficiency, reduced latency, and consistent reliability for batch and streaming transfers across multiple storage platforms (e.g., S3, ADLS, GCS, HDFS, relational databases, and data warehouses).
- Coordinate connector connectivity, throughput efficiency, error-handling mechanisms, and schema evolution during cross-storage data migrations and replication tasks.
- Drive self-service ingestion frameworks, allowing customers to easily register new data sources and targets without manual infrastructure setup.
2. Workflow Orchestration & Pipeline Management- Take responsibility for the operational health and user experience of the pipeline orchestration engine within the Studio (e.g., DAG-based scheduling, event-driven triggers, dependency management).
- Define standards and guidelines for customer pipeline development, ensuring fault tolerance, automatic retry policies, and backfilling capabilities.
- Maintain pipeline observability, providing customers with comprehensive monitoring dashboards, real-time logging, and configurable alerting for job failures or bottlenecks.
3. Customer Enablement & Platform Operations- Act as the functional owner and advocate for Studio users, streamlining onboarding, documentation, template creation, and self-service workflows.
- Monitor compute and network costs associated with large-scale data transfers and processing workloads, establishing prioritization quotas and cost-optimization practices.
- Troubleshoot customer partner concern issues related to pipeline execution failures, lockouts, or network efficiency bottlenecks between storage tiers.
4. Data Governance & Security Oversight- Ensure all cross-storage data movements align with organizational security policies, encryption standards (in-transit and at-rest), and compliance frameworks (e.g., GDPR, SOC 2, HIPAA).
- Coordinate identity management, granular credential mapping, and Role-Based Access Control (RBAC) across external storage endpoints and Studio pipeline tasks.
Qualifications & RequirementsEducation & Experience
- Education: Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field, or equivalent experience.
- Experience: 5+ years of hands-on experience in Data Engineering, Data Platform Management, or Cloud Data Operations, with 2+ years in a team lead, product owner, or manager capacity.
Technical Skills- Data Movement & Storage: Demonstrated proficiency with various storage architectures (S3, ADLS, GCS, HDFS, SFTP) and tools/frameworks for high-throughput data transfer.
- Orchestration Frameworks: Extensive hands-on experience with enterprise orchestration tools (e.g., Apache Airflow, Prefect, Dagster, Temporal, or cloud-native workflow managers).
- ETL / ELT & Processing: Experience with distributed computing frameworks (e.g., Apache Spark, Apache Flink).
- Storage & Catalog Protocols: Knowledge of open table formats (e.g., Apache Iceberg, Delta Lake, Apache Hudi) and metadata catalog integration.
- Scripting & Querying: Strong proficiency in SQL and Python.
Leadership & Operations- Strong track record of improving developer/user experience for self-service analytics platforms.
- Excellent communication skills to interface effectively with both software engineers and business collaborators.
- Hands-on experience with SLAs/SLOs, monitoring tooling (e.g., Prometheus, Grafana, Datadog), and DataOps methodologies.
Additional DetailsThe base pay range for this position is expected in the range below:
$136,000 - $228,600
Base pay offered may vary depending on multiple individualized factors, including location, skills, and experience. The total compensation package for this position may also include other elements, including a target bonus and restricted stock units (as applicable) in addition to a full range of medical, financial, and/or other benefits (including 401(k) eligibility and various paid time off benefits, such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.
If hired, employees will be in an "at-will position" and the Company reserves the right to modify base salary (as well as any other discretionary payment or compensation program) at any time, including for reasons related to individual performance, Company or individual department/team performance, and market factors.
Remote roles are not eligible for U.S. visa sponsorship.
