The Sr. Data Engineer will be responsible for the design and development of data pipelines to support System Integrations as well as data pipelines that power our Analytical Platforms. In collaboration with Product Owners and other technical teams, the Sr. Data Engineer will lead significant enterprise-wide work as part of larger cross-functional teams supporting multiple major project initiatives.
- A candidate will deliver scalable, flexible, and high-quality solutions that conform to Subway's data design and governance strategies, leveraging our cloud data platform built on Snowflake or Databricks along with other modern cloud-based technologies.
- Design and develop data pipelines and integrations that are performant, scalable and flexible.
- Work with project teams to deliver system integration pipelines, ensuring a high degree of reliability and resiliency.
- Work with business users and project teams to provide SME guidance, finalize pipeline requirements and develop level of effort estimates.
- Develop automated testing and deployments scripts to support integrations and pipelines.
- Provide database support by coding utilities, responding to user questions, and resolving problems.
- Create and maintains documentation of the data pipelines including data flow and data lineage documentation.
- Responsible for tier 3 support and assisting our Operations Team as required to provide a great customer experience for our users.
- Responsible for overall data quality as it relates to our Enterprise Data Warehouse and associated Data Marts. Provide data analysis as required to troubleshoot data issues.
- Assist with integration related code reviews and mentorship of junior developers.
- Implement data governance and master data management principles as part of data pipeline development and delivery.
- Additional responsibilities that may apply due to country, location, or department differences.
Skills and Abilities Required - Lakehouse Architecture: Hands-on experience building lakehouse solutions on Databricks (Delta Lake, Unity Catalog) or Snowflake (Iceberg Tables, Horizon Catalog).
- Medallion Architecture: Proven ability to design Bronze/Silver/Gold layers for curated, analytics-ready data.
- Streaming or Batch Processing: Experience implementing Lambda or Kappa architectures using Databricks Structured Streaming / DLT or Snowflake Dynamic Tables / Snowpipe Streaming.
- Semantic Layer: Experience building and managing semantic models using Databricks AI/BI Genie or Unity Catalog Metrics or Snowflake Semantic Views / Cortex Analyst to enable consistent, governed metrics for BI and AI consumption.
- ETL/ELT Orchestration: Building pipelines with Airflow, Databricks Lakeflow, or Snowflake Openflow; familiarity with dbt is a plus.
- Programming: Strong PySpark or advanced SQL skills; Python for data engineering or automation.
- Data Modeling: Dimensional modeling, Data Vault, or schema design for analytical workloads.
- Performance Tuning: Cluster/warehouse sizing, partitioning, clustering keys, Z-ordering, or query optimization.
- Governance & Security: Working knowledge of Unity Catalog (Databricks) or Horizon Catalog (Snowflake) for lineage, access control, or data quality.
- CI/CD & DevOps: Git, Databricks Asset Bundles or Snowflake CLI/Schemachange, automated testing, or deployment pipelines.
- Cloud Ecosystem: AWS (S3, Glue, Kinesis), Azure, or GCP services supporting modern data platforms.
- AI/ML Enablement: Familiarity with Databricks Mosaic AI or Snowflake Cortex for GenAI or ML use cases.
- Collaboration: Strong communication skills to partner with Product Owners, Analysts, or cross-functional engineering teams.
Minimum Education Required Bachelor
Computer or Information Science or related field or equivalent combination of education and experience.
Experience 5-7 years' experience in creating quality data pipelines and system integrations with at least 3 years of experience in a cloud environment.
