Requisition ID: 19863
Remote Work Available: Yes
Job Title: Senior Data Engineer
Department: Data Engineering
Reports to: Senior Director of Data Platform Engineer
Location: Remote
Responsibilities- Design and Implement Snowflake-Native Data Architectures
- Lead the creation and optimization of modern data architectures on Snowflake, including multi-layer pipelines (bronze/silver/gold), real-time streaming ingestion, and governed analytics layers.
- Ensure that the architecture supports high-volume data workloads, near-real-time freshness requirements, and integrates seamlessly with upstream operational systems and downstream analytics consumers.
- Lead the Development of Complex, End-to-End Data Pipelines Using Native Snowflake Services
- Architect and build data pipelines using Dynamic Tables for declarative SQL-based transformations with automated dependency management and incremental refresh.
- Implement real-time and near-real-time ingestion using Snowpipe and Snowpipe Streaming.
- Design event-driven and procedural pipeline logic using Streams and Tasks for complex orchestration scenarios (MERGE, SCD patterns, external function calls).
- Leverage Snowpark (Python) for advanced transformations that require procedural logic beyond SQL.
- Collaborate with Data Scientists, Analysts, and Other Stakeholders
- Work closely with analytics and data science teams to understand data requirements and translate them into scalable, performant Snowflake solutions.
- Build and maintain Semantic Views and governed consumption layers that provide self-service access to curated data.
- Provide technical guidance on Snowflake best practices for data usage, cost optimization, and warehouse sizing.
- Ensure Data Security, Governance, and Compliance Standards Are Met
- Implement and manage Snowflake data governance features including Dynamic Data Masking, Row Access Policies, Object Tagging, and Data Classification.
- Establish and maintain data governance frameworks ensuring data quality and compliance with relevant regulations (SOX, GDPR, HIPAA).
- Manage role-based access control (RBAC), data sharing, and cross-account governance using Snowflake's native security model.
Skills- Deep Understanding of Data Engineering Concepts
- Extensive knowledge of data modeling, including designing and maintaining relational, dimensional, and semi-structured data models within Snowflake.
- Proficiency in data warehousing concepts with hands-on experience designing multi-layer transformation pipelines (staging, intermediate, marts).
- Strong understanding of incremental processing patterns, change data capture (CDC), and slowly changing dimension (SCD) strategies.
- Expertise in Snowflake Platform and Native Data Pipeline Services
- Deep proficiency with Snowflake Dynamic Tables (TARGET_LAG, REFRESH_MODE, pipeline dependency graphs, incremental vs. full refresh).
- Hands-on experience with Snowpipe and Snowpipe Streaming for continuous and real-time data ingestion.
- Strong knowledge of Snowflake Streams and Tasks for event-driven and procedural pipeline orchestration.
- Experience with Snowpark (Python/Scala) for complex data transformations and UDFs/UDTFs.
- Familiarity with Snowflake Cortex AI functions for embedding AI/ML capabilities into data pipelines.
- Proficiency in SQL, Python, and Data Engineering Frameworks
- Advanced SQL skills including window functions, CTEs, recursive queries, semi-structured data handling (VARIANT, OBJECT, ARRAY), and performance optimization.
- Python proficiency for Snowpark development, automation scripting, and integration work.
- Experience with modular SQL transformation development, testing, and documentation using native Snowflake patterns and shared engineering standards.
- Familiarity with orchestration platforms such as Apache Airflow / Astronomer for pipeline scheduling and monitoring.
- Experience with Real-Time Data Processing and Streaming Architectures
- Hands-on experience designing solutions for real-time and near-real-time data pipelines using Snowpipe Streaming and Kafka connectors.
- Understanding of event-driven architectures and their integration with Snowflake's continuous data pipeline features.
- Experience with change data capture (CDC) patterns and tools (Debezium, Fivetran, custom CDC).
- Strong Knowledge of Cloud Infrastructure and Cost Optimization
- Expertise in cloud platforms (AWS, Azure, or GCP) with a focus on integration with Snowflake (external stages, storage integrations, PrivateLink).
- Experience with Snowflake cost management including warehouse sizing strategies, auto-suspend/resume, resource monitors, and query optimization.
- Familiarity with infrastructure-as-code tools (Terraform, Pulumi) for managing Snowflake resources declaratively.
Contributions- Lead the Technical Design of New Projects
- Responsible for making critical decisions regarding Snowflake architecture patterns, product direction, and delivery tradeoffs, including when to use Dynamic Tables, Streams/Tasks, Materialized Views, and external orchestration based on business requirements and user needs.
- Develop and enforce best practices for pipeline development, testing, deployment, and monitoring within the team.
- Design and document data contracts and SLAs for pipeline freshness (TARGET_LAG), data quality, and availability.
- Mentor and Develop Junior Engineers
- Provide technical leadership and mentorship to junior and mid-level engineers, fostering a culture of continuous learning and improvement.
- Lead code reviews, pair programming sessions, and technical workshops focused on Snowflake-native development patterns and engineering best practices.
- Create reusable patterns, templates, and documentation for common pipeline scenarios (CDC ingestion, SCD management, real-time aggregations).
- Operate as a Product Minded Data Engineering Leader
- Partner with business stakeholders, analytics teams, and engineering leaders to shape the roadmap for data products and platform capabilities based on business value, user needs, and operational priorities.
- Translate ambiguous business problems into clearly defined product requirements, success metrics, delivery plans, and prioritized engineering work.
- Own the lifecycle of key data products and shared platform services, including intake, prioritization, stakeholder alignment, adoption, and continuous improvement.
- Continuously Evaluate and Improve Existing Systems
- Regularly review current data systems to identify opportunities for migration to modern patterns (for example, migrating legacy Streams/Tasks to Dynamic Tables where appropriate).
- Monitor pipeline health, refresh performance, and cost efficiency using Snowflake's INFORMATION_SCHEMA, ACCOUNT_USAGE, and alerting capabilities.
- Implement optimizations including incremental refresh tuning, warehouse right-sizing, and query performance improvements.
Experience- Substantial Professional Experience in Data Engineering
- Typically requires 7+ years of experience in data engineering or related fields.
- Proven track record of designing and implementing large-scale data solutions on Snowflake or similar cloud data platforms.
- 2+ years of hands-on experience with Snowflake-native pipeline services (Dynamic Tables, Streams/Tasks, Snowpipe).
Training & Certifications- Snowflake and Cloud Data Engineering Certifications
- Preferred certifications include SnowPro Advanced: Data Engineer, SnowPro Core, or AWS/Azure/GCP data engineering certifications.
- Continuous learning through relevant certifications and training to stay current with Snowflake platform releases and modern data engineering practices.
- Familiarity with Snowflake's release cycle and ability to evaluate new features (for example, Cortex AI, Document AI, Iceberg Tables) for team adoption.
Salary will be commensurate with experience. A comprehensive benefits package including paid time off is offered with this position.
