1-3 years experience in data engineering or analytics engineering with a focus on SQL workloads
Strong SQL skills, including window functions, CTEs, and query tuning
Hands-on experience with Snowflake, including cost management and semi-structured data handling
Proficient in Python for data tasks, including libraries like pandas or Polars
Familiarity with Docker for containerizing jobs and managing scheduled tasks
Basic knowledge of Git and CI/CD practices
Keen attention to numerical accuracy and troubleshooting discrepancies
Responsibilities
Design and maintain Python jobs that execute SQL in Snowflake for various datasets
Write efficient and readable SQL for large transaction tables
Package jobs as Docker containers and ensure they are idempotent and observable
Implement data-quality checks to ensure accuracy and consistency
Monitor Snowflake costs and optimize query performance
Collaborate with stakeholders to translate business questions into SQL queries
Apply security controls suitable for a payments environment
Document job libraries and contribute to team code reviews and testing
Explore AI tools to enhance development efficiency
Benefits
Flexible work environment
Opportunities for professional development
Collaborative team culture
Access to cutting-edge technology
Health and wellness programs
Full Job Description
Shift4 is looking for a Data Engineer to join the Shift4 data team and build the SQL and Python based jobs that turn raw payments data in Snowflake into reconciled, reliable datasets and reports. We currently have a library of containerized Python scripts that run SQL against Snowflake on a schedule. You will extend and improve this platform, making it more reliable, scalable and maintainable as we continue to evolve it.
This is a hands-on build role: you write the queries, wrap them in clean, testable Python, package them to run in containers, and make sure they produce correct numbers every time. Pipeline infrastructure (ingestion into Snowflake) is owned by the Data Platform team; you own what happens once the data is there, while embracing the Shift4 way.
Responsibilities
Design, write, and maintain Python jobs that execute SQL over Snowflake to produce transaction, settlement, fee, and reconciliation datasets for internal and merchant-facing consumers
Write performant, readable SQL: window functions, CTEs, incremental logic, and semi-structured (JSON/VARIANT) handling on large transaction tables
Package jobs as Docker containers and run them on the existing scheduler; make every job idempotent, rerunnable, parameterized, and observable (logging, alerting on failure or bad output)
Implement data-quality checks and reconciliation controls to ensure data is accurate and totals consistently align across sources and reporting periods.
Keep an eye on Snowflake cost: warehouse sizing, clustering, query profiling, and avoiding wasteful scans
Work with finance, operations, and product stakeholders to translate payments questions into correct, maintainable queries
Apply security controls appropriate for a payments environment: secrets handling, least-privilege access, PII masking, audit trails
Document the job library and contribute to code review, testing, and CI practices for the team
Proactively explore and apply AI tools to improve efficiency across the development lifecycle (query authoring, testing, documentation)
Qualifications
1 - 3 years experience in data engineering, analytics engineering, or backend roles building SQL-heavy data workloads in production
Strong SQL: window functions, CTEs, aggregation and joins over large tables, query tuning and reading execution plans
Hands-on Snowflake experience: warehouses and credit/cost awareness, stages and COPY INTO, streams and tasks, time travel, VARIANT/semi-structured data; Snowpark or the Snowflake Python connector
Solid Python for data work: pandas or Polars, parameterized SQL, configuration and secrets management, structured logging, error handling, unit tests
Docker fundamentals: building images, running scheduled batch jobs in containers, debugging container failures
Git and basic CI/CD habits; comfortable with code review and writing testable, rerunnable jobs
Attention to numerical correctness: you notice when totals do not tie out and you dig until you know why
Nice to have
Payments domain knowledge: authorization, capture, settlement, refunds, chargebacks, interchange and scheme fees, merchant and acquirer data models, reconciliation
Orchestration and transformation tooling: Airflow, Prefect, Dagster, dbt
AWS from the developer side: S3, IAM, Secrets Manager, ECS/Fargate scheduled tasks
Data-quality frameworks (Great Expectations, dbt tests, Soda) and pipeline monitoring/alerting
Experience in a PCI or otherwise regulated environment
Exposure to BI tools consuming the outputs (Sigma, Tableau, Power BI, Looker)
