Shift4 is looking for a Data Engineer to join the Shift4 data team and build the SQL and Python based jobs that turn raw payments data in Snowflake into reconciled, reliable datasets and reports. We currently have a library of containerized Python scripts that run SQL against Snowflake on a schedule. You will extend and improve this platform, making it more reliable, scalable and maintainable as we continue to evolve it.
This is a hands-on build role: you write the queries, wrap them in clean, testable Python, package them to run in containers, and make sure they produce correct numbers every time. Pipeline infrastructure (ingestion into Snowflake) is owned by the Data Platform team; you own what happens once the data is there, while embracing the Shift4 way.
Responsibilities- Design, write, and maintain Python jobs that execute SQL over Snowflake to produce transaction, settlement, fee, and reconciliation datasets for internal and merchant-facing consumers
- Write performant, readable SQL: window functions, CTEs, incremental logic, and semi-structured (JSON/VARIANT) handling on large transaction tables
- Package jobs as Docker containers and run them on the existing scheduler; make every job idempotent, rerunnable, parameterized, and observable (logging, alerting on failure or bad output)
- Implement data-quality checks and reconciliation controls to ensure data is accurate and totals consistently align across sources and reporting periods.
- Keep an eye on Snowflake cost: warehouse sizing, clustering, query profiling, and avoiding wasteful scans
- Work with finance, operations, and product stakeholders to translate payments questions into correct, maintainable queries
- Apply security controls appropriate for a payments environment: secrets handling, least-privilege access, PII masking, audit trails
- Document the job library and contribute to code review, testing, and CI practices for the team
- Proactively explore and apply AI tools to improve efficiency across the development lifecycle (query authoring, testing, documentation)
Qualifications - 1 - 3 years experience in data engineering, analytics engineering, or backend roles building SQL-heavy data workloads in production
- Strong SQL: window functions, CTEs, aggregation and joins over large tables, query tuning and reading execution plans
- Hands-on Snowflake experience: warehouses and credit/cost awareness, stages and COPY INTO, streams and tasks, time travel, VARIANT/semi-structured data; Snowpark or the Snowflake Python connector
- Solid Python for data work: pandas or Polars, parameterized SQL, configuration and secrets management, structured logging, error handling, unit tests
- Docker fundamentals: building images, running scheduled batch jobs in containers, debugging container failures
- Git and basic CI/CD habits; comfortable with code review and writing testable, rerunnable jobs
- Attention to numerical correctness: you notice when totals do not tie out and you dig until you know why
Nice to have - Payments domain knowledge: authorization, capture, settlement, refunds, chargebacks, interchange and scheme fees, merchant and acquirer data models, reconciliation
- Orchestration and transformation tooling: Airflow, Prefect, Dagster, dbt
- AWS from the developer side: S3, IAM, Secrets Manager, ECS/Fargate scheduled tasks
- Data-quality frameworks (Great Expectations, dbt tests, Soda) and pipeline monitoring/alerting
- Experience in a PCI or otherwise regulated environment
- Exposure to BI tools consuming the outputs (Sigma, Tableau, Power BI, Looker)
