Databricks Data Engineer

CGI

$79K — $156K *
Finance & Insurance
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years in Data Engineering, ETL/Data Warehouse, or Big Data roles; 3+ years specifically in Azure Databricks development.
  • Proficient in Databricks, PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS.
  • Experience in designing scalable ETL/ELT pipelines for financial datasets.
  • Strong skills in data transformation, mapping, cleansing, reconciliation, and quality control.
  • Familiarity with credit card/payments domain; understanding of accounts, transactions, and authorizations.
  • Experience with Azure DevOps, Git, CI/CD, and Agile/Scrum methodologies.
  • Strong experience in financial services with knowledge of data controls and reconciliation.

Responsibilities

  • Design and maintain Azure Databricks pipelines for credit card data processing.
  • Develop scalable solutions for managing high volume datasets using PySpark and Python.
  • Create transformation logic for various types of credit card data.
  • Translate business needs into reusable Databricks workflows and ETL/ELT processes.
  • Support modernization of credit card platforms, including data mapping and transformation.
  • Implement controls for data quality, reconciliation, and cleansing across environments.
  • Optimize Databricks pipelines through collaboration with various stakeholders.

Benefits

  • Competitive compensation package.
  • Comprehensive insurance options from day one.
  • 401(k) matching contributions and a share purchase plan.
  • Paid time off including vacation, holidays, and sick leave.
  • Paid parental leave for new parents.
  • Opportunities for learning and tuition assistance.
  • Access to wellness and well-being programs.
Full Job Description
Databricks Data Engineer

Category: Software Development/ Engineering

Main location: United States, Georgia, Atlanta

Position ID:J0826-1837

Employment Type: Full Time

Position Description:

This position is open to candidates located anywhere within the United States and may be performed remotely. Preference will be given to candidates located in or near Atlanta, Georgia, or within commuting distance of a CGI office.

Your future duties and responsibilities:

. Design, develop, and maintain Azure Databricks pipelines for credit card data ingestion, transformation, integration, and downstream consumption.
. Develop scalable processing solutions using PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS for high volume card datasets.
. Build transformation logic for cardholder, account, transaction, authorization, payment, balance, fee, interest, rewards, and servicing data.
. Translate business requirements and source to target mappings into reusable Databricks notebooks, workflows, jobs, and ETL/ELT processes.
. Support credit card platform conversion and modernization, including data mapping, historical/incremental loads, transformation, and reconciliation.
. Implement automated data quality, balancing, reconciliation, cleansing, and exception handling controls across source and target environments.
. Protect sensitive cardholder information, including PAN and PCI sensitive data, through appropriate masking, tokenization, encryption, and data handling practices.
. Troubleshoot and optimize Databricks pipelines while collaborating with credit card SMEs, architects, analysts, and application teams.

Required qualifications to be successful in this role:

. 5+ years of Data Engineering, ETL/Data Warehouse, or Big Data experience, including 3+ years of hands on Azure Databricks development.
. Strong hands on development experience with Databricks, PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS.
. Experience designing and developing scalable ETL/ELT pipelines for large volume transactional and financial datasets.
. Strong experience with data transformation, source to target mapping, cleansing, reconciliation, data quality, and exception handling.
. Credit card/payments domain experience with knowledge of card accounts, transactions, authorizations, payments, balances, billing, and settlement.
. Experience with Azure DevOps, Git, CI/CD, and Agile/Scrum development practices.
. Strong banking/financial services experience and understanding of financial data controls and reconciliation.

Desired Qualifications

. Experience with TSYS or another major credit card processing platform.
. Experience supporting credit card conversions, migrations, or platform modernization.
. Knowledge of the end to end card lifecycle, including authorization, authentication, clearing, settlement, posting, and reconciliation.
. Experience with Mastercard and/or Visa data, interfaces, reporting, rewards, or loyalty programs.
. Understanding of PCI DSS, PAN protection, tokenization/de tokenization, encryption, and sensitive cardholder data management.
. Experience with advanced Databricks capabilities such as Unity Catalog, Workflows, Auto Loader, and Lakeflow/Delta Live Tables.
. Experience modernizing Informatica PowerCenter or SSIS workloads to Databricks/PySpark.
. Azure and/or Databricks certifications and experience with large scale banking conversion initiatives

CGI anticipates accepting applications for this position through September 4, 2026.

This position can be performed remotely from anywhere in the United States.

Other Information:
CGI is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. The determination of this range includes various factors not limited to skill set, level, experience, relevant training, and licensure and certifications. To support the ability to reward for merit based performance, CGI typically does not hire individuals at or near the top of the range for their role. Compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range for this role in the U.S. is $79,600.00 - $156,700.00.

CGI's benefits are offered to eligible professionals on their first day of employment to include:
. Competitive compensation
. Comprehensive insurance options
. Matching contributions through the 401(k) plan and the share purchase plan
. Paid time off for vacation, holidays, and sick time
. Paid parental leave
. Learning opportunities and tuition assistance
. Wellness and Well being programs

#LI-JN1

Skills:
  • Banking
  • Data Warehousing
  • Databricks
  • Financial Services
  • Python

Similar Jobs

More Jobs at CGI

More Finance & Insurance Jobs

Find similar Databricks Data Engineer jobs: