Associate Data Engineer - DataBricks

Sumitomo Mitsui Financial Group, Inc. • $90K — $120K *
Charlotte, NC 28269In-Person
Finance & Insurance
Less than 5 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 3+ years of experience in data engineering in financial services, preferably in compliance/regulatory contexts.
  • Advanced proficiency in SQL, Python, and Azure Databricks.
  • Hands-on experience with Azure Data Factory and Databricks notebooks.
  • Expertise in data wrangling and analytics for compliance use cases.
  • Familiarity with data governance frameworks and audit processes.
  • Strong communication and collaboration skills.
  • Experience with Git/GitLab and CI/CD tools, like Azure DevOps.
  • Preferred: Certifications in Azure Data Engineering, Databricks, or Financial Compliance Technology.

Responsibilities

  • Design and implement ETL workflows using Azure Data Factory and Databricks.
  • Integrate and normalize data from structured and unstructured sources while ensuring data quality.
  • Develop and optimize secure pipelines for sensitive financial datasets.
  • Automate data ingestion processes for real-time compliance analytics.
  • Implement efficient storage formats and strategies for data retrieval.
  • Utilize Databricks and Delta Lake for distributed processing and versioned data storage.
  • Enforce data retention and purging policies that comply with global regulations.
  • Migrate legacy application logic to modern Azure data environments.
  • Maintain data lineage and audit trails using Azure Purview or other frameworks.
  • Collaborate with governance teams to define data access controls.
  • Engage in Agile development cycles and support production deployments.

Benefits

  • Hybrid work model allowing flexibility between home and office.
  • Opportunity to work on mission-critical projects with complex financial datasets.
  • Collaborative environment with governance and compliance teams.
  • Focus on high-impact, end-to-end data engineering initiatives.
Full Job Description
SUMMARY:

Join SMBC's Compliance Technology team as a Data Engineer specializing in Azure and Databricks. You will architect and build secure, scalable data pipelines and a cloud-native data platform that powers real-time compliance analytics and regulatory reporting across global markets. Work collaboratively as a team player on mission-critical projects involving complex financial datasets, leveraging Azure Data Factory, Databricks, Delta Lake, and Python to transform raw data into trusted, governed insights. This is a high-impact role where a motivated self-starter owns, end-to-end data engineering initiatives, collaborates with governance teams, and influence the architecture of SMBC's next-generation compliance systems.

PRINCIPAL DUTIES & RESPONSIBILITIES:

  • Own the design and implementation of ETL workflows using Azure Data Factory and Databricks for compliance-related data ingestion and transformation.
  • Integrate and normalize data from multiple structured and unstructured sources, ensuring adherence to strict data quality governance framework.
  • Develop secure, auditable, and optimized pipelines for performance and reliability across sensitive financial datasets.
  • Automate ingestion and transformation processes to enable real-time compliance analytics and reporting.
  • Implement efficient storage formats and partitioning strategies for fast loading and retrieval.
  • Utilize Databricks and Delta Lake for distributed processing and versioned data storage.
  • Enforce data retention, archiving, and purging policies aligned with global regulatory requirements.
  • Migrate legacy application logic into modern Azure Databricks/Data Lake and SQL environments.
  • Maintain comprehensive data lineage, metadata, and audit trails using Azure Purview or custom frameworks.
  • Collaborate with governance teams and compliance analysts to define requirements and manage access controls.
  • Participate in Agile development cycles, sprint planning, and compliance reviews.
  • Review code changes and support production deployments.


JOB SPECIFICATIONS:

  • 3+ years of experience in data engineering within financial services, preferably compliance/regulatory.
  • Advanced proficiency in SQL, Python, and Azure Databricks.
  • Hands-on experience with Azure Data Factory, Databricks notebooks, and ADLS.
  • Expertise in data wrangling, modeling, and analytics enablement for compliance use cases.
  • Experience integrating Microsoft ADF and Databricks for complex ETL development.
  • Familiarity with data governance, quality frameworks, and audit processes.
  • Strong communication and collaboration skills.
  • Experience with Git/GitLab and CI/CD tools such as Azure DevOps.
  • Preferred: Certifications in Azure Data Engineering, Databricks, or Financial Compliance Technology.
  • Preferred: Bachelor's or Master's in Computer Science, Data Engineering, or related field

SMBC's employees participate in a Hybrid workforce model that provides employees with an opportunity to work from home, as well as, from an SMBC office. SMBC requires that employees live within a reasonable commuting distance of their office location. Prospective candidates will learn more about their specific hybrid work schedule during their interview process. Hybrid work may not be permitted for certain roles, including, for example, certain FINRA-registered roles for which in-office attendance for the entire workweek is required.

About Sumitomo Mitsui Financial Group, Inc.

Sumitomo Mitsui Financial Group, Inc. Careers

There has never been a more opportune time to join the dynamic team at Sumitomo Mitsui Financial Group, Inc. (SMFG)—a leading force in the financial services industry recognized for its leadership in innovation and diversity.

Explore Job Opportunities

Sumitomo Mitsui Financial Group, Inc. offers a plethora of job opportunities that cater to a variety of skills and interests. The company is renowned for its commitment to professional growth and leadership development, making it an ideal environment for ambitious individuals looking to advance their careers.

Experience Professional Growth

At SMFG, career advancement is not just a possibility but a priority. The company supports its team members with extensive training programs, including leadership development and diversity training, ensuring that every employee has the tools and knowledge necessary to succeed.

Join a Diverse and Inclusive Team

Diversity and inclusion are at the core of the company culture at Sumitomo Mitsui Financial Group, Inc. With a global team that values unique perspectives and fosters a collaborative and inclusive environment, SMFG is a place where everyone can thrive.

Internship Programs

For those starting their career journey, SMFG offers internship programs that provide a robust foundation in the financial sector. Interns gain invaluable experience, working alongside seasoned professionals and engaging in projects that offer real-world applications of their studies.

Benefits and Culture

Sumitomo Mitsui Financial Group, Inc. is dedicated to not only attracting but also retaining top talent by offering competitive benefits that enhance both personal and professional life. The company culture promotes work-life balance, employee well-being, and continuous learning.

Innovative Work Environment

Innovation is a key driver of SMFG’s success. Employees are encouraged to bring forward-thinking ideas to the table and are provided with the resources to transform these ideas into actionable solutions that drive the financial industry forward.

Networking and Career Development

Networking opportunities within SMFG are abundant. Employees are encouraged to connect with colleagues and industry leaders through various platforms and events, enhancing their professional network and opening doors to myriad career opportunities.

Apply for a Position

Sumitomo Mitsui Financial Group, Inc. is actively hiring and looking for talented individuals who are passionate, curious, and driven. Explore open positions that match your skills and interests on the SMFG careers page.

Stay Connected with SMFG Careers

Keep up to date with the latest career tips, industry insights, and company news from Sumitomo Mitsui Financial Group, Inc. Subscribe to receive updates and stay informed about new job openings and employment trends.

Prepare for Your Interview

Aspiring to join SMFG? Prepare your resume to reflect your best self and gear up for the interview process where you can showcase your skills and passion for finance and innovation.

Career Opportunities Await

At Sumitomo Mitsui Financial Group, Inc., the potential for professional development and personal growth is limitless. Discover the exciting and rewarding career opportunities that await at SMFG, where every position contributes to the company’s global success and leadership in the financial industry.
Learn more about Sumitomo Mitsui Financial Group, Inc.
Industry
Finance & Insurance
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Sumitomo Mitsui Financial Group, Inc.

More Finance & Insurance Jobs

Find similar Associate Data Engineer - DataBricks jobs:

NationwideCharlotte, NC