Specialist - Data Engineering

LTM

• $80K — $110K *
Mississauga, ON L4T 0A1In-Person
Information Technology
15+ years of experience
Reposted Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 6-8 years of experience in Big Data and Data Engineering roles
  • Hands-on experience with PySpark and Apache Spark
  • Solid understanding of SQL and relational database concepts
  • Experience in Linux/UNIX environments
  • Familiarity with the Hadoop ecosystem including Hive and HDFS
  • Exposure to cloud platforms such as Azure Databricks, ADF, or Synapse

Responsibilities

  • Design, develop, and maintain data pipelines using PySpark
  • Perform data transformations and validations with Spark SQL DataFrames
  • Write and optimize SQL queries for data extraction and reporting
  • Conduct data quality checks and ensure consistency
  • Support existing data pipelines, troubleshooting issues and performing root cause analysis
  • Collaborate with cross-functional teams in Agile environments
  • Document technical workflows, standards, and procedures

Benefits

  • Collaborative work environment
  • Opportunities for professional development and growth
  • Engagement with cutting-edge technologies
  • Supportive team culture focused on Agile practices
  • Flexibility in adapting to dynamic project needs
Full Job Description
Role description

Job Title: Big data Developer

Work Location : Mississauga-Ontario-CA

Job Summary

  • Design develop and maintain data pipelines using PySpark for largescale data processing
  • Perform data transformations aggregations and validations using Spark SQL DataFrames
  • Write and optimize SQL queries for data extraction reconciliation and reporting
  • Perform data quality checks profiling and consistency validations
  • Support existing data pipelines by troubleshooting failures and performing root cause analysis
  • Work closely with data engineers analysts QA and DevOps teams in Agile delivery models
  • Optimize performance of Spark jobs for efficiency and scalability
  • Maintain technical documentation workflows and runbooks
  • Follow enterprise standards for coding deployment and change management26 years of experience in Big Data Data Engineering roles
  • Strong Handson experience with PySpark Apache Spark
  • Solid understanding of SQL and relational database concepts
  • Experience working in Linux UNIX environments
  • Understanding of ETL concepts and largescale data processing
  • Strong analytical and debugging skills
  • Experience supporting production data pipelines Exposure to Azure Databricks ADF Synapse or similar cloud data platforms
  • Experience with Hadoop ecosystem Hive HDFS
  • Knowledge of orchestration tools eg Airflow
  • Python scripting beyond Spark use cases
  • Familiarity with Agile DevOps delivery practices
  • Domain exposure eg Banking BFSI Analytics is an advantage Strong problem-solving and ownership mindset
  • Ability to work collaboratively in Cross functional teams
  • Good communication and documentation skills
  • Ability to adapt quickly in dynamic project environments
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at LTM

More Information Technology Jobs

Find similar Specialist - Data Engineering jobs:

NationwideMississauga, ON