We are looking for a Senior AWS Data Engineer to design and lead scalable, secure, and high-performance cloud data platforms across AWS/Azure.
" in
Mississauga, ON (Hybrid).Senior AWS Data Engineer (Strong Databricks)Location: Mississauga, ON (Hybrid)
Full Time/CTHExperience: 8+ YearsRole ExpectationsThe Senior Data Engineer will be responsible for hands-on development, implementation, optimization, and maintenance of enterprise-scale data solutions. The role requires strong technical ownership of data pipelines, data processing frameworks, and cloud data platforms, focusing on performance, reliability, data quality, and cost efficiency.
Key Responsibilities- Design, develop, and maintain scalable cloud-based data pipelines and data platforms using AWS and/or Azure.
- Build and optimize data lakes, data warehouses, and lakehouse solutions.
- Implement Medallion Architecture (Bronze, Silver, and Gold layers) for efficient data processing and data quality.
- Develop and maintain robust ETL/ELT pipelines for batch and real-time data processing.
- Build data integration workflows using SQL, Python, Spark, Databricks, Kafka, Airflow, dbt, or equivalent technologies.
- Develop and optimize complex SQL queries, data models, and transformation logic.
- Work with structured, semi-structured, and unstructured data from multiple sources.
- Implement data quality checks, validation frameworks, error handling, and monitoring mechanisms.
- Apply data governance, security, access control, and compliance best practices.
- Optimize data pipeline performance, scalability, reliability, and cloud infrastructure costs.
- Implement CI/CD pipelines, automated testing, and DevOps practices for data engineering solutions.
- Collaborate with Data Architects, Analytics Engineers, Data Scientists, and business stakeholders to deliver reliable data solutions.
- Contribute to technical design documents, data flow diagrams, and implementation plans.
- Troubleshoot production issues and ensure data pipeline availability, accuracy, and timely delivery.
- Mentor junior Data Engineers and promote engineering best practices.
Must-Have Skills- 8-10+ years of experience in Data Engineering, with strong hands-on experience building production-grade data pipelines.
- Strong hands-on experience with AWS and/or Azure data services.
- Expertise in ETL/ELT development, data transformation, and data pipeline orchestration.
- Strong knowledge of Data Lakes, Lakehouses, Data Warehousing, and Medallion Architecture.
- Advanced SQL skills and proficiency in Python or equivalent programming languages.
- Hands-on experience with Apache Spark and/or Databricks.
- Experience with Kafka, Airflow, dbt, or equivalent streaming and orchestration technologies.
- Experience processing structured, semi-structured, and unstructured data.
- Strong understanding of data modeling, data quality, and data validation techniques.
- Knowledge of data governance, security, access management, and compliance practices.
- Experience with Git, CI/CD, automated testing, and DevOps practices.
- Strong analytical, problem-solving, communication, and collaboration skills.
Good to Have- AWS or Azure certifications.
- Databricks certification or hands-on experience with the Databricks platform.
- Experience with Snowflake or other modern cloud data warehouses.
- Experience with real-time data processing and streaming architectures.
- Familiarity with Infrastructure as Code (Terraform, CloudFormation, or Bicep).
- Knowledge of data observability, monitoring, and data lineage tools.
- Exposure to Data Mesh concepts and distributed data architectures
