3-5 years of experience with Apache Spark and Python.
Strong background in data pipeline development and optimization.
Mandatory DASCA certification for Big Data roles.
Familiarity with business data analytics frameworks and tools.
Experience mentoring team members and promoting best practices.
Responsibilities
Design and develop high-performance data processing applications using Apache Spark.
Write clean and efficient Python code for data engineering tasks.
Analyze complex datasets to extract insights and identify trends.
Collaborate with stakeholders to create tailored data solutions.
Mentor junior team members and share knowledge within the team.
Conduct code reviews to maintain adherence to standards.
Manage and monitor data pipeline deployments for performance.
Benefits
Opportunity to work in a collaborative and innovative environment.
Access to ongoing professional development and training programs.
Potential for career advancement within a growing organization.
Flexible work arrangements to support work-life balance.
Participation in team-building and knowledge-sharing activities.
Full Job Description
Role description
Job Title: Big data Developer Work Location : Mississauga , Canada Job Description:
Seeking a Specialist with 3 to 5 years of experience in Apache Spark and Python to design develop and optimize scalable data processing solutions within the Python Data domain
Develop and maintain scalable and robust data pipelines using Apache Spark and Python Collaborate with data scientists and analysts to enhance and optimize data workflows Ensure data quality integrity and consistency across large and complex datasets Implement efficient data transformations and aggregations to support analytics and reporting Integrate data solutions seamlessly into existing systems by working closely with cross functional teams Monitor and troubleshoot data processing jobs to maintain high reliability and performance Stay updated with emerging trends technologies and best practices in big data and Python development
Roles and Responsibilities
Design develop and optimize high performance data processing applications leveraging Apache Spark Write clean reusable and efficient Python code to support various data engineering initiatives Analyse complex datasets to extract actionable insights and identify trends Collaborate with stakeholders to gather requirements and deliver tailored data solutions that meet business objectives Mentor junior team members and promote knowledge sharing within the team Participate actively in code reviews to ensure adherence to coding standards and best practices Manage deployment of data pipelines and continuously monitor their health and performance in production environments
Mandatory certificate DASCA Big Data Associate Big Data Analyst
