Context
As part of our data-driven strategy, we are looking for an experienced Data Engineer to design, develop, and optimize data pipelines on our Azure platform. You will play a key role in the ingestion, processing, transformation, and delivery of large-scale data through Azure Data Lake, Azure Data Factory, Azure Synapse, and especially Databricks.
Missions/Main Duties
• Design and develop data ingestion pipelines (batch and real-time) from various sources (API, SFTP, databases, etc.)
• Set up and orchestrate data flows using Azure Data Factory or Azure Synapse Pipelines
• Develop complex data processing workflows with Apache Spark through Databricks (PySpark or Scala)
• Ensure data quality, traceability, and governance, particularly through Delta Lake
• Optimize the performance of pipelines and Spark processing jobs
• Collaborate with Data Science, BI, and Architecture teams to ensure platform consistency and scalability
• Participate in the industrialization, documentation, and security of data flows
• Contribute to the evolution of the data architecture and the implementation of DevOps/DataOps best practices (CI/CD, monitoring, testing, etc.)
Profile/Skills
Required Technical Skills:
• Expertise in Microsoft Azure: Data Lake Storage Gen2, Data Factory, Synapse, Key Vault, etc.
• Strong command of Databricks (PySpark / Spark SQL / Delta Lake)
• Solid experience in developing data pipelines
• Strong data modeling skills (structured and semi-structured data)
• Proficiency with Avro, Parquet, and JSON formats
• Knowledge of data security, quality, and governance principles
• Experience with Git, CI/CD (Azure DevOps or GitHub Actions), and pipeline testing
Preferred Skills:
• Scala, advanced SQL
• Knowledge of event-driven architectures (Event Hub, Kafka)
• Understanding of MLOps or experience collaborating with Data Scientists
• Microsoft (DP-203) or Databricks certification is a plus
Required Experience:
• At least 5 years of experience as a Data Engineer
• Hands-on experience with large-scale data projects in an Azure & Databricks environment
Personal Qualities:
• Analytical mindset, rigor, and autonomy
• Ability to work in an agile and collaborative environment
• Strong technical communication skills
Workplace Type
Hybrid
