Job SummaryThe role involves designing, developing, and optimizing data models, ETL pipelines, and scalable data solutions using Azure Data Services, Spark, and Databricks. The position requires strong experience in data engineering, SQL, and cloud-based data platforms, with a focus on supporting business requirements, ensuring data quality, and maintaining high-performance data workflows.
Key Responsibilities- Design, develop, and optimize data models and schemas aligned with business needs.
- Develop, implement, and maintain efficient ETL pipelines using Spark, Databricks, and related tools.
- Write complex SQL queries, stored procedures, and functions for data extraction, transformation, and loading.
- Collaborate with analytics and business teams to gather requirements and deliver data solutions.
- Build scalable data solutions using Azure Data Lake, Azure Data Factory, and Azure Databricks.
- Ensure data quality, governance, and security across all data processes.
- Monitor, optimize, and troubleshoot data pipelines to ensure performance and availability.
- Document data architecture, workflows, and procedures, and participate in code reviews and best-practice development.
Required Qualifications- Overall, 9+ years of experience.
- 6+ years of experience in data engineering with strong expertise in data modeling and SQL.
- Should have recent experience in the banking/financial services/capital markets domain
- Proven experience with ETL development and data pipeline orchestration.
- Hands-on experience with Azure Data Factory, Azure Data Lake, and Azure Databricks.
- Strong knowledge of Spark, Python, or Scala for data processing.
- Familiarity with data governance, security, and compliance best practices.
- Strong problem-solving skills and attention to detail.
- Ability to work independently and collaboratively within a team.
- Bachelor's degree in Computer Science, Information Technology, or a related field.
Preferred Qualifications- Certifications in Azure data services or Databricks.
- Experience with BI tools and reporting.
- Knowledge of other cloud platforms.
Certifications- Certifications in Azure Data Services or Databricks
