Azure Data Engineer

Compunnel

• $120K — $145K *
Information Technology
11 - 15 years of experience
Job Overview by Ladders

Qualifications

  • 15+ years of overall professional experience in data engineering and integration
  • 8-10 years in data modeling and governance practices
  • Expertise in Microsoft Azure services, especially Azure Databricks and Azure Data Factory
  • Proficient in developing scalable ETL/ELT pipelines
  • Strong programming skills in Python, SQL, Scala, and Spark notebooks
  • Experience with relational databases like Microsoft SQL Server and Oracle
  • Understanding of big data solutions and architectures

Responsibilities

  • Design and optimize ETL/ELT pipelines for various data processing needs
  • Implement data solutions using Azure Databricks and other Azure services
  • Establish enterprise data architecture and storage mechanisms
  • Develop data processing solutions with Spark and programming languages
  • Design data models for analytics and reporting workflows
  • Support data migration and system modernization initiatives
  • Collaborate with stakeholders to define and deliver data solutions

Benefits

  • Opportunities for professional development and advanced training
  • Access to cutting-edge cloud technologies and platforms
  • Collaborative work environment with cross-functional teams
  • Exposure to complex enterprise-level data projects
  • Potential for flexible working arrangements
Full Job Description
Job Summary Azure Data Engineer responsible for designing, developing, and supporting enterprise-scale data engineering, migration, modernization, and integration initiatives on Microsoft Azure. The role requires strong expertise in Azure Databricks, Azure Data Factory, ADLS, Azure Synapse, Azure SQL Database, Azure Functions, Logic Apps, Spark, Python, SQL, and enterprise data architecture. The position will focus on building batch and real-time data pipelines, data modeling, data governance, data quality, and integration across complex data environments. Key Responsibilities • Design, develop, and optimize scalable ETL/ELT pipelines and data flows using Azure Data Factory for batch, real-time, and near-real-time data processing. • Develop and implement data solutions using Azure Databricks, Delta Lake, ADLS, Azure Synapse, Azure SQL Database, Azure Functions, Logic Apps, Event Hubs, Web Apps, and API Management. • Design and implement enterprise data architecture, data sourcing, ingestion, curation, transformation, and storage mechanisms. • Develop Spark-based data processing solutions using Python, SQL, Scala, and Spark notebooks. • Design and implement data models supporting enterprise reporting, analytics, data warehousing, and operational data requirements. • Support data migration, system modernization, and integration initiatives across complex enterprise environments. • Develop and maintain real-time and near-real-time data warehousing solutions and data processing architectures. • Work with Microsoft SQL Server and Oracle databases to develop, optimize, and maintain data solutions. • Analyze and reverse-engineer complex legacy implementations, including Oracle PL/SQL logic and database behavior. • Apply data management principles across metadata management, master data management, data governance, and data quality initiatives. • Support enterprise data integration and migration activities, including migration to Salesforce where applicable. • Collaborate with technical and business stakeholders to understand data requirements and translate them into scalable data engineering solutions. • Troubleshoot data pipelines, integrations, database issues, and production data processing challenges. Required Qualifications • 15+ years of overall professional experience with strong experience in enterprise data migration, system modernization, and integration initiatives. • 8-10 years of experience in data modeling, data/information architecture, metadata management, master data management, data governance, and data quality. • Strong hands-on experience with Microsoft Azure data services, including Azure Databricks, ADLS, Azure Data Factory, Azure Synapse, and Azure SQL Database. • Strong experience with Azure Databricks and its components, including Delta Lake. • Strong experience designing and developing ETL/ELT pipelines and data flows using Azure Data Factory. • Hands-on experience with Azure Functions, Logic Apps, Event Hubs, Web Apps, and API Management. • Strong programming experience with Python, SQL, Spark notebooks, and Scala. • Strong development and design experience with relational database environments, including Microsoft SQL Server and Oracle. • Strong understanding of data architecture, data management, data sourcing, ingestion, curation, transformation, and storage mechanisms. • Experience with big data concepts and scalable data processing architectures. • Experience designing real-time and near-real-time data warehousing solutions. • Strong understanding of Oracle PL/SQL logic and database behavior, with the ability to analyze and reverse-engineer complex legacy implementations. • Strong experience with enterprise data integration and data migration initiatives. • Strong ETL/data integration experience with SSIS and/or Informatica as additional technologies. Preferred Qualifications • Experience supporting migration initiatives involving Salesforce. • Experience working with complex enterprise data environments and modernization programs.

Similar Jobs

More Jobs at Compunnel

  • Mainframe Developer
    $110K — $130K *
    Atlanta, GA 30349 (Fulton County)
    Information Technology
    In-Person
  • Product Specialist 4
    $95K — $115K *
    Dover, DE 19901 (Kent County)
    Information Technology
    In-Person
  • Market Data Analyst
    $80K — $95K *
    New York, NY 10025 (New York County)
    Finance & Insurance
    In-Person
  • Senior SQL Performance Engineer
    $110K — $130K *
    Florham Park, NJ 07932 (Morris County)
    Information Technology
    In-Person
  • Java AWS Developer
    $110K — $130K *
    Atlanta, GA 30349 (Fulton County)
    Enterprise Technology
    In-Person

More Information Technology Jobs

Find similar Azure Data Engineer jobs: