Senior Data Engineer

Paul Hastings

$134K — $194K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's degree in Computer Science, Engineering or related field required.
  • 7+ years of data engineering or BI development experience.
  • 3+ years building data solutions in Microsoft Azure Cloud.
  • 2+ years using Azure DevOps for automated builds and pipeline orchestration.
  • Proven experience working with structured and semi-structured data.

Responsibilities

  • Design, build and manage scalable data pipelines using Azure Data Factory, ADLS and Databricks.
  • Develop ELT frameworks for data transformation from various sources.
  • Implement DevOps pipelines for code versioning, testing, deployment and monitoring.
  • Ensure pipeline performance and observability using Azure Monitor.
  • Collaborate with BI teams to optimize Power BI datasets and data models.
  • Automate environment setup and configuration using infrastructure-as-code.
  • Mentor junior data engineers and advocate for best practices in CI/CD.

Benefits

  • Medical, Dental, Vision, Life/AD&D, Long Term Care, and Short/Long-Term Disability coverage.
  • Flexible Spending Accounts and Health Savings Account options.
  • Generous PTO program and 10 paid holidays annually.
  • Comprehensive family support including paid parental leave and tutoring assistance.
  • Professional development programs and an anniversary bonus program.
Full Job Description


Job Title
Senior Data Engineer

Location
Atlanta, Boston, Charlotte, Chicago, Dallas, Houston, Los Angeles, New York

Regular/Temporary
Regular

Job Description Summary
We have an opening for a Senior Data Engineer.

The Senior Data Engineer is responsible for designing, building and maintaining modern data pipelines and data integration solutions in the Azure Cloud. This role enables enterprise analytics by delivering clean, timely and well-structured data to platforms such as Power BI. The Senior Data Engineer collaborates closely with architects, analysts and business stakeholders to deliver scalable, secure and efficient data solutions using Azure Data Factory, Azure Data Lake Storage (ADLS), Databricks, and Azure DevOps.

Job Description

In this capacity, the Senior Data Engineer will:
  • Design, build and manage scalable data pipelines using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS) and Azure Databricks;
  • Develop ELT frameworks for ingesting and transforming structured and semi-structured data from on-prem and cloud sources;
  • Implement robust DevOps pipelines for code versioning, testing, deployment and monitoring using Azure DevOps;
  • Ensure pipeline performance, reliability and observability using tools like Azure Monitor and Log Analytics;
  • Collaborate with BI teams to support Power BI datasets and ensure optimal data model performance;
  • Automate environment setup, configuration and permissions using infrastructure-as-code where possible;
  • Write modular, testable code in Python or PySpark and manage version control with Git;
  • Support production operations, root cause analysis and performance tuning;
  • Maintain technical documentation for pipelines, data mappings and deployment processes;
  • Work as part of an Agile delivery team, participating in sprints, retrospectives and code review; and
  • Mentor junior data engineers and promote best practices in CI/CD, version control, and release management.


Proficiencies:
  • Advanced proficiency in Azure Data Factory, Azure Data Lake Storage (Gen2) and Azure Databricks;
  • Experience building and managing CI/CD pipelines in Azure DevOps for data solutions;
  • Strong SQL and ELT experience; familiarity with schema design (e.g., star, snowflake, denormalized);
  • Hands-on expertise with PySpark and Databricks notebooks, including Delta Lake workflows;
  • Familiarity with Git, branching strategies and repository management within Azure Repos;
  • Understanding of data security and privacy best practices in Azure (RBAC, managed identities, encryption);
  • Experience with Power BI dataflows, datasets and refresh configuration (preferred);
  • Familiarity with data cataloging and governance tools (e.g., Microsoft Purview) a plus; and
  • Experience with Agile tools such as Azure Boards or JIRA.


Qualifications:
  • Bachelor's degree in Computer Science, Engineering or related field (required);
  • 7+ years of data engineering or BI development experience;
  • 3+ years building data solutions in Microsoft Azure Cloud;
  • 2+ years using Azure DevOps for automated builds, releases and pipeline orchestration;
  • Proven experience working with structured and semi-structured data (Parquet, JSON, Avro); and
  • Experience in professional services, legal, or financial environments preferred.


The salary wage range for New York that we expect to pay for this position is a minimum of $134,500 and a maximum of $194,750 annually. The actual pay wage may vary based on experience or other relevant factors.

The salary wage range for Los Angeles that we expect to pay for this position is a minimum of $125,800 and a maximum of $182,450 annually. The actual pay wage may vary based on experience or other relevant factors.

The salary wage range for Boston that we expect to pay for this position is a minimum of $134,500 and a maximum of $194,750 annually. The actual pay wage may vary based on experience or other relevant factors.

Employees will be provided with an excellent career opportunity in a collaborative environment, in addition to a generous total compensation package with the opportunity to earn bonuses based on individual contribution and firm profitability.

The actual salary offered will depend on a variety of factors, including without limitation, the qualifications of the individual applicant for the position, years of relevant experience, level of education attained, certifications or other professional licenses held, and if applicable, the location in which the applicant lives and/or from which they will be performing the job. This role is exempt meaning it is not overtime pay eligible.

Eligible employees can participate in the Firm's comprehensive benefits program, which include the following:

  • Medical, Dental, Vision, Life/AD&D, Long Term Care, and Short- and Long-Term Disability
  • Flexible Spending Accounts and Health Savings Account
  • Healthcare Concierge and Advocacy
  • Lifestyle Spending Account
  • Voluntary 401(k) Plan and Profit Sharing
  • 10 Paid Holidays per year and a generous PTO Program
  • Family Support including Paid Parental Leave, Fertility Benefits, Breast Milk Shipping, Back-up Child Care, Elder Care, and Tutoring
  • Wellbeing programs (Employee Assistance Program, Relationship Support, Mental Health, Menopause and Midlife Health and Well-Being Events)
  • Retirement Plan Consulting
  • Anniversary Bonus Program
  • Professional Development Programs
  • Transportation and Commuter Benefits
  • International Travel Insurance
  • Critical Illness, Hospital Indemnity and Accident Insurance
  • Auto/Home/Pet Insurance
  • Prepaid Legal Insurance
  • Employee Discounts
  • And More!

Similar Jobs

More Jobs at Paul Hastings

  • Senior Data Engineer
    $134K — $194K *
    New York, NY 10025 (New York County)
    Information Technology
    In-Person
  • Senior Data Engineer
    $134K — $194K *
    Remote
    Information Technology
    Remote in Boston, MA
  • Senior Data Engineer
    $134K — $194K *
    Atlanta, GA 30349 (Fulton County)
    Information Technology
    In-Person
  • Senior Data Engineer
    $134K — $194K *
    Boston, MA 02115 (Suffolk County)
    Information Technology
    In-Person
  • Senior Data Engineer
    $134K — $194K *
    Dallas, TX 75217 (Dallas County)
    Information Technology
    In-Person

More Information Technology Jobs

Find similar Senior Data Engineer jobs: