Senior Data Engineer

Dynamic Lifecycle Innovations

$85K — $120K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's degree in computer science, IT, software engineering, or a related field; experience may substitute for degree.
  • 5+ years of hands-on experience in enterprise data pipelines and application integrations.
  • Production experience with cloud-based ingestion, transformation, and data delivery for analytics.
  • Advanced proficiency in SQL, Python, and PySpark for data transformation.
  • Familiarity with Git-based source control and CI/CD practices.
  • Preferred: hands-on experience with Azure Databricks and Delta Lake.
  • Preferred: experience with MuleSoft Anypoint Platform and Power BI.

Responsibilities

  • Own and manage the Azure Databricks lakehouse, defining its architecture and governance.
  • Design and deploy scalable data ingestion pipelines from various sources.
  • Develop and maintain data layers using Delta Lake, SQL, and PySpark.
  • Optimize Databricks workloads for performance and cost efficiency.
  • Create and support APIs and automated workflows using integration technologies.
  • Implement data quality controls and lead troubleshooting efforts.
  • Collaborate with stakeholders to design durable technical solutions.

Benefits

  • Comprehensive health benefits starting the first of the month following hire.
  • 401(k) with company match.
  • Profit sharing opportunities.
  • Generous paid time off and paid holidays.
Full Job Description
Senior Data Engineer

You'll start with the Predictive Index (PI) assessment (takes
Work Location: Onalaska, WI (Hybrid options)

Compensation & Benefits

Total Compensation: $85,000 - $120,000 annually

Benefits Include:
  • Comprehensive health benefits starting the first of the month following hire
  • 401(k) with company match
  • Profit sharing
  • Generous paid time off and paid holidays

Your Purpose

As our Senior Data Engineer, you'll serve as the primary technical owner of our Azure Databricks lakehouse and the integrations that keep data moving across our enterprise. You'll build reliable bronze, silver, and gold data layers, create trusted data products for Power BI, analytics, automation, and AI, and develop secure application integrations using MuleSoft and other technologies.

Your work will turn complex data into dependable, reusable assets, helping teams make better decisions while establishing the engineering standards, architecture, and governance that allow our data platform to scale.

What You'll Do
  • Own and operate the Azure Databricks lakehouse, helping shape its architecture, standards, governance, reliability, and future roadmap.
  • Design, build, deploy, and support scalable ingestion pipelines from enterprise applications, APIs, relational databases, flat files, SaaS platforms, and external sources.
  • Develop and maintain bronze, silver, and gold data layers using Delta Lake, SQL, Python, and PySpark, including dimensional models and business-ready datasets.
  • Optimize Databricks workloads for performance, scalability, maintainability, reliability, and cost.
  • Design and support APIs, application interfaces, automated workflows, and file-based integrations using MuleSoft and other integration technologies.
  • Implement data quality controls, reconciliation, lineage, monitoring, logging, alerting, and recovery practices; lead root-cause analysis when issues occur.
  • Partner with business stakeholders, BI developers, application owners, architects, vendors, and consultants to translate needs into durable technical solutions and documentation.

What You Bring (Briefcase)
  • Bachelor's degree in computer science, information technology, software engineering, or a related field. Additional relevant experience will be considered in lieu of formal education.
  • Five or more years of progressive, hands-on experience designing, developing, deploying, and supporting enterprise data pipelines, analytical data models, and application integrations.
  • Production experience with cloud-based ingestion, transformation, dimensional modeling, data quality, monitoring, troubleshooting, and curated data delivery for reporting or analytics.
  • Advanced SQL skills and proficiency with Python and PySpark for production-grade pipelines and transformations.
  • Experience applying Git-based source control, CI/CD, automated deployment, and environment configuration practices to data engineering workloads.
  • Preferred: Hands-on experience with Azure Databricks, Delta Lake, Unity Catalog, Databricks Workflows, Auto Loader, Lakeflow Declarative Pipelines, or comparable modern data-platform tooling.
  • Preferred: Experience with MuleSoft Anypoint Platform or another enterprise integration/iPaaS platform, plus Power BI datasets or semantic models.
  • Preferred: Experience integrating ERP, CRM, finance, logistics, WMS, manufacturing, or other operational systems into an enterprise data platform.

Skills & Abilities (Head)
  • Deep understanding of lakehouse and medallion architecture, including bronze, silver, and gold data layers.
  • Strong command of batch, streaming, incremental, change-data-capture, API-based, and file-based ingestion patterns.
  • Ability to design dimensional models and curated data products that support reporting, self-service analytics, machine learning, and AI.
  • Strong diagnostic thinking-you can trace discrepancies across source systems, transformation logic, integrations, and reporting layers to find the real cause.
  • Sound judgment in balancing speed with long-term maintainability, scalability, security, governance, reliability, and cost.
  • Clear communication and documentation skills, with the ability to translate business requirements into technical designs and collaborate across technical and non-technical teams.
  • Working knowledge of secure cloud-data practices, including access control, secrets management, encryption, service principals, and environment separation.

Who You Are (Heart)

You're a hands-on builder who enjoys creating order from complexity and taking meaningful ownership of the platforms behind the scenes.

You are curious enough to explore better approaches, disciplined enough to build for production, and persistent enough to stay with a difficult data issue until it is truly resolved.

You collaborate with humility, communicate clearly, share what you know, and care deeply about earning trust in both the data and the relationships surrounding it.

Most of all, you're energized by the chance to build something that scales-and to see your work improve decisions across an entire business.

Similar Jobs

More Jobs at Dynamic Lifecycle Innovations

More Information Technology Jobs

Find similar Senior Data Engineer jobs: