Data Engineer (Databricks + Informatica + Azure)

Allata

$90K — $130K *
Healthcare
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • 5-7 years of experience in data engineering and ETL processes.
  • Proficient in Informatica and Databricks or similar data platforms.
  • Expertise in data architecture and data warehousing.
  • Experienced in custom coding for distributed systems using SQL, Python, or PySpark.
  • Familiar with cloud relational databases such as MS SQL Server or AWS RDS.
  • Knowledge of batch and streaming data processing techniques.
  • Understanding of data governance in regulated environments.

Responsibilities

  • Design and develop scalable data pipelines in cloud environments.
  • Optimize ETL/ELT processes using industry best practices.
  • Implement data models based on Data Lakehouse principles.
  • Ensure data quality and performance across all data layers.
  • Collaborate with stakeholders to address complex healthcare data needs.
  • Create reusable data transformation logic and modular components.
  • Support CI/CD deployment processes for data solutions.

Benefits

  • Work within a high-performing team focused on technical excellence.
  • Opportunity to contribute to significant healthcare projects.
  • Environment that promotes autonomy and data-driven decision-making.
  • Exposure to modern cloud-native architectures and data methodologies.
Full Job Description
We are seeking a skilled Data Engineer to join our consulting team on a strategic project with one of our key clients in the healthcare industry. You will be responsible for designing, building, and optimizing scalable data solutions that support analytics, reporting, and advanced data use cases in regulated environments.

In this role, you will work alongside data architects, analysts, and business stakeholders, translating healthcare data requirements into robust technical solutions. You will operate within high-performing teams in an environment where technical excellence, autonomy, and data-driven decision-making are valued.

Role & Responsibilities:

  • Design, develop, and maintain scalable data pipelines using modern distributed data processing platforms and cloud environments.
  • Build and optimize ETL/ELT processes following industry best practices and cloud-native architectures.
  • Implement data models aligned with modern Data Lakehouse principles and data architecture frameworks.
  • Ensure data quality, consistency, and performance across ingestion, staging, and curated data layers.
  • Collaborate with data architects, analysts, and business stakeholders to understand complex healthcare data requirements.
  • Develop reusable data transformation logic and modular processing components for efficient, maintainable systems.
  • Support deployment processes following CI/CD and DevOps best practices.
  • Monitor and optimize data workflows for performance, scalability, and reliability in production environments.
  • Contribute to data governance, security, and compliance practices relevant to regulated healthcare environments.


Hard Skills - Must have:

  • Proven hands-on experience with Informatica as a data integration and ETL platform.
  • Strong experience with Databricks or similar distributed data processing platforms.
  • Core expertise in data architecture, data integrations, data warehousing, and ETL/ELT process design.
  • Applied experience developing and deploying custom scripts and modules for distributed computing environments (custom code execution across parallel executors and worker nodes).
  • Strong proficiency in SQL, Python, and PySpark (or equivalent distributed processing languages) for data transformation and processing.
  • Solid knowledge of cloud and hybrid relational database systems such as MS SQL Server, PostgreSQL, Oracle, Azure SQL, AWS RDS, or comparable engines.
  • Hands-on experience with batch and streaming data processing techniques and data compaction strategies.


Soft Skills / Business Specific Skills:

  • Strong analytical and problem-solving skills.
  • Ability to work effectively in cross-functional and distributed teams.
  • Clear communication skills, with the ability to explain technical concepts to non-technical stakeholders.
  • Proactive mindset with a strong sense of ownership.
  • Commitment to delivering high-quality, reliable data solutions.

Similar Jobs

More Jobs at Allata

More Healthcare Jobs

Find similar Data Engineer (Databricks + Informatica + Azure) jobs: