This role is responsible for designing, building, and maintaining data flows between corporate IT systems and ProMach's data lake. Leveraging tools such as Apache Spark and Delta Lake, you will manage data ingestion and transformation, integrate with cloud services, ensure data quality, and collaborate with analysts to deliver data-driven insights. Success in this position requires expertise in data modeling, SQL, Python, and CI/CD pipelines.
Key Responsibilities- Data Pipeline Development:
Design, build, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake. - ETL/ELT Processes:
Develop and manage robust ETL/ELT workflows for data extraction, transformation, and loading from diverse source systems. - Monitor and resolve any issues with scheduled jobs that move data.
- Data Architecture and Modeling:
Contribute to designing and implementing data models and architecture for data lakes and data warehouses. - Provide guidance on architecture and platforms that facilitate reporting and analytics needs
- Cloud Integration:
Integrate data solutions with cloud platforms (Azure, AWS, or GCP) and related services. - Data Quality and Governance:
Implement data quality checks, monitoring, and data governance best practices to ensure consistency, accuracy, and reliability. - Automation:
Implement and maintain CI/CD pipelines to automate deployment and streamline workflows. - Collaboration:
Work closely with business analysts and stakeholders to define data requirements and deliver actionable insights.
Required Skills and Qualifications- Proven experience in data engineering and cloud-based development.
- 3-5 Years' experience in a similar role
- Expertise in Databricks and the broader Databricks ecosystem (Databricks SQL, Delta Lake, Databricks Runtime).
- Proficiency in SQL, Python, and Apache Spark.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Strong understanding of data modeling, ETL/ELT, and data warehousing concepts.
- Experience with CI/CD tools and practices.
- Excellent problem-solving, analytical, and communication skills.
- Eager to learn new skills and step in to deliver where needed.
- Bachelor's Degree in Information Systems, Computer Science, or related field, or equivalent work experience.
Preferred Experience- Experience with ERP systems (SAP, Infor) and Salesforce CRM.
- Familiarity with BI tools such as Tableau.
- Manufacturing industry experience.
- Motivated, committed and energetic self-starter dedicated to providing high quality and responsive IT service
What's in it for you?There's no monopoly on innovative ideas or limits to how far you can advance with our team. We offer an open, communicative environment that fosters individual initiative. We pride ourselves in being a company where people are happy and choose to build a career.
In addition to growth opportunities, as a ProMach employee, you receive more than just a paycheck. Total compensation includes your salary, comprehensive medical/dental programs, as well as life insurance, a generous paid time off program, and a retirement savings plan with a company match. And because we understand the importance of feeling protected, there is no waiting period for benefits - you are eligible on your first day of employment!
Please note that the pay ranges listed on external job boards, outside our posting details, do not accurately reflect our internal compensation ranges.
