Role Overview:This role requires a skilled Incorta Data Engineer with strong hands-on experience in the Incorta platform, focusing on designing and developing data models, schemas, and dashboards. The successful candidate will also be proficient in PySpark for building and optimizing large-scale data pipelines, integrating data from various sources, and ensuring data accuracy and system performance.
Key Responsibilities:- Design and develop data models, schemas, and dashboards using Incorta.
- Build and optimize data pipelines using PySpark for large-scale data processing.
- Ingest, transform, and integrate data from multiple sources including ERP systems (SAP/Oracle), databases, flat files, and APIs.
- Collaborate with business stakeholders to understand reporting and analytics requirements.
- Develop and maintain Incorta schemas, business views, and insights dashboards.
- Optimize query performance and data load processes within Incorta.
- Perform data validation, testing, and troubleshooting to ensure data accuracy.
- Integrate Incorta with enterprise systems such as SAP, Oracle, or other data sources.
- Monitor system performance and implement improvements for scalability and efficiency.
- Maintain documentation for data models, pipelines, and reporting solutions.
Required Skills:- Incorta: Strong hands-on experience with Incorta platform, including schemas, dashboards, SPARK SQL, Incorta business views, and insights dashboards.
- Data Engineering: Proficiency in PySpark for data transformation and processing, data pipeline development and optimization, large-scale data processing, data ingestion, and integration.
- Data Warehousing & Modeling: Good understanding of data warehousing concepts, ETL processes, data modeling techniques, and schema design.
- Databases & Big Data: Experience working with large datasets (big data environment), SQL, and Spark SQL.
- ERP & Integration: Experience integrating data from ERP systems (SAP, Oracle), and integration with databases, flat files, APIs, and other data sources.
- Performance & Quality: Query performance optimization, data load optimization, data validation, testing, troubleshooting, data accuracy assurance, system performance monitoring, scalability, and efficiency improvements.
- Collaboration & Documentation: Collaboration with business stakeholders to understand reporting and analytics requirements, and documentation of data models, pipelines, and reporting solutions.
- Soft Skills: Strong analytical and problem-solving skills.
Qualifications:- 6-8 years of experience in a relevant data engineering or Incorta development role.
