Senior Data Engineer - Ab Initio / SnowflakeDescription:
We are looking for a Senior Data Engineer with strong experience in Ab Initio and Snowflake to design, develop, and support enterprise-scale data integration and data warehouse solutions.
You will work within a large-scale Azure data environment, supporting high-volume ETL/ELT processes, complex data pipelines, metadata management, and cloud modernization initiatives. The role is ideal for a hands-on technical expert comfortable working with complex data ecosystems and hundreds of Ab Initio processes.
Responsibilities:
- Design, develop, and support high-volume ETL/ELT data integration solutions using Ab Initio and Snowflake.
- Develop and maintain Ab Initio graphs, Plans, PSETS, Continuous Flows, and batch processing frameworks.
- Build reusable ETL components, transformations, DML files, XFRs, and PDL scripts.
- Manage and support Ab Initio EME / Metadata Hub, including metadata, lineage, impact analysis, and dataset mapping.
- Develop and optimize Snowflake solutions, including Snowpipe, Streams, Tasks, Time Travel, and Zero-Copy Cloning.
- Perform advanced SQL development, query optimization, data reconciliation, and data quality validation.
- Design and maintain data warehouse solutions using dimensional modeling, Star/Snowflake schemas, and Data Vault concepts.
- Develop automation and operational support tools using Python and Shell scripting (KSH/Bash).
- Support batch, real-time, file-based, and API-driven data integrations.
- Participate in CI/CD, Git-based development, code promotion, branching, and release management processes.
- Troubleshoot production issues and ensure the reliability, scalability, and performance of data pipelines.
- Collaborate with technical and business teams to translate data requirements into scalable solutions.
- Work with healthcare data domains, including claims, membership, providers, health plans, benefits, and employer/group data.
Your Qualifications:
- 6+ years of hands-on experience with Ab Initio development.
- 4+ years of experience with Snowflake development and architecture.
- Advanced SQL skills, including complex joins, subqueries, stored procedures, dynamic SQL, and query tuning.
- Strong experience with Ab Initio EME / Metadata Hub, version control, sandbox management, check-in/check-out, and tag promotion.
- Experience with data warehouse architecture, dimensional modeling, Star/Snowflake schemas, Data Vault, MDM concepts, and source-to-target mapping.
- Proficiency in Python and Shell scripting (KSH/Bash).
- Strong understanding of ETL/ELT patterns, batch and real-time processing, API integrations, and large-scale data movement.
- Experience with DevOps and CI/CD, including Git, branching strategies, automated deployments, and release management.
- Experience working with Azure-based data environments.
- Knowledge of healthcare data, particularly:
- Claims: Professional, institutional, medical, pharmacy, encounter, and adjudication data.
- Membership: Enrollment, eligibility, coverage, benefits, population health, and member attribution.
- Provider: Directories, networks, contracting, NPI, attribution, and provider hierarchies.
- Health Plans: Benefits, products, groups, and employer data.
- Strong analytical, troubleshooting, and problem-solving skills.
- Ability to work independently in a complex enterprise data environment and collaborate effectively with cross-functional teams.
