SummaryAs a
Senior Cloud Data Engineer - GCP in Data & Analytics' Data Core team, you'll design, build and run the Google Cloud data pipelines behind Surescripts' products. You'll work hands-on in BigQuery and Airflow (Cloud Composer), building reliable, scalable data pipelines and leveraging Dataflow and other GCP services where appropriate. Partnering with data analysts, data scientists and engineers, you'll help deliver high-quality data that supports better healthcare decisions and advanced analytics across the organization. You'll have ownership in shaping and improving the cloud data platform that powers enterprise analytics and reporting.
Responsibilities- Design, build and maintain data pipelines using BigQuery and Airflow (Cloud Composer), leveraging Dataflow and other GCP services as needed for ingestion, transformation and loading.
- Optimize pipelines for performance, scalability and cost.
- Build data-quality checks, validation and monitoring; troubleshoot and resolve pipeline issues.
- Develop and maintain BigQuery data models and schemas that support analytics, BI and machine learning initiatives.
- Partner with analysts and data scientists to turn data requirements into technical solutions.
- Manage infrastructure as code and integrate pipelines into CI/CD workflows.
- Apply data governance and PHI security best practices, and help maintain our data catalog.
- Evaluate and leverage AI-assisted tools and emerging technologies to improve engineering efficiency, code quality, documentation and operational effectiveness.
- Document pipelines and processes, and bring forward improvements to how we build.
Required Qualifications- Bachelor's degree in Computer Science, Mathematics, Information Technology or a related field, or equivalent experience.
- 5+ years of professional data engineering experience.
- Solid understanding of data warehousing, data lake concepts and data modeling.
- 2+ years building production data pipelines on Google Cloud Platform, including BigQuery and Airflow/Cloud Composer.
- Strong SQL and Python skills.
- Experience with data-quality tools and techniques.
- Experience with infrastructure-as-code tools and CI/CD practices.
- Experience utilizing AI-assisted development tools to improve productivity, accelerate code development, troubleshoot issues and enhance engineering workflows.
- Ability to explain complex technical work clearly and to work independently and as part of a team.
Preferred Qualifications- Hands-on Dataflow experience.
- Experience with Terraform.
- Experience with Pub/Sub, Cloud Storage and other GCP services.
- Experience with Looker, Tableau or Power BI.
- Healthcare data experience, especially pharmacy, claims or prescription data.
- Experience supporting machine learning or data science workloads.
- Experience with HIPAA and PHI data security and governance.
- Google Cloud Professional Data Engineer certification.
