5+ years of hands-on experience in DataOps and cloud infrastructure management.
Strong expertise in AWS services, including EC2, EKS, and S3.
Proficient in Python, PySpark, and SQL for data manipulation and processing.
Experience with Infrastructure as Code tools like Terraform and AWS CloudFormation.
Awareness of FinOps for managing cloud costs effectively.
Responsibilities
Build and manage automated systems, pipelines, and tools for data handling.
Monitor and improve data platforms to enhance data flow and uptime.
Design and optimize data pipelines for ETL/ELT workloads with CI/CD capabilities.
Implement observability and health checks to ensure data reliability.
Lead incident response and root cause analysis process for data issues.
Manage the automated testing and release lifecycle of data engineering solutions.
Collaborate with managed service providers to adopt new DataOps practices.
Benefits
Long-term role with opportunities for growth and development.
Comprehensive involvement in cutting-edge DataOps practices.
Engage in innovative cloud solutions with industry-leading tools.
Work in a collaborative environment directly impacting data quality.
Contribute to cost optimization and efficiency in large data environments.
Full Job Description
Job Description
Role: DataOps Engineer
Location: Erie, PA (Day 1 Complete Onsite)
Duration: Long Term
Skills:
DataOps builds and manages the automated systems, pipelines, and tools that move and test data inside a company.
Operate, automate, monitor, and continuously improve core data platforms to reduce waste, improve data flow, and ensure maximum uptime for structured datasets and analytics.
Design and optimize robust data pipeline automation for ETL/ELT workloads, including orchestration, scheduling, and CI/CD to streamline data extraction and processing.
Implement deep observability, manage logs, automate health checks, and rigorously track SLAs/SLOs to increase data reliability.
Lead rapid incident response, stakeholder communications, and Root Cause Analysis (RCA) to continuously identify process gaps and correct them.
Manage the end-to-end release lifecycle, executing automated testing (unit, performance, and end-to-end tests), smooth deployments, rollbacks, and performance tuning.
Partner with managed service providers to test and adopt new solutions that adhere to DataOps best practices and ensure contract SLAs are met.
Translate technical metrics into clear executive reports and facilitate collaboration with data and BI teams to enhance the quality of data products.
Strong hands-on experience with AWS, Medallion architecture, and modern DataOps practices. Deep understanding of complex XML handling.
Deep expertise in AWS Infrastructure (EC2, EKS, S3, Glue, Lambda), IAM, and ensuring stringent security standards are applied across all data pipelines.
Proficient in Infrastructure as Code using Terraform and AWS CloudFormation.
Design data engineering assets, develop reusable frameworks and patterns, enforce standards, and leverage AI-assisted coding tools (Codex).
Strong FinOps awareness to monitor, control, and optimize cloud costs across large data environments.
