Hi
We are seeking an experienced AWS Data Platform Engineer to design, build, and operate enterprise-grade data pipelines across both on-premises and AWS cloud environments. This role will be instrumental in building the foundation of our enterprise data platform, enabling secure, scalable, and governed data ingestion, processing, and analytics. The ideal candidate has hands-on experience developing distributed data pipelines, deploying cloud-native infrastructure, and integrating data from diverse enterprise systems while adhering to security, governance, and operational best practices.
Key Responsibilities
• Design, develop, and maintain scalable data pipelines across hybrid (on-premises and AWS) environments.
• Build and optimize streaming and batch data ingestion pipelines for structured and unstructured data.
• Develop reusable data engineering frameworks that support enterprise-scale analytics and AI initiatives.
• Integrate data from multiple enterprise sources, including:
• IoT and sensor data
• Finance and ERP systems
• Procurement applications
• Security and audit logs
• Operational databases
• APIs and event streams
• Deploy and manage cloud-native data infrastructure using Infrastructure as Code (Terraform).
• Build secure, governed data platforms following enterprise security and data governance standards.
• Implement CI/CD pipelines and automate deployments for data platform components.
• Collaborate with architecture, security, DevOps, and business teams to deliver reliable, scalable data solutions.
Required Technical Skills
• Python
• SQL
• Terraform (Infrastructure as Code)
• Apache Kafka
• Apache Flink
• Apache Beam
• Apache Spark
• Apache Airflow
• Kubernetes (Amazon EKS)
• Trino and/or Amazon Athena
• Amazon S3
• AWS Lambda
• AWS IAM
• AWS Lake Formation
• GitHub
• Harness
• JFrog
• Argo CD
Preferred Skills
• Experience building enterprise data platforms in hybrid cloud environments.
• Experience with event-driven and streaming architectures.
• Knowledge of distributed systems and large-scale data processing.
• Strong understanding of AWS networking, security, IAM, VPCs, and data governance.
• Experience with monitoring, observability, and production support for data platforms.
• Familiarity with enterprise architecture patterns and metadata-driven data platforms.
