JOB SUMMARY
The Sr. AWS Data Engineer will be responsible for designing, developing, and implementing enterprise-scale data solutions within a modern AWS-based cloud data platform. This role will focus on building scalable data pipelines, developing ETL/ELT solutions, supporting data lake and data warehouse initiatives, and enabling high-volume data ingestion and processing. The ideal candidate will possess deep AWS data engineering expertise, strong DevOps knowledge, and experience delivering robust, cloud-native data solutions within Agile environments.
KEY RESPONSIBILITIES
• Design, develop, and maintain scalable data pipelines to support data ingestion, transformation, processing, and storage within AWS environments.
• Build and implement enterprise ETL and ELT solutions that support analytics, reporting, and business intelligence requirements.
• Develop and support cloud-based data lake and data warehouse architectures using AWS-native services.
• Implement both real-time and batch data processing solutions utilizing AWS data engineering technologies.
• Design and optimize data models, workflows, and integration frameworks to improve scalability, performance, and reliability.
• Develop, automate, and maintain CI/CD pipelines for data engineering solutions using Jenkins and CloudFormation.
• Collaborate with developers, QA teams, architects, data consumers, and business stakeholders to deliver high-quality data solutions.
• Monitor, troubleshoot, and optimize performance across AWS data platforms and applications.
• Participate in architecture reviews, technical design discussions, and cloud solution planning.
• Implement best practices related to data governance, security, monitoring, operational excellence, and code quality.
• Support DevOps initiatives by automating deployment, infrastructure provisioning, and operational processes.
• Identify and resolve data quality, performance, integration, and processing issues across the platform.
• Create and maintain technical documentation, operational procedures, and knowledge-sharing materials.
• Contribute to Agile development activities including sprint planning, code reviews, and continuous improvement initiatives.
REQUIRED QUALIFICATIONS
• Minimum 8 years of experience as a Data Engineer with strong AWS cloud expertise.
• AWS Certification required:
o AWS Certified Solutions Architect - Associate or Professional
o AWS Certified Developer - Associate
• Hands-on experience with AWS services including:
o Amazon S3
o AWS Glue
o Amazon Athena
o Amazon EMR
o Apache Spark
o Amazon Redshift
o Amazon RDS
o AWS Lambda
o AWS Step Functions
o AWS Identity and Access Management (IAM)
• Strong experience designing, building, and supporting enterprise data pipelines.
• Extensive experience with ETL and ELT framework development and optimization.
• Experience designing and implementing cloud-based data lake and data warehouse solutions.
• Strong programming skills in:
o Python
o SQL
o Shell Scripting
• Experience implementing CI/CD pipelines using Jenkins and CloudFormation.
• Strong understanding of DevOps methodologies, automation practices, and continuous delivery processes.
• Experience working within Agile software development environments.
• Excellent analytical, troubleshooting, and problem-solving skills.
• Strong communication and collaboration skills.
PREFERRED QUALIFICATIONS
• Experience with streaming and real-time data technologies, including:
o Amazon Kinesis
o Apache Kafka
• Experience with orchestration and workflow management tools such as Apache Airflow.
• Experience with DevOps and software delivery tools including:
o Jenkins
o CloudBees
o SonarQube
o Artifactory
• Experience using containerization technologies such as Docker.
• Familiarity with source control platforms including Bitbucket.
• Experience working with Atlassian tools such as Jira and Confluence.
• Knowledge of Role-Based Access Control (RBAC) and AWS data security models.
• Experience supporting large-scale enterprise cloud modernization initiatives.
• Experience implementing monitoring, logging, and observability solutions for data platforms.
• Strong understanding of data architecture, governance, and cloud security best practices.
CERTIFICATIONS
• AWS Certified Solutions Architect - Associate or Professional (Required).
• AWS Certified Developer - Associate (Required).
• AWS Certified Data Engineer - Associate (Preferred).
• AWS Certified Data Analytics - Specialty (Preferred).
• Apache Spark or Databricks certifications (Preferred).
• Relevant DevOps or Cloud Engineering certifications are a plus.
ADDITIONAL SKILLS
• Strong knowledge of cloud-native data engineering best practices.
• Ability to design highly scalable and resilient data processing frameworks.
• Experience documenting technical solutions, operational procedures, and troubleshooting guides.
• Strong collaboration skills with cross-functional teams across Engineering, QA, Architecture, and Business units.
• Ability to balance multiple priorities in a fast-paced Agile environment while maintaining high-quality deliverables.
