5-7 years of hands-on experience in AWS services and cloud architecture.
Proficiency in Python and PySpark for data processing tasks.
Strong understanding of event-driven architectures and scalable data pipeline design.
Experience with AWS Glue for ETL and data catalog management.
Hands-on knowledge of Terraform for Infrastructure as Code implementation.
Familiarity with CI/CD practices and automation pipelines in cloud environments.
Responsibilities
Develop a migration plan for fraud detection campaigns to AWS.
Design event-driven architectures for processing fraud event data.
Build and manage scalable data pipelines using AWS Glue and Python.
Optimize data storage in Parquet format for analytics with Amazon Athena.
Develop integrations that automatically update customer profiles.
Implement automation for fraud-related outbound calls based on customer updates.
Conduct end-to-end testing and document system architecture.
Benefits
Opportunity to work with cutting-edge AWS technologies.
Engage in real-time data processing for fraud detection.
Collaborate in a dynamic and innovative team environment.
Continual learning and development through hands-on projects.
Full Job Description
Develop a comprehensive plan for migrating near real-time fraud detection campaigns from on-premises systems to AWS.
Design and implement event-driven architectures to process inbound dialer data (fraud events) using services such as Amazon EventBridge, Kafka, Kinesis Data Streams, and Kinesis Firehose.
Build and manage scalable data pipelines using AWS Glue (ETL jobs, Crawlers), PySpark, and Python for data ingestion, transformation, and processing.
Configure and manage Glue Crawlers to automatically Client schemas and update the Data Catalog.
Store and optimize data using Parquet format and enable analytics through Amazon Athena for efficient querying.
Develop integrations between Customer Profiles and messaging platforms to automatically trigger profile updates and downstream processes.
Implement automation to trigger fraud-related outbound calls based on updates in customer profiles.
Design and orchestrate workflows using AWS Step Functions to manage complex processing pipelines.
Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
Optimize system architecture for scalability, reliability, cost-efficiency, and ensure data integrity and security.
Conduct end-to-end testing of the entire framework to validate functionality, performance, and reliability.
Deploy, automate, and manage resources using CI/CD pipelines.
Continuously monitor system performance and implement optimizations post-deployment.
Maintain detailed documentation of architecture, workflows, and operational processes.
Technical Skills
Strong expertise in AWS services including:
Lambda
S3
EventBridge
Kinesis (Data Streams & Firehose)
Glue (ETL + Crawlers)
Step Functions
Amazon Connect
Athena
Macie
Proficient in:
Python
PySpark
Experience in:
Glue Crawlers for schema discovery and cataloging
Parquet-based data storage
Building scalable data pipelines
Strong understanding of event-driven architectures (Pub/Sub model)
Hands-on experience with Terraform (Infrastructure as Code)
Familiarity with CI/CD tools and automation pipelines
Preferred Qualifications
AWS Certifications:
AWS Certified Developer
AWS Solutions Architect
Experience with:
Messaging platforms like Kafka and Amazon EventBridge
Designing real-time data processing systems
Using Glue Crawlers + Athena for data lake architectures
