Role Overview:We are seeking a highly experienced Senior Data Engineer with a strong background in Python and AWS to design, build, and implement robust data pipelines and real-time data integrations. This role is critical for developing scalable solutions leveraging AWS native services and advanced data technologies.
Key Responsibilities:- Design and construct sophisticated data pipelines utilizing AWS Native services and Python.
- Develop and implement APIs, specifically REST/Flask, for seamless real-time data integrations.
- Architect and build secure authentication and authorization mechanisms, including SSO, OAuth, and Entra.
- Engage in the design and implementation of solutions involving vector databases and embeddings for Retrieval Augmented Generation (RAG).
- Contribute to the development of event-driven architectures.
Required Skills:- Proven experience as a Senior Data Engineer, with a focus on Python and AWS, ideally 10+ years in this specific domain.
- Strong proficiency in Python programming.
- Extensive experience with AWS Cloud services, including Terraforms, Lambda, Stepfunctions, Textract, AWS Bedrock for AI LLM's, S3, SQS, and SNS.
- Hands-on experience with Amazon RDS, particularly PostgreSQL.
- Proficiency in utilizing Amazon API Gateway.
- Familiarity with event-driven architecture principles.
- Experience with ASP.NET 4.5.
Qualifications:- Minimum of 8-10 years of overall professional experience in relevant data engineering roles.
