Degree in Computer Science, Information Systems, or related field.
5+ years of experience as a Data Engineer or Data Lake Architect on AWS.
Deep knowledge of AWS services like S3, EMR, Redshift.
Proficient in advanced SQL for Redshift.
Experience with ETL process design on AWS.
Familiarity with Jenkins, Bitbucket, and Webhooks.
Knowledge in Docker for containerized workloads.
Responsibilities
Design, build, and maintain the Data Pipelines.
Develop collaborative relationships with business sponsors and IT teams.
Lead the development of ETL processes for various data sources.
Refine the internal Python data management framework.
Enhance existing CI/CD framework with BitBucket and Jenkins.
Ensure code deployment standards are met across environments.
Participate in architecture design and code reviews.
Mentor junior team members in Data Engineering practices.
Benefits
Dynamic work environment focused on a data-driven culture.
Opportunity to work with state-of-the-art technologies.
Collaborative work with key business stakeholders.
Mentorship opportunities within the team.
Involvement in innovative data engineering practices.
Full Job Description
Client is seeking a talented and motivated Senior Data Engineer to join their Data and Analytics team. Client is constantly enhancing their Data Engineering Platform for business and operational analytics. At client, they have state of the art technologies such as Oracle Fusion, Sales Force, Adobe to name a few. The Data platform will allow users to answer "what, "when and "how questions and allow them to ask "what if. You should be passionate about solving business problems using data. Integrating data from disparate sources to find hidden gems should be your second nature. You will be part of the team that is driven by data driven culture
Responsibilities
Design, build and maintain all facets of the Data Pipelines for Data Engineering team. Develop collaborative relationships and work with key business sponsors and other IT teams to solve complex problems with simple solutions Lead the development of ETL processes to integrate data from a wide variety of sources including S3 objects, Kafka, REST APIs, SQL Server Continue refining the Python data management framework developed internally Enhance existing data centric CI/CD framework utilizing BitBucket and Jenkins Ensure established standards are followed for code deployment between environments Participate in architecture design sessions and perform code reviews Follow and improve the established standards for design, development, and migration Mentor junior team members Qualifications Degree in Computer Science, Information Systems, Information Technology, or related engineering field Five + years experience as a Data Engineer or as a Data Lake Architect on AWS platform Required experience Deep knowledge of AWS Services such as S3, EMR, Redshift, DataPipeline, RDS, DynamoDB, STEP functions, Lambda Advanced SQL knowledge for Redshift ETL process design and implementation on AWS Jenkins, Bitbucket, Webhooks Containerized workload management using Docker Streaming data processing (Kafka)
Location: NYC
Duration: FTE
Salary: DOE
Work Authorization: No visa sponsorship at this time
