AWS Data Center Capacity Delivery (DCCD) is looking for a Data Engineer to support data center construction globally. We work on the most challenging problems, with thousands of variables impacting the data center delivery - and we're looking for talented people who want to help.
You'll join a diverse team of software, hardware, and network engineers, construction specialists, security experts, operations managers, and other vital roles. You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. You'll experience an inclusive culture that welcomes bold ideas and empowers you to own them to completion.
We're looking for Data Engineer to help us grow our Data Lake and Data Warehouse Systems, which is being built using a serverless architecture, with 100% native AWS components including Redshift Spectrum, Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch and more! We own a world-class data lake that is used to drive multi-billion dollar decisions on a regular cadence and we're looking to improve on filling the lake quickly, with as little human intervention needed and democratize the data in the lake.
Our Data Engineers build the ETL and analytics solutions for our internal customers to answer questions with data and drive critical improvements for the business. Our Data Engineers use best practices in software engineering, data management, data storage, data compute, and distributed systems. We are passionate about solving business problems with data!
Key job responsibilities
- Design and implement scalable, fault-tolerant data pipelines using AWS technologies and internal Amazon tools to extract, transform, and load data from multiple sources leveraging and implementing AI solutions as required.
- Collaborate cross-functionally with BIEs, Data Scientists, PMs, and SDEs to understand data requirements and deliver customized data solutions.
- Automate infrastructure deployment with CI/CD pipelines and ensure streamlined processes for deployment and maintenance.
- Ensure data quality through robust validation, cleansing, and deduplication techniques.
- Implement data governance standards, including access control, encryption, data retention, deletion policies, and audit mechanisms to ensure compliance and security.
- Continuously improve and optimize data pipelines and infrastructure, staying up to date with emerging technologies and implementing automation and monitoring tools.
- Build a scalable and reliable data platform supporting analytics for intuitive, self-service data products.
- Write high quality code and build scalable applications that interface with critical services and APIs to extract and process unstructured data
- Work with a range of data technologies, including Python, EMR, Spark, Iceberg, Airflow, and many AWS data services like Glue, Athena, Redshift to create end-to-end pipelines that consolidate data from disparate systems.
About the team
DCCD -CAT is a central data and analytics team within the DCCD tooling org that plays a pivotal role in supporting analytics for data center construction space suporting cost,controls and commissioning domains. We own data platform, reporting, dashboards, measurement, and analytical solutions for DCCD org.
BASIC QUALIFICATIONS
- Bachelor's degree
- 3+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with SQL
- Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence
- Knowledge of distributed systems as it pertains to data storage and computing
- Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam
- Experience as a data engineer or related specialty (e.g., software engineer, business intelligence engineer, data scientist) with a track record of manipulating, processing, and extracting value from large datasets
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
- Experience with Apache Spark / Elastic Map Reduce
PREFERRED QUALIFICATIONS
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
- Master's degree in computer science, engineering, analytics, mathematics, statistics, IT or equivalent
- Experience programming with at least one modern language such as C++, C#, Java, Python, Golang, PowerShell, Ruby
- Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
- Experience in translating business needs into detailed feature requirements
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, WA, Seattle - 132,100.00 - 178,800.00 USD annually
