Design, develop, and maintain data pipelines to handle large and complex data sets using Big Data technologies such as Hadoop, Spark, and Kafka. Ensure that data pipelines are scalable, reliable, and performant, and that they adhere to data governance, data security, and data quality best practices. Help develop sustainable data solutions with current and leading next generation data technologies to transform business capabilities. Collaborate with end users to ensure data availability and accessibility. Work with cross-functional teams to identify and implement process improvements for data pipelines and data workflows. Monitor and troubleshoot data pipeline issues to ensure high availability and reliability. Participate in code reviews and provide feedback to other team members to ensure that code is high quality and adheres to best practices. Remain current with industry trends and emerging technologies in Big Data and data engineering and evaluate their potential impact on our data infrastructure and systems. Lead, mentor, and develop up and coming engineers. Engage in technology strategy and data architecture across enterprise. The position is based out of and reports to Thrivent's HQ in Minneapolis, MN, but allows for the option to telecommute from a home office anywhere in the US.
Requirements:- Minimum of a Bachelor's or foreign equivalent degree in Management Information Systems, Computer Science, or related field plus 6 years of progressively responsible post baccalaureate experience as a data engineer.
- At least 6 years of Data Engineering experience with Big Data Technologies: Apache Spark and Hadoop.
- At least 5 years of experience with the following:
- Programming with Python, Java or Scala;
- Designing and architecting big data solutions using Spark and Hadoop.
- At least 4 years of experience with the following:
- Building data pipelines, CICD pipelines, and fit for purpose data stores;
- BI development and reports;
- Data Warehouse modelling with star schema; modelling data in big data eco system;
- Developing and executing test cases.
At least 3 years of experience building data pipelines using Snowflake, Databricks and Kafka.
At least 2 years of experience with the following:
- Streamsets and Bamboo;
- Working with Informatica, IICS and DB2;
- Cloud technologies: AWS, Azure, Google Cloud, OpenStack, Docker, Ansible, Chef, Lambdas, Microservices and Terraform.
Salary: at least $173,400/year
OtherThe following benefits may be offered: various bonuses (including, for example, annual or long-term incentives); medical, dental, and vision insurance; health savings account; flexible spending account; 401k; pension; life and accidental death and dismemberment insurance; disability insurance; supplemental protection insurance; 20 days of Paid Time Off each year; Sick and Safe Time; 10 paid company holidays; Volunteer Time Off; paid parental leave; EAP; well-being benefits, and other employee benefits. Eligibility for receipt of these benefits is subject to the applicable plan/policy documents. Thrivent's plans/policies are subject to change at any time at Thrivent's discretion.
