Employer: CDK Global LLC
Job Title: Sr. Software Engineer
Worksite: 11809 Domain Drive, Suite 200 Austin TX 78758.
#LI-DNI
Job Description: Design and develop EMR pipelines by using AWS services like SQS QUEUE, EC2 instances, AWS data pipeline, S3 buckets, AWS glue, RDS and others. Create extract-transform-load (ETL) EMR pipelines based on HADOOP, hive, Yarn resource manager, NIFI, spark and python frameworks in AWS. Interpret the data mapping document to identify the source systems like SQL server and develop required spark transformations for ingesting the ETL data into titan platform. Optimize spark jobs using Pyspark after complete analysis of multiple parameters and opportunities to improve the target systems along with data quality checks. Create spark data frames/RDD's and load the data in different formats JSON, Parquet, AVRO, CSV and others. Evaluate new architectures and technologies such as snowflake, Debezium and other tools to improve performance and efficiency of ETL tasks. Responsible for completing the data requests from CDK product customers and help them debug and resolve any data quality issues in a timely manner. Work closely with the CDK customers and provide the feedback to the CDK service team to improve reliability of our products. Work in the scaled agile methodologies to increase the quality of the deliverables. Monitor and resolve production 11/12 issues. 100% Telecommuting.
Requirements: Bachelor's degree or foreign equivalent in Computer Science, Information Technology, Computer Engineering or a related field plus 5 years of professional experience as a Software Developer or related occupation. Alternatively, a Master's degree or foreign equivalent in Computer Science, Information Technology, Computer Engineering or a related field plus 3 years of professional experience as a Software Developer or related occupation. Additionally, the applicant must have employment experience with: 1) Designing, developing, and migrating data pipelines to latest data frameworks such as Databricks; 2) Creating Extract-Transform-Load (ETL) pipelines based on Hadoop, Hive, Nifi, Spark and Python frameworks in cloud AWS/Snowflake; 3) Map data between source systems and data lake and develop required data transformations for ingesting the source data into data lake (TITAN Platform); 4) Creating spark data frames/RDD's and load the data in different formats such as Json, Parquet, Avro, and CSV; 5) Evaluating new architectures and technologies such as Snowflake, Debezium, Kafka and other tools to improve performance and efficiency of ETL tasks.
To Apply: Applicants who are interested in this position visit https://cdk.wd1.myworkdayjobs.com/en-US/CDK or email resume to [redacted] Reference Req# JR9104.
#recruit
#LI-DNI
CDK Global is committed to fair and equitable compensation practices. Compensation packages are based on several factors, including but not limited to skills, experience, certifications, and work location. The total compensation package for this position may also include annual performance bonus, benefits and/or other applicable incentive compensation plans.We offer Medical, dental, and vision benefits in addition to:
- Paid Time Off (PTO)
- 401K Matching Program
- Tuition Reimbursement
At CDK, we believe inclusion and diversity are essential in inspiring meaningful connections to our people, customers and communities. We are open, curious and encourage different views, so that everyone can be their best selves and make an impact.
