Position will be responsible for the integration, expansion and support of the North American Automotive data warehouse solutions. The Data Warehouse Developer is responsible for interpreting, organizing, designing, executing, and coordinating technical assignments within the Data Warehouse. Candidates must have hands on experience with Oracle 12c in an Exadata platform and with InfoSphere Datastage 11.5 tools. Knowledge of AWS tools, Spark, Python is also preferred. Applicant will work closely with a multitude of groups throughout the organization which will require the ability to communicate with associates with backgrounds ranging from DB development, Application Support, Networking, Operating System Administrators, Software Development Managers and Business Users from a wide range of operating units.
- Participate in the requirement gathering process and develop complex source to target mapping rules
- Conduct Data Profiling in support of the design and testing processes
- Create ETL design documents that supports best practices and development standards
- Contribute to the development of best practice document and ensure adherence to best practices
- Create staging strategy for optimum reusability and performance
- Integrate data from multiple source systems into Data Warehouse using SCD type 1 and type 2 approaches
- Participate in architectural decisions for data integration from source to staging to ODS to AFDW
- Create ETL flows to Integrate the On-Prem data to Cloud Snowflake warehouse using Nifi/PySpark/SnowSql
- Configure EC2 Clusters, S3 Buckets, EMR and other AWS services needed for AFDW
- Involved in developing and maintaining Scheduling and Sequence jobs with complex dependencies.
- Troubleshooting of jobs and addressing production issues like data issues, ENV issues, performance tuning and enhancements.
- Use relevant test types and develop test strategy for all data integration scenarios.
- Automate resiliency process for achieving complete restartability and recoverability of the load processes with zero data loss
- Extract, transform, and load data from various Databases, Sequential files, XML documents.
- Coordinate with offshore development team and mentor developers on proper usage of DataStage tool and adoption of best practices and standards.
- Maintain metadata at appropriate levels commensurate with best practices
- Bachelor's Degree in Computer Science or equivalent
- 3+ years of experience on data population to Data Warehouse(s) built using IBM's Industry specific Reference Data Models based on Data-vaulting approach for the Banking industries
- 3+ years ETL experience using IBM InfoSphere 11.5 tools in a Grid environment.
- 3+ years of advanced SQL skills in Oracle based on Exadata platform.
- 3+ years of advanced Unix scripting experience
- 1+ years of AWS(S3, EC2 , EMR, Cloudwatch, Lambda)
- 1+ years of SPARK/ Python/ PySpark
- 1+ years of Devops tools like Jenkins/Urbancode
- Must have experience working on a Data Warehouse built using 3rd Normal Form and Datamarts built on dimensional model.
- Strong experience with staging strategy, Change Data Capture, DB and Datastage partitioning, DB Compression, and ETL solutioning to populate staging area.
- Possesses extensive experience in Unit Testing, Functional Testing, System Testing, Integration Testing, Regression Testing, User Acceptance Testing (UAT) and Performance Testing.
- Ability to work in a rapid-development environment.
- Ability to work closely with customers in analyzing requirements.
- Ability to work flexible hours.
- Strong communication skills, both written and spoken
- Comfortable working independently, but has experience working in a team environment
- Skills in establishing and maintaining effective working relationships with clients and staff
- Ability to read, write, speak and understand English required
- Can discern when escalation is needed.
- Experience in Agile / SCRUM (with sprints) methodologies preferred
- Finance or Automotive Industry experience preferred