Python Developer

Confidential Company  •  Sunnyvale, CA

8 - 10 years experience  •  Staffing & Recruiting

Salary depends on experience
Posted on 02/21/18 by Krishna Reddy
Confidential Company
Sunnyvale, CA
8 - 10 years experience
Staffing & Recruiting
Salary depends on experience
Posted on 02/21/18 Krishna Reddy

Job Title:             Python Developer with Unix Cron & AWS

Job Location:     Sunnyvale, CA

Job Description:

?                   Minimum 9+ years of software programming experience

?                   Strong experience in ETL

?                   SQL scriptingexperience

?                   Strong Python Development experience around data management.

?                   Strong in Unix Cron not just exposure

?                   Must have the ability to work in a dynamic, fast-paced environment

?                   Strong communication skills to interact with Agile team members

Data Sources: Cloud based as well as on premise example: SAP CRM, ECC, people soft, PLM, Calliduscloud

 

High-level architecture:

?                   Amazon S3 bucket have folders for specific source files delivered by Fusion middleware.

?                   Data quality check is not handled by the sources.

?                   Anaconda Python based programs are in place to clean any abnormalities in the source files and re-write them back in their respective folders this is to avoid load failures.

?                   Most of the known errors are handled using this mechanism, moving forward you will require to enhance these scripts to include a fix for any new errors.

?                   Currently mechanism is processing one file at a time, target is to move it to parallel processing.

?                   Next is a Python program for data pipeline, based on AWS API and snowflake API. This is to write files on snowflake EDW

?                   There are 3 different layers on the snowflake side Pre=processing, staging and base tables

?                   UNIX cron is used for scheduling purpose.

Not the right job?
Join Ladders to find it.
With a free Ladders account, you can find the best jobs for you and be found by over 20,0000 recruiters.