10+ years of development experience in Python or another modern programming language
10+ years of experience with SQL and relational databases
10+ years of experience in custom ETL design and maintenance
5+ years of experience with workflow management engines (Airflow, Luigi, etc.)
Must have domain experience in supply chain (E-commerce/manufacturing)
Responsibilities
Design, build, and launch efficient data pipelines
Apply expertise to create scalable data warehouses
Source external data securely from partners
Design data models for optimal storage and retrieval
Implement data quality checks to ensure high data quality
Optimize and maintain existing data pipelines
Own the end-to-end data engineering component of solutions
Benefits
Hybrid/remote work flexibility
Long-term contract opportunity
Exposure to cutting-edge data technologies
Opportunity to work in a startup-like culture
Potential for professional development in data engineering practices
Full Job Description
Title: Sr. Data Engineer (SCM) Location: Bay area, CA (Hybrid/Remote) Duration: Long term contract
Description:
Design, build and launch efficient & reliable data pipelines to move and transform data (both large and small amounts)
Apply proven expertise and build high-performance scalable data warehouses
Securely source external data from numerous partners
Intelligently design data models for optimal storage and retrieval
Deploy inclusive data quality checks to ensure high quality of data
Optimize existing pipelines and maintain of all domain-related data pipelines
Ownership of the end-to-end data engineering component of the solution
Experience in creation of data pipelines using Python.
Python for data manipulation & transformation (python dictionaries, data frames, data stream, joins of all kinds, outside of SQL IDEs)
Research skills - go figure it out and come back with working model. Pros/cons analysis skills
Strong work ethic, independent problem-solving skills, self-motivated, comfortable with startup culture
Excellent and crisp communicator
Must have domain experience in supply chain (E-commerce /manufacturing etc)
Minimum Qualifications
BS/MS in Computer Science or a related technical field
10+ years of Python or other modern programming language development experience
10+ years of SQL and relational databases experience
10+ years experience in custom ETL design, implementation and maintenance
5+ years of experience with workflow management engines (i.e. Airflow, Luigi, Prefect, Dagster, Digdag, Google Cloud Composer, AWS Step Functions, Azure Data Factory, UC4, Control-M)
