Full Job Description
DataStage ETL Engineer
- Build and support ETL pipelines that move high-volume data across data warehouse and big data platforms.
Qualifications
- 5+ years of ETL development experience with IBM DataStage, SQL, UNIX/Linux shell scripting and distributed big data systems (Hadoop, Spark).
- 3+ years of Python experience, plus hands-on work with workflow orchestration tools (such as Airflow or Control-M).
- Experience building ETL pipelines in Python using pandas.
- Strong experience with data migration, data analysis, transformation, conversion, system interfaces, high-volume data loading, data modeling and SQL performance tuning.
- Working knowledge of Apache Spark and RDDs (Resilient Distributed Datasets) using Python (PySpark) or Scala.
- Nice to have : Teradata Vantage, Greenplum, Hive
#M1
#LI-DP1
Ref: #404-IT Pittsburgh