Proficient in Big Data technologies and Hadoop ecosystem
Strong programming skills in Scala, PySpark, and Python
Experienced with AWS services including Glue, EMR, and Lambda
Experience in requirement gathering and solution design
Responsibilities
Support and migrate existing data pipelines
Automate and monitor data processing workflows
Design and deploy Big Data analytics solutions on cloud
Translate business requirements into technical specifications
Analyze complex customer data for campaign performance
Integrate data from multiple sources for reporting
Develop scalable and cost-effective data solutions
Benefits
Collaborative work environment with cross-functional teams
Opportunities for professional development in cloud technologies
Exposure to cutting-edge Big Data tools and platforms
Flexible working arrangements within preferred states
Potential for impact on real-world advertising analytics
Full Job Description
Description of the Project
To integrate complex real-world Customer TV and Digital Ad viewing data to perform pre- campaign planning and post-campaigns performance measurements using Big Data Platform (Databricks, Glue, EMR, Step-functions, Lambda, PySpark and other AWS services to support existing workloads, Automate and monitor and based on requirement develop an elastic, scalable, high performance and cost-effective solution in collaboration with cross functional teams.
Key Goals Specified by Manthan / Client(s)
Position is responsible for the support, migration, automation and monitoring of existing data pipelines and based on new requirements, design, develop, test, and deploy the Big Data Analytics solutions on cloud (AWS) and Databricks.
Collaborate with key stake holders and translate business requirements to technical requirements & implement solutions.
Dive into large, noisy, and complex real-world customer TV and Digital Ad viewing data to do pre-campaign planning and post campaigns performance measurement using Big data Platform (Databricks, PySpark, Glue, EMR).
Responsible for support, migration, automation and monitoring of data pipelines to ingest data, integrate data from multiple data sources (On-Premise & Cloud) and create aggregated data sets for reporting needs and appropriate notifications.
Job Description
8+ years of overall software development
Experience on Big Data Technologies, Hadoop Eco system and Technologies with experience programming in Scala, PySpark and Python
Requirement gathering, design and Support, Migration, Automation
Monitoring of data pipelines with PySpark, AWS services
Location States ( NJ, NY, MA, CT, NC, PA preferred)