Consultant, Big Data Engineering
• Purpose of the Team:
o Provide strategic direction and expertise to all of MasterCard’s Big Data environments.
o Provide Big Data hands-on support, thought leadership and technology roadmaps to support MasterCard’s vision and evolution.
• Purpose of the Job:
o Assume a lead position in providing strategic direction, evaluation and recommendation of Big Data products to support MasterCard’s current and future vision of Big Data.
o Represent Big Data Engineering alongside business partners to provide expertise that supports significant M&A, strategic and new technology infrastructure.
o Design and implement large scale data migration and data processing on Hadoop using massively parallel processing programming (MPP) techniques.
Essential responsibilities of the position:
• Work directly with application and development teams to completely understand requirements for project and ad hoc requests and provide accurate estimates.
• Provide technical leadership to a team of subject matter experts that support mission-critical Big Data environments for highly-available and contingent platforms.
• Create and maintain documentation and define best practices in support of proper change management, security and operational requirements.
• Investigate and understand new technologies and additional capabilities of current platforms that support MasterCard’s vision of Big Data, leading their delivery adoption.
• Mentor and develop the technical skills and abilities of the team in the areas of Big Data concepts and application of technology to serve business needs
• Direct maintenance and operational support activities related to Big Data platforms per MasterCard standards and industry best practices.
All About You
• Essential knowledge, skills and experience:
o Strong knowledge of Hadoop platforms and other distributed data processing platforms.
o Advanced Linux knowledge is a must. Understanding of shell, debugging, etc.
o Strong background in delivering mission-critical Big Data project work while interacting with diverse and experienced teammates across the spectrum of enterprise operational functions.
o The candidate must possess strong problem solving skills and engage in troubleshooting complex problems and require coordination across multiple teams.
o Experience with CI/CD (continuous integration / continuous integration) and infrastructure automation tools and techniques.
• Desirable or additional capabilities:
o SQL knowledge a must. Experience with advanced data warehousing and MPP knowledge is a plus.
o Understanding of and experience with network engineering in support of large IT organizations.
o Background building and maintaining extract/transform/load (ETL) processes. Practical experience dealing with large ETL pipelines is a plus.
o Experience building and maintaining Big Datatechnologies on public or private cloud technologies.