Senior Big Data Engineer (Java, Spark, Kafka)
What does a Big Data Engineer, Senior do?
As an experienced member of our Data Engineering Platform Group, you will be responsible for building and taking ownership over the successful design and development of data engineering projects within Client's Enterprise Data Commerce Solutions division. You will be required to apply your depth of knowledge and expertise to all aspects of the data engineering lifecycle, as well as partner continuously with your many stakeholders daily to stay focused on common goals. You'll work in a collaborative, trusting, thought-provoking environment-one that encourages diversity of thought and creative solutions that are in the best interests of our customers globally. You will Lead large-scale data engineering, integration and warehousing projects, build custom integrations between cloud-based systems using APIs and write complex and efficient queries to transform raw data sources into easily accessible models by using the Data integration tool with coding across several languages such as Java, Python, and SQL. Additional responsibilities include, but are not limited to Architect, build, and launch new data models that provide intuitive analytics to the team and Build data expertise and own data quality for the pipelines you create.
Basic Qualifications for consideration:
• 10+ Overall industry experience
• 7+ years' experience with building large scale big data applications development
• Bachelors in computer science or related field
• Provide technical leadership in developing data solutions and building frameworks.
• Expertise in solutions for processing large volumes of data, using data processing tools and Big Data platforms.
• Experience building Data Lake, EDW and data applications using Azure, AWS and
• Hands-on experience in cloud Data stack (preference is Azure)
• Understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS, SQL experience
• Hands on experience on major programming/scripting languages like Java
• Java experience with OOPS concepts, multithreading
• Nice to have experience deploying code on containers.
• Conduct code reviews and strive for improvement in software engineering quality.
• Hands-on experience in production rollout and infrastructure configuration
• Demonstrable experience of successfully delivering big data projects using Kafka, Spark
• Exposure working on NoSQL Databases such as Cassandra, HBase, DynamoDB, and Elastic Search
• Experience working with PCI Data and working with data scientists is a plus.
• In depth knowledge of design principles and patterns
• Able to tune big data solutions to improve performance.
Preferred Skills, Experience and Education:
• Exposure to Big Data tools and solutions a strong plus
• Exposure to Relational Modeling, Dimensional Modeling, and Modeling of Unstructured Data
• Bachelors in computer science or related field
• Experience in Design and architecture review
• Experience in Banking, Financial domain