Objective & Deliverables
The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long term IT strategy and planning to include high level analysis, professional reports and presentations, and mentoring, support and training.
Technical Knowledge and Skills:
4+ years of hands-on Development, Deployment and production Support experience in Big Data environment.
4-5 years of programming experience in Java, Scala, Python.
Proficient in SQL and relational database design and methods for data retrieval.
Knowledge of NoSQL systems like HBase or Cassandra
Hands-on experience in Cloudera Distribution 6.x
Hands-on experience in creating, indexing Solr collections in Solr Cloud environment.
Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL.
Must have experience with developing Hive QL, UDF's for analyzing semi structured/structured datasets.
Must have experience with Spring framework, Web Services, REST API's and MicroServices.
Hands-on experience ingesting and processing various file formats like Avro/Parquet/Sequence Files/Text Files etc.
Must have working experience in the data warehousing and Business Intelligence systems.
Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs.
Successful track record of building automation scripts/code using Java, Bash, Python etc. and experience in production support issue resolution process.
Experience in building Client models using MLLib or any Client tools.
Hands-on experience working in Real-Time analytics like Spark/Kafka/Storm
Experience with Graph Databases like Neo4J, Tiger Graph, Orient DB
Agile development methodologies.
Top Must Haves:
5+ years of strong hands-on Design and Development experience in Hadoop/Big Data Environment.
Strong Hands-on coding experience in Java, Scala and Python languages
Strong Hands-on experience with Spring framework, Web Services, REST API's and MicroServices.
3+ years of Hands-on experience building data pipelines using Hadoop components Sqoop, Kafka, Hive, SOLR, Map Reduce, Spark, Spark SQL, HBase etc.
