Hadoop Database Development Lead
Full Time/Perm/Direct Hire
Our client is looking for candidates with data warehouse and people management experience. Experience with Hadoop technology not mandatory at this time.
Leads database development staff while focusing on the analysis, development and maintenance of the Data Lake / Data Hub as well as the feed to/from all subscriber applications using the Hadoop ecosystem.
It involves development and support of integrations with multiple systems and ensuring accuracy and quality of data by implementing business and technical reconciliations.
Development and support will be required for Data Lake / Hub implementation on the Hadoop environment.
1.development and maintenance of the Hadoop Platform and various associated components for data ingestion, transformation and processing
2.development and maintenance of integrations with various bespoke Financial (Applications and Management Information Systems), Underwriting and Claims Management Systems
- Minimum of Eleven or more years working as a productive member of a development team
- Bachelors degree in Computer Science or other computer-related field
- Experience of leading midsized team of data warehouse / Hadoop / integration developers
- Experience of working both independently and collaboratively with various teams and global stakeholders (Business Analysts / Architects / Support / Business) in an agile approach while working on projects or data quality issues
- Experience in development and support of Data Lakes, using Hadoop Platform and associated technologies such as Hive, Spark, Sqoop
- Demonstrable experience on Hadoop ecosystem (including HDFS, Spark, Sqoop, Flume, Hive, Impala, Map Reduce, Sentry, Navigator)
- Experience on Hadoop data ingestion using ETL tools (e.g. Talend, Pentaho, Informatica) and Hadoop transformation (including MapReduce, Scala)
- Experience working on Unix / Linux environment, as well as Windows environment
- Experience on Java or Scala or Python in addition to exposure to Web Application Servers preferred
- Exposure to NoSQL (HBase) preferred
- Experience in creating Low Level Designs
- Prior experience of analysis and resolution of data quality and integration issues
- Experience in providing and maintaining technical documentation, specifically Data Mapping and Low Level / ETL (or ELT) Design Documentation
- Experience in supervising the work of, and mentoring junior team members, in a project environment
- Experience on Continuous Integration preferred
- Experience on large scale (>1TB raw) data processing, ETL and Stream processing preferred
- Experience in working as SME for enterprise systems or integrations
- Experience of Database queries (SQL) and procedure performance tuning
- Experience in development of complex / enterprise data warehouse implemented over standard RDBMS (preferably using SQL Server Integration Services (SSIS) and SQL Server Objects preferred
- Insurance Domain Experiencepreferred
o For BizTalk Server
o For all SQL Server Database Objects
o For T-SQL Query and Stored Procedure Optimization, Table Indexing, and Constraints
o For SQL Server Integration Services (SSIS) Data Integration Tasks