Sr Hadoop Developer


Costa Mesa, CA

Industry: Financial Services


11 - 15 years

Posted 384 days ago

  by    Kumar Kankipati

Required Skills :

Candidates MUST have 12+ years of working experience. 2+ years of experience with Hadoop,Hive,Spark,Java, Scala, Sqoop, Cassandra.

Job Description :


Senior Hadoop Developer

Costa Mesa, CA

12 month contract


• Design a generic framework for high throughput and streaming data ingestion, curation and linkage for the Experian Data Fabric – a Data as a Service platform intended to be configured/extended for deployments across the globe

• Develop high throughput and streaming data pipelines using Experian Lambda Architecture; Internal bulk sources to include server logs, file uploads, database logs, relational sources and CDC; streaming sources to include web sockets, HTTP client callbacks, and kafka events

• Develop rules-driven, data transformation layer that can be rapidly configured / deployed at business units and geographies globally

• Build components for standardization, data curation, identity resolution and end-to-end traceability at dataset and record levels

• Develop data provisioning components to deliver data for operational analytics as well as research purposes

• Develop ultra-low latency micro services to serve data from the State Container within 10 milliseconds


• 5+ years hands-on experience as Software Developer/Engineer

• 4+ years Java/Scala development background

• 1+ yearexperienceworking with native MapReduce and/or spark

• Desired - 2+ years data engineering/ETL development experienceworking with data at scale

• Desired - 2+ experience with Heroku or Docker

• 1+ yearexperience across one or more of the following Hadoop, MapReduce, HDFS, Cassandra, HBase, Hive, Flume, Sqoop, Spark, Kafka, etc.

• Bachelor's degree in Computer Science, Engineering or quantitative discipline

·         Designs, develops, tests, and evaluates software and systems that enable computers to perform their applications, applying principles and techniques of computer science, engineering, and mathematical analysis.

·         Excludes paraprofessional positions and requires a degree in software engineering design and development.

·         Researches, designs, and develops computer software systems, in conjunction with hardware product development, applying principles and techniques of computer science, engineering, and mathematical analysis.

·         Analyzes software requirements to determine feasibility of design within time and cost constraints.

·         Consults with hardware engineers and otherengineering staff to evaluate interface between hardware and software, and operational and performance requirements of overall system. Formulates and designs software system, using scientific analysis and mathematical models to predict and measure outcome and consequences of design.

·         Develops and directs software system testing procedures, programming, and documentation.

·         Consults with customer concerning maintenance of software system.

·         May coordinate installation of software system.

·         12+ years of experiencerequired.

$80K - $110K