Big Data Specialist


New York, NY

Industry: Staffing & Recruiting


5 - 7 years

Posted 164 days ago

  by    Radhika Arora

This job is no longer available.

Our client, a global Investment Bank is seeking a Big Data Specialist, this candidate will join the Data Analytics team, which works under the Markets Execution Technology division, this team is responsible for delivering research, pre-trade Analytics, pricing and structuring, execution and post trade functions supporting the business. 

The candidate will be part of the team and participate in setting up an efficient calculation engine using J2EE, object oriented programming, big datatechnologies and a client server architecture for the Research business.

This candidate will be a key member in the application design and development of the distributed compute cluster, web server and database setup, and accessory scripts and processes. Development will include logical data modelling, setting up a cluster, partitioning, ingestion of real time streaming feeds and an interface to allow 100’s of simultaneous interactive web requests. The team will develop solutions with both on-the-fly analytic calculations and pre-computed analytics efficiently stored for quick reads. The platform will ingest streaming and stored content, and store both raw and computed content. Compute will include aggregations, filtering, bespoke analytic calculations across bonds and indices spanning 30 years of historic data. 

This candidate must be:

  • An individual with strong analytic mind and an ability to grasp broad business requirements
  • Adept at moving across programming languages and operating systems
  • Excellent at problem solving


  • 7+ years of Java Experience 
  • 3+ years of experience in Big Datatechnologies 
  • Experience setting up and maintaining distributed clusters
  • Experience implementing distributed and scalable algorithms (Spark, Hadoop)
  • Experience with Cassandra, Spark, HDFS, Impala, HBASE, Kafka. Vertica a plus.
  • Proficiency in Avro, Parquet, JSON Formats.
  • Hands on programming experience with YARN/Spark using Java/Scala
  • Experience with Map Reduce
  • Experience with stream processing
  • Experience developing optimum strategies for distributing data over a cluster
  • Demonstrate strong interest in process automation and orchestration.
  • Very good understanding of relational as well as columnar databases
  • SQL application development
  • Experience developing object-oriented applications in Java
  • Experience developing on web technologies (Apache/Tomcat)
  • Experience in working with streaming data and time-series analytics
  • Experienceworking with different caching strategies
  • Experience working in development teams using agile techniques
  • BA or MSdegree in Engineering, Computer Science or Information Technology