Advanced Cognitive Platform Data Engineer

IBM   •  

San Francisco, CA

Industry: Technology


11 - 15 years

Posted 216 days ago

This job is no longer available.

Job Description

Do you think BIG Data is better data? Do you think data lakes are more fun than real lakes? Do you get excited about building new cognitive tools in the Big Data ecosystem? Are you self-motivated and like working on teams? Do you enjoy building scalable platforms on the cloud? This role might be for you!

IBM is seeking Big Data Engineers with technical experience in building scalable distributed software on the cloud that will combine cognitive computing/Watson technology with traditional data engineering/science and apply it at scale to transform enterprise business processes. The ideal candidate will also be able to modify a traditional open-source or other data stack to incorporate cognition. Successful candidates should come from a strong data engineering background and need to have experience with un-structured and structured data and being able to transform and analyze the data using various tools. We are building new platforms, and may need to move our data to new technologies requiring data parity guarantees between new and old feeds.

Required Professional and Technical Expertise

• 10+ years of related work experience in software engineering, architecture and design
• Experienced in designing and delivering large scale distributed software systems, preferably in a large scale global business
• Experienced in service-oriented architecture (SOA) principles
• Experienced in cloud, microservices and Web services standards and best practices
• 5+ years of experience in architecting and developing a highly scalable, distributed system using open source tools and big data technologies: Hadoop, HBase, Spark, Kafka and Kubernetes
• Experience in cloud-based Agile, DevOps experience with PaaS and IaaS, using Big Data batch and streaming tools, SQL, NoSQL, relational database design, DFS, TensorFlow and methods for efficiently retrieving data
• Ability to define, develop, apply and evangelize advanced software patterns, cognitive capabilities and standards to solve complex problems in Cognitive, NL, ML, DL and AI
• Manage risk and build prototypes or POCs where needed to validate technical approach
• Hands-on ability to gather and process raw data at scale, including writing scripts, web scraping, calling APIs, and writing SQL queries
• Hands-on programming in Java, Scala, Python, SQL
• Excellent technical and critical thinking skills
• Excellent verbal and written communication skills

Location Statement

We will consider for employment qualified applicants with arrest and conviction records

Required Technical and Professional Expertise

• 10+ years of related work experience in software engineering, architecture and design

Preferred Technical and Professional Experience

Master’s or Doctorate preferred

Eligibility Requirements