Role Overview:This role is for a Java Data Engineer responsible for the end-to-end lifecycle of scalable applications, services, and data pipelines. The position involves extensive backend development using Java, Scala, and Python, with a focus on building large-scale ETL/ELT, batch, and streaming data solutions. A key aspect includes integrating AI and Generative AI capabilities into applications and data workflows.
Key Responsibilities:- Design, develop, test, deploy, and maintain scalable applications, services, and data pipelines using Java, Scala, and Python.
- Develop backend services, RESTful APIs, and microservices utilizing Java and Spring Boot.
- Build and maintain large-scale ETL/ELT, batch, and streaming data pipelines.
- Work with structured and unstructured data across relational databases, data lakes, and distributed data platforms.
- Integrate AI and Generative AI capabilities into applications and data workflows using LLMs and AI APIs.
Required Skills:- Proficiency in Java, Scala, and Python for application and data pipeline development.
- Experience with Spring Boot for backend services, RESTful APIs, and microservices.
- Hands-on experience with Apache Spark, PySpark, and/or Scala-based data processing.
- Strong SQL skills and experience with databases such as PostgreSQL, MySQL, or Oracle.
- Experience with messaging and streaming platforms like Apache Kafka.
- Familiarity with Prompt Engineering, RAG, embeddings, vector search, and vector databases.
- Experience with cloud platforms including AWS, Azure, or GCP.
- Knowledge of modern CI/CD practices.
- Strong understanding of software engineering principles, design patterns, testing, debugging, performance, and production support.
- Excellent problem-solving, communication, and collaboration skills in an an Agile environment.
Qualifications:- 6+ years of relevant experience.
Preferred Skills:- Apache Spark
- Kafka
- Core Java
