Job Summary
We are seeking a Lead Data Engineer with 10-12+ years of experience to design, build, and optimize modern enterprise data platforms and AI-enabled data solutions. This hands-on individual contributor role focuses on developing scalable data ecosystems, enabling AI-driven data access, and delivering intelligent data solutions using Microsoft Fabric Data Agent/Fabric IQ or Databricks Genie. The ideal candidate will possess strong expertise in data engineering, cloud data platforms, and AI/LLM technologies.
Key Responsibilities
• Design, build, and optimize enterprise-scale data platforms and modern data lake architectures.
• Implement, configure, and support Microsoft Fabric Data Agent/Fabric IQ or Databricks Genie solutions.
• Enable AI-driven data access and develop data foundations for Agentic AI use cases.
• Configure and optimize data agents, retrieval systems, and AI-powered data workflows.
• Design, develop, and maintain scalable data pipelines, ingestion frameworks, and ETL/ELT processes.
• Build and optimize data transformation workflows to support enterprise analytics and AI initiatives.
• Collaborate with AI, analytics, and business teams to deliver intelligent, data-driven solutions.
• Ensure data quality, governance, security, scalability, and performance across enterprise data platforms.
• Evaluate and implement modern data engineering best practices, tools, and emerging technologies.
• Lead technical solution design, development, testing, and production deployment with minimal supervision.
• Troubleshoot and optimize data platform performance and reliability.
Required Qualifications
• 10-12+ years of professional experience in Data Engineering.
• Strong hands-on expertise with modern cloud data platforms and enterprise data architectures.
• Experience implementing Microsoft Fabric Data Agent/Fabric IQ or Databricks Genie.
• Strong knowledge of Artificial Intelligence (AI), Large Language Models (LLMs), and configuring data agents for AI-powered solutions.
• Experience building or enabling AI agents on enterprise data platforms and data lake environments.
• Strong experience designing and developing data pipelines, ETL/ELT processes, and data transformation frameworks.
• Experience with data modeling and enterprise analytics platforms.
• Strong understanding of enterprise data governance, security, and performance optimization.
• Ability to independently drive technical initiatives from architecture and design through production deployment.
• Excellent analytical, problem-solving, communication, and collaboration skills.
Preferred Qualifications
• DP-700 Certification.
• Experience with Microsoft Fabric, Azure Data Services, Databricks, and modern lakehouse architectures.
• Experience with Agentic AI, Retrieval-Augmented Generation (RAG) architectures, vector databases, or AI-driven analytics solutions.
• Experience implementing scalable cloud-native data engineering solutions and enterprise AI platforms.
