OZ - Principal Consultant - DatabricksWhat We're Looking For:We're seeking a highly experienced Principal Consultant - Databricks to serve as the senior technical authority for complex enterprise data-platform engagements. The ideal candidate combines deep Azure Databricks and Lakehouse expertise with strong consulting, architecture, delivery leadership, and executive communication skills.
Position Overview:As a Principal Consultant - Databricks, you will own the technical vision, target-state architecture, standards, and roadmaps for Azure Databricks engagements from discovery through production delivery. You will shape secure, scalable Lakehouse solutions, guide multidisciplinary teams, support pre-sales and delivery planning, and ensure that platform decisions align with customer outcomes, operational requirements, security, performance, reliability, and cost goals. This is a remote U.S.-based role working primarily in the Eastern Time Zone.
Key Responsibilities:- Own Technical Vision and Architecture: Define target-state architecture, principles, standards, and roadmaps for complex Azure Databricks engagements.
- Lead Discovery and Architecture Workshops: Translate business priorities, security needs, operational constraints, and success measures into actionable solution designs.
- Design Enterprise Lakehouse Platforms: Architect secure and scalable solutions spanning ingestion, storage, transformation, orchestration, analytics, data sharing, and AI/ML enablement.
- Establish Platform Standards: Define engineering, governance, identity, networking, CI/CD, observability, reliability, performance, and cost-optimization practices.
- Provide Delivery Oversight: Review critical implementation artifacts, identify risks early, guide technical decisions, and maintain alignment between sold and delivered solutions.
- Support Pre-sales and Solutioning: Contribute to qualification, solution shaping, estimates, staffing plans, proposals, statements of work, demonstrations, and executive presentations.
- Build Reusable Consulting Assets: Develop reference architectures, accelerators, assessment methods, estimation models, delivery playbooks, and thought leadership.
- Mentor and Collaborate: Guide architects, engineers, and consultants while partnering with Databricks, Microsoft, customer stakeholders, sales teams, and internal practices.
RequirementsRequirements:- 10+ years of experience in enterprise data and analytics consulting, including:
- 5+ years architecting and delivering production Databricks or Lakehouse solutions
- 3+ years in senior technical leadership, solution architecture, or principal-level consulting roles.
- 3+ years in an insurance firm in a consulting capacity
- Demonstrated experience as the senior technical authority for complex enterprise data-platform engagements, with accountability from discovery through production delivery.
- Deep Azure Databricks expertise across platform architecture, data engineering, Unity Catalog governance, operationalization, performance, and cost management.
- Strong command of Apache Spark, PySpark, Python, SQL, Delta Lake, batch and streaming patterns, ETL/ELT, data modeling, data quality, performance tuning, and production troubleshooting.
- Working knowledge of Microsoft Entra ID, Azure networking and private connectivity, ADLS, secrets and key management, monitoring, and related Azure data services.
- Experience with Git-based development, automated testing, CI/CD, environment promotion, infrastructure as code, deployment automation, observability, runbooks, and support models.
- Strong consulting and business acumen, with the ability to connect architecture decisions to customer outcomes, delivery risk, scope, margin, adoption, and platform consumption.
- Executive-level written, verbal, facilitation, and presentation skills, with the technical depth to engage architects and engineers.
- Experience in discovery, estimation, staffing, proposal or SOW development, technical pre-sales, delivery leadership, and multidisciplinary team guidance.
- Self-directed, collaborative leadership with sound judgment, resilience, empathy, and the ability to create structure in ambiguous situations.
Preferred Qualifications:- Databricks Certified Data Engineer Professional strongly preferred.
- Azure Platform Architect Badge and/or Databricks Data + AI Platform Administrator accreditation preferred.
- Databricks Partner Champion or Delivery Expert recognition, partner or Center of Excellence leadership, co-sell experience, or work in regulated and security-sensitive industries.
- Candidates without an appropriate Databricks credential must be willing to complete an OZ-approved, role-aligned certification or accreditation path after joining.
- Databricks Certified Generative AI Engineer Associate or Machine Learning Professional is a plus.
- Experience with large-scale platform modernization, cloud transformation, data-warehouse migration, or consolidation involving Databricks.
- Experience creating reusable offerings, accelerators, reference architectures, assessment frameworks, delivery playbooks, or intellectual property.
- Experience overseeing production analytics, machine-learning, or AI workloads beyond proof-of-concept delivery.
Tools & Technologies:- Azure Databricks, Delta Lake, Unity Catalog, Databricks SQL, Lakeflow Jobs and pipelines, Apache Spark, PySpark, Python, and SQL.
- Microsoft Azure, Microsoft Entra ID, Azure Storage/ADLS, Azure networking, Private Link, Key Vault, and Azure Monitor.
- Git, GitHub and/or Azure DevOps, automated testing, CI/CD, Terraform or comparable infrastructure as code, Databricks CLI and APIs, and Declarative Automation Bundles.
- MLflow, model serving, Power BI, Microsoft Fabric, Azure Data Factory, and Microsoft Purview exposure preferred.
Competencies:- Databricks and Lakehouse architecture leadership
- Enterprise data-platform design and governance
- Technical pre-sales and solution shaping
- Delivery oversight and risk management
- Performance, reliability, security, and cost optimization
- Executive stakeholder communication and facilitation
- Team mentorship and cross-functional collaboration
Benefits- Competitive Pay
- 401K
- Health Coverage
