Data Platform Architect DataBricks

Compunnel

• $150K — $180K *
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 10+ years of experience in data platform architecture or distributed systems.
  • Demonstrated experience in cloud migration involving large datasets.
  • Strong understanding of data warehousing fundamentals, including schema design.
  • Expertise in Medallion architecture and large-scale data platforms.
  • Proficient in Databricks, Apache Spark, and Kafka for distributed processing.
  • Experience with AWS, Azure, or GCP cloud platforms.
  • Strong programming skills in Java, Python, and SQL.

Responsibilities

  • Define architecture for enterprise data platforms and analytics solutions.
  • Lead cloud migrations from on-premises platforms.
  • Design scalable, secure architectures for petabyte-scale workloads.
  • Implement Medallion and lakehouse architectures.
  • Establish governance frameworks and best practices.
  • Collaborate with teams to align data strategies with cloud initiatives.
  • Evaluate technologies for data management and observability.

Benefits

  • Mentorship opportunities for engineering teams.
  • Involvement in strategic modernization initiatives.
  • Potential for remote work flexibility.
  • Collaboration with cross-functional teams.
  • Opportunities for professional certification and development.
Full Job Description
div:has([data-free-thinking-preview-answer=true])+:is(.text-message,.relative:has(>.text-message))]:-mt-2 grow">

Job Summary:
We are seeking an experienced Data Platform Architect to lead the strategic modernization of an enterprise data ecosystem. This role is responsible for defining and executing architecture for large-scale data platform transformations across on-premises and cloud environments. The ideal candidate will have extensive experience with large-scale data migration and modernization, greenfield data platform implementations, lakehouse and Medallion architectures, distributed data processing, and cloud-native technologies.

Key Responsibilities:
• Define target-state architecture for enterprise data platforms and cloud-native analytics solutions.
• Lead large-scale data migrations and modernization initiatives from on-premises platforms to cloud environments.
• Design scalable, resilient, secure, and highly available data architectures capable of supporting petabyte-scale workloads.
• Design and implement Medallion and lakehouse architectures.
• Establish architectural standards, governance frameworks, and engineering best practices.
• Partner with infrastructure, security, application, and architecture teams to align data platform initiatives with enterprise cloud strategies.
• Evaluate and recommend technologies for data storage, processing, streaming, governance, and observability.
• Provide technical leadership and mentorship to engineering teams implementing Spark, Kafka, Databricks, and cloud-native services.
• Drive performance optimization, cost management, scalability, and operational excellence across the data platform.
• Support migration roadmaps, risk assessments, and modernization strategies for executive stakeholders.
• Design and support large-scale data movement and transport solutions.
• Apply data warehousing principles, including schema design, star and snowflake schemas, fact and dimension modeling, and Slowly Changing Dimensions (SCDs).

Required Qualifications:
• 10+ years of experience in data platform architecture, data engineering, or distributed systems.
• Demonstrated experience leading enterprise-scale cloud migration and modernization initiatives involving hundreds of terabytes or petabytes of data.
• Experience with large-scale data migration from platforms such as Vertica, Hadoop, Teradata, Oracle, Snowflake, or similar technologies.
• Strong experience designing and implementing greenfield large-scale data platforms.
• Strong expertise in Medallion architecture.
• Strong experience with big data movement and transport.
• Strong understanding of data warehousing fundamentals, including star and snowflake schemas, fact and dimension modeling, and SCDs.
• Strong experience with Databricks and Apache Spark.
• Strong experience with Apache Kafka and distributed data processing.
• Experience with AWS, Azure, or GCP cloud platforms.
• Strong software engineering background with Java and Python programming experience.
• Strong SQL skills and experience with object-oriented programming concepts.
• Experience designing highly available, scalable, and resilient distributed systems.
• Strong understanding of data governance, security, compliance, and operational monitoring.
• Experience with enterprise messaging or data transport technologies such as AMPS, MQ, Hadoop, or similar platforms.

Preferred Qualifications:
• Experience implementing lakehouse architectures.
• Experience designing streaming and real-time analytics platforms.
• Experience working within highly regulated financial services or enterprise environments.
• Master's degree.
• Experience with Databricks and cloud architecture best practices.

Certifications:
• Databricks certification(s) preferred.
• AWS Solutions Architect certification(s) preferred.

Similar Jobs

More Jobs at Compunnel

More Information Technology Jobs

Find similar Data Platform Architect DataBricks jobs: