Full Job Description
Job Title: US_East | DATA ARCHITECT / DATA OFFICER_L4
JOB SUMMARY
The role involves architecting, designing, and delivering enterprise-scale data platforms using Databricks Lakehouse, with a focus on hands-on development of complex data pipelines, Spark jobs, and cloud-native data solutions. It requires leading implementations with Databricks technologies, designing and optimizing data processing frameworks, and developing ingestion, transformation, and orchestration solutions leveraging Azure/AWS data services. The position also includes establishing best practices for data management, collaborating with stakeholders, providing technical leadership, and supporting solutioning and growth initiatives.
Key Responsibilities
• Architect, design, and deliver enterprise-scale data platforms using Databricks Lakehouse.
• Remain hands-on with development of complex data pipelines, Spark jobs, and cloud-native data solutions.
• Lead implementation using Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog, and Workflows.
• Design and optimize batch and real-time data processing frameworks.
• Develop data ingestion, transformation, and orchestration solutions leveraging Azure/AWS data services.
• Establish best practices for data modeling, governance, security, monitoring, and performance tuning.
• Collaborate with business stakeholders, product owners, and engineering teams to translate requirements into scalable solutions.
• Provide technical leadership, code reviews, mentoring, and delivery oversight.
• Support discussions, solutioning, estimations, and account growth initiatives.
Required Qualifications
• 12+ years of experience in Data Engineering and Data Architecture.
• 5+ years of strong hands-on Databricks experience.
• Expert-level proficiency in PySpark, Spark SQL, Python, SQL, and Delta Lake.
• Strong experience with Azure Databricks and related services (ADF, ADLS Gen2, Synapse) or equivalent AWS ecosystem.
• Experience building large-scale ETL/ELT pipelines and modern Lakehouse architectures.
• Strong understanding of data governance, security, CI/CD, DevOps, and cloud architecture.
• Experience with performance tuning, optimization, and cost management within Databricks.
• Excellent communication and stakeholder management skills.
Preferred Qualifications
• Banking/Financial Services experience.
• Exposure to AI/ML, GenAI, and advanced analytics on Databricks.
Certifications
• Databricks Certified Professional or Architect certification.