Strong AWS expertise, particularly with data services
Proficient with Databricks Lakehouse, PySpark, and Spark SQL
Experience in building enterprise-level data solutions and pipelines
Familiarity with compliance and security standards in data handling
Degree in Computer Science or related field; advanced degree preferred
Databricks or AWS certifications are a plus
Responsibilities
Design and develop scalable data pipelines using Databricks and AWS
Build and maintain enterprise data lake and Lakehouse solutions
Implement frameworks for real-time and batch data ingestion
Optimize performance of Spark workloads and data processes
Collaborate with teams to integrate Databricks with AWS services
Support data migration initiatives from traditional environments to Databricks
Participate in code and architecture reviews to enforce governance
Benefits
Opportunities for continuous learning and professional development
Exposure to cutting-edge technologies and practices
Collaborative work environment with cross-functional teams
Potential for career advancement in a growing field
Flexible work arrangements and a supportive organizational culture
Full Job Description
Role description
Job Title: DATABRICKS Architect
Work Location
Irving TX
Job Description:
Role Overview
We are seeking a highly skilled Databricks Engineer with AWS expertise to support Data Services portfolio
The role will be responsible for designing developing optimizing and supporting enterprise scale data platforms data pipelines and analytics solutions leveraging Databricks Lakehouse AWS Cloud and modern data engineering practices
The candidate will work closely with data architects' application teams business stakeholders and platform engineering teams to deliver scalable secure and high-performance data solutions
Key Responsibilities
Design develop and maintain scalable data pipelines using Databricks PySpark Spark SQL and Delta Lake
Build and support enterprise data lake and Lakehouse solutions on AWS
Develop batch and Realtime ingestion frameworks from multiple source systems
Implement data transformation enrichment cleansing and validation processes
Optimize Spark workloads for performance scalability and cost efficiency
Design and implement data models supporting reporting analytics and downstream applications
Integrate Databricks with AWS services including S3 Glue Lambda Redshift EventBridge and IAM
Support data migration and modernization initiatives from traditional data warehouses to Databricks Lakehouse
Implement CICD pipelines and Infrastructure as Code IaC using Terraform and Gitbased repositories
Collaborate with security teams to ensure compliance with client governance risk and data security standards
Troubleshoot production incidents and provide L3 operational support
Participate in architecture reviews code reviews and technical governance activities
Mandatory Technical Skills
Databricks
Databricks Lakehouse Architecture
Databricks Workflows
Delta Lake
Unity Catalog
Databricks SQL
Auto Loader
Structured Streaming
Databricks Performance Tuning
Medallion Architecture
AWS
Amazon S3
AWS Glue
AWS Lambda
AWS IAM
Amazon Redshift
AWS CloudWatch
SNS SQS
EventBridge
KMS
Secrets Manager
Data Engineering
PySpark
Apache Spark
Python
SQL
Data Warehousing Concepts
ETLELT Design
Data Modeling
Data Quality Frameworks
Batch RealTime Processing
Preferred Skills
Snowflake
Microsoft Fabric
GCP BigQuery
Kafka
Airflow
dbt
Terraform
Docker
Kubernetes OpenShift
Informatica
Collibra
Data Vault Modeling
Dimensional Data Modeling
Banking Domain Experience
Strong experience within Banking and Financial Services
Understanding of enterprise data governance regulatory reporting risk compliance and audit requirements
Experience working in large global organizations with stringent security and operational standards
Qualifications
Bachelors or Masters degree in Computer Science Engineering Information Systems or related discipline
Databricks Certified Data Engineer Preferred
AWS Certified Data Analytics or AWS Solution Architect Preferred