Full Job Description
Role: Databricks Engineer
Location: Remote
Position: 1
Employment Type: W2 Only
Role Overview:
We are seeking an experienced Databricks Engineer with strong expertise in Databricks Lakehouse, PySpark, SQL, Medallion Architecture, and Unity Catalog. The ideal candidate will be responsible for designing, developing, optimizing, and securing scalable data pipelines while implementing best practices for data governance, performance tuning, and CI/CD within the Databricks platform.
Key Responsibilities:
• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and SQL.
• Implement and optimize Medallion Architecture for efficient data processing.
• Perform performance tuning and optimization of Databricks workloads and Lakehouse environments.
• Configure and manage Unity Catalog, including row level and column level security.
• Manage data lineage, governance, and secure data sharing across enterprise environments.
• Implement and maintain CI/CD pipelines and automated testing processes within Databricks.
• Collaborate with cross functional teams to deliver reliable and scalable data solutions.
• Ensure data quality, security, and compliance across data engineering processes.
• Troubleshoot and resolve data pipeline and platform performance issues.
• Support ongoing enhancements and operational activities within the Databricks ecosystem.
Required Skills:
• Strong proficiency in PySpark.
• Advanced SQL experience.
• Hands on experience with Databricks Lakehouse Platform.
• Experience implementing Medallion Architecture.
• Performance tuning and optimization within Databricks.
• Experience configuring Unity Catalog.
• Knowledge of row level and column level security.
• Experience managing data lineage and secure data sharing.
• Hands on experience with CI/CD implementation and testing processes in Databricks.
• Strong analytical and problem solving skills.
• Excellent verbal and written communication skills.
Preferred Skills:
• SAS
• SQL
• Unix
• Linux
Expected Deliverables:
• Scalable Databricks Data Pipelines.
• Optimized Lakehouse Performance.
• Medallion Architecture Implementation.
• Unity Catalog Security Configuration.
• Data Lineage and Governance Solutions.
• CI/CD Pipeline Implementation.
• Technical Documentation and Operational Support.
Education Requirement:
• Bachelor's degree required.
Our Perks and Benefits:
Our benefits and rewards program has been thoughtfully designed to recognize your skills and contributions, elevate your learning experience, and provide care and support for you and your loved ones.
We also offer:
• Health Insurance with Dental & Vision
• 401K Plan
• Life Insurance, STD & LTD
• Paid Vacations & Holidays
• Continuous Skill Based Development and Career Advancement Opportunities