Databricks Data Scientist

Prophecy Technologies

$110K — $130K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's or master's degree in data science, Computer Science, Statistics, Engineering, or a related field.
  • 4+ years of experience in data science or advanced analytics.
  • Hands-on experience with Databricks and Apache Spark.
  • Strong programming skills in Python (PySpark, Pandas, NumPy, Scikit-learn).
  • Experience building and deploying ML models in production.
  • Solid understanding of SQL and data modeling.
  • Experience with MLflow, model lifecycle management, and experimentation.

Responsibilities

  • Design, develop, and deploy machine learning models using Databricks (MLflow, Spark ML, Python).
  • Implement end-to-end ML pipelines (data ingestion to monitoring).
  • Collaborate with data engineers to ensure reliable, high-quality datasets in the Lakehouse.
  • Leverage Databricks Lakehouse (Delta Lake, Unity Catalog) for scalable analytics.
  • Optimize Spark jobs for performance and cost efficiency.
  • Apply best practices for data governance, lineage, and security.
  • Configure and enable Databricks Genie for self-service analytics.

Benefits

  • Flexible work hours and remote work options.
  • Opportunities for professional development and certification.
  • Access to cutting-edge technologies and tools.
  • Collaborative work environment with cross-functional teams.
  • Health and wellness programs.
Full Job Description
Role Overview:

We are seeking a Databricks Data Scientist with strong experience in Databricks Lakehouse, advanced analytics, and Genie (AI/BI) to design, build, and deploy scalable data science and AI solutions. This role will focus on transforming enterprise data into actionable insights using machine learning, natural language analytics, and self-service BI powered by Databricks Genie.

Key Responsibilities:
  • Design, develop, and deploy machine learning models using Databricks (MLflow, Spark ML, Python).
  • Implement end-to-end ML pipelines (data ingestion ? training ? deployment ? monitoring).
  • Collaborate with data engineers to ensure reliable, high-quality datasets in the Lakehouse.
  • Leverage Databricks Lakehouse (Delta Lake, Unity Catalog) for scalable analytics.
  • Optimize Spark jobs for performance and cost efficiency.
  • Apply best practices for data governance, lineage, and security.
  • Configure and enable Databricks Genie for self-service analytics.
  • Design semantic layers and curated datasets optimized for natural language queries.
  • Partner with business stakeholders to translate questions into Genie-enabled insights.
  • Work closely with product owners, analysts, and business leaders to identify high-value use cases.
  • Communicate complex analytical results in a clear, business-friendly manner.

Required Skills:
  • Hands-on experience with Databricks and Apache Spark.
  • Strong programming skills in Python (PySpark, Pandas, NumPy, Scikit-learn).
  • Experience building and deploying ML models in production.
  • Solid understanding of SQL and data modeling.
  • Experience with MLflow, model lifecycle management, and experimentation.

Qualifications:
  • Bachelor's or master's degree in data science, Computer Science, Statistics, Engineering, or a related field.
  • 4+ years of experience in data science or advanced analytics.

Preferred Skills:
  • Digital : Databricks

Similar Jobs

More Jobs at Prophecy Technologies

More Information Technology Jobs

Find similar Databricks Data Scientist jobs: