Databricks Senior Developer

CRISIL Limited

• $130K — $155K *
New York, NY 10025In-Person
Finance & Insurance
5 - 7 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 5-7 years of data engineering experience with 3+ years of hands-on Databricks proficiency in production settings.
  • Expertise in migrating on-premise data systems to cloud platforms.
  • Strong skills in Python, PySpark, and SQL; experience with converting SQL dialects to Spark SQL is a bonus.
  • Knowledge of cloud computing, especially within Azure; integration of cloud functions with data platforms is a plus.
  • Familiarity with CI/CD practices, data quality frameworks, and tools like Databricks Workflows, Airflow, or dbt.

Responsibilities

  • Develop and optimize data pipelines using Databricks and cloud technologies.
  • Lead the migration of on-premises data systems to cloud-based solutions.
  • Collaborate with teams to ensure data quality and adherence to best practices.
  • Design and implement CI/CD processes for data applications.
  • Monitor and troubleshoot data workflows to enhance performance.

Benefits

  • Opportunities for career development and continuous learning.
  • Collaborative and innovative work environment.
  • Access to cutting-edge tools and technologies.
  • Flexible work arrangements and work-life balance support.
Full Job Description
Job Description

Qualifications & Requirements

  • Experience
    5 to 7 years of data engineering experience, with at least 3+ years of hands-on Databricks experience in production environments.
  • Migration Expertise
    Proven experience in migrating on-premises data systems to cloud platforms.
  • Technical Skills
    Strong proficiency in Python, PySpark, and SQL. Experience in converting legacy SQL dialects to Spark SQL is a plus.
  • Cloud Knowledge
    Familiarity with cloud computing concepts, preferably within Azure ecosystems. Experience integrating cloud functions or runtime services with data platforms is desirable.
  • Best Practices & Tools
    Solid understanding of CI/CD, data quality frameworks, and orchestration tools such as Databricks Workflows, Airflow, or dbt.


Preferred Skills

  • Experience with large-scale data modernization programs
  • Exposure to financial services or regulated data environments
  • Strong problem-solving and stakeholder communication skills
  • Investment banking background would be added advantage.
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at CRISIL Limited

More Finance & Insurance Jobs

Find similar Databricks Senior Developer jobs:

NationwideNew York, NY