Consolidated Precision Products Corp.

Azure Databricks Engineer (Dallas, TX)

Consolidated Precision Products Corp. • $120K — $160K *
Dallas, TX 75217In-Person
Information Technology
8 - 10 years of experience
More than 3 months ago
Job Overview by Ladders

Qualifications

  • Minimum 10 years of hands-on experience in data engineering and cloud-based platforms.
  • At least 5 years of experience with Azure Databricks, building data pipelines.
  • Strong expertise in Azure services like Data Lake, Synapse Analytics, and SQL Database.
  • Proficiency in Spark, PySpark, Scala, or SQL for large-scale data processing.
  • Expertise in Databricks notebooks, clusters, and libraries.
  • Familiarity with data orchestration tools like Azure Data Factory.

Responsibilities

  • Design and optimize scalable data pipelines and data lakes using Azure Databricks.
  • Lead the implementation of end-to-end analytics solutions on Azure.
  • Collaborate with data scientists to translate business needs into technical solutions.
  • Manage and optimize ETL/ELT processes across data sources.
  • Identify and resolve performance issues in Databricks environments.
  • Mentor junior engineers on best practices in data engineering.
  • Implement monitoring systems to ensure the integrity of data workflows.

Benefits

  • Professional development opportunities and mentorship.
  • Access to cutting-edge technology for innovative projects.
  • Flexible working arrangements to enhance work-life balance.
  • Comprehensive health and wellness programs.
  • Participation in a collaborative and diverse team environment.
Full Job Description
Key Responsibilities:
  • Azure Databricks Architecture & Design:
    • Design, build, and optimize scalable, high-performance data pipelines and data lakes using Azure Databricks.
    • Architect and implement end-to-end analytics solutions leveraging Databricks and other Azure services (e.g., Azure Data Lake, Azure SQL, Azure Blob Storage).
    • Lead the design of cloud-based architectures using Azure Databricks for data processing, transformation, and reporting.
  • Data Engineering & Integration:
    • Design and implement ETL/ELT processes to ingest, process, and transform data across various sources (structured and unstructured).
    • Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and develop data solutions.
    • Manage data integration workflows between Databricks and other platforms like Power BI, Azure SQL, Synapse, etc.
  • Optimization & Performance Tuning:
    • Identify performance bottlenecks in Databricks environments and optimize clusters, queries, and code.
    • Continuously improve and scale data pipelines to accommodate growing data volumes and business needs.
  • Collaboration & Leadership:
    • Mentor and guide junior engineers and team members in best practices related to Azure Databricks and data engineering.
    • Work closely with cross-functional teams (data science, analytics, business intelligence) to deliver integrated solutions.
    • Provide technical leadership in cloud data architecture and data engineering.
  • Monitoring & Security:
    • Implement monitoring and alerting systems to ensure data pipelines and workflows are running smoothly.
    • Ensure compliance with data security and governance standards across the data ecosystem.
  • Continuous Improvement & Innovation:
    • Stay up to date with the latest advancements in Azure Databricks, cloud technologies, and data engineering trends.
    • Continuously evaluate and introduce new technologies and methodologies to enhance the data platform.
Required Skills and Qualifications:
  • Experience:

    • Minimum 10 years of hands-on experience in data engineering and cloud-based data platforms.
    • At least 5 years of experience working with Azure Databricks, building data pipelines, and performing data engineering tasks.
    • Strong experience with Azure services such as Azure Data Lake, Azure Synapse Analytics, Azure Blob Storage, Azure SQL Database, etc.
  • Technical Skills:

    • Proficiency in Spark, PySpark, Scala, or SQL for large-scale data processing.
    • Expertise in Databricks notebooks, clusters, job scheduling, and libraries.
    • Deep understanding of cloud-native data engineering practices and architecture on Azure.
    • Familiarity with data modeling, data lakes, and data warehouse concepts.
    • Expertise in Azure services: Data Factory, Data Lake, Synapse Analytics, Event Hubs, Cosmos DB.
  • Data Pipeline and ETL/ELT Development:

    • Experience building and optimizing ETL/ELT workflows on Databricks.
    • Knowledge of data orchestration tools such as Azure Data Factory, Apache Airflow, or similar.
  • Big Data Technologies:

    • Proficiency in handling large-scale distributed data processing with tools like Apache Spark.
    • Experience with technologies such as Kafka, Delta Lake, and Databricks Runtime.
    • Strong understanding of Delta Lake and Lakehouse architecture.
  • Programming Languages:

    • Strong programming skills in Python, Scala, or Java.
    • Experience with SQL-based querying and optimizations.
  • Cloud & DevOps:

    • Strong understanding of Azure cloud services and architecture.
    • Familiarity with DevOps principles, CI/CD pipelines, and automation tools. Terraform required.
    • Knowledge of Git, Azure DevOps, or similar version control and deployment systems.
  • Collaboration & Leadership:

    • Excellent communication skills to work with both technical and non-technical stakeholders.
    • Proven track record of mentoring and leading teams of data engineers.
    • Experience managing complex projects and working with cross-functional teams.
Preferred Qualifications:
  • Certifications:

    • Microsoft Certified: Azure Data Engineer Associate or equivalent certification.
    • Databricks Certified Associate Developer for Apache Spark.
  • Education:

    • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

About Consolidated Precision Products Corp.

Consolidated Precision Products Corp. is a manufacturer of complex metal components and products for the aerospace and defense industries. The company was founded in 1991 and is headquartered in Carrollton, Texas. Consolidated Precision Products Corp. operates a network of manufacturing facilities across the United States, as well as in Mexico and Europe. The company's products include castings, forgings, and machined components, as well as assemblies and sub-assemblies. Consolidated Precision Products Corp. is committed to providing high-quality products and services to its customers.
Learn more about Consolidated Precision Products Corp.
Size
3,000 employees
Industry
Aerospace & Defense
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Information Technology Jobs

Find similar Azure Databricks Engineer (Dallas, TX) jobs:

NationwideDallas, TX