Director, Data Engineer/Architect (Databricks Pro Certified)

Infinitive Inc

$170K — $215K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
  • Proven experience as a Data Architect, focused on Databricks.
  • Proficiency in Databricks and optimizing Spark applications.
  • Experience with AWS services like S3, Redshift, and Glue.
  • Strong knowledge of Unity Catalog for data asset management.
  • Familiarity with data modeling, ETL processes, and data warehousing.
  • Databricks Certified Professional.

Responsibilities

  • Design, develop, and implement data architectures in Databricks on AWS and Azure.
  • Collaborate with stakeholders to gather business requirements and convert them to technical specs.
  • Develop and maintain ETL processes for efficient data integration.
  • Ensure data quality, integrity, and security in all activities.
  • Optimize data storage and retrieval for performance and cost-efficiency.
  • Lead data governance policy implementation.
  • Mentor data engineering teams and provide technical guidance.

Benefits

  • Comprehensive health benefits including medical, dental, and vision coverage.
  • 401(k) plan with company match to support retirement savings.
  • Employee development programs for skill enhancement and career growth.
  • Flexible work hours and potential for remote work options.
  • Opportunities to work with the latest industry technologies and innovations.
Full Job Description
*Candidates must possess US Citizenship with the ability to gain a government security clearance. Job Summary: We are seeking a highly skilled Data Engineer/Architect with extensive experience in Databricks on AWS or Azure to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining scalable data architectures, ensuring data integrity, and optimizing data flow and collection. This role requires a deep understanding of cloud-based data solutions and strong expertise in leveraging Databricks to drive data initiatives. Key Responsibilities: • Design, develop, and implement data architectures and solutions on Databricks in both AWS and Azure. • Collaborate with stakeholders to understand business requirements and translate them into technical specifications. • Develop and maintain ETL processes to ensure efficient data flow and integration across systems. • Ensure data quality, integrity, and security in all data-related activities. • Optimize data storage and retrieval strategies to improve performance and cost-efficiency. • Lead the implementation of data governance policies and procedures. • Provide technical guidance and mentorship to data engineering teams. • Stay updated with the latest industry trends and best practices in data architecture, Databricks, cloud data warehouses, AWS and Azure. Required Skills and Qualifications: • Bachelor's or Master's degree in Computer Science, Information Technology, or related field. • Proven experience as a Data Architect, with a strong focus on Databricks. • Proficiency in Databricks, including the ability to build and optimize Spark applications. • Extensive experience with AWS services such as S3, Redshift, EMR, Lambda, Glue, and others. • Strong knowledge of Unity Catalog and its application in managing and securing data assets. • Strong knowledge of data modeling, ETL processes, and data warehousing concepts. • Proficiency in SQL and experience with big data technologies (e.g., Hadoop, Spark). • Familiarity with data governance and data security practices. • Databricks Certified Professional. • Excellent problem-solving skills and attention to detail. • Strong communication and collaboration skills. Preferred Qualifications: • AWS Certified Solutions Architect or similar certification. • Experience with other cloud platforms (e.g., Azure, GCP). • Knowledge of machine learning and data science concepts. • Experience with data visualization tools (e.g., Tableau, Power BI). • Prior experience in a leadership or mentorship role. Infinitive is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. The determination of this range includes various factors not limited to skill set, level, experience, relevant training, and licensure and certifications. Compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range for this role in the U.S. is $170,000.00- $215,000.00.

Similar Jobs

More Jobs at Infinitive Inc

More Information Technology Jobs

Find similar Director, Data Engineer/Architect (Databricks Pro Certified) jobs: