Texas Health and Human Services Commission requires the services of
2 Systems Analyst 3, hereafter referred to as Candidate(s), who meets the general qualifications of
Systems Analyst 3, Applications/Software Development and the specifications outlined in this document for the
Texas Health and Human Services Commission.
The Databricks Engineer should design, develop, and optimize scalable data solutions on Databricks, leveraging PySpark or Scala for large-scale data processing. Build and maintain ingestion pipelines, Declarative Pipelines (DLT), and Medallion Architecture (Bronze, Silver, Gold) to support enterprise analytics and reporting. Develop robust data models and implement data quality, validation, and governance frameworks. Create dynamic dashboards, Databricks Apps, and analytical solutions to deliver actionable business insights. Optimize workloads, monitoring, and operational processes to ensure scalability, security, and cost efficiency.
Required Qualifications- 8+ years of experience in IT, supporting the design, development, deployment, or delivery of technology solutions.
- 8+ years of experience with Databricks, including building and optimizing ETL/ELT data pipelines using Apache Spark.
- 8+ years of experience in data warehousing and dimensional data modeling, including star and snowflake schemas.
- 8+ years of proficiency in SQL and Python (or Scala) for large-scale data processing.
- 8+ years of experience designing and developing dashboards and applications natively within Databricks, such as Databricks SQL Dashboards and Databricks Apps.
- 8+ years of experience implementing data governance, data quality, and data security best practices.
- 8+ years of experience implementing Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT) to build and manage production data pipelines.
- 8+ years of experience with Delta Lake, medallion architecture (bronze, silver, and gold layers), data lakehouse design, and creating and scheduling offline jobs using Lakeflow Jobs (formerly Databricks Workflows) or similar orchestration tools such as Apache Airflow.
- Excellent verbal and written communication skills, with the ability to present technical concepts and insights to both technical and business stakeholders.
Preferred Qualifications- 1+ year of experience working in public sector or state government environments.
- Databricks certification, such as Databricks Certified Data Engineer Associate or Professional.
- 1+ year of experience with CI/CD practices for data pipelines, including DevOps and Git-based workflows.
