OverviewThe Senior Data Governance Engineer, Databricks & AWS is responsible for managing AIT's enterprise data dictionary, metadata, governance standards, and Databricks/AWS configuration, provisioning, and access controls. The role ensures enterprise data is trusted, secure, well-documented, and accessible by strengthening data governance, quality, and compliance practices.Additionally, the position supports governance initiatives through automation, scripting, and targeted pipeline support, while leveraging AI-enabled tools such as Databricks Genie to enhance data discovery, documentation, and self-service analytics. Working closely with business and technology stakeholders, the role provides technical leadership, mentors teammates, establishes scalable governance standards, and may evolve to include people management responsibilities as AIT's data platform grows.
Responsibilities
- Enterprise Data Dictionary, Metadata & Lineage: Own the development, implementation, and ongoing maintenance of AIT's enterprise data dictionary. Establish consistent definitions, naming standards, metadata, business terminology, and technical documentation for enterprise data assets. Develop and maintain lineage and source-to-target documentation that improves transparency and trust in enterprise data. Partner with business stakeholders, data owners, and subject matter experts to validate definitions and resolve inconsistencies. Integrate data dictionary, metadata, catalog, and governance practices with Unity Catalog, Databricks, and related enterprise tooling. Drive adoption of common data definitions and make trusted information easier for teammates to discover and understand. Establish the review cadence, change-control process, and ownership model that keep definitions current as the business evolves.
- Platform Configuration, Provisioning & Access Management: Administer Databricks workspace and Unity Catalog configuration, including catalogs, schemas, grants, groups, service principals, and related platform settings. Own user and workload provisioning and permissioning across the platform onboarding, role and entitlement assignment, periodic access review, and deprovisioning. Administer and secure the AWS environment supporting AIT's Databricks platform, including IAM, S3, networking, connectivity, and permissions. Establish and maintain security, access, resource, and configuration standards, and document them so access decisions are consistent and auditable. Monitor platform capacity, utilization, and cloud costs, and apply FinOps and cost-governance controls for Databricks and associated AWS resources. Partner with infrastructure, security, and technology teams to ensure the platform meets AIT requirements for availability, security, scalability, and operational support.
- AI-Enabled Data Discovery & Delivery: Use Databricks Genie and other approved AI and GenAI capabilities to improve enterprise data discovery, analytics, documentation, and self-service. Configure and curate AI-enabled data experiences that allow users to interact more effectively with governed enterprise data. Ensure the dictionary and metadata layer is structured to support reliable AI-assisted discovery and natural- language querying. Identify opportunities to automate or accelerate documentation and data-management activities through AI- assisted tooling. Establish appropriate patterns and guardrails for reliable use of AI-enabled data capabilities. Evaluate emerging Databricks and AI capabilities and recommend adoption where they provide measurable business value.
- Data Platform Engineering & Automation: Build and maintain targeted data pipelines and transformations in Databricks using SQL, Python/PySpark, and Delta Lake to support governance, documentation, and reporting needs. Automate recurring platform, provisioning, and documentation tasks through scripting, platform APIs, and Infrastructure as Code such as Terraform. Implement data quality controls, monitoring, and alerting, and drive data reliability issues to resolution. Apply sound engineering practices for source control, testing, deployment, and observability in the work this role produces. Support existing pipelines and troubleshoot platform, access, and data-processing issues in partnership with the broader data engineering team. Produce clear technical documentation and operational procedures that improve platform sustainability and supportability.
- Collaboration & Technical Leadership: Partner with technology teams, analysts, business stakeholders, data owners, and other subject matter experts to translate requirements into governed, well-documented data. Contribute to AIT's data governance, security, access-control, retention, lineage, and lifecycle-management standards. Provide technical leadership, mentoring, and knowledge sharing to teammates working with AIT's data platform. Communicate complex technical and data-definition concepts clearly to both technical and non-technical audiences. Proactively identify platform and governance risks, gaps, and opportunities and recommend practical paths forward. As the data platform function grows, take on additional leadership responsibilities, which may include future direct reports.
- Performs other duties as assigned
- Complies with all policies and standards
- The wage range for this position is $118,330-$177,490 and may vary based on geography as well as relative knowledge, skills, abilities, and experience.
Qualifications
Education High School (required) Bachelor's Degree Computer Science, Engineering, Information Systems, Data Science, or a related technical field (required)
Experience 5+ years Data governance, data management, analytics engineering, or data platform experience, including hands-on work with modern cloud data technologies (required) 3+ years Administering or configuring cloud-based data platforms on AWS, including IAM, S3, networking, security, and access management (required) 3+ years Hands-on Databricks experience, including Unity Catalog administration, permissions, and Delta Lake (required) 3+ years Developing or maintaining enterprise data dictionaries, metadata, data catalogs, business glossaries, lineage, or related data-governance capabilities (required) 1+ years Using AI/GenAI-enabled tools, such as Databricks Genie, for data discovery, analytics, engineering, or documentation (preferred)
Knowledge, Skills, and Abilities Databricks administration, including Unity Catalog, permissions, and Delta Lake - Advanced Medium AWS platform administration, including IAM, S3, networking, security, and FinOps - Advanced Medium SQL, Python, and PySpark for data transformation, scripting, and automation - Intermediate Medium Data modeling, metadata management, data dictionaries, business glossaries, and data catalogs - Expert Medium Databricks Genie and AI-assisted data tooling- Advanced Medium CI/CD, Git, automation, and Infrastructure as Code such as Terraform - Intermediate Medium Data governance, data quality, security, and lineage - Expert Medium Access provisioning, permissioning, and identity/entitlement administration - Advanced Medium Problem solving, navigating ambiguity, and creating structure - Advanced Medium Technical documentation and knowledge sharing - Advanced Medium Communication and cross-functional collaboration - Advanced Medium Performance optimization, troubleshooting, and root-cause analysis - Advanced Medium
Licenses and Certifications - Databricks Certified Data Engineer 6 Associate (or higher) (required) - AWS Certified Solutions Architect 6 Associate (or higher) (preferred)
