TAIT is looking for a hands-on Data Engineer to build, scale, and operate the governed data platform that will underpin the next phase of the company's technology transformation. Reporting to the Data Architect, this role will turn the target data architecture into reliable pipelines, reusable data products, trusted analytical models, and production platform capabilities.
This role is ideal for an experienced engineer who can move between architecture and execution and who has a proven track record delivering production-grade Databricks solutions in AWS. The successful candidate will do more than move data from one system to another: they will have delivered projects that combine complex enterprise and operational data to enable sophisticated analysis, forecasting, optimization, AI/ML use cases, and confident executive decision-making.
The Data Engineer will work closely with the Data Architect and partner across Enterprise Technology, Architecture/AI, Integration, Infrastructure and Cloud, Information Security, and business teams including Finance, Sales, Project Delivery, Operations, Procurement, Manufacturing, and People/Resourcing. The role will help establish clear sources of truth and enable analytics across systems such as Salesforce CRM and CPQ, Kantata PSA, Epicor and the next-generation ERP, MES, Autodesk/PLM, HCM, service management platforms, and other internal and external data sources.
Responsibilities: Databricks and AWS Data Platform Engineering
- Design, build, deploy, and operate production data pipelines and lakehouse capabilities using Databricks on AWS.
- Implement scalable ingestion, transformation, orchestration, and serving patterns for batch, near-real-time, streaming, API, file, and change-data-capture workloads.
- Use AWS services and controls - including S3, IAM, KMS, VPC networking, Secrets Manager, CloudWatch, and appropriate catalog or integration services - to create secure and supportable data solutions.
- Develop reliable data layers, optimizing workloads, storage design, cluster or serverless configuration, performance, and platform cost.
- Establish automated deployment practices using Git, CI/CD, infrastructure as code, automated testing, and environment promotion across development, test, and production.
- Implement monitoring, observability, alerting, recovery, service-level objectives, and operational runbooks for critical data products and pipelines.
Analytics Enablement and Data Products
- Translate high-value business questions into well-defined data products, curated datasets, and semantic-ready models for Power BI, advanced analytics, and AI-enabled solutions.
- Integrate data from multiple enterprise and operational sources to support strategic analytics, dashboards, KPIs, project and customer profitability, resource and capacity analysis, financial planning, forecasting, scenario modeling, operational optimization, and other sophisticated analytical use cases.
- Partner with analysts, product owners, and business subject-matter experts to ensure data products are analytically fit for purpose, clearly defined, and adopted by their intended users.
- Create reusable datasets and feature-ready data that accelerate experimentation and responsible delivery of machine learning, generative AI, and other advanced analytical capabilities.
- Define and track technical and business measures for data-product adoption, quality, timeliness, reliability, and realized value.
Roadmap Delivery
- Partner with the Integration team and Data Architect to define data contracts, APIs, event patterns, and reusable integration standards, including appropriate alignment with MuleSoft and other enterprise integration capabilities.
- Support major platform implementations and migrations by developing conversion pipelines, reconciliation routines, archival strategies, historical-data models, and post-go-live analytical capabilities.
- Create repeatable onboarding patterns that accelerate M&A data discovery, integration, harmonization, and reporting across acquired businesses.
- Reduce data and technology debt by replacing brittle point-to-point extracts, undocumented transformations, and duplicate datasets with governed, reusable platform services.
Data Quality, Governance, and Security
- Work with the Data Architect to implement enterprise data standards, naming conventions, reference architectures, reusable patterns, and engineering guardrails.
- Build automated data-quality controls, source-to-target reconciliation, exception handling, freshness checks, and issue-management processes for critical data domains.
- Implement metadata management, cataloging, lineage, ownership, and access controls using Databricks governance capabilities and supporting enterprise tools.
- Apply security by design, including least-privilege access, encryption, secrets management, environment separation, auditability, retention, and appropriate handling of confidential or regulated data.
- Help define and maintain trusted sources of truth, master and reference-data practices, data definitions, and documentation that make strategic metrics traceable and explainable.
Delivery, Collaboration, and Continuous Improvement
- Own engineering work from discovery through production support, including estimation, design, development, testing, documentation, deployment, adoption, and continuous improvement.
- Communicate technical tradeoffs, dependencies, risks, and recommendations clearly to both technical and non-technical audiences.
- Participate in design reviews, code reviews, incident reviews, architecture forums, and roadmap planning; provide practical feedback that improves quality without slowing delivery unnecessarily.
- Work effectively with global internal teams, nearshore and offshore partners, software vendors, and implementation partners while maintaining clear engineering accountability and standards.
- Mentor less-experienced engineers and analysts, share reusable practices, and help build a strong data-engineering discipline within GTS.
Required Qualifications: - Bachelor's degree in computer science, engineering, information systems, data science, or a related discipline, or equivalent practical experience.
- 7+ years of professional experience in data engineering, analytics engineering, data platform engineering, or a closely related role.
- A proven track record designing, delivering, and operating production-grade solutions in Databricks on AWS, including responsibility for architecture implementation, pipeline development, deployment, support, performance, and cost.
- Evidence of delivering multiple end-to-end projects where engineered data products enabled sophisticated analysis - such as cross-system KPI reporting, forecasting, optimization, profitability analysis, anomaly detection, predictive modeling, scenario analysis, or AI/ML - with clear business outcomes.
- Advanced SQL and strong Python/PySpark skills, including experience developing maintainable, tested, production-quality code for large and complex datasets.
- Strong knowledge of Spark, Delta Lake or comparable lakehouse patterns, dimensional and analytical data modeling, partitioning, schema evolution, data contracts, and performance tuning.
- Hands-on experience with core AWS data-platform capabilities such as S3, IAM, KMS, VPC networking, CloudWatch, Secrets Manager, and related services used to secure, integrate, and operate Databricks workloads.
- Experience building batch and incremental pipelines using APIs, files, databases, SaaS connectors, change data capture, and at least one orchestration framework or managed workflow capability.
- Experience with Git-based development, CI/CD, automated data testing, code review, environment management, and infrastructure-as-code practices.
- Practical experience implementing data quality, observability, cataloging, lineage, access control, and operational support for business-critical data products.
- Strong problem-solving, documentation, and communication skills, including the ability to translate ambiguous business needs into pragmatic technical solutions and explain data issues in business terms.
- Ability to work independently in a changing environment, prioritize based on business value and risk, and collaborate effectively across architecture, integration, infrastructure, security, applications, analytics, and business teams.
Preferred Qualifications: - Databricks certification and/or AWS certification relevant to data engineering, analytics, or cloud architecture.
- Experience with Unity Catalog or comparable governance capabilities, including fine-grained access, lineage, cataloging, and multi-environment administration.
- Experience preparing governed data models for Power BI, including semantic-model design, incremental refresh patterns, performance optimization, and row-level security.
- Experience integrating enterprise applications such as Salesforce, Epicor or another ERP, Kantata or another PSA, CPQ, MES, PLM, HCM, EPM, and service management platforms.
- Experience with MuleSoft or another enterprise integration platform and with designing API- or event-driven data exchange patterns.
- Experience enabling machine learning or AI solutions through feature engineering, model-ready datasets, ML lifecycle tooling, vector or unstructured-data pipelines, or responsible AI data controls.
- Experience in a global, project-based, manufacturing, engineering, live-entertainment, or operationally complex business environment.
- Experience supporting enterprise transformation, ERP replacement, legacy-platform retirement, cloud migration, and/or M&A integration.
#LI-ML1
