OverviewAdastra is looking for a hands-on Data Engineer to join our team supporting a modern enterprise data platform. You'll design and run Databricks pipelines that turn raw operational, marketing, and retail data into trusted, analytics-ready data products the business relies on every day.
Working as an embedded member of the data team, you'll own pipelines end to end on the Databricks Lakehouse Platform. If you love clean data, well-tuned Spark jobs, and shipping solutions that people actually use, this role is for you.
Primary Location: Across Canada or USA
Work Model: Hybrid 2-Days Onsite
Employment Type: Full-Time or Contractor
Vacancy Status: New
RESPONSIBILITIES
- Design, build, and maintain scalable batch and streaming pipelines on Databricks using PySpark and Spark SQL within a medallion (Bronze/Silver/Gold) architecture
- Ingest and unify data from operational databases, cloud storage, APIs, and marketing platforms such as Braze
- Model and optimize Delta Lake tables for performance, reliability, and cost efficiency using techniques such as partitioning, liquid clustering, OPTIMIZE, and VACUUM
- Deliver analytics-ready data for reporting on sales, shipments, orders, and other key business metrics, supporting Power BI dashboards and downstream analytics
- Orchestrate jobs using Databricks Workflows, managing dependencies, retries, alerting, and SLAs
- Apply Unity Catalog for data governance, lineage, and access control
- Work to a high engineering standard using Git, CI/CD, testing, code reviews, and clear documentation
- Monitor pipeline health, troubleshoot issues, and tune Spark jobs and cluster configurations for speed and cost optimization
- Partner with business and technical stakeholders to deliver reliable, high-quality data products
QUALIFICATIONS, SKILLS & EXPERIENCE
- Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field
- 5+ years of data engineering experience building and operating production data pipelines
- Hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake, Databricks Workflows, and Unity Catalog
- Advanced experience with PySpark and Spark SQL
- Strong SQL development and performance tuning skills
- Solid Python development skills and experience with modern engineering practices
- Experience using Git, CI/CD pipelines, testing frameworks, and code review processes
- Experience working with AWS, Azure, or GCP and their core compute and storage services
- Strong understanding of data modeling concepts, including dimensional and medallion architectures
- Experience designing ELT/ETL solutions and integrating data from databases, APIs, files, and event streams
- Strong communication skills and the ability to work independently within a client-facing environment
NICE TO HAVE
- Databricks Certified Data Engineer Associate or Professional certification
- Experience with Structured Streaming, Auto Loader, Kafka, or Kinesis
- Experience working with marketing or customer data platforms such as Braze
- Retail, grocery, or consumer packaged goods (CPG) industry experience
- Experience supporting Power BI reporting and self-service analytics initiatives
