What you will be doing: - Own the end to end architecture: ingestion, lakehouse and warehouse layers, transformation, modeling, and delivery to reporting and analytics tools.
- Build it. Write and deploy pipelines, Spark and SQL transformations, and data models, and review the code the rest of the team writes.
- Design lakehouse architectures on Databricks or Fabric, including medallion layering, Delta table design, and the semantic layer that sits on top.
- Make the platform call and defend it. Databricks, Fabric, or a mix, based on the client's workloads, existing licensing, skills, and cost profile.
- Set up governance and access control using Unity Catalog, Purview, or the equivalent, and apply it consistently rather than bolting it on at the end.
- Design and build ingestion from a wide range of sources: databases, SaaS applications, APIs, files, and legacy systems that were never meant to be integrated.
What we need from you:- Eight or more years in data engineering, with at least three years owning architecture and design decisions.
- Proven hands on delivery. You should be able to point at platforms you designed and also built.
- Deep hands on experience with Databricks or Microsoft Fabric, and working knowledge of the other. Comfortable explaining where each one fits and where it does not.
- Strong Spark and SQL. Solid Python or an equivalent language used in production data work.
- Experience with Delta Lake, medallion architecture, and lakehouse patterns in production, not just in a demo.
- Power BI, including how the semantic model is built and where it should live.
- Solid dimensional and semantic modeling skills. You should have opinions about where business logic belongs and be able to explain why.
- Working knowledge of data governance: classification, lineage, cataloging, and access control.
- CI/CD, version control, and testing practices applied to data work, not just to application code.
What sets you apart from other applicants:- Azure Government or GCC High delivery experience.
- Unity Catalog or Purview implementation experience.
- Azure Data Factory, Synapse, or Azure SQL background.
- Consulting or client facing delivery experience.
- Experience in a regulated environment with real compliance constraints.
- Databricks or Microsoft certifications.
What you will receive: - Love your career: Competitive total rewards package. Blog during work hours; take a day off and volunteer for your favorite charity.
- Love your work/life balance: Flexibly work remotely from your home, there's no daily travel requirement to an office! All you need is a stable internet connection.
- Love your coworkers: Collaborate with some of the best and brightest in the industry!
- Love your development: Hone your skills or learn new ones with our substantial training allowance; participate in professional development days, attend training, become certified, whatever you like!
- Love your workspace: We give you all the equipment you need to work from home including a laptop with your choice of OS, and an annual budget to personalize your work environment!
- Love yourself: Pythian cares about the health and well-being of our team. You will have an annual wellness budget to make yourself a priority (use it on gym memberships, massages, fitness and more). Additionally, you will receive a generous amount of paid vacation and sick days, as well as a day off to volunteer for your favorite charity.
The pay range for this role is:
160,000 - 200,000 USD per year (USA)
