About the role:
This is a full-time position for a Principal Data Engineer / Solution Architect specializing in Microsoft Fabric and Azure data architectures to support a fast-moving project for a key client in the Life Sciences / Biotech industry.
In this role, you will serve as a senior technical resource responsible for supporting, enhancing, and executing data engineering solutions across Microsoft Fabric, Fabric Data Factory, Microsoft SQL, ADLS Gen2, Databricks Unity Catalog, PySpark in Fabric, and related Azure ecosystem services. The project operates at high velocity, requiring someone who can ramp up quickly, navigate technical ambiguity, and keep active delivery moving seamlessly. You will help stabilize, troubleshoot, and advance an active migration from traditional Azure infrastructure to Microsoft Fabric.
Location & Logistics: Candidates must be based in Boston, MA or fully capable of traveling to Boston as required by project milestones, maintaining 100% working time overlap with US East Coast business hours.
Qualification / Skill Set Requirement:
- Core Technical Expertise:
- 8+ years of hands-on data engineering, data architecture, or complex data platform development experience.
- Strong, practical experience building, supporting, and troubleshooting data pipelines in Microsoft Fabric and Azure Data Factory (ADF).
- Advanced proficiency in Microsoft SQL (query development, stored procedures, schema design, indexing, performance tuning) and Python / PySpark in Fabric.
- Hands-on experience managing and supporting Databricks Unity Catalog (catalog/schema/table organization, access controls, permissions, and governance administration).
- Deep familiarity with Azure Data Lake Storage Gen2 (ADLS Gen2), including data organization, security, and cloud service integrations.
- Architecture & Migration Experience:
- Proven experience supporting or leading platform migrations, modernizations, or transitions (specifically Azure-to-Microsoft Fabric).
- Familiarity with medallion architecture, lakehouse patterns, and modern data warehouse design.
- Experience with Azure administration activities supporting data platforms (access management, service configuration, and monitoring).
- Industry & Domain Experience:
- Required: Prior hands-on experience working with Life Sciences, pharmaceutical, biotechnology, medical device, clinical, or regulatory data environments.
- Knowledge of appropriate data handling, quality, compliance awareness, and business context in regulated settings.
- Logistics & Operating Style:
- Based in Boston, MA or willing/able to travel to Boston, MA for client engagements and alignment meetings.
- Full working time overlap with US East Coast business hours (ET).
- Demonstrated ability to operate effectively in a startup-speed environment with shifting priorities, tight timelines, and minimal ramp-up time.
- Certifications Preferred:
- Microsoft Certified: Azure Solutions Architect Expert
- Microsoft Certified: Fabric Data Engineer Associate
- Microsoft Certified: Azure Data Engineer Associate
- Microsoft Certified: Azure AI Engineer Associate: nice to have
Responsibilities
- Fabric & Pipeline Engineering: Design, build, enhance, and support scalable data pipelines and integrations using Microsoft Fabric Data Factory, PySpark in Fabric, and Azure Data Factory.
- Migration Support & Stabilization: Drive active migration efforts to Microsoft Fabric, including pipeline transition, validation, performance tuning, and operational stabilization.
- Databricks Governance: Manage and organize Databricks Unity Catalog structures, permissions, and access control policies.
- Data Asset Management: Develop and optimize Microsoft SQL data assets, complex stored procedures, queries, and ADLS Gen2 storage layers.
- Troubleshooting & Quality Assurance: Quickly diagnose and resolve issues across data pipelines, SQL queries, Fabric PySpark notebooks, Unity Catalog, and cloud integration layers.
- Technical Leadership & Knowledge Transfer: Assist with knowledge transfer from outgoing project resources, document technical designs and operational workflows, and mentor delivery team members.
- Stakeholder Collaboration: Partner closely with technical, project, and business stakeholders to translate complex requirements into reliable, compliant data solutions during US Eastern business hours.
