Data Engineer

The Data Engineer builds and maintains the data infrastructure that supports operational and digital analytics across the organization. This position designs, develops, and operates scalable data pipelines and ingestion workflows on a Microsoft Azure-based platform to ensure data is reliable, governed, and available for analytics and reporting. This role partners closely with IT, Finance, Operations, and Digital stakeholders to support the organization's continued migration to a centralized, cloud-native data architecture.

Work Location and Onsite Requirement: This position is Hybrid and requires a few days in the Alexandria VA office.

Essential Duties and Responsibilities

• Design, develop, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of structured and semi-structured data.
• Ingest data from diverse sources, including relational databases, REST APIs, flat files, and third-party platforms.
• Build and manage Azure Data Factory pipelines to orchestrate, schedule, and monitor data workflows.
• Develop Azure Functions and Logic Apps to support event-driven ingestion and lightweight system integrations.
• Administer and configure Databricks workspaces, including cluster management, access controls, and workspace governance.
• Develop data processing and transformation logic using Python and PySpark in Databricks notebooks and jobs.
• Apply data architecture standards and evaluate technical trade-offs related to medallion architecture, data modeling, and governance best practices.
• Implement and maintain data governance controls, including access controls, data classification, and lineage tracking.
• Manage Azure Storage resources, including Azure Data Lake Storage Gen2, to support proper organization, partitioning, and lifecycle management.
• Validate data quality, completeness, and reliability throughout the pipeline and support discrepancy resolution with analysts, developers, and downstream consumers.
• Collaborate with analysts and developers to deliver curated, conformed datasets for reporting and analytics.
• Troubleshoot pipeline failures, performance bottlenecks, and data discrepancies.
• Document architecture decisions, data flows, pipeline logic, and operational runbooks.

Qualifications Guidelines

To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skills, and/or abilities required. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.

Education/Experience

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field required
• 3+ years of experience in data engineering roles required
• Hands-on experience with Microsoft Azure data services required
• Experience supporting Finance, Operations, or Digital reporting environments preferred
• Experience in multi-unit retail or restaurant environments preferred

Language Ability

Excellent verbal and written communication skills are required. This position requires the ability to translate business requirements into technical solutions, collaborate effectively with IT, Finance, Operations, and Digital stakeholders, and document technical work clearly for both technical and non-technical audiences.

Math Ability

This position requires strong analytical skills, including the ability to interpret data, evaluate metrics, identify trends, and assess risks across data pipelines and reporting environments.

Reasoning Ability

This position requires strong analytical and problem-solving skills, including the ability to assess complex data issues, evaluate technical trade-offs, and make sound decisions related to data architecture, pipeline design, data quality, and system integrations. The candidate must be able to troubleshoot discrepancies, identify root causes, prioritize risks, and develop reliable solutions in collaboration with technical and business stakeholders.

Computer Skills

Proficiency in Microsoft Office Suite is required. Strong proficiency in Python, PySpark, advanced SQL, Microsoft Azure data services, Databricks, Azure Data Factory, Azure Functions, Logic Apps, and Azure Data Lake Storage Gen2 is required. Experience with Databricks workspace administration, cluster configuration, Unity Catalog, access management, SQL sources, REST APIs, CSV/XML formats, data quality frameworks, governance controls, validation strategies, and cloud-based access controls is preferred.

Supervisory Responsibilities

This position has no supervisory responsibilities.

Physical Demands

This position is primarily office-based and requires the ability to remain in a stationary position for prolonged periods while using a computer, telephone, and standard office equipment. The position may occasionally require standing, walking, reaching, grasping, handling objects, and lifting or moving up to 20 pounds. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.

Special Requirements/Certification

Power BI experience, including data modeling, DAX measures, and report publishing in Power BI Service, is preferred. Databricks and Microsoft Azure certifications, such as Databricks Certified Data Engineer Associate/Professional, Databricks Certified Associate Developer for Apache Spark, Azure Data Engineer Associate (DP-203), or Azure Fundamentals (AZ-900), are preferred but not required.

To address your goal of ensuring only candidates who live near the office apply-given that the position is hybrid and requires in-office days-it's important to make this requirement explicit in the job description. This will help filter out applicants from distant locations and set clear expectations.

Recommended Addition to the Job Description:

Example Implementation:

You can add the above section under the "Special Requirements/Certification" or "Physical Demands" section, or as a separate section titled "Work Location and Onsite Requirement" in your job posting. This makes the expectation clear to all applicants and helps reduce applications from candidates who are not local.

Why This Works:

• It sets clear geographic expectations for applicants.
• It reinforces the hybrid/in-office requirement, reducing confusion.
• It helps your recruiting team focus on qualified, local candidates.

Sample Placement in Job Description:

Work Location and Onsite Requirement: This position is hybrid and requires regular in-office attendance. Only candidates who reside within a reasonable commuting distance to the office will be considered. Applicants from outside the local area or those seeking fully remote work will not be considered for this role.

The above job description is not intended to be an all-inclusive list of duties and standards of the position. Incumbents will follow any other instructions and perform other related duties as assigned by their supervisor, policy, or company management. Work location and onsite attendance requirements will be determined by the company in accordance with company practice and business needs.