Youll need to have:
Bachelors or advanced degree in Computer Science, Data Engineering, or a related field.
415 years of professional data engineering experience.
Strong Python, SQL, and Spark (PySpark) skills, and/or Kafka.
Snowflake (Snowpipe, Tasks, Streams) as a complementary warehouse.
Databricks (Delta formats, workflows, cataloging) or equivalent Spark platforms.
Hands-on experience building ETL/ELT with Prefect (or Airflow), dbt, Spark, and/or Kafka.
Experience onboarding datasets to cloud data platforms (storage, compute, security, governance).
Familiarity with Azure/AWS/GCP data services (e.g., S3/ADLS/GCS; Redshift/BigQuery; Glue/ADF).
Git-based workflows CI/CD and containerization with Docker (Kubernetes a plus).
It is a bonus to have:
Advanced APIM practices (custom policies, OAuth2/JWT, mTLS, private endpoints) and Azure AD integration.
Integrating datasets into MCP tools/providers for LLM/agent applications; familiarity with frameworks such as LangChain or LlamaIndex.
Data observability/quality tools (e.g., Great Expectations, Monte Carlo, Datafold) and strong lineage practices.
Exposure to financial datasets and controls (PII handling, encryption, masking).
Salary Range $135,000-$150,000 a year
