About the RoleYou'll design, build, and maintain the robust data pipelines and infrastructure required for large-scale AI research, applied AI, production systems. We are building a data lake that ingests a broad and growing set of financial and market datasets from many vendors and sources. You will take ownership of it end to end: hardening what exist today, making the core architectural calls on storage, orchestration, and data modeling, and building out the rest.
Your job is to ingest all of it, model it correctly, and make it queryable, versioned, and trustworthy for both humans and AI agents.
Responsibilities- Own the data platform end to end: ingestion, storage, transformation, cataloging, governance, and access across research and production.
- Build reliable pipelines: onboard large, heterogeneous datasets; handle schema changes, late data, revisions, backfills, and vendor failures.
- Guarantee research-grade correctness: point-in-time data, lineage, versioning, and reproducibility; prevent look-ahead and survivorship bias.
- Create trusted data products: normalize identifiers, timestamps, corporate actions, reference data, and unstructured sources into accessible datasets and APIs.
- Enable AI-native systems: build permission-aware interfaces, metadata, datasets, and benchmarks designed for both researchers and agents.
- Partner across the team: work directly with research and engineering to turn ambiguous problems into durable data capabilities.
Requirements- Exceptional software and data engineering skills: strong Python and SQL, plus experience with distributed processing and modern data infrastructure.
- Production ownership: demonstrated ability to design, deploy, operate, and improve business-critical data systems at scale.
- Temporal-data expertise: revisions, event time versus knowledge time, reproducibility, and point-in-time querying.
- Strong data judgment: messy, changing, unstructured, or adversarial data; rigor about correctness, provenance, and privacy.
- AI-native workflow: uses coding agents daily and builds data systems that are safe, well-described, and machine-consumable, for agents as much as humans.
- Financial-data experience strongly preferred: market data, security identifiers, corporate actions, or quantitative research.
