About the roleYou are a skilled and motivated Data Platform Engineer. You will:
- Design, build, and maintain data pipelines at scale. We run Apache Airflow 3 on Astronomer with pipelines that process terabytes of real-world physical data across many file types-imagery, audio, point clouds, and more. You will develop and evolve DAGs that orchestrate complex, multi-step workflows: dozens of tasks, fan out/in in pipelines, Python and Kubernetes operators split across generalized and specialized node pools, and dynamic DAG generation. You will work closely with our in-house ML team (feature pipelines and model deployment live in these DAGs) and coordinate with our hardware team on ingestion and formats. Scope is a mix of pipeline development and platform ownership and we are happy to adjust the scope and balance of responsibilities based on your interests and strengths.
- Help us scale and harden our data platform. We have one dedicated data engineer today; you will be the second. The broader engineering team is highly collaborative and you will work with members of the full-stack and machine learning teams. We are looking for someone to improve DAG design and execution, resource and cost tuning, reliability and observability, and contribute to how we run Airflow and Kubernetes in the cloud. If you enjoy writing pipelines and improving the platform that runs them, this role has room for both.
- Stay curious, collaborative, and cross-functional. We are a small team where many people wear multiple hats. You will work alongside ML engineers, hardware engineers, and software engineers. Turning a technically complex set of requirements from a critical industry into a rich data set is at the center of what we do. We take pride in managing complexity and providing high-fidelity data that our customers can use to make better-informed decisions.
- Be an owner at Treeswift; make the company better in whatever form that takes. We value the full picture you bring-whether that's deep expertise in orchestration, a knack for debugging at scale, or hidden talents outside work. We launched our platform last fall and have only scratched the surface of what's possible in terms of finding ways to add value for our customer. You will partner closely with some of the largest utilities in the country and contribute to efforts to develop new workflows in work planning, construction and disaster response.
This is a full-time, hybrid role based out of our Lower Manhattan, NYC office (2 days per week in person, currently pinned to Tuesdays and Wednesdays).
Experience / Skills- Bachelor's degree in Computer Science, Computer Engineering, Math, or a related field (or equivalent experience).
- 4+ years of data engineering or backend engineering experience with a focus on pipelines, orchestration, or platform.
- Hands-on experience building and maintaining production data pipelines (e.g. Airflow, Prefect, Luigi, or similar). We use Python for our pipeline environment, machine learning, and developer tooling; we don't require Python expertise and are happy for you to learn on the job.
- Experience with cloud object storage and data-at-scale (we use AWS and S3; cloud experience is required, but prior AWS experience is not).
- Comfort with Kubernetes and container-based deployments in practice: running workloads on K8s, resource and volume configuration, and debugging pod/worker issues.
- Ability to own work end-to-end: design, implement, test, and operate pipelines and related tooling. You are comfortable picking up new parts of the stack when needed.
- Strong collaboration and communication; you work well with ML, hardware, and product stakeholders and can explain tradeoffs clearly.
Nice-to-haves- Experience in early-stage or fast-moving environments where scope and ownership evolve.
- Experience with Apache Airflow (especially 3.x) and/or Astronomer.
- Experience with geospatial data, imagery, lidar, or point clouds.
- Interest in utilities, forestry, or field operations and how data pipelines support those domains.
What We Value- Mission first. We value team members who are focused on working towards the best outcomes for the customer and the business.
- Curiosity. We are constantly learning - from each other and from our customers.. We aren't afraid to learn we were wrong in the process and adapt our approach accordingly.
- Owners mindset. If you see an opportunity for improvement, run with it. We believe that good ideas can come from anywhere, no matter your role.
Benefits- Competitive salary and equity package
- Comprehensive medical, dental, and vision coverage for you and your eligible dependents
- Life insurance and short- and long-term disability coverage
- 16 weeks of fully paid parental leave to support all new parents
- Flexible, unlimited paid time off
- 401(k) retirement savings plan
- Free OneMedical membership
- Commuter benefits
- Snacks, goodies, and team lunches provided twice a week to keep you fueled and connected with your colleagues.
The estimated salary range for this position is $180,000 - 230,000 USD. Total compensation for this position is determined by skills, qualifications, relevant work experience, location, and other factors. This salary estimate excludes the value of any potential bonuses; the value of any benefits offered; and the potential future value of any long-term incentives. This information is provided per the New York City Human Rights Law. Please note that the range provided is applicable only to New York City-based applicants. Base compensation may vary if the work location is outside of New York City.
