Senior Software Engineer, Data Platform

Persona Identities, Inc

$130K — $180K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years of experience in software engineering with a focus on data infrastructure or large-scale distributed systems.
  • Proficient in Python, with openness to increasing proficiency in Go.
  • Hands-on experience with big data or streaming technologies like Spark, Kafka, or Flink, especially with interest in Iceberg.
  • Experience in building scalable data pipelines for both structured and unstructured data.
  • Proven ability to lead complex, multi-quarter infrastructure projects and articulate architectural decisions to various stakeholders.
  • Experience in mentoring engineers, capable of operating efficiently in ambiguous situations.
  • Strong foundation in distributed systems, storage, and performance/cost optimization.

Responsibilities

  • Build and scale the datalake, ensuring efficient storage design and fast access.
  • Develop reliable high-throughput data ingestion services for seamless data flow.
  • Own streaming and batch ingestion pipelines handling billions of records and terabytes of data daily.
  • Create new query and analytics capabilities that cater to unique use cases outside traditional warehouses.
  • Operate a low-latency serving layer for real-time access to platform data.
  • Collaborate with infrastructure and product teams to secure and scale the platform.

Benefits

  • Medical, dental, and vision insurance for full-time US-based employees.
  • 401(k) contribution of 3%.
  • Unlimited Paid Time Off (PTO) with quarterly mental health days.
  • Professional development stipend and wellness benefits.
  • Private medical insurance and 6% employer pension contribution for full-time UK employees.
  • Monthly wellness stipend and co-working stipend for UK employees.
Full Job Description
About the role

We are looking for passionate engineers who are excited to build the data foundation for the identity layer of the internet. As a senior engineer on the Data Platform team at Persona, you'll design, build, and operate the systems that move and store data at massive scale - an Iceberg datalake, the streaming and batch pipelines that feed it, a multi-provider warehouse layer, and the services built on top of it. You'll work the full depth of that stack, from raw ingestion through storage layout and compute all the way to the query layer - owning the performance, reliability, and cost of each. You'll set technical direction for major pieces of this platform - making the architecture, storage-format, and build-vs-buy decisions that shape it for years - and raise the bar for the engineers around you. This is a high-ownership role on a small team, working closely with the rest of engineering, data science, and infrastructure to make the platform fast, reliable, and cost-efficient at hundreds of billions of records per day.

What you'll do at Persona
  • Build and scale our datalake - the pipelines that feed it and the storage design that keeps it fast and efficient.
  • Develop high-throughput data-movement and ingestion services that bring data into the platform reliably and at scale.
  • Own streaming and batch ingestion and the change-data-capture pipelines that absorb billions of records and terabytes of data per day.
  • Build new query and analytics capabilities directly on the datalake - for use cases that don't fit a traditional warehouse.
  • Operate the low-latency serving layer for real-time, high-concurrency access to platform data.
  • Partner with infrastructure and product teams to provision, secure, and scale the platform.


What you'll bring to Persona
  • 5+ years of experience in software engineering, with deep expertise in data infrastructure or large-scale distributed systems.
  • Proficiency in Python - our primary language across pipelines. Our newest services are written in Go, so you either work in Go today or are excited to ramp into it.
  • Hands-on experience with big-data or streaming technologies such as Spark, Kafka, or Flink, and interest in lakehouse table formats like Iceberg.
  • Experience building and operating scalable data pipelines across structured and unstructured data.
  • A track record of leading complex, multi-quarter infrastructure projects end to end and driving architecture decisions across teams.
  • Experience mentoring engineers and operating effectively in ambiguity - defining the problem, not just the solution.
  • Strong fundamentals in distributed systems, storage, and performance/cost optimization.
  • Excellent communication and collaboration skills across engineering, data science, and infrastructure teams.
  • Nice to have: ClickHouse, DuckDB, Google Cloud (GCP), Kubernetes, MySQL, MongoDB, Snowflake.


Benefits and Perks

For full-time US-based employees (excluding internship and contractor opportunities), Persona offers a wide range of benefits, including medical, dental, and vision, 3% 401(k) contribution, unlimited PTO, quarterly mental health days, family planning benefits, professional development stipend, wellness benefits, among others.

For full-time UK employees, Persona offers a wide range of benefits, including private medical insurance, dental insurance, a 6% employer pension contribution, unlimited PTO, a monthly wellness stipend, professional development stipend, co-working stipend, and more.

As part of our interview process, all candidates will be asked to verify their identity with Persona. This step is used solely to confirm that candidates are who they say they are, and will have no impact on hiring decisions.

Similar Jobs

More Jobs at Persona Identities, Inc

More Information Technology Jobs

Find similar Senior Software Engineer, Data Platform jobs: