About the Team:The
Search team (part of Coach & Orchestration) owns the full path from raw content to search results - the pipelines and infrastructure that get content indexed, the services that query it, and the systems that serve it with high relevance and low latency. We're looking for a Backend Engineer who can own this end-to-end: from data ingestion and Elasticsearch index design through the retrieval/query services that power search in production.
You'll bring strong backend and data engineering fundamentals - pipeline design, orchestration, data modeling, and service/API development - with enough exposure to embeddings, vector search, and ML-adjacent concepts to support our hybrid (lexical + vector) retrieval today and our move toward ranking and relevance improvements tomorrow. You'll work at the intersection of data infrastructure, backend services, and search, ensuring our indices are fresh and our retrieval services are performant, reliable, and built to support increasingly sophisticated search.
About the Role:To support collaboration, we ask employees to be in the office at least two days a week: Wednesday and Thursday.
In this role, you will:- Design, build, and maintain data pipelines that ingest, transform, and load content into Elasticsearch indices at scale.
- Own index design - mappings, analyzers, sharding strategy, and lifecycle management - balancing indexing throughput, query latency, and storage cost.
- Build and operate the infrastructure for hybrid retrieval, combining lexical (BM25) search with dense vector similarity (kNN/HNSW) in Elasticsearch.
- Design and maintain the backend retrieval/query services that sit in front of Elasticsearch - API design, request routing, caching, and query fan-out.
- Integrate embedding generation into pipelines - batching, caching, and re-embedding workflows when models or content change - using off-the-shelf or hosted embedding models.
- Partner with product and applied ML teams to support the evolution from retrieval into multi-stage ranking, including feeding features to future learning-to-rank systems.
- Monitor and troubleshoot cluster health, service latency, indexing throughput, and query performance; drive improvements in reliability and observability.
- Implement zero-downtime reindexing and index cutover strategies (aliasing, blue/green indices) to support continuous schema and data evolution.
- Establish data quality and validation practices to catch pipeline failures and indexing issues before they reach production.
- Collaborate with infrastructure/platform teams on cluster sizing, service scaling, and cost optimization.
- Support experiment rollout for retrieval and ranking changes, working with feature flagging or A/B test infrastructure.
- Stay current on Elasticsearch/OpenSearch and retrieval-infrastructure best practices, evaluating what's worth adopting.
What you bring to the table:- Minimum 4+ years of experience in backend or data engineering, with hands-on ownership of production data pipelines and/or backend services.
- Strong SQL and experience with data warehouses (Snowflake, BigQuery, Redshift, or similar).
- Proficiency in Python (or Java/Scala) for pipeline and service development, and experience with orchestration tools (Airflow, Dagster, Prefect, or similar).
- Experience with dbt for data transformation, modeling, and testing within the warehouse.
- Hands-on experience with Elasticsearch or OpenSearch in production - index design, mappings, ILM, sharding, and cluster tuning.
- Experience designing and operating backend services/APIs (REST or gRPC) - request handling, caching, and performance optimization for low-latency, read-heavy systems.
- Experience with service observability - tracing, metrics, and alerting (Datadog, Prometheus/Grafana, or similar).
- Comfort with containerization and deployment (Docker, Kubernetes) for production services.
- Clear, effective communication, with the ability to collaborate well with data scientists, ML engineers, and product partners.
- Comfort operating in cloud infrastructure (AWS/GCP/Azure), including cost and performance tradeoffs for search infrastructure.
Bonus points if you have:- Experience with batch and/or streaming data processing (Spark, Kafka, Flink, or similar).
- Practical experience with vector search - dense_vector fields, kNN/HNSW, and combining lexical and vector scores for hybrid retrieval.
- Working familiarity with embedding models (open-source or hosted/API-based) - generating, storing, versioning, and refreshing embeddings at scale.
- Understanding of retrieval evaluation basics (recall@k, NDCG, MRR).
- Experience with learning-to-rank libraries (e.g., LightGBM, XGBoost rankers) or exposure to reranking pipelines.
- Experience with reciprocal rank fusion (RRF) or other hybrid score-blending techniques.
- Familiarity with vector databases beyond Elasticsearch (FAISS, ScaNN, pgvector, etc.).
- Prior experience scaling search/retrieval infrastructure in a high-traffic consumer or enterprise product.
Compensation, Benefits & Perks:- Collaborate with your manager and team to create a healthy work-life balance
- 20 vacation days that we expect you to take!
- Competitive health, dental, and vision insurance (100% employee and 75% dependent PPO, Dental, VSP Choice)
- Employer-sponsored 401k plan with company match
- Access to LinkedIn Learning and other resources to support professional growth
- Paid Family Leave, FSA, HSA, Commuter benefits, and Wellness benefits
- 40 hours of annual paid time off to participate in volunteer programs of choice
