Overview

You'll build and scale the real-time and batch data platform that powers a large enterprise contact center solution. Our products demand ultra-low-latency decisioning for live interactions and cost-efficient big-data analytics for historical insights. We're primarily on Azure today and expanding to GCP and AWS. Data is the backbone for our AI features and product intelligence.

Primary charter: complex contact center analytics and operational intelligence: an AI-enabled enterprise contact center analytics. Our vision is a flexible AI-enabled data platform that unifies contact center KPIs, customer/business outcomes, and AI quality/performance, and pervasively applies AI to deliver advanced features that help users easily leverage rich contact center data alongside business data and AI performance monitoring to drive decisions end-to-end.

Job Description

Team & Tech
• Cloud: Azure (primary), expanding to GCP/AWS
• Platform: Databricks, Spark (batch + streaming), Airflow, Apache Superset, Kafka
• Data & governance: Delta Lake, Unity Catalog, Delta Sharing
• Infra & delivery: Terraform, Docker/Kubernetes, CI/CD (GitHub Actions/Azure DevOps)
• Interfaces: REST/gRPC; schemas with Avro/Protobuf
• Processing alternatives: Apache Flink/Apache Beam where appropriate; custom
processors/services in Go for specialized low-latency needs
• App stack: React + TypeScript (front-end), Go (preferred) and Java (backend)
• Focus: Real-time streaming, lakehouse analytics, reliability, and cost efficiency
• Experimentation & metrics: MLflow for experiment tracking and AI quality/performance
metrics
• Tooling integration: MCP (Model Context Protocol) to expose/consume data tools for agents

What you'll do
• Design, build, and operate low-latency streaming pipelines (Kafka, Spark Structured
Streaming) and robust batch ETL/ELT on Databricks Lakehouse.
• Establish reliable orchestration and dependency management (Airflow), with strong SLAs
and on-call readiness for business-critical data flows.
• Model, optimize, and document curated datasets and interfaces that serve analytics, product
features, and AI workloads.
• Implement data quality checks, observability, and backfills; drive root-cause analysis and
incident prevention.
• Partner with application teams (Go/Java), analytics, and ML/AI to ship data products into
production.
• Build and maintain datasets and services that power RAG pipelines and agentic AI workflows
(tool-use/function calling).
• When Spark/Databricks isn't optimal, design and operate custom processors/services in Go
to meet strict latency or specialized transformation requirements.
• Instrument prompt/response and token usage telemetry to support LLMOps evaluation and
cost optimization; provide datasets for labeling and golden sets.
• Improve performance and cost (storage/compute), review code, and raise engineering
standards.

Security & Compliance
• Design data solutions aligned to enterprise security, privacy, and compliance requirements
(e.g., SOC 2, ISO 27001, GDPR/CCPA as applicable), partnering with Security/Legal.
• Implement RBAC/ABAC and least-privilege access; manage service principals, secrets, and
key rotation; enforce encryption in transit and at rest.
• Govern sensitive data: classification, PII handling, masking/tokenization, retention/archival,
lineage, and audit logging across pipelines and storage.
• Build observability for data security and quality; support incident response, access reviews,
and audit readiness.
• Embed controls in CI/CD (policy checks, dependency vulnerability scanning) and ensure
infra-as-code adheres to guardrails.
• Partner with security engineering on penetration tests, threat modeling, and red-team
exercises; remediate findings and document controls.
• Contribute to compliance audits (e.g., SOC 2/ISO 27001) with evidence collection and
continuous control monitoring; support DPIAs/PIAs where required.

Qualifications
• 6+ years building production-grade data pipelines at scale (streaming and batch).
• Deep proficiency in Python and SQL; strong Spark experience on Databricks (or similar).
• Advanced SQL: window functions, CTEs, partitioning/z-ordering, query planning and tuning
in lakehouse environments.
• Hands-on with Kafka (or equivalent) and an orchestrator (Airflow preferred).
• Strong data modeling skills and performance tuning for low latency and high throughput.
• Production mindset: SLAs, monitoring, alerting, CI/CD, and on-call participation.
• Proficient using AI coding assistants (Cursor, Claude Code) as part of daily development.
• Proficiency building data services/processors in Go (or willingness to ramp quickly), and
familiarity with alternative frameworks (e.g., Flink/Beam) is a plus

Required

-- Claude Code and/or Cursor - Desirable daily or heavy weekly use. --

Preferred qualifications
• Experience in multi-cloud or cloud migration (Azure plus either GCP or AWS).
• Exposure to building data for AI/RAG, LLM-powered features, and agentic AI patterns (tool-
use/function calling, planning/execution, memory).
• Familiarity with LLMOps telemetry (prompt/response logs, token budgets) and agent
evaluation pipelines.
• Background in high-scale product engineering (vs. internal IT-only projects).
• Contact center or CRM data familiarity (nice-to-have, not required).
• Bachelor's or Master's in CS/EE/Math or similar; strong academic background and/or top-tier
companies.

The pay range for this opportunity is from $128,200 to $157,000 + performance-related bonus + benefits. This range represents the anticipated low and high end of the salary for this position. This role is also eligible to receive an annual bonus that aligns with individual and company performance. Actual salaries will vary and are based on factors such as a candidate's qualifications, skills, competencies.

#LI-CS1

Experience

4 - 6 Years of Experience

Education

Bachelor degree or equivalent experience
Advance Degree preferred

Preferred Certifications