Role Overview:The Senior AIOps ML Engineer will be responsible for designing, building, and optimizing a robust Lakehouse architecture for petabyte-scale multi-domain observability data. This role involves developing and deploying advanced machine learning models for AIOps, including streaming anomaly detection, root-cause analysis, and incident forecasting. Key responsibilities also include managing the end-to-end MLOps lifecycle, ensuring data quality and performance, and integrating AIOps insights with incident management platforms. The engineer will also focus on security and compliance observability, collaborating with security teams, and contributing to organizational engineering standards and mentorship.
Key Responsibilities:- Lakehouse Architecture & Data Engineering: Design and evolve Lakehouse schema (Delta Lake / Apache Iceberg) for multi-domain observability data at petabyte scale. Build and maintain robust ingestion pipelines from OTel Collector through Kafka to the Lakehouse, ensuring exactly-once semantics and strict schema enforcement. Implement dbt transformation models to generate mart-ready, denormalized fact and dimension tables. Define and enforce data quality contracts and SLAs. Optimize query performance utilizing partitioning strategies, Z-ordering, bloom filters, and materialized views.
- ML Model Development & AIOps: Design, train, and deploy machine learning models for streaming multivariate anomaly detection, root-cause analysis, and incident forecasting. Build low-latency streaming inference pipelines (Flink / Spark Streaming) for real-time anomaly scoring. Develop sophisticated log intelligence models (clustering, NLP classification, error deduplication). Implement unsupervised and semi-supervised methods for User Experience frustration detection and KPI correlation. Own the ML feature store, managing feature engineering, versioning, and backfill pipelines. Instrument model performance tracking, including drift detection, accuracy monitoring, and automated retraining triggers.
- AIOps Platform & Productionization: Design and operate the end-to-end AIOps workflow, spanning signal ingestion, feature computation, model inference, alert routing, and auto-remediation hooks. Build high-performance model serving infrastructure supporting real-time REST/gRPC endpoints and async batch scoring with strict p99 latency SLOs. Integrate AIOps insights with incident management platforms (PagerDuty, Opsgenie) and internal runbooks. Define and publish metrics from the Business KPI mart to quantify business impact.
- Security & Compliance Observability: Partner with the Security team to build the Security mart schema, including threat feed ingestion, UEBA baselines, and CVE correlation pipelines. Train anomalous-access and lateral-movement detection models. Ensure all data handling adheres strictly to data residency requirements, PII masking standards, and audit-log protocols.
- Collaboration & Engineering Standards: Define telemetry schema contracts with the OTel Instrumentation team. Author ML platform RFCs and contribute actively to observability data model standards. Mentor junior ML and data engineers and conduct rigorous design reviews.
Required Skills:- AI Agents
- Kafka and Streaming technologies (Flink / Spark)
- Lakehouse architectures (Delta Lake / Apache Iceberg)
- Machine Learning (Anomaly detection, time-series analysis)
- Observability tools and concepts (OTel, APM, Logs)
- MLOps practices (feature store management, drift detection, model retraining)
- Strong proficiency in SQL and Python
Qualifications:- 10+ years of experience in a relevant field.
