ELK Observability Lead

Prophecy Technologies

$120K — $150K *
US-AnywhereRemote in Chicago, IL
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 10+ years of experience in observability and related fields.
  • Expert level proficiency in Elasticsearch.
  • Strong knowledge of distributed architecture and cluster design.
  • Experience with search and index optimization, including Query DSL and aggregations.
  • Demonstrated ability in performance tuning and scaling of data platforms.
  • Proficiency in Index Lifecycle Management (ILM), data streams, and sharding strategies.

Responsibilities

  • Lead the design and implementation of the ELK Stack for enterprise observability.
  • Own and execute the observability strategy in collaboration with cross-functional teams.
  • Drive performance and reliability optimization of the ELK platform.
  • Oversee the development of data ingestion pipelines to ensure reliable data flow.
  • Enable advanced observability through analytics and visualizations in Kibana.
  • Establish security, data governance, and compliance best practices within the ELK ecosystem.
  • Act as a technical leader and mentor for observability best practices.
  • Integrate observability into CI/CD pipelines with DevOps and application teams.

Benefits

  • Collaborative work environment with cross-functional teams.
  • Opportunity to lead and shape observability strategy for the enterprise.
  • Exposure to advanced technologies and tools in observability and data visualization.
  • Professional development opportunities, including mentoring and skills building.
Full Job Description
Role Overview:

The ELK Observability Lead will be responsible for spearheading the design, implementation, and continuous enhancement of the ELK Stack (Elasticsearch, Logstash, Kibana) to provide comprehensive enterprise-wide observability, logging, and monitoring capabilities. This role involves owning the observability strategy and collaborating with cross-functional teams to translate operational requirements into scalable dashboards, alerts, and reporting solutions.

Key Responsibilities:
  • Lead the design, implementation, and ongoing enhancement of the ELK Stack (Elasticsearch, Logstash, Kibana) to deliver enterprise-wide observability, logging, and monitoring capabilities.
  • Own the observability strategy by collaborating with cross-functional teams to translate business and operational requirements into scalable dashboards, alerts, and reporting solutions for diverse stakeholders.
  • Drive performance optimization and reliability of the ELK platform, including proactive monitoring, troubleshooting, and fine-tuning of Elasticsearch clusters for high-volume data ingestion, storage, and retrieval.
  • Oversee the development and governance of data ingestion pipelines using Logstash and other tools, ensuring robust, scalable, and reliable data flow from multiple heterogeneous sources.
  • Enable advanced observability use cases by delivering meaningful analytics and intuitive data visualizations in Kibana to support IT operations, incident response, and business insights.
  • Establish and enforce best practices for security, data governance, access control, and compliance within the ELK ecosystem.
  • Act as a technical leader and SME for observability, mentoring team members, driving capability building, and providing guidance on ELK best practices and modern observability approaches.
  • Work closely with DevOps, SRE, and application teams to integrate observability into CI/CD pipelines and ensure end-to-end visibility across systems and services.

Required Skills:
  • Elasticsearch (Expert level)
  • Distributed Architecture & Cluster Design
  • Search & Index Optimization (Query DSL, Aggregations)
  • Performance Tuning & Scaling
  • ILM, Data Streams, Sharding Strategy
  • Digital : Elastic Search

Qualifications:
  • 10+ years of experience

Preferred Skills:
  • Prior experience in AIOps, ITOM, or observability solutions.
  • Familiarity with scripting and automation (e.g., Python, Bash).
  • Knowledge of infrastructure-as-code tools like Terraform or Ansible.

Similar Jobs

More Jobs at Prophecy Technologies

More Information Technology Jobs

Find similar ELK Observability Lead jobs: