Log Ingestion with Logstash & Beats
- ELK Stack architecture: data flow from sources to Elasticsearch
- Filebeat: log file tailing, multiline patterns, modules
- Metricbeat: system and service metrics collection
- Logstash pipeline: input, filter, output plugins
- Grok filter: pattern matching for unstructured log parsing
- Mutate, date, geoip, useragent filters
- Logstash performance: pipeline workers, batch size, persistent queues
- Kafka as Logstash input: consuming from topics with consumer group management
- Elastic Agent: unified agent replacing multiple Beats, Fleet management
- Ingest pipelines: lightweight processing inside Elasticsearch without Logstash
Elasticsearch Operations & Kibana Dashboards
- Index templates: component templates, composable templates
- ILM: data streams, rollover, hot-warm-cold architecture
- Kibana Fleet: managing Elastic Agents at scale
- Kibana Dashboards for log analytics: log rate, error spike detection
- Machine Learning: anomaly detection for log patterns
- APM integration: traces alongside logs in Kibana
- Alerting on log patterns: watcher, Kibana rules
- Security: Elasticsearch security, encrypted transport, audit logging
- High availability: master node quorum, cross-zone replica placement
- Kubernetes log collection: DaemonSet Filebeat, Kubernetes metadata enrichment
Platform and operations teams who can build and operate a complete ELK logging pipeline — from log shipping through enrichment, indexing, search, and dashboards.
- Deploy Beats agents and Logstash pipelines to collect and parse logs from diverse sources
- Build Grok patterns to parse custom application log formats
- Configure ILM data streams for cost-effective log retention
- Build Kibana dashboards for operational visibility and set up log-based alerting
Book the ELK Stack training
Available as a 2-day complete course or broken into a 1-day Elasticsearch/Kibana session plus a 1-day Logstash/Beats ingest session.
Get in touch