Logging — OpenSearch & Kibana
Turn thousands of raw log lines per second into filtered, structured, alert-wired intelligence. Find what broke, when, and in which service — in seconds.
Log Management Ecosystem
What we cover
Scattered, unsearchable logs full of noise — turned into a system engineers actually reach for when something goes wrong.
Centralised Log Architecture
Design the right collection topology for microservices, system logs, and security events. Decide what to collect, what to drop, and where to route — before writing config.
OpenSearch Cluster Setup
Single-node or multi-node OpenSearch cluster deployment, TLS setup, user and role management, and index template configuration.
Kibana / OpenSearch Dashboards
Service-level log views, error rate trends, and anomaly detection dashboards. Saved searches your engineers will actually use.
Log Pipeline — Fluent Bit & Vector
Kubernetes pod log collection via Fluent Bit DaemonSet or Vector agent. Parser and filter chains to reduce log noise before it hits storage.
Index Lifecycle Management
Retention policies: how long logs stay in hot/warm/cold tiers, when they roll over, when they delete. Storage cost under control.
Alerting and Anomaly Detection
Log-based alerting: fire when error rate crosses a threshold, or when a specific string appears N times in 5 minutes. Alertmanager or OpenSearch Alerting integration.
Log Infrastructure Setup Process
From log inventory to a searchable, alert-wired, cost-optimised system — step by step.
- 1
Log Inventory
Identify sources, volumes, retention requirements, and existing noise. Decide which logs you actually need — clear decisions before any config is written.
- 2
Pipeline Design
Collection, parse, and filter chain with Fluent Bit or Vector. Cut unnecessary log noise at the source: low latency, low loss.
- 3
Storage & Retention
OpenSearch ILM: hot/warm/cold tiers and automatic deletion. Storage cost optimised, retention policy matched to your SLA.
- 4
Dashboards & Alerting
Kibana saved searches, anomaly detection, and Alertmanager integration. Actionable notifications when error rate thresholds are breached.
Raw Noise → Structured Log
Left: raw log manually grep'd over SSH, no idea which service it belongs to. Right: structured JSON queryable in OpenSearch in milliseconds, ready for alerting.
# Raw logs — ssh prod@10.0.0.5 "cat /var/log/app.log"
2024-01-15T09:23:41.332Z ERROR connection timed out
at socket.connect (/app/node_modules/pg/lib/connection.js:61)
at Object.connect (/app/src/db/pool.js:18)
at processTicksAndRejections (node:internal/process/task_queues:95)
{ errno: -110, code: 'ETIMEDOUT', syscall: 'connect' }
2024-01-15T09:23:41.887Z INFO Request received
2024-01-15T09:23:41.901Z WARN Slow query detected (2341ms)
2024-01-15T09:23:42.101Z ERROR connection timed out
at socket.connect (/app/node_modules/pg/lib/connection.js:61)
...
# Which service? Which pod? Which namespace?
# 50 services x 3 replicas = 150 SSH connections// Structured log — queryable and filterable in OpenSearch
{
"@timestamp": "2024-01-15T09:23:41.332Z",
"level": "ERROR",
"service": "payment-api",
"namespace": "production",
"pod": "payment-api-7f8b4-xk9p2",
"node": "worker-node-03",
"message": "connection timed out",
"error_code": "ETIMEDOUT",
"duration_ms": null,
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"
}
// OpenSearch query: level:ERROR AND service:payment-api
// Result: 47 ETIMEDOUT errors in the last hour — DB connection pool exhaustedAvg. MTTD
✕ 45–90 minutes (SSH + grep)
✓ < 3 minutes (OpenSearch)
Log Storage
✕ All logs, retained indefinitely
✓ ILM — 60% cost reduction
Alert Coverage
✕ Manual checks, issues go unnoticed
✓ Automated, threshold-based alerts
Typical Log Pipeline Architecture
Technologies
Who is this for?
Teams adopting microservices who need visibility into what each service is doing. Specifically, engineering teams who spend hours SSH-ing into individual pods to tail logs when something breaks in production — and want a better answer than "check each service manually."
Ready to talk through your stack?
A direct technical conversation about where you are and what needs fixing.
Request a free assessment