Let's discover together the next generation of observability with logs and traces: Quickwit

Today, in cloud-native projects, an observability stack is systematically in place. There are many such stacks (Elasticstack, Graylog, Grafana…). All those stacks, except for SaaS solutions like Datadog, rely on two solutions for log storage and search: Elasticsearch or Grafana Loki.

Both solutions have their pros and cons: on one hand, Elasticsearch is based on a very powerful search engine (Apache Lucene) but consumes a significant amount of resources to maintain performant indexes. It requires a primary node and replicas, with all data stored on block storage (SSD), which can become excessively expensive. On the other hand, Loki is very fast at ingestion and less costly because it stores logs on object storage but lacks the indexing power of Elasticsearch, making searches often slow and complex.

In this presentation, we’ll present Quickwit which combines the best of both worlds. We’ll explain how it achieves this by rewriting a search engine comparable to Lucene with high performance and leveraging the gains to store indexed data in object storage.

We’ll also see in this presentation that Quickwit can serve as a backend for storing traces via OpenTelemetry and is compatible with Jaeger UI, allowing traces to be stored durably over time.

This presentation will include a demo where Quickwit will ingest logs and traces from a Python application via OTLP/grpc. We’ll showcase dashboards with the Grafana plugin, enabling the correlation of logs and traces