Seq 5.1 Plans

Nicholas Blumhardt -

Seq 5.1 is on its way! Version 5.0 was a major turning point for Seq, and we're still coming to terms with everything we can now build on its foundations.

The focus for Seq 5.1 is refinement: we want to improve the ergonomics of workspaces, round out Docker support, pick off some long-awaited enhancements to dashboards, and give a performance boost to several important categories of query.

Work on all of this is underway now. Some of these features are ready to try in the current preview MSI on https://getseq.net/download and the :preview tag on Docker Hub; you can find more details on these in this post.

Dashboards in 5.1

Dashboards are the perfect demonstration of how much useful information can be easily extracted from structured logs. With each Seq release we make a special point of extending the capabilities of dashboards, and in 5.1 this means:

  • Login-free dashboards - it's now possible to add &apiKey=... and &theme=... parameters to a dashboard URL to show the dashboard on an overhead display/information radiator without needing to log in. The API key needs to have the Read permission assigned to it.
  • Tabular charts - show a rowset directly on the dashboard: top five orders, slowest URLs, and much more.
  • Time-axis reference line on hover - mousing over a chart will now show a reference line at the same x-axis point on other charts.
  • having, order by and limit support - the dashboard query builder now permits a wider range of SQL constructs to be used. This works particularly nicely with the new tabular chart type.

You can see the reference line and an example tabular chart in the screenshot at the top of this post.

Pre-filtering for faster searches

Seq uses a tiered storage architecture, split into a fast memory store over a much slower, disk-backed persistent store. In the past, we viewed the persistent store as an historical archive, but in our quest to support ever-larger log volumes on the same hardware, we've started squeezing more performance out of the persistent store with the aim of making it more comfortable to use in day-to-day diagnostic analysis.

In version 5.0 we shipped an all-new native storage engine (which has been hugely successful), and signal indexes to speed up search on low-cardinality fields like Application, @Level, StatusCode etc.

Version 5.1 introduces a new technique called pre-filtering to speed up free-text searches, and searches on high-cardinality fields such as RequestId, CorrelationId and so-on.

The essence of pre-filtering is to detect fragments of unique text in the query, for example juVa98HA9dK418hf in RequestId = 'juVa98HA9dK418hf', and to skip parsing and loading of events that don't include this text in their UTF-8 encoded JSON representation. There are some subtle edge-cases to work around, but, thanks to the blistering performance of Rust's regex::RegexSet we can squeeze 6× more throughput out of many searches against the persistent store with this technique.

Pluggable inputs and GELF ingestion

Seq version 5.0 was the first to run in Docker on Linux, but we're already hearing from a lot of users who are deploying it this way. To fit even more nicely into the Docker ecosystem, 5.1 adds support for ingesting logs from Docker's own logging infrastructure, via the GELF format and protocol.

Instead of building GELF input directly into Seq 5.1, we've opened up an extension point for custom inputs and shipped GELF support as a plug-in. This means more new input types are coming in the future, and there are already a RabbitMQ input and an Azure Event Hubs input from Michael Hinni in the works.

Read more about how the Docker logging integration works in our earlier blog post Collecting Docker logs with Seq.

General improvements

We've taken on board a whole lot of feedback and made quite a few other improvements, including:

Getting the latest preview

We'll keep posting new builds as we work through the remaining tasks in the release. We're aiming to have a final build completed in late March, if all goes to plan. In the meantime, feedback via the comments section here, support@getseq.net, or the Seq issue tracker, will be appreciated.

Happy logging!

Download the Seq 5.1 preview or pull the datalust/seq:preview container.