New Alerts, DR with zero-downtime upgrades: Seq 2021.3 has shipped! 🎉

Nicholas Blumhardt -

Today we're very pleased to announce the release of Seq 2021.3. You can download the Windows installer from datalust.co, or pull the latest datalust/seq image from Docker Hub.

Seq 2021.3 includes improvements across the whole product.

New, completely rewritten Alerts — We've redesigned Seq Alerts as a full-fledged, top-level feature. Alerts get their own status-oriented dashboard, a much better editing experience, and rich, multi-channel notifications.

Disaster recovery instances — Reliably and securely replicate all Seq data to two nodes, preventing data loss even in the face of a total machine failure.

Zero-downtime upgrades — Fail over to a second Seq node to upgrade or perform maintenance on the first, all the while seamlessly ingesting live data and serving user queries.

Improvements under Docker — Seq 2021.3 plays much more nicely with the Linux kernel's virtual memory manager, improving performance and stability. The datalust/seq Docker container now natively supports TLS, strengthening security and in some cases avoiding the need for a reverse proxy. Managing Seq on Docker is made easier with init script support.

PostgreSQL metadata storage 2021.3 adds PostgreSQL (alongside MSSQL and the embedded metadata store) as a robust option for storing Seq's internal configuration.

And there's a lot more: bug fixes, query language improvements, a clearer search bar layout, icon updates, better secret storage, a dedicated /health endpoint, new seqcli features, millisecond precision in the date range picker… Read on for a summary of the major features, or check out the 2021.3 issue tracker milestone for all of the details.

What is Seq?

Seq is a centralized search and analysis server for structured application logs. It combines a flexible JSON data model and familiar query language to drive real-time log exploration, dashboarding, and alerting.

We build Seq to help teams easily identify and diagnose problems in complex applications and microservices.

Alerts in 2021.3

Structured log data is perfect for alerting. For any event or query result you can find with search, you can set an alert to notify you when that condition next occurs.

Is an app throwing exceptions unexpectedly? Have response times spiked? Are connections to a back-end web API timing out more than usual? Seeing a high rate of login failures? With the data of interest in the Events screen, press the (rather magical) bell 🔔 icon, and in a few clicks you'll have a matching alert!

This not only works for simple signals and searches, but also for more complex SQL-style queries.

The new Alerts dashboard, pictured at the top of this post, provides at-a-glace status information for all the alerts on a Seq server. The notification history for an alert is tracked, so you can find out when and how often an alert has been triggered, and click through to the underlying data.

Alert notifications can be sent to email, Slack, Teams, and many other integrations.

If you're familiar with Seq's earlier alerting implementation, you'll also be pleased to find that notifications now include a sample of contributing events 😎.

Read more in the new Alerts documentation.

Disaster recovery and zero-downtime upgrades

2021.3 is the first Seq release to support multi-node deployment, and a huge milestone on Seq's clustering roadmap.

The DR configuration in Seq 2021.3 serves two important purposes:

  • All event data is precisely and securely replicated to a second Seq node, so that if the first Seq node is completely lost, historical data can be recovered
  • By switching between nodes, Seq itself, along with the hardware and operating system it runs on, can be upgraded and maintained during business hours without any interruption of log ingestion or access to the Seq UI

We've paid special attention to making DR instances easy to configure and maintain. If you'd like a walk-through of the process, or need some help deploying DR in your own environment, we'd love to help: please get in touch via support@datalust.co.

The DR configuration in 2021.3 provides redundancy, but does not implement high availability (HA) or scale-out. These are a major part of our aims for 2022, and we'll be able to talk more about our plans early in the new year.

Read more about setting up a DR instance in the Seq documentation.

Improvements under Docker

In the three years since Seq added Docker support, we've seen a massive shift towards Docker deployment. We've been continually learning, and 2021.3 has the benefit of a lot more experience running Seq under Docker.

This release addresses a common cause of Seq being OOM-killed by the Docker runtime: improvements in Seq's storage engine release disk pages faster, leaving more container memory for query execution and to absorb alloction spikes. The result is a smoother, more stable Docker hosting experience with fewer restarts.

Also in 2021.3, administration is made easier through init script support. Init scripts are regular shell scripts placed in either a mounted /seqinit directory, or under /data/Init, that perform configuration tasks and interact with the Seq command line interface. When the datalust/seq container starts, it will detect new init scripts and run them before starting the Seq server process.

Native support for TLS/SSL termination means that the Seq container can now be deployed in production without a reverse proxy.

Finally, secret key providers make it possible to secure Seq's internal encryption key using an external key management service, avoiding the use of environment variables or plain-text configuration for this on Docker/Linux.

What else is new?

A lot! You'll immediately notice the new search bar button layout and icon set. While a tiny amount of muscle memory will need to be reprogrammed, we've been living with this design for a few months now and feel like it's a worthwhile improvement.

You'll notice that the JSON and CSV export buttons have moved to a much more discoverable position above the result set that they act on.

Also of note -

Upgrading

Seq 2021.3 is a highly-compatible in-place update. All recent Seq versions can be upgraded by running the Windows MSI or pulling the datalust/seq:latest tag from Docker Hub.

Since Seq 2021.2, the Alerts API has changed significantly. If you're integrating with Seq alerts programmatically and need help to move your code across from the dashboard-based alerting implementation, please reach out and we'll be happy to assist.

Check out the upgrade guide for version-specific instructions, or if you're upgrading from Seq 4.2 and earlier.

We hope you enjoy Seq 2021.3!

— The Seq Team