Today we're pleased to announce the production release of Seq 3.0, a major update to our log server for .NET apps. Downloads are now live on the Seq website.
What is Seq?
Seq helps .NET teams make sense of complex, asynchronous and distributed applications. Logs are centralized to Seq for efficient monitoring, troubleshooting and integration. It runs on-premises or in the cloud, giving organizations complete control over data security and retention. This makes Seq a compelling solution in diverse industries spanning insurance, finance, government services, and health care.
Seq has best-in-class support for the modern structured logging techniques embodied in libraries like Serilog, Semantic Logging and ASP.NET Core's forthcoming Microsoft.Extensions.Logging. Structured logging makes advanced filtering and correlation of log events possible without regular expressions or log parsing, drastically reducing the time and effort needed to find and fix issues in production environments.
For .NET developers specifically, Seq is a natural fit. It has no JVM dependencies, is lightweight enough to install and run on individual developer workstations (including under the free single-user license), and integrates with all popular .NET tooling. Log filtering uses a familiar C#-inspired syntax to soften the learning curve, and Seq plug-ins as well as the supported API client are all written using C# and .NET. Seq is built by .NET developers, for .NET developers.
Seq 3.0
The centerpiece of Seq 3.0 is an exciting new query engine, capable of grouping, aggregation, and time slicing. This expands the diagnostic capabilities of Seq to make many more questions about system behavior and performance answerable directly from log events.
Today's release also adds integration with Azure Active Directory, automated configuration backups, easier troubleshooting via the new Ingestion Log, performance improvements and a more polished user experience to the existing feature set. Complete release notes follow at the end of this post.
New Query Engine
Filtering vs. Queries - Seq already has a sophisticated filter syntax that can be used to select events based on text, event types, property values and regular expressions. Filtering is how Seq helps you find that needle in the haystack; filters can be grouped into signals, saved for quick access, and used to trigger notifications via Seq apps.
There are still many questions that filtering alone can't answer.
- How many times has this event occurred each hour, per machine?
- Are response times improving?
- What message types have been processed today?
These questions call for aggregates like count(), min(), max(), mean(), percentile() and distinct(). They group results either by a property or by time slice, and they produce a rowset rather than a stream of events.
Rather than extend filters with these capabilities, Seq 3.0 introduces SQL as an alternative query syntax for this purpose. Because SQL is familiar to most developers, there's less to learn: with a few exceptions, aggregate queries in Seq are very "guessable".
Basing aggregate queries in Seq on an industry standard also means that there's a clear path to evolve more advanced capabilities, and opportunities to work better with other SQL-oriented tooling in the future.
Right now, exploring a log stream with SQL is also a lot of fun! Here are the three examples from above.
How many times has this event occurred each hour, per machine?
select count(*) from stream group by time(1h), MachineName
Seq implicitly returns columns used as groupings like time and MachineName in this example, to reduce the amount of typing needed.
Are response times improving?
select mean(Elapsed) as Response,
percentile(Elapsed, 90) as Response90th
from stream
group by time(1d)
order by time desc
What message types have been processed today?
select distinct(MessageType) from stream
When you run these queries in Seq, you'll need to select the time range to use: the default will be to query the last 24 hours. Keep in mind that, like filters, queries running over archived (non-memory-resident) events can take a while, so choosing a modest time range at first is always a good idea.
How is the syntax determined? - Seq continues to support all of the filtering capabilities from earlier releases. If what's typed into the filter bar starts with select then Seq will assume you're writing a SQL query and parse it accordingly. Otherwise, whatever you type will be treated as a filter if possible, or free text if not.
(For the curious - if you want to search for the word select as a filter, just enclose it in double quotes like "select".)
Does this mean I can use Seq as a database? - No, Seq would make a lousy general-purpose database :-). The SQL operations being added are the ones that make sense for ad-hoc log analysis: you'd quickly hit limitations using it as an application data store.
Is Seq targeting metrics collection scenarios? - Though Seq 3.0 can do light metrics duties with these features, particulary time() groupings and aggregate functions, the focus of SQL queries is ad-hoc log analysis, not primary metrics workloads. Metrics packages (like InfluxDB, from which the time() syntax is borrowed, and Grafana, an excellent dashboarding UI for InfluxDB and other back-ends) can store points much more efficiently than is possible for log events, use techniques like downsampling to work better with high volume numeric information, and focus much more strongly on graphical presentation rather than raw data.
SQL queries are a powerful foundation that we'll keep building on throughout the 3.x cycle. More information on what's supported today can be found in the documentation.
Installing and Upgrading
Download the new version from the Seq website. It's an MSI installer, and getting started is a five-minutes process. Seq ships with a free single-user license that you can use on developer workstations, or you can get a 30-day trial key to enable authentication and SSL support.
Seq 3.0 is an in-place upgrade from all prior Seq versions back to Seq 1.0. Customers with a current paid license can upgrade to Seq 3.0 for free within the support period, or obtain a trial key to use the new release for 30 days.
If you have any trouble installing or upgrading to Seq 3.0, we're here to help through the discussion forum and support@getseq.net.
Release Notes for Seq 3.0.30
Since Seq 2.4.2:
- Aggregate/SQL query support with
time()groupings - Azure Active Directory authentication provider support
- Automated metadata and configuration backups
- Enable minimizing the Signal Bar (#277)
- Better diagnostics for ingestion problems with a new new Ingestion Log (#339)
- Persist startup-time configuration to a JSON file in the storage root (#376)
- Prevent internal log space consumption when invalid JSON payloads are repeatedly sent to the ingestion endpoint (#375)
- Substantial reduction in CPU usage for high-throughput deployments (#380)
- Encoding improvements for newly-written extent files, ~30% space reduction and up to 50% faster access time
- Show week-by-hour as a dash time-range option (#353)
- Tooltips added to filter bar buttons (#374)
- Per-event size limits on ingestion (in addition to overall payload size)
- Display the NuGet package id for installed Seq apps (#356)
- Show a notification when the Seq server is warming up and query performance may be degraded (#326)
- Detect and reconfigure obsolete
--data=storage layouts on upgrade (#247) - Show search progress by date rather than just scanned event count (#369)
- Don't show a leading zero in AM/PM time format (#368)
- Fix case-sensitivity of function names when used in filter wildcard expressions (#366)
- Simpler Clear button to close timeline view (#365)
- Fix error "The database engine is already initialized" immediately after restart (#362)
- Fix failure to ingest values of
Double.MinValue(#358) - Disable conflicting autocomplete in Edge browser (#350)
- Ingestion throughput on I/O starved machines can be improved using buffered writes
- The
seq installcommand now optionally accepts a user account to run the service under - Encryption used for feed passwords etc. is now portable between machines (avoids DPAPI machine key)
seq versioncommand to determine the server version from the console- Default Boolean app setting values to
falserather than raise an error when not set - Prevent NuGet feeds from being deleted when configured apps depend on them
- Audit all successful, as well as failed, log-in attempts to Seq's internal log
- Simple CSS override-based theming in Settings > Theme
- Renamed the Metrics settings screen to the less ambiguous Diagnostics
- Support
0xsyntax for hexadecimal literals in filters and queries - Introduces new built-in
@Message,@EventTypeand@MessageTemplateproperties in filters and queries - Introduces new
Round(f,n),ToHexString(n)andToIsoString(dt)built-in functions in filters and queries - Fixes ignored "run on existing" option when adding an app instance (#381)
- And many small cosmetic improvements...