Building modern Seq Apps in C#

Nicholas Blumhardt -

Seq Apps are plug-ins that read or write events to the Seq event stream. Seq apps have been around a long time, but they've been refreshed and updated recently, first to support non-.NET programming languages, and more recently, to support .NET Standard and a more efficient C# API.

Not much has been written about the .NET or C# changes yet, which is a shame because it's now super-easy, and a lot of fun, to programmatically analyze and act on log events.

This is the first of two parts. In it we'll write a simple output app, one that can receive events from Seq. Outputs are used for a lot of different things: triggering alerts via channels such as email and Slack, archiving data to long-term storage, automatically raising tickets in issue trackers, replicating Seq data to other systems, and more. The second part will look at input apps.

The plan

One of the fears of companies handling payments is for credit card numbers to inadvertently end up in application logs. There are many steps usually taken to prevent it, but the nature of log collection makes it hard to guarantee with absolute certainty that card numbers can never find their way into exception messages, chunks of request/response payloads, and so on.

If your domain is sensitive (health, banking, handling PII...) then you'll be able to think of a few similar things that you'd prefer didn't appear in logs.

The simple app we'll build is going to apply a regular expression to the full JSON representation of each log event, looking for 13 to 16-digit numbers joined with dashes and/or spaces, and raise the alarm if there's any match.

The example here is easy to understand but isn't meant to be ultra-realistic or production-quality. Most apps would need to parse and manipulate the JSON event data, and if you're working with similar requirements, your checks might have to be a bit more sophisticated 😁.

For the impatient, you can grab the source code for this example straight from GitHub and start hacking away.

File > New Project

Seq Apps are regular C# classes, in regular .NET Standard assemblies.

To build one, the first step is to create a .NET Standard class library, either in Visual Studio, or from the dotnet command line.

mkdir Seq.App.Detective
cd Seq.App.Detective
dotnet new classlib

The app needs a reference to the Seq.Apps package:

dotnet add package Seq.Apps -v 5.1.0

Does it build?

dotnet build

👍

The [SeqApp] type

After deleting the default Class1.cs file, it's time to add the plug-in type. Here's RegexDetective.cs:

using System;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Seq.Apps;

[SeqApp("Regular expression detector",
    Description = "Checks complete event documents for text matching a regular expression.")]
public class RegexDetective : SeqApp, ISubscribeToJsonAsync
{
    const int MaximumFragmentChars = 4096;

    static readonly string Pattern = "\\b(?:\\d[ -]*?){13,16}\\b";

    public Task OnAsync(string json)
    {
        var matches = Regex.Matches(json, Pattern);

        if (matches.Count > 0)
        {
            var positions = matches.Cast<Match>()
                .Select(m => m.Index)
                .ToArray();

            var sanitized = Regex.Replace(json, Pattern, m => new string('#', m.Value.Length));
            var fragment = sanitized.Substring(0, Math.Min(sanitized.Length, MaximumFragmentChars));

            Log.ForContext("Pattern", Pattern)
                .ForContext("Fragment", fragment)
                .ForContext("Positions", positions, destructureObjects: true)
                .Error("Match detected");
        }

        return Task.CompletedTask;
    }
}

You'll notice:

  • The class is annotated with [SeqApp]. This provides the description and help text for the Seq Settings > Apps screen.
  • The class inherits from SeqApp. The base type has properties like Host and Log for interacting with the outside world.
  • The class implements ISubscribeToJsonAsync - this is the newest of several ways for the app to receive events, in this case, they're passed to the app as raw JSON.

The logic is all there in OnAsync(): check for regular expression matches, and report the offending events when they're found.

Using Log....Error() sends diagnostic events back to Seq. There, they can be forwarded to a different app for alerting, for example to the Slack or email outputs.

Time to build!

dotnet build

Testing and debugging

Before packaging up the app and installing it into Seq, it can be tested locally. You'll need seqcli version 5.1.213 or newer for this.

Check the version of seqcli you have installed:

seqcli version

If it's not up-to-date, grab the latest archive for your operating system from the releases page.

Things will be made easier if you have Seq running locally on your development machine.

Once this is ready to go, log some events:

seqcli log -m "Processing a card"
seqcli log -m "Processing 1234 567890 1234"

Since they're the most recent events on the server, it's easy to read them back with:

seqcli search -c 2

You should see something like:

[2019-04-11T14:24:21.0563248+10:00 INF] Processing 1234 567890 1234 {}
[2019-04-11T14:23:54.7078096+10:00 INF] Processing a card {}

To test the app, we'll need to change into the directory containing the built DLLs:

cd bin\Debug\netstandard2.0

Using --json we can pipe the search results into the app:

seqcli search -c 2 --json | seqcli app run

Aha! A match:

{"@t":"2019-04-11T04:24:45.5443570Z","@mt":"Match detected","@l":"Error","Positions":[55],"Fragment":"{\"@t\":\"2019-04-11T04:24:21.0563248Z\",\"@mt\":\"Processing ################\"}","Pattern":"(?:\\d[ -]*?){13,16}"}

Tip: if this fails, you might be running into some character encoding issues that appear under Windows PowerShell. Using CMD or PowerShell Core 6.x will help.

App settings

There isn't anything specific to credit card matching in our app, except for the regular expression we're searching for. To generalize the app, we can accept the pattern as an app setting, and configure different instances of the app to use different patterns:

        [SeqAppSetting(HelpText = "A regular expression to search for.")]
        public string Pattern { get; set; }

It's enough to make the pattern a public property and apply the [SeqAppSetting] attribute.

After rebuilding the package, we can now try different patterns on the command-line:

seqcli search -c 2 --json | seqcli app run -p Pattern="card"

And match different events:

{"@t":"2019-04-11T05:05:33.9565488Z","@mt":"Match detected","@l":"Error","Positions":[57],"Fragment":"{\"@t\":\"2019-04-11T04:23:54.7078096Z\",\"@mt\":\"Processing a ####\"}","Pattern":"card"}```

Packaging and deployment

Because our app doesn't have any dependencies apart from Seq.Apps, packaging it for Seq is as simple as:

dotnet pack

If you're writing a non-trivial app, chances are there are additional assemblies your app will need. Seq expects these to be in the NUPKG, too (it won't recursively install packages), so to do that you'll need to make a small modification to your CSPROJ file and use dotnet publish alongside pack in your build script. (These changes are already present in the example repository.)

Once we have a NUPKG file, it's time to load it up in Seq. If you already have a NuGet feed configured, push the package there. Otherwise, you can add the build output location (or a file share) as a feed under Seq's Settings > Feeds:

Now, to install the package, go to Settings > Apps > Install from NuGet, choose the feed, and enter the package id:

With the app installed, Add instance will present some settings for the app: an informational title, the source of events to send to it, and the pattern input for the app itself:

Stream incoming events needs to be checked if you want the app to receive events as they arrive.

The app in Seq

Now, whenever a matching event is logged, it should be detected:

seqcli log -m "This one is tricky" -p CardNumber=12345678901234

And, there it is!

The source code for this example is published on GitHub. Check out apps that others have written by searching the seq-app tag on NuGet.