Seq Apps are plug-ins that read or write events to the Seq event stream. Seq apps have been around a long time, but they've been refreshed and updated recently, first to support non-.NET programming languages, and more recently, to support .NET Standard and a more efficient C# API.
Not much has been written about the .NET or C# changes yet, which is a shame because it's now super-easy, and a lot of fun, to programmatically analyze and act on log events.
This is the first of two parts. In it we'll write a simple output app, one that can receive events from Seq. Outputs are used for a lot of different things: triggering alerts via channels such as email and Slack, archiving data to long-term storage, automatically raising tickets in issue trackers, replicating Seq data to other systems, and more. The second part will look at input apps.
The plan
One of the fears of companies handling payments is for credit card numbers to inadvertently end up in application logs. There are many steps usually taken to prevent it, but the nature of log collection makes it hard to guarantee with absolute certainty that card numbers can never find their way into exception messages, chunks of request/response payloads, and so on.
If your domain is sensitive (health, banking, handling PII...) then you'll be able to think of a few similar things that you'd prefer didn't appear in logs.
The simple app we'll build is going to apply a regular expression to the full JSON representation of each log event, looking for 13 to 16-digit numbers joined with dashes and/or spaces, and raise the alarm if there's any match.
The example here is easy to understand but isn't meant to be ultra-realistic or production-quality. Most apps would need to parse and manipulate the JSON event data, and if you're working with similar requirements, your checks might have to be a bit more sophisticated 😁.
For the impatient, you can grab the source code for this example straight from GitHub and start hacking away.
File > New Project
Seq Apps are regular C# classes, in regular .NET Standard assemblies.
To build one, the first step is to create a .NET Standard class library, either in Visual Studio, or from the dotnet command line.
mkdir Seq.App.Detective
cd Seq.App.Detective
dotnet new classlib
The app needs a reference to the Seq.Apps package:
dotnet add package Seq.Apps -v 5.1.0
Does it build?
dotnet build
👍
The [SeqApp] type
After deleting the default Class1.cs file, it's time to add the plug-in type. Here's RegexDetective.cs:
using System;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Seq.Apps;
[SeqApp("Regular expression detector",
Description = "Checks complete event documents for text matching a regular expression.")]
public class RegexDetective : SeqApp, ISubscribeToJsonAsync
{
const int MaximumFragmentChars = 4096;
static readonly string Pattern = "\\b(?:\\d[ -]*?){13,16}\\b";
public Task OnAsync(string json)
{
var matches = Regex.Matches(json, Pattern);
if (matches.Count > 0)
{
var positions = matches.Cast<Match>()
.Select(m => m.Index)
.ToArray();
var sanitized = Regex.Replace(json, Pattern, m => new string('#', m.Value.Length));
var fragment = sanitized.Substring(0, Math.Min(sanitized.Length, MaximumFragmentChars));
Log.ForContext("Pattern", Pattern)
.ForContext("Fragment", fragment)
.ForContext("Positions", positions, destructureObjects: true)
.Error("Match detected");
}
return Task.CompletedTask;
}
}
You'll notice:
- The class is annotated with
[SeqApp]. This provides the description and help text for the Seq Settings > Apps screen. - The class inherits from
SeqApp. The base type has properties likeHostandLogfor interacting with the outside world. - The class implements
ISubscribeToJsonAsync- this is the newest of several ways for the app to receive events, in this case, they're passed to the app as raw JSON.
The logic is all there in OnAsync(): check for regular expression matches, and report the offending events when they're found.
Using Log....Error() sends diagnostic events back to Seq. There, they can be forwarded to a different app for alerting, for example to the Slack or email outputs.
Time to build!
dotnet build
Testing and debugging
Before packaging up the app and installing it into Seq, it can be tested locally. You'll need seqcli version 5.1.213 or newer for this.
Check the version of seqcli you have installed:
seqcli version
If it's not up-to-date, grab the latest archive for your operating system from the releases page.
Things will be made easier if you have Seq running locally on your development machine.
Once this is ready to go, log some events:
seqcli log -m "Processing a card"
seqcli log -m "Processing 1234 567890 1234"
Since they're the most recent events on the server, it's easy to read them back with:
seqcli search -c 2
You should see something like:
[2019-04-11T14:24:21.0563248+10:00 INF] Processing 1234 567890 1234 {}
[2019-04-11T14:23:54.7078096+10:00 INF] Processing a card {}
To test the app, we'll need to change into the directory containing the built DLLs:
cd bin\Debug\netstandard2.0
Using --json we can pipe the search results into the app:
seqcli search -c 2 --json | seqcli app run
Aha! A match:
{"@t":"2019-04-11T04:24:45.5443570Z","@mt":"Match detected","@l":"Error","Positions":[55],"Fragment":"{\"@t\":\"2019-04-11T04:24:21.0563248Z\",\"@mt\":\"Processing ################\"}","Pattern":"(?:\\d[ -]*?){13,16}"}
Tip: if this fails, you might be running into some character encoding issues that appear under Windows PowerShell. Using
CMDor PowerShell Core 6.x will help.
App settings
There isn't anything specific to credit card matching in our app, except for the regular expression we're searching for. To generalize the app, we can accept the pattern as an app setting, and configure different instances of the app to use different patterns:
[SeqAppSetting(HelpText = "A regular expression to search for.")]
public string Pattern { get; set; }
It's enough to make the pattern a public property and apply the [SeqAppSetting] attribute.
After rebuilding the package, we can now try different patterns on the command-line:
seqcli search -c 2 --json | seqcli app run -p Pattern="card"
And match different events:
{"@t":"2019-04-11T05:05:33.9565488Z","@mt":"Match detected","@l":"Error","Positions":[57],"Fragment":"{\"@t\":\"2019-04-11T04:23:54.7078096Z\",\"@mt\":\"Processing a ####\"}","Pattern":"card"}```
Packaging and deployment
Because our app doesn't have any dependencies apart from Seq.Apps, packaging it for Seq is as simple as:
dotnet pack
If you're writing a non-trivial app, chances are there are additional assemblies your app will need. Seq expects these to be in the NUPKG, too (it won't recursively install packages), so to do that you'll need to make a small modification to your CSPROJ file and use dotnet publish alongside pack in your build script. (These changes are already present in the example repository.)
Once we have a NUPKG file, it's time to load it up in Seq. If you already have a NuGet feed configured, push the package there. Otherwise, you can add the build output location (or a file share) as a feed under Seq's Settings > Feeds:
Now, to install the package, go to Settings > Apps > Install from NuGet, choose the feed, and enter the package id:
With the app installed, Add instance will present some settings for the app: an informational title, the source of events to send to it, and the pattern input for the app itself:
Stream incoming events needs to be checked if you want the app to receive events as they arrive.
The app in Seq
Now, whenever a matching event is logged, it should be detected:
seqcli log -m "This one is tricky" -p CardNumber=12345678901234
And, there it is!
The source code for this example is published on GitHub. Check out apps that others have written by searching the seq-app tag on NuGet.