Durable log shipping turned out to be an unexpectedly popular feature in the Serilog sink for Seq. In this mode, the sink writes log events to a local buffer file, and sends them to Seq in the background using a cursor into the buffer file.
The advantage over regular batched HTTP shipping is that events aren’t lost when connectivity with the Seq server is temporarily broken, or if the app terminates while events are still queued to be sent.
Some of the limitations of the approach prevent the technique being used more widely however:
- Only one process can use a buffer file; this is tough to get right when using IIS hosted ASP.NET apps, or when collecting logs from Windows desktop applications that can run more than one instance
- Issues in the shipping process are hard to surface; oversized events, file corruption, connection issues – because the sink has no UI of its own, and nowhere it can reasonably show status information, management is harder if there are a lot of apps involved
For these reasons, we’ve only recommended using durable mode when absolutely necessary – but for many, especially occasionally-connected apps – that necessity is apparent.
A new component, Seq Forwarder, is designed to pick up where durable log shipping leaves off. Initially, the forwarder will target the “ocasionally connected” scenario, but future work on additional input types may also address the durable-during-termination requirement.
Seq Forwarder runs as a Windows Service on the remote machines, and accepts events through HTTP connections on localhost. Instead of the Seq default port 5341, the forwarder listens on 15341:
The HTTP endpoint is identical to the Seq Server one, so standard client libraries can be used.
Log.Logger = new LoggerConfiguration()
.WriteTo.Seq("http://localhost:15341")
.CreateLogger();
Log.Information("Hello, Seq Forwarder!");
The forwarder doesn’t itself require an API key – make sure inbound connections on port 15341 are denied by the application machine’s firewall (on Windows this will be the default).
Events are buffered in a local database (via Lightning.NET) until they can be batched up and sent to the remote Seq machine.
The size of the buffer can be capped: by default, up to 64 MB will be used, so that an unmonitored machine doesn’t exhaust its local storage. The forwarder uses a configuration file in C:\ProgramData\Seq\Forwarder and provides a command-line tool for administration:
seq-forwarder config -k storage.bufferSizeBytes -v 1073741824
seq-forwarder restart
While it’s in development, Seq Forwarder is not listed on the main website. Grab the MSI installer (with setup wizard) here.
We'd love your feedback and help shaping the direction of this project and we're excited about the possibilities it presents. Please file any issues you find on the Seq issue tracker.