Store and Forward Messaging Edit on GitHub


Jasper supports durable message persistence using your application's database for "store and forward" queueing with all possible Jasper transport options, including the built in Store and Forward Messaging, RabbitMQ Transport. and Azure Service Bus Transport.

If a Jasper system that uses durable messaging goes down before all the messages are processed, the persisted messages will be loaded from storage and processed when the system is restarted. Jasper does not include any kind of persistence in the core Jasper library, so you'll have to use an extension library to add that behavior. Today the options are:

  1. A Marten/Postgresql backed option
  2. A Sql Server backed option

With an option based on EF Core planned for later.

To use the built in TCP Transport in a durable way, just use the schema durable instead of tcp like so:


public class DurableTransportApp : JasperRegistry
{
    public DurableTransportApp()
    {
        // Set up a listener (this is optional)
        Transports.DurableListenerAt(2200);

        // Or, alternatively set up durable listening by Uri
        Transports.ListenForMessagesFrom("tcp://localhost:2200/durable");

        // Or, alternatively set up durable listening by Uri
        Transports.ListenForMessagesFrom("durable://localhost:2200");

        // With the RabbitMQ transport
        Transports.ListenForMessagesFrom("rabbitmq://server1/durable/queue1");
    }
}

See the blog post Durable Messaging in Jasper for more context behind the durable messaging.

Message Storage in Testing

Let's say that we're all good developers who invest in automated testing of our applications. Now, let's say that we're building a Jasper application that uses Sql Server backed message persistence like so:


public class MyJasperApp : JasperRegistry
{
    public MyJasperApp()
    {
        // Enables Sql Server-backed message persistence using
        // a connection string from your application's appsettings.json
        Settings.PersistMessagesWithSqlServer((c, settings) =>
        {
            settings.ConnectionString = c.Configuration["connectionString"];
        });
    }
}

If we write integration tests for our application above, we need to guarantee that as part of the test setup the necessary Sql Server schema objects have been created in our test database before we run any tests.

Fortunately, Jasper comes with an extension method hanging off of IJasperHost called RebuildMessageSchema() that will completely rebuild all the necessary schema objects for message persistence. Below is an example of using an xUnit shared fixture approach for integration tests of the MyJasperApp application.


public class MyJasperAppFixture : IDisposable
{
    public MyJasperAppFixture()
    {
        Host = JasperHost.For<MyJasperApp>();

        // This extension method will blow away any existing
        // schema items for message persistence in your configured
        // database and then rebuilds the message persistence objects
        // before the *first* integration test runs
        Host.RebuildMessageStorage();
    }

    public IJasperHost Host { get;  }

    public void Dispose()
    {
        Host?.Dispose();
    }


}

// An xUnit test fixture that uses our MyJasperAppFixture
public class IntegrationTester : IClassFixture<MyJasperAppFixture>
{
    private readonly MyJasperAppFixture _fixture;

    public IntegrationTester(MyJasperAppFixture fixture)
    {
        _fixture = fixture;
    }
}

Message Storage from the Command Line

As of Jasper v0.9.5, Jasper comes with a built in command for adminstering database backed persistence. Assuming that you're using Jasper's command line support, you have the command storage with several options.

At the command line in the root of your application, you can rebuild the message storage schema objects with:

dotnet run -- storage rebuild

You can also query the current counts of persisted input, output, and scheduled messages with:

dotnet run -- storage counts

You can dump the SQL to create the necessary database objects to a file for usage in database migration scripts with:

dotnet run -- storage script --file SomeFilePath.sql

And lastly, if you just want to clear out any persisted incoming, outgoing, or scheduled messages in your application's database, use:

dotnet run -- storage clear

Durable Messaging to External Systems

To utilize Jasper's durable messaging support and associated outbox support with other external systems, just utilize a Jasper message handler to do the actual integration with the external system. For example, you can send messages to an external web service by making the HttpClient call inside of a Jasper message handler through a durable loopback queue.