Automated Testing Support

Jasper was built intentionally with testability as a first class design goal.

General Guidance

• For any kind of integration testing, the Jasper team suggests bootstrapping your Jasper application to an IHost in a test harness as closely to the production application setup as you can -- minus inconvenient external dependencies
• To isolate your Jasper application from any kind of external transports that you might not want to access locally, use the transport stubbing explained in a section below
• If your message handlers do not use any kind of Jasper middleware, it might be easy enough to simply resolve your handler class from the underlying IoC container. Use IHost.Services.GetRequiredService<T>() to resolve the handler objects, and Lamar is able to figure out a construction strategy on the fly without any kind of prior registration.
• If your message handlers do not involve any kind of cascading messages, use ICommandBus.Invoke() to execute a message inline
• If your message handler does involve cascading messages, use the message tracking support explained in the next session
• If you are trying to coordinate a test across multiple Jasper applications, see the section on Message Tracking across Systems

Message Tracking

Jasper is a successor to a much earlier project called FubuMVC. One of the genuinely successful parts of FubuMVC was a mechanism to coordinate automated testing of its messaging support as described in these blog posts:

• Automated Testing of Message Based Systems
• Reliable and “Debuggable” Automated Testing of Message Based Systems in a Crazy Async World

Jasper's version of this feature is much improved from FubuMVC, and comes out of the box to make testing scenarios easier.

Automated testing against asynchronous processing applications can be very challenging. Let's say that when you're application handles a certain message it also sends out a couple cascading messages that in turn cause changes in state to the system that you want to verify in your automated tests. The tricky part is how to invoke the original message, but then waiting for the cascading operations to complete before letting the test harness proceed to verifying the system state. Using Thread.Sleep() is one alternative, but usually results in either unnecessarily slow or horrendously unreliable automated tests.

To at least ameliorate the issues around timing, Jasper comes with the "Message Tracking" feature that can be used as a helper in automated testing. To enable that in your applications, just include the extension as shown below:

public class AppUsingMessageTracking : JasperOptions
{
    public override void Configure(IHostEnvironment hosting, IConfiguration config)
    {
        if (hosting.IsDevelopment() || hosting.IsEnvironment("Testing"))
        {
            // This is necessary to add the message tracking
            // to your Jasper application
            Extensions.UseMessageTrackingTestingSupport();
        }
    }
}

Now, in testing you can use extension methods off of IHost that will execute an action with the service bus and wait until all the work started (messages sent should be received, cascading messages should be completed, etc.) has completed -- or it times out in a reasonable time. The message tracking will throw an exception if it times out without completing, and the exception will list out all the detected activity to try to help trouble shoot where things went wrong.

To use the message tracking, consider this skeleton of a test:

public async Task invoke_a_message()
{
    using (var host = JasperHost.For<AppUsingMessageTracking>())
    {
        await host.ExecuteAndWait(x => x.Invoke(new Message1()));

        // check the change in system state after the original
        // message and all of its cascading messages
        // finish
    }
}

The other usages of message tracking are shown below:

public async Task other_usages()
{
    using (var runtime = JasperHost.For<AppUsingMessageTracking>())
    {
        // Call IMessageContext.Invoke() and wait for all activity to finish
        await runtime.InvokeMessageAndWait(new Message1());

        // Configurable timeouts
        await runtime.InvokeMessageAndWait(new Message1(),
            10000);

        // More general usage to send a single message and wait
        // for all activity to complete
        await runtime.ExecuteAndWait(() => runtime.Send(new Message1()));


        // Using an isolated message context
        await runtime.ExecuteAndWait(c => c.Send(new Message1()));

        // Assert that there were no exceptions during the processing
        // If there are, this will throw an AggregateException of
        // all encountered exceptions in the message processing
        var session = await runtime.ExecuteAndWait(c => c.Send(new Message1()));
    }
}

All of these methods return an ITrackedSession object that gives you access to the tracked activity. Here's an example from Jasper's tests that uses ITrackedSession:

[Fact]
public async Task track_outgoing_to_tcp_when_stubbed()
{
    using (var host = JasperHost.For(options =>
    {
        options.Endpoints.PublishAllMessages().ToPort(7777);
        options.Endpoints.StubAllExternallyOutgoingEndpoints();
        options.Extensions.UseMessageTrackingTestingSupport();
    }))
    {
        var message = new Message1();

        // The session can be interrogated to see
        // what activity happened while the tracking was
        // ongoing
        var session = await host.SendMessageAndWait(message);

        session.FindSingleTrackedMessageOfType<Message1>(EventType.Sent)
            .ShouldBeSameAs(message);
    }
}

The ITrackedSession interface looks like this:

public interface ITrackedSession
{
    TrackingStatus Status { get; }

    T FindSingleTrackedMessageOfType<T>();

    IEnumerable<object> UniqueMessages();

    IEnumerable<object> UniqueMessages(EventType eventType);

    T FindSingleTrackedMessageOfType<T>(EventType eventType);

    EnvelopeRecord[] FindEnvelopesWithMessageType<T>(EventType eventType);

    EnvelopeRecord[] FindEnvelopesWithMessageType<T>();

    EnvelopeRecord[] AllRecordsInOrder();

    bool HasNoRecordsOfAnyKind();

    EnvelopeRecord[] AllRecordsInOrder(EventType eventType);
}

Message Tracking with External Transports

By default, the message tracking runs in a "local" mode that logs any outgoing messages to external transports, but doesn't wait for any indication that those messages are completely received on the other end. To include tracking of activity from external transports even when you are testing one application, use this syntax shown in an internal Jasper test for the Azure Service Bus support:

[Fact]
public async Task can_stop_and_start()
{
    using (var host = JasperHost.For<ASBUsingApp>())
    {
        await host
            // The TrackActivity() method starts a Fluent Interface
            // that gives you fine-grained control over the
            // message tracking
            .TrackActivity()
            .Timeout(30.Seconds())
            // Include the external transports in the determination
            // of "completion"
            .IncludeExternalTransports()
            .SendMessageAndWait(new ColorChosen {Name = "Red"});

        var colors = host.Get<ColorHistory>();

        colors.Name.ShouldBe("Red");
    }
}

Stubbing Outgoing External Transports

Sometimes you'll need to develop on your Jasper application without your application having any access to external transports for one reason or another. Jasper still has you covered with the feature JasperOptions.Endpoints.StubAllExternallyOutgoingEndpoints() shown below:

public static IHostBuilder CreateHostBuilder() =>
    Host.CreateDefaultBuilder()

        // This adds Jasper with inline configuration
        // of JasperOptions
        .UseJasper((context, opts) =>
        {
            // This is an example usage of the application's
            // IConfiguration inside of Jasper bootstrapping
            var port = context.Configuration.GetValue<int>("ListenerPort");
            opts.Endpoints.ListenAtPort(port);

            // If we're running in development mode and you don't
            // want to worry about having all the external messaging
            // dependencies up and running, stub them out
            if (context.HostingEnvironment.IsDevelopment())
            {
                // This will "stub" out all configured external endpoints
                opts.Endpoints.StubAllExternallyOutgoingEndpoints();

                opts.Extensions.UseMessageTrackingTestingSupport();
            }
        });

When this is active, Jasper will simply not start up any of the configured listeners or subscribers for external transports. Any messages published to these endpoints will simply be ignored at runtime -- but you can still use the message tracking feature shown above to capture the outgoing messages for automated testing.

Integration with Storyteller

Jasper comes with a pre-built recipe for doing integration or acceptance testing with Storyteller using the Jasper.TestSupport.Storyteller extension library.

To get started with this package, create a new console application in your solution and add the Jasper.TestSupport.Storyteller Nuget dependency. Next, in the Program.Main() method, use this code to connect your application to Storyteller:

JasperStorytellerHost.Run<MyJasperAppOptions>(args);

In this case, MyJasperAppRegistry would be the name of whatever the JasperRegistry class is for your application.

If you want to hook into events during the Storyteller bootstrapping, teardown, or specification execution, you can subclass JasperStorytellerHost<T> like this:

public class MyJasperStorytellerHarness : JasperStorytellerHost<MyJasperAppOptions>
{
    public MyJasperStorytellerHarness()
    {
        // Customize the application by adding testing concerns,
        // extra logging, or maybe override service registrations
        // with stubs
        Registry.Services.AddSingleton<ISomeService, ISomeService>();
    }

    protected override void beforeAll()
    {
        // Runs before any specification are executed one time
        // Perfect place to load any kind of static data

        // Note that you have access to the JasperRuntime
        // of the running application here
        Host.Services.GetService<ISomeService>().StartUp();
    }

    protected override void afterEach(ISpecContext context)
    {
        // Called immediately after each specification is executed
        Host.Services.GetService<ISomeService>().CleanUpTestRunData();
    }

    protected override void beforeEach()
    {
        // Called immediately before each specification is executed
        Host.Services.GetService<ISomeService>().LoadTestingData();
    }

    protected override void afterAll()
    {
        // Called right before shutting down the Storyteller harness
        Host.Services.GetService<ISomeService>().Shutdown();
    }
}

Then, your bootstrapping changes slightly to:

StorytellerAgent.Run(args, new MyJasperStorytellerHarness());

MessagingFixture

Jasper.Storyteller also comes with a MessagingFixture base class you can use to create Storyteller Fixtures that send messages to the running service bus with some facility to use the built in

Unknown topic key 'documentation/testing/message_tracking' -- CTRL+SHIFT+R to force refresh the topic tree

to "know" when all the activity related to the message being sent has completed.

Here's a sample MessagingFixture from the sample project:

public class TeamFixture : MessagingFixture
{
    [FormatAs("A new team {team} has joined the league")]
    public Task CreateNewTeam(string team)
    {
        // This method sends a message to the service bus and waits
        // until it can detect that the message has been fully processed
        // on the receiving side or timed out
        return Host.SendMessageAndWait(new TeamAdded {Name = team});
    }

    [FormatAs("On {day}, the score was {homeTeam} {homeScore} vs. {visitorTeam} {visitorScore}")]
    public Task RecordGameResult(DateTime day, string homeTeam, int homeScore, string visitorTeam, int visitorScore)
    {
        var message = new GamePlayed
        {
            Date = day.Date,
            Home = new TeamResult {Name = homeTeam, Score = homeScore},
            Visitor = new TeamResult {Name = visitorTeam, Score = visitorScore}
        };

        return Host.SendMessageAndWait(message);
    }

    [FormatAs("Send an un-handled message")]
    public Task SendUnHandledMessage()
    {
        return Host.SendMessageAndWait(new UnhandledMessage());
    }
}

Diagnostics

If there is any messages sent or received by the service bus feature during a Storyteller specification, there will be a custom results tab called "Messages" in the Storyteller specification results that presents information about the message activity that will look like this:

content/storyteller-messaging-log.png