Runar Ovesen Hjerpbakk

Software Philosopher

Testing Azure Blob Storage using xUnit

Azure Blob Storage is persistent cloud data storage for any kind of unstructured data. Slack Profilebot uses this service to store a whitelist of users with incomplete profiles that should not be reminded to update their profiles. Every user is represented by a file containing that user’s id. This is complete overkill, I know, but it served a purpose. It taught me both how to use and test Blob Storage.

Using Azure Blob Storage from C#

Microsofts documentation has really improved these last couple of years, so getting started is as easy as reading this short documentation and following it.

In my code I used the Microsoft Azure Storage Client Library for .NET nuget package and you can see it in action in Profilebot’s FaceWhitelist class.

Run tests dependant on Azure Blob Storage

The FaceWhitelistTests need to run both locally and on the build server. Microsoft has made available a storage emulator which can be used during development and testing. You will not pollute Azure with data from the tests, and your running costs will depend only on your production scenario.

After the Microsoft Azure Storage Emulator is installed, it can be started like any other application.

Then all that your code needs to do is to connect to Blob Storage using this connection string:

var storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true;");

Automatic testing with xUnit.net

Writing automated tests against Blob Storage is no different than writing tests against any other data layer. You need to ensure that the data layer is in a known state prior to the test run, and be certain that any changes to the data are what you expected and actually from your test run.

A simple known state is a totally empty Blob Storage without any data, thus the flow for the tests can go like this:

  1. Stop the storage emulator if it’s running (argument stop)
  2. Clear all data (argument clear all)
  3. Start the storage emulator (argument start)
  4. Run the integration tests
  5. Stop the storage emulator after the tests have run to completion (argument stop)

I write my unit and integration tests with xUnit.net and this framework has the concept of Class Fixtures:

Class Fixtures should be used when you want to create a single test context and share it among all the tests in the class, and have it cleaned up after all the tests in the class have finished.

The simple example looks like this:

public class DatabaseFixture : IDisposable {
    public DatabaseFixture() {
        Db = new SqlConnection("MyConnectionString");
        // ... initialize data in the test database ...
    }

    public void Dispose() {
        // ... clean up test data from the database ...
    }

    public SqlConnection Db { get; private set; }
}

public class MyDatabaseTests : IClassFixture<DatabaseFixture> {
    DatabaseFixture fixture;

    public MyDatabaseTests(DatabaseFixture fixture) {
        this.fixture = fixture;
    }

    // ... write tests, using fixture.Db to get access to the SQL Server ...
}

Here the connection to the db is created only once and disposed when the tests are completed.

Doing the same for tests using Blob Storage can look like this. The storage emulator is started in its own process using the required argument.

public class BlobStorageFixture : IDisposable {
    readonly Process process;

    public BlobStorageFixture() {
        process = new Process {
            StartInfo = {
                UseShellExecute = false,
                FileName = @"C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator\AzureStorageEmulator.exe",
            }
        };

        StartAndWaitForExit("stop");
        StartAndWaitForExit("clear all");
        StartAndWaitForExit("start");
    }

    public void Dispose() {
        StartAndWaitForExit("stop");
    }

    void StartAndWaitForExit(string arguments) {
        process.StartInfo.Arguments = arguments;
        process.Start();
        process.WaitForExit(10000);
    }
}

public class FaceWhitelistTests : IClassFixture<BlobStorageFixture> {
    public FaceWhitelistTests(BlobStorageFixture fixture) { }

    [Fact]
    public async Task WhitelistUser_NewUser_IsWhitelisted() {
        var faceWhitelist = Create();
        var userToWhiteList = new SlackUser {Id = Path.GetRandomFileName()};

        await faceWhitelist.WhitelistUser(userToWhiteList);

        await VerifyUserIsWhiteListed(userToWhiteList);
    }

    // ... more test cases exist IRL

    static FaceWhitelist Create() =>
    new FaceWhitelist(BlobStorageConfiguration.Local);

    static async Task VerifyUserIsWhiteListed(SlackUser user) {
        var storageAccount = CloudStorageAccount.Parse(BlobStorageConfiguration.Local.ConnectionString);
        var blobClient = storageAccount.CreateCloudBlobClient();
        var container = blobClient.GetContainerReference("whitelist");
        using (var memoryStream = new MemoryStream()) {
            await container.GetBlockBlobReference(user.Id).DownloadToStreamAsync(memoryStream);
            var actualUserId = Encoding.UTF8.GetString(memoryStream.ToArray());
            Assert.Equal(user.Id, actualUserId);
        }
    }
}

Profilebot’s builds run on AppVeyor and their build server images come pre-loaded with the Azure tooling, including the storage emulator. So this little BlobStorageFixture trick was all that was needed for once again achieving 99% code coverage (though Visual Studio has it at 100%…).