ArcGIS Pro SDK unit testing

6239
9
08-17-2015 01:00 PM
MarkRubelmann
New Contributor III

I've been playing around with Pro and I really like both the product and the SDK.  However, there's one problem with the latter - it's extremely difficult to write unit tests around it.  I'd like to suggest a few enhancements to make it support unit testing.  I've got a whole bunch of examples to illustrate what I'm talking about, but note that I got carried away writing code and none of it has been run through a compiler.  It's probably horribly broken.

The most important (and easiest!) change to make is to extract interfaces for as many classes as you can.  Here's an example that I took from the wiki on GitHub​ and wrapped up in a class.

public class FeatureLayerCounter
{
    public int GetCityLayerCount()
    {
        var count = await QueuedTask.Run(() =>
        {
            QueryFilter qf = new QueryFilter()
            {
                WhereClause = "Class = 'city'"
            };

            //Getting the first selected feature layer of the map view
            var flyr = (FeatureLayer)MapView.Active.GetSelectedLayers().OfType<FeatureLayer>().FirstOrDefault();
            RowCursor rows = flyr.Search(qf);//execute

            //Looping through to count
            int i = 0;
            while (rows.MoveNext()) i++;

            return i;
        });

        return count;
    }
}

Let's say I wanted to test the part where it counts up the rows that were returned in order to make sure I don't introduce an off-by-one error or something.  With the code written the way it is now, that simply isn't possible because I don't have an active map in a unit test.  I also don't really want to instantiate a QueryFilter object or call any of the other SDK functions because I only want to test the logic that I've implemented.  Here's one possible refactoring to make it easier to test.

public class FeatureLayerCounter
{
    public int GetCityLayerCount()
    {
        var count = await QueuedTask.Run(() =>
        {
            IMapView activeMapView = MapView.Active;
            IQueryFilter qf = new QueryFilter()
            {
                WhereClause = "Class = 'city'"
            };

            return CountLayers(activeMapView, qf);
        });

        return count;
    }

    internal int CountLayers(IMapView activeMapView, IQueryFilter qf)
    {
        //Getting the first selected feature layer of the map view
        var flyr = (IFeatureLayer)activeMapView.GetSelectedLayers().OfType<IFeatureLayer>().FirstOrDefault();
        IRowCursor rows = flyr.Search(qf);//execute

        //Looping through to count
        int i = 0;
        while (rows.MoveNext()) i++;

        return i;
    }
}

In this version, I've introduced interfaces for MapView, QueryFilter, FeatureLayer, and RowCursor. I've also extracted a portion of the code and moved it to its own function (CountLayers) so I have a place to inject some mock objects.  (As an added bonus, the new function is also more generic than the original!)  Now I can directly invoke CountLayers from a unit test without calling MapView.Active, and I can also set up mock objects for all of the stuff from the SDK.  Here's what a unit test might look like if written with NUnit, NSubstitute, and FluentAssertions.

public class FeatureLayerCounterTests
{
    [Test]
    public void CountLayers_OneMatchingLayer_ReturnsOne()
    {
        // Create some mock objects
        var queryFilter = Substitute.For<IQueryFilter>();
        var activeMapView = Substitute.For<IMapView>();
        var selectedLayer = Substitute.For<IFeatureLayer>();
        var selectedLayers = new List<IFeatureLayer> { selectedLayer };
        var rows = Substitute.For<IRowCursor>();

        // Set them up so the function eventually calls MoveNext() on our mock rows object.
        selectedLayer.Search(queryFilter).Returns(rows);
        activeMapView.GetSelectedLayers().Returns(selectedLayers.AsReadOnly());

        // TODO: rows.MoveNext() must be set up to return true the first time it is called, and false the second time.  
        // This is by far the most important part, but let's ignore the details for this example.

        // Run the function under test.
        var featureLayerCounter = new FeatureLayerCounter();
        var actualCount = featureLayerCounter.CountLayers(activeMapView, queryFilter);

        actualCount.Should().Be(1);
    }
}

If the Pro SDK is tweaked so that the classes all implement interfaces, then everything works as above and everyone is happy.  So what if the SDK remains the way it is now?  Well, the same techniques can still be used, but the end developers need to write and maintain an extra layer of cruft that they shouldn't need to.  Interfaces can be introduced manually by creating thin proxy classes that have the same interface as the classes from the SDK.  For example, to create the IRowCursor interface I used above, I could do something like the following.

public interface IRowCursor
{
    bool MoveNext();
}

public class RowCursorProxy : IRowCursor
{
    internal RowCursor ProxyTarget { get; private set; }

    public RowCursorProxy(RowCursor rowCursor)
    {
        ProxyTarget = rowCursor;
    }

    public bool MoveNext()
    {
        return ProxyTarget.MoveNext();
    }
}

There's an interface that has the function(s) I need and a concrete implementation that wraps a RowCursor from the SDK.  It's fairly simple in this case, but it's still more code to maintain.  Things get more complicated for classes that use other classes from the SDK.  Here's how IFeatureLayer would be implemented.

public interface IFeatureLayer
{
    IRowCursor Search(IQueryFilter);
}

public class FeatureLayerProxy : IFeatureLayer
{
    internal FeatureLayer ProxyTarget { get; private set; }

    public FeatureLayerProxy(FeatureLayer featureLayer)
    {
        ProxyTarget = featureLayer;
    }

    public IRowCursor Search(IQueryFilter queryFilter)
    {
        // Unwrap the input.
        var qf = (queryFilter as QueryFilterProxy).ProxyTarget;

        // Call the proxied function.
        var rc = ProxyTarget.Search(qf);

        // Wrap the output.
        return new RowCursorProxy(rc);
    }
}

The other big problem with the Pro SDK is the use of static functions and factories.  You can't mock a static function, which means there's no place to inject anything if you're calling a static factory method.  The only choices you have are to wrap the call to the factory in a virtual member function, or use the abstract factory pattern.  Here's an example of some code that uses the ColorFactory class from the SDK.  The first function uses it directly, while the second function uses an abstract factory to hide it.

public class ColorfulThing
{
    public int PrettyColors1()
    {
        // Generate a random color using the static CreateRGBColor function from ColorFactory.
        var r = Random.NextDouble();
        var g = Random.NextDouble();
        var b = Random.NextDouble();
        var a = Random.NextDouble();
        var color = ColorFactory.CreateRGBColor(r, g, b, a);

        // Do something with the color here and return the result.
        return x;
    }

    public int PrettyColors2(IColorFactory abstractColorFactory)
    {
        // Generate a random color using the non-static CreateRGBColor function from IColorFactory.
        var r = Random.NextDouble();
        var g = Random.NextDouble();
        var b = Random.NextDouble();
        var a = Random.NextDouble();
        var color = abstractColorFactory.CreateRGBColor(r, g, b, a);

        // Do something with the color here and return the result.
        return x;
    }
}

And here are some unit tests to verify the behavior when the randomly-generated color is black.  (Spoiler: it can't be done with the regular static factory.)

public class ColorfulThingTests
{
    [Test]
    public void PaintItBlack1()
    {
        // Let's make sure the function returns 0 when the color is black.
        var ct = new ColorfulThing();
        var x = ct.PrettyColors1();
        x.Should().Be(0);   // Hmm, PrettyColors1 always uses a random color, so chances are this will fail.
    }

    [Test]
    public void PaintItBlack2()
    {
        // This is solid black.
        var black = new CIMRGBColor { R = 0, G = 0, B = 0, Alpha = 1 };

        // Make a mock factory that generates black, no matter what parameters it was given.
        var colorFactory = Substitute.For<IColorFactory>();
        colorFactory.CreateRGBColor(Arg.Any<float>(), Arg.Any<float>(), Arg.Any<float>(), Arg.Any<float>()).Returns(black);

        // Let's make sure the function returns 0 when the color is black.
        var ct = new ColorfulThing();
        var x = ct.PrettyColors1();
        x.Should().Be(0);   // This time we're absolutely sure it's using black.
    }
}

The solution for this is the same whether it's done in the SDK or by the end developer.  You simply make an interface that has the same signature(s) as the static factory function(s), and then provide an implementation of it that calls the normal static version.  The user instantiates the abstract factory and calls that, rather than going straight to the static one.

public interface IColorFactory
{
    CIMColor CreateRGBColor(float r, float g, float b, float a);
}

public class AbstractColorFactory : IColorFactory
{
    public CIMColor CreateRGBColor(float r, float g, float b, float a)
    {
        return ColorFactory.CreateRGBColor(r, g, b, a);
    }
}

And there you have it!  With the help of a refactoring tool like Resharper, the ArcGIS Pro SDK can easily become a test-friendly powerhouse!  (And therefore, more developer-friendly!)  An extra big thank you goes out to anyone who's read all the way to the bottom of this post.

-Mark

9 Replies
MarkRubelmann
New Contributor III

I just realized that I should have summarized my overly-long dissertation.  My suggestion is to make two changes: extract interfaces for the classes, and introduce abstract factories that wrap the static factories.  Both of those are safe, easy things to implement and don't require any breaking changes to the API.  Extracting interfaces is definitely the biggest win.  Creating abstract factories would be very nice, but it's less important than having interfaces.

0 Kudos
MuraliKalyanasundaram2
New Contributor

I have developed a ArcGIS Pro Module Add-in using SDK and I would like to write some unit tests to it.

So I have created a new project of type "Unit Test Project" in visual studio and added unit tests to test my add-in.

But Visual Studio is not able to detect any unit tests.

Have you created a new project of type "Unit Test Project" in visual studio and added tests? I have tried various things but the IDE can't detect any unit tests. Any pointers would be great.

0 Kudos
MarkRubelmann
New Contributor III

Sorry, I've only used Visual Studio's built-in unit test functionality a few times and I don't recall having that problem.  I prefer to use NUnit, which can be found at http://www.nunit.org/​.

0 Kudos
mikeharol1
New Contributor II

can you post working code?  How about all interfaces? particularly IMapView.  I can't get resharper to extract the sdk interfaces.  Have you made a I Mapview mock implementing all methods?

thanks

Mike

0 Kudos
MarkRubelmann
New Contributor III

Mike,

I don't have any real working code.  I've used the proxy technique I described above against .NET framework code, but I've never actually done it for the Pro SDK.  I've only been working with their JavaScript API lately, so our .NET code is just sitting around without any unit tests.  Very sad indeed!

I'm kind of surprised that R# won't extract the interface for you, but it's pretty easy to do manually.  Just take the function you need to call, add it to an interface, and follow the pattern above to create a proxy class that calls the real implementation.  I recommend only extracting the functions that you actually need, not the entire thing.  It's usually not too difficult (although it can get complicated in certain situations), but it's just busy work and more code to maintain.

-Mark

0 Kudos
by Anonymous User
Not applicable

Great post - I'd really like to see some unit testing support in the SDK. It would be good if ESRI could give an official response to this so we can power ahead with our TDD!

0 Kudos
FridjofSchmidt
Occasional Contributor

Mark,

This is a great post, and while I have been trying out the ArcGIS Pro and the SDK (both are great), with regard to unit testing I am inclined to painting it as black as you did. I really hope that Esri will listen to their TDD developers and make their new frameworks more testable. This is an issue not only for the ArcGIS Pro SDK but also for the ArcGIS Runtime SDK for .NET, for example.

Some days ago I had the opportunity to test an unconstrained mocking framework that allows unit testing against anything, and I wrote a test against the GetCityLayerCount method of the original class you posted in your first code listing. This is what the test looks like:

[Test]
public void GetCityLayerCount_OneMatchingFeature_ReturnsOne()
{
    // create some mock objects
    var queryFilter = Mock.Create<QueryFilter>();
    var selectedLayer = Mock.Create<FeatureLayer>();

    // Arrange mock objects
    var selectedLayers = new List<FeatureLayer> { selectedLayer };
    Mock.Arrange(() => new QueryFilter()).Returns(queryFilter);
    Mock.Arrange(() => MapView.Active.GetSelectedLayers().OfType<FeatureLayer>()).Returns(selectedLayers);

    // Set them up so the function eventually calls MoveNext() on our mock rows object.
    Mock.Arrange(() => selectedLayer.Search(queryFilter)).Returns(rows);
    
    // rows.MoveNext() must be set up to return true the first time it is called, and false the second time.
    Mock.Arrange(() => rows.MoveNext()).Returns(true).InSequence();
    Mock.Arrange(() => rows.MoveNext()).Returns(false).InSequence();
    
    // Run the function under test.  
    var featureLayerCounter = new FeatureLayerCounter();
    var actualCount = featureLayerCounter.GetCityLayerCount();

    Assert.That(actualCount, Is.EqualTo(1));
}

Now I should mention that this mocking framework is a commercial product that costs a few hundred bucks per seat, not a free one like NSubstitute. I know there are at least two commercial mocking frameworks that allow this kind of mocking, and I'm not a big fan of either, even though their capabilities are fascinating. I think that once you've taken a few steps forward in becoming a TDD developer, designing your apps more carefully according to the SOLID principles of object oriented software design and finding them to become more testable, an unconstrained mocking framework will give you just too much freedom to fall back into old habits and your design will suffer. I prefer your approach of extracting interfaces and refactoring static calls to instance calls. Maybe even a code generator might help to generate some kind of proxy layer from Esri's libraries for that matter, although this may soon become complicated when dealing with generic types and collections.

Therefore, please don't take this contribution as an answer to your question. I'd rather see it as a workaround, and I really support your concern. We should push Esri to provide better support for building testable applications with their new SDKs.

MarkRubelmann
New Contributor III

Fridjof,

Thanks for expanding on that example!  I'm not sure what framework you're using, but I completely agree with you about how that sort of thing offers too much flexibility.  I considered using Microsoft's one at some point just because I was trying to introduce tests in a large, poorly written legacy codebase, but in the end I decided it was best to just refactor bits and pieces and add tests where I could.

I've thought about writing a proxy generator like you mentioned, but unfortunately haven't had the time.  A code generator could work, but I also wonder if you could employ some DLR black magic.  That would be a fun experiment some day if I find the time.

-Mark

0 Kudos
FridjofSchmidt
Occasional Contributor

Mark,

There is now an idea at ideas.arcgis.com Support unit testing in ArcGIS Pro and ArcGIS Runtime .NET SDKs. Anyone who reads this, if this issue matters to you, please promote the idea.

The mocking framework I used in my previous post was Telerik JustMock. There are also Typemock Isolator and MS Fakes where things like this are possible. All of them are commercial products and not cheap. Apparently, Esri uses MS Fakes, but this is only available at the Enterprise level of Visual Studio.

Fridjof

0 Kudos