Last Updated December 2019, 6 min. read

Test your game code with the Unity Test Framework

What you will get from this page: Tips on how to use the Unity Test Framework (UTF) to QA your projects. The UTF is one of the most important tools for quality assurance; it is used to run automated tests both in the Editor and on supported platforms. And it's available to all Unity users!  

A new Test Runner API brings greater flexibility and extensibility to the UTF, for most of your testing needs. It’s available via the Unity Package Manager, so we can ship bug fixes and new updates faster. This also means you can access the source code of the UTF locally. You can look at the source code, step through it when debugging, and modify it.  

To get the full story, watch the UTF session at Unite Copenhagen, featuring Unity tooling and testing developers Richard Fine and Christian Warnecke.

Test your game code with Unity Test Framework

Getting started with the Unity Test Framework

If you are new to the Unity Test Framework (UTF), read the documentation for an introduction. In brief, the UTF enables Unity users to test their code in both Edit Mode and Play Mode, and also on target platforms such as Standalone, Android, iOS, and others.

UTF uses a Unity integration of the NUnit library, which is an open-source unit testing library for .Net languages. For more information about NUnit, see the official NUnit website and the NUnit documentation on GitHub

Newcomers will find these blog posts informative:

Performance benchmarking in Unity: How to get started

Testing test-driven development with the Unity Test Runner

Overview of the Test Runner API

You can run your tests programmatically from any script via the Test Runner API (see API below). It allows you to retrieve a list of tests that will run in Edit Mode, Play Mode, or both, without running them. You can hook into some register/unregister callbacks at the start and finish of each test and at each level within the testing cycle, that is, on the entire test assembly, on each individual test fixture, and on each test class and test. 

At the start of each test, you get information about the test route that is about to be run. When the test finishes, you see the test results. 

In addition to running the UTF in Play Mode in the Unity Editor, a new customization point lets you run it on target devices. This is called before you build the Player; you can modify the Player build options, for example, to change the test run settings and to specify the build locations.

TestRunnerAPI.cs (C#)
void Execute(ExecutionSettings executionSettings);
void RegisterCallbacks<T>(T testCallbacks, int priority = 0) where T : ICallbacks;
void UnregisterCallbacks<T>(T testCallbacks) where T : ICallbacks;
void RetrieveTestList(TestMode testMode, Action<ITestAdaptor> callback);

Splitting build and run

Splitting the build and run processes is useful when you want to run tests on a target device that is not attached to your local machine, for example, if it is in the cloud (or is multiple devices in the cloud). 

To do this, you first need to customize the test Player build process itself. Here’s how to do that: 

  • Disable the AutoRun, so that once you build the Player it doesn’t launch and run the tests. 
  • Save it to a known location rather than into the system’s temporary folder (where it would be saved by default. 

Then you add custom result reporting on the Player side (using the callback interface) to capture all the results and save them to an XML file, or whatever format works for your project. 

See the following code examples for splitting build and run. In the Unite session (at 6:28) Richard Fine takes you step by step through the code for both parts of this application – the build and result reporting.

Splitting build and run: build

Build:

 

SplitBuildAndRun:build.cs (C#)
[assembly:TestPlayerBuildModifier(typeof(SetupPlaymodeTestPlayer))]
public class SetupPlaymodeTestPlayer : ITestPlayerBuildModifier {
   public BuildPlayerOptions ModifyOptions(BuildPlayerOptions playerOptions) {
   	playerOptions.options &= ~(BuildOptions.AutoRunPlayer | BuildOptions.ConnectToHost);
   	var buildLocation = Path.GetFullPath("TestPlayers");
   	var fileName = Path.GetFileName(playerOptions.locationPathName);
   	if (!string.IsNullOrEmpty(fileName))
       	buildLocation = Path.Combine(buildLocation, fileName);
   	playerOptions.locationPathName = buildLocation;
   	return playerOptions;
   }
}

Splitting build and run: save results in run

Run: 

SplitBuildAndRun:SaveResults.cs (C#)
[assembly:TestRunCallback(typeof(ResultSerializer))]
public class ResultSerializer : ITestRunCallback {
   public void RunStarted(ITest testsToRun) { }
   public void TestStarted(ITest test) { }
   public void TestFinished(ITestResult result) { }
   public void RunFinished(ITestResult testResults) {     
   	var path = Path.Combine(Application.persistentDataPath, "testresults.xml");
   	using (var xw = XmlWriter.Create(path, new XmlWriterSettings{Indent = true}))
       	testResults.ToXml(true).WriteTo(xw);
   	System.Console.WriteLine($"***\n\nTEST RESULTS WRITTEN TO\n\n\t{path}\n\n***");
   	Application.Quit(testResults.FailCount > 0 ? 1 : 0);
   }
}

Launching specific tests from a menu item

Often, when developers write validations, the results are displayed in the Console window, where it’s easy for them to be lost in the flow of messages. What if you could instead get validation results in a test suite with clear indicators in a dedicated place in the Editor? You can do this by launching specific tests for a menu item.  

You start with a method that’s wired up to a MenuItem, exactly the same way as you would do with any other Editor extension menu item. That method then creates a ScriptableObject callback object. You use a ScriptableObject instead of a regular class so that the tests can do things such as cause a domain reload while keeping your callbacks intact. When the ScriptableObject is enabled, it is registered for callbacks, and when it’s disabled, it is unregistered. 

Then you set up filters. Often, when you launch a test for a MenuItem, you want to run tests for a specific category or group. With filters you can execute the test run asynchronously. It will run over the course of several frames, allowing you to have UnityTest, UnitySetUp, and UnityTearDown tick the engine loop during the test. When the test is finished and registers RunFinished, you can set it to display a message or open a results window – whatever makes sense for your workflow.

See the code example below. Richard goes through the code setup for this application in the Unite session (at 11:50).  

LaunchTestsFromMenuItem.cs (C#)
public class RunTestsFromMenu : ScriptableObject, ICallbacks {
   [MenuItem(“Tools/Run useful tests”)] public static void DoRunTests() {
   	CreateInstance<RunTestsFromMenu>().StartTestRun();
   }
   private void StartTestRun() {
   	hideFlags = HideFlags.HideAndDontSave;
   	CreateInstance<TestRunnerApi>().Execute(new ExecutionSettings {
       	filters = new [] { new Filter{ categoryNames = new[] { “UsefulTests” }}}
   	});
   }
   public void OnEnable() { CreateInstance<TestRunnerApi>().RegisterCallbacks(this); }
   public void OnDisable() { CreateInstance<TestRunnerApi>().UnregisterCallbacks(this); }
   /* ...RunStarted, TestStarted, TestFinished... */
   public void RunFinished(ITestResultAdaptor result) {       
   	…
   	DestroyImmediate(this);
   }

Running tests before the build

Running tests before the build can be tricky because the build process requires that the tests are run from a callback, so there’s no opportunity to pump the engine update loop. But the benefit is that you can check that basic functionality is working, prior to spending time actually building (which can take many minutes for some projects). 

You can implement this application using the IPreprocessBuildWithReport interface, the same way you might implement any other kind of build preprocessing. To get the results, register a callback as usual.

Because you can’t go into Play Mode in the middle of a build, you can use the Test Runner API to run specific tests in Edit Mode. You can select those tests by filtering by category, such as a pre-build validation test category. You can run these tests synchronously. 

Once the test is finished, check the results. If something failed, you can throw a BuildFailed exception, which will cause the build process to abort. 

This application can be broken down into two parts – the ResultCollector and the preprocessor – which Richard details in their talk (at 15:20). 

Watch Christian and Richard demo the Test Runner API live. Check out the whole session for even more QA tips! 

RunTestsBeforeBuild.cs (C#)
   public ITestResultAdaptor Result { get; private set; }     
   public void RunStarted(ITestAdaptor testsToRun) { }
   public void TestStarted(ITestAdaptor test) { }
   public void TestFinished(ITestResultAdaptor result) { }
   public void RunFinished(ITestResultAdaptor result)
   {
   	Result = result;
   }
}

Did you like this content?

We use cookies to ensure that we give you the best experience on our website. Visit our cookie policy page for more information.

Got it