vsts-agent Build Definition for .NET Core (with Test Results)

The other day I cloned a .NET Core sample app MusicStore from GitHub to VSTS and wanted to configure a build definition and run unit and integration tests using hosted build agent. The Visual Studio Build Definition Template which I chose did not work out very well 😦

Clone the MusicStore to your Git Repo in the Code hub and it should look something like this

VSTS Code Hub after MusicStore GitHub Repository cloned

At this point you can clone the code if you want to see the sample in Visual Studio.  Now we’ll create a Build Definition in VSTS for the project.  You can now organize your build defintions in folders which I think is pretty cool and has been a feature I’ve been after for a long time.

Create a new build definition | Visual Studio Template

I selected the Visual Studio template and clicked Next.

Select the Repository where you put the MusicStore sample

You can select the code repository, branch and build folder for your definition then click Create.

Visual Studio Build Template (not saved)

At this point you can press Save give it a Name and it will save your new build definition with default settings.  This is going to fail to build the .NET Core solution but this is our starting point, we’ll fix it after.   After you save you will be able to click the Queue new build… link

Queue new build…

And the build fails….

Build failed…

To see where things went sideways you can click the Build 20160917.1 link to go to the summary page.  The title of the link can also be customized later in the settings for your build to suit your preferences.

Build Summary Page

You will see error messages like the following about a missing lock file for the projects referenced in the solution

Project <name> does not have a lock file. Please run “dotnet restore” to generate a new lock file.

Luckily for us the hosted agent has the bits we need to make this work.  We will go back and edit the build definition and add a task to perform the restore.  Click on the link with the name of your build definition and then click Edit.

Let’s Add a new task Command Line to run the dotnet.exe command mentioned in the error above with restore as its parameter.

Click Add then Close, the task will be added to the bottom of the tasks and now select it and drag it up between Nuget restore and Build Solution.

Add Run dotnet.exe restore and disable Nuget restore task

for Tool specify C:\Program Files\dotnet\dotnet.exe and for Arguments specify restore. Note: I left the Working folder empty so it will default to the root of the branch.  This is so all projects are parsed under both src and tests.

Save your changes (enter an optional comment) and click the Queue a new build… link and wait for the build to complete and now you should have a successful build as you can see in the screenshot below.

Build Succeeded

If you go to the Summary page for the build you will see you have an Issues listed that no test assemblies were found.

Build Succeeded | Issues | No test assemblies found matching the pattern

Well we would really like to run the unit tests and the integration tests that are part of the MusicStore sample so let’s edit our build definition again and disable the Test Assemblies task and add two Command Line tasks to run each of the test projects in the MusicStore sample.

Here is how your build definition should look now with the two Command Line tasks and the disabled Test Assemblies task.

dotnet.exe test

I entered the following values for the Command Line tasks to execute each set of tests.

First One

• Tool: C:\Program Files\dotnet\dotnet.exe
• Arguments: test -xml .\TEST-MusicStore.xml
• Working folder: test/MusicStore.Test

Second One

• Tool: C:\Program Files\dotnet\dotnet.exe
• Arguments: test -xml .\TEST-E2E.xml
• Working folder: test/E2ETests

You’ll notice I added the argument -xml to the dotnet.exe test command, this is so we can save the results for each test run and use them later to publish the results of our tests to the build summary.  For now let’s Save the changes to the build definition and Queue a new build to make sure everything is green.

Both dotnet.exe test tasks ran successfully

This is great except we still do not see our test results in the build summary, luckily a task exists so we can publish the results.  Let’s go and add the Publish Test Results task now.

Add Publish Test Results task from the Task catalogue

Add a build step and select the Publish Test Results task to your build definition and place it immediately before the Publish symbols path task

I used the following values for the Publish Test Results task:

• Test Result Format: XUnit
• Test Results Files: **\test-*.xml
• Merge Test Results: Checked

Under Advanced

• Platform: $(BuildPlatform)
• Configuration: $(BuildConfiguration)
• Upload Test Attachments: Checked

Publish Test Results

Save your build definition and Queue a new build so we can check out the Test Results from the build.

After the build has run the Summary page should now have a populated Test Results widget (see screenshot below)

Published Test Results

You can of course view more details about your test run by going to the Test tab of the Build Summary (shown below)

If you click on the 43/55 Passed – XUnit_TestResults link you can get to the Test Runs Run Summary for your build in the Test Hub.

Test Run Summary

If you click Test Results on this page, you can create Bugs for any of your failed tests.

Hope this helps, if you have any feedback regarding the above steps feel free to leave a comment below.