The other day I cloned a .NET Core sample app MusicStore from GitHub to VSTS and wanted to configure a build definition and run unit and integration tests using hosted build agent. The Visual Studio Build Definition Template which I chose did not work out very well 😦
Clone the MusicStore to your Git Repo in the Code hub and it should look something like this
At this point you can clone the code if you want to see the sample in Visual Studio. Now we’ll create a Build Definition in VSTS for the project. You can now organize your build defintions in folders which I think is pretty cool and has been a feature I’ve been after for a long time.
I selected the Visual Studio template and clicked Next.
You can select the code repository, branch and build folder for your definition then click Create.
At this point you can press Save give it a Name and it will save your new build definition with default settings. This is going to fail to build the .NET Core solution but this is our starting point, we’ll fix it after. After you save you will be able to click the Queue new build… link
And the build fails….
To see where things went sideways you can click the Build 20160917.1 link to go to the summary page. The title of the link can also be customized later in the settings for your build to suit your preferences.
You will see error messages like the following about a missing lock file for the projects referenced in the solution
Project <name> does not have a lock file. Please run “dotnet restore” to generate a new lock file.
Luckily for us the hosted agent has the bits we need to make this work. We will go back and edit the build definition and add a task to perform the restore. Click on the link with the name of your build definition and then click Edit.
Let’s Add a new task Command Line to run the dotnet.exe command mentioned in the error above with restore as its parameter.
Click Add then Close, the task will be added to the bottom of the tasks and now select it and drag it up between Nuget restore and Build Solution.
for Tool specify C:\Program Files\dotnet\dotnet.exe and for Arguments specify restore. Note: I left the Working folder empty so it will default to the root of the branch. This is so all projects are parsed under both src and tests.
Save your changes (enter an optional comment) and click the Queue a new build… link and wait for the build to complete and now you should have a successful build as you can see in the screenshot below.
If you go to the Summary page for the build you will see you have an Issues listed that no test assemblies were found.
Well we would really like to run the unit tests and the integration tests that are part of the MusicStore sample so let’s edit our build definition again and disable the Test Assemblies task and add two Command Line tasks to run each of the test projects in the MusicStore sample.
Here is how your build definition should look now with the two Command Line tasks and the disabled Test Assemblies task.
I entered the following values for the Command Line tasks to execute each set of tests.
- Tool: C:\Program Files\dotnet\dotnet.exe
- Arguments: test -xml .\TEST-MusicStore.xml
- Working folder: test/MusicStore.Test
- Tool: C:\Program Files\dotnet\dotnet.exe
- Arguments: test -xml .\TEST-E2E.xml
- Working folder: test/E2ETests
You’ll notice I added the argument -xml to the dotnet.exe test command, this is so we can save the results for each test run and use them later to publish the results of our tests to the build summary. For now let’s Save the changes to the build definition and Queue a new build to make sure everything is green.
This is great except we still do not see our test results in the build summary, luckily a task exists so we can publish the results. Let’s go and add the Publish Test Results task now.
Add a build step and select the Publish Test Results task to your build definition and place it immediately before the Publish symbols path task
I used the following values for the Publish Test Results task:
- Test Result Format: XUnit
- Test Results Files: **\test-*.xml
- Merge Test Results: Checked
- Platform: $(BuildPlatform)
- Configuration: $(BuildConfiguration)
- Upload Test Attachments: Checked
Save your build definition and Queue a new build so we can check out the Test Results from the build.
After the build has run the Summary page should now have a populated Test Results widget (see screenshot below)
You can of course view more details about your test run by going to the Test tab of the Build Summary (shown below)
If you click on the 43/55 Passed – XUnit_TestResults link you can get to the Test Runs Run Summary for your build in the Test Hub.
If you click Test Results on this page, you can create Bugs for any of your failed tests.
Hope this helps, if you have any feedback regarding the above steps feel free to leave a comment below.