Did your application deploy successfully? Use a selenium test in your release pipeline with a Microsoft-hosted agent

UI Testing with Microsoft-hosted Agents

Microsoft-hosted agents are pre-configured for UI testing and UI tests for both web apps and desktop apps. The agents are also pre-configured with popular browsers and matching web-driver versions that can be used for running Selenium tests. The browsers and corresponding web-drivers are updated on a periodic basis.  (ref.)

It’s great to have a complete CI-CD pipeline but if you have no way of knowing whether it actually works or not after it gets deployed you’ll soon be getting emails when it breaks. We can of course do better by including some tests which run after we deploy to the slot and then again after we swap slots with production.

I also grabbed a screenshot of the web page while it was executing the test and saved it as an attachment with the test.

Configuration

First we need a selenium test to execute against our web application that we deployed, this is quite easy to do. If you want an end-end example you can refer to the following example.

Here is a Test Method which at the moment uses Google Chrome in headless mode to navigate to my web apps home page and look for the name of my Azure Key Vault that I have displayed on the home page.

My Test Project exists in its own solution (independent of the .NET Core Web App) and has its own Build Pipeline which creates an artifact (see below) that is linked to the CoreWebApp Release Pipeline.

The artifact produced from that build pipeline has my test assembly which is executed by the VS Test tasks in the Core Web App release pipeline before and after the slot swap task. I have two (2) different .runsettings files that are used with the different VS Test tasks.

Release Pipeline

Here is my release pipeline configured with both of the build artifacts (Core Web App and Selenium Tests), the CD trigger is only configured on the Web App artifact in the picture below.

So now when my release is triggered, it not only builds the environment in Azure it also validates that the website is functional before it performs the slot swap.

I can even view a screen capture that was taken during the execution of the test method on either of the slots in my App Service (from the Attachments) if I desire.

Hope you find this post helpful, next I think I’m going to expand my tests to execute Edge, Chrome and Firefox selenium tests on the hosted agent. I also want to integrate Application Insights into the pipeline… so many ideas!

If you want some recipes to help you get started with Azure DevOps (Server or Services) then I definitely recommend my good friends Tarun and Utkarsh‘s cookbook for sale on Amazon!