Did your application deploy successfully? Use a selenium test in your release pipeline with a Microsoft-hosted agent

UI Testing with Microsoft-hosted Agents

Microsoft-hosted agents are pre-configured for UI testing and UI tests for both web apps and desktop apps. The agents are also pre-configured with popular browsers and matching web-driver versions that can be used for running Selenium tests. The browsers and corresponding web-drivers are updated on a periodic basis.  (ref.)

It’s great to have a complete CI-CD pipeline but if you have no way of knowing whether it actually works or not after it gets deployed you’ll soon be getting emails when it breaks. We can of course do better by including some tests which run after we deploy to the slot and then again after we swap slots with production.

I also grabbed a screenshot of the web page while it was executing the test and saved it as an attachment with the test.


First we need a selenium test to execute against our web application that we deployed, this is quite easy to do. If you want an end-end example you can refer to the following example.

Here is a Test Method which at the moment uses Google Chrome in headless mode to navigate to my web apps home page and look for the name of my Azure Key Vault that I have displayed on the home page.

CheckCoreWebAppDemo Test Method
CheckCoreWebAppDemo Test Method

My Test Project exists in its own solution (independent of the .NET Core Web App) and has its own Build Pipeline which creates an artifact (see below) that is linked to the CoreWebApp Release Pipeline.

Selenium Test Artifact
Test Project Artifact – used by the Release Pipeline

The artifact produced from that build pipeline has my test assembly which is executed by the VS Test tasks in the Core Web App release pipeline before and after the slot swap task. I have two (2) different .runsettings files that are used with the different VS Test tasks.

Visual Studio Test tasks used in release pipeline
Visual Studio Test tasks – before and after Swap Slots

Release Pipeline

Here is my release pipeline configured with both of the build artifacts (Core Web App and Selenium Tests), the CD trigger is only configured on the Web App artifact in the picture below.

Release Pipeline with Artifacts
Release Pipeline with Artifacts

So now when my release is triggered, it not only builds the environment in Azure it also validates that the website is functional before it performs the slot swap.

100% Test Pass!
Eureka! 100% Test Pass

I can even view a screen capture that was taken during the execution of the test method on either of the slots in my App Service (from the Attachments) if I desire.

Test Method Attachments w/Preview
Test Method Attachments w/preview

Hope you find this post helpful, next I think I’m going to expand my tests to execute Edge, Chrome and Firefox selenium tests on the hosted agent. I also want to integrate Application Insights into the pipeline… so many ideas!

If you want some recipes to help you get started with Azure DevOps (Server or Services) then I definitely recommend my good friends Tarun and Utkarsh‘s cookbook for sale on Amazon!

About Wes MacDonald

Wes MacDonald is a DevOps Consultant for LIKE 10 INC., a DevOps consulting firm providing premium support, guidance and services for Azure, Microsoft 365 and Azure DevOps.

No comments yet... Be the first to leave a reply!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: