Key Results: Benchmarking AWS and Azure

One of the Key results for this quarters OKR for AWS migration is to perform a stress test against AWS. The following details this as well as other benchmarks targeting different users

Stress Testing

The following is a table of HTTP result counts against both Azure and AWS on one of that larger TIS components (TCS) against one of the slower endpoints.

{table}

Insights:

Response times

At the end of the day, the end users will be the main customer of the TIS system, we need to show that the move has not had any detrimental affect on the day to day work.

The following are some response times from the browser as the user will see them while working on TIS

Azure TIS after login:

AWS TIS after login:

Azure TIS view person:

AWS TIS view persion:

Azure TIS view post:

Aws TIS view post:

The following is a graph of response times from the same endpoint with indications of percentiles

{graph}

Insights:

Reliability

Below is a demo of the reliability checks in play

{video}

Build times

One thing to make the development experience better for developers is to have fast turnaround (feedback) from external systems. Typically, when a developer develops a feature, they would push code to a central repository regularly, this code is typically a possible release candidate and therefore needs to go through a pipeline different quality checks. This pipeline could take some time to complete, so you don’t want this developer waiting around for some feedback.

Below is several screen caps of the same pipelines for the same code/features with how long it takes to respond

Admins UI Azure:

Admins UI AWS:

TCS Azure:

TCS AWS:

ESR Inbound reader Azure:

ESR Inbound reader AWS:

Insights:

Comparing pipelines in both AWS and Azure, it's easy to see that there is up to circa 1minute improvements in some stages over Azure. If multiple developer pushes multiple times a day, the compound savings could penitentially be enormous

Cost

Currently in Azure, we have an inventory of virtual machines and registries using storage space, the average monthly cost to run TIS in Azure is…