2021/2: Q2 | Review#3 (2021-07-20 to 2021-08-03)

Incident logs

2021-06-08 Placements were not marked as exported, because of TCS unavailability

2021-07-30 Old RabbitMQ prod broker deleted before the EsrDataExport service was moved to new RabbitMQ broker

2021-07-23 TIS-GMC ETL failure

2021-07-23 TIS-NDW ETL failure

[missing incident log for NIMDTA subnet]

2021-07-21 GMC API Failure

TISSS

Improve synchronisation of placement data

@Andy Dingley

In certain cases the synchronisation of data changes can take much longer than originally anticipated.

For example, if a Site is updated in TIS we need to update all trainee placement data related to that site to account for any changes to the name or address fields (which are included in the placement data displayed to trainees). On the extreme end we see over 20,000 placements associated with a single site and over 64,000 placements linked to a single post.

Previously we tried to update all of those trainee placements at the same time at a considerable time cost. We have an imposed time limit of 15 minutes per update, anything that takes longer than 15 minutes was flagged as “failed”.

To solve the issue we are modifying the update to a multi-step approach:

1. Identify all the placements that need to be updated

2. Send those placement to a queue

3. Update each placement individually

Open

Allowing a focus on individual placements rather than large groups avoids the 15 minute time limit, but it also allows us to assign additional workers* much more easily. Previously a single worker would be expected to handle 1000s of placements at once, now we can have multiple workers getting their fair share of those placements from the queue.

_{*Worker meaning any process that can handle the data updates}

Replace dropdown list with new searchable autocomplete input for Programme Specialty, Specialty 1 & 2 for Award of CCT on Form R Part A

@Edward Barclay

Demo from live

Reval

Data Migration (Update)

@Cai Willis

Short update of the on going work.

TIS

Fix Site inaccurate data affected by previous bug

@Marcello Fabbri (Unlicensed)

Explanation of what specifically about Sites has been fixed

Open

Add Logging from NDW-ETL

@Andy Dingley

When something goes wrong, we now have logging that will be available no matter when the ETL fails. We can better diagnose what happened rather than rely on trying to deduce what happened.

We’ve got things running in better place (ECS). Better visibility and control, which will mean better use of resources.

National Insurance Number (NINo) available for reporting

@Reuben Roberts

@Jayanta Saha

Very sensitive information held in TIS, exchanged with ESR and now available for limited reporting via Tableau.

Confirmed 27th July that this NINo is available.

Bulk delete person records

@Andy Dingley

1. Deleted some remaining placements

2. Scripted removal of data identified by NW office

ESR

NIMDTA

OKR

Objective: Solve MFA and complete second live pilot

Key Result#1: Run an experiment with existing pilot Trainees on using an Authenticator App. Get sufficient feedback to develop it into a solution for the next pilot.

Key Result#2: Coordinate a 2nd pilot (confirm with teams running ARCPs - ask monthly. Any number would be insightful, ideally between 30-50).

Key Result#3: Run the 2nd pilot to include live ARCP Form R completion via the app (with associated HEE admin unlocking of Form Rs submitted in error, if needed).

Key Result#4: Ensure data from all fields in Form R is stored in a format enabling it to be extracted on demand for the purposes of reporting.