2022-11-01: Hack day – User management and Notifications

Topic selection process

We decided to select a topic for each of the two teams in advance. Ideas were proposed by team members, and stakeholders were invited to complete a survey to prioritise these (results at right).

Thereafter the two teams selected their preferred topic for the hack day:

The London team selected #3: How can we improve user management in system so that it's more transparent and relevant?

The Manchester team selected #6: How can we provide timely alerts to users so that they are given notice in good time for any actions they need to take (e.g. Trainee notification service for upcoming placements)?

Survey prioritisation of topics

The survey prioritisation of the ideas was as follows (highest to lowest):

1. How can we improve the Local Office Admin's ability to manage assessments?

2. How can we determine the value of integrating TIS and Horus / E-portfolios?

3. How can we improve user management in system so that it's more transparent and relevant?

4. How can we streamline our infrastructure in order to save HEE money and simply future TIS Team working?

5. How can we improve all FE Apps delivery and availability?

6. How can we provide timely alerts to users (e.g. Trainee notification service for upcoming placements)?

7. Investigate and plan a piece of work on the current roadmap, e.g. how can we support TIS admins with a move to paperless office?

8. How can we improve the user experience for PGDiTs interacting with TSS on their mobile devices?

9. How can we better understand the benefits of an app that can work offline?

Also two further ideas were put forward to consider:
10. Improving the admin experience, searching, filtering.
11. On a daily basis, there are times when I cannot log into TIS. I sometimes have to wait an hour or turn off my laptop and then turn it back on again (although this doesn't always work). This is a disaster if I'm in the middle of an ARCP. Please look into fixing this issues. I know this happens to all my team members not just me.

Subject matter experts

3 subject matter experts agreed to attend the Manchester session. These were Ashley Barrett, Lesley McGinty and Asmaa Yehia (all working in Jane McGovern's team).

Team members

London: Andy Nash (facilitator), Ashley Ransoo, John Simmons, Joseph Kelly, Naz Akhtar, Saedhia Mohammed, Steve Howard

Manchester: Andy Dingley, Cai Willis, Doris Wong, Ed Barclay, Reuben Roberts (facilitator), Yafang Deng

Structure of the day

• 1 hour: discovery (who are the users? what are their problems (might be different problems for different users)? what are the risks and assumptions we need to shine a light on before committing to potentially wasted development effort?)

• 5 mins assessment: collating the discovery

• 1 hour: alpha experiments (testing assumptions and risks and narrowing down a range of possible solutions, to the preferred one to develop in beta)

• 5 mins assessment: collating the alpha

• 4 hours (incl. breaks): beta development (incrementing a solution - developing a bit, reconvening, developing a bit more, reconvening. Avoiding long periods of silo/solo working)

• 1 hour: live presentation back to the other team (and maybe some stakeholders of the chosen ideas selected in advance of the day)

Intro

TODO - or break it down as for Manchester below, by Agile phase?

Hypotheses

TODO

Experiment outline

TODO

Experiment detail

TODO

Discovery phase

The discovery phase was conducted as a general discussion with the SME’s. The various sorts of notifications that trainees and admin staff would find useful were sketched-out.

The existing notifications of changes to TIS data for a trainee (e.g. placements, personal details) are currently relayed to trainees via a (normally monthly) rotation spreadsheet, which highlights new, deleted and edited data. This spreadsheet is used to generate change notifications that are emailed to the affected Trainees. In some regions, the Trust will be the recipient of the rotation spreadsheet and distribute the notifications; in others it will be the Lead employer which receives the spreadsheet and relays the notifications to both the trainees and the Trusts.

The inability to summarise which trainees have (or have not) completed their FormRs makes it difficult for the Trainee Programme Managers to ensure that the panel for an ARCP are assembled in time, or to inform the trainee that their ARCP will be delayed unless they can submit their FormR in time. A report providing the current status for trainees would be appreciated by the TPMs.

Problem statement:

Alpha phase

The focus in the Alpha phase was ensuring we had a clear view of the workflow of the existing emailed notifications (top part of the diagram to right), and to sketch out a proposed process for the hack-day solution (bottom part of the diagram). We did not explore a range of alternative solutions in detail, although we did discuss different methods of notifying trainees (email, SMS, in-app notification). For an MVP we decided to focus on a like-for-like implementation of email notifications.

The existing workflow would not be replaced, except to remove the mailing of trainee notification emails, since the rotation spreadsheet is used in a variety of other ways.

We decided that a feasible solution should not notify a trainee of every data change, since admin staff sometimes make a number of edits to a record in a short period of time. Rather we would determine the net changes on a daily basis and send notifications based on that aggregate / net set of data changes. This would broadly mirror the way the rotation spreadsheet identifies data changes.

We also tightened up the scope of the work in a few ways.

We identified some data changes that could be excluded from the scope:

• Past changes

• Post number changes, which should not trigger notifications to trainees, but as the existing process involves creating and deleting posts, these would be hard to distinguish from genuine data changes that should generate notifications

The range of data changes that should trigger notifications is wider than the data currently held in TSS, but we decided to limit the hack-day work to TSS data changes only: replicating the cascading effects of changes to e.g. a Programme that affects many placements within TIS was felt to be a duplication of that functionality in TSS, and in due course all TIS data should be housed in TSS.

We similarly excluded notification email bounce-back handling from the proposed solution. While an edge-case, there are situations where trainee’s mailboxes are full, triggering a bounce, or their email address has changed. These would need to be handled manually.

Beta phase

Initial work in the beta phase was put towards agreeing a more detailed architecture for the solution identified in the Alpha. This was carved-up into sections for different teams to work on.

Andy Dingley: the initial detection of TSS data changes, adding these as appropriately structured SQS messages to a FIFO queue.

Doris Wong / Reuben Roberts / Yafang Deng: process the SQS messages to produce aggregate changes to placements over the course of a day, and put these onto the SNS queue

Cai Willis / Ed Barclay: process the messages from the SNS queue via a fanout of SQS queues consumed by Lambda functions to generate emails to trainees.

During the course of the afternoon we realised that the middle component was lagging somewhat, due to complexity around setting up the new component via Terraform and configuring its dependencies (e.g. Redis data cache, SQS, SNS, Spring Batch). In general, we found that this was a set of tasks we were less familiar with, and this delayed engagement with the actual business problem. The algorithm for maintaining original and current states for trainee data, given incoming data change messages might relate to any combination of fields, also posed a challenge.

With time running short, we decided to remove the data aggregation step from the MVP, and to connect the initial data change detection step directly to the final email generation step. With seconds to spare, this was tested and debugged to produce a workable proof-of-concept, if not exactly an MVP.

Presentation phase

In the absence of teleconferencing facilities, this was conducted via Teams, which proved highly awkward, but we were able to successfully demonstrate the work.

Edit placement in TIS. When this data change arrives in TSS, it generates an email as follows:

Hack day Retros are best carried out at the end of a Hack day, while everything is still fresh in everyone’s minds. Because of time and teleconferencing constraints, this was unfortunately not possible. Many participants were sick / on leave in the following few days.