Context:
The previous page describes the various mechanisms by which a user can delete a field/record on TIS currently, and provides suggestions as to how this might be tackled in future (with re-work on the FE and BE), however, there is no downstream review of impact. This page gives an overview of the following
- tables and fields used across TIS and where they're used
- suggestion of deletion mechanism
- commentary on downstream impacts of deletion mechanism
In conclusion, deletion could continue to be applied by status, however each status would imply a particular action. These are described below:
Inactive
...
Page content:
Table of Contents
Background
Requires further discussion with Joanne Watson (Unlicensed) / Ray / IG - TISNEW-257
There is a requirement to enable permanent retirement of defined records e.g. when a trainee record falls outside of the data retention period. This is also an item on the list here. An initial recommendation from the data leads is that this should be an automated process that would trigger the following
- Change of status in Back End that removes "Inactive" trainees after a defined duration has elapsed e.g. 7 years from their programme end date
- Remove view of the record in the Front End and should not be searchable/accessible via any means from the UI.
- Notify users in bulk of records that have been archived - AR 29/08: What does this mean and entail ???
- Apply status of "Archive/Suppressed"
- All archived data should be retrievable via NDW - AR 29/08: Has this been verified if compliant with IG?
Statuses
Inactive
- the record with a status of 'Inactive' can be viewed on TIS by applying appropriate filtering where applicable or visible as a read-only value where are are legacy reference values. For example, reference values that have been made inactive in their respective table are rendered read only, or can be removed from a TIS form
- the inactive field record cannot be viewed in the TIS UI drop downs. For example, reference values that have been made inactive in their respective table are no longer viewable within drop downs across TIS forms to edit or create records with.
- the inactive form value cannot be linked to any other recordused to create a new record or update an existing record with an inactive value. For example an inactive post cannot be linked to a trainee and become a placement
- Sent to NDW to allow reporting on
Delete
- the field/form record with a status of 'Delete' cannot be viewed on the TIS UI.
- the field/form record remains in the Back End of TIS to allow it be shared with NDW and retrieved if required (until it must be filed or archived)
- the field/form is sent to NDW as a "deleted" value
Archive
- the field/form Sent to NDW to allow reporting on
Archive/Suppressed (AR 28/08: Solutionising? This status does not exist at the moment and the scope of Archiving as a requirement still being reviewed)
- the record cannot be viewed on the TIS UI via any means
- the field/form record cannot be viewed in the TIS back endend – AR 29/08: what does this mean?
- the field/form record must be stored in an archived location (TBD)
- the records must not be reportable by the NDW (Need to check)
- the archiving process is automated (TBC)
- archiving should be read only (snapshot), without requirement for reference values
Note:
A "form" is the TIS UI format of a particular set of data in TIS e.g. Person screen = a form
A "record" can refer to either a table or a form within TIS, so is not used in this page to remove confusion
This will need to be updated once "missing" fields have been agreed for the TIS UI (see here: NDW UAT feedback)
Impact Analysis
The fields listed are only those displayed in the TIS front end:
...
RolePermission
- whole entry status can be: INACTIVE / DELETE / ARCHIVE on the FE
- Should be stored against a relevant person ID (tbc) when deleted or archived
...
DELETE
ARCHIVE
...
- deletion will remove the role from the FE, but retain the it in the BE
- permissions will be removed to the user on login
- a role must be assigned to any user who needs to use the system
- roles should be archived against a person record (i.e. for users who are also managed within TIS)
- roles should be archived against users (i.e. for users who are not managed within TIS)
...
- the permission is removed where the role is removed
...
Role
- whole entry status can be: INACTIVE / DELETE / ARCHIVE on the FE
...
INACTIVE
DELETE
...
- inactive will remove the role from being applied in the FE
- delete will remove the value from the table
- delete will remove data from forms where the role is in use
This is held in Keycloak currently, this is only relevant where role/permission management moves into TIS (as reference data)
Assumption is that reference data archiving is not necessary
...
UserRole
- whole entry status can be: INACTIVE / DELETE / ARCHIVE on the FE
- Should be stored against a relevant person ID (tbc)
...
DELETE
ARCHIVE
...
- usernames are mandatory so at least 1 is necessary for a user to be active in TIS
- delete will remove their ability to login to TIS
- archive will retain a copy of the username details
...
DELETE
ARCHIVE
...
- deletion will remove the role from the FE, but retain the it in the BE
- permissions will be removed to the user on login
- a role must be assigned to any user who needs to use the system
- roles should be archived against a person record (i.e. for users who are also managed within TIS)
- roles should be archived against users (i.e. for users who are not managed within TIS)
...
This is held in Keycloak currently, this is only relevant where role/permission management moves into TIS
Is this data linked to the data held in the RolePermission or is it held separately? If so, then would the status of one or both instances of a like-role need to be altered?
...
PersonalDetails
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
TrainingNumber
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
GdcDetails
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
INACTIVE
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
GmcDetails
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
INACTIVE
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
ContactDetails
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
RightToWork
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Qualification
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Comment
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
PersonTrust
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
PersonOwner
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
RotationPerson
- entry status can be: INACTIVE / DELETE / ARCHIVE
...
DELETE
ARCHIVE
...
Rotation
- Whole entry status can be: INACTIVE / DELETE / ARCHIVE
...
DELETE
...
INACTIVE
DELETE
ARCHIVE
...
RotationPost
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
PlacementSupervisor
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
TBC
...
TBC
...
Programme
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
INACTIVE
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Placement
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Curriculum
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Post
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
INACTIVE
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
PostFunding
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Specialty
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
INACTIVE
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
ProgrammeMembership
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Assessment
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
INACTIVE
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
AssessmentDetail
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
AssessmentOutcome
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Revalidation
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
Reference Tables - these have been summarised since the rules should be similar in the majority of instances (some have other values which will be impacted if the below rules are adhered)
- entry status can be: INACTIVE / DELETE / ARCHIVE
- Should be stored against a relevant ID (tbc)
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
DELETE
ARCHIVE
...
INACTIVE
DELETE
...
- -AR 29/09: what does this mean? Archiving is not about taking a backup/snapshot!
Archiving process – what and when to archive?
| # | Artefact | What and when to archive (V10) | What it means for TIS? | Questions |
|---|---|---|---|---|
| 1 | Persons | The Code of Practice provides retention timescales for a full staff record, staff record summary and staff training records as follows:
Proposals: Scenario 1:
Scenario 2:
Scenario 3:
|
The dates for each scenarios need to be reviewed if similar approach will be undertaken for TIS. | |
| 2 | Posts | The Code of Practice does not specify the action to be taken in relation to training posts. However as funding may be attached to post records it is important to recognise that these posts may have historical value. In addition, on the V10 system the removal of any parent records could impact on any related/linked records i.e. removing a post record will result in the linked information being removed from any related child records. Therefore it is proposed that only where: Scenario 1: Record status is inactive and has no placements. Record has no funding information attached Action to take – delete record. Until more work can be taken around the management of posts on V10 is it suggested that the above change is the only one that is made.
| ||
| 3 | Contacts | There also exists on V10 a large number of Contact records. Many of these records contain little information saved for the name of the individual. Some however contain full trainer records so a ‘one size fits all’ solution will not work. It is therefore suggested that more research be done into the extent of the use of the ‘Contact’ record type in local offices. | What does Contacts mean for TIS? Does this also consist of removal of consultant records? | |
| 4 | Placements | Should placements be archived? If so what rules to be applied in TIS? | ||
| 5 | Assessments | Should Assessments be archived? If so what rules to be applied in TIS? | ||
| 6 | Documents | Should Documents be archived? If so what rules to be applied in TIS? | ||
| 7 | Revalidations | Should revalidation episodes be archived? If so what rules to be applied in TIS? | ||
| 8 | Concerns | Should revalidation concerns be archived? If so what rules to be applied in TIS? |
Recommendations
| # | Recommendation | Comments |
|---|---|---|
| 1 | All local offices to undertake basic housekeeping by reviewing records that have already been set to delete and purged from the system where appropriate. NOTE: we should not archive any deleted records. | |
| 2 | Archiving should take place after the Membership number work (GMC, GDC and Public Health numbers) has been completed by all local offices to ensure partial trainee records are not archived. | |
| 3 | Those that should not be deleted should be reset to ‘Inactive’ to ensure they are not purged from the system. | |
| 4 | Those Staff records that meet the archiving criteria above (based on timescales for retaining a full staff record or a Summary Staff Record) should be identified on the V10 system via queries run in the Data Repository using a view that allows access to all local office data. | |
| 5 | These records should be reviewed and, where deemed appropriate, placed into an HEE archiving repository. The records should then be set to delete and purged from V10. | |
| 6 | Both the V10 system and the archived records repository should be interrogated when responding to Subject Access requests. | |
| 7 | HEE to arrive at an agreement for managing the Staff/contact situation on V10. The categorising of records in this way is resulting in ever more complex issues for local offices to deal with. | |
| 8 | Local Offices to investigate the records with a Record Type of ‘Contact’. Those records with no educational/training value should be considered for deletion; those that have a historical educational/training value be reviewed for archiving and deletion. | |
| 9 | Only when the funding for all posts has been added to V10 should we consider looking further at archiving post records. | |
| 10 | Longer term, an archiving function should form a part of the Trainee Information System (TIS). A comprehensive policy for managing records in TIS should be developed as part of the TIS project to include parent and child record archiving and the archiving of reference data. | |
| 11 | Consideration should be given to how access to the records that have been archived can be future proofed to ensure they remain accessible for the required time period. |
Questions and Assumptions
| # | Question | Comments |
|---|---|---|
| 1 | Other than reference date and records (forms), is there anywhere else, or data type, that it should be possible to apply inactive as a status? | |
| 2 | Other than within record, are there other scenarios you might want to delete (trashcan) a data entity? for example, particular fields? | |
| 5 | What data will need to be accessible (GDPR) and for how long? | |
| 6 | Are there any notifications we need to provide to users / trainees regarding the editing/deleting/archiving of their data? |