Archiving - V10 v.s. TIS (DRAFT)

Owned by Former user (Deleted)
Last updated: 13 Jun 2019 by Ashley Ransoo

Page content:


Background

Requires further discussion with Joanne Watson (Unlicensed) / Ray / IG - TISNEW-257

Hicom Document:
https://hee-tis.atlassian.net/wiki/download/attachments/476053520/Intrepid_ConsolidationDataRetention_Proposal%20V4.docx?version=1&modificationDate=1535620301647&cacheVersion=1&api=v2

Proposal document:
https://hee-tis.atlassian.net/wiki/download/attachments/476053520/V10%20Consolidation%20project%20-%20archiving%20proposal%20(draft)%2029.9.16%20(1)%20(1).docx?version=1&modificationDate=1534494485661&cacheVersion=1&api=v2

There is a requirement to enable permanent retirement of defined records e.g. when a trainee record falls outside of the data retention period. This is also an item on the list here.  An initial recommendation from the data leads is that this should be an automated process that would trigger the following

  • Change of status in Back End that removes "Inactive" trainees after a defined duration has elapsed  e.g. 7 years from their programme end date
  • Remove view of the record in the Front End and should not be searchable/accessible via any means from the UI.
  • Notify users in bulk of records that have been archived - AR 29/08: What does this mean and entail ???
  • Apply status of  "Archive/Suppressed"
  • All archived data should be retrievable via NDW - AR 29/08: Has this been verified if compliant with IG?

Statuses

Inactive

  • the record with a status of 'Inactive' can be viewed on TIS by applying appropriate filtering where applicable or visible as a read-only value where are are legacy reference values. For example, reference values that have been made inactive in their respective table are rendered read only
  • the inactive record cannot be viewed in the TIS UI drop downs. For example, reference values that have been made inactive in their respective table are no longer viewable within drop downs across TIS forms to edit or create records with.
  • the inactive value cannot be used to create a new record or update an existing record with an inactive value. For example an inactive post cannot be linked to a trainee and become a placement
  • Sent to NDW to allow reporting on


Delete

  • the record with a status of 'Delete' cannot be viewed on the TIS UI.
  • the record remains in the Back End of TIS to allow it be shared with NDW and retrieved if required (until it must be filed or archived)
  • Sent to NDW to allow reporting on


Archive/Suppressed (AR 28/08: Solutionising? This status does not exist at the moment and the scope of Archiving as a requirement still being reviewed)

  • the record cannot be viewed on the TIS UI via any means
  • the record cannot be viewed in the TIS back end – AR 29/08: what does this mean?
  • the record must be stored in an archived location (TBD) 
  • the records must not be reportable by the NDW (Need to check)
  • the archiving process is automated (TBC)
  • archiving should be read only (snapshot), without requirement for reference values -AR 29/09: what does this mean? Archiving is not about taking a backup/snapshot!


Archiving process – what and when to archive?

#ArtefactWhat and when to archive (V10)What it means for TIS?Questions
1
Persons

The Code of Practice provides retention timescales for a full staff record, staff record summary and staff training records as follows:

  • Full staff record – to be kept until the staff member’s 75th birthday or 6 years after the leaving date.
  • Summary Staff record – to be made after the 6 years retention timescale of the full staff record has passed and should be kept until the staff member’s 75th birthday. See appendix one for details of the contents of the summary record.
  • Staff training records – Clinical training records to be retained until 75th birthday or 6 years after the staff member leaves, whichever is longest. Statutory and mandatory training records should to be kept for ten years after training completed and other training records to be kept for 6 years after the training has been completed

Proposals:

Scenario 1:

  • Record type = Staff (is a trainee record).
  • Last programme end date is equal to or earlier than 3/8/10.
  • There are no subsequent placements on the record.
  • Action to take - Extract a Staff Record Summary from V10 for archiving and delete entire V10 record.

Scenario 2:

  • Record type = Staff (is a trainee record).
  • Last programme end date is equal to or earlier than 3/8/10.
  • There are subsequent non training grade placements.
  • Action to take - Extract a Staff Record Summary from V10 for archiving and delete all programme details (including associated placements and assessments) on V10.
  • Action to take - Set record type to Both to maintain the non-training grade placements (see Contacts and Recommendations section for additional details)

Scenario 3:

  • Record type = Both (is/has been both a trainee and non-trainee record)
  • Last programme end date is equal to or earlier than 3/8/10.
  • There are subsequent non training grade placements.
  • Action to take - Extract a Staff Record Summary from V10 for archiving and delete all programme details (including associated placements and assessments) on V10.
  • Action to take - Retain record type as Both to maintain the non-training grade placements (see Contacts and Recommendations section for additional details)

  1. What is the definition of 'leaving date'?
  2. What is the definition of Full, Summary and Staff training records?
  • Full Staff Record:
  • Staff Summary Record:

  • Staff Training Record:


The dates for each scenarios need to be reviewed if similar approach will be undertaken for TIS.

2
Posts

The Code of Practice does not specify the action to be taken in relation to training posts. However as funding may be attached to post records it is important to recognise that these posts may have historical value. In addition, on the V10 system the removal of any parent records could impact on any related/linked records i.e. removing a post record will result in the linked information being removed from any related child records. Therefore it is proposed that only where:

Scenario 1:

Record status is inactive and has no placements.

Record has no funding information attached 

Action to take – delete record.

Until more work can be taken around the management of posts on V10 is it suggested that the above change is the only one that is made.

Scenario 2:

Once all the appropriate Staff records have been archived and deleted from the system we can then archive post records that fulfil the following criteria:

Record status is inactive and contains no placements for a person whose Programme and placement information is still either Current or Inactive.



3
Contacts
There also exists on V10 a large number of Contact records. Many of these records contain little
information saved for the name of the individual. Some however contain full trainer records so a
‘one size fits all’ solution will not work.
It is therefore suggested that more research be done into the extent of the use of the ‘Contact’
record type in local offices.

What does Contacts mean for TIS? Does this also consist of removal of consultant records?
4
Placements


Should placements be archived? If so what rules to be applied in TIS?
5
Assessments


Should Assessments be archived? If so what rules to be applied in TIS?
6
Documents


Should Documents be archived? If so what rules to be applied in TIS?
7
Revalidations


Should revalidation episodes be archived? If so what rules to be applied in TIS?
8
Concerns


Should revalidation concerns be archived? If so what rules to be applied in TIS?


Recommendations

#RecommendationComments
1All local offices to undertake basic housekeeping by reviewing records that have already been set to delete and purged from the system where appropriate. NOTE: we should not archive any deleted records.
2Archiving should take place after the Membership number work (GMC, GDC and Public Health numbers) has been completed by all local offices to ensure partial trainee records are not archived.
3Those that should not be deleted should be reset to ‘Inactive’ to ensure they are not purged from the system.
4Those Staff records that meet the archiving criteria above (based on timescales for retaining a full staff record or a Summary Staff Record) should be identified on the V10 system via queries run in the Data Repository using a view that allows access to all local office data.
5These records should be reviewed and, where deemed appropriate, placed into an HEE archiving repository. The records should then be set to delete and purged from V10.
6Both the V10 system and the archived records repository should be interrogated when responding to Subject Access requests.
7HEE to arrive at an agreement for managing the Staff/contact situation on V10. The categorising of records in this way is resulting in ever more complex issues for local offices to deal with. 
8Local Offices to investigate the records with a Record Type of ‘Contact’. Those records with no educational/training value should be considered for deletion; those that have a historical educational/training value be reviewed for archiving and deletion.
9Only when the funding for all posts has been added to V10 should we consider looking further at archiving post records.
10
Longer term, an archiving function should form a part of the Trainee Information System (TIS). A comprehensive policy for managing records in TIS should be developed as part of the TIS project to include parent and child record archiving and the archiving of
reference data.

11Consideration should be given to how access to the records that have been archived can be future proofed to ensure they remain accessible for the required time period.
 

Questions and Assumptions


#QuestionComments
1Other than reference date and records (forms), is there anywhere else, or data type, that it should be possible to apply inactive as a status?
2Other than within record, are there other scenarios you might want to delete (trashcan) a data entity? for example, particular fields?
5What data will need to be accessible (GDPR) and for how long?
6Are there any notifications we need to provide to users / trainees regarding the editing/deleting/archiving of their data?



Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213