15/07/2020
- Ashley Ransoo
- Adewale Adekoya
Working Group
TIS: @Alistair Pringle (Unlicensed), @Ashley Ransoo , @Adewale Adekoya, @James Harris @Sebastian Kwok (Unlicensed)
North East, North West and YH: @David Short (Unlicensed), @Andrew.Crozier (Unlicensed), @Liam.Lofthouse (Unlicensed) , @Helan Raynor (Unlicensed)
South (WES/TV/SEV/PEN): @Sue Barrand (Unlicensed), @Mike.Richardson (Unlicensed)
London/KSS/SE: @Chris Norman (Unlicensed), @Maria Kist (Unlicensed) @Sarah Krause @Benazir.Khan (Unlicensed)
East of England, Midlands: @Gwilym Williams (Unlicensed) @Caroline Grisdale
Discussion/Action
Link to Metabase dashboard for data quality issues to resolve - https://build.tis.nhs.uk/metabase/dashboard/18
# | Discussion/Action | Responsible | Outcome | Status |
|---|---|---|---|---|
1 | Resolve Duplicates - TISDEV-4841: Resolve the duplicate invalid GMC numbers, such as 'Retired,' and make them 'UNKNOWN' | Data leads | First discussed on DQ on 04/06: 08/01: TISNEW-2364-Merging, then removal of Duplicate Person Records - Round 1DONE - Work completed for Round 1 on Stage and testing began. Since this was a flyway script, the importance was stressed on completion of the testing on stage as this would block anything else from being released to Production in out release pipeline DS: Has done some testing. SB: leave manager tests carried out. JH/DS/AH: re-use some of the scripts that were used during consolidation for testing on metabase for next week. DS: Prog membership, placements and assessments - Looks good so far. How long more this would need? - conclude by Wed 16/01.( and are going to test this on Wednesday afternoon) 21/09: The scripts have been run on Prod over the weekend. has taken extracts of the data on Friday afternoon ahead of the scripts being run. 4/02: Round two will be needed. LaSE still to advise DS on what to do with the records that weren't an easy match. Chris picking this up - London may be ok but KSS need more attention 4/02: Leave manager - need to do checks for the circa 500 records to make sure that the round 2 de-duping would not leave orphaned records (as per the procedure for Round 1 01/04: to provide the list of Id's/template for the second round of duplication (second strongest match rule). To send the list out to all regions be checked. Jen would have done for the South - to be checked? 15/04: Still awaiting for EMD, South and London. 24/06: Jen to speak to David tomorrow (25th) re queries she has found in the South. James to ask Gwil where has gotten to with this - returns from leave on the 1st July 08/07: David to resend file to Gwil and Jen is no longer confused as she now understands that this part of the exercise will not remove ALL erroneous records. Work is continuing........ 22/07: Ongoing, waiting for South and EMD data 02/09 EM have submitted, awaiting the South (ongoing) 14/10: Back with DS - to compare and rebuild the new list of what's been merged
11/11: Completed by South. With David at the moment. 09/12: There will be some duplicates because of Study Leave. The merging would have to be done at some point on TIS. Almost ready to go. 06/01: DS post on slack for additional feedback. 03/02: DS posted on slack for everyone else to confirm this is OK. (De-dupe Round 2 spreadsheet) 20/04: Recommendation: to be done ahead of Northern Ireland and HCS onboarding onto TIS.
04/05:
20/05: Round 2 de-duplication file -  We can go ahead and de-duplicate the records identified (~20) @David Short (Unlicensed) to share those records, @Ashley Ransoo @James Harris @Sebastian Kwok (Unlicensed) create ticket for Round 2. Some coordination with Hicom after the bulk has been done. 03/06: Dig out the template/ticket and the script for refinement. (@David Short (Unlicensed) @Ashley Ransoo) Iteration 1: Exclude Hicom Records https://hee-tis.atlassian.net/browse/TISNEW-4705 Iteration 2: Hicom records only https://hee-tis.atlassian.net/browse/TISNEW-4871 Include the 38 records to Round 2 - Iteration 2 (Hicom coordinated) 15/06: Template in TIS format expected by end today. 29/06: Sue: Hicom sent over a national list (personTISIds) of ~18 approvers to exclude in the merge. (Based on anyone that has ever approved a study leave application). Some coordination with Hicom for these records. The list on Slack to go first iteration excluding these 18 records. Second iteration look at the Hicom coord. ones. Sue to check against the master file. 29/06: Received and ticket updated. 15/07: South - sent findings on merged records of duplicate Trainees where duplicate trainee record has leave application records 
| |
2 | Identification and removal of Consultant data on TIS | All | Removal of consultant data has a dependency on De-duplication work. 22/10: Dependency on the De-dupe work. Follow up with Chris after the de-dupe work has taken place. This is to identify which Consultant data belongs to which Trust so that they can have this if the trust wants. 29/10: Linked to the removal of the consultant information from TIS we also need to address the issue of the non-trainee posts. So a further piece of work needs to take place to identify and hard delete these posts. Will need to take a copy of this so that (like with the non-trainee records) we can pass this to the trusts if needed. JW to create ticket related to the deletion of posts. 19/11 - Once de-dupe is done, check the full list of People from Chris. Please review against the surviving records (IDs) onTISNEW-2057. 14/01: Wait until the testing of the de-dupe has been done on stage then carry on.- 04/02: Awaiting on DS on testing outcome for TISNEW-2057 to prioritise TISNEW-2083. Does the script attached to 2083 has everything? Check with Rob about urgency of this re IG - JW to do this 01/04: to check with Rob/Ben. 15/04: response from Rob 'From an IG perspective - of we don't need it then we should not keep it.' Supervisors and Leave approvers taken into account on Chris's script. 24/06: The South has undertaken a post audit and set to Inactive Consultant/non current posts which has resulted in a reduction of approx. 2000 posts Suggestion that this is something that the other three regions can do while the Person de-dup is in the process of being completed. Suspect most of the Consultant posts in the North and Mids and East are now inactive meaning that the majority of outstanding posts belong to KSS (possibly) Once this has been completed for Posts they can remain as Inactive rather than undertake another unnecessary step. 22/07: Still waiting on the de-duplication work. To discuss about Supervisors after this piece of work. 17/02: Intrepid/Leave Manager roles to be sorted out ahead of de-dupe of consultants on TIS 20/04:
04/05: Report has been amended - Query saved on Data warehouse. Exclude list of Approvers before removing. 21/05: Update from the south and EM. 03/06: Waiting on Hicom. 29/06: Sue: Hicom sent over a national list (personTISIds) of ~18 approvers to exclude in the merge. (Based on anyone that has ever approved a study leave application). Some coordination with Hicom for these records. The list on Slack to go first iteration excluding these 18 records. Second iteration look at the Hicom coord. ones. Sue to check against the master file.
| On hold until Point 1 has been completed to the group's satisfaction for People area |
3 | Sites created with Trust code. Trust codes are not unique. | 01/03/2019: Tom De Salis: Cleansing the NTS data revealed the fact that 3 of our CCGs merged into a super-CCG last year, which means their sites have the wrong Trust/Trust code on TIS. This is an issue on both the Site reference table and on all the posts related to these sites. It affects around 100 sites.
AR: Site is linked to Trust via the TrustId. When creating a Site, a valid Trust Code is entered and the Trust ID is fetched from the Trust reference table to attach to the Site. However, since the Trust Code is not currently unique in the Trust reference table, the site might be linked to incorrect Trusts. E.g. Trust Code RQ3:   Trust Code VTS: Are these sites linked to the correct trusts? 01/04: Check if the FE is displaying INACTIVE values when creating site. Need to understand what the problem is. to speak to Tom de Salis/invite him to the next data leads call .
15/04:
24/06: Regions to act on report created by Chris. Not for immediate attention due to the current work ongoing in Local Offices. 19/08: Ongoing 07/11/2019 Similar issue raised Lynn Bradley team https://hee-tis.atlassian.net/browse/TISNEW-3571 another ticket raised for all duplicate trust code https://hee-tis.atlassian.net/browse/TISNEW-3577
11/11: Post the spreadsheet on DQ channel to sort out the duplicate Trusts codes and ensure 1 only is CURRENT for each code. Trusts do not have Local Office boundaries, owner field can be ignored. 03/02: Inactivate and ensure only unique trust codes are left
20/04: Check if there are still records on metabase.@Ashley Ransoo / @James Harris https://build.tis.nhs.uk/metabase/question/316 15/07: Sites and posts linked to the trusts that remain in the above query. | Not for immediate attention as priority in LOs is August intake and ARCPS | |
5 | Programmes and rotations on TIS which are current but do not seem to being used and need to be inactivated. | All | https://build.tis.nhs.uk/metabase/question/320 20/05: GW suggestion to look at what GMC approved programmes are per site before inactivating 03/06: 15/07:
Impact of inactivating Rotations associated with Programmes on Posts. Rotations are associated to Programmes and to Posts. 15/06: We will need to supply a list of Programmes to Hicom to add to Oriel as part of Oriel 2 development to facilitate the ingestion of trainees between the Oriel and TIS.
|
|
6 | List of curricula that we could bulk inactivate | All | James: I have started compiling a list of curricula that we could bulk inactivate when we are ready link is https://healtheducationengland.sharepoint.com/:x:/s/ProgrammesAnalysisTeam-MaE/EYOGs7vMqntKim8ycVrduwwBxVi8ktOmu8NhDyORj8VjQg?e=vKD255 09/12: 05/04: Report to be created on metabase @James Harris Logic to be confirmed - @Mike.Richardson (Unlicensed) @Sue Barrand (Unlicensed)
Ticket with list to make inactive with logic confirmed Ticket https://hee-tis.atlassian.net/browse/TISNEW-3386 : - Only make Current curricula available for creating/updating programme memberships with. 03/12: POG to be made 0 on all curriculum. 17/02: Same validation to be applied via bulk - https://hee-tis.atlassian.net/browse/TISNEW-3888 03/06: Ticket for Curricula to bulk inactivate - @Ashley Ransoo
15/06: We’ve inactivated 144 curricula on Stage and Prod. James suggested we inactivate the curriculum first and later look at other stuff as need to check what issues that would cause. Mike: In the South we have 250 trainees in legacy curricula & 300 assessments against them. We will be able to get them changed to non-legacy curricula but it's worth mentioning that removing a curriculum from a programme will also remove the curriculum from any assessment made against it, so those will need to be updated too. (cc @Sue Barrand) (edited) https://build.tis.nhs.uk/metabase/question/327 https://build.tis.nhs.uk/metabase/question/326 15/06: 144 of them have been inactivated. There seems to be lots of new records added with legacy. After the ARCP seasons completed (Beginning of Sept 2020) to inactivate the remaining ones. Caroline - Check those that have been bulk uploaded recently (GIM) Mike - Check those that have been bulk uploaded recently Add Start date to query 327. Check Bulk Upload validating correctly? - https://hee-tis.atlassian.net/browse/TISNEW-3888 There will still be some that are legacy but no ‘legacy’ in their names that would need inactivating (smaller no.). 29/06: POG removed to stop auto-calculation on prog/curr membership. | |
7 | Issues around unique site codes | All | 09/12:
16/12: Further discussions on #dataleads
20/04: Will be discussed next at PAG. 04/05: Site code added to NDW.
| |
8 | Programme memberships with invalid or null dates | All | 31/12: We are working on a ticket in current sprint to set the Status field (Now Training Status) based on the programme membership. There are a number of programme memberships where we cannot set this at the moment as they have either missing start/end dates or the end date is greater than start date, and as such are invalid. Can the below be looked at by data leads for their respective regions: https://build.tis.nhs.uk/metabase/question/296 - add TIS Person ID 17/02: Some of those would not be corrected, old data.
Ticket to turn Training Status to INACTIVE on retrospective trainee records with no prog. mem. - 15/07: EMD - EOE - Caroline to check NTH/NW/YH - LDN - Sarah to check. South - | |
9 | Record Status > Training Status field | All | 03/01: It appears that some Local Offices have been using this field to record Trainers approval. It was agreed at PAG for this field to be auto-populated based on trainee being on a current/inactive programme membership. Any impact on Leave manager / Accent and NDW reporting? 06/01:
03/02: Trainers status have been migrated to the new Trainers Approval area on TIS within the Approval Status. All trainers have been given a status of CURRENT with empty start and end dates, the idea being to have a bulk tool to update those and make the status calculated in the future. 20/04: Bulk People Update based on TIS Person ID coming in the next sprint. (This is on Prod) 15/07: Bulk Site Approval ticket pending. |
|
10 | Updating of Reference tables | Data leads | Who should be able to update what? E.g. Specialty list being updated without other Local Offices knowing, impacts on all records linked to that reference value. Should there be category/group of reference lists that can only be updated by specific people/roles/data leads on TIS or an outside business process to agree who does the update for such references. ticket to support updating the remaining post and stop the wiping out of the specialty on the associated fields. https://hee-tis.atlassian.net/browse/TISNEW-385403/02: Ticket to restrict all except Site, Trusts, Programmes. - @Ashley Ransoo, @James Harris https://hee-tis.atlassian.net/browse/TISNEW-3887There are a number of HEE TIS Admin that needs tidying up. This is the role that will have the permissions left on to manage all reference tables. https://build.tis.nhs.uk/metabase/question/294 20/04: The fix for the bug has been pushed to Prod. Gastro-enterology changed to Gastroenterology. Look at defining Reference tables for TIS: | |
11 | Assessment Bulk Update tool on TIS | All | Review Columns, validations and rules for a proposed Bulk Assessments Update tool using TIS_Assessment_ID's: 17/02: Data leads to review and feed back. North - EMD - LDN - South - | |
12 | Site Approvals Bulk Upload on TIS | All | Fields to review and feedback Approvals - Field Validation#FieldValidation-BulkUploadSiteApprovalsColumnHeaders | |
13 | Recording Programme Curriculum Type on TIS | All/James | https://hee-tis.atlassian.net/browse/TISNEW-4389 Background There is a requirement from GMC to be record this consistently and therefore be able to run reports on this information. Programme Curriculum Types - GMC Approved
We need to be able to record the above and send to the National Datawarehouse for reporting purposes. P.S: The terminology of the Programme Curriculum Types need to be discussed with Data leads - 04/05 - confirmed as above | |
14 | Email address not being unique | Gwil/All |
15/07: DQ dashboard report - to look at and resolve duplicates. Metabase query to be added. |
|
15 | Covid Self Declaration Form/ via Placement comments info | All | To be recorded against Placements comments. Placement comments get removed and replaced with the values on update template that can be downloaded from TIS. If previous comments are to be kept, download report with current comments and add to that with a prefix of “COVID-19” 15/07: Can be closed | CLOSED |
16 | Trusts accounts Auditing on TIS | Rob/Alistair/All/IG | Which accounts are still being used? Simon/Ashley has created a report of last logged in users by roles on TIS. Users that are using Tableau might not have necessarily logged in on TIS. 03/06: There is a way to find out last logged in date on Tableau. - Use this to cross reference the two reports. (@James Harris )
15/07: Tableau logins s/s sent to Adrian. Can be closed. Dealt by User Management group. | CLOSED |
17 | WTE > 1 on Trainee Placements |
| To agree some rules on what to change the non-trainee placements to. https://build.tis.nhs.uk/metabase/question/323 03/06: When there is downstream impact on NDW, to have a process in place to check between the 2 teams. Use the TIS Tech Share session (1st Thursday of every sprint). (e.g. London downstream processes and impacts)
|
|
18 | North West Dental Data |
| Doing a post redistribution of Dental Posts. No current DFT data in NorthWest.
|
|
19 | Recording of ‘Main Programme’ in ACCS Posts to aide reporting |
| ACCS posts require multiple programmes attached. This causes duplicates when running reports on post numbers. Suggestion from PAG is to record ‘Main ACCS Theme/Programme’ in a field against these posts, e.g. PostFamily.
13/07 Closed as it was rejected and sent back to pag | Closed |
20 | CURRENT Posts with white spaces in NPNs | All | There are about 50 posts on TIS that have been created with spaces in their NPNs (leading/trailing/extra white space) This ticket is to prevent this from happening in future edits/creates: https://hee-tis.atlassian.net/browse/TISNEW-4707 However, the existing posts that need sorting out are here: |
|
21 | Adding/Amending Trusts - Mergers in the South | Sue/All | https://hee-tis.atlassian.net/browse/TISNEW-4721
Programme in UM - Refreshed as and when a change is made.
|
|
22 | Bulk People Update - Update and replace all v.s Update | All | Although we've discussed this on our data leads calls previously re the trainer approvals roles and how it would behave on bulk people update(new) v.s. bulk people upload(existing), we'd like to add this as an item to review with you : People Upload (Existing):
People Update (new):
|
|
23 | Null NTN and Training Pathways from TIS. | All | An issue was raised recently about the suffix missing from some NTNs. This is caused by the TrainingPathway being NULL (No Value Provided on the FE). The reason for this is the TrainingPathway auto-calc was only added to the bulk upload in December 2019 which is why the vast majority of these are Feb starters. Also, TrainingPathway only recalculates if the curriculum is changed, not if any other field is amended such as the StartDate (also not if the value had been changed to CESR). Therefore, if a trainee was uploaded prior to December 2019 and then deferred say to August for example, there TrainingPathway would not auto-calculate.We thought your local teams might need to go and enter a value for these trainees so have produced a metabase question to identify them. It's written in TIS_Interim so you can copy the SQL and run in the NDW if you prefer. |
|
24 | 3 Large Trusts reported missing trainees - ESR. (Mainly Foundation Trainees for South) | Mike SOUTH/WMD/NE/YH | Is it volume related for August rotations? Discussion at 4p.m. today. 13/07: Ashley did some investigation on missing apps and found out that a job failed on the 6th of May (exactly 13 weeks ahead of the big Aug Rotation).We discussed later in the evening, and the job was re-run the following day (30/06/2020). This should have resolved the issue of missing applicants. 13/07 resolved | Closed |
25 | Updating of NPNs on TIS and implications to ESR | All/ Phil James | #1 - Updating of NPN’s on Placements that have potentially been exported to ESR #2 - Updating of NPN’s on Posts that have placements attached to it (e.g. CURRENT/PAST& FUTURE)
Tickets to be created - Bulk/UI restriction to the NPN update on posts. - To raise at PAG. #3 - Updating of NPN’s on ESR Position ID where there have potentially been applicants exported previously and implications
|
|
26 | Tickets from Jira related to this Group of Data Leads - Notifications on updates to them? | Gwil/All/ SM’s @Andy Nash (Unlicensed) @Philip Wilsdon (Unlicensed) |
|
|
Jira Tickets (Reference Tables)
Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213