Data leads:
Discussion/Action
# | Discussion/Action | Responsible | Outcome | Status |
|---|---|---|---|---|
| 1 | Resolve Duplicates - TISDEV-4841: Resolve the duplicate invalid GMC numbers, such as 'Retired,' and make them 'UNKNOWN' | Data leads | Discussed on DQ on 04/06:
04/06: The following has been agreed:
DS: List of persons with same GMC's, check the DW to see if some of those have been resolved? TISDEV-4841 to be paused from releasing to Prod until David Short (Unlicensed) has done his investigation of the ones that could potentially be merged. Merge Rules to be shared. To create tickets following that. 11/06: Ongoing and will probably be ready mid-week. 18/06: On hold until the document uploads are available on People records? The idea being if de-duplicated before the documents have been added, we won't be able to attach the documents to the deduped person records. Get an update from Ben/Anita regarding document management in a couple of weeks. 02/07: Benjamin Wittonare you able to give an update on this? 09/07: Ben: We are expecting the document delta from Hicom this week. The next step will be to get them onto Azure and then apply the agreed meta tags. Do let me know if you need any other details/updates. 16/07: Delta received. Document work to continue in this sprint. Following which David can work his magic on the duplicates. David Short (Unlicensed) does your initial work need to be revisited? and will there be any dev requirement related to your dup work? We will need to include this in Planning so will need to know 23/07: Work in progress, script modified. (David Short (Unlicensed), bullet points of what's required to bring into the sprint as User Stories) 13/08: Document Management migration complete - when can work start on the duplicate records David Short (Unlicensed) ? 20/08: DS: Scripts being worked on. Next step to create the stories in Jira. 03/09: David: Looking at having the scenarios by end of the week. 10/09: DS confirmed this is a trainee on his list too - TISNEW-1894 - Getting issue details... STATUS 10/09: DS demo'ed some 'strong match' scenarios that can be safely de-duped. Sue/ Joanne Watson (Unlicensed): impact on leave manager when de-duped. Output of Round 1 of de-duplication: https://healtheducationengland.sharepoint.com/:x:/r/sites/WPI-North/Shared Documents/TIS/North (all)/TIS Person De-Dup Project/Duplicates Output v1.xlsx?d=w70c77e6caa5b4cc1a2408663a1e77fd6&e=4%3a3feb8d869af74e61aeae63ea2a9baff4&at=9 17/09: Tickets for dev to start looking at. (AR and DS) Considering other priorities in the next sprint, we are looking at mid to late October. 24/09: TISNEW-2057 - Getting issue details... STATUS - DS to send the rules to add to the ticket. (This has been added to Sprint 59 which is due to start on Thursday - do we have all the info we need?) 01/10: Methodology attached to the ticket. David Short (Unlicensed) to separate out only the records that need to be merged for Round 1 to attach to the ticket. 15/10: TISNEW-2057 has been pulled out of sprint due to impact on leave manager records. Merging on the TIS side would leave orphan records on the Intrepid/TWS side. 22/10: Awaiting confirmation on the above regarding the correct records out of the duplicates to get rid of. 29/10: List received from David - will add to the ticket and work with devs to get it into a forthcoming sprint. 05/11: Awaiting confirmation on the merge rules from Data leads, as the work involves merging data to the survivor person record then removing the duplicate. DS: There are 2 main merge rules that need to be reviewed by other data leads and may require some tweaking. Person Merge Rules (DRAFT) (David Short (Unlicensed), James Harris, Alice Thompson, Sue Barrand (Unlicensed)) 1.All associated records (placements, programme/curriculum memberships, ARCPs, etc.) belonging to a duplicate person record need to be transferred to the surviving person record. 2.Where the surviving person record has a field containing a NULL or Zero-Length-Strings value but the same field in the duplicate person record contains a non-NULL/ZLS value, this value should be copied into the surviving person record. This should exclude address fields due to possible town/county/postcode mismatches.
12/11: The merge rules have been updated on the ticket and David has provided attachments for:
26/11: The devs have done a spike on this and came up with the following approach, Proposed Solution: With regard to the attached template "TIS People Upload Template - Duplicated records.xlsx", we propose to do a series of 'updates to' and 'insert into' the surviving record. Once this gets completed, we will proceed to delete the duplicate records. Updates: i.e. if a value is present on the duplicate record for the above but null on the surviving, we will be copying them onto the surviving record.
3) Insert: 26/11: Deleting records on Intrepid - will the ETL take care of this? - follow up with the devs/hicom/Adrian Ashley - The expectation is to get them deleted on Intrepid as well. 03/12:
08/01:
DS: Has done some testing. SB: leave manager tests carried out. JH/DS/AH: re-use some of the scripts that were used during consolidation for testing on metabase for next week. DS: Prog membership, placements and assessments - Looks good so far. How long more this would need? - conclude by Wed 16/01.(Sue Barrand (Unlicensed), David Short (Unlicensed) and James Harris are going to test this on Wednesday afternoon) 21/09: The scripts have been run on Prod over the weekend. David Short (Unlicensed) has taken extracts of the data on Friday afternoon ahead of the scripts being run. 4/02: Round two will be needed. LaSE still to advise DS on what to do with the records that weren't an easy match. Chris picking this up - London may be ok but KSS need more attention 4/02: Leave manager - need to do checks for the circa 500 records to make sure that the round 2 de-duping would not leave orphaned records (as per the procedure for Round 1 | |
| 2 | Identification and removal of Consultant data on TIS | All | 13/08:
Additional note - during my catch up with Hicom, a proposed solution from Hicom included looking at the role which a record is associated with (i.e. does the record have a role of 'DR in training' or 'GP Tutor' etc.). I am seeking assistance from the NDW Team to produce an output which shows all records and with what role they have (e.g. clinical supervisor, medical director, leave manager etc.). There is an option to work with Regional Data Quality Leads/Local Teams to confirm which roles are they actively using in TIS. Actions from the meeting:
Additional actions:
04/02: Joanne has reviewed the roles and Ola has done the work on making sure each role has the correct permissions. 20/08: CN: ~20000 records potentially identified. Approx. half from KSS and NW. Removal of non-training grades to follow after removal of consultants - Create Jira ticket to be prioritised. 03/09: Validate 'survivor Ids' on TIS against Intrepid Ids for Leave manager and CPPS 10/09: BW: Waiting on duplication work to be done, then look at. Dental role approval and accreditation process - piece of work required to align the process. e.g. YH prefixed roles. Benjamin Witton: Publish on DQ channel for data leads to review or pass to colleagues/quality team to review.e.g. definitions against the roles. 17/09 - (Regarding Roles) - BW update (by BW) - Starter for ten published to Data Quality Leads to review in their respective Regions and Local Teams. Return asked for by Monday 24th September. Link to access the starter is below: 24/09: Jira Tickets to be created. Benjamin Witton to share work re Roles on DQ channel and follow up. 01/10: TISNEW-2083 22/10: Dependency on the De-dupe work. Follow up with Chris after the de-dupe work has taken place. This is to identify which Consultant data belongs to which Trust so that they can have this if the trust wants. 29/10: Linked to the removal of the consultant information from TIS we also need to address the issue of the non-trainee posts. So a further piece of work needs to take place to identify and hard delete these posts. Will need to take a copy of this so that (like with the non-trainee records) we can pass this to the trusts if needed.JW to create ticket related to the deletion of posts. 19/11 - Once de-dupe is done, check the full list of People from Chris. Please review against the surviving records (IDs) on TISNEW-2057. 14/01: Wait until the testing of the de-dupe has been done on stage then carry on.- Chris Norman (Unlicensed) 04/02: Chris Norman (Unlicensed), David Short (Unlicensed): Awaiting on DS on testing outcome for TISNEW-2057 to prioritise TISNEW-2083. Does the script attached to 2083 has everything? Check with Rob about urgency of this re IG - JW to do this | |
| 3 | Trust access - role logic | All / Andy Horton (Unlicensed) / James Harris | Objective: Gain understanding of what the local office requirements for trust access should be based on - NPN seems to be the incorrect basis as in TWS it is based on Training and/or Employing body Email conversation excerpts: 07/09: - JH: This is what was on Intrepid and works for me Q - So if we are using the Trust of the placement site – then we can still only grant access to a post & its occupant to that one trust Q - e.g. a Severn GP trainee goes to Royal United Hospital (RD130) for a placement. The parent trust in the reference table for that site would be Royal United Hospitals Bath NHS Foundation Trust (RD1). We would also want however for Gloucestershire Hospitals NHS Foundation Trust (RTE) to have access as the lead employer. The only way I’d picture that happening would be through the ‘OR’ statement on Employing Body and Training Body that James mentioned below. Q - small pockets of users that are trust based but have their data profiles restricted by Programme. We have done this for the new GP lead employer in Thames Valley as Pennine Acute Trust won the tender as well as some local trust based administrators that only look after GP. This allowed us to prevent them from seeing trainees that were not relevant to them at the trust they are based but also an easy way of seeing the trainees out in GP practices as they don’t really have a parent trust, we have the CCG in that field for GP practices. I suspect we may need to find a workaround to this in the short term perhaps using reporting. Joanne Watson (Unlicensed) / Ashley Ransoo - Requirement for TIS: (Ticket to be created and get an estimate from Dev) TISNEW-1987 - Getting issue details... STATUS People - If either the trainnee's EmployingBody OR TrainingBody for their Current Placement Post match the Trust, then make available to the Trust. Posts - If either the User is at a trust which is the EmployingBody OR TrainingBody of the Post, then make available to the Trust. 17/09: JW: ...very rough guideline of the complexity of amending the current logic on the trust role was given an 8 (out of a possible 10). This is rather large and I have concerns that we might not make the deadline of the end of next week for the role to be amended, tested and users issued with their login details. So, with the above in mind I considered what else we could do if we approach this from another angle. Please could I ask you to consider the following - instead of changing the trust role we leave it as it currently is - so placement>post>site>trust but for those users who are lead employers or who just need to see the trainees and posts from a number of trusts we give them the access to that range of trusts so they can see the entire trainee landscape they should. We would obviously need this info from yourselves but it would save us a lot of dev time. This isn't elegant but should get us across the line for the 1st October. I then suggest that I ask Ashley to speak to you all asap and get full understanding of the different roles in trust-land so that we can correctly create roles that are needed e.g. the one to allow users to only see trainees based on programme etc. We can then get these roles created and rolled out. GW/AH: There might be an IG Issue as they would be able to see trainees that they should not be able to see. JW: Will be discussed at pre-planning. This will be taken into the next sprint. Review progress on next call. 24/09: Being worked on in Current Sprint. Joanne Watson (Unlicensed)/ Alistair Pringle (Unlicensed) - To get an update from dev 01/10: Trust role on Prod. Need to create the trust access role based on Programme next. 22/10: Andy will confirm the details ahead of next sprint planning. 29/10: Andy confirmed that the logic was acceptable for the South. Forwarded to LaSE and North for info. Ticket will go into Sprint 61. JW to speak to Simon/Paul about a bulk addition of these users/ 05/11: In progress in current sprint 12/11: How many different user roles are we eventually going to have for Trusts access? - To review with the devs/ UM 19/11: Yes, if there is a requirement to, would need to be ticketed up as and when. 26/10: Programme based access, released on Friday. James testing it at the moment. 03/10: This has been released as part of last sprint. Can this be closed?
14/01: Andy Horton (Unlicensed) to test with future programme membership. Provide some examples. 04/02: Paul has applied the fix - will be on Prod on the 5th. JW to test before alerting users | |
| 4 | Rob - New approach with data leads/data group: | Rob/Data Leads | To discuss with the existing data leads... Data Group • Ownership of the data model in TIS inc. multi-professional 05/11: BW when is this going to get formalised and therefore move this call entirely to the new 'TIS Data Group'. 12/11: Data quality dashboard? Alistair Pringle (Unlicensed) to speak to Rob. | CLOSE |
| 5 | NETs Data | Many thanks for previously organising the email addresses to come to us out of TIS from the regions. The launch went well. We need to be able to identify the training programmes the trainees are on that were sent to us to help us calculate response rates. We have this for the London region but is it possible to request this for the South, North and Midlands and East regions? As I say, it is only the training programme – e.g. Foundation, GP, General Surgery etc. - Richard Higgins 26/11: Alistair Pringle (Unlicensed) get a list of names for each region and post on DQ channel. 03/12: Awaiting for the list. 14/01: James Harris will try and do a national pull. Andy has done for the South. | CLOSE | |
| 6 | ESR | Missing applicants from the South. Familiarisation of the ESR error reports.
03/12: Andy Horton (Unlicensed) - Examples of notifications where the next post holders are empty to be provided for further investigation. 08/01: More examples sent through Naz. Comms to be sent trust regarding support. 14/01: Investigation carried out and sent to Mike. Discussed on the ESR call last week. The outcome falls into 3 categories which Mike can explain.
Decision support tree being put in place (Craig Elton from ESR and Mike supporting this) A demo of how to investigate ESR issues raised by trusts has been scheduled for this Friday. - Intended for HEE staff with access to Metabase.(record the session) 21/01: Session was done on Friday 18/01 on how to investigate ESR issues. Note: This is not to be shared with Trusts or anyone outside HEE due to IG. Link to recording: 04/02: only outstanding piece of work related to this is to get the data our of Metabase and into NDW and then Tableau for wider investigation 04/03: The ticket is being worked on in this Sprint. TISNEW-1926 | CLOSE | |
| 7 | Records Live on TIS but Deleted on Intrepid | JT/Adrian | This was reported around Christmas period and was resolved, however there are some outstanding actions either between TIS/NDW TIS-NDW External Flows diagram review? Impact Assessment when changes made on a particular flow/code Dashboard for data quality checks. ETL changes if Hicom products are not re-procured at the end of March 2019. 04/02: Is there an update to this? John Thompson has this been rectified? | Closed |
| 8 |
| Data Leads | JH 31/01/2019: From a conversation around the Placement Planner Tool, it was brought up that there were current Programmes that didn't have any People or Posts attached. It seemed like this should be something we discuss on the data leads call. From first thought it would seem we should inactivate the programmes. This spreadsheet lists those Programmes. There is a tab for programmes with both no people or posts attached, one for just no posts and one for just no people. AR 25/01: Whilst discussing the Bulk Post Upload requirements, it became apparent that 'Programme name' when creating Posts is not a mandatory field on TIS. There are Posts with no programmes at the moment. The consensus was to make this field mandatory when creating posts, however, there will need to be a tidying up exercise by the data leads regarding those posts. Tableau - Alice and James (Intrepid/TIS views for data audit will show posts with no programme attached) 04/02: Data leads/local offices to de-activate those that can be de-activated and remainder to be left. Not an urgent piece of work. 04/02: Data leads to check the report JH sent showing the placements with no start or end date or both approx. 530 | |
| 9 | TISNEW-2378 suggests that some Trust users may be attached to inactive trusts. | Is this an issue and who would like to take this piece of work on? 04/02: JH to create report showing the users who are attached to inactive trusts and share with group for decision on how to fix this if needed. JH found 16 users who are attached to an inactive Trust - Scarborough & North East Yorkshire Healthcare NHS Trust, Rotherham Doncaster and South Humber Mental Health NHS Foundation Trust. Kay and Helan at Y&H to be asked if the users Current trust is the right trust by JW | ||
| 10 | Sites created with Trust code. Trust codes are not unique. | 01/03/2019: Tom De Salis: Cleansing the NTS data revealed the fact that 3 of our CCGs merged into a super-CCG last year, which means their sites have the wrong Trust/Trust code on TIS. This is an issue on both the Site reference table and on all the posts related to these sites. It affects around 100 sites.
AR: Site is linked to Trust via the TrustId. When creating a Site, a valid Trust Code is entered and the Trust ID is fetched from the Trust reference table to attach to the Site. However, since the Trust Code is not currently unique in the Trust reference table, the site might be linked to incorrect Trusts. E.g. Trust Code RQ3:
Trust Code VTS:
Are these sites linked to the correct trusts James Harris?
|
Complexity and volume - David may need to work with Dev in next sprint in order to prepare this. Prioritisation required. 
0 Comments