Data leads:
Discussion/Action
# | Discussion/Action | Responsible | Outcome | Status | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Resolve Duplicates - TISDEV-4841: Resolve the duplicate invalid GMC numbers, such as 'Retired,' and make them 'UNKNOWN' | Data leads | 5 | NETs Data | CLOSE | 6 | ESR | CLOSE | 7 | Records Live on TIS but Deleted on Intrepid | JT/Adrian | Closed | 8 |
| Data Leads | 9 | Is this an issue and who would like to take this piece of work on? 04/02: JH to create report showing the users who are attached to inactive trusts and share with group for decision on how to fix this if needed. JH found 16 users who are attached to an inactive Trust - Scarborough & North East Yorkshire Healthcare NHS Trust, Rotherham Doncaster and South Humber Mental Health NHS Foundation Trust. Kay and Helan at Y&H to be asked if the users Current trust is the right trust by JW | 10Discussed First discussed on DQ on 04/06:
04/06: The following has been agreed:
DS: List of persons with same GMC's, check the DW to see if some of those have been resolved? TISDEV-4841 to be paused from releasing to Prod until David Short (Unlicensed) has done his investigation of the ones that could potentially be merged. Merge Rules to be shared. To create tickets following that. 11/06: Ongoing and will probably be ready mid-week. 18/06: On hold until the document uploads are available on People records? The idea being if de-duplicated before the documents have been added, we won't be able to attach the documents to the deduped person records. Get an update from Ben/Anita regarding document management in a couple of weeks. 02/07: Benjamin Wittonare you able to give an update on this? 09/07: Ben: We are expecting the document delta from Hicom this week. The next step will be to get them onto Azure and then apply the agreed meta tags. Do let me know if you need any other details/updates. 16/07: Delta received. Document work to continue in this sprint. Following which David can work his magic on the duplicates. David Short (Unlicensed) does your initial work need to be revisited? and will there be any dev requirement related to your dup work? We will need to include this in Planning so will need to know Complexity and volume - David may need to work with Dev in next sprint in order to prepare this. Prioritisation required. 23/07: Work in progress, script modified. (David Short (Unlicensed), bullet points of what's required to bring into the sprint as User Stories) 13/08: Document Management migration complete - when can work start on the duplicate records David Short (Unlicensed) ? 20/08: DS: Scripts being worked on. Next step to create the stories in Jira. 03/09: David: Looking at having the scenarios by end of the week. 10/09: DS confirmed this is a trainee on his list too -
10/09: DS demo'ed some 'strong match' scenarios that can be safely de-duped. Sue/ Joanne Watson (Unlicensed): impact on leave manager when de-duped. Output of Round 1 of de-duplication: https://healtheducationengland.sharepoint.com/:x:/r/sites/WPI-North/Shared Documents/TIS/North (all)/TIS Person De-Dup Project/Duplicates Output v1.xlsx?d=w70c77e6caa5b4cc1a2408663a1e77fd6&e=4%3a3feb8d869af74e61aeae63ea2a9baff4&at=9 17/09: Tickets for dev to start looking at. (AR and DS) Considering other priorities in the next sprint, we are looking at mid to late October. 24/09:
01/10: Methodology attached to the ticket. David Short (Unlicensed) to separate out only the records that need to be merged for Round 1 to attach to the ticket. 15/10: TISNEW-2057 has been pulled out of sprint due to impact on leave manager records. Merging on the TIS side would leave orphan records on the Intrepid/TWS side. 22/10: Awaiting confirmation on the above regarding the correct records out of the duplicates to get rid of. 29/10: List received from David - will add to the ticket and work with devs to get it into a forthcoming sprint. 05/11: Awaiting confirmation on the merge rules from Data leads, as the work involves merging data to the survivor person record then removing the duplicate. DS: There are 2 main merge rules that need to be reviewed by other data leads and may require some tweaking. Person Merge Rules (DRAFT) (David Short (Unlicensed), James Harris, Alice Thompson, Sue Barrand (Unlicensed)) 1.All associated records (placements, programme/curriculum memberships, ARCPs, etc.) belonging to a duplicate person record need to be transferred to the surviving person record. 2.Where the surviving person record has a field containing a NULL or Zero-Length-Strings value but the same field in the duplicate person record contains a non-NULL/ZLS value, this value should be copied into the surviving person record. This should exclude address fields due to possible town/county/postcode mismatches.
12/11: The merge rules have been updated on the ticket and David has provided attachments for:
26/11: The devs have done a spike on this and came up with the following approach, Proposed Solution: With regard to the attached template "TIS People Upload Template - Duplicated records.xlsx", we propose to do a series of 'updates to' and 'insert into' the surviving record. Once this gets completed, we will proceed to delete the duplicate records. Updates: i.e. if a value is present on the duplicate record for the above but null on the surviving, we will be copying them onto the surviving record. 2) We will be directly updating the surviving records with the presented GmcDetail or GdcDetail. https://hee-tis.atlassian.net/secure/attachment/21793/DeDup_NoLeave_GMC_GDC_PH.xlsx3) Insert: 26/11: Deleting records on Intrepid - will the ETL take care of this? - follow up with the devs/hicom/Adrian Ashley - The expectation is to get them deleted on Intrepid as well. 03/12:
08/01:
DS: Has done some testing. SB: leave manager tests carried out. JH/DS/AH: re-use some of the scripts that were used during consolidation for testing on metabase for next week. DS: Prog membership, placements and assessments - Looks good so far. How long more this would need? - conclude by Wed 16/01.(Sue Barrand (Unlicensed), David Short (Unlicensed) and James Harris are going to test this on Wednesday afternoon) 21/09: The scripts have been run on Prod over the weekend. David Short (Unlicensed) has taken extracts of the data on Friday afternoon ahead of the scripts being run. 4/02: Round two will be needed. LaSE still to advise DS on what to do with the records that weren't an easy match. Chris picking this up - London may be ok but KSS need more attention 4/02: Leave manager - need to do checks for the circa 500 records to make sure that the round 2 de-duping would not leave orphaned records (as per the procedure for Round 1 | 2 | Identification and removal of Consultant data on TIS | All | 3 | Trust access - role logic | All / Andy Horton (Unlicensed) / James Harris |
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
People - If either the trainnee's EmployingBody OR TrainingBody for their Current Placement Post match the Trust, then make available to the Trust.
Posts - If either the User is at a trust which is the EmployingBody OR TrainingBody of the Post, then make available to the Trust.
17/09:
JW:
...very rough guideline of the complexity of amending the current logic on the trust role was given an 8 (out of a possible 10). This is rather large and I have concerns that we might not make the deadline of the end of next week for the role to be amended, tested and users issued with their login details.
So, with the above in mind I considered what else we could do if we approach this from another angle. Please could I ask you to consider the following - instead of changing the trust role we leave it as it currently is - so placement>post>site>trust but for those users who are lead employers or who just need to see the trainees and posts from a number of trusts we give them the access to that range of trusts so they can see the entire trainee landscape they should. We would obviously need this info from yourselves but it would save us a lot of dev time.
This isn't elegant but should get us across the line for the 1st October. I then suggest that I ask Ashley to speak to you all asap and get full understanding of the different roles in trust-land so that we can correctly create roles that are needed e.g. the one to allow users to only see trainees based on programme etc. We can then get these roles created and rolled out.
GW/AH: There might be an IG Issue as they would be able to see trainees that they should not be able to see.
JW: Will be discussed at pre-planning. This will be taken into the next sprint. Review progress on next call.
24/09: Being worked on in Current Sprint. Joanne Watson (Unlicensed)/ Alistair Pringle (Unlicensed) - To get an update from dev
01/10: Trust role on Prod. Need to create the trust access role based on Programme next.
22/10: Andy will confirm the details ahead of next sprint planning.
29/10: Andy confirmed that the logic was acceptable for the South. Forwarded to LaSE and North for info. Ticket will go into Sprint 61. JW to speak to Simon/Paul about a bulk addition of these users/
05/11: In progress in current sprint
12/11: How many different user roles are we eventually going to have for Trusts access? - To review with the devs/ UM
19/11: Yes, if there is a requirement to, would need to be ticketed up as and when.
26/10: Programme based access, released on Friday. James testing it at the moment.
03/10: This has been released as part of last sprint. Can this be closed?
- some of the issues could not be replicated.
- Trust level access v/s Programme level access - understanding the interactions between the two? - Alistair Pringle (Unlicensed)
- Roles: Definition (review confluence and update with the devs on what each of them gives access to)
- Report of Roles against individuals (check with NDW)
14/01: Andy Horton (Unlicensed) to test with future programme membership. Provide some examples.
04/02: Paul has applied the fix - will be on Prod on the 5th. JW to test before alerting users
To discuss with the existing data leads...
Data Group
• X4 Regional Data Leads; NDW link; Service Manager
• Self-ordering team
• Transparency
o One action list
o MS Teams
o Slack
• Ownership of the data model in TIS inc. multi-professional
• Manage business-facing reference tables and have a change control process
• Simplify reference data and standardise to HEE / DH / ESR standards
• Management of data issues as and when
• Develop Dashboards
o Data Quality
o Management information
• Participate as data experts in any process alignment, research or development work
• Help identify, rationalise and simplify data flows and data handling e.g. bulk uploads, ESR etc
• Prioritise data activity
• Work with NDW to ensure data relevance
• Develop and support national reports and extracts
• Undertake testing where there is a data
05/11: BW when is this going to get formalised and therefore move this call entirely to the new 'TIS Data Group'.
12/11: Data quality dashboard? Alistair Pringle (Unlicensed) to speak to Rob.
08/01:
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
DS: Has done some testing. SB: leave manager tests carried out. JH/DS/AH: re-use some of the scripts that were used during consolidation for testing on metabase for next week.
DS: Prog membership, placements and assessments - Looks good so far.
How long more this would need? - conclude by Wed 16/01.(Sue Barrand (Unlicensed), David Short (Unlicensed) and James Harris are going to test this on Wednesday afternoon)
21/09: The scripts have been run on Prod over the weekend. David Short (Unlicensed) has taken extracts of the data on Friday afternoon ahead of the scripts being run.
4/02: Round two will be needed. LaSE still to advise DS on what to do with the records that weren't an easy match. Chris picking this up - London may be ok but KSS need more attention
4/02: Leave manager - need to do checks for the circa 500 records to make sure that the round 2 de-duping would not leave orphaned records (as per the procedure for Round 1
Removal of consultant data has a dependency on De-duplication work.
22/10: Dependency on the De-dupe work. Follow up with Chris after the de-dupe work has taken place. This is to identify which Consultant data belongs to which Trust so that they can have this if the trust wants.
29/10: Linked to the removal of the consultant information from TIS we also need to address the issue of the non-trainee posts. So a further piece of work needs to take place to identify and hard delete these posts. Will need to take a copy of this so that (like with the non-trainee records) we can pass this to the trusts if needed. JW to create ticket related to the deletion of posts.
19/11 - Once de-dupe is done, check the full list of People from Chris. Please review against the surviving records (IDs) on TISNEW-2057.
14/01: Wait until the testing of the de-dupe has been done on stage then carry on.- Chris Norman (Unlicensed)
04/02: Chris Norman (Unlicensed), David Short (Unlicensed): Awaiting on DS on testing outcome for TISNEW-2057 to prioritise TISNEW-2083. Does the script attached to 2083 has everything?
Check with Rob about urgency of this re IG - JW to do this
- Programmes with no Posts or People
- Posts with no Programmes
View file | ||||
---|---|---|---|---|
|
JH 31/01/2019:
From a conversation around the Placement Planner Tool, it was brought up that there were current Programmes that didn't have any People or Posts attached. It seemed like this should be something we discuss on the data leads call. From first thought it would seem we should inactivate the programmes.
This spreadsheet lists those Programmes. There is a tab for programmes with both no people or posts attached, one for just no posts and one for just no people.
AR 25/01: Whilst discussing the Bulk Post Upload requirements, it became apparent that 'Programme name' when creating Posts is not a mandatory field on TIS. There are Posts with no programmes at the moment. The consensus was to make this field mandatory when creating posts, however, there will need to be a tidying up exercise by the data leads regarding those posts.
Tableau - Alice and James (Intrepid/TIS views for data audit will show posts with no programme attached)
04/02: Data leads/local offices to de-activate those that can be de-activated and remainder to be left. Not an urgent piece of work.
04/02: Data leads to check the report JH sent showing the placements with no start or end date or both approx. 530
TISNEW-2378 suggests that some Trust users may be attached to inactive trusts.
Is this an issue and who would like to take this piece of work on?
04/02: JH to create report showing the users who are attached to inactive trusts and share with group for decision on how to fix this if needed.
JH found 16 users who are attached to an inactive Trust - Scarborough & North East Yorkshire Healthcare NHS Trust, Rotherham Doncaster and South Humber Mental Health NHS Foundation Trust. Kay and Helan at Y&H to be asked if the users Current trust is the right trust by JW
01/04/2019: LaSE- Creation of Trust Users. What is the priority on this? The last we had was this would be required for beginning of May, which has therefore made the IMT posts ticket a higher priority than this.
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
01/04/2019: The following ticket has been taken into our current sprint due to the urgency LaSE requires the IMT posts to be on TIS (by mid-April). However we are still awaiting for the template to be completed and sent over to the TIS development team.
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
- Do we know when the post update tool might be available to update the posts? Is it worth waiting?
- Would we be able to do an update on the back-end to the site reference table?
AR:
Site is linked to Trust via the TrustId. When creating a Site, a valid Trust Code is entered and the Trust ID is fetched from the Trust reference table to attach to the Site.
However, since the Trust Code is not currently unique in the Trust reference table, the site might be linked to incorrect Trusts.
E.g.
Trust Code RQ3:
Trust Code VTS:
Are these sites linked to the correct trusts James Harris?
...