Attendees: Alistair Pringle (Unlicensed), Sue Barrand (Unlicensed), James Harris, Andy Horton (Unlicensed), David Short (Unlicensed), Chris Norman (Unlicensed), Adewale Adekoya, Ashley Ransoo
Discussion/Action :
First discussed on DQ on 04/06:
08/01: 
TISNEW-2364 - Merging, then removal of Duplicate Person Records - Round 1 DONE - Work completed for Round 1 on Stage and testing began. Since this was a flyway script, the importance was stressed on completion of the testing on stage as this would block anything else from being released to Production in out release pipeline
DS: Has done some testing. SB: leave manager tests carried out. JH/DS/AH: re-use some of the scripts that were used during consolidation for testing on metabase for next week.
DS: Prog membership, placements and assessments - Looks good so far.
How long more this would need? - conclude by Wed 16/01.( and are going to test this on Wednesday afternoon)
21/09: The scripts have been run on Prod over the weekend.
has taken extracts of the data on Friday afternoon ahead of the scripts being run. 4/02: Round two will be needed. LaSE still to advise DS on what to do with theAttendees: Alistair Pringle (Unlicensed), Sue Barrand (Unlicensed), James Harris, Andy Horton (Unlicensed), David Short (Unlicensed), Chris Norman (Unlicensed), Adewale Adekoya, Ashley Ransoo
Discussion/Action :
# | Discussion/Action | Responsible | Outcome | Status | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Resolve Duplicates - TISDEV-4841: Resolve the duplicate invalid GMC numbers, such as 'Retired,' and make them 'UNKNOWN' | Data leads | First discussed on DQ on 04/06: 08/01: DS: Has done some testing. SB: leave manager tests carried out. JH/DS/AH: re-use some of the scripts that were used during consolidation for testing on metabase for next week. DS: Prog membership, placements and assessments - Looks good so far. How long more this would need? - conclude by Wed 16/01.( and are going to test this on Wednesday afternoon) 21/09: The scripts have been run on Prod over the weekend. has taken extracts of the data on Friday afternoon ahead of the scripts being run.4/02: Round two will be needed. LaSE still to advise DS on what to do with the records that weren't an easy match. Chris picking this up - London may be ok but KSS need more attention 4/02: Leave manager - need to do checks for the circa 500 records to make sure that the round 2 de-duping would not leave orphaned records (as per the procedure for Round 1 01/04: to provide the list of Id's/template for the second round of duplication (second strongest match rule). To send the list out to all regions be checked. Jen would have done for the South - to be checked?15/04: Still awaiting for EMD, South and London. 24/06: Jen to speak to David tomorrow (25th) re queries she has found in the South. James to ask Gwil where has gotten to with this - returns from leave on the 1st July 08/07: David to resend file to Gwil and Jen is no longer confused 22/07: Ongoing, waiting for South and EMD data 02/09 EM have submitted, awaiting the South (ongoing) | 2 | Identification and removal of 14/10: Back with DS - to compare and rebuild the new list of what's been merged
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 2 | Identification and removal of Consultant data on TIS | All | Removal of consultant data has a dependency on De-duplication work. 22/10: Dependency on the De-dupe work. Follow up with Chris after the de-dupe work has taken place. This is to identify which Consultant data belongs to which Trust so that they can have this if the trust wants. 29/10: Linked to the removal of the consultant information from TIS we also need to address the issue of the non-trainee posts. So a further piece of work needs to take place to identify and hard delete these posts. Will need to take a copy of this so that (like with the non-trainee records) we can pass this to the trusts if needed. JW to create ticket related to the deletion of posts. 19/11 - Once de-dupe is done, check the full list of People from Chris. Please review against the surviving records (IDs) on TISNEW-2057. 14/01: Wait until the testing of the de-dupe has been done on stage then carry on.- 04/02: Awaiting on DS on testing outcome for TISNEW-2057 to prioritise TISNEW-2083. Does the script attached to 2083 has everything? Check with Rob about urgency of this re IG - JW to do this 01/04: to check with Rob/Ben.15/04: response from Rob 'From an IG perspective - of we don't need it then we should not keep it.' Supervisors and Leave approvers taken into account on Chris's script. 24/06: The South has undertaken a post audit and set to Inactive Consultant/non current posts which has resulted in a reduction of approx. 2000 posts Suggestion that this is something that the other three regions can do while the Person de-dup is in the process of being completed. Suspect most of the Consultant posts in the North and Mids and East are now inactive meaning that the majority of outstanding posts belong to KSS (possibly) Once this has been completed for Posts they can remain as Inactive rather than undertake another unnecessary step. 22/07: Still waiting on the de-duplication work. To discuss about Supervisors after this piece of work. | On hold until Point 1 has been completed to the group's satisfaction for People area | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 3 | Sites created with Trust code. Trust codes are not unique. | 01/03/2019: Tom De Salis: Cleansing the NTS data revealed the fact that 3 of our CCGs merged into a super-CCG last year, which means their sites have the wrong Trust/Trust code on TIS. This is an issue on both the Site reference table and on all the posts related to these sites. It affects around 100 sites.
AR: Site is linked to Trust via the TrustId. When creating a Site, a valid Trust Code is entered and the Trust ID is fetched from the Trust reference table to attach to the Site. However, since the Trust Code is not currently unique in the Trust reference table, the site might be linked to incorrect Trusts. E.g. Trust Code RQ3:
Trust Code VTS:
Are these sites linked to the correct trusts ?
01/04: Check if the FE is displaying INACTIVE values when creating site. Need to understand what the problem is. to speak to Tom de Salis/invite him to the next data leads call .
15/04:
24/06: Regions to act on report created by Chris. Not for immediate attention due to the current work ongoing in Local Offices. 19/08: Ongoing | Not for immediate attention as priority in LOs is August intake and ARCPS | 4 | Funding Body | All | 15/04: Agreement to retain the Funding Body field in TIS. Data leads to come up with a list of criteria to autopopulate this field? For discussion Ticket to be created following this exercise. Take the discussion to DQ channel. to summarise the criteria on the channel. 24/06: Data leads have agreed and approved the logic. Ticket to be raised (and ensure that part of the function allows manual override of the auto-populated field) 08/07: JW to raise ticket... 22/07: Check with Joanne. (applies to only newly created posts) 15/08: TISNEW-3241 (auto-population on TIS) created - data leads to review please and discuss this in relation to Point 8 below 16/09: Should be able to change default value. 30/09: This Funding Bodies to be maintained in a separate reference table. This to include the Trusts, Local Offices and others. When adding new/changing existing trusts, this to also reflect in the funding body list. Organisation code of the Trusts to be included.
Are there any possible potential data conflict in NDW and Intrepid post fundings? | 5 | Unknown trust codes linked to Sites | Data leads | 6 | Dental data on TIS | All | 24/06: No DFT in the NW and Orthodontics in KSS. 08/07: David to be in receipt of this info in near future - suspect for David to import into TIS 22/07: KSS: Nothing to be done, NW: Will be done imminently 19/08: Awaiting on NW. 30/09: It's in progress. | 7 | Missing data from placements is causing entire specialties not to render on the PPT FE | All | 08/07: Ashley to produce a report showing any current placements are missing mandatory information so this can be added to the placements in question. 09/07:07/11/2019 Similar issue raised Lynn Bradley team
| Not for immediate attention as priority in LOs is August intake and ARCPS | ||||||||||||||||||||||||||||||||||||||||||||
| 4 | Funding Body | All | 15/04: Agreement to retain the Funding Body field in TIS. Data leads to come up with a list of criteria to autopopulate this field? For discussion Ticket to be created following this exercise. Take the discussion to DQ channel. to summarise the criteria on the channel. 24/06: Data leads have agreed and approved the logic. Ticket to be raised (and ensure that part of the function allows manual override of the auto-populated field) 08/07: JW to raise ticket... 22/07: Check with Joanne. (applies to only newly created posts) 15/08: TISNEW-3241 (auto-population on TIS) created - data leads to review please and discuss this in relation to Point 8 below 16/09: Should be able to change default value. 30/09: This Funding Bodies to be maintained in a separate reference table. This to include the Trusts, Local Offices and others. When adding new/changing existing trusts, this to also reflect in the funding body list. Organisation code of the Trusts to be included.
Are there any possible potential data conflict in NDW and Intrepid post fundings? - None | CLOSED | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 5 | Unknown trust codes linked to Sites | Data leads | 11 | NTN Generation | Data Leads | Training Pathways:Ola spotted that there are 56 sites which have an unknown Trust code attached to them. Data leads have been asked to investigate the ones that belong to their region and update the unknown trust code with a known trust code where needed/possible (posted in Data Leads channel on the 3/6/2019) TISNEW-3041 24/06: 49 records still left to be looked at. https://build.tis.nhs.uk/metabase/question/279 The last column tells you what's missing. I have limited it to placements with an end date greater than 08-07-2018 assuming the placement end date itself is not missing. It is still a large number - 4546 records Placements with a missing end date and the highest start date available in the database (1/03/2017): https://build.tis.nhs.uk/metabase/question/278 Solution could be to only get the PPT to show the placements for the past 24 months - AP to speak Ola about this. 19/08: Ongoing piece of work to tidy up data | 8 | Varchar max causing issues in Tableau | All | 9 | Curriculum GMC Reference Number | All |
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
30/09:
- This is not in Assessment.
- Ticket to be tidied up: ProgrammeCurriculum needs to include another field called ProgrammeCurriculumGMCReferenceNumber both in the FE and BE.
- Does not need to be under Programme membership.
- It's like a Programme number format, 6 character format alphanumeric. (non-mandatory)
- This field needs to be sent to NDW.
- To get the data from GMC via GMC Connect, to input into this field on TIS.
E&T Currencies:
Build a report which derives the currency code from data we already have on TIS
Regions to look at when they have the time. South has completed their part.
08/07: work ongoing
24/06: No DFT in the NW and Orthodontics in KSS.
08/07: David to be in receipt of this info in near future - suspect for David to import into TIS 
22/07: KSS: Nothing to be done, NW: Will be done imminently
19/08: Awaiting on NW.
30/09: It's in progress.
08/07: Ashley to produce a report showing any current placements are missing mandatory information so this can be added to the placements in question.
09/07:https://build.tis.nhs.uk/metabase/question/279
The last column tells you what's missing. I have limited it to placements with an end date greater than 08-07-2018 assuming the placement end date itself is not missing. It is still a large number - 4546 records 
Placements with a missing end date and the highest start date available in the database (1/03/2017):
https://build.tis.nhs.uk/metabase/question/278
Solution could be to only get the PPT to show the placements for the past 24 months - AP to speak Ola about this.
19/08: Ongoing piece of work to tidy up data
08/07: Workarounds in place but would be good to know the scale of the issue - JT to provide more info when he returns from leave (again....)
22/07: Determine data type TIS is sending to NDW? awaiting for JT, post on tis-ndw channel.
19/08: Recommendations for length required.
Speak to JT
04/09: JH: From discussion in the dataleads slack channel, David and I have agreed on some recommended field sizes and David has put in the recommendations for each field in this spreadsheet. Please could you add any comments in the dataleads channel on slack?
16/09: JT: The only one I would potentially query is vwPerson.Role, as this is a commonly used search term and ergo might benefit from an index, which can’t be added for Varchar(MAX).
I wonder if there is a data quality issue worth investigating? I find it hard to believe there is a legitimate person record with over 4000 characters in role. Ultimately, a separate PersonRole table might be the best solution (I will check with Pavel).
JH/DS: To have another look at Roles and rationalise.
Work would be required on both TIS and NDW side to change to the defined size.
30/09: To ticket this up.
- Tableau does not handle varcharmax properly
- SSIS has the same issue
- Refine with devs to add limit to the fields both FE and BE on TIS
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
19/08: Needs adding to TIS for ARCP return. Add to Assessments bulk upload.
Can be done via Intrepid avoiding the need to add it to TIS Assessments upload.
Add to NDW - TISNEW-1547-Assessment:CurriculumGMCReferenceNumber not in NDW
Jira Legacy server System JIRA serverId 4c843cd5-e5a9-329d-ae88-66091fcfe3c7 key TISNEW-1547
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
- CCT
- CESR
- N/A - Yes for Core and Foundation Programmes
- CEGPR - No
- Any others? - No
16/09:(Ashley Ransoo)
- Curriculum > Leads to CCT field - This needs to be added to NDW. Needs tidying up.
- Following the tidying up, wherever the value is YES, it should default the Training pathway field to CCT, where No, then to N/A. - New ticket for this
Jira Legacy server System JIRA serverId 4c843cd5-e5a9-329d-ae88-66091fcfe3c7 key TISNEW-3394 Jira Legacy server System JIRA serverId 4c843cd5-e5a9-329d-ae88-66091fcfe3c7 key TISNEW-3389 - Bulk Upload - People Import to be extended (Add/Update) - Another ticket
30/09: AR: There are users on TIS that have been setup with the incorrect roles combination.
e.g. HEE Admin and one of the other roles in (Trust Observer/Admin, Programme Observer/Admin) or user has HEE Admin and one of the Trust/Programme roles.
- To be looked at along with the User Management enhancements
- Tidying up of roles not needed
- Look at a better way of managing roles for TIS
- IG implications
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Ashley Ransoo - ticket this up.
|
30/09:
- This is not in Assessment.
- Ticket to be tidied up: ProgrammeCurriculum needs to include another field called ProgrammeCurriculumGMCReferenceNumber both in the FE and BE.
- Does not need to be under Programme membership.
- It's like a Programme number format, 6 character format alphanumeric. (non-mandatory)
- This field needs to be sent to NDW.
- To get the data from GMC via GMC Connect, to input into this field on TIS.
29/10:
- DS: "...the list of programme codes is the best I'd get, there's no matrix of programme to curriculum."
- There are gaps for some programmes, where a GMC Curriculum Reference number could not be provided
- Why the gaps and why some regions/programmes have no such matrix to curriculum? Are they not being used/not required for GMC ARCP return for these programmes?
E&T Currencies:
Build a report which derives the currency code from data we already have on TIS
16/09:
- Education and Training currencies to be added to TIS
- Only Postgraduates are on TIS
- Instead of Adding a field onto TIS to be maintained by a national process and team, in the short term a report to be produced in NDW based on
- Post information to determine if higher specialty
- Longer term, look at TIS if needed, as finance do not really use TIS nationally.
- Possibility of this changing in the future as well as to how this is recorded
- to be done by the end of this month - James and Chris very keen on
30/09: Awaiting for dental and PH and some other specialties from Rozeen. In progress... Needs to get raised with BM's re timescales.
14/10: A report is being run nationally (agreed by Jon Howes).JH to share the output.
30/09: AR: There are users on TIS that have been setup with the incorrect roles combination.
e.g. HEE Admin and one of the other roles in (Trust Observer/Admin, Programme Observer/Admin) or user has HEE Admin and one of the Trust/Programme roles.
- To be looked at along with the User Management enhancements
- Tidying up of roles not needed
- Look at a better way of managing roles for TIS
- IG implications
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
14
/10: User management reviewed and stories for improving ahead of the above ticket being tackled.
Ashley Ransoo - ticket this up.
| Jira Legacy | ||||
|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
- Review of how this works in the current ARCP return
- If the Curricula dates change how do we capture/handle this on the assessment records proactively?
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Training Pathways:
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
- CCT
- CESR
- N/A - Yes for Core and Foundation Programmes
- CEGPR - No
- Any others? - No
16/09:(Ashley Ransoo)
- Curriculum > Leads to CCT field - This needs to be added to NDW. Needs tidying up.
- Following the tidying up, wherever the value is YES, it should default the Training pathway field to CCT, where No, then to N/A. - New ticket for this
Jira Legacy server System JIRA serverId 4c843cd5-e5a9-329d-ae88-66091fcfe3c7 key TISNEW-3394 Jira Legacy server System JIRA serverId 4c843cd5-e5a9-329d-ae88-66091fcfe3c7 key TISNEW-3389 - Bulk Upload - People Import to be extended (Add/Update) - Another ticket - N/A
04/11: Maria Kist (Unlicensed) asked a question about the other training pathways (...CCTi, CCTn etc. and whether those should be factored into the NTN generation?
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
- Review of how this works in the current ARCP return
- If the Curricula dates change how do we capture/handle this on the assessment records proactively?
- JH: So for the automate NTN ticket as part of refinement we need to consider how a current NTN is autogenerated but also how a future programme membership NTN is autogenerate
- JH: For current it is easy as just use the curricula as per today's date
- JH:For future I think it should be generate the NTN based upon the first day of the programme membership
14/10: For TIS: Ashley Ransoo update ticket with...
- If the curriculum attached to the Prog membership is amended, and there are assessment records for the attached curriculum and the Review date is within the Curriculum membership start and end dates, then it should update the TrainingNumberAtTime against the assessment records.
- It is not run as of today but as of the Curriculum dates.
- Recording of the historical NTNs not required for TIS.
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
14/10: Agreed
- If they are on F2 - they would have completed their academic component at the end of it.
- Kay to speak to other Foundation Teams.
24/10/2019 Raised by James Harris
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
24/10/2019 Raised by Kay Appleyard
Often the wrong programme name has been picked but it is not obvious as all we can see in the saved record is Orthodontics and not the region that it is attached to. This is an easy mistake to make but not so easy to detect. Showing the region code would highlight this as an obvious error. I did send this to the service desk a while ago so it may be on a to do list somewhere
When type and select from the drop down this is what you see:
But after you select, this is what you see:
Ade Added. To be transferred for next meeting and ticket yet to be created
SuppoRTT Data Programme: Project Plan
| View file | ||||
|---|---|---|---|---|
|