Data alignment
Page Content:
Working group
@James Harris , @Alistair Pringle (Unlicensed) , @Ashley Ransoo, @Mark.Oliver (Unlicensed) from NIMDTA
GP Data - V9
GP data for Northern Ireland is currently on V9 unlike the rest which is on V10 of Intrepid. Link to the Data extract:
Hicom confirmed the work would start around end of August 2020, probably taking around 2 weeks.
Working group: Mark, 2 members from NI GP team, TIS.
GP V9 Field | Area/Grouping | NIMDTA/TIS Comments | Migrate to V10? Y/N | Field Exists on TIS? Y/N |
---|---|---|---|---|
Name | GP Practice | A record for the practice should already exist on the “site” reference table. | Y | Y |
no email address against “site” on V10 | N | N | ||
Practice Manager | for v10, PM would be added as a contact and linked to the site. Cannot be linked to site on TIS though – may not be much value in bringing across on that basis? 13/05 TIS: Workaround in Local Offices using Local table. Will be looked at in the future if there is a need. | N | N | |
GP Practice VTS | no VTS field against “site” – Trust field can be used ~5 VTS for NI 13/05: Sites are linked to Trusts in TIS. | Y | Y | |
GP Practice Area | no Practice Area field against “site” – Trust field can be used 13/05: Same as VTS usually with some exceptions with schemes. Sites mapped to Trusts. | Y | Y | |
Record Status |
|
|
| |
Trainees Allowed | There is a “post count” against the “site” record on V10 13/05: They are available in National Data Warehouse. (NDW) | N | N | |
GP Practice Type – Postgraduate | this may transfer to V10, but I do not believe it can go to TIS | N | N | |
GP Practice Type – Undergraduate | this may transfer to V10, but I do not believe it can go to TIS | N | N | |
GP Practice Type – Foundation | this may transfer to V10, but I do not believe it can go to TIS | N | N | |
GP Practice Type – Retainer | this may transfer to V10, but I do not believe it can go to TIS 13/05: Sites>Posts>Programmes on TIS | N | N | |
GP Practice Type – Returner | this may transfer to V10, but I do not believe it can go to TIS | N | N | |
GP Practice Type – Research | this may transfer to V10, but I do not believe it can go to TIS | N | N | |
Posts | GP Practice > Posts | based upon separate “GP Posts” table which is not used – not required | N | N |
GPs | GP Practice > GPs | this may transfer to V10, but I do not believe linking the trainers, to practices / sites within TIS is possible. Most that could be done may be recording Practice Name and Address details as the GP’s address against their new “Contact” record on V10. 13/05: Post per site, placement, then CS/ES supervisors linked to placements on TIS. Data can be made available in NDW if migrated to V10. | Y | N |
Notes | GP Practice > Notes | 13/05: Make available in NDW. In TIS Posts > Training description? Posts are linked to Sites. Depending on the types of notes available, decide where to put those. | Y | N |
GPs | GPs | GP would be added as a new “contact” person record on V10, in the “Educational Supervisor – GP” role | Y | Y |
Surname | GPs > GP Details |
| Y | Y |
GP practice | GPs > GP Practice Details | Practice and address details may transfer to V10, but I do not believe linking the trainers, to practices / sites within TIS is possible. Most that could be done may be recording Practice Name and Address details as the GP’s address against their new “Contact” record on V10. | Y | Y |
Practice Manager | for v10, PM would be added as a contact and linked to the site. Cannot be linked to site on TIS though – may not be much value in bringing across on that basis? 13/05: Make available in NDW | Y | N | |
Practice email | for v10, PM would be added as a contact and linked to the site. Cannot be linked to site on TIS though – may not be much value in bringing across on that basis? 13/05: Make available in NDW | Y | N | |
Notes | GP > Notes | 13/05: In TIS Person > Comments | Y | Y |
Trainer | GP > Types | this may transfer to V10, but no corresponding role on TIS 13/05: Roles against Person. ES/CS/LeaveManager actively used with a number of others. Consider using existing ones on TIS. | Y | Y |
Foundation Trainer | this may transfer to V10, but no corresponding role on TIS 13/05: Roles against Person. ES/CS/LeaveManager actively used with a number of others. Consider using existing ones on TIS. | Y | Y | |
Trainer Group Convener | this may transfer to V10, but no corresponding role on TIS 13/05: Roles against Person. ES/CS/LeaveManager actively used with a number of others. Consider using existing ones on TIS. | Y | Y | |
OOH Supervisor | this may transfer to V10, but no corresponding role on TIS 13/05: Roles against Person. ES/CS/LeaveManager actively used with a number of others. Consider using existing ones on TIS. | Y | Y | |
Educational Supervisor | Will transfer to V10 and TIS 13/05: Roles against Person. ES/CS/LeaveManager actively used with a number of others. Consider using existing ones on TIS. | Y | Y | |
TPD | this may transfer to V10, but no corresponding role on TIS 13/05: We have Programme Observer role on TIS assigned to TPDs to have access to their programmes. | Y | Y | |
GP Appraiser | this may transfer to V10, but no corresponding role on TIS |
| Y | |
GP > Contact Details | V10 has 2 fields for email addresses, but TIS has only one. A decision will need to be made in relation to which address to store against the trainer record. 13/05: Make all email addresses available in NDW. Ideally, all persons to have a ‘hscni’ email address to be used on TIS. TIS normally takes the ones from Oriel. | Y | Y | |
Prev. no. trainees appointed | GP > Trainees | do not believe there is a corresponding field in V10 13/05: Make available in NDW reports. | Y | N |
Current no. trainees appoint’d | do not believe there is a corresponding field in V10 13/05: Make available in NDW reports. | Y | N | |
Total no. trainees appointed | do not believe there is a corresponding field in V10 13/05: Make available in NDW reports. | Y | N | |
Record Added By | GP > History | an audit trail exists but may not be exactly the same | Y | N |
Record Added Location | an audit trail exists but may not be exactly the same | Y | N | |
Record Added Date Added | an audit trail exists but may not be exactly the same 13/05: On TIS we have Placements audits (Added/Amended). | Y | N | |
Last Amended By | an audit trail exists but may not be exactly the same | Y | N | |
Last Amended Location | an audit trail exists but may not be exactly the same | Y | N | |
Last Amended Date changed | an audit trail exists but may not be exactly the same | Y | N | |
Staff Post Records | Staff / Staff Post Records | 13/05: On TIS can be recorded against deferral posts Reports can be generated in NDW for deferred start dates. | Y | N |
GP Trainee | Staff > GP Post | used to identify trainees who are out of sync 27/05: Identification of Out of Sync trainees - Currently achieved via a "GP Trainee" drop-down field on Intrepid V9. Out of sync is a particular area for concern as this is an important field in relation to reporting requirements. We can Create reports in NDW to pick that up. Programme membership dates is used to identify out-of-sync trainees on TIS. Combination of Reporting and Deferral posts could be used. Deferral posts created on TIS to record those trainees. Out-of-sync classification (anything that affects the Programme dates):
An additional column to the report to flag as ‘out-of-sync' based on a number of reasons to identify the ‘out-of-sync category/reason’ in relevant columns. Look at the V10 field we receive from Hicom for this. Might need to map to VTS on TIS and add comments as a short term workaround when we migrate. @Mark.Oliver (Unlicensed) - circulate report. | Y | N |
VTS code | identifies the VTS Area / scheme that the trainee is in. Separate instances of the GP Programme are set up on V10 and the trainees V10 programme memberships may need to be adjusted so that they are in the correct instance of the programme based on this value. 27/05: Look at the V10 field we receive from Hicom for this. Might need to map to VTS on TIS and add comments as a short term workaround when we migrate. | Y | N | |
Programme Director | Staff > Training Specialty | identifies the trainees Programme Director – should move to V10, not to TIS 13/05: Can be brought as a Programme Observer on TIS so they can access their programmes. 27/05: Linking of Training Programme Directors to individual trainees On TIS ES and CS can be linked to trainees via placements. TPD’s aren’t linked to Programmes on TIS at the moment but on our roadmap for future development. When closer to development to look at what would work for all. More investigation required before moving to possible solutions. | Y | N |
GP Trainee | used to identify trainees who are out of sync | Y | N | |
GP Trainer | Staff Post > Trainer | Identifies the trainee’s supervisor during their placement - should map to the Educational Supervisor fields against placement record on V10 13/05: Should map across to corresponding V10 field. | Y | Y |
College number | Posts | 27/05: Both these fields should contain the same information? Can the ‘Post family’ field be used on TIS to record this? @Mark.Oliver (Unlicensed) to investigate if exists on V10 and what is actually recorded/whether different from Local Post No. | Y | N |
Local post no | 27/05: Both these fields should contain the same information ‘Local Post no’ exists on TIS ‘Training description’ is another field on TIS used and not standardised. | Y | Y |
Merge Rules
We are reiterating the need for Northern Ireland to check their data quality ahead of the proposed merge with TIS. The merge rules suggested below appears in the last column. The data quality has to be sorted as a priority ahead of any development work to merge and bring across to TIS. The merge rules we’ve considered look at:
Identification of duplicates
How do we determine the updated record and therefore bring across to TIS
How to determine which records we would want to overwrite etc. or ignore completely
The suggested overarching order is to look at merging and bringing across in the following order:
Sites > Trusts >
Reference Data > Curricula>
Programmes > Posts>
People >Programme memberships> Placements >Assessments>
Look at Supervisors (ES/CS), whether this can be done separate or part of people
and then Full Copy of related records (e.g. assessments etc.)
Merge Rules used at consolidation previously with Intrepid
After a person record merge, if a GMC/GDC/PH number is specified, that is the value that should be assigned to the GMC/GDC/PH field of the surviving record. In the case of blanks, the current value (whatever that they be) of the surviving record is retained.
All associated records (placements, programme/curriculum memberships, ARCPs) belonging to a duplicate person record, regardless of whether they are different to Surviving record, need to be transferred to the surviving person record.
Where the surviving person record has a field containing a NULL or Zero-Length-Strings value but the same field in the duplicate person record contains a non-NULL/ZLS value, this value should be copied into the surviving person record. This should exclude address fields due to possible town/county/postcode mismatches.
https://hee-tis.atlassian.net/secure/attachment/21792/TIS%20FrontEnd%20Fields.docx - list of front end fields where values can be taken from duplicates to surviving where there is a blank in the surviving record. Note, I have removed the GMC, GDC and PH number fields as they are subject to some special rules already stated above.
Sensitive Data - Keep sensitive data from the person ID's in column A
Use the address that are against the survivor records.
Documents: Pull all the docs from the Duplicate across to the Survivor records regardless of whether these are duplicates.
Note:
During testing on TIS Stage environment, any Data added to Intrepid would not be copied over to TIS, unless we re-run script to pull data.
NDW to take snapshots as required before testing commence
Intrepid V10 Data
Item No. | Hicom DR views | Comments/Discussions | Actions/Next steps | Merge Rules |
---|---|---|---|---|
05/12/2019 - Kick off meeting | ||||
1 | vwAccredetationEvidence, vwAccreditationPathway | Not in use |
|
|
2 | vwAssessment | Follow the same rules which were used for consolidation |
|
|
3 | vwCourseAttendance | V9 but unsure if in use. Most recent data is from 2014 |
|
|
4 | VwCurriculum | The year to be removed from the curriculum names |
|
|
5 | VwCurriculumGrade |
|
|
|
6 | VwcurriculumMembership | Follow the same rules which were used for consolidation |
|
|
7 | EmployerReports | Pre-employment checks done on behalf of trusts in NI |
|
|
8 | StudyLeave | Not in use |
|
|
9 | Ext prefixed Tables (External)
|
|
|
|
10 | FormR | Not required |
|
|
11 | vwGPVisit | Not required |
|
|
12 | vwGrade |
|
|
|
13 | vwNTN |
|
|
|
14 | vwPerson |
| Total Person records = 6644 UNKNOWN GMC = 439 NULL GMC = 591 WITH GMC = 5614
|
|
15 | VwPlacement and also applied to Post, Programme |
|
|
|
12/12/2019 - Session 2 | ||||
16 | vwPersonExam | Related to GP Exams etc. not used in NI |
|
|
17 | vwPersonQualification | Data pulled from Oriel, which is already on TIS |
|
|
18 | vwPlacementSupervisor |
|
|
|
19 | vwPost | Ensure there are no duplicates and overlap with TIS posts |
|
|
20 | vwPostFunding | Not recorded for NI |
|
|
21 | vwPostFundingComponent | Not recorded for NI |
|
|
22 | vwPostOtherGrade |
|
|
|
23 | vwProgramme | Programme Code: ensure no duplicates and overlap with TIS before copying over |
|
|
24 | vwProgrammeCurriculum |
|
|
|
25 | vwProgrammeNTN |
|
|
|
26 | VwProgrammePost
| Rotations on TIS the moment and only used by London |
|
|
27 | ProgrammeRotationMembership ProgrammeRotationPost |
|
|
|
28 | vwProgrammeSchool |
|
|
|
29 | vwProgrammeSupervisor |
|
|
|
30 | VwQuestionnaire | Not in use |
|
|
31 | vwRevalidationEpisode | Not in use |
|
|
32 | vwSecurityDataProfileMembership |
|
|
|
33 | vwSite |
|
|
|
34 | Vwsitetype | Is it used to indicate Practices? Based on placement specialty |
|
|
35 | Specialty |
|
|
|
36 | vwTrust |
|
|
|
|
|
|
|
|
Questions and Assumptions
No. | Question/Assumption | Owner |
---|---|---|
1. | Permissions to update Reference tables for NI
NI 12/02:
| TIS |
2. | Pre-employment checks - No table available?
12/02:
| Mark (NI)/TIS |
3. | GP data from V9
12/02: In progress/review with NI. | Mark (NI) |
4. | No use of ESR in NI |
|
5. | NI Sites are not on TIS except for 2 of them which we could find. NI should align their Sites to ODS, i.e. the Site codes should be aligned, Inactivate unused values etc.
| Mark (NI) |
6. | Does NI have CCG’s? 12/02:
| Mark (NI) |
7. | Are the Intrepid IDs unique across various implementations of Intrepid in Hicom? I.e. no overlap with the Intrepid ID’s used against HEE TIS data?
| Mark/Hicom |
8. | Where the trainees already exist on TIS and is also a trainee within the NI dataset, should NI be able to see their HEE programme memberships, placements, rotations, assessments, documents etc. and vice-versa? 12/02: Yes. Given I am a NI TIS Admin Given I have a relevant HEE Admin role | IG/Data Protection/TIS/NI Data Sharing Agreement and TIS DPIA |
9. | Observation: There are no trainees valid PH numbers in NI dataset.
|
|
10. | Ownership of data - where the ownership cannot be determined, then both HEE and NI would be able to see the record. |
|
11. | ‘Leaving Reasons’ consolidated list from TIS - send to @Mark.Oliver (Unlicensed)
|
|
12. | TIS is currently looking at building/Fixing Reval - Engage NI |
|
13. | Pre-registration Pharmacy programmes and Healthcare Scientists programmes are managed under separate schemes in Northern Ireland and not under NIMDTA. |
|
14. | There will be about 40-50 Admin users for NIMDTA. |
|
|
|
|
|
|
|
|
|
|
Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213