People - Scenarios | Bulk Imports
- Ashley Ransoo
- Former user (Deleted)
- Alistair Pringle (Unlicensed)
Page content
Scope
- Ability to bulk import people (trainees and non-trainees) via spreadsheet
- Ability to bulk import trainees from Oriel into TIS
As-Is User Interface for Oriel Import
As-Is Recruitment Import / Bulk Upload people
To- Be Bulk Upload/Update People - (trainees and non-trainees)
Process Name | PP6 - Bulk Upload/Update of People | Comments |
|---|---|---|
| Description | A user should be able to add a list of individuals to create multiple person records in one go or update existing records. | A link under 'People' meganav for Bulk Upload People is envisaged. For TIS, the only mandatory fields identified in the template are:
|
| Actors | Local office admin | |
| Pre-conditions | Logged in Authorised User | |
| Post-conditions |
| |
| Process steps |
| Existing person record on TIS can be matched with to update the record. The rules for finding an existing person record is Surname AND GMC/GDC/PH number match with an existing active Person record on TIS. |
| Alternative flow | PP5 - Create person (single) | |
| Rules | Error: this GMC/GDC/Public Health number is already held by a person with a different Surname on the database Error: mandatory fields not completed Error: Structure of the template expected by the import does not match, i.e. file format/column names/order expected do not match Error: Duplicate rows identified in the spreadsheet | Error message appears to let the operator know what the reasons for non import are. The errors should be corrected on the import spreadsheet before the import function is tried again. It is possible to select records to be imported if some of the records are correct but the best course of action is to correct all inaccuracies and import the entire spreadsheet in one go. The total number of records that have been imported is shown to the operator. Mandatory fields should be defined on the import spreadsheet to avoid the non import of records. Specification of Addition/Update/Matching Criteria Column headers: Bulk People Upload Validations:
Rules: 5. Column headers marked as 'Matching Criteria' are used to determine if the Person already exists on TIS and therefore Add/Update the rest of the specified fields. 6. Column headers marked with an asterisk ( * ), are mandatory. However, one of the registration numbers (GMC/GDC/PH) should also be provided and not marked as an asterisk. |
| JIRA Reference | TISDEV-2579 - Getting issue details... STATUS | |
| Audit Log | Who created the people record and when | Log changes to any of the field values and when and who made the changes. Visually presented to all |
To- Be Oriel Import of trainees to TIS scenario
Process Name | PP7 - Bulk Upload/Update of Oriel Trainees into TIS | Comments |
|---|---|---|
| Description | Automated ingestion process of trainees import should automatically map trainees to programmes, map reference data (e.g. immigration status, title etc.) along with non-reference data (e.g. Surname, Date of birth etc.) during the creation or update of the trainees records on TIS. | Trainees may already exist on TIS and therefore the existing records will be updated with the details at the point of upload. (Both from Oriel Stored data and Recruitment Import spreadsheet) |
| Actors | Local Office Admin | |
| Pre-conditions | Trainee records along with their documents are extracted daily from Oriel All data and documents exported by Oriel are stored within TIS on a Data Structure in Azure. The data and documents are updated in an import tracking table to make them queryable. The records can be uniquely identified by their Oriel ApplicantID’s. The stored Oriel data can be consumed during bulk upload of the Recruitment Import spreadsheet into TIS. | |
| Post-conditions | New trainees records created or existing trainee records updated Programme and curriculum membership details added to existing person record on TIS or created with a new person record Documents pulled from Oriel Azure Blob available on trainee records. | |
| Process steps |
| Proposed Business Process:
The Bulk People Import Template, when used to import Oriel sourced trainees, will have a limited data set: A: Applicant ID, Surname, GMC ID - sourced from an Oriel report Existing person record on TIS can be matched with to update the record. The rules for finding a matching person record are |
| Alternative flow | PP6-Bulk upload of People | |
| Rules |
| Where mandatory fields are not completed at time of import suggest a quality dashboard/data extract be produced to indicate the missing values in each person record for re-import. |
| JIRA Reference | TISDEV-3047 - Getting issue details... STATUS TISDEV-3459 - Getting issue details... STATUS | |
| Audit Log | Who created the people record and when | Log changes to any of the field values and when and who made the changes. Visually presented to all |
Discussion and assumptions
| # | Question | Comments | Owner |
|---|---|---|---|
| 1 | Where do you envisage the Oriel bulk import and Bulk upload people navigation to be? | A link under People meganav. | Joanne Watson (Unlicensed) |
| 2 | Note: The Oriel import in Intrepid isn't a template as it is a direct link from Oriel to V10. The user must select the successful applicants by programme and bring them into the V10 system. The new Oriel Import created in Intrepid v10.76 allows trainees to be directly imported from Oriel once they have had their offer accepted and the vacancy has been updated in Oriel to produce a “Workforce Extract” file. It is possible to put your new (or existing) trainees into programmes and placements via the new import interface but these are not field mapped as with the information below. | Oriel Import user guide - /wiki/spaces/TISDEV/pages/55738406 | |
| 3 | How do you envisage the page of the Oriel bulk import to be like before creating them on TIS and what fields should be made available on that page forr editing? Which of those should be mandarory? | The interface may not be needed at all if we manage to map programme details to the trainees in the oriel import such that as part of the Oriel integration the ingest is completely automated so applicant records are added straight into their programmes in TIS. The problem is that the applicant files do not have the programme numbers so we need to find a way of mapping them. Each programme has the Local office name against it and we have from Oriel most of the reference data from Hicom. Next step is to get a list of programmes for Data managers to tidy up to match the GMC approved list of programmes. (Sunil Rochani (Unlicensed) will extract the spreaddheet to send to them) and ask them to look at creating a spreadsheet per local office that maps the programme number we have against the data in the Oriel application files. Fields currently populated from oriel:
Fields that can be manually populated: Programme details:
Placement details:
| Joanne Watson (Unlicensed) |
| 4 | PP6 - What are the required validation rules for bulk upload of people? Which of these are MVP? |
Some rules could be:
| Joanne Watson (Unlicensed) |
| 5 | Some trainees may not have a specific ID, so if they are not this type of trainee (doctors, dentists, non-medical public health trainee), we assume they should have a type of ID, if no ID is included in the spreadsheet, they cannot be uploaded; however other types of trainee (healthcare scientists & pharmacists) type trainees can be uploaded without an ID. |
| Joanne Watson (Unlicensed) |
| 6 | Should we also allow bulk update of records via the bulk upload of people for records that already exist on the system? | Yes - GMC number is the unique identifier Foundation - Some trainees may not have GMC number, a second source of identifying the person would be the email address. | Joanne Watson (Unlicensed) |
| 7 | PP7 - What are the required validation rules for creating trainees using the Oriel import? Which of these are MVP? | As per #3: The interface may not be needed at all if we manage to map programme details to the trainees in the oriel import.
| Joanne Watson (Unlicensed) |
| 8 | Should TIS allow more than one trainees with the same GMC/GDC/PH number to be created within TIS? If not, then editing of 2 records on TIS to have the same GMC/GDC/PH number should also not be allowed. | No. Added validation rule for the second statement to the following story: TISDEV-2575 - Getting issue details... STATUS | Joanne Watson (Unlicensed) |
| 9 | Should the Oriel import user interface where you do the editing allow records/rows that pass validation to be created even though the lot may have rows that fail validation? I.e. either the whole import fails beacuse of 1 bad row Or allow good rows to be imported regardless and flag bad rows that needs to be corrected/re-imported. | Agreed that the rows that pass validation will upload successfully whilst the rows that failed to upload will be flagged to the user to be corrected for re-uploading. | |
| 10 | What are the filters that should be available on the Oriel import interface to facilitate the import, should it be necessary to have one? E.g. import by Local Office, or Programme. | As per #3. | Joanne Watson (Unlicensed) |
| 11 | Are the programmes and placements created on Intrepid ahead of the trainees being imported from Oriel? | So that the admin can then choose a Programme to import trainees against. Programmes and Posts should be on TIS ahead of the import. Terminology incorrect on the Oriel import UI - Placements won't be there but Posts will. | Alistair Pringle (Unlicensed)Joanne Watson (Unlicensed) No. You cannot add a Programme or placement to a trainee until they exist on the system. The Programme itself will already exist before the trainee is imported but this then allows you to create a Programme membership once the trainee is imported. AR: Alistair Pringle (Unlicensed) - I think there is a gap in the understanding of how trainees end up on Intrepid. I have found and reviewed the user guide (/wiki/spaces/TISDEV/pages/55738406) and on V10 "It is possible to put your new (or existing) trainees into programmes and placements via the new import interface". So my understanding is you can add the Programme and Placement details at the point of importing new trainees to the system. |
| 12 | Reference data from Oriel may not match reference data on Intrepid or TIS. Since going forward, reference data will have to map to ESR, we should therefore get Oriel reference data to be mapped to TIS reference data (which is aligned to ESR) when parsing the xml files. Is this a correct assumption to make? | When bi-directional flows in ESR is enabled, ESR will have to be the master. The reasoning behind this is in a scenario where records hava been updated in ESR and therefore updated on TIS but not Oriel. If Oriel then sends updates to TIS, it may not find the record to update. | Graham O'Regan (Unlicensed) |
| 13 | How will the Oriel import uniquely match trainees that are already on TIS to receive updates via Oriel import? Is there some sort of table that reference unique records on intrepid to unique records on Oriel? E.g <ApplicantID> from Oriel to <PersonID> from Intrepid? | Quoted from Intrepid "Trainee matches are established using their Surname and GMC Number values." This is true for one programme, but if an applicant has applied to a different programme in a subsequent year with a different Surname for e.g. what currently happens? Does a duplicate record gets created in Intrepid? SR: In TIS <Applicant ID> is uniquely matched to TIS ID for a person record. | Graham O'Regan (Unlicensed) Joanne Watson (Unlicensed) |
| 14 | Will the automated import/ingest of oriel trainees into TIS be tied up with their account/profile creation? I.e. People records created as well as their Account/Profile with potentially notifications sent out to them with their login details? Speaking to Joanne Watson (Unlicensed), Form R Part-A needs to be completed before they start training. This will take burden off Local Offices as well as a means to get trainees to fill in incomplete information on their profile. | Graham O'Regan (Unlicensed) | |
| 15 | On the data leads call on 04/12/2017, Claire/Alice mentioned that all distinct combinations of specialties/grade/start date for each LO would be required from Hicom/Oriel if we were to automate Oriel import. Digging further, firstly, Start date is not received in the Oriel XML but duration is. Secondly, there is not such a thing as a definitive list of programmes on Oriel by LO. When a vacancy is advertised, admins upload programmes (sub-preferences in Oriel) to the vacancy, as long as the values exist in Oriel reference data they are valid combinations, they may not conform to the GMC approved list of programmes. So not a definitive list. Also, although Oriel has a taxonomy, not every programme in Intrepid DR can be mapped to the Oriel taxonomy. The only straightforward way I see this would be possible is to have ProgrammeName/ProgrammeNumber on Oriel. How do we progress programme mapping if possible at all, thoughts? | Outcome of Meeting held on 05/12/2017: The following proposed as MVP process and currently being discussed with stakeholders. Oriel-TIS Integration recent and previous investigation summary Post-MVP: - Work with Oriel SML's to make changes to accomodate for the Programme details to be mapped | |
Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213