Page Content
Background
The Oriel Workforce Extract is a file based (XML) export from Oriel of Trainee details, the programmes they've been accepted on along with documents submitted with their Oriel application. There is currently an ETL that receives those files and documents from Oriel and stores them on Azure. (check if this is still running). This extract includes Trainees that are new to TIS (e.g. Foundation trainees) and those applying for Higher Specialties. The former result in creation of new trainee records on TIS whilst the latter result in additional Programme memberships and curriculum memberships being added onto their existing TIS record.
Previously, the process on Intrepid has been been to input the curriculum and programme details via Intrepid at the point of importing them into Intrepid to create their records. The reason for this is because post and programme details are not captured on Oriel and therefore unable to do a direct mapping to the Intrepid system when transferring the Offer Accepted Oriel Applicants.
A piece of work started to look at how to automatically ingest trainees into TIS when Intrepid is retired as a system. For MVP, it was initially proposed to have a semi-automated ingestion of some of the Oriel trainee details whilst HEE Admins perform a bulk upload of trainees to TIS. We went live in April/May 2018 with a bulk upload solution as an MVP, but this did not include the semi-automated transfer of Oriel details from the Oriel Workforce Extract files. We are now reviving this piece of work in order to deliver a potential full automation of extracting and creating trainees records from Oriel to TIS. In order to achieve this there are potential changes to be done on both TIS and Oriel sides.
In December 2023, a piece of work was started by the Business Analyst and The data team to Understand the process followed in each local office for onboarding new PGDiTs from Oriel to TIS. This piece of work is to achieve an understanding of the processes followed by each local office so that all perspectives can be considered when deciding solution options for the semi-automated ingestion method for Oriel trainee details into TIS and bulk upload by HEE Admins, ensuring alignment with local office practices.
(Previous discovery) Oriel Workforce Extract on Azure
/wiki/spaces/TISDEV/pages/59051715
N3 Location
TIS uses the following share location for Oriel:
Service | Server | Share |
---|---|---|
Oriel | 10.196.16.50 | C:\WorkforceExtract |
(Previous discovery) MVP - Proposed Business Process to Import Trainees from Oriel into TIS
26/08/2020: The below process was not fully achieved and excluded the semi-auto ingestion of Oriel trainee details from Azure extracted Oriel Files. A bulk upload tool was developed for MVP for uploading Oriel trainees which requires pre-population from Oriel Reporting tool or in some Local Office’s what is known as the Rotor tool to transform/map the Oriel data along with Reference data (e.g. Nationality, sexual Orientation etc.) to what TIS can successfully upload.
Current Work Contacts
TIS Contacts
Kavitha Shankar- Data Analyst - kavitha.shankar1@nhs.net
Stan Ewenike - Data Analyst - stan.ewenike@nhs.net
James Harris - Data Lead - james.harris39@nhs.net
Adewale Adekoya - Business Analyst - adewale.adekoya@nhs.net
Ashley Ransoo - Business Analyst - ashley.ransoo@nhs.net
East and West Midlands
Lauren Squires - Regional Programme Information Coordinator - Lauren.squires1@nhs.net
Yorkshire and Humber
Kay Appleyard (Main contact) - Information Manager - kay.appleyard2@nhs.net
Philip Balcomb - Data Officer - philip.balcomb@nhs.net
Sarah Hill - sarah.hill131@nhs.net
North West
Liam Lofthouse (Main contact)- Information Manger - liam.lofthouse1@nhs.net
Jane Lovelady - jane.lovelady1@nhs.net
North East
Leane Lowe - Lead Employer Trust (Senior People Services)- leanne.lowe@nhs.net
South West
Louise Peckham - Education Programme Manager – Secondary Care Medicine, IMT, & Medical Ophthalmology- louise.peckham@nhs.net
London
Stuart Morris - Service Delivery Manager- stuart.morris@nhs.net
Sarah Krause - Senior BI Analyst - sarah.krause@nhs.net
Wessex/Thames Valley
Maxine Grout - Thames Valley - Maxine.Grout@nhs.net
Jemma Martell - Wessex - Jemma.martell@nhs.net
(Current Work) High Level Activity Flow
(Current Work) Analysis of the current Process
Activity | E&W Midlands | Y&H | NW | NE | SW | London | TV/Wessex | |
Download Records with relevant documents; who performs this role? | The recruitment Team | The recruitment Team | The recruitment Team | HR/Lead Employer Team runs/extract Oriel Custom Report of trainees after Offer deadline | The recruitment Team depending on specialty IDs supplied by the programme Teams | The recruitment Team | ||
Download Records with relevant documents; How many people in the LO has got access to Oriel to the full full report? | One person in the Team has got access | One person in the Team has got access | One person in the Team has got access | One person in the Team has got access | One person in the Team has got access | One person in the Team has got access | ||
Download Records with relevant documents; How often is the download made | The download is weekly to check any new record | The download is weekly to check any new record | The download is weekly to check any new record and inform the Data and Programme Team | Download from Oriel and upload into TIS are not weekly, it happens when it needs to happen. They constantly check Oriel for new starter and upload them into TIS until post is full. | There is no Weekly extract from Oriel to Upload into TIS in this region but a big upload. There might be situations where some programmes’ recruitment might still be on even after deadline, then a report is pulled after the deadline of such extra recruitment. | The Download is automated in London | ||
Download Records with relevant documents; Any Document downloaded with Report? | No document downloaded with report | No document downloaded with report but the TPD do download the following which differs from school to school and documents are saved to the trainee file
| No document downloaded with report | No document | No document downloaded with report | No document downloaded with report | ||
Run/Extract reports ; Type of reports? | One full report generated. Oriel Custom Report which contains information around equality, diversity, ethnicity, gender, disability etc ……….. Still to ask for more information | 4 reports generated yearly:
| One Full report is generated excluding the foundation doctor | The downloads; Core, Higher, academic, Foundation, Dentist depending on the national timeline. Academic Round- Academics trainees are first pulled then Core trainees in round 1(Academic could be a bit of Round 1 because of clinical interviews), round 2 will be higher trainees and so on depending on the timeline. | Core, Higher and academic. No Foundation and GP | Reports generated by the recruitment team includes Foundation Doctors Records which follows the same process | ||
Run/Extract reports ; What format? | CSV | CSV | CSV | CSV | CSV | XIsx | ||
Run/Extract reports ; Where is the report saved? | SharePoint | SharePoint- The Data Analyst save in SharePoint the extracts | SharePoint | SharePoint | SharePoint | SharePoint | ||
Mapping of reports Who does the mapping? | The LO Admin | Data Analyst - The recruitment Team notify the Data analyst moment the reports are downloaded from Oriel | Data Officer | HR/Lead Employer Team | Programme Team | Data Focus Admins for each School/Specialty in HET. | ||
Mapping of reports What activities is done in the mapping? | 1. Manually remove person’s record without GMC number>fills in the recruitment Import template with Oriel received trainee details including ApplicantIDs, Surname and GMC AND with fields that do not come from Oriel e.g Programme and curriculum details> Using the People Bulk Upload | Check if necessary data are in the report> If contain necessary data then run a VBA process to arrange columns and merge files.which will be a Mirror of the 4 report> If there are missing GMC number, create a separate file for person’s record without reference number (GMC/GDC) in Oriel and follow the process when GMC number is received>Add record to the Export Data file which can be accessed by all the member of the Data Team who will follow the next part of the process>Add the Programme and curricula details. For more complicated area e.g ACCS or Academics who are already on TIS, the Specialist Data Analysts add the programme and curricula>Check out missing data e.g. address line 2, gender fields etc>Create a file to be uploaded to TIS from the Import sheet which is unto populated>Uploads the files with the required details to TIS using the People/People Updates bulk upload function in TIS> other specialised Data Analyst then go into TIS to add placement. If foundation records, TPM get updated emails of Foundation trainees in late August time from Trainee and then update TIS using People Bulk update template | The Recruitment Team notify the Data Officer and the Programme Team (but sensitive data are made hidden for the programme Team)> Data Officer set up the recruitment template> Align template with Oriel report by using series of formulae and look up tables> Data Officer import raw Oriel data into recruitment template which converts these raw into a TIS bulk upload template > If no missing data, import data to stage to test and fix errors> but if missing data, missing data is highlighted in the bulk template> Hide sensitive data and lock password to protect> save in SharePoint> Each school/programme review and confirm data > Data Officer upload to TIS> Notify Programme/School record is created > Programme/School bulk add placement (this could be manual).
| Manually Map/Match the custom Reports with the new starter template which is a mirror of the TIS People Bulk Upload Template; 1) Manually Add programme and curriculum details (which includes start and end dates) to the template 2) Carry out pre-employment Check on Trainees before uploaded on Trac (using a different template) which is the pre-employment system 3) Complete Occupational Health template and upload to Occupational Health system> After 1 - Manually Add programme and curriculum details (which includes start and end dates) to the template, Check for missing GMC/GDC> if missing GMC, Ask Trainee or go to GMC connect to get number and if not supplied, GMC/GDC number is not available, it goes into TIS as UNKNOWN. Most of the time to meet CoP deadline. Some international Trainees might not have GMC/GDC number> Split the Template into different specialties and upload each into TIS> after record is created in TIS, Lead Employer Team/HR get the Trainee Rotations from TPDs/specialties by send template without sensitive data to TPDs (Lead Employer Team/HR hide sensitivity data from TPDs. TPDs gets names, interview ranks and email)> Lead Employer Team/HR Update the placement template with the rotations received from TPDs> Lead Employer Team/HR Bulk upload Placement into TIS> Lead Employer Team/HR ask the Programme team to check and confirm the grades | After the recruitment Admin saved the report in the common SharePoint, the programme Teamm filter on GMC number to check missing GMC number> if missing, email trainee to send number within a deadline> if not submitted withing deadline, programme check GMC connect> Update Oriel report with number> Programme Team amend on the Oriel report the field’s language that are not matching the TIS fields language e.g. Ethnic origin, Gender, Sexuality, Religion, Nationality etc AND add Programme and curriculum details for each trainee on the Oriel report>Programme team transfer the Oriel report into the TIS People bulk upload template> upload to TIS> After record is created, Programme Admin/ Specialties Bulk upload Placement into TIS after the people bulk upload is done | After report is saved is auto saved into specialty SharePoint> Preprogramme and Team Data Focus Admin in HET Check data correctness. If second run duplicates is removed> Update with programme and curriculum and other missing data like GMC/GDC Number> Request Missing GMC/GDC number from Trainees> if not received, Update Spreadsheet with ‘UNKNOWN’ or ‘0’> Data Focus Admins for each School/Specialty uploads the spreadsheet to TIS using the People/People Updates bulk upload function into TIS>Programme Admin Bulk upload Placement into TIS | ||
Mapping of reports What template is use? | Template used; Oriel Custom Report - Sent by the recruitment Team to the Admin, Recruitment Import Template/TIS Bulk upload template - use by Admin to upload into TIS and which is also splitted into different programme team. | This uses a series of formulas and look up tables, which should hopefully make things a little easier to understand | ||||||
Mapping of reports What software/app is used? | No identified software Note:
| Software - VBA/sql script /snag to identify anomalies | his uses a series of formulas and look up tables, which should hopefully make things a little easier to understand Note
| No app/software | ||||
Bulk Upload data Person Record to TIS Who Upload to TIS | Admin | Data Analyst | Data Officer | Lead Employer Team/HR | Data Focus Admins for each School/Specialty in HET | |||
Bulk Upload data Person Record to TIS What Template? | Use the People Bulk Upload Template/recruitment Import template | Export Sheet/Import Sheets | Recruitment Template/TIS Bulk Template | TIS Bulk Template | People Uploads Bulk Upload Template | |||
Bulk Upload/Upload Placement Who add placement | Programme Admin- Each Programme Admins are informed by the LO Admin when their records are created in TIS. Admin share individual folder with the programme Admin. The Programme Admin inform the TPDs of who is joining their programme, upload the placement/rotation into TIS | Data Analyst | Programme teams/Schools | Lead Employer Team/HR | Programme Admin/ Specialties manager |
| ||
Bulk Upload/Upload Placement What Template? | Bulk Placement template | Manually Add placement when trainee records are created | Manual/ Bulk placement template | TIS bulk placement template | Bulk placement template | TIS bulk placement template Note: GP manually upload placement | ||
Is it the same process for Foundation and other specialties? | No not really any difference between foundation, GP, academic or the rest of rounds 1 and 2 in terms of process – just different timelines (as the recruitment rounds happen at different stages) and the local team that is directed to that information. UKFPO and GPNRO do provide some reports for foundation and GP recruitment directly to local teams, but it is often not enough for them to work off of, so they typically wait for our process even if the national recruitment offices sent in information earlier. |
|
| Yes, follow the same process | No Contacts: Foundation; MADDOCK, Suzanne (NHS ENGLAND - T1510) suzanne.maddock2@nhs.net GP; SLOMAN, Richard (NHS ENGLAND - T1510) richard.sloman@nhs.net | Yes Reports generated by the recruitment team includes Foundation Doctors Records which follows the same process | ||
Pains/Challenges | Lot of manual mapping of data to suit TIS Language |
| Pain Point
| Pain Point
| Pain Point
| Pain Points
| ||
Other Comments |
|
|
|
|
|
|
(Current Work) Options
# | Option | Description | Impact | Decision | Comments |
---|---|---|---|---|---|
Do Nothing | |||||
(Current Work) Discussion and Assumptions
# | Question | Comment | Owner | Jira Ticket |
---|---|---|---|---|
1 | Mapping and Analysing LOs process | This ticket was raised by the TIS data team. Mapping sessions was carried out by the BA and the TIS data Team with each LO | Data Lead |
(Previous discovery) Discussion and Assumptions
# | Question | Comment | Owner | Jira Ticket |
---|---|---|---|---|
1 | In the workbook on sub heading 3, there are fields that are reference data from Oriel and require further data mapping to be done. | Agreed that wherever possible we will adopt Oriel ref tables to avoid having differences and having to map the data. Ashley to check impact on ESR interface to determine which tables we can do this for. Agreed that nationality will have to align with ESR as data passes through interface so this must be mapped. | Data Leads | |
2 | In the workbook on sub heading 3, there are fields that are not necessarily mandatory on Oriel and would therefore only pull through if the applicant/trainee has provided the information on their application form, e.g. National Insurance number, Religious belief, Sexual Orientation etc. | Recommendation is that if completed in Oriel that it is pulled through into TIS. | Data Leads | |
3 | In the workbook on sub heading 3, there are fields that are on the Recruitment Import template but not Oriel. Which of these fields are required in the template and where would these be sourced from? Which of these should be mandatory? | Feedback received from Claire and Alice as to which one to keep on the template and which one to remove. Draft version consolidated and added to dev story. Concern over mandatory fields in the template. E.g. DOB may not be available for non-trainees. | Data Leads | |
4 | Workbook on sub heading 4 was put together with input provided by Claire Lehoux. Although separate from Reference data mapping, there still need to have legacy data tidied up in the DR. Not related to the Oriel integration but for data quality in general. | Data Leads | ||
5 | Workbook on sub heading 5 has a list of Immigration statuses discrepancies that requires mapping. Oriel reference data for 2017 and 2018 have different list of values that will be passed on to TIS. | Agreed not to map and just take Oriel values as Intrepid data is not accurate and up to date. Regarding the 2 new questions added for immigration on 2018 application form in Oriel, the recommendation is to record these as 2 new fields on TIS.
Oriel to fix their extract to send those values. | Data Leads | |
6 | Reference data changes on Oriel will need to be coordinated with TIS going forward. | Data Leads | ||
7 | Custom report can be built on Oriel to extract a list of trainees for specific programme/s where they have accepted an offer or withdrawn after accepting. This task is to:
| Ashley and Claire met to discuss and Recruitment lead is currently working on this. Concern raised over Dental trainees. They are expected to use the agreed approach in order to import trainees from Oriel into TIS. Ashley - To speak to Joanne Watson (Unlicensed) Alistair Pringle (Unlicensed), to add channel on Microsoft Teams to get feedback over: - Done
| Data Leads | |
8 | Documents will be be extracted from Oriel and stored on TIS in a data structure on Azure to be consumed by the Bulk Upload function during trainee records creation. | For info only. | Data Leads | |
9 | The Recruitment Import template may need to be changed to have new columns or some columns removed if not needed. For e.g. ApplicantID may need to be added to accomodate for the above Process. Some of the columns in the current Recruitment import template in its current format, will overlap with the fields that are received from Oriel. Do we want to keep them in the template? Which of the overlapped fields would be best to be manually filled? | ApplicantID will be added. - Done Next steps are: SImilar action to #3. ApplicantID = ORIEL PIN | Data Leads | |
10 | The Recruitment Import template will be used both for creating and updating records on TIS. In a scenarion where records already eixist on TIS, what should be the best course of action by the Upload function? For e.g. should all or a selection of fields on TIS be updated/overwritten? | The template will be used with the relevant fields populated so that existing person record on TIS can be matched with to update the record. The rules for finding a matching person record are If an existing person exists on TIS, then update with populated values from the spreadsheet and pull Oriel stored data from Azure storage to update the person record with programme and curriculum details. | Data Leads | |
11 | Oriel XML file format |
(Current Work) Jira Tickets
Summary | Jira Ticket | |
---|---|---|
1 | Understanding process followed in local offices for onboarding new PGDiTs from Oriel to TIS |
Add Comment