Streamlining – ESR/TIS interface – Briefing Note

Introduction

The purpose of this short note is to provide a briefing on issues arising from the interfacing/data exchange between the NHS’s National Electronic Staff Record system (ESR) and Health Education England’s Trainee Information System (TIS). To do this the paper will outline the background and history of work to date and identify issues relating to this work. The paper seeks advice and direction in relation to the significant risks associated with the identified issues.

Background

ESR is a payroll and HR system used by Trusts throughout the NHS. 470 organisations in England and Wales are currently using the system. It is used to get junior doctors on payroll and manage training and development. The primary functions of the ESR Interface to TIS are;

• Reconcile the Inbound Positions from ESR against Posts in TIS to make sure that both systems match.
• Send Placements from TIS to ESR for future posts, this includes sending new starters who have never been placed before.
• Receive Trainee profile updates from ESR to TIS,
• Send Placements updates from TIS to ESR where changes occur after they've been sent to ESR.

The ESR solution is based on Oracle's e-Business suite R12 and has been in place since 2006 providing almost 12 years of great benefits to the NHS. It is currently integrated with Wales Deanery Intrepid solution delivered by Hicom, Empower solution delivered by Northgate and the Trainee Information System (TIS) solution delivered by Health Education England for the purpose of transferring junior doctors placement information.

Whilst the benefits derived from the integration have been extensive to the NHS Employer organisations, it is limited to what it can provide when compared to modern interfaces in many respects. We recognise that the historic framework contracts ESR has had are ineffable and makes it difficult to react into current practice.  

TIS Programme and ESR work to date

HEE has used the Intrepid system for approximately 20 years which is supplied by Hicom. Each Local office (13 of them grouped into 4 regions), used Intrepid individually built up separate implementation and databases.

As well as the core Intrepid system, Hicom developed and supplied several modules to HEE Local Offices and covers features like Placement Management, Leave Management, Case Management and Course Management. The total cost of ownership per year for HEE was in the region of £1.6m.

The TIS programme was put together to:

• develop an in-house team within HEE to build a replacement for Intrepid
• develop a national system providing consistency across local offices
• give HEE the capability to replace the modules
• provide the ability to continue to develop the product in a use-centred approach and meeting the business needs.

• As part of the TIS solution, a replacement for the Intrepid-ESR interface was developed to carry on the automatic transfer of junior doctor placement between TIS and ESR to avoid a downgrade in the service when switching to TIS. The Minimum Viable Product (MVP) included exchanging trainee placements, changes to their placements by way of notifications to ESR and data quality reports generated where gaps in the data are identified. This is also known as the one-directional flow. The TIS-ESR interface involves exchanging of files generated by either parties in a sequential order every evening via scheduled jobs similar to the Intrepid-ESR interface. The specification provided to TIS to integrate with ESR was largely one that was developed between Hicom, ESR and IBM. This method is known to have several issues.

There is a future requirement to build a bi-directional link to exchange updates of trainees personal details to feed into TIS. Whilst TIS masters training information, ESR is deemed to master the trainees’ details once they start employment.

The bi-directional flow that existed between ESR-Intrepid has initially had HEE Local Offices faced with several data quality issues regarding their trainees’ details updated inaccurately on Intrepid. As a result, most of the Local offices switched off the bi-directional flow or accepting a bare minimum set of data to be updated with on Intrepid.  

TIS has started scoping the dataset that needs to be included in a bi-directional flow for TIS if one is to be built. It is currently being analysed with the various systems in use by HEE where trainees have self-service access to. Given the numerous issues we have encountered with the current method; it is not deemed sustainable to build onto it a bi-directional flow using a similar approach to Intrepid.

ESR Interfaces

There are basically 3 interfaces on ESR through which the above are received or sent:

• Deanery Update Interface/Deanery Reconciliation and Medical Trainee Outbound Interface
• Deanery Inbound Recruitment Interface
• Generic Inbound Notification Interface

Key Issues

The use of the phrase “system interfaces” is not technically correct. Modern systems exhibit clear technical interfaces which allow an on-demand connection and provide a documented interface specification [Chris – can you put in a para about API’s with ref/footnotes).

The current “interface” between TIS and ESR is essentially and file transfer and import into the various data storage arrangements. It is this approach which generates issues in two key areas:

Data transfer approach

Data management within receiving systems

The current approach requires the import of data from a delimited file. This is essentially a text file with a record per line with the end of record indicated by a marker (a comma or pipe symbol).

In addition to the technical limitations of the approach explained and also linked above, the below are more specific issues we have encountered on the interface:

• Issues where markers are either missing or use of other special characters in the file rendering the whole of it invalid and therefore not processed
• National Post Numbers not matching the format that TIS expects and use of special characters breaking the processing of the file
• Limitation of ESR to handle retrospective reference data that would have been correct at the time of recording on TIS but not acceptable to ESR as they only expect a specific version of the reference data.
• Limitation of ESR to handle trainee placements information beyond 3 months’ future time projected hire date
• The exchange of notifications,e. changes to placements subsequent to them being transferred to ESR being inadequate. The changes to placements are not exchanged in real time but relies on changes audited for the day to be built and sent in a file at the end of the day. The changes are limited to only 5 different types of notifications that ESR can infer. E.g. a medical rotation is due to take place against a post, an applicant has withdrawn/replaced from a post etc.
• If for some reason, there is an outage, it involves a lot of manual work by an experienced individual to process the backlog of files to bring the interface back up and sync the data
• The time at which the files are currently exchanged are out of normal working hours, which means if there are issues, they are not noticed until the next day

Associated risks

The implication of the above issues are detrimental to data quality in many ways:

• A file being unprocessed means trainees placements are not transferred to ESR on time to follow the code of practice and have a direct impact on trainees payroll due to delays in record update
• National Post Numbers mismatch, has a similar implication as the above unless noticed on time
• Changes to placements not transferred in real time but at the end of the day, means out of date information are sent to ESR, again with a similar implication to the above
• Inferring reference data on trainees supplied information rather than being able to retrospectively maintain them by ESR, creates inaccuracies in reporting and would create inaccuracies if a future bi-directional flow is to be built
• Over-reliance on a 3 months’ future start date rule, means ESR does not have the full picture of oncoming trainees for the posts beyond 3 months’ time
• Manual interfering and fixing backlog of unprocessed files are prone to human error and therefore impacting on data quality
• The time taken to bring the data back in synch in an event of an outage (e.g. due to network issues), means that there could be further inaccuracies created by missing applicants which were due to be transferred during that period
• In addition to the delays caused by aforementioned issues, there is also an added delay by the ESR queue processing of files, normally 24 hours plus

Timeliness – the current approach is essentially a “bulk” import and export of data this means that data are processed on a daily/weekly basis, with an inherent “lag” in the systems (the time taken to export and import data) of 24 hours plus, this carries a clear number of usage risks.

Managing risks and issues

As detailed above the bi-directional “interface” has a number of significant challenges.

Due to the age and contractual arrangement for ESR approaches to date have been forced down the technical route detailed above, this puts particular pressure on the TIS development team as there is more flexibility in the technology in use on that side.

The team can increase the levels of data validation (filtering/checking), however the underlying issues will remain.

Conclusion

There are significant risks and issues in this area of work that are detailed above.

The mitigation of these risks will require significant person resource investment by the TIS team.

The team require guidance as to how much of this work should be conducted in the knowledge that the process will be error-prone.

As the team are unaware of the future development program for ESR the above question is one that the TIS team cannot address.

R.Hill, A.Ransoo, C.Mills – 7/12/18