The below table is an extract of all the jobs with narratives of their purposes and what time they run in BST (during summer) and UTC (during winter) in the United Kingdom. The timeline below was validated on 26/07/2021.
NOTE: Timeline below showing all schedules in BST (UTC+1) unless specified otherwise and where they are triggered. Where the jobs are in UTC, their BST equivalent were added to the timeline. All schedules on this page are to be kept up-to-date by TIS devs & Ops and NDW team whenever there is a change.
...
Purpose and Timing of
...
Jobs
The below table is an extract of all the above jobs with narratives of their purposes and what time they run in BST (during summer) and UTC (during winter) in the United Kingdom. The table was last validated and updated on 2609/07/202102/2022.
No. | A. Job Name | B. HEE or NIMDTA | C. Purpose of the job | D. Job type | E. Start time (BST) | F. End Time (BST) | G. Start time (UTC) | H. End time (UTC) | I. Where notified on slack? | J. Add/Update Slack Alerts? -
| K. Notes | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Elasticsearch-snapshot-prod | HEE | ?? | Jenkins | 00:58 | 01:02 | 23:58 | 12:02 | #Jenkins_Azure | Yes | Check if this is still used John Simmons (Deactivated) / Liban Hirey (Unlicensed) | ||||||||
2 | GMC Synch Prod | HEE | Runs the GMC ETL process on prod. Updates the Revalidation and Concerns Elastic Search indexes with data received from GMC. Updates Profile Traineeprofile table with doctors TISIds and GMC numbers received from GMC. | Jenkins | 01:05 | 01:10 | 00:05 | 00:10 | #monitoring | ||||||||||
3 | Intrepid-Reval-ETL-all-prod | HEE | Runs the Reval ETL process on prod. It used to pull data from Intrepid but not anymore since TIS switchover. It enriches the Revalidation and Connection Discrepancies Elastic Search indexes with data from TCS and Assessments service for the TISIds. | Jenkins | 02:00 | 02:17 | 02:00 | 02:17 |
No Notification
Add slack notification to #monitoring?
#monitoring | Yes | ||||||||
4 | ndw-etl-prod | HEE | A spring-batch application which runs nightly to extract data from TIS, denormalise to the National Data Warehouse (NDW) schema, and write to TIS_NDW_INTEREIM_PROD interim database for the NDW PROD to import from.?? | AWS ECS | 03:00 | 03:30 | 02:00 | 02:30 | #monitoring |
- |
ndw |
#monitoring-prod
Move to ->
#monitoring-ndw
Yes | ||||||||
5 | ndw-etl-preprod | HEE | A spring-batch application which runs nightly to extract data from TIS, denormalise to the National Data Warehouse (NDW) schema, and write to TIS_NDW_INTERIM_STAGE interim database for the NDW UAT to import from.?? | AWS ECS | 03:40 | 04:35 | 02:40 | 03:35 |
#monitoring-ndw |
Yes | |||||||||||
6 | ndw-etl-nimdta | NIMDTA | A spring-batch application which runs nightly to extract data from TIS, denormalise to the National Data Warehouse (NDW) schema, and write to TIS Northern Ireland. (Interim database for NDW) | AWS ECS | 03:30 | 03:35 | 02:30 | 02:35 | #monitoring-ndw | New | |
7 | HEE NDW Merge ETL (UAT and PROD) | HEE | Runs the MSSQL job on NDW to merge TIS data to NDW databases.UAT then PROD. ?? | NDW MSSQL | 02:30 | 03:30 | 01:30 | 02:30 | N/A | Ask Pavel - James Harris |
NIMDTA NDW Merge ETL
NIMDTA
NDW MSSQL
8 | NDW-Hicom ETL (SQL07) | HEE | Populating TIS data (merged from the NDW Merge ETL earlier) to Intrepid database to send to Hicom. ?? | NDW MSSQL | 04:00 | Before 04:30 | 04:00 | Before 04:30 | N/A | ||
9 | HEE Person Owner Rebuild | HEE | Rebuilds Person Ownership table in TCS. | TIS SYNC | 01:05 | 01:05 | 00:05 | 00:05 | #monitoring-prod | ||
10 | NIMDTA Person Owner Rebuild | NIMDTA | Rebuilds Person Ownership table in TCS. | TIS SYNC | 01:05 | 01:05 | 00:05 | 00:05 | #monitoring-prod | New | |
11 | HEE PersonPlacement EmployingBodyTrust | HEE | Populates the PersonTrust table with the Employing body trust of the placement in TCS | TIS SYNC | 01:09 | 01:39 | 00:09 | 00:39 | #monitoring-prod | ||
12 | NIMDTA PersonPlacement EmployingBodyTrust | NIMDTA | Populates the PersonTrust table with the Employing body trust of the placement in TCS | TIS SYNC | 01:09 | 01:09 | 00:09 | 00:09 | #monitoring-prod | New | |
13 | HEE PersonPlacement TrainingBodyTrust | HEE | Populates the PersonTrust table with the Training body trust of the placement in TCS | TIS SYNC | 01:29 | 02:00 | 00:29 | 01:00 | #monitoring-prod | ||
14 | NIMDTA PersonPlacement TrainingBodyTrust | NIMDTA | Populates the PersonTrust table with the Training body trust of the placement in TCS | TIS SYNC | 01:29 | 01:29 | 00:29 | 00:29 | #monitoring-prod | New | |
15 | HEE Post EmployingBodyTrust | HEE | Populates the PostTrust table with the employing body in TCS | TIS SYNC | 02:09 | 02:10 | 01:09 | 01:10 | #monitoring-prod | ||
16 | NIMDTA Post EmployingBodyTrust | NIMDTA | Populates the PostTrust table with the employing body in TCS | TIS SYNC | 02:09 | 02:09 | 01:09 | 01:09 | #monitoring-prod | New | |
17 | HEE Post TrainingBodyTrust | HEE | Populates the PostTrust table with the Training body in TCS | TIS SYNC | 02:19 | 02:20 | 01:19 | 01:20 | #monitoring-prod | ||
18 | NIMDTA Post TrainingBodyTrust | NIMDTA | Populates the PostTrust table with the Training body in TCS | TIS SYNC | 02:19 | 02:19 | 01:19 | 01:19 | #monitoring-prod | ||
19 | HEE Person Record Status | HEE | Populates the Person record Status based on whether their programme membership has started/ended | TIS SYNC | 02:00 | 02:00 | 01:00 | 01:00 | #monitoring-prod | ||
20 | HEE Person Record Status | NIMDTA | Populates the Person record Status based on whether their programme membership has started/ended | TIS SYNC | 02:00 | 02:00 | 01:00 | 01:00 | #monitoring-prod | ||
21 | HEE Person Elasticsearch Synch | HEE | Populates the Elasticsearch clusters for Person? | TIS SYNC | 02:29 | 02:40 | 01:29 | 01:40 | #monitoring-prod | Yes | Update the Slack notification Person Sync → Person Elastic Search |
22 | NIMDTA Person Elasticsearch Synch | NIMDTA | Populates the Elasticsearch clusters for Person? | TIS SYNC | 02:29 | 02:29 | 01:29 | 01:29 | #monitoring-prod | Yes | Update the Slack notification Person Sync → Person Elastic Search |
23 | NHSE PostFunding Status Sync Job | NHSE | Updates Post Funding Status | TIS SYNC | 02:45 | 02:45 | 01:45 | 01:45 | #monitoring-prod | Yes | Slack notification-PostFundingStatusSyncJob |
24 | NIMDTA PostFunding Status Sync Job | NIMDTA | Updates Post Funding Status | TIS SYNC | 02:45 | 02:45 | 01:45 | 01:45 | #monitoring-prod | Yes | Slack notification-PostFundingStatusSyncJob |
25 | ESR Inbound Data Reader Notifier | HEE | ESR sends us |
RMC/F files. | RabbitMQ | real time | real time | real time | real time | #monitoring-esr | New Usually at 14:55 |
UK time from ESR | |||||||||||
26 | ESR Inbound Data Reader Notifier | HEE | ESR sends us APC files. | RabbitMQ | real time | real time | real time | real time | #monitoring-esr | New Usually at |
07:25 |
UK time from ESR | |||||||||||
27 | ESR Inbound Data Reader Notifier | HEE | ESR sends us DCC files. | RabbitMQ | real time | real time | real time | real time | #monitoring-esr | New Usually at 16:25 |
UK time from ESR | ||||
28 | ESRExporter Stage > PendingAppExport | HEE | Creates CSV files for APP Files and uploads to ESR SFTP | RabbitMQ/SFTP Sync |
12:00 |
12:10 |
11:00 |
11:10 | #sentry-esr | Yes | Add slack notification to #monitoring-esr. | |
29 | ESRExporter Stage > PendingNotificationExport | HEE | Creates CSV files for DNC Files and uploads to to ESR SFTP | RabbitMQ/SFTP Sync |
13:00 |
13:10 |
12:00 |
12:10 | #sentry-esr | Yes | Add slack notification #monitoring-esr | |
30 | ESRExporter Prod> PendingAppExport | HEE | Creates CSV files for APP Files and uploads to ESR SFTP | RabbitMQ/SFTP Sync |
12:00 |
12:10 |
11:00 |
11:10 | #sentry-esr | Yes | Add slack notification #monitoring-esr. | |
31 | ESRExporter Prod> PendingNotificationExport | HEE | Creates CSV files for DNC Files and uploads to to ESR SFTP | RabbitMQ/SFTP Sync |
13:00 |
13:10 |
12:00 |
12:10 | #sentry-esr | Yes | Add slack notification to #monitoring-esr | ||||||||
32 | ESRDataWriter Stage | HEE | Matches ESR Assignments and their Positions to TIS PersonID's and posts/placements. This then updates Personal details. | RabbitMQ | real time | real time | real time | real time | #sentry-esr | Yes | Add slack notification #monitoring-esr (combined/single) |
ESR don’t really do often do testing from their environment. | |||
33 | ESRDataWriter Prod | HEE | Matches ESR Assignments |
in Positions to TIS PersonID's |
in posts/placements. This then updates Personal & Contact details. | RabbitMQ | real time | real time | real time | real time | #sentry-esr | Yes | Add slack notification #monitoring-esr (combined/single) Usually start around 14:55 ( |
UK time) from ESR |
. Only structural issues triggered to Sentry in #sentry-esr | |||||||||||
34 | ESRReconciliation Stage | HEE | Matches ESR Positions to TIS Posts. | RabbitMQ | real time | real time | real time | real time | #sentry-esr | Yes | Add slack notification #monitoring-esr (combined/single) |
ESR don’t really do often do testing from their environment. | |||||||||||
35 | ESRReconciliation Prod | HEE | Matches ESR Positions to TIS Posts. | RabbitMQ | real time | real time | real time | real time | #sentry-esr | Yes | Add slack notification #monitoring-esr (combined/single) Usually start around 14:55 ( |
UK time) from ESR. |
36 | New Revel GMC Sync Prod | HEE | Cron job triggered by Tis-revalidation-recommendation service that runs the GMC ETL process on prod. Updates DoctorsForDB document in new Revel MongoDB and Elastic Search indexes for new Connections | RabbitMQ | 01:00 | 01:05 | 00:00 | 00:05 | #monitoring | ||||||||||
37 | TIS-SYNC SERVICE | HEE | TIS SYNC | 04:30 | 04:47 | #monitoring-prod |
|