Page Comparison

How it works?

...

As of

Overview

In order to provide a list of doctors for revalidation administrators to work with, we rely on a nightly bulk update of all connected doctors from the GMC.

The Short Version

The GMC provide a SOAP web service GetDoctorsForDB (where DB stands for “Designated Body”) that returns the details of all doctors currently connected to a given designated body. This web service is used to collect data from the GMC and save it to a DocumentDB (where DB stands for “database”) database.

Any changes to data in the DocumentDB database are subsequently copied to Elasticsearch (AWS Opensearch) via a CDC (Change Data Capture) process. Elasticsearch is the technology that drives the list pages, providing super fast searching and filtering capabilities.

Further Reading:

• GMC (DocumentDB) Change Data Capture

How it works (The Longer Version)

1. The tis-revalidation-recommendation service (github link) is a Spring Boot application running as three load-balanced tasks in ECS. This service manages a scheduled job that kicks off the overnight sync process at midnight
   
   The cron expressions that determine the start time of the job are stored in Parameter Store for preprod and prod
   
   preprod: /tis/revalidation/preprod/recommendation/cron/nightlysync
   Prodprod: /tis/revalidation/prod/recommendation/cron/nightlysync
   

2. The recommendation service would start preparing the data for the GMC sync job. The doctors in the DoctorsForDB database have the field “existsInGmc” set to false. (Effectively this marks all doctors as “disconnected” as GMC only sends us connected doctors)
   After that, a message will be pushed to the Rabbit queue to notify the GMC-client service to start the overnight sync job.When the scheduled job in tis-revalidation-recommendation (see 1.) has started, the first operation is to effectively disconnect all doctors by removing their designated bodies and setting the flag existsInGmc to false. As GMC only send us connected doctors, this approach ensures that doctors that were disconnected externally show up as disconnected in our system.
   
   This has the effect of “hiding” all the doctors for recommendations and current connections, so doctors cannot be worked on at this time.
   
   Note 1: these changes will be propagated to Elasticsearch via the CDC process at this point.
   
   Note 2: The tis-revalidation-recommendation service serves double duty as the “revalidation doctor service” - this is an old architecture decision that hasn’t yet been seriously re-examined
   

3. Once all of the doctors in our system have been “disconnected” as described above, a message is published to the main reval exchange in our RabbitMQ instance (hosted using AmazonMQ). The routing details are as follows:

   Code Block
   exchange: reval.exchange queue: reval.queue.gmcsync.requested.gmcclient routingKey: reval.gmcsync.requested

   
   GMC-Client would send SOAP requests This message is consumed by the tis-gmc-client (github link) service, a Spring Boot application running as a single task in ECS.
   

4. The tis-gmc-client service then sends a SOAP request to GetDoctorsForDB to GMC for retrieving the list of doctors with their designated body.
   

5. GMC-Client would push GMC doctors' data to the RabbitMQ with routingKey reval.gmcsync which is binding to two queues, one for Recommendation and one for Connection service.
   

6. Recommendation service would listen to the queue reval.queue.gmcsync.recommendation for updating the MongoDB DoctorsForDB document.
   Doctors' data will be updated with their updated designated body. Any doctors who are not sent back from the queue, means they are not connected to any office. This would leave the disconnected doctors with a existsInGmc set to false, and their record will not be shown in recommendation summary page anymore (but will still show up in connections discrepancies).each Designated Body Code stored in the following Parameter Store locations for each environement:
   preprod: tis-revalidation-preprod-gmc-designated-bodies-codes
   prod: tis-revalidation-prod-gmc-designated-bodies-codes
   

7. The GMC’s GetDoctorsForDB endpoint returns a body of xml data containing the details for all the doctors currently connected to the given DB. This is processed into DTOs (Data Transfer Objects) and then the details of each doctor are individually published to be consumed to RabbitMQ with the following routing details:

   Code Block
   exchange: reval.exchange.gmcsync queue: reval.queue.gmcsync.recommendation routingKey: reval.gmcsync

8. At the same time, the Connection service would listen to The tis-revalidation-recommendation service consumes the messages from the queue reval.queue.gmcsync.connection for updating the Elastic Search Master index and the corresponding Connection indexes. Please find the document Elastic Search Index Update Event for Connections (outdated) for more information. At the time of writing, the connections application uses masterdoctorindex via an alias.recommendation and updates the doctor’s details stored in the DocumentDB database.
   
   There are several things to note at this point:

   1. Any doctor returned from GMC has the existsInGmc flag field set to true, and their designated body is updated.

   2. Any doctor not returned from GMC remains in the database but disconnected (i.e. null designated body and existsInGmc flag field set to false).

   3. The workflow fields that appear as TIS Status and GMC Status columns in the recommendations doctors list are both updated here. There is logic to “re-set” these status' when a doctor comes under notice once more.
      

9. As doctors are updated in the DocumentDB database, these changes are propagated to Elasticsearch via the via the CDC process.
   
   Note: this in of itself is a lengthy process, first the changes are propagated to the masterDoctorIndex, then they are published back to the recommendation service as the recommendation service manages its own elasticsearch index.
   
   Note 2: The connections list is backed by the masterDoctorIndex, so there is no dedicated connections architecture for this process
   

How to Trigger GMC Sync Manually

...

In case of overnight GMC sync failure, that might lead missing of doctors from the recommendation summary page and may block the daily operation of the Reval users. Instead of instead of waiting until the job re-run in the next mid-night, we can re-run the job manually to reduce the downtime.

NOTE (18/08/2023): The above is not necessarily advisable, as the sync job takes far to long to complete. Relevant Spike ticket here:

Jira Legacy
server System JIRA serverId 4c843cd5-e5a9-329d-ae88-66091fcfe3c7 key TIS21-4956

1. Login to the AWS console

2. Go to API Gateway

3. Choose the API revalidation-preprod-gateway for Preprod or revalidation-prod-gateway for Prod

4. Find the endpoint v1/admin/ under “Resources” and click on POST

5. Click Test to run the endpoint, then the overnight sync job will start

6. You will see a timeout error, this is fine

7. Logs can be checked from Recommendation service
   

Legacy

...

Stuff (To be deleted?)

...

GMC Sync Walkthrough: https://web.microsoftstream.com/video/adefa444-30e5-4748-8e97-991d12caad16

...