Versions Compared

Old Version 8

changes.mady.by.user Doris.Wong

Saved on

compared with

New Version 9

changes.mady.by.user Cai Willis

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Known Issues:

04/07/2022 Post deployment review / release washup

Running the sync job

The reval sync job can be run manually from the API Gateway for prod and preprod at /api/sync.

...

  • tis-revalidation-integration (ECS)

  • tis-revalidation-recommendation (ECS)

  • tis-revalidation-connection (ECS)

  • TIS-TCS (EC2)

  • ElasticSearch (EC2)

  • RabbitMQ (Amazon MQ)

  • SQS (SQS)

Sync job data flow

  1. Image RemovedImage Added

    When /sync endpoint called in Integration service, the Master index is cleared and rebuilt

  2. “syncStart“ Message sent to TCS to start extracting data

    Code Block
    Rabbit Queue:  
  exchange: reval.exchange
  queue: reval.queue.connection.syncstart
  routingKey: reval.connection.syncstart

  3. TCS send trainee data with latest programme/curriculum information of the trainee one by one via the queue. “syncEnd“ signal will be sent together with the last trainee.

    Code Block
    Rabbit Queue:  
  exchange: reval.exchange
  queue: reval.queue.connection.syncdata
  routingKey: reval.connection.syncdata

    Integration service process to insert trainee data from TCS to the Master Index

  4. When the “syncEnd” message is received by Integration, “gmcSyncStart” message is sent to Recommendation to get GMC data

    Code Block
    Rabbit Queue:  
  exchange: reval.exchange
  queue: reval.queue.recommendation.syncstart
  routingKey: reval.recommendation.syncstart

  5. Recommendation get trainee data from “DoctorsForDB” and send it over to Integration one by one via the queue. “syncEnd“ signal will be sent together with the last trainee.

    Code Block
    SQS Queue:  
  Prod: tis-revalidation-sync-gmc-queue-prod
  Preprod: tis-revalidation-sync-gmc-queue-preprod

    Integration service process to insert/update (if exist) trainee data from Recommendaition to the Master Index

  6. When the “syncEnd” message is received by Integration, “getMaster” message is sent to Recommendation and Connection service at the same time to get data from Master Index

    Code Block
    Rabbit Queue:  
  exchange: reval.exchange
  Recommendation queue: reval.queue.indexrebuildgetmastercommand.requested.recommendation
  Connection queue: reval.queue.indexrebuildgetmastercommand.requested.connection
  routingKey: reval.indexrebuildgetmastercommand.requested

  7. Recommendation service cleans and rebuilds Recommendation Index, then maps and populates the trainee data to Recommendation Index
    Connection service cleans and rebuilds Connected, Disconnected, Exception Index, then maps and populates the trainee data to corresponding indexes according to business logic

...