...
- Route all other traffic through Green
Change ESR ETL to allow a parameter to be passed which provides a specific date for the notification load so that they can be picked up retrospectively if necessary (rather than just using the current system date)
Kick off the notification load routine on Blue
- Wait for process to finish before re-enabling Blue for other traffic
Detection / Timeline
- 0200 0600 7th May: ESR ETL started.
- 0600 7th May: ESR ETL found to have failed (at some point)
- 0900 7th May: ESR ETL (notification-daily-load) Started again
- 1039 7th May: ESR ETL found to have failed (at some point)
- 1300 7th May: ESR ETL (notification-daily-load) Started again
- 1500 7th May: All jobs jobs disabled from 8pm that evening to allow notification-daily-load opportunity to run
- 2000 7th May: ESR ETL (notification-daily-load) Started again (to give largest window before other processes start)
- 0400 8th May: ESR ETL failed (TCS GC overhead)
- 0900 8th May: Hard memory limit removed from TCS
- 1400 9th May: ESR-ETL code change to allow parameter to be passed to specify the date of the notifications (instead of the default of LocalDate.now())
- 1500 9th May: ESR ETL started
- 1730 10th May: ESR ETL completed
...