Background
Outcome of this spike 09/01/2019:
What does a message bus do in lay-man terms?
- A publish-subscribe paradigm, services can subscribe to / Postman analogy
- A queue dealing with requests in a guaranteed way and provides traceability
- No blockages
What are the problems we are trying to address by the use of it?
- Duplication issue with bulk
- No retry at the moment, whereas if in a queue it can be re-tried
- Can monitor what message are being sent
What are the benefits/business value we will achieve by having this?
- Durability
- Less potential for re-work in case of failures
- Reduction of errors and duplication of work
- Less developer work to create a new consumer/service – does not require to write from start to finish however adds more complexity
- Kubernetes and Bus together – more resilience from a user perspective
- Devs do not need to know all the dependencies!
- More robust and reliable TIS
Relevant data
Where could and should this this be applied on TIS (defining the MVP and the MMF)?
- Possibly do this to Audit Logs as a first instance
- Then look at User management / Profile
- Educating the devs
- Resource to support this
- Cost consideration for the solution choice, impact on Azure subscriptions
- Cost vs Benefit
- Disaster recovery aspect
- We should consider the various technologies available and whether they have what we are looking for a TIS message bus? RabbitMQ, Kafka, WebSphere JMS etc. part of spike TISNEW-2401
- NDW – discuss with NDW people whether this could be done to NDW as well as opposed to ETLs.
- Sending to external sources – e.g. Hicom, means that we have to expose the BUS to third parties. API preferred way – No Bus.
Are we after an adaptive and incremental approach to exploring, defining, and implementing a message bus or its alternative?
- Yes
Do we need a Proof-of-concept to prove the above?
- Already carried out in Sept 2018 – The outcome was it is achievable, tested with 10K requests and all of them received. TISNEW-2402
Options considered
| Option 1: | Option 2: | |
|---|---|---|
| Description | Continue without a message bus | Implement a message bus |
| Pros and cons | POSITIVES No dev work required. NEGATIVES No queuing means potential for duplications being added, especially via bulk upload function. No traceability. No restartability. | POSITIVES Addresses all the negatives from Option #1. Future-proofs our development approach further. Could be rolled out elsewhere - e.g. in place of ETLs with the NDW team. NEGATIVES Is a not insignificant amount of development. Need to source the appropriate software to use. Unclear the overall cost of implementation (time and money). |
| Estimated cost | MEDIUM (Time and Effort) Problems with discovering and having to fix duplicates caused because of the lack of message bus. | LARGE (Time and Cost) |
| Status | IN PROGRESS | TO DO |
Action items
- Tickets for separate areas where this could be applied so that PO’s can prioritise – Add to Epic TISNEW-2396
- Use TISNEW-2401 to help with spiking and creating further tickets
Add Comment