2023-01 Elastic Search Index(es) Quantity and Shape

Problems:
Same work as recommendations assumption, we SHOULD do the same

• CDC and the "separate indexes"

• Level of change: we could do minimal updates to what is implemented now

• “Logic” Implementing in different languages / at different times

  • Principal: Do it once!

  • Logic in writing? "just the mapping"? Mapping is logic... still have 2 processes

  • Logic

• Filters would need to be duplicated if using different indexes:

  • Implementation-dependent

• Hypothesis: Performance of querying a single index vs. separate indexes

• Hypothesis: Performance of writing to several indexes
  

• Hypothesis: It would all be much simpler if we were copying all programme memberships separately

• What level of mapping is required between the base (master index) and connections?

  • Next to none... probably not actually necessary

Principals:

• Write it (code) once - Questions over this. How much does this run counter to Microservice Design

• Unnecessary redundancy. Data stored in several places.

  • When should we have copies of the data?:

Solutions:

• Assumption:

• Fields available in the Base (Master) Index similar enough to the fields needed across all connections screens.

• Data Design

  • Steve's query on schema design

  • When do we Specifically do we need to persist:

    • connection status field, given we have gmcDesignatedBody

    • Programme Memberships & Whether a discrepancy is hidden

Three ways of caching data, the first might not be viable because of connection specific info:

• Single Index across REVAL? (Probably not, given the likely need of holding connection specific info?)

• Single Index for Connections

• Single Index for each tab