Date | |||||||||
Authors | Yafang DengJoseph (Pepe) Kelly catherine.odukale (Unlicensed) Fiaz Adam (Deactivated) Rob Pink Fiaz Adam (Deactivated) | ||||||||
Status | DocumentingDone | ||||||||
Summary |
| ||||||||
Impact | One file was shown “IN PROGRESS” and bulk upload was not picking up the following files any more. Those other files were delayed until we changed the status of the blocking file. They were then processed. |
Non-technical Description
When files are uploaded on to the bulk upload page, the files are uploaded to some storage and the jobs are recorded with the status “PENDING“. Then the bulk upload service starts to pick up the first file, and set its status to “IN PROGRESS“, leaving all other following files “PENDING“. Only when the current file is marked with “COMPLETED“, the following “PENDING“ ones will get processed.
We had a problem with Tis, the system was not doing Bulk upload whichFiles that were uploaded with a bulk upload, which is when more than a few files are sent to be uploaded, was not being completed. This is likely due to a timeout issue, the reason is because of the limits of connection between the Bulk Upload service and a system named TCS.
...
Trigger
A file of person update template with 1180 records was uploaded.
On prod, the id (Epoch time) of the file is: 1674667043653 and the record id in
genericupload.ApplicationTypeis 28781.
...
Action Items | Comments | Owner | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| |||||||||
Find the limit of Person Update before we hit the timeouts and apply the limit on the Generic upload service Add a message on UI to inform users not to exceed the limit | catherine.odukale (Unlicensed)
| |||||||||
Generic upload sends “keep-alive“ packets to keep connection |
|
Lessons Learned
When we find out that there’re logs showing the data have already been updated, it’s good to have a quick comparison on current data and data from spreadsheet. So Generic Upload service can be resumed more quickly.