Date |
| ||||||||
Authors | |||||||||
Status | Documenting | ||||||||
Summary |
| ||||||||
Impact | Data in some of the TIS reports were stale and reflected what was on TIS from the last successful refresh |
| Table of Contents |
|---|
Non-technical Description
An issue was discovered (on 11th Oct 2023) where TIS Data Extracts' refresh were failing since 2 days prior. The impact of this was the data Data in some of the TIS reports were stale and reflected what was on TIS from the last successful refresh.
Trigger
...
Detection
Email notifications in Site Owner’s inbox
...
Resolution
Re-application of missing permisssions on the NDW service account used to refresh data extracts on Tableau Server.
Re-run refresh post-fix
...
Timeline
BST unless otherwise stated
09:42 Site Owner raises the alarm on the number of email notifications in own inbox about failed tableau refreshes on TIS PID Data site
10:15 - PO and team informed. Brief team discussion on priority and problem resolution delegation
11:00 - Quick analysis of number of data sources with failed refreshes and dates of last successful refresh established
11:56 - Tableau Administrator/Data Service team got notified
14:09 - Tableau Administrator/Data Service team confirms identification of the root cause of the problem
13:00 - Further analysis to establish scope/coverage of the problem and extent of stale data
07:21 - Requested progress update on permanent fix from Tableau Administrator/Data Service team
07:29 - Updated PO on state of affairs at the time
09:33 - Received confrmation of permanent fix in progres from Tableau Administrator/Data Service team
11:35 - Communication sent out on MS Teams channels to stakeholders
14:23 - Received notification of completion of permanent fix and verified data sources now refreshed
14:38 - Updated stakeholders on MS Teams channels
Root Cause(s)
Modifications made to the permissions on the NDW service account used to refresh data extracts on Tableau Server
...
Action Items
Action Items | Owner | |
|---|---|---|
Carry out analysis to establish scope/coverage of the problem and extent of stale data | ||
Liaise with dependent services to investigate the problem | ||
Implement permanent fix to problem | Tableau Administrator/Data Service team | |
...
Lessons Learned
We were fortunate that the Site Owner was working.
Possible update of distribution list to include more recipients of error/failure notifiations. Suggestions for potential inclusions are: PO, DM, Lead developer, analysts from both teams