2018-12-21 NDW ETL created duplicate entries on NDW Prod

Date	21 Dec 2018
Authors	Jayanta Saha, John Simmons (Deactivated), Chris Mills (Unlicensed)
Status	Resolved
Summary	TIS Prod → NDW Prod ETL created duplicate entries in the NDW D/B. This occurred as a recent code change resulted in the ETL running from both Blue and Green (which aren't synced, so caused dupes)
Impact	NDW Prod D/B had every record duplicated.

Impact

When the TIS Prod → NDW Prod ETL started, data from both Blue and Green was transferred, resulting in a duplicate of each record in NDW. All reporting then required some post-processing to account for removal of duplicates.

Recent TIS-DEVOPS deployment code changes which were kicking off the TIS-NDW-ETL running on both Blue and Green.

NDW databases were getting duplicates in different tables. NDW team, or one of their downstream users noticed the problem and alerted TIS on Slack #tis-ndw-etl channel.

.
20-12-2018 - NDW Team reported the duplicate issues and DEVOPS started looking at the docker logs and pull them in a file
21-12-2018 - Similar issues happened today and Dev team checked the docker logs, run queries to check if the row count matches in the TCS and NDW db tables. Row count did not match as expected. DEVOPs team checked the version of etls on both Stage and Prod and found they are same. Finally they checked the TIS-DEVOPS code base found the issue.

. To check docker logs use the following command docker logs ndw-etl_ndw-etl_1 --tail 3000