Legacy Data Migration Job

The migration ETL is written in Spring Batch in tis-revalidation-migration service.

It consists of an ItemRreader to read data from the legacy Reval MySQL database, an ItemProcessor to map the trainee data into the format we need in the new Reval, and an ItemWriter to insert the data into new Reval MongoDB.

How to run:

1. Clear/drop the revalidation.recommendation collection in the Reval MongoDB

2. The migration will start automatically once the service runs.
   To run the task manually:
   i. Find the subnet id for the AZ the database is in, e.g. 
   export SUBNET_ID=`aws ec2 describe-subnets --output text --query "Subnets[*].SubnetId" --filters Name=tag:Name,Values=tis_preprod_private_*2a`
   ii. Find the security group to run with, e.g. 
   export SECURITY_GROUP=`aws ec2 describe-security-groups --filters Name=group-name,Values=tis_preprod* --output text --query "SecurityGroups[*].GroupId"`
   iii. Now run the task, e.g.
   aws ecs run-task --task-definition tis-revalidation-migration --cluster revalidation-preprod --network-configuration "awsvpcConfiguration={subnets=[$SUBNET_ID],securityGroups=[$SECURITY_GROUP]}"

3. Data will be populated to the revalidation.recommendation collection in the Reval MongoDB after migration.