Migrating Mongo Data (2022)
It’s always nice taking your problems and making it someone else’s* so we’re moving to MongoDB’s Atlas DBaaS. As part of https://hee-tis.atlassian.net/browse/TIS21-3007 we have considered a number of methods for transferring data:
* when you have less expertise and would need to invest much more rather than working together
Atlas Live Migration
Pros
It can be used online (source dbs do not need to be offline). So live data can be migrated.
Cons
It is only UI driven.
TLS/SSL (Live Migration pre-req)
Generate a self-signed certificate openssl req -newkey rsa:4096 -x509 -sha512 -days 90 -nodes -out mongo.crt -keyout mongo.pem -subj "/C=GB/ST=England/L=London/O=Health Education England/OU=Trainee Information Systems
Deployment would require significant co-ordination and may require downtime if we were unable to add optional SSL with the nodes up.
Can we connect unencrypted to our cluster when SSL is enabled?… Yes.
According to Upgrade a Cluster to Use TLS/SSL , we can set an preferSSL
config option and leave requireSSL
inactive. This will enable our services to continue to connect and the Live Migration to connect.
Good Old Fashioned Downtime and Backup->Restore
Steps:
A bunch of prep work inc. updated config for the new clusters.
Stop ESR services
run the “one-liner” command to backup & restore See MongoDB website (avoiding the URI “gotcha”).
If using the --oplog flag is feasible it will reduce any downtime
Start services with the updated config
“Super Easy”, a number of caveats are given on the page
mongodump
can target individual databases reducing the scope of individual steps
Copies Secondary indicies
Runs from anywhere that mongo tools are installed #N.B. Achieved as a PoC by running a mongo container.
Will this do DocumentDB? Is it adaptable for moving clusters for services like Reval?
Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213