Migrating Mongo Data (2022)

It’s always nice taking your problems and making it someone else’s* so we’re moving to MongoDB’s Atlas DBaaS. As part of https://hee-tis.atlassian.net/browse/TIS21-3007 we have considered a number of methods for transferring data:

* when you have less expertise and would need to invest much more rather than working together

Atlas Live Migration

Pros

It can be used online (source dbs do not need to be offline). So live data can be migrated.

Cons

It is only UI driven.

TLS/SSL (Live Migration pre-req)

Generate a self-signed certificate openssl req -newkey rsa:4096 -x509 -sha512 -days 90 -nodes -out mongo.crt -keyout mongo.pem -subj "/C=GB/ST=England/L=London/O=Health Education England/OU=Trainee Information Systems

Deployment would require significant co-ordination and may require downtime if we were unable to add optional SSL with the nodes up.

Can we connect unencrypted to our cluster when SSL is enabled?… Yes.

According to https://www.mongodb.com/docs/v4.0/tutorial/upgrade-cluster-to-ssl/ , we can set an preferSSL config option and leave requireSSL inactive. This will enable our services to continue to connect and the Live Migration to connect.

Good Old Fashioned Downtime and Backup->Restore

Steps:

1. A bunch of prep work inc. updated config for the new clusters.

2. Stop ESR services

3. run the “one-liner” command to backup & restore See MongoDB website (avoiding the URI “gotcha”).

   1. If using the --oplog flag is feasible it will reduce any downtime

4. Start services with the updated config

“Super Easy”, a number of caveats are given on the page

mongodump can target individual databases reducing the scope of individual steps

Copies Secondary indicies

Runs from anywhere that mongo tools are installed #N.B. Achieved as a PoC by running a mongo container.

Will this do DocumentDB? Is it adaptable for moving clusters for services like Reval?