How to create a scrambled dump of Mesh data for sharing with new devs
This is a draft
For some tickets, it's helpful, if not essential, for developers to have a copy of Mesh data. However, we don't want to give away peoples' PII. To solve this, we've written a scramble script that can be used to "sterilize" an existing copy of the mesh data.
To acquire this data, follow the below steps:
- Log into the Mesh AWS Account
- Navigate to the
meshdb-backups
S3 bucket, and download a copy of a recent backup. - Use the following command to copy it into your running postgres container
docker cp <path to downloaded backup>/2025-02-19-214000.psql.bin meshdb-postgres-1:/
- Import the backup into your database
docker run -it meshdb-postgres-1 pg_restore -d meshdb -U meshdb -j 6 --verbose /2025-02-19-214000.psql.bin
- Be patient. This will take several minutes.
- Run meshdb and spot check the admin panel to ensure the restore worked well
- Use the scramble script to sanitize the data
$ python src/manage.py scramble_members
- Drop the users
- Dump the database and copy it back over to your host
`docker exec -it meshdb-postgres-1 pg_dump -d meshdb -U meshdb -Fc -f scrambled.psql.bin`
`docker cp meshdb-postgres-1:/scrambled.psql.bin ~/Downloads`