Skip to main content

How to create a scrambled dump of Mesh data for sharing with new devs

This is a draft

For some tickets, it's helpful, if not essential, for developers to have a copy of Mesh data. However, we don't want to give away peoples' PII. To solve this, we've written a scramble script that can be used to "sterilize" an existing copy of the mesh data.

To acquire this data, follow the below steps:

  1. Log into the Mesh AWS Account
  2. Navigate to the meshdb-backups S3 bucket, and download a copy of a recent backup.
  3. Use the following command to copy it into your running postgres container
    docker cp <path to downloaded backup>/2025-02-19-214000.psql.bin meshdb-postgres-1:/
  4. Import the backup into your database
    docker run -it meshdb-postgres-1 pg_restore -d meshdb -U meshdb -j 6 --verbose /2025-02-19-214000.psql.bin
  5. Be patient. This will take several minutes.
  6. Run meshdb and spot check the admin panel to ensure the restore worked well
  7. Use the scramble script to sanitize the data
    $ python src/manage.py scramble_members
  8. Drop the users
  9. Dump the database and copy it back over to your host
    `docker exec -it meshdb-postgres-1 pg_dump -d meshdb -U meshdb -Fc -f scrambled.psql.bin`
    `docker cp meshdb-postgres-1:/scrambled.psql.bin  ~/Downloads`