How to Import/Export CSV files of MeshDB data

The data in MeshDB is best viewed using the tooling available at db.nycmesh.net:

Admin Panel: https://db.nycmesh.net/admin
Legacy Query form: https://forms.nycmesh.net/query
SQL Explorer: https://db.nycmesh.net/explorer/play/

However, for some applications these resources may not provide all of the required query and filtering functionality. In these cases, users may want to export meshDB data for direct editing/analysis in spreadsheet editors or other local tools. This page summarizes the options available to do that.

CSV Export in MeshDB Format

To export individual MeshDB tables in the format used by MeshDB, such as the Members table, first open the MeshDB Admin Panel. Next select the table of interest, Members in this case:

On the next page, select "Export" to enter the export ~~dialog~~dialog:

[ REDACTED SCREENSHOT HERE ]

Finally, select "csv" as the output format, click "SUBMIT", and the data will be ~~download.~~downloaded:

Since we chose to export the data in MeshDB format, it does not include related columns such as install number or node number. These other columns would need to be downloaded separately from their respective tables an joined together with the id column on the member table. This is not trivial, so for common cases, like looking up a member's install by their email address, we provide CSV exports that combine these tables automatically, see below.

CSV Export in Legacy Spreadsheet Format

For convenience and compatibility with legacy scripts and workflows, MeshDB provides a CSV export capability which approximates the columns available in the "New node form responses" spreadsheet

The CSV exports provided in this section are an approximation of the legacy spreadsheet, but are not a perfect recreation. In particular, the notes fields do not match 1:1 with the original spreadsheet due to the way that the notes are parsed, split, and automatically created during the spreadsheet import process. Some smaller details like the exact install used for the `X-` pointer on NN re-use may also vary a bit. This query should not be used to replace the contents of the existing spreadsheet wholesale without careful consideration

To access this capability, use the following links, or navigate via the SQL Explorer homepage:

These queries can also be accessed by the following direct links:

Use the "Download" button and select "CSV" as the type:

CSV Import in MeshDB Format

These instructions cover the general process for importing a CSV into MeshDB, for a bulk ~~typo~~import ~~fix~~of partner data for example. For creating many installs in a large building, there is a more specific guide here

Editing large quantities of MeshDB data using CSV import can unintentionally have a large impact on the quality of MeshDB data. Though edits using this feature are supported, this process should primarily be used for imports of new data, rather than edits to existing data. For bulk edits to existing data, a Django Migration in the source code should probably be used instead

To import data to individual MeshDB tables in the format used by MeshDB, such as the Members table, first open the MeshDB Admin Panel. Next select the table of interest, Members in this case:

On the next page, select "Import" to enter the import dialog

[ REDACTED SCREENSHOT HERE ]

In the import dialog, select the CSV file you would like to import from (likely an edited export created using the instructions above):

On the following screen, confirm the accuracy of the data, including that the rows are showing as either "Update" or "Create" as expected based on the desired editing: