How to Import/Export CSV files of MeshDB data
The data in MeshDB is best viewed using the tooling available at db.nycmesh.net:
- Admin Panel: https://db.nycmesh.net/admin
- Legacy Query form: https://forms.nycmesh.net/query
- SQL Explorer: https://db.nycmesh.net/explorer/play/
However, for some applications these resources may not provide all of the required query and filtering functionality. In these cases, users may want to export meshDB data for direct editing/analysis in spreadsheet editors or other local tools. This page summarizes the options available to do that.
CSV Export in MeshDB Format
To export individual MeshDB tables in the format used by MeshDB, such as the Members
table, first open the MeshDB Admin Panel. Next select the table of interest, Members
in this case:
On the next page, select "Export" to enter the export dialog
Finally, select "csv" as the output format, click "SUBMIT", and the data will be download. Since we chose to export the data in MeshDB format, it does not include related columns such as install number or node number. These other columns would need to be downloaded separately from their respective tables an joined together with the id
column on the member table. This is not trivial, so for common cases, like looking up a member's install by their email address, we provide CSV exports that combine these tables automatically, see below.
CSV Export in Legacy Spreadsheet Format
For convenience and compatibility with legacy scripts and workflows, MeshDB provides a CSV export capability which approximates the columns available in the "New node form responses" spreadsheet
The CSV exports provided in this section are an approximation of the legacy spreadsheet, but are not a perfect recreation. In particular, the notes fields do not match 1:1 with the original spreadsheet due to the way that the notes are parsed, split, and automatically created during the spreadsheet import process. Some smaller details like the exact install used for the `X-` pointer on NN re-use may also vary a bit. This query should not be used to replace the contents of the existing spreadsheet wholesale without careful consideration
To access this capability, use the following links, or navigate via the SQL Explorer homepage:
These queries can also be accessed by the following direct links:
- Export Legacy Spreadsheet Installs/Nodes Tab
- Export Legacy Spreadsheet Links Tab
- Export Legacy Spreadsheet Sectors Tab
Use the "Download" button and select "CSV" as the type:
CSV Import in MeshDB Format
These instructions cover the general process for importing a CSV into MeshDB, for a bulk typo fix for example. For creating many installs in a large building, there is a more specific guide here
Editing large quantities of MeshDB data using CSV import can unintentionally have a large impact on the quality of MeshDB data. Though edits using this feature are supported, this process should primarily be used for imports of new data, rather than edits to existing data. For bulk edits to existing data, a Django Migration in the source code should probably be used instead
To import data to individual MeshDB tables in the format used by MeshDB, such as the Members
table, first open the MeshDB Admin Panel. Next select the table of interest, Members
in this case:
On the next page, select "Import" to enter the import dialog
In the import dialog, select the CSV file you would like to import from, likely an edited export