biommap/utils
nobohan 9520ee73b5 upd utils README 2021-09-22 10:37:06 +02:00
..
gis #17 Add possibility to import ARHEM haies remarquables 2021-09-15 11:46:55 +02:00
.gitignore import external data and qualification (#17): improvements 2021-09-15 10:57:31 +02:00
README.md upd utils README 2021-09-22 10:37:06 +02:00
geojson2sql.py geojson2sql: refine name of properties 2021-09-21 17:01:50 +02:00
lib64 add script for importing data 2021-09-13 16:38:41 +02:00
requirements.txt add script for importing data 2021-09-13 16:38:41 +02:00

README.md

utils

This folder contains some scripts for preparing some data for the municipality of Wasseiges.

Init

The scripts are python scripts that should run into a virtual environment.

First, install the local dependencies:

$ pip3 install -r requirements.txt
$ source bin/activate

Import of the layer Arbres remarquables

(venv) python3 geojson2sql.py

Some notes about this import

The import script creates database entries by filling not all the fields of the table. Only mandatory (non-nullable) fields and informative fields are filled.

Here is the column list and the corresponding values list for the table of sites

    column_list = ['id_site', 'uuid_sinp', 'id_program', 'name', 'geom', 'timestamp_create', 'id_type', 'obs_txt']
    values_list = ['__ID__', '__UUID__', '2', 'properties.SPFR', '__POINT__', '__NOW__', '1', 'géoportail wallon' ]

Some values in these list MUST follow some special conventions:

  • the "ID" value will be converted to a unique identifier for the site that will be created by adding 1000000 to the index of the geojson table.
  • the "UUID" value will be converted to a a formula that creates uuid value
  • the "POINT" value will be converted to a PostGIS ST_GeomFromText formula based on the coordinates of the geojson file
  • the "NOW" value will be converted to a formula that creates a timestamp

Furthermore, the id_program and id_type MUST match the correct values! The first field will decide which program the sites will be attached to.

Lastly, if the values starts with properties., it will be mapped to a property in the geojson. E.g, properties.COMMENTAIR.

For populating the table of visits, the same function is applied. But there is another special convention that applies for building the json details of the visit: the "JSON" value will be converted to a dedicated formula that will build the json object in the database.

Here is the column list and the corresponding values list for the table of visits

    column_list = ['id_site', 'json_data', 'timestamp_create', 'obs_txt']
    values_list = ['__ID__', '__JSON__', '__NOW__', 'import']

Populate the database

Simply execute the sql files onto the database to import the data.

Note that you may activate the uuid extension on the database the first time you use it (for using the function uuid_generate_v4()). This can be done by CREATE EXTENSION IF NOT EXISTS "uuid-ossp";