Provoly Stream

The aim of this project is to contain all the modules that need to perform data transformations and convert them into Provoly items.

To do this, the data model, storage type and dataset definition must first be entered into Provoly. Transformed items inserted into topics will then be consumed by Provoly and inserted into the corresponding database, in the corresponding dataset.

The dataset definition must be MODIFIABLE or OPEN. It’s not possible yet to define CLOSED dataset with data as it comes in.

Steps

1. Define the data model for transformed items in Provoly

There are three types of storage in Provoly (ELASTIC,POSTGIS,KUZZLE), but not all of them are available. Make sure storage is available before.

Example :

{
    "id": "e62a08cb-485e-40f6-aa30-e04c1a001bb5",
    "name": "Enrich equipment",
    "storage": "KUZZLE"
    "attributes": [
        {
            "id": "f5c5c117-43c8-4d24-9d8f-fcf4a076855c",
            "technicalName": "family",
            "field": "802c64b3-8c16-4e92-ba8f-f51fb8cd9378",
        },
        {
            "id": "a31782be-c4e2-4c65-ac6f-211e1002706e",
            "technicalName": "code",
            "field": "d3640d94-1cda-4f2a-9365-6a88530fae64",
        },
        {
            "id": "cd9d6da7-39ff-4761-8412-5b933a25f92a",
            "technicalName": "consumption",
            "field": "c6096007-0cc6-43e5-920f-f8fe25724cf6",
        },
        {
            "id": "bd750fbe-a979-4bcb-b7f9-ca5eba2b14a5",
            "technicalName": "measuredAt",
            "field": "795e696f-1951-4837-8eaa-0a6f6bdd9ab8",

        }
    ]
}

2. Define the dataset definition in Provoly

Example :

{
"id": "b4e7cde4-a18e-4ae8-bd48-acf928c7330c",
"name": "Enrich equipment dataset",
"oClass": "e62a08cb-485e-40f6-aa30-e04c1a001bb5",
"type": "MODIFIABLE",
"groups": ["AUTHENTICATED"]
}

3. Retrieve the id of the generated dataset version

This id is necessary because it must be set in the configuration ( application.yaml) of the module that will perform the transformation