Contributing

This guide explains how to contribute to the project.

Joining the project team 

Communication

Communication within the team is done on :

a Mattermost workspace
daily standup meetings

The first thing to do when joining the project is to get in touch so we help you join these spaces.

Project management

We use :

a board of the tasks (issues)
work on issues is always done on feature branches which are merged into master after review

Front end and back end development 

Install

To install front end dependencies, execute:

make install-front

Run

To run the frontend for development purposes, open a terminal and execute:

make run-front

To run the backend for development purpose (with hot-reload), open another terminal and execute:

make run-back

During the first run, dependencies will be downloaded
You’ll need to install psql to interact with the Postgres database

The users for login are:

User with non super-user access: - username: “user” - password: “fish”
User with super-user access: - username: “superuser” - password: “fish”

Then, insert the GIS layers to the postgres database by executing (make sure you have psql installed):

./infra/remote/postgis_insert_layers.sh

Export the required environment variables:

export DB_HOST=db
export DB_NAME=monitorfishdb
export DB_SCHEMA=public
export DB_USER=postgres
export DB_PASSWORD=postgres

Finally add to Geoserver the layers by executing (make sure to remove your proxy if you have one configured with unset HTTP_PROXY and unset http_proxy):

./infra/init/geoserver_init_layers.sh

Test

To run all tests and checks clean architecture principles are respected, execute:

make test

To manually add a VMS position (in NAF format) for the vessel CABO ARTA:

curl --data '//SR//AD/FRA//FR/NLD//RD/20210431//NA/CABO ARTA//RT/2133//FS/NLD//RC/PCVC//XR/FG78//IR/XXX2545115//DA/20210431//TI/2130//LT/55.099//LG/3.869//SP/0//CO/173//TM/POS//ER//'  -X POST http://localhost:8880/api/v1/positions -H "Content-Type:application/text"

Lint

To lint backend code, execute from the backend folder:

./mvnw antrun:run@ktlint-format

Documentation

API documentation can be found at http://localhost:8880/swagger-ui.html

Frontend documentation can be generated with

jsdoc frontend/src/domain/ frontend/src/types/ frontend/src/api/ -r -P frontend/ -d docs

Problems & solutions

Windows & Docker

During the build, there is an error while seting up docker which is used for the test database
- Explanation : Testing uses docker. To be able to connect to docker, Intellij requires that TLS is disabled in docker.
- Solution : Open docker Configuration, General and click on “Expose Daemon on xxxx Witout TLS”
Running the application with IntelliJ does not work. Spring complains about a BuildProperties beans that is missing
- Explanation: The buildProperties bean is constructed based on the META-INF/build-info.properties file. This file is generated by Maven, but not by IntelliJ
- Solution: Tell IntelliJ to delegate the build to maven. To do this you must:
1. go to Build,Execution,Deployment> Build Tools > Maven > Runner
2. click the checkbox : “Delegate IDE Build/run actions to maven”

Data pipeline development and data science 

Data processing and ETL operations are done in python using prefect and pandas.

Data and data sources

Data is not open and can only be accessed from inside the RIE network. To work with data, you will need to either access the private development server or work with data extracts.

Install

The project runs on python 3. To manage python versions, we suggest you use pyenv. See the official documentation for installation instructions.
After installing python 3 (the exact version currently used can be found in the .python-version file), install development dependencies :
- If you do not have poetry, install it to manage python dependencies:
```
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 -
```
- Within the pipeline folder, install python dependencies :
```
poetry install
```
To install pre-commit hooks (isort, black and flake8 are configured in the pyproject.toml), run
```
poetry run pre-commit install
```

Test

To run tests on the data pipeline, run

make test-pipeline

Documentation 

Documentation is written as .rst source files, from which the documentation (this website) is built using sphinx and hosted on ReadTheDocs.

Writing documentation

To update the documentation, simply edit the .rst source file in pipeline/docs/source.
To build the documentation locally and see the result of your updates :
- you need to install the python development dependencies as described in Install if you have not yet done so
- build the html documentation with :
```
make build-docs-locally
```
This builds the documentation in pipeline/docs/build/html. You can view the built documentation in english and in french by opening en/index.html and fr/index.html in a web browser.

Translations

The documentation is translated with the recommended process of ReadTheDocs using Transifex.

Setting up Transifex

In order to translate documentation using Transifex, you must set it up :

you need to have transifex-client installed, which is part of the python development dependencies. If you have not yet done so, install development dependencies
ask us for an invitation to join our organization on Transifex
in your Transifex account, create an API token in User Setttings > API token
in pipeline/docs, run (with your API token instead of $TOKEN) :
```
tx init --token $TOKEN --no-interactive
```

Pushing documentation to Transifex for translation

After updating the .rst sources in english, if you wish to translate it :

push the documentation to Transifex with :
```
make push-docs-to-transifex
```
This does several things :
- cut the .rst sources into strings of uninterrupted text
- create a mapping of these strings that Transifex uses to keep translation synchronized
- push everything to Transifex
translate the corresponding strings in Transifex

Pulling the translated documentation from Transifex

When you are done translating in Transifex, you can pull the translated material back from Transifex with :

make pull-translated-docs-from-transifex

Translated material comes in the form of .po and .mo files in the locale directory, from which sphinx will look for translations during the build.

You can then build the documentation locally to check the result.

Updating the documentation online

The documentation is built by ReadTheDocs and hosted on ReadTheDocs. One the .rst source files and the translations are updated and pulled from Transifex, simply pushed the changes to master (better, create a branch and PR) and ReadTheDocs will build and update the documentation online automatically.

Contributing

Joining the project team

Communication

Project management

Front end and back end development

Install

Run

Test

Lint

Documentation

Problems & solutions

Windows & Docker

Data pipeline development and data science

Data and data sources

Install

Test

Documentation

Writing documentation

Translations

Setting up Transifex

Pushing documentation to Transifex for translation

Pulling the translated documentation from Transifex

Updating the documentation online

Joining the project team 

Front end and back end development 

Data pipeline development and data science 

Documentation 