documentation for installation process

This commit is contained in:
Julien Fastré 2024-05-03 13:16:51 +02:00
parent 394c752fc3
commit 1230b7f4c0
Signed by: julienfastre
GPG Key ID: BDE2190974723FCB

View File

@ -14,273 +14,305 @@
Installation & Usage Installation & Usage
#################### ####################
.. toctree::
:maxdepth: 2
prod.rst
load-addresses.rst
prod-calendar-sms-sending.rst
msgraph-configure.rst
You will learn here how to install a new symfony project with chill, and configure it.
Requirements Requirements
************ ============
- This project use `docker <https://docker.com>`_ to be run. As a developer, use `docker-compose <https://docs.docker.com/compose/overview/>`_ to bootstrap a dev environment in a glance. You do not need any other dependencies ; The installation is tested on Debian-like linux distribution. The installation on other operating systems is not documented.
- Make is used to automate scripts.
Installation You have to install the following tools on your computer:
************
If you plan to run chill in production: - `PHP <https://www.php.net/>`_, version 8.3+, with the following extensions: pdo_pgsql, intl, mbstring, zip, bcmath, exif, sockets, redis, ast, gd;
- `composer <https://getcomposer.org/>`_;
- `symfony cli <https://symfony.com/download>`_;
- `node, we encourage you to use nvm to configure the correct version <https://github.com/nvm-sh/nvm>`_. The project contains an
:code:`.nvmrc` file which selects automatically the required version of node (if present).
- `yarn <https://classic.yarnpkg.com/lang/en/docs/install/>`_. We use the version 1.22+ for now.
- `docker and the plugin compose <https://docker.com>`_ to run the database
1. install it locally first, and check if everything is ok on your local machine; Chill needs a redis server and a postgresql database, and some stuffes like a "relatorio service" which will
2. once ready, build the image from your local machine, and deploy them. generate documents from templates. **All those stuffes are available trough docker with the plugin compose**. We do not provide
information of how to run it without it.
If you want to develop some bundles, the first step is sufficient (until you deploy on production).
1. Get the code Install a new project
=============== =====================
Clone or download the chill-skeleton project and `cd` into the main directory.
Initialize project and dependencies
***********************************
.. code-block:: bash .. code-block:: bash
git clone https://gitea.champs-libres.be/Chill-project/chill-skeleton-basic.git symfony new --version=5.4 my_chill_project
cd chill-skeleton-basic cd my_chill_project
We strongly encourage you to initialize a git repository at this step, to track further changes.
As a developer, the code will stay on your computer and will be executed in docker container. To avoid permission problem, the code should be run with the same uid/gid from your current user. This is why we get your current user id with the command ``id -u`` in each following scripts.
2. Prepare composer to download the sources
===========================================
As you are running in dev, you must configure an auth token for getting the source code.
.. warning
If you skip this part, the code will be downloaded from dist instead of source (with git repository). You will probably replace the source manually, but the next time you will run ```composer update```, your repository will be replaced and you might loose something.
1. Create a personal access token from https://gitlab.com/-/profile/personal_access_tokens, with the `read_api` scope.
2. add a file called ```.composer/auth.json``` with this content:
.. code-block:: json
{
"gitlab-token": {
"gitlab.com": "glXXX-XXXXXXXXXXXXXXXXXXXX"
}
}
2. Prepare your variables and environment
=========================================
Copy ```docker-compose.override.dev.yml``` into ```docker-compose.override.yml```
.. code-block:: bash .. code-block:: bash
cp docker-compose.override.dev.template.yml docker-compose.override.yml # add the flex endpoints required to custom recipes
cat <<< "$(jq '.extra.symfony += {"endpoint": ["flex://defaults", "https://gitlab.com/api/v4/projects/57371968/repository/files/index.json/raw?ref=main"]}' composer.json)" > composer.json
# install chill and some dependencies
# TODO fix the suffix "alpha1" and replace by ^3.0.0 when version 3.0.0 will be released
composer require chill-project/chill-bundles v3.0.0-alpha1 champs-libres/wopi-lib dev-master@dev champs-libres/wopi-bundle dev-master@dev
2. Prepare your variables and docker-compose We encourage you to accept to include the "Docker configuration from recipes": this is the documented way to run database.
============================================ You must also accept to configure recipes from the contrib repository, unless you want to configure by yourself the bundles).
Have a look at the variable in ``.env`` and check if you need to adapt them. If they do not adapt with your need, or if some are missing:
1. copy the file as ``.env.local``: ``cp .env .env.local``
2. you may replace some variables inside ``.env``
Prepare also you docker-compose installation, and adapt it to your needs:
1. If you plan to deploy on dev, copy the file ``docker-compose.override.dev.template.yml`` to ``docker-compose.override.yml``.
2. adapt to your needs.
**Note**: If you intend to use the bundle ``Chill-Doc-Store``, you will need to configure and install an openstack object storage container with temporary url middleware. You will have to configure `secret keys <https://docs.openstack.org/swift/latest/api/temporary_url_middleware.html#secret-keys>`_.
3. Run the bootstrap script
===========================
This script can be run using `make`
.. code-block:: bash .. code-block:: bash
make init # fix some configuration
./post-install-chill.sh
This script will : # install node dependencies
yarn install
1. force docker-compose to, eventually, pull the base images and build the image used by this project ; # and compile assets
2. run an install script to download `composer <https://getcomposer.org>`_ ; yarn run encore production
3. install the php dependencies
4. build assets
.. warning::
The script will work only if the binary ``docker-compose`` is located into your ``PATH``. If you use ``compose`` as a docker plugin,
you can simulate this binary by creating this file at (for instance), ``/usr/local/bin/docker-compose`` (and run ``chmod +x /usr/local/bin/docker-compose``):
.. code-block:: bash
#!/bin/bash
/usr/bin/docker compose "$@"
.. note:: .. note::
In some cases it can happen that an old image (chill_base_php82 or chill_php82) stored in the docker cache will make the script fail. To solve this problem you have to delete the image and the container, before the make init : If you encounter this error during assets compilation (:code:`yarn run encore production`) (repeated multiple times):
.. code-block:: txt
[tsl] ERROR in /tmp/chill/v1/public/bundles/chillcalendar/types.ts(2,65)
TS2307: Cannot find module '../../../ChillMainBundle/Resources/public/types' or its corresponding type declarations.
run:
.. code-block:: bash .. code-block:: bash
docker-compose images php rm -rf public/bundles/*
docker rmi -f chill_php82:prod
docker-compose rm php
Then restart the compilation of assets (:code:```yarn run encore production```)
4. Start the project Configure your project
==================== **********************
You should carefully read the configuration files in :code:`chill/config/packages`, especially if you have
custom developments. But most of the time, this should be fine.
You have to configure some local variable, which are described in the :code:`.env` file. The secrets should not be stored
in this :code:`.env` file, but instead using the `secret management tool <https://symfony.com/doc/current/configuration/secrets.html>`_
or in the :code:`.env.local` file, which should not be commited to git repository.
You do not need to set variables for the smtp server, redis server and relatorio server, as they are generated automatically
by the symfony server, from the docker compose services.
The only required variable in the :code:`ADMIN_PASSWORD`. You can generate an hashed and salted admin password using the command
:code:`symfony console security:hash-password <your password> 'Symfony\Component\Security\Core\User\User'`. Then,
you can either:
- add this password to the :code:`.env.local` file, you must escape the character :code:`$`: if the generated password
is :code:`$2y$13$iyvJLuT4YEa6iWXyQV4/N.hNHpNG8kXlYDkkt5MkYy4FXcSwYAwmm`, your :code:`.env.local` file will be:
.. code-block:: env
ADMIN_PASSWORD=\$2y\$13\$iyvJLuT4YEa6iWXyQV4/N.hNHpNG8kXlYDkkt5MkYy4FXcSwYAwmm
- add the generated password to the secret management (**note**: you must add the generated hashed password to the secrets env,
not the password in clear text).
Prepare migrations and other tools
**********************************
To continue the installation process, you will have to run migrations:
.. code-block:: bash .. code-block:: bash
docker-compose up # start databases and other services
docker compose up -d
# the first start, it may last some seconds, you can check with docker compose ps
# run migrations
symfony console doctrine:migrations:migrate
# setup messenger
symfony console messenger:setup-transports
# prepare some views
symfony console chill:db:sync-views
**On the first run** (and after each upgrade), you must execute *post update commands* and run database migrations. With a container up and running, execute the following commands: .. warning::
If you encounter error while running :code:`symfony console messenger:setup-transports`, you can set up the messenger
transport to redis, by adding this in the :code:`.env.local` or :code:`.env` file:
.. code-block:: env
MESSENGER_TRANSPORT_DSN=redis://${REDIS_HOST}:${REDIS_PORT}/messages
Start your web server locally
*****************************
At this step, Chill will be ready for serving locally, but does not contains any configuration. You can run the project
locally using the `local symfony server <https://symfony.com/doc/current/setup/symfony_server.html>`_:
.. code-block:: bash .. code-block:: bash
# mount into to container # see the whole possibilities at https://symfony.com/doc/current/setup/symfony_server.html
./docker-php.sh symfony server:start -d
bin/console chill:db:sync-views
# and load fixtures
bin/console doctrine:migrations:migrate
Chill will be available at ``http://localhost:8001.`` Currently, there isn't any user or data. To add fixtures, run If you need to test the instance with accounts and some basic configuration, install fixtures (see below).
Add capabilities for dev
========================
If you need to add custom bundles, you can develop them in the `src/` directory, like any other symfony projects. You
can rely on the whole chill framework from there: there is no need to add them to the original `chill-bundles`.
You will require some bundles to have development tools:
- add fixtures
- add profiler and var-dumper to debug
Install fixtures
****************
.. code-block:: bash .. code-block:: bash
# mount into to container # generate fixtures for chill
./docker-php.sh composer require --dev doctrine/fixtures-bundle nelmio/alice
# and load fixtures (do not this for production) # now, you can generate fixtures (this will reset your database)
bin/console doctrine:fixtures:load --purge-with-truncate symfony console doctrine:fixtures:load
There are several users available: This will generate user accounts, centers, and some basic configuration.
- ``center a_social`` The accounts created are: :code:`center a_social`, :code:`center b_social`, :code:`center a_direction`, ... The full list is
- ``center b_social`` visibile in the "users" table: :code:`docker compose exec database psql -U app -c "SELECT username FROM users"`.
The password is always ``password``. The password is always :code:`password`.
Now, read `Operations` below. For running in production, read `prod_`. .. warning::
The fixtures are not fully functional. See the `corresponding issue <https://gitlab.com/Chill-Projet/chill-bundles/-/issues/280>`_.
Add web profiler and debugger
*****************************
.. code-block:: bash
composer require --dev symfony/web-profiler-bundle symfony/var-dumper
Working on chill bundles
************************
If you plan to improve the chill-bundles repository, this is great!
You must download chill-bundles as a git repository (and not as an archive, which is barely editable).
In your :code:`composer.json` file, add those lines:
.. code-block:: diff
{
"config": {
+ "preferred-install": {
+ "chill-project/chill-bundles": "source",
+ }
}
Then, run :code:`composer update` to take changes into account.
Code style, code quality and other tools
****************************************
For development, you will also have to install:
- `php-cs-fixer <https://cs.symfony.com/>`_
We also encourage you to use tools like `phpstan <https://phpstan.org>`_ and `rector <https://getrector.com>`_.
Commit and share your project
=============================
If multiple developers works on the project, you can commit your symfony project and share it with other people.
When another developers clone your project, they will have to:
- run :code:`composer install` and :code:`yarn install` to install the same dependencies as the initial developer;
- run :code:`yarn run encore production` to compile assets;
- copy eventually variables from the :code:`.env.local` files;
- start the docker compose stack, using :code:`docker compose`, and run migrations, set up transports, and prepare chill db views
(see the corresponding command above)
Update
======
In order to update your app, you must update dependencies:
- for chill-bundles, you can `set the last version <https://gitlab.com/Chill-Projet/chill-bundles/-/releases>`_ manually
in the :code:`composer.json` file, or set to the version to `^3.0.0` and run :code:`composer update` regularly
- run :code:`composer update` and :code:`yarn update` to maintain your dependencies up-to-date.
After each update, you must update your database schema:
.. code-block:: bash
symfony console doctrine:migrations:migrate
symfony console chill:db:sync-views
Operations Operations
********** ==========
Build assets Build assets
============ ************
run those commands: run those commands:
.. code-block:: bash .. code-block:: bash
make build-assets # for production (or in dev, when you don't need to work on your assets and need some speed)
yarn run encore production
# in dev, when you wan't to reload the assets on each changes
yarn run encore dev --watch
How to execute the console ? How to execute the console ?
============================ ****************************
.. code-block:: bash .. code-block:: bash
# if a container is running # start the console with all required variables
./docker-php.sh symfony console
# if not # you can add your command after that:
docker-compose run --user $(id -u) php bin/console symfony console list
How to create the database schema (= run migrations) ? How to generate documents
====================================================== *************************
Documents are generated asynchronously by `"consuming messages" <https://symfony.com/doc/current/messenger.html#consuming-messages-running-the-worker>`_.
You must generate them using a dedicated process:
.. code-block:: bash .. code-block:: bash
# if a container is running symfony console messenger:consume async priority
./docker-php.sh
bin/console doctrine:migrations:migrate
bin/console chill:db:sync-views
# if not
docker-compose run --user $(id -u) php bin/console doctrine:migrations:migrate
docker-compose run --user $(id -u) php bin/console chill:db:sync-views
To avoid memory issues, we encourage you to also use the :code:`--limit` parameter of the command.
How to read the email sent by the program ? How to read the email sent by the program ?
=========================================== *******************************************
Go at ``http://localhost:8005`` and you should have access to mailcatcher. In development, there is a built-in "mail catcher". Open it with :code:`symfony open:local:webmail`
In case of you should click on a link in the email, be aware that you should remove the "s" from https.
How to load fixtures ? (development mode only)
==============================================
.. code-block:: bash
# if a container is running
./docker-php.sh
bin/console doctrine:fixtures:load
# if not
docker-compose run --user $(id -u) php bin/console doctrine:fixtures:load
How to open a terminal in the project
=====================================
.. code-block:: bash
# if a container is running
./docker-php.sh
# if not
docker-compose run --user $(id -u) php /bin/bash
How to run cron-jobs ? How to run cron-jobs ?
====================== **********************
Some command must be executed in :ref:`cron jobs <cronjob>`. To execute them: Some command must be executed in :ref:`cron jobs <cronjob>`. To execute them:
.. code-block:: bash .. code-block:: bash
# if a container is running symfony console chill:cron-job:execute
./docker-php.sh
bin/console chill:cron-job:execute
# some of them are executed only during the night. So, we have to force the execution during the day:
bin/console chill:cron-job:execute 'name-of-the-cron'
# if not
docker-compose run --user $(id -u) php bin/console chill:cron-job:execute
# some of them are executed only during the night. So, we have to force the execution during the day:
docker-compose run --user $(id -u) php bin/console chill:cron-job:execute 'name-of-the-cron'
How to run composer ? What about materialized views ?
===================== *******************************
.. code-block:: bash There are some materialized views in chill, to speed up some complex computation on the database.
# if a container is running In order to refresh them, run cron job or refresh them manually in your database.
./docker-php.sh
composer
# if not
docker-compose run --user $(id -u) php composer
How to access to PGADMIN ? How to run tests for chill-bundles
========================== **********************************
Pgadmin is installed with docker-compose, and is available **only if you uncomment the appropriate lines into ``docker-compose.override.yml``.
You can access it at ``http://localhost:8002``.
Credentials:
- login: from the variable you set into ``docker-composer.override.yml``
- password: same :-)
How to run tests ?
==================
Tests reside inside the installed bundles. You must `cd` into that directory, download the required packages, and execute them from this place. Tests reside inside the installed bundles. You must `cd` into that directory, download the required packages, and execute them from this place.
@ -290,87 +322,29 @@ Exemple, for running unit test inside `main` bundle:
.. code-block:: bash .. code-block:: bash
# mount into the php image
./docker-php.sh
# cd into main directory # cd into main directory
cd vendor/chill-project/chill-bundles cd vendor/chill-project/chill-bundles
# download deps
git submodule init
git submodule update
composer install composer install
# run tests # run tests
bin/phpunit src/Bundle/path/to/your/test bin/phpunit src/Bundle/path/to/your/test
Or for running tests to check code style and php conventions with csfixer and phpstan: Or for running tests to check code style and php conventions with csfixer and phpstan:
.. code-block:: bash Troubleshooting
===============
# run code style fixer
bin/grumphp run --tasks=phpcsfixer
# run phpstan
bin/grumphp run --tasks=phpstan
.. note::
To avoid phpstan block your commits:
.. code-block:: bash
git commit -n ...
To avoid phpstan block your commits permanently:
.. code-block:: bash
./bin/grumphp git:deinit
How to run webpack interactively
================================
Executing :code:`bash docker-node.sh` will open a terminal in a node container, with volumes mounted.
How to switch the branch for chill-bundles, and get new dependencies
====================================================================
During development, you will switch to new branches for chill-bundles. As long as the dependencies are equals, this does not cause any problem. But sometimes, a new branch introduces a new dependency, and you must download it.
.. warning::
Ensure that you have gitlab-token ready before updating your branches. See above.
In order to do that without pain, use those steps:
0. Ensuire you have a token, set
1. at the app's root, update the ``composer.json`` to your current branch:
.. code-block:: json
{
"require": {
"chill-bundles": "dev-<my-branch>@dev"
}
2. mount into the php container (``./docker-php.sh``), and run ``composer update``
Error `An exception has been thrown during the rendering of a template ("Asset manifest file "/var/www/app/web/build/manifest.json" does not exist.").` on first run Error `An exception has been thrown during the rendering of a template ("Asset manifest file "/var/www/app/web/build/manifest.json" does not exist.").` on first run
==================================================================================================================================================================== ********************************************************************************************************************************************************************
Run :code:`make build-assets` Run :code:`make build-assets`
Running in production Running in production
********************* =====================
Currently, to run this software in production, the *state of the art* is the following : Currently, to run this software in production, the *state of the art* is the following :
1. Run the software locally and tweak the configuration to your needs ; 1. Run the software locally and tweak the configuration to your needs ;
2. Build the image and store them into a private container registry. This can be done using :code:`make build-and-push-image`. 2. Build the image and store them into a private container registry.
To be sure to target the correct container registry, you have to adapt the image names into your ``docker-compose.override.yml`` file.
3. Push the image on your registry, or upload them to the destination machine using ``docker image save`` and ``docker image load``.
3. Run the image on your production server, using docker-compose or eventually docker stack. You have to customize the variable set in docker-compose.
See also the :ref:`running-production-tips-and-tricks` below.
.. warning:: .. warning::
@ -382,25 +356,13 @@ Currently, to run this software in production, the *state of the art* is the fol
There are security issues if you keep the same variable than for production. There are security issues if you keep the same variable than for production.
.. _running-production-tips-and-tricks: Going further
=============
Tips and tricks .. toctree::
=============== :maxdepth: 2
Operation on database (backups, running custom sql, replication) are easier to set when run outside of a container. If you run into a container, take care of the volume where data are stored.
The PHP sessions are stored inside redis. This is useful if you distribute the traffic amongst different php server: they will share same sessions if a request goes into a different instance of the container.
When the PHP servers are shared across multiple instances, take care that some data is stored into redis: the same redis server should be reachable by all instances.
It is worth having an eye on the configuration of logstash container.
Design principles
*****************
Why the DB URL is also set in environment, and not in .env file ?
=================================================================
Because, at startup, a script does check the db is up and, if not, wait for a couple of seconds before running ``entrypoint.sh``. For avoiding double configuration, the configuration of the PHP app takes his configuration from environment also (and it will be standard in future releases, with symfony 4.0).
prod.rst
load-addresses.rst
prod-calendar-sms-sending.rst
msgraph-configure.rst