* allow more than 15 character in regroupment name
* remove unused methods in PickCenterType
* show only active Regroupment in form
* remove dead code and commented code
The documents are now generated in a queue, using symfony messenger. This queue should be configured:
```yaml
# app/config/messenger.yaml
framework:
messenger:
# reset services after consuming messages
# reset_on_message: true
failure_transport: failed
transports:
# https://symfony.com/doc/current/messenger.html#transport-configuration
async: '%env(MESSENGER_TRANSPORT_DSN)%'
priority:
dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
failed: 'doctrine://default?queue_name=failed'
routing:
# ... other messages
'Chill\DocGeneratorBundle\Service\Messenger\RequestGenerationMessage': priority
```
`StoredObject`s now have additionnal properties:
* status (pending, failure, ready (by default) ), which explain if the document is generated;
* a generationTrialCounter, which is incremented on each generation trial, which prevent each generation more than 5 times;
The generator computation is moved from the `DocGenTemplateController` to a `Generator` (implementing `GeneratorInterface`.
There are new methods to `Context` which allow to normalize/denormalize context data to/from a messenger's `Message`.
698 Filtrer les actions par agent traitant: utiliser le formulaire avec AddPersons, et pas un EntityType
See merge request Chill-Projet/chill-bundles!480
Now, by default:
- the creator will receive a notification for the last step only;
- the participant / assignee for each step won't receive a notification (unless they explicitly choose to receive one)
Resolve "La modale "on the fly" pour créer une personne ne fonctionne pas: il manque le centre"
Closes#29
See merge request Chill-Projet/chill-bundles!468
* reset admin vertical menu custom styles
* disable icons in admin vertical menu
* fix main content positioning in admin section
* fix table appearance in admin crud template
* new scope/center form: move submit button in record_action sticky area
* edit scope/center form: move submit button in record_action sticky area
* improve homogeneity in admin index pages: centers/scopes/users/jobs
* remove centered div old tags
Component works in modal or in main frame. CSS Flex position parent-children directives were broken.
Hope that fix don't introduce others display bug in any cases !
The range generator failed during week-end, because no range is selected (lastStart and endDate is null).
with this commit, the rangeGenerator may return null instead of an array, which means no sms will be send.
Those filters applyed on persons and allowed to select person which has a new, closing, or active accompanying period. But this is deprecated: there are now a dedicated Export for that ( CountPersonWithAccompanyingCourse)
Squashed commit of the following:
commit dc2bbc8f4da24549a1d42feb3b453af9f79ab2ff
Author: Julien Fastré <julien.fastre@champs-libres.coop>
Date: Tue Oct 18 10:40:14 2022 +0200
Fixes: remove iterator_to_array on formatter, which causes a PHP crashes
commit 4ce56a01a7
Author: Julien Fastré <julien.fastre@champs-libres.coop>
Date: Mon Oct 17 17:41:14 2022 +0200
DX: use tags and dependency injection to build ExportManager
* filters call static entity method
* add renderString option to manage add_children
* add tests on new entity method getDescendantsWithThisFor..()
* rename function in repository
* rename 'Select2..' classes by 'Pick..' and replace all occurences
Select2UserLocationType.php is called in another context:
UserController call UserCurrentLocationType which instanciate new Select2UserLocationType.php FormType
Each time a step is changed on an history, a record is stored in a
dedicated table.
When the acp's opening date is moved, the first row is adapted to match the new opening's date. This
mechanisme does not work if the opening date is move beyon the first end
date (if any), nor on the closing date.
In a ManyToMany relationship, doctrine does take care only of the owning
side of a relationship to inspect changes.
([see
documentation](https://www.doctrine-project.org/projects/doctrine-orm/en/current/reference/unitofwork-associations.html#important-concepts))
This commit mark as "internal" the methods add/removeSocialAction on the
inversed side (`Evaluation`) and let the methods add/removeEvaluation
on the owning side (`SocialAction`) update the inversed side.
The usage is also adapted into SocialWorkMetadata's importer.
In a ManyToMany relationship, doctrine does take care only of the owning
side of a relationship to inspect changes.
([see
documentation](https://www.doctrine-project.org/projects/doctrine-orm/en/current/reference/unitofwork-associations.html#important-concepts))
This commit mark as "internal" the methods add/removeSocialAction on the
inversed side (`Evaluation`) and let the methods add/removeEvaluation
on the owning side (`SocialAction`) update the inversed side.
The usage is also adapted into SocialWorkMetadata's importer.
The association between Person and Center is now stored in a dedicated
Entity: `PersonCenterHistory`, which have a date interval (start date
and endDate). The SQL counterpart is a table, with a constraint which
ensure that no person might be associated with two center at the same time.
For ease, a view is created to get the current center associated with
the person.
The dedicated migration creates also:
* indexes for a rapid search for person at current date;
* and populate the table from current data, setting the startdate to the
person's creation date and time if any, `NOW()` unless.
The `Person` entity is also updated to use the information from the
PersonCenterHistory classes, but this commit does not yet delete the
`Center` column.
composition
A filter and two aggregators create a join between composition and
household, but with a parameter at different date.
Each aggregator and filter now have a custom alias, to allow to set this
date parameter on each join.
EntityRepository for ActivityType
* [activity][export] Feature: improve label for aliases in "Filter by activity type"
* [activity][export] DX/Feature: use of an `ActivityTypeRepositoryInterface` instead of the old-style EntityRepository
- initiate new Vue component
- cleaning buildForm in Filter
- add HiddenType with ModelTransformer
- use ObjectToIdTransformer in HiddenType fields
- setting mount() and teleport
- list fetch url in api.js
- html/bootstrap form structure
- add Multiselect actionType
- i18n import
- add multiselect for goals and results
- add and remove options in multiselect (wip)
- fix basic vue add/remove element from data store
- vue cleaning
- add ids in value hiddenFields
- adapt Filter in AlterQuery (wip)
- improve code lisibility
- put orderby at the end
- addSelect with TO_CHAR() function (cancel EXRACT())
- uniformize addSelect cases
- orderBy with variable
- improve getLabels
when a specific join has to be shared between combined filters/aggregators, we need to check if it has been already loaded.
we cannot load it into indicator because result is wrong
* get the last updated time from the stored object's storage
* improve perf on loading content from stored object's storage: keep the
response in cache
* [person][export] Fixed: rename the alias for `accompanying_period` to `acp` in filter associated with person
* [activity][export] Feature: improve label for aliases in "Filter by activity type"
* [activity][export] DX/Feature: use of an `ActivityTypeRepositoryInterface` instead of the old-style EntityRepository
* [person][export] Fixed: some inconsistency with date filter on accompanying courses
* [person][export] Fixed: use left join for related entities in accompanying course aggregators
* [workflow] Feature: allow user to copy and send manually the access link for the workflow
* [workflow] Feature: show the email addresses that received an access link for the workflow
## Test releases
### 2.0.0-beta2
* [workflow]: Fixed: the notification is sent when the user is added to the first step.
* [budget] Feature: allow to desactivate some charges and resources, adding an `active` key in the configuration
* [person] Feature: on Evaluation, allow to configure an URL from the admin
### 2022-06
* [workflow]: added pagination to workflow list page
* [homepage_widget]: null error on tasks widget fixed
* [person-thirdparty]: fix quick-add of names that consist of multiple parts (eg. De Vlieger) within onthefly modal person/thirdparty
* [search]: Order of birthdate fields changed in advanced search to avoid confusion.
* [workflow]: Constraint added to workflow (https://gitlab.com/champs-libres/departement-de-la-vendee/accent-suivi-developpement/-/issues/675)
* [social_action]: only show active objectives (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/625)
* [household]: Reposition and cut button for enfant hors menage have been deleted (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/620)
* [admin]: Add crud for composition type in admin (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/611)
* [social_action]: only show active objectives (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/625)
## Test releases
### 2022-05-30
* fix creating a new AccompanyingPeriodWorkEvaluationDocument when replacing the document (the workflow was lost)
### 2022-05-27
* [storedobject] add title field on StoredObject entity + use it in activity documents (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/604)
* [main] add a "read more..." on comment embeddable when overflown (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/604)
* [person] add closing motive to closed acc course (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/603)
* [person] household filiation: fetch person info when unfolding person (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/586)
* [admin] repair edit of social action in the admin (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/601)
* [admin]: add select2 to Goal form type entity fields (https://gitlab.com/champs-libres/departement-de-la-vendee/accent-suivi-developpement/-/issues/702)
* [main] allow hide permissions group list menu (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/577)
* [main] allow hide change user password menu (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/577)
* [main] filter user jobs by active jobs (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/577)
* [main] add civility to User (entity, migration and form type) (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/577)
* [admin] refactorisation of the admin section: reorganisation of the menu, translations, form types, new entities (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/592)
* [admin] add admin section for languages and countries (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/596)
* [activity] activity admin: translations + remove label field for comment on admin activity type (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/587)
* [address] can add extra address info even if noAddress (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/576)
### 2022-05-06
* [person] add civility when creating a person (with the on-the-fly component or in the php form) (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/557)
* [person] add address when creating a person (with the on-the-fly component or in the php form) (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/557)
* [person] add household creation API point (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/557)
### 2021-04-29
* [person] prevent circular references in PersonDocGenNormalizer (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/527)
* [person] add maritalStatusComment to PersonDocGenNormalizer (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/582)
* Load relationships without gender in french fixtures
* Add command to remove old draft accompanying periods
* [parcours]: If users assings him/herself as referrer and job is not null. Update parcours job (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/578)
### 2021-04-28
* [address] fix bug when editing address: update location and addressreferenceId + better update of the map in edition (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/593)
* [main] avoid address reference search on undefined post code (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/561)
* [person] prevent duplicate relationship in filiation/household graph (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/560)
* [Documents] Validate storedObject and allow for null data (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/565)
* [parcours]: Comments can be unpinned + edit/delete for all users that are allowed to edit parcours (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/566)
### 2021-04-26
* [Datepickers] datepickers fixed when using keyboard to enter date (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/545)
* [social_action] Display 'agents traitants' in parcours resumé and social action list (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/568)
* [Person_search] Closed parcours shown within an accordeon that can be opened/closed (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/574)
### 2021-04-24
* [notification email on course designation] allow raw string in email content generation
* [Accompanying period work] list evaluations associated to a work by startDate, and then by id, from the most recent to older
* [Documents] Change wording 'créer' to 'enregistrer' (https://gitlab.com/champs-libres/departement-de-la-vendee/accent-suivi-developpement/-/issues/634)
* [Parcours]: The number of 'mes parcours' displayed (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/572)
* [Hompage_widget]: Renaming of tabs and removal of social actions tab (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/570)
* [activity]: Ignore thirdparties when creating a social action via an activity (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/573)
* [parcours]: change wording of warning message and button when user is not associated to a household yet (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/590#note_918370943)
* [Accompanying period work evaluations] list documents associated to a work by creation date, and then by id, from the most recent to older
* [Course comment] add validationConstraint NotNull and NotBlank on comment content, to avoid sql error
* [Notifications] delay the sending of notificaiton to kernel.terminate
* [Notifications / Period user change] fix the sending of notification when user changes
* [Activity form] invert 'incoming' and 'receiving' in Activity form
* [Activity form] keep the same order for 'attendee' field in new and edit form
* [list with period] use "sameas" test operator to introduce requestor in list
* [notification email on course designation] allow raw string in email content generation
* [Accompanying period work] list evaluations associated to a work by startDate, and then by id, from the most recent to older
* [evaluation_document] changing date to datetime in order to display the time at which document was created (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/569)
### 2021-04-13
* [person] household address: add a form for editing the validFrom date (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/541)
* [person] householdmemberseditor: fix composition type bug in select form (vuejs) (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/543)
* [docgen] add more persons choices in docgen for course: amongst requestor (if person), resources of course (if person), and PersonResource (if person);
* [docgen] add a new context with a list of activities in course
* [docgen] add a comment in budget lines
* [notifications] allow to send a notification to an email address. The address receive an access link
* [adresses] add constraints in database to avoid errors later: postcode not null, and validfrom <= validto
* [accompanying work editor] add a label on document title input
### 2021-04-07
* notification list: move action buttons outside of the toggle
* fix detecting of non-read notification
* filter users which are disabled in search user api
* order query for location and add pagination in list
* allow every person which has part for a workflow to see the workflow page
* able to see the workflow if the evaluation document has been deleted
* hardcode the list of supported mime types for edition with collabora
* list of accompanying course: allow to see the pinned comment in list_item
### 2021-04-06
* [main] notification toggle read: correct js syntax for compilation in production (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/548)
* [parcours] Display of interlocuteurs changed to flex-table in parcours edit page to prevent cut-off of information (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/535)
* [activity] espace entre les boutons pour supprimer les documents
### continuous release in February and March
* Creation of PickCivilityType, and implementation in PersonType and ThirdpartyType
* [person] Accompanying course evaluation documents: disable the WOPI edit link if mimetype not supported and if no keyInfos
* [activity] display error messages above the form in creating a new location (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/481)
* [activity] show required field in activity edit/new by an asterix in the vuejs fields (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/494)
* [ACL] fix allow to see the course, event if the scope'course does not contains the scope's user
@@ -52,6 +185,7 @@ and this project adheres to
* [household] bugfix if position of member is null, renderbox no longer throws an error (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/480)
* [parcours] location cannot be removed if linked to a user (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/478)
* [person] email added to twig personRenderbox (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/490)
* [activity] Only youngest descendant is kept for social issues and actions (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/471)
* [person] Add link to current household in person banner (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/484)
* [address] person badge in address history changed to open OnTheFly with all person info (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/489)
* [person] Change 'personne' with 'usager' and '&' with 'ET' (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/499)
* [notification] Display of social action within workflow notification set to display block (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/537)
* [onthefly] trim trailing whitespace in email of person and thirdparty (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/542)
* [person] fix bug when phonenumber or mobile number is left empty (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/546)
* [action] Only youngest descendant is kept for social issues and actions (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/471)
## Test releases
### test release 2022-02-21
@@ -108,8 +242,6 @@ and this project adheres to
* [bug]: fix confidential toggle of address in thirdpartyrenderbox (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/460)
## Test releases
* Creation of PickCivilityType, and implementation in PersonType and ThirdpartyType
* [homepage widget] add vue homepage_widget with asynchone loading, give a global view resume of the user concerned actions, notifications, etc.
* [thirdparty] Add a contact to a thirdparty from within onTheFly (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/345)
## Test releases
=======
* [homepage widget] add vue homepage_widget with asynchone loading, give a global view resume of the user concerned actions, notifications, etc.
>>>>>>> issue422_and_others_on_AddPersons
=======
* [homepage widget] add vue homepage_widget with asynchone loading, give a global view resume of the user concerned actions, notifications, etc.
* [thirdparty] Add a contact to a thirdparty from within onTheFly (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/345)
>>>>>>> b0d50d315c8e00959a967badac9cf5057ab2b4bc
### test release 2021-01-31
* [person] accompanying course: optimisation: do not fetch some resources for the banner (https://gitlab.com/champs-libres/departement-de-la-vendee/chill/-/issues/409)
Par bundle, toutes les traductions des pages twig se trouvent dans un seul fichier `translations/messages.fr.yaml`.
## Emplacement des fichiers
Les controllers, form type & templates twig sont placés à la racine des dossiers `Controller`, `Form`&`Ressources/views`, respectivement. Pour les pages Admin, on ne les mets plus dans des sous-dossiers Admin.
## Assets: nommage des entrypoints
Trois types d'entrypoint:
@@ -146,19 +153,20 @@ Ces règles n'ont pas toujours été utilisées par le passé. Elles sont souhai
Les routes sont nommées de cette manière:
`chill_bundle_entite_action`
`chill_(api|crud)_bundle_(api)_entite_action`
1. d'abord chill_ (pour tous les modules chill)
2. ensuite une string qui est identique, par bundle
3.si le point est un point d'api (json), alors ajouter la string `api`
4. ensuite une string qui indique sur quelle entité porte la route, voire également les sous-entités
5. ensuite une action (`list`, `view`, `edit`, `new`, ...)
2. ensuite `crud` ou `api`, optionnel, automatiquement ajouté si la route est générée par la configuration
3.ensuite une string qui indique le bundle (`main`, `person`, `activity`, ...)
4. ensuite, `api`, si la route est une route d'api.
5. ensuite une string qui indique sur quelle entité porte la route, voire également les sous-entités
6. ensuite une action (`list`, `view`, `edit`, `new`, ...)
Le fait d'indiquer `api` en 3 permet de distinguer les routes d'api qui sont générées par la configuration (qui sont toutes préfixées par `chill_api`, de celles générées manuellement. (Exemple: `chill_api_household__index`, et `chill_person_api_household_members_move`)
Le fait d'indiquer `api` en quatrième position permet de distinguer les routes d'api qui sont générées par la configuration (qui sont toutes préfixées par `chill_api`, de celles générées manuellement. (Exemple: `chill_api_household__index`, et `chill_person_api_household_members_move`)
Si les points 4 et 5 sont inexistants, alors ils sont remplacés par d'autres éléments de manière à garantir l'unicité de la route, et sa bonne compréhension.
### URL
### Nommage des URL
Les URL respectent également une convention:
@@ -219,6 +227,14 @@ Les éléments suivants devraient se trouver dans la liste:
Ces éléments peuvent être entrecoupés de l'identifiant d'une entité. Dans ce cas, cet identifiant se place juste après l'entité auquel il se rapporte.
#### Pour les URL de l'espace Admin
Même conventions que dans les autres pages html de l'application, **mais `admin` est ajouté en deuxième position**. Soit:
// introduce a random component to ensure a roll of task execution when multiple instances are hosted on same machines
&&mt_rand(0,5)===0;
}
publicfunctiongetKey():string
{
return'arbitrary-and-unique-key';
}
publicfunctionrun():void
{
// here, we execute the command
}
}
How are cron job scheduled ?
============================
If the command :code:`chill:cron-job:execute` is run with one or more :code:`job` argument, those jobs are run, **without checking that the job can run** (the method :code:`canRun` is not executed).
If any :code:`job` argument is given, the :code:`CronManager` schedule job with those steps:
* the tasks are ordered, with:
* a priority is given for tasks that weren't never executed;
* then, the tasks are ordered, the last executed are the first in the list
* then, for each tasks, and in the given order, the first task where :code:`canRun` return :code:`TRUE` will be executed.
The command :code:`chill:cron-job:execute` execute **only one** task.
Export is an important issue for the Chill software : users should be able to :
Export is an important issue within the Chill software : users should be able to :
- compute statistics about their activity ;
- list "things" which make part of their activities.
- list "things" which are a part of their activities.
The `main bundle`_ provides a powerful framework to build custom queries with re-usable parts across differents bundles.
..contents:: Table of content
:local:
..seealso::
..seealso::
`The issue where this framework was discussed <https://git.framasoft.org/Chill-project/Chill-Main/issues/9>`_
Provides some information about the pursued features and architecture.
@@ -32,37 +32,37 @@ Concepts
Some vocabulary: 3 "Export elements"
------------------------------------
Four terms are used for this framework :
Four terms are used for this framework :
exports
provides some basic operation on the date. Two kind of exports are available :
Exports
provide some basic operation on the data. Two kinds of exports are available :
- computed data : it may be "the number of people", "the number of activities", "the duration of activities", ...
- list data : it may be "the list of people", "the list of activity", ...
- list data : it may be "the list of people", "the list of activities", ...
filters
The filters make a filter on the date: it removes some information the user doesn't want to introduce in the computation done by export. In other word, filters make a filter...
Filters
The filters create a filter on the data: it removes some information the user doesn't want to introduce in the computation done by the export.
Example of filter: "people under 18 years olds", "activities between the 1st of June and the 31st December", ...
Example of a filter: "people under 18 years olds", "activities between the 1st of June and the 31st December", ...
aggregators
The aggregator aggregates the data into some group (some software use the term 'bucket').
Aggregators
The aggregator aggregates the data into some group (some software use the term 'bucket').
Example of aggregator : "group people by gender", "group people by nationality", "group activity by type", ...
Example of an aggregator : "group people by gender", "group people by nationality", "group activity by type", ...
formatters
The formatters format the data into a :class:`Symfony\Component\HttpFoundation\Response`, which will be returned "as is" by the controller to the web client.
Formatters
The formatters format the data into a :class:`Symfony\Component\HttpFoundation\Response`, which will be returned "as is" by the controller to the web client.
Example of formatter: "format data as CSV", "format data as ods spreadsheet", ...
Example of a formatter: "format data as CSV", "format data as an ods spreadsheet", ...
Anatomy of an export
---------------------
An export may be explained as a sentence, where each part of this sentence refers to one or multiple exports element. Examples :
An export can be thought of as a sentence where each part of this sentence refers to one or multiple export elements. Examples :
**Example 1**: Count the number of people having at least one activity in the last 12 month, and group them by nationality and gender, and format them in a CSV spreadsheet.
Here :
Here :
-*count the number of people* is the export part
-*having at least one activity* is the filter part
@@ -72,10 +72,10 @@ Here :
Note that :
-aggregators, filters, exports and aggregators are cross-bundle. Here the bundle *activity* provides a filter which apply on an export provided by the person bundle ;
-there may exists multiple aggregator or filter for one export. Currently, only one export is allowed.
-Aggregators, filters, exports and formatters are cross-bundle. Here the bundle *activity* provides a filter which is applied on an export provided by the person bundle ;
-Multiple aggregator or filter for one export may exist. Currently, only one export is allowed.
**Example 2**: Count the average duration of an activity with type "meeting", which occurs between the 1st of June and the 31st of December, group them by week, and format the data in a OpenDocument spreadsheet.
**Example 2**: Count the average duration of an activity with type "meeting", which occurs between the 1st of June and the 31st of December, group them by week, and format the data in an OpenDocument spreadsheet.
Here :
Here :
-*count the average duration of an activity* is the export part
-*activity with type meeting* is a filter part
@@ -102,7 +102,7 @@ Here :
The result might be :
+-----------------------+----------------------+
| Week | Number of activities |
| Week | Number of activities |
+=======================+======================+
| 2015-10 | 10 |
+-----------------------+----------------------+
@@ -116,77 +116,77 @@ The result might be :
Authorization and exports
-------------------------
Exports, filters and aggregators should not make see data the user is not allowed to see.
Exports, filters and aggregators should not show data the user is not allowed to see within the application.
In other words, developers are required to take care of user authorization for each export.
It should exists a special role that should be granted to users which are allowed to build exports. For more simplicity, this role should apply on center, and should not requires special circles.
There should be a specific role that grants permission to users who are allowed to build exports. For more simplicity, this role should apply on a center, and should not require special circles.
How does the magic works ?
How does the magic work ?
===========================
To build an export, we rely on the capacity of the database to execute queries with aggregate (i.e. GROUP BY) and filter (i.e. WHERE) instructions.
An export is an SQL query which is initiated by an export, and modified by aggregators and filters.
..note::
..note::
**Example**: Count the number of people having at least one activity in the last 12 month, and group them by nationality and gender
1. The report initiate the query
1. The report initiates the query
..code-block::SQL
SELECTcount(people.*)FROMpeople
2. The filter add a where and join clause :
2. The filter adds a where and join clause :
..code-block::SQL
SELECTcount(people.*)FROMpeople
RIGHTJOINactivity
SELECTcount(people.*)FROMpeople
RIGHTJOINactivity
WHEREactivity.dateISBETWEENnowAND6monthago
3. The aggregator "nationality" add a GROUP BY clause and a column in the SELECT statement:
3. The aggregator "nationality" adds a GROUP BY clause and a column in the SELECT statement:
Each filter, aggregator and filter may collect parameters from the user by providing a form. This form is appended to the export form. Here is an example.
Each filter, aggregator and filter may collect parameters from the user through a form. This form is appended to the export form. Here is an example.
The screenshot show the export form for ``CountPeople`` (Nombre de personnes). The filter by date of birth is checked (*Filtrer par date de naissance de la personne*), which allow to show a subform, which is provided by the :class:`Chill\PersonBundle\Export\Filter\BirthdateFilter`. The other filter, which are unchecked, does not show the subform.
The screenshot shows the export form for ``CountPeople`` (Nombre de personnes). The filter by date of birth is checked (*Filtrer par date de naissance de la personne*), which triggers a subform, which is provided by the :class:`Chill\PersonBundle\Export\Filter\BirthdateFilter`. The other unchecked filter does not show the subform.
Two aggregators are also checked : by Country of birth (*Aggréger les personnes par pays de naissance*, corresponding class is :class:`Chill\PersonBundle\Export\Aggregator\CountryOfBirthAggregator`, which also open a subform. The aggregator by gender (*Aggréger les personnes par genre*) is also checked, but there is no corresponding subform.
Two aggregators are also checked : by Country of birth (*Aggréger les personnes par pays de naissance*, the corresponding class is :class:`Chill\PersonBundle\Export\Aggregator\CountryOfBirthAggregator`, which also triggers a subform. The aggregator by gender (*Aggréger les personnes par genre*) is also checked, but there is no corresponding subform.
The Export Manager
------------------
The Export manager (:class:`Chill\MainBundle\Export\ExportManager` is the central class which register all exports, aggregators, filters and formatters.
The Export manager (:class:`Chill\MainBundle\Export\ExportManager` is the central class which registers all exports, aggregators, filters and formatters.
The export manager is also responsible for orchestrating the whole export process, producing a :class:`Symfony\FrameworkBundle\HttpFoundation\Request`to each export request.
The export manager is also responsible for orchestrating the whole export process, producing a :class:`Symfony\FrameworkBundle\HttpFoundation\Request`for each export request.
The export form step
--------------------
The form step allow to build a form, aggregating different parts of the module.
The form step allows you to build a form, combining different parts of the module.
The building of forms is separated between different subform, which are responsible for rendering their part of the form (aggregators, filters, and export).
The building of forms is split into different subforms, where each one is responsible for rendering their part of the form (aggregators, filters, and export).
..figure:: /_static/puml/exports/form_steps.png
:scale:40%
@@ -194,12 +194,12 @@ The building of forms is separated between different subform, which are responsi
The formatter form step
-----------------------
The formatter form is processed *after* the user filled the export form. It is built the same way, but receive in parameters the data entered by the user on the previous step (i.e. export form). It may then adapt it accordingly (example: show a list of columns selected in aggregators).
The formatter form is processed *after* the user filled the export form. It is built the same way, but receives the data entered by the user on the previous step as parameters (i.e. export form). It may then adapt it accordingly (example: show a list of columns selected in aggregators).
Processing the export
---------------------
The export process may be explained by this schema :
The export process can be explained by this schema :
@@ -219,20 +219,20 @@ This is an example of the ``CountPerson`` export :
:language:php
:linenos:
***Line 36**: the ``getType`` function return a string. This string will be used to find the aggregtors and filters which will apply to this export.
***Line 41**: a simple description to help user to understand what your export does.
***Line 36**: the ``getType`` function returns a string. This string will be used to find the aggregtors and filters which will apply to this export.
***Line 41**: a simple description to help users understand what your export does.
***Line 46**: The title of the export. A summary of what your export does.
***Line 51**: The list of roles requires to execute this export.
***Line 51**: The list of roles required to execute this export.
***Line 56**: We initiate the query here...
***Line 59**: We have to filter the query with centers the users checked in the form. We process the $acl variable to get all ``Center`` object in one array
***Line 63**: We create the query, with a query builder.
***Line 74**: We simply returns the result, but take care of hydrating the results as an array.
***Line 103**: return the list of formatters types which are allowed to apply on this filter
***Line 59**: We have to filter the query with centers the users checked in the form. We process the $acl variable to get all ``Center`` objects in one array
***Line 63**: We create the query with a query builder.
***Line 74**: We return the result, but make sure to hydrate the results as an array.
***Line 103**: return the list of formatter types which are allowed to be applied on this filter
Filters
-------
This is an example of the *filter by birthdate*. This filter ask some information in a form (`buildForm` is not empty), and this form must be validated. To performs this validations, we implement a new Interface: :class:`Chill\MainBundle\Export\ExportElementValidatedInterface`:
This is an example of the *filter by birthdate*. This filter asks some information through a form (`buildForm` is not empty), and this form must be validated. To perform this validation, we implement a new Interface: :class:`Chill\MainBundle\Export\ExportElementValidatedInterface`:
- This project use `docker <https://docker.com>`_ to be run. As a developer, use `docker-compose <https://docs.docker.com/compose/overview/>`_ to bootstrap a dev environment in a glance. You do not need any other dependencies ;
- Make is used to automate scripts.
Installation in development mode
********************************
Installation
************
If you plan to run chill in production:
1. install it locally first, and check if everything is ok on your local machine;
2. once ready, build the image from your local machine, and deploy them.
If you want to develop some bundles, the first step is sufficient (until you deploy on production).
1. Get the code
===============
Clone or download the chill-app project and `cd` into the main directory.
Clone or download the chill-skeleton project and `cd` into the main directory.
As a developer, the code will stay on your computer and will be executed in docker container. To avoid permission problem, the code should be run with the same uid/gid from your current user. This is why we get your current user id with the command ``id -u`` in each following scripts.
2. Prepare your variables
=========================
2. Prepare composer to download the sources
===========================================
Have a look at the variable in ``.env.dist`` and in ``app/config/parameters.yml.dist`` and check if you need to adapt them. If they do not adapt with your need, or if some are missing:
As you are running in dev, you must configure an auth token for getting the source code.
1. copy the file as ``.env``: ``cp .env.dist .env``
.. warning
If you skip this part, the code will be downloaded from dist instead of source (with git repository). You will probably replace the source manually, but the next time you will run ```composer update```, your repository will be replaced and you might loose something.
1. Create a personal access token from https://gitlab.com/-/profile/personal_access_tokens, with the `read_api` scope.
2. add a file called ```.composer/auth.json``` with this content:
..code-block::json
{
"gitlab-token":{
"gitlab.com":"glXXX-XXXXXXXXXXXXXXXXXXXX"
}
}
2. Prepare your variables and environment
=========================================
Copy ```docker-compose.override.dev.yml``` into ```docker-compose.override.yml```
Have a look at the variable in ``.env`` and check if you need to adapt them. If they do not adapt with your need, or if some are missing:
1. copy the file as ``.env.local``: ``cp .env .env.local``
2. you may replace some variables inside ``.env``
Prepare also you docker-compose installation, and adapt it to your needs:
1. If you plan to deploy on dev, copy the file ``docker-compose.override.dev.template.yml`` to ``docker-compose.override.yml``.
2. adapt to your needs.
**Note**: If you intend to use the bundle ``Chill-Doc-Store``, you will need to configure and install an openstack object storage container with temporary url middleware. You will have to configure `secret keys <https://docs.openstack.org/swift/latest/api/temporary_url_middleware.html#secret-keys>`_.
3. Run the bootstrap script
@@ -60,10 +111,32 @@ This script can be run using `make`
This script will :
1. force docker-compose to, eventually, pull the base images and build the image used by this project ;
2. run an install script to download `composer <https://getcomposer.org>`_ ;
2. run an install script to download `composer <https://getcomposer.org>`_ ;
3. install the php dependencies
4. build assets
..warning::
The script will work only if the binary ``docker-compose`` is located into your ``PATH``. If you use ``compose`` as a docker plugin,
you can simulate this binary by creating this file at (for instance), ``/usr/local/bin/docker-compose`` (and run ``chmod +x /usr/local/bin/docker-compose``):
..code-block::bash
#!/bin/bash
/usr/bin/docker compose "$@"
..note::
In some cases it can happen that an old image (chill_base_php or chill_php) stored in the docker cache will make the script fail. To solve this problem you have to delete the image and the container, before the make init :
..code-block::bash
docker-compose images php
docker rmi -f chill_php:prod
docker-compose rm php
4. Start the project
====================
@@ -76,13 +149,20 @@ This script will :
..code-block::bash
make migrate
# mount into to container
./docker-php.sh
# and load fixtures
bin/console doctrine:migrations:migrate
Chill will be available at ``http://localhost:8001.`` Currently, there isn't any user or data. To add fixtures, run
A basic configuration of `sami <https://github.com/FriendsOfPhp/Sami>`_ is embedded within the project.
During development, you will switch to new branches for chill-bundles. As long as the dependencies are equals, this does not cause any problem. But sometimes, a new branch introduces a new dependency, and you must download it.
A configuration file for `phpDocumentor <https://www.phpdoc.org>`_ is present.
..warning::
Ensure that you have gitlab-token ready before updating your branches. See above.
In order to do that without pain, use those steps:
0. Ensuire you have a token, set
1. at the app's root, update the ``composer.json`` to your current branch:
..code-block::json
{
"require":{
"chill-bundles":"dev-<my-branch>@dev"
}
2. mount into the php container (``./docker-php.sh``), and run ``composer update``
Error `An exception has been thrown during the rendering of a template ("Asset manifest file "/var/www/app/web/build/manifest.json" does not exist.").` on first run
Because, at startup, a script does check the db is up and, if not, wait for a couple of seconds before running ``entrypoint.sh``. For avoiding double configuration, the configuration of the PHP app takes his configuration from environment also (and it will be standard in future releases, with symfony 4.0).
* authenticate users using Microsoft Graph, with relatively small adaptations;
* synchronize calendar in both ways (`see the user manual for a large description of the feature <https://gitea.champs-libres.be/Chill-project/manuals>`_).
Both can be configured separately (synchronising calendars without SSO, or SSO without calendar). When calendar sync is configured without SSL, the user's email address is the key to associate Chill's users with Microsoft's ones.
Configure SSO
-------------
On Azure side
*************
Configure an app with the Azure interface, and give it the name of your choice.
Grab the tenant's ID for your app, which is visible on the main tab "Vue d'ensemble":
..figure:: ./saml_login_id_general.png
This the variable which will be named :code:`SAML_IDP_APP_UUID`.
Go to the "Single sign-on" ("Authentication unique") section. Choose "SAML" as protocol, and fill those values:
..figure:: ./saml_login_1.png
1. The :code:`entityId` seems to be arbitrary. This will be your variable :code:`SAML_ENTITY_ID`;
2. The url response must be your Chill's URL appended by :code:`/saml/acs`
3. The only used attributes is :code:`emailaddress`, which must match the user's email one.
..figure:: ./saml_login_2.png
You must download the certificate, as base64. The format for the download is :code:`cer`: you will remove the first and last line (the ones with :code:`-----BEGIN CERTIFICATE-----` and :code:`-----END CERTIFICATE-----`), and remove all the return line. The final result should be something as :code:`MIIAbcdef...XyZA=`.
This certificat will be your :code:`SAML_IDP_X509_CERT` variable.
The url login will be filled automatically with your tenant id.
Do not forget to provider user's accesses to your app, using the "Utilisateurs et groupes" tab:
..figure:: ./saml_login_appro.png
You must know have gathered all the required variables for SSO:
..code-block::
SAML_BASE_URL=https://test.chill.be # must be
SAML_ENTITY_ID=https://test.chill.be # must match the one entered
The sync processe might be configured in the same app, or into a different app.
The synchronization processes use Oauth2.0 for authentication and authorization.
..note::
Two flows are in use:
* we authenticate "on behalf of a user", to allow users to see their own calendar or other user's calendar into the web interface.
Typically, when the page is loaded, Chill first check that an authorization token exists. If not, the user is redirected to Microsoft Azure for authentification and a new token is grabbed (most of the times, this is transparent for users).
* Chill also acts "as a machine", to synchronize calendars with a daemon background.
One can access the configuration using this screen (it is quite well hidden into the multiple of tabs):
..figure:: ./oauth_app_registration.png
You can find the oauth configuration on the "Securité > Autorisations" tab, and click on "application registration" (not translated).
Add a redirection URI for you authentification:
..figure:: ./oauth_api_authentification.png
The URI must be "your chill public url" with :code:`/connect/azure/check` at the end.
Allow some authorizations for your app:
..figure:: ./oauth_api_autorisees.png
Take care of the separation between autorization "on behalf of a user" (déléguée), or "for a machine" (application).
Some explanation:
* Users must be allowed to read their user profile (:code:`User.Read`), and the profile of other users (:code:`User.ReadBasicAll`);
* They must be allowed to read their calendar (:code:`Calendars.Read`), and the calendars shared with them (:code:`Calendars.Read.Shared`);
The sync daemon must have write access:
* the daemon must be allowed to read all users and their profile, to establish a link between them and the Chill's users: (:code:`Users.Read.All`);
* it must also be allowed to read and write into the calendars (:code:`Calendars.ReadWrite.All`)
* for sending invitation to other users, the permission (:code:`Mail.Send`) must be granted.
At this step, you might choose to accept those permissions for all users, or let them do it by yourself.
Grab your client id:
..figure:: ./oauth_api_client_id.png
This will be your :code:`OAUTH_AZURE_CLIENT_ID` variable.
Generate a secret:
..figure:: ./oauth_api_secret.png
This will be your :code:`OAUTH_AZURE_CLIENT_SECRET` variable.
And get you azure's tenant id, which is the same as the :code:`SAML_IDP_APP_UUID` (see above).
You can now process for the first api authorization on the application side, (unless you did it in the Azure interface), and get a first token, by using :
This will generate a url that you can use to grant your app for your tenant. The redirection may fails in the browser, but this is not relevant: if you get an authorization token in the CLI, the authentication works.
Run the processes to synchronize
--------------------------------
The calendar synchronization is processed using symfony messenger. It seems to be intersting to configure a queue (in the postgresql database it is the most simple way), and to run a worker for synchronization, at least in production.
The association between chill's users and Microsoft's users is done by this cli command:
In order to generate the application key, secret, and consumerKey, refers to their `documentation <https://docs.ovh.com/gb/en/api/first-steps-with-ovh-api/>`_.
Before to be able to send your first sms, you must enable your account, grab some credits, and configure a sender. The service_name is an internal configuration generated by OVH.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU
Free Documentation License".
.._prod:
Installation for production
###########################
An installation use these services, which are deployed using docker containers:
* a php-fpm image, which run the Php and Symfony code for Chill;
* a nginx image, which serves the assets, and usually proxy the php requests to the fpm image;
* a redis server, which stores the cache, sessions (this is currently hardcoded in the php image), and some useful keys (like wopi locks);
* a postgresql database. The use of postgresql is mandatory;
* a relatorio service, which transform odt templates to full documents (replacing the placeholders);
Some external services:
* (required) an openstack object store, configured with `temporary url <https://docs.openstack.org/swift/latest/api/temporary_url_middleware.html>` configured (no openstack users is required). This is currently the only way to store documents from chill;
* a mailer service (SMTP)
* (optional) a service for verifying phone number. Currently, only Twilio is possible;
* (optional) a service for sending Short Messages (SMS). Currently, only Ovh is possible;
The `docker-compose.yaml` file of chill app is a basis for a production install. The environment variable in the ```.env``` and ```.env.prod``` should be overriden by environment variables, or ```.env.local``` files.
This should be adapted to your needs:
* The image for php and nginx apps are pre-compiled images, with the default configuration and bundle. If they do not fullfill your needs, you should compile your own images.
.. TODO:
As the time of writing (2022-07-03) those images are not published yet.
* Think about how you will backup your database. Some adminsys find easier to store database outside of docker, which might be easier to administrate or replicate.
Cron jobs
=========
The command :code:`chill:cron-job:execute` should be executed every 15 minutes (more or less).
This command should never be executed concurrently. It should be not have more than one process for a single instance.
Post-install tasks
==================
- import addresses. See :ref:`addresses`.
Tweak symfony messenger
=======================
Calendar sync is processed using symfony messenger.
You can tweak the configuration
Going further:
* Configure the saml login and synchronisation with Outlook api
message: "#^Method Chill\\\\ActivityBundle\\\\Export\\\\Export\\\\StatActivityDuration\\:\\:getDescription\\(\\) should return string but return statement is missing\\.$#"
message: "#^Method Chill\\\\ActivityBundle\\\\Export\\\\Export\\\\StatActivityDuration\\:\\:getTitle\\(\\) should return string but return statement is missing\\.$#"
message: "#^Method Chill\\\\ThirdPartyBundle\\\\Search\\\\ThirdPartySearch\\:\\:renderResult\\(\\) should return string but return statement is missing\\.$#"
return['Filtered activity by location: only %locations%',[
'%locations%'=>implode(', ',$locations),
]];
}
publicfunctiongetTitle():string
{
return'Filter activity by location';
}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
