From 786c3e6184c14d8c4fadc42b2f453a4a009b8be1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Julien=20Fastr=C3=A9?= Date: Tue, 9 Feb 2016 21:34:23 +0100 Subject: [PATCH] [wip] first documentation for exports --- source/development/exports.rst | 140 +++++++++++++++++++++++++++++++++ source/development/index.rst | 1 + 2 files changed, 141 insertions(+) create mode 100644 source/development/exports.rst diff --git a/source/development/exports.rst b/source/development/exports.rst new file mode 100644 index 000000000..22cf426ab --- /dev/null +++ b/source/development/exports.rst @@ -0,0 +1,140 @@ +.. Copyright (C) 2014 Champs Libres Cooperative SCRLFS + Permission is granted to copy, distribute and/or modify this document + under the terms of the GNU Free Documentation License, Version 1.3 + or any later version published by the Free Software Foundation; + with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. + A copy of the license is included in the section entitled "GNU + Free Documentation License". + + +Exports +******* + +Export is an important issue for the Chill software : users should be able to : + +- compute statistics about their activity ; +- list "things" which make part of their activities. + +The `main bundle`_ provides a powerful framework to build custom queries with re-usable parts across differents bundles. + +.. contents:: Table of content + :local: + +.. seealso:: + + `The issue where this framework was discussed `_ + Provides some information about the pursued features and architecture. + +Concepts +======== + + +Some vocabulary: 3 "Export elements" +------------------------------------ + +Three terms are used for this framework : + +exports + provides some basic operation on the date. Two kind of exports are available : + + - computed data : it may be "the number of people", "the number of activities", "the duration of activities", ... + - list data : it may be "the list of people", "the list of activity", ... + +filters + The filters make a filter on the date: it removes some information the user doesn't want to introduce in the computation done by export. In other word, filters make a filter... + + Example of filter: "people under 18 years olds", "activities between the 1st of June and the 31st December", ... + +aggregators + The aggregator aggregates the data into some group (some software use the term 'bucket'). + + Example of aggregator : "group people by gender", "group people by nationality", "group activity by type", ... + + +Anatomy of an export +--------------------- + +An export may be explained as a sentence, where each part of this sentence refers to one or multiple exports element. Examples : + +**Example 1**: Count the number of people having at least one activity in the last 12 month, and group them by nationality and gender + +Here : + +- *count the number of people* is the export part +- *having at least one activity* is the filter part +- *group them by nationality* is the aggregator part +- *group them by gender* is a second aggregator part + +Note that : + +- aggregators, filters and exports are cross-bundle. Here the bundle activity provides a filter which apply on an export provided by the person bundle ; +- there may exists multiple aggregator or filter for one export. Currently, only one export is allowed. + +The result might be : + ++-----------------------+----------------+---------------------------+ +| Nationality | Gender | Number of people | ++=======================+================+===========================+ +| Russian | Male | 12 | ++-----------------------+----------------+---------------------------+ +| Russian | Female | 24 | ++-----------------------+----------------+---------------------------+ +| France | Male | 110 | ++-----------------------+----------------+---------------------------+ +| France | Female | 150 | ++-----------------------+----------------+---------------------------+ + +**Example 2**: Count the average duration of an activity with type "meeting", which occurs between the 1st of June and the 31st of December, and group them by week. + +Here : + +- *count the average duration of an activity* is the export part +- *activity with type meeting* is a filter part +- *activity which occurs between the 1st of June and the 31st of December* is a filter +- *group them by week* is the aggregator part + +The result might be : + ++-----------------------+----------------------+ +| Week | Number of activities | ++=======================+======================+ +| 2015-10 | 10 | ++-----------------------+----------------------+ +| 2015-11 | 12 | ++-----------------------+----------------------+ +| 2015-12 | 10 | ++-----------------------+----------------------+ +| 2015-13 | 9 | ++-----------------------+----------------------+ + +Authorization and exports +------------------------- + +Exports, filters and aggregators should not make see data the user is not allowed to see. + +In other words, developers are required to take care of user authorization for each export. + +It should exists a special role that should be granted to users which are allowed to build exports. For more simplicity, this role should apply on center, and should not requires special circles. + +How the magic works +=================== + +To build an export, we rely on the capacity of the database to execute queries with aggregate (i.e. GROUP BY) and filter (i.e. WHERE) instructions. + +An export is an SQL query which is initiated by an export, and modified by aggregators and filters. + +.. note:: + + **Example 1**: Count the number of people having at least one activity in the last 12 month, and group them by nationality and gender + + 1. The report initiate the query :code:`SELECT count(people.*) FROM people` + 2. The filter add a where and join clause : :code:`SELECT count(people.*) FROM people RIGHT JOIN activity WHERE activity.date IS BETWEEN now AND 6 month ago` + 3. The aggregator "nationality" add a GROUP BY clause and a column in the SELECT statement: :code:`SELECT people.nationality, count(people.*) FROM people RIGHT JOIN activity WHERE activity.date IS BETWEEN now AND 6 month ago GROUP BY nationality` + 4. The aggregator "gender" do the same job as the nationality aggregator : it adds a GROUP BY clause and a column in the SELECT statement : :code:`SELECT people.nationality, people.gender, count(people.*) FROM people RIGHT JOIN activity WHERE activity.date IS BETWEEN now AND 6 month ago GROUP BY nationality, gender` + + +.. todo:: + + Continue to explain the export framework + +.. _main bundle: https://git.framasoft.org/Chill-project/Chill-Main diff --git a/source/development/index.rst b/source/development/index.rst index b3e59b134..b480425fe 100644 --- a/source/development/index.rst +++ b/source/development/index.rst @@ -25,6 +25,7 @@ As Chill rely on the `symfony `_ framework, reading the fram Database migrations Searching Timelines + Exports Testing manual/index.rst