More complex questionnaire setups

.gitignore

@@ -4,6 +4,7 @@
 /tests/StackName.stack
 /tests/ChainName.chain
 /tests/ClusterName.cluster
+dist/
 
 .coverage
 htmlcov/
@@ -23,18 +24,19 @@ temp.chain
 tests/Example Data (A).cache
 *.css
 *.ttf
+*.js
 *.eot
 *.svg
 *.woff
+
 *.js
 
-docs/API/_build/
 docs/API/_static/
 docs/API/_templates/
 
 *.egg-info
 *.egg
 .eggs/
 .cache
-
-build
+.pytest_cache/*
+build

.pytest_cache/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/latest/cache.html) for more information.

README.md

@@ -14,7 +14,10 @@ Quantipy is an open-source data processing, analysis and reporting software proj
 
 #### Contributors
 - Alexander Buchhammer, Alasdair Eaglestone, James Griffiths, Kerstin Müller : https://yougov.co.uk
-- Datasmoothie’s Birgir Hrafn Sigurðsson and Geir Freysson: http://datasmoothie.io/
+- Datasmoothie’s Birgir Hrafn Sigurðsson and Geir Freysson: http://datasmoothie.com/
+
+### Python 3 compatability
+Efforts are underway to port Quantipy to Python 3 in a [seperate repository](https://www.github.com/quantipy/quantipy3).
 
 ## Docs
 [View the documentation at readthedocs.org](http://quantipy.readthedocs.io/)

docs/API/_build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 0288e8257f20bf0e318a1e13f4f950ad
+config: 3c8d31b3ee2b61e092a93342a4c7e2d0
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/API/_build/html/_sources/sites/api_ref/00overview.rst.txt

@@ -0,0 +1,16 @@
+
+API references
+==============
+
+.. toctree::
+   :maxdepth: 3
+
+   Chain
+   Cluster
+   DataSet
+   quantify_engine
+   QuantipyViews
+   Rim_scheme
+   Stack
+   View
+   ViewMapper

docs/API/_build/html/_sources/sites/api_ref/DataSet.rst.txt

@@ -0,0 +1,8 @@
+.. toctree::
+   :maxdepth: 3
+
+DataSet
+=======
+
+.. autoclass:: quantipy.DataSet
+	:members:

docs/API/_build/html/_sources/sites/lib_doc/batch/00_overview.rst.txt

@@ -0,0 +1,35 @@
+
+=========
+``Batch``
+=========
+
+``qp.Batch`` is a subclass of ``qp.DataSet`` and is a container for
+structuring a ``qp.Link`` collection's specifications.
+
+``qp.Batch`` is not only a subclass of ``qp.DataSet``, it also takes a
+DataSet instance as input argument, inheriting  a few of its attributes, e.g.
+``_meta``, ``_data``, ``valid_tks`` and ``text_key``.
+All other ``Batch`` attributes are used as construction plans for populating a
+``qp.Stack``, these get stored in the belonging ``DataSet`` meta component in
+``_meta['sets']['batches'][batchname]``.
+
+In general, it does not matter in which order ``Batch`` attributes are set by
+methods, the class ensures that all attributes are kept consistent.
+
+All next sections are working with the following ``qp.DataSet`` instance::
+
+	import quantipy as qp
+
+	dataset = qp.DataSet('Example Data (A)')
+	dataset.read_quantipy('Example Data (A).json', 'Example Data (A).csv')
+
+The json and csv files you can find in ``quantipy/tests``.
+
+.. toctree::
+  :maxdepth: 5
+  :includehidden:
+
+  01_create_load
+  02_variables
+  03_properties
+  04_subclass

docs/API/_build/html/_sources/sites/lib_doc/batch/01_create_load.rst.txt

@@ -0,0 +1,27 @@
+.. toctree::
+  :maxdepth: 5
+  :includehidden:
+
+=========================================
+Creating/ Loading a ``qp.Batch`` instance
+=========================================
+
+As mentioned, a ``Batch`` instance has a close connection to its belonging
+``DataSet`` instance and we can easily create a new ``Batch`` from a ``DataSet``
+as per::
+
+	batch1 = dataset.add_batch(name='batch1')
+	batch2 = dataset.add_batch(name='batch2', ci=['c'], weights='weight')
+
+It is also possible to load an already existing instance out of the meta
+stored in ``dataset._meta['sets']['batches']``::
+
+	batch = dataset.get_batch('batch1')
+
+Both methods, ``.add_batch()`` and ``.get_batch()``, are an easier way to
+use the ``__init__()`` method of ``qp.Batch``.
+
+An other way to get a new ``qp.Batch`` instance is to copy an existing one, in
+that case all added open ends are removed from the new instance::
+
+	copy_batch = batch.copy('copy_of_batch1')

docs/API/_build/html/_sources/sites/lib_doc/batch/02_variables.rst.txt

@@ -0,0 +1,119 @@
+.. toctree::
+  :maxdepth: 5
+  :includehidden:
+
+===========================================
+Adding variables to a ``qp.Batch`` instance
+===========================================
+
+-----------------
+x-keys and y-keys
+-----------------
+
+The included variables in a ``Batch`` constitute the main structure for the
+``qp.Stack`` construction plan. Variables can be added as x-keys or y-keys, for
+arrays all belonging items are automatically added and the ``qp.Stack`` gets
+populated with all cross-tabulations of these keys:
+
+>>> batch.add_x(['q1', 'q2', 'q6'])
+>>> batch.add_y(['gender', 'q1'])
+Array summaries setup: Creating ['q6'].
+
+x-specific y-keys can be produced by manipulating the main y-keys, this edit
+can be extending or replacing the existing keys:
+
+>>> batch.extend_y(['locality', 'ethnicity'], on=['q1'])
+>>> batch.replace_y(['locality', 'ethnicity'], on=['q2'])
+
+With these settings the construction plan looks like that:
+
+>>> print batch.x_y_map
+OrderedDict([('q1', ['@', 'gender', 'q1', 'locality', 'ethnicity']), 
+             ('q2', ['locality', 'ethnicity']), 
+             ('q6', ['@']), 
+             (u'q6_1', ['@', 'gender', 'q1']), 
+             (u'q6_2', ['@', 'gender', 'q1']), 
+             (u'q6_3', ['@', 'gender', 'q1'])])
+
+------
+Arrays
+------
+
+A special case exists if the added variables contain arrays. As default for all
+arrays in x-keys array summaries are created (array as x-key and ``'@'``-referenced total as
+y-key), see the output below (``Array summaries setup: Creating ['q6'].``).
+If array summaries are requested only for a selection of variables or for none,
+use ``.make_summaries()``:
+
+>>> batch.make_summaries(None)
+Array summaries setup: Creating no summaries!
+
+Arrays can also be transposed (``'@'``-referenced total as x-key and array name
+as y-key). If they are not in the batch summary list before, they are
+automatically added and depending on the ``replace`` parameter only the
+transposed or both types of summaries are added to the ``qp.Stack``:
+
+>>> batch.transpose_array('q6', replace=False)
+Array summaries setup: Creating ['q6'].
+
+The construction plan now shows that both summary types are included:
+
+>>> print batch.x_y_map
+OrderedDict([('q1', ['@', 'gender', 'q1', 'locality', 'ethnicity']),
+             ('q2', ['locality', 'ethnicity']),
+             ('q6', ['@']),
+             ('@', ['q6']),
+             (u'q6_1', ['@', 'gender', 'q1']), 
+             (u'q6_2', ['@', 'gender', 'q1']), 
+             (u'q6_3', ['@', 'gender', 'q1'])])
+
+--------------------
+Verbatims/ open ends
+--------------------
+
+Another special case are verbatims. They will not be aggregated in a ``qp.Stack``,
+but they have to be defined in a ``qp.Batch`` to add them later to a ``qp.Cluster``.
+
+There are two different ways to add verbatims: Either all to one ``qp.Cluster``
+key or each gets its own key. But both options can be done with the same method.
+
+For splitting the verbatims, set ``split=True`` and insert as many titles as
+included verbatims/ open ends:
+
+>>> batch.add_open_ends(['q8a', 'q9a'], break_by=['record_number', 'age'],
+						split=True, title=['oe_q8', 'oe_q9'])
+
+For collecting all verbatims in one Cluster key, set ``split=False`` and add
+only one ``title`` or use the default parameters:
+
+>>> batch.add_open_ends(['q8a', 'q9a'], break_by=['record_number', 'age'])
+
+--------------------
+Special aggregations
+--------------------
+
+It is possible to add some special aggregations to a ``qp.Batch``, that are
+not stored in the main construction plan ``.x_y_map``. One option is to give a
+name for a Cluster key in which all y-keys are cross-tabulated against each
+other:
+
+>>> batch.add_y_on_y('y-keys')
+
+Another possibility is to add a ``qp.Batch`` instance to an other instance.
+The added Batch loses all information about verbatims and ``.y_on_y``, that
+means only the main construction plan in ``.x_y_map`` gets adopted. Each of
+the two batches is aggregated discretely in the ``qp.Stack``, but the added
+instance gets included into the ``qp.Cluster`` of the first ``qp.Batch`` in
+a key named by its instance name.
+
+>>> batch1 = dataset.get_batch('batch1')
+>>> batch2 = dataset.get_batch('batch2')
+>>> batch2.add_x('q2b')
+Array summaries setup: Creating no summaries!
+>>> batch2.add_y('gender')
+>>> batch2.as_addition('batch1')
+Batch 'batch2' specified as addition to Batch 'batch1'. Any open end summaries and 'y_on_y' agg. have been removed!
+
+The connection between the two ``qp.Batch`` instances you can see in ``.additional``
+for the added instance and in ``._meta['sets']['batches']['batchname']['additions']``
+for the first instance.

docs/API/_build/html/_sources/sites/lib_doc/batch/03_properties.rst.txt

@@ -0,0 +1,58 @@
+.. toctree::
+  :maxdepth: 5
+  :includehidden:
+
+================================
+Set properties of a ``qp.Batch``
+================================
+
+The section before explained how the main construction plan (``batch.x_y_map``)
+is built, that describes which x-keys and y-keys are used to add ``qp.Link``\s
+to a ``qp.Stack``. Now you will get to know how the missing information for the
+``Link``\s are defined and which specific views get extracted for the
+``qp.Cluster`` by adding some property options the ``qp.Batch`` instance.
+
+----------------------------------------
+Filter, weights and significance testing
+----------------------------------------
+
+``qp.Link``\s can be added to a ``qp.Stack`` data_key-level by defining its x
+and y-keys, which is already done in ``.x_y_map``, and setting a filter.
+This property can be edited in a ``qp.Batch`` instance with the
+following methods:
+
+>>> batch.add_filter('men only', {'gender': 1})
+>>> batch.extend_filter({'q1': {'age': [20, 21, 22, 23, 24, 25]}})
+
+Filters can be added globally or for a selection of x-keys only. Out of the
+global filter, ``.sample_size`` is automatically calculated for each ``qp.Batch``
+defintion.
+
+Now all information are collected in the ``qp.Batch`` instance and the ``Stack``
+can be populated with ``Link``\s in form of ``stack[data_key][filter_key][x_key][y_key]``.
+
+For each ``Link`` ``qp.View``\s can be added, these views depend on a weight
+definition, which is also defined in the ``qp.Batch``:
+
+>>> batch.set_weights(['weight_a'])
+
+Significance tests are a special ``View``; the sig. levels which they are
+calculated on can be added to the ``qp.Batch`` like this:
+
+>>> batch.set_sigtests(levels=[0.05])
+
+-----------------------
+Cell items and language
+-----------------------
+
+As ``qp.Stack`` is a container for a large amount of aggregations, it will
+accommodate various ``qp.View``\s. The ``qp.Batch`` property ``.cell_items`` is
+used to define which specfic ``Views`` will be taken to create a ``qp.Cluster``:
+
+>>> batch.set_cell_items(['c', 'p'])
+
+The property ``.language`` allows the user to define which ``text`` labels from
+the meta data should be used for the extracted ``Views`` by entering a valid
+text key:
+
+>>> batch.set_language('en-GB')

docs/API/_build/html/_sources/sites/lib_doc/batch/04_subclass.rst.txt

@@ -0,0 +1,26 @@
+.. toctree::
+  :maxdepth: 5
+  :includehidden:
+
+================================
+Inherited ``qp.DataSet`` methods
+================================
+
+Being a ``qp.DataSet`` subclasss, ``qp.Batch`` inherits some of its methods.
+The important ones are these which allow the manipulation of the meta component.
+That means meta-edits can be applied globally (run methods on ``qp.DataSet``) or
+``Batch``-specific (run methods on ``qp.Batch``). Batch meta-edits
+always overwrite global meta-edits and while building a ``qp.Cluster`` from a
+``qp.Batch``, the modified meta information is taken from ``.meta_edits``.
+
+The following methods can be used to create meta-edits for a ``qp.Batch``:
+
+>>> batch.hiding('q1', [2], axis='y')
+>>> batch.sorting('q2', fix=[97, 98])
+>>> batch.slicing('q1', [1, 2, 3, 4, 5], axis='x')
+>>> batch.set_variable_text('gender', 'Gender???')
+>>> batch.set_value_texts('gender', {1: 'Men', 2: 'Women'})
+>>> batch.set_property('q1', 'base_text', 'This var has a second filter.')
+
+Some methods are not allowed to be used for a ``Batch``. These will raise a
+``NotImplementedError`` to prevent inconsistent case and meta data states.

docs/API/_build/html/_sources/sites/lib_doc/builds/00_overview.rst.txt

@@ -0,0 +1,9 @@
+======
+Builds
+======
+
+.. toctree::
+	:maxdepth: 5
+	:includehidden:
+
+	01_chains

docs/API/_build/html/_sources/sites/lib_doc/builds/01_chains.rst.txt

@@ -0,0 +1,21 @@
+.. toctree::
+ 	:maxdepth: 5
+	:includehidden:
+
+=================
+Combining results
+=================
+
+--------------------------------
+Organizing ``View`` aggregations
+--------------------------------
+
+-------------------------------
+Creating ``Chain`` aggregations
+-------------------------------
+
+What is a ``Chain``?
+--------------------
+
+Customizing results
+-------------------