From 0afa1a07fdd76b0a3f6e0ab4469aa7d866fbe7ab Mon Sep 17 00:00:00 2001 From: Kim Scott Date: Wed, 29 Mar 2017 15:54:27 -0400 Subject: [PATCH 1/8] update documentation to example of nesting randomizers --- docs/source/experiments.md | 120 ++++++++++++++++++++++++++++++++++++- 1 file changed, 119 insertions(+), 1 deletion(-) diff --git a/docs/source/experiments.md b/docs/source/experiments.md index 030b486..42ab917 100644 --- a/docs/source/experiments.md +++ b/docs/source/experiments.md @@ -293,7 +293,125 @@ But what we really want to do is have some kids think about how tasty broccoli i Notice that since both of the frames in the `frameList` were of the same kind, we could define the kind in `commonFrameProperties`. We no longer define `id` values for the frames, as they will be automatically identified as `instruct-and-manip-1` and `instruct-and-manip-2`. -If we wanted to have 75% of participants think about how tasty broccoli is, we could also weight the parameter sets by providing the optional parameter `"parameterSetWeights": [3, 1]"` to the randomizer frame. One use of this function is to stop testing conditions that you already have enough children in as data collection proceeds. +If we wanted to have 75% of participants think about how tasty broccoli is, we could also weight the parameter sets by providing the optional parameter `"parameterSetWeights": [3, 1]"` to the randomizer frame. + +> Note: One use of parameterSetWeights is to stop testing conditions that you already have enough children in as data collection proceeds. + +#### Nested randomizers + +The frame list you provide to the randomParameterSet randomizer can even include other randomizer frames! This allows you to, for instance, define a **trial** that includes several distinct **blocks** (say, an intro, video, and then 4 test questions), then show 10 of those trials with different parameters - without having to write out all 60 blocks. There's nothing "special" about doing this, but it can be a little more confusing. + +Here's an example. Notice that `"kind": "choice"`, `"sampler": "random-parameter-set"`, `"frameList": ...`, and `commonFrameProperties` are `commonFrameProperties` of the outer frame `nested-trials`. That means that every "frame" we'll create as part of `nested-trials` will itself be a random-parameter-set generated list with the same frame sequence, although we'll be substituting in different parameter values. (This doesn't have to be the case - we could show different types of frames in the list - but in the simplest case where you're using randomParameterSet just to group similar repeated frame sequences, this is probably what you'd do.) The only thing that differs across the two (outer-level) **trials** is the `parameterSet` used, and we list only one parameter set for each trial, to describe (deterministically) how the outer-level `parameterSet` values should be applied to each particular frame. + +```json + "nested-trials": { + "kind": "choice", + "sampler": "random-parameter-set", + "commonFrameProperties": { + "kind": "choice", + "sampler": "random-parameter-set", + "frameList": [ + { + "nPhase": 0, + "doRecording": false, + "autoProceed": false, + "parentTextBlock": { + "title": "Parents!", + "text": "Phase 0: instructions", + "emph": true + }, + "images": [ + { + "id": "protagonist", + "src": "PROTAGONISTFACELEFT", + "left": "40", + "bottom": "2", + "height": "60", + "animate": "fadein" + } + ], + "audioSources": [ + { + "audioId": "firstAudio", + "sources": [{"stub": "0INTRO"}] + } + ] + }, + { + "nPhase": 1, + "doRecording": false, + "autoProceed": false, + "parentTextBlock": { + "title": "Parents!", + "text": "Phase 1: instructions", + "emph": true + }, + "images": [ + { + "id": "protagonist", + "src": "PROTAGONISTFACELEFT", + "left": "40", + "bottom": "2", + "height": "60" + } + ], + "audioSources": [ + { + "audioId": "firstAudio", + "sources": [{"stub": "1INTRO"}] + } + ] + } + ], + "commonFrameProperties": { + "kind": "exp-lookit-dialogue-page", + "doRecording": true, + "nTrial": "NTRIAL", + "backgroundImage": "BACKGROUNDIMG", + "baseDir": "https://s3.amazonaws.com/lookitcontents/politeness/", + "audioTypes": ["mp3", "ogg"] + } + }, + "frameList": [ + { + "parameterSets": [ + { + "PROTAGONISTFACELEFT": "PROTAGONISTFACELEFT_1", + "BACKGROUNDIMG": "BACKGROUNDIMG_1", + "0INTRO": "0INTRO_1", + "1INTRO": "1INTRO_1", + "NTRIAL": 1 + } + ] + }, + { + "parameterSets": [ + { + "PROTAGONISTFACELEFT": "PROTAGONISTFACELEFT_2", + "BACKGROUNDIMG": "BACKGROUNDIMG_2", + "0INTRO": "0INTRO_2", + "1INTRO": "1INTRO_2", + "NTRIAL": 2 + } + ] + } + ], + "parameterSets": [ + { + "PROTAGONISTFACELEFT_1": "order1_test1_listener1.png", + "PROTAGONISTFACELEFT_2": "order1_test1_listener1_second.png", + "BACKGROUNDIMG_1": "order1_test1_background.png", + "BACKGROUNDIMG_2": "order1_test1_background.png", + "0INTRO_1": "polcon_example_1intro", + "1INTRO_1": "polcon_example_1intro", + "0INTRO_2": "polcon_example_1intro", + "1INTRO_2": "polcon_example_1intro" + } + ] + } +``` + + ### Testing your study From a64cd53dddb3867a5696ae280d5b171be11c9b73 Mon Sep 17 00:00:00 2001 From: Kim Scott Date: Fri, 31 Mar 2017 16:45:01 -0400 Subject: [PATCH 2/8] update documentation to describe condition information stored by randomizers (where to find randomizer-specific documentation) --- docs/source/experiments.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/source/experiments.md b/docs/source/experiments.md index 42ab917..603a6bb 100644 --- a/docs/source/experiments.md +++ b/docs/source/experiments.md @@ -522,17 +522,23 @@ And descriptions of these properties are enumerated below: #### Sessions and `expData` in detail -Continuing with the example from above, lets walk through the data collected during a session (note: some fields are hidden): +Lets walk through an example of data collected during a session (note: some fields are hidden): ```json { "sequence": [ "0-intro-video", - "1-survey-2", + "1-survey", "2-exit-survey" ], "conditions": { - "survey-randomizer": 1 + "1-survey": { + "parameterSet": { + "QUESTION1": "What is your favorite color?", + "QUESTION2": "What is your favorite number?" + }, + "conditionNum": 0 + } }, "expData": { "0-intro-video": { @@ -541,7 +547,7 @@ Continuing with the example from above, lets walk through the data collected dur "timestamp": "2016-03-23T16:28:20.753Z" }] }, - "1-survey-2": { + "1-survey": { "formData": { "name": "Sam", "favPie": "pecan" @@ -570,10 +576,10 @@ Things to note: the overall sequence where this **frame** appeared, and `` is the identifier of the frame as defined in the 'frames' property of the experiment structure. Notice in particular that since 'survey-2' was randomly selected, it appears here. -- 'conditions' has the key/value pair `"1-survey-randomizer": 1`, where the format `-` corresponds +- 'conditions' has the key/value pair `"1-survey": 1`, where the format `-` corresponds with the `` from the 'sequence' of the *original* experiment structure, and the `` again corresponds with the identifier of the frame as defined in - the 'frames' property of the experiment structure. + the 'frames' property of the experiment structure. Data will be stored in conditions for the first frame created by a randomizer (top-level only for now, i.e. not from nested randomizers). The data stored by a particular randomizer can be found under `methods: conditions` in the [randomizer documentation](http://centerforopenscience.github.io/exp-addons/modules/randomizers.html) - 'expData' is an object with three properties (corresponding with the values from 'sequence'). Each of these objects has an 'eventTimings' property. This is a place to collect user-interaction events during an experiment, and by default contains the 'nextFrame' event which records when the user progressed to the next **frame** in the 'sequence'. You can see which events a particular frame records by looking at the 'Events' tab in its [frame documentation](http://centerforopenscience.github.io/exp-addons/modules/frames.html). Other properties besides 'eventTimings' are dependent on the **frame** type. You can see which properties a particular frame type records by looking at the parameters of the `serializeContent` method under the 'Methods' tab in its [frame documentation](http://centerforopenscience.github.io/exp-addons/modules/frames.html). Notice that 'exp-video' captures no data, and that both 'exp-survey' **frames** capture a 'formData' object. From a6e874e743112f0ad102d54325ce84b2be4b48fb Mon Sep 17 00:00:00 2001 From: Kim Scott Date: Fri, 31 Mar 2017 16:48:02 -0400 Subject: [PATCH 3/8] split experiment doc into experiment-creation and experiment-data --- docs/source/experimentdata.md | 170 ++++++++++++++++++++++++++++++++++ docs/source/experiments.md | 167 --------------------------------- docs/source/index.rst | 1 + 3 files changed, 171 insertions(+), 167 deletions(-) create mode 100644 docs/source/experimentdata.md diff --git a/docs/source/experimentdata.md b/docs/source/experimentdata.md new file mode 100644 index 0000000..187061f --- /dev/null +++ b/docs/source/experimentdata.md @@ -0,0 +1,170 @@ +# Experiment data + +### Accessing experiment data + +You can see and download collected data from sessions marked as 'completed' (user filled out the exit survey) directly from the Experimenter application. + +You can also download JSON study or accounts data from the command line using the python script [experimenter.py](https://github.com/CenterForOpenScience/lookit/blob/develop/scripts/experimenter.py) ([description](https://github.com/CenterForOpenScience/lookit/pull/85)); you'll need to set up a file config.json with the following content: + +```json +{ + "host": "https://staging-metadata.osf.io", + "namespace": "lookit", + "osf_token": "YOUR_OSF_TOKEN_HERE" +} + +``` + +You can create an OSF token for the staging server [here](https://staging.osf.io/settings/tokens/). + +The collection name to use to get study session records is `session[STUDYIDHERE]s`, e.g. `session58d015243de08a00400316e0s`. + +### Structure of session data + +The data saved when a subject participates in a study varies based on how that experiment is defined. The general structure for this **session** data is: + +```json +{ + "type": "object", + "properties": { + "profileId": { + "type": "string", + "pattern": "\w+\.\w+" + }, + "experimentId": { + "type": "string", + "pattern": "\w+" + }, + "experimentVersion": { + "type": "string" + }, + "completed": { + "type": "boolean" + }, + "sequence": { + "type": "array", + "items": { + "type": "string" + } + }, + "conditions": { + "type": "object" + }, + "expData": { + "type": "object" + }, + "feedback": { + "$oneOf": [{ + "type": "string" + }, null] + }, + "hasReadFeedback": { + "$oneOf": [{ + "type": "boolean" + }, null] + }, + "globalEventTimings": { + "type": "array", + "items": { + "type": "object" + } + } + }, + "required": [ + "profileId", + "experimentId", + "experimentVersion", + "completed", + "sequence", + "expData" + ] +} +``` + +And descriptions of these properties are enumerated below: + +- *profileId*: This unique identifier of the participant. This field follows the form: `.`, where `` is the unique identifier of the associated account, and `` is the unique identifier of the profile active during this particular session (e.g. the participating child). Account data is stored in a separate database, and includes demographic survey data and the list of profiles associated with the account. +- *experimentId*: The unique identifier of the study the subject participated in. +- *experimentVersion*: The unique identifier of the version of the study the subject participated in. TODO: more on JamDB, versioning +- *completed*: A true/false flag indicating whether or not the subject completed the study. +- *sequence*: The sequence of **frames** the subject actually saw (after running randomization, etc.) +- *conditions*: For randomizers, this records what condition the subject was assigned +- *expData*: A JSON object containing the data collected by each **frame** in the study. More on this to follow. +- *feedback*: Some researchers may have a need to leave some session-specific feedback for a subject; this is shown to the participant in their 'completed studies' view. +- *hasReadFeedback*: A true/false flag to indicate whether or not the given feedback has been read. +- *globalEventTimings*: A list of events recorded during the study, not tied to a particular frame. Currently used for recording early exit from the study; an example value is + +```json +"globalEventTimings": [ + { + "exitType": "browserNavigationAttempt", + "eventType": "exitEarly", + "lastPageSeen": 0, + "timestamp": "2016-11-28T20:00:13.677Z" + } + ] +``` + +### Sessions and `expData` in detail + +Lets walk through an example of data collected during a session (note: some fields are hidden): + +```json +{ + "sequence": [ + "0-intro-video", + "1-survey", + "2-exit-survey" + ], + "conditions": { + "1-survey": { + "parameterSet": { + "QUESTION1": "What is your favorite color?", + "QUESTION2": "What is your favorite number?" + }, + "conditionNum": 0 + } + }, + "expData": { + "0-intro-video": { + "eventTimings": [{ + "eventType": "nextFrame", + "timestamp": "2016-03-23T16:28:20.753Z" + }] + }, + "1-survey": { + "formData": { + "name": "Sam", + "favPie": "pecan" + }, + "eventTimings": [{ + "eventType": "nextFrame", + "timestamp": "2016-03-23T16:28:26.925Z" + }] + }, + "2-exit-survey": { + "formData": { + "thoughts": "Great!", + "wouldParticipateAgain": "Yes" + }, + "eventTimings": [{ + "eventType": "nextFrame", + "timestamp": "2016-03-23T16:28:32.339Z" + }] + } + } +} +``` + +Things to note: +- 'sequence' has resolved to three items following the pattern `-`, where `` is the order in + the overall sequence where this **frame** appeared, and `` is the identifier of the frame as defined in + the 'frames' property of the experiment structure. Notice in particular that since 'survey-2' was randomly selected, + it appears here. +- 'conditions' has the key/value pair `"1-survey": 1`, where the format `-` corresponds + with the `` from the 'sequence' of the *original* experiment structure, and the `` again corresponds + with the identifier of the frame as defined in + the 'frames' property of the experiment structure. Data will be stored in conditions for the first frame created by a randomizer (top-level only for now, i.e. not from nested randomizers). The data stored by a particular randomizer can be found under `methods: conditions` in the [randomizer documentation](http://centerforopenscience.github.io/exp-addons/modules/randomizers.html) +- 'expData' is an object with three properties (corresponding with the values from 'sequence'). Each of these objects has an 'eventTimings' property. This is a place to collect user-interaction events during an experiment, and by default contains the 'nextFrame' event which records when the + user progressed to the next **frame** in the 'sequence'. You can see which events a particular frame records by looking at the 'Events' tab in its [frame documentation](http://centerforopenscience.github.io/exp-addons/modules/frames.html). Other properties besides 'eventTimings' are dependent on + the **frame** type. You can see which properties a particular frame type records by looking at the parameters of the `serializeContent` method under the 'Methods' tab in its [frame documentation](http://centerforopenscience.github.io/exp-addons/modules/frames.html). Notice that 'exp-video' captures no data, and that both 'exp-survey' **frames** capture a 'formData' object. diff --git a/docs/source/experiments.md b/docs/source/experiments.md index 603a6bb..f92334d 100644 --- a/docs/source/experiments.md +++ b/docs/source/experiments.md @@ -416,170 +416,3 @@ Here's an example. Notice that `"kind": "choice"`, `"sampler": "random-parameter ### Testing your study Experimenter has a built-in tool that allows you to try out your study. However, some functionality may not be exactly the same as on Lookit. We recommend testing your study from Lookit, which will be how participants experience it. - -### Experiment data - -You can see and download collected data from sessions marked as 'completed' (user filled out the exit survey) directly from the Experimenter application. - -You can also download JSON study or accounts data from the command line using the python script [experimenter.py](https://github.com/CenterForOpenScience/lookit/blob/develop/scripts/experimenter.py) ([description](https://github.com/CenterForOpenScience/lookit/pull/85)); you'll need to set up a file config.json with the following content: - -```json -{ - "host": "https://staging-metadata.osf.io", - "namespace": "lookit", - "osf_token": "YOUR_OSF_TOKEN_HERE" -} - -``` - -You can create an OSF token for the staging server [here](https://staging.osf.io/settings/tokens/). - -The collection name to use to get study session records is `session[STUDYIDHERE]s`, e.g. `session58d015243de08a00400316e0s`. - -The data saved when a subject participates in a study varies based on how that experiment is defined. The general structure for this **session** data is: - -```json -{ - "type": "object", - "properties": { - "profileId": { - "type": "string", - "pattern": "\w+\.\w+" - }, - "experimentId": { - "type": "string", - "pattern": "\w+" - }, - "experimentVersion": { - "type": "string" - }, - "completed": { - "type": "boolean" - }, - "sequence": { - "type": "array", - "items": { - "type": "string" - } - }, - "conditions": { - "type": "object" - }, - "expData": { - "type": "object" - }, - "feedback": { - "$oneOf": [{ - "type": "string" - }, null] - }, - "hasReadFeedback": { - "$oneOf": [{ - "type": "boolean" - }, null] - }, - "globalEventTimings": { - "type": "array", - "items": { - "type": "object" - } - } - }, - "required": [ - "profileId", - "experimentId", - "experimentVersion", - "completed", - "sequence", - "expData" - ] -} -``` - -And descriptions of these properties are enumerated below: - -- *profileId*: This unique identifier of the participant. This field follows the form: `.`, where `` is the unique identifier of the associated account, and `` is the unique identifier of the profile active during this particular session (e.g. the participating child). Account data is stored in a separate database, and includes demographic survey data and the list of profiles associated with the account. -- *experimentId*: The unique identifier of the study the subject participated in. -- *experimentVersion*: The unique identifier of the version of the study the subject participated in. TODO: more on JamDB, versioning -- *completed*: A true/false flag indicating whether or not the subject completed the study. -- *sequence*: The sequence of **frames** the subject actually saw (after running randomization, etc.) -- *conditions*: For randomizers, this records what condition the subject was assigned -- *expData*: A JSON object containing the data collected by each **frame** in the study. More on this to follow. -- *feedback*: Some researchers may have a need to leave some session-specific feedback for a subject; this is shown to the participant in their 'completed studies' view. -- *hasReadFeedback*: A true/false flag to indicate whether or not the given feedback has been read. -- *globalEventTimings*: A list of events recorded during the study, not tied to a particular frame. Currently used for recording early exit from the study; an example value is - -```json -"globalEventTimings": [ - { - "exitType": "browserNavigationAttempt", - "eventType": "exitEarly", - "lastPageSeen": 0, - "timestamp": "2016-11-28T20:00:13.677Z" - } - ] -``` - -#### Sessions and `expData` in detail - -Lets walk through an example of data collected during a session (note: some fields are hidden): - -```json -{ - "sequence": [ - "0-intro-video", - "1-survey", - "2-exit-survey" - ], - "conditions": { - "1-survey": { - "parameterSet": { - "QUESTION1": "What is your favorite color?", - "QUESTION2": "What is your favorite number?" - }, - "conditionNum": 0 - } - }, - "expData": { - "0-intro-video": { - "eventTimings": [{ - "eventType": "nextFrame", - "timestamp": "2016-03-23T16:28:20.753Z" - }] - }, - "1-survey": { - "formData": { - "name": "Sam", - "favPie": "pecan" - }, - "eventTimings": [{ - "eventType": "nextFrame", - "timestamp": "2016-03-23T16:28:26.925Z" - }] - }, - "2-exit-survey": { - "formData": { - "thoughts": "Great!", - "wouldParticipateAgain": "Yes" - }, - "eventTimings": [{ - "eventType": "nextFrame", - "timestamp": "2016-03-23T16:28:32.339Z" - }] - } - } -} -``` - -Things to note: -- 'sequence' has resolved to three items following the pattern `-`, where `` is the order in - the overall sequence where this **frame** appeared, and `` is the identifier of the frame as defined in - the 'frames' property of the experiment structure. Notice in particular that since 'survey-2' was randomly selected, - it appears here. -- 'conditions' has the key/value pair `"1-survey": 1`, where the format `-` corresponds - with the `` from the 'sequence' of the *original* experiment structure, and the `` again corresponds - with the identifier of the frame as defined in - the 'frames' property of the experiment structure. Data will be stored in conditions for the first frame created by a randomizer (top-level only for now, i.e. not from nested randomizers). The data stored by a particular randomizer can be found under `methods: conditions` in the [randomizer documentation](http://centerforopenscience.github.io/exp-addons/modules/randomizers.html) -- 'expData' is an object with three properties (corresponding with the values from 'sequence'). Each of these objects has an 'eventTimings' property. This is a place to collect user-interaction events during an experiment, and by default contains the 'nextFrame' event which records when the - user progressed to the next **frame** in the 'sequence'. You can see which events a particular frame records by looking at the 'Events' tab in its [frame documentation](http://centerforopenscience.github.io/exp-addons/modules/frames.html). Other properties besides 'eventTimings' are dependent on - the **frame** type. You can see which properties a particular frame type records by looking at the parameters of the `serializeContent` method under the 'Methods' tab in its [frame documentation](http://centerforopenscience.github.io/exp-addons/modules/frames.html). Notice that 'exp-video' captures no data, and that both 'exp-survey' **frames** capture a 'formData' object. diff --git a/docs/source/index.rst b/docs/source/index.rst index 9dac5ec..2d63b29 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -17,6 +17,7 @@ Contents: staging experiments + experimentdata experiments_ref stimuli development From 2bcec094ec5bb7ca5ecdfb1ac4501d799feb522c Mon Sep 17 00:00:00 2001 From: Fitz Elliott Date: Fri, 6 Oct 2017 13:57:23 -0400 Subject: [PATCH 4/8] bump exp-addons dep to get last-modified date from session * Needed to export the last-modified timestamp in the CSV for ISP output. [SVCS-382] https://github.com/CenterForOpenScience/exp-addons/pull/238 --- lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib b/lib index a09006d..5492446 160000 --- a/lib +++ b/lib @@ -1 +1 @@ -Subproject commit a09006d94ebc3c0f1b56a8a2cc53ef296f311576 +Subproject commit 5492446b29550835c7fbb645ccddf465fd4ebf86 From 512c34c1c93f31931cdfc1c42f81e5c34248d496 Mon Sep 17 00:00:00 2001 From: Addison Schiller Date: Mon, 17 Jul 2017 13:56:34 -0400 Subject: [PATCH 5/8] add date completed field to export data [SVCS-382] Closes: #144 --- app/components/export-tool/component.js | 1 + app/components/export-tool/isp-fields.js | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/app/components/export-tool/component.js b/app/components/export-tool/component.js index 484e4b6..adc0c10 100644 --- a/app/components/export-tool/component.js +++ b/app/components/export-tool/component.js @@ -114,6 +114,7 @@ export default Ember.Component.extend({ const newRecord = { PID: record.profileId, SID: record.extra.studyId, + completed: record.modifiedOn, locale: record.extra.locale }; diff --git a/app/components/export-tool/isp-fields.js b/app/components/export-tool/isp-fields.js index 4273615..c4a6cc1 100644 --- a/app/components/export-tool/isp-fields.js +++ b/app/components/export-tool/isp-fields.js @@ -47,5 +47,6 @@ export default [ ...generateNumericFields('Honest', 10), ...generateNumericFields('Micro', 6), ...generateNumericFields('Narq', 6), - ...generateNumericFields('ReligionScale', 17) + ...generateNumericFields('ReligionScale', 17), + 'completed' ]; From 907460915b34cb948316f1e3512e538f1dc02f3f Mon Sep 17 00:00:00 2001 From: Fitz Elliott Date: Fri, 6 Oct 2017 14:43:40 -0400 Subject: [PATCH 6/8] fix data export tests to accommodate last-modified date --- tests/fixtures/isp-survey.csv | 4 ++-- tests/fixtures/isp-survey.json | 1 + tests/integration/components/export-tool-test.js | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/fixtures/isp-survey.csv b/tests/fixtures/isp-survey.csv index 130b61f..871d0d9 100644 --- a/tests/fixtures/isp-survey.csv +++ b/tests/fixtures/isp-survey.csv @@ -1,2 +1,2 @@ -PID,SID,locale,Age,Gender,Ethnicity,Language,SocialStatus,BirthCity,BirthCountry,Residence,Religion1to10,ReligionYesNo,ReligionFollow,EventTime,WhatResponse,WhereResponse,WhoResponse,ThreeCat_rsq1,ThreeCat_rsq2,ThreeCat_rsq3,ThreeCat_rsq4,ThreeCat_rsq5,ThreeCat_rsq6,ThreeCat_rsq7,ThreeCat_rsq8,ThreeCat_rsq9,ThreeCat_rsq10,ThreeCat_rsq11,ThreeCat_rsq12,ThreeCat_rsq13,ThreeCat_rsq14,ThreeCat_rsq15,ThreeCat_rsq16,ThreeCat_rsq17,ThreeCat_rsq18,ThreeCat_rsq19,ThreeCat_rsq20,ThreeCat_rsq21,ThreeCat_rsq22,ThreeCat_rsq23,ThreeCat_rsq24,ThreeCat_rsq25,ThreeCat_rsq26,ThreeCat_rsq27,ThreeCat_rsq28,ThreeCat_rsq29,ThreeCat_rsq30,ThreeCat_rsq31,ThreeCat_rsq32,ThreeCat_rsq33,ThreeCat_rsq34,ThreeCat_rsq35,ThreeCat_rsq36,ThreeCat_rsq37,ThreeCat_rsq38,ThreeCat_rsq39,ThreeCat_rsq40,ThreeCat_rsq41,ThreeCat_rsq42,ThreeCat_rsq43,ThreeCat_rsq44,ThreeCat_rsq45,ThreeCat_rsq46,ThreeCat_rsq47,ThreeCat_rsq48,ThreeCat_rsq49,ThreeCat_rsq50,ThreeCat_rsq51,ThreeCat_rsq52,ThreeCat_rsq53,ThreeCat_rsq54,ThreeCat_rsq55,ThreeCat_rsq56,ThreeCat_rsq57,ThreeCat_rsq58,ThreeCat_rsq59,ThreeCat_rsq60,ThreeCat_rsq61,ThreeCat_rsq62,ThreeCat_rsq63,ThreeCat_rsq64,ThreeCat_rsq65,ThreeCat_rsq66,ThreeCat_rsq67,ThreeCat_rsq68,ThreeCat_rsq69,ThreeCat_rsq70,ThreeCat_rsq71,ThreeCat_rsq72,ThreeCat_rsq73,ThreeCat_rsq74,ThreeCat_rsq75,ThreeCat_rsq76,ThreeCat_rsq77,ThreeCat_rsq78,ThreeCat_rsq79,ThreeCat_rsq80,ThreeCat_rsq81,ThreeCat_rsq82,ThreeCat_rsq83,ThreeCat_rsq84,ThreeCat_rsq85,ThreeCat_rsq86,ThreeCat_rsq87,ThreeCat_rsq88,ThreeCat_rsq89,ThreeCat_rsq90,NineCat_rsq1,NineCat_rsq2,NineCat_rsq3,NineCat_rsq4,NineCat_rsq5,NineCat_rsq6,NineCat_rsq7,NineCat_rsq8,NineCat_rsq9,NineCat_rsq10,NineCat_rsq11,NineCat_rsq12,NineCat_rsq13,NineCat_rsq14,NineCat_rsq15,NineCat_rsq16,NineCat_rsq17,NineCat_rsq18,NineCat_rsq19,NineCat_rsq20,NineCat_rsq21,NineCat_rsq22,NineCat_rsq23,NineCat_rsq24,NineCat_rsq25,NineCat_rsq26,NineCat_rsq27,NineCat_rsq28,NineCat_rsq29,NineCat_rsq30,NineCat_rsq31,NineCat_rsq32,NineCat_rsq33,NineCat_rsq34,NineCat_rsq35,NineCat_rsq36,NineCat_rsq37,NineCat_rsq38,NineCat_rsq39,NineCat_rsq40,NineCat_rsq41,NineCat_rsq42,NineCat_rsq43,NineCat_rsq44,NineCat_rsq45,NineCat_rsq46,NineCat_rsq47,NineCat_rsq48,NineCat_rsq49,NineCat_rsq50,NineCat_rsq51,NineCat_rsq52,NineCat_rsq53,NineCat_rsq54,NineCat_rsq55,NineCat_rsq56,NineCat_rsq57,NineCat_rsq58,NineCat_rsq59,NineCat_rsq60,NineCat_rsq61,NineCat_rsq62,NineCat_rsq63,NineCat_rsq64,NineCat_rsq65,NineCat_rsq66,NineCat_rsq67,NineCat_rsq68,NineCat_rsq69,NineCat_rsq70,NineCat_rsq71,NineCat_rsq72,NineCat_rsq73,NineCat_rsq74,NineCat_rsq75,NineCat_rsq76,NineCat_rsq77,NineCat_rsq78,NineCat_rsq79,NineCat_rsq80,NineCat_rsq81,NineCat_rsq82,NineCat_rsq83,NineCat_rsq84,NineCat_rsq85,NineCat_rsq86,NineCat_rsq87,NineCat_rsq88,NineCat_rsq89,NineCat_rsq90,PosNeg,SitSimilarity,BBI1,BBI2,BBI3,BBI4,BBI5,BBI6,BBI7,BBI8,BBI9,BBI10,BBI11,BBI12,BBI13,BBI14,BBI15,BBI16,Risk,BFI1,BFI2,BFI3,BFI4,BFI5,BFI6,BFI7,BFI8,BFI9,BFI10,BFI11,BFI12,BFI13,BFI14,BFI15,BFI16,BFI17,BFI18,BFI19,BFI20,BFI21,BFI22,BFI23,BFI24,BFI25,BFI26,BFI27,BFI28,BFI29,BFI30,BFI31,BFI32,BFI33,BFI34,BFI35,BFI36,BFI37,BFI38,BFI39,BFI40,BFI41,BFI42,BFI43,BFI44,BFI45,BFI46,BFI47,BFI48,BFI49,BFI50,BFI51,BFI52,BFI53,BFI54,BFI55,BFI56,BFI57,BFI58,BFI59,BFI60,SWB1,SWB2,SWB3,SWB4,IntHapp1,IntHapp2,IntHapp3,IntHapp4,IntHapp5,IntHapp6,IntHapp7,IntHapp8,IntHapp9,Constru1,Constru2,Constru3,Constru4,Constru5,Constru6,Constru7,Constru8,Constru9,Constru10,Constru11,Constru12,Constru13,Tight1,Tight2,Tight3,Tight4,Tight5,Tight6,ChangeYesNo,ChangeDescribe,ChangeSuccess,Trust1,Trust2,Trust3,Trust4,Trust5,LOT1,LOT2,LOT3,LOT4,LOT5,LOT6,Honest1,Honest2,Honest3,Honest4,Honest5,Honest6,Honest7,Honest8,Honest9,Honest10,Micro1,Micro2,Micro3,Micro4,Micro5,Micro6,Narq1,Narq2,Narq3,Narq4,Narq5,Narq6,ReligionScale1,ReligionScale2,ReligionScale3,ReligionScale4,ReligionScale5,ReligionScale6,ReligionScale7,ReligionScale8,ReligionScale9,ReligionScale10,ReligionScale11,ReligionScale12,ReligionScale13,ReligionScale14,ReligionScale15,ReligionScale16,ReligionScale17 -"5514232571-something","someString","en-US",32,1,"Special","Tagalog",6,"Big Crater","The Moon",1,1,2,,"15:00","Eating something","Daydreaming","alone",1,2,2,1,2,3,3,1,2,1,2,2,3,2,3,3,2,1,2,3,1,2,1,2,1,2,3,3,2,2,3,2,3,3,2,2,2,2,1,2,1,2,2,1,3,2,2,3,3,2,1,2,1,2,2,2,3,3,2,2,2,1,2,2,3,2,2,1,2,2,3,2,2,2,1,1,2,1,2,3,3,3,1,2,2,2,2,2,1,2,2,6,5,1,4,7,8,6,5,6,6,4,8,3,8,7,4,2,3,7,7,4,4,5,1,6,8,8,4,3,6,6,7,7,3,5,5,5,4,5,6,6,5,2,7,4,4,6,7,5,1,6,2,5,5,4,9,9,4,6,5,3,6,5,7,5,3,2,4,5,8,5,4,5,3,3,4,3,5,9,7,7,3,6,3,4,6,5,2,5,6,4,4,7,7,4,5,3,7,7,3,7,4,4,6,7,4,8,0,2,3,4,2,5,2,5,2,4,1,5,2,4,2,5,1,5,2,4,1,5,1,5,4,3,2,5,5,5,1,4,1,5,1,1,1,2,4,5,1,5,5,1,5,1,5,1,1,5,4,4,2,1,5,1,1,5,5,5,4,7,7,7,6,4,4,1,1,1,1,1,1,1,1,1,2,2,8,2,2,3,2,2,3,2,3,4,3,3,3,2,4,2,,,5,4,3,3,1,3,3,3,4,2,4,1,3,3,2,5,2,4,2,4,1,4,1,3,2,4,,2,4,2,3,3,3,1,2,2,2,2,2,3,4,3,2,4,2,3,3,4,3,3 +PID,SID,locale,Age,Gender,Ethnicity,Language,SocialStatus,BirthCity,BirthCountry,Residence,Religion1to10,ReligionYesNo,ReligionFollow,EventTime,WhatResponse,WhereResponse,WhoResponse,ThreeCat_rsq1,ThreeCat_rsq2,ThreeCat_rsq3,ThreeCat_rsq4,ThreeCat_rsq5,ThreeCat_rsq6,ThreeCat_rsq7,ThreeCat_rsq8,ThreeCat_rsq9,ThreeCat_rsq10,ThreeCat_rsq11,ThreeCat_rsq12,ThreeCat_rsq13,ThreeCat_rsq14,ThreeCat_rsq15,ThreeCat_rsq16,ThreeCat_rsq17,ThreeCat_rsq18,ThreeCat_rsq19,ThreeCat_rsq20,ThreeCat_rsq21,ThreeCat_rsq22,ThreeCat_rsq23,ThreeCat_rsq24,ThreeCat_rsq25,ThreeCat_rsq26,ThreeCat_rsq27,ThreeCat_rsq28,ThreeCat_rsq29,ThreeCat_rsq30,ThreeCat_rsq31,ThreeCat_rsq32,ThreeCat_rsq33,ThreeCat_rsq34,ThreeCat_rsq35,ThreeCat_rsq36,ThreeCat_rsq37,ThreeCat_rsq38,ThreeCat_rsq39,ThreeCat_rsq40,ThreeCat_rsq41,ThreeCat_rsq42,ThreeCat_rsq43,ThreeCat_rsq44,ThreeCat_rsq45,ThreeCat_rsq46,ThreeCat_rsq47,ThreeCat_rsq48,ThreeCat_rsq49,ThreeCat_rsq50,ThreeCat_rsq51,ThreeCat_rsq52,ThreeCat_rsq53,ThreeCat_rsq54,ThreeCat_rsq55,ThreeCat_rsq56,ThreeCat_rsq57,ThreeCat_rsq58,ThreeCat_rsq59,ThreeCat_rsq60,ThreeCat_rsq61,ThreeCat_rsq62,ThreeCat_rsq63,ThreeCat_rsq64,ThreeCat_rsq65,ThreeCat_rsq66,ThreeCat_rsq67,ThreeCat_rsq68,ThreeCat_rsq69,ThreeCat_rsq70,ThreeCat_rsq71,ThreeCat_rsq72,ThreeCat_rsq73,ThreeCat_rsq74,ThreeCat_rsq75,ThreeCat_rsq76,ThreeCat_rsq77,ThreeCat_rsq78,ThreeCat_rsq79,ThreeCat_rsq80,ThreeCat_rsq81,ThreeCat_rsq82,ThreeCat_rsq83,ThreeCat_rsq84,ThreeCat_rsq85,ThreeCat_rsq86,ThreeCat_rsq87,ThreeCat_rsq88,ThreeCat_rsq89,ThreeCat_rsq90,NineCat_rsq1,NineCat_rsq2,NineCat_rsq3,NineCat_rsq4,NineCat_rsq5,NineCat_rsq6,NineCat_rsq7,NineCat_rsq8,NineCat_rsq9,NineCat_rsq10,NineCat_rsq11,NineCat_rsq12,NineCat_rsq13,NineCat_rsq14,NineCat_rsq15,NineCat_rsq16,NineCat_rsq17,NineCat_rsq18,NineCat_rsq19,NineCat_rsq20,NineCat_rsq21,NineCat_rsq22,NineCat_rsq23,NineCat_rsq24,NineCat_rsq25,NineCat_rsq26,NineCat_rsq27,NineCat_rsq28,NineCat_rsq29,NineCat_rsq30,NineCat_rsq31,NineCat_rsq32,NineCat_rsq33,NineCat_rsq34,NineCat_rsq35,NineCat_rsq36,NineCat_rsq37,NineCat_rsq38,NineCat_rsq39,NineCat_rsq40,NineCat_rsq41,NineCat_rsq42,NineCat_rsq43,NineCat_rsq44,NineCat_rsq45,NineCat_rsq46,NineCat_rsq47,NineCat_rsq48,NineCat_rsq49,NineCat_rsq50,NineCat_rsq51,NineCat_rsq52,NineCat_rsq53,NineCat_rsq54,NineCat_rsq55,NineCat_rsq56,NineCat_rsq57,NineCat_rsq58,NineCat_rsq59,NineCat_rsq60,NineCat_rsq61,NineCat_rsq62,NineCat_rsq63,NineCat_rsq64,NineCat_rsq65,NineCat_rsq66,NineCat_rsq67,NineCat_rsq68,NineCat_rsq69,NineCat_rsq70,NineCat_rsq71,NineCat_rsq72,NineCat_rsq73,NineCat_rsq74,NineCat_rsq75,NineCat_rsq76,NineCat_rsq77,NineCat_rsq78,NineCat_rsq79,NineCat_rsq80,NineCat_rsq81,NineCat_rsq82,NineCat_rsq83,NineCat_rsq84,NineCat_rsq85,NineCat_rsq86,NineCat_rsq87,NineCat_rsq88,NineCat_rsq89,NineCat_rsq90,PosNeg,SitSimilarity,BBI1,BBI2,BBI3,BBI4,BBI5,BBI6,BBI7,BBI8,BBI9,BBI10,BBI11,BBI12,BBI13,BBI14,BBI15,BBI16,Risk,BFI1,BFI2,BFI3,BFI4,BFI5,BFI6,BFI7,BFI8,BFI9,BFI10,BFI11,BFI12,BFI13,BFI14,BFI15,BFI16,BFI17,BFI18,BFI19,BFI20,BFI21,BFI22,BFI23,BFI24,BFI25,BFI26,BFI27,BFI28,BFI29,BFI30,BFI31,BFI32,BFI33,BFI34,BFI35,BFI36,BFI37,BFI38,BFI39,BFI40,BFI41,BFI42,BFI43,BFI44,BFI45,BFI46,BFI47,BFI48,BFI49,BFI50,BFI51,BFI52,BFI53,BFI54,BFI55,BFI56,BFI57,BFI58,BFI59,BFI60,SWB1,SWB2,SWB3,SWB4,IntHapp1,IntHapp2,IntHapp3,IntHapp4,IntHapp5,IntHapp6,IntHapp7,IntHapp8,IntHapp9,Constru1,Constru2,Constru3,Constru4,Constru5,Constru6,Constru7,Constru8,Constru9,Constru10,Constru11,Constru12,Constru13,Tight1,Tight2,Tight3,Tight4,Tight5,Tight6,ChangeYesNo,ChangeDescribe,ChangeSuccess,Trust1,Trust2,Trust3,Trust4,Trust5,LOT1,LOT2,LOT3,LOT4,LOT5,LOT6,Honest1,Honest2,Honest3,Honest4,Honest5,Honest6,Honest7,Honest8,Honest9,Honest10,Micro1,Micro2,Micro3,Micro4,Micro5,Micro6,Narq1,Narq2,Narq3,Narq4,Narq5,Narq6,ReligionScale1,ReligionScale2,ReligionScale3,ReligionScale4,ReligionScale5,ReligionScale6,ReligionScale7,ReligionScale8,ReligionScale9,ReligionScale10,ReligionScale11,ReligionScale12,ReligionScale13,ReligionScale14,ReligionScale15,ReligionScale16,ReligionScale17,completed +"5514232571-something","someString","en-US",32,1,"Special","Tagalog",6,"Big Crater","The Moon",1,1,2,,"15:00","Eating something","Daydreaming","alone",1,2,2,1,2,3,3,1,2,1,2,2,3,2,3,3,2,1,2,3,1,2,1,2,1,2,3,3,2,2,3,2,3,3,2,2,2,2,1,2,1,2,2,1,3,2,2,3,3,2,1,2,1,2,2,2,3,3,2,2,2,1,2,2,3,2,2,1,2,2,3,2,2,2,1,1,2,1,2,3,3,3,1,2,2,2,2,2,1,2,2,6,5,1,4,7,8,6,5,6,6,4,8,3,8,7,4,2,3,7,7,4,4,5,1,6,8,8,4,3,6,6,7,7,3,5,5,5,4,5,6,6,5,2,7,4,4,6,7,5,1,6,2,5,5,4,9,9,4,6,5,3,6,5,7,5,3,2,4,5,8,5,4,5,3,3,4,3,5,9,7,7,3,6,3,4,6,5,2,5,6,4,4,7,7,4,5,3,7,7,3,7,4,4,6,7,4,8,0,2,3,4,2,5,2,5,2,4,1,5,2,4,2,5,1,5,2,4,1,5,1,5,4,3,2,5,5,5,1,4,1,5,1,1,1,2,4,5,1,5,5,1,5,1,5,1,1,5,4,4,2,1,5,1,1,5,5,5,4,7,7,7,6,4,4,1,1,1,1,1,1,1,1,1,2,2,8,2,2,3,2,2,3,2,3,4,3,3,3,2,4,2,,,5,4,3,3,1,3,3,3,4,2,4,1,3,3,2,5,2,4,2,4,1,4,1,3,2,4,,2,4,2,3,3,3,1,2,2,2,2,2,3,4,3,2,4,2,3,3,4,3,3,"2017-10-06T19:33:00.000Z" diff --git a/tests/fixtures/isp-survey.json b/tests/fixtures/isp-survey.json index 6e49bdd..cf3f879 100644 --- a/tests/fixtures/isp-survey.json +++ b/tests/fixtures/isp-survey.json @@ -1,5 +1,6 @@ [ { + "modifiedOn": "2017-10-06T19:33:00.000Z", "sequence": [ "0-0-overview", "1-1-free-response", diff --git a/tests/integration/components/export-tool-test.js b/tests/integration/components/export-tool-test.js index 01aed73..d797a7b 100644 --- a/tests/integration/components/export-tool-test.js +++ b/tests/integration/components/export-tool-test.js @@ -124,8 +124,8 @@ test('CSV for ISP - Full Example', function(assert) { assert.strictEqual( this.$('.export-tool-textarea').val(), svJoin([ - 'PID,SID,locale,Age,Gender,Ethnicity,Language,SocialStatus,BirthCity,BirthCountry,Residence,Religion1to10,ReligionYesNo,ReligionFollow,EventTime,WhatResponse,WhereResponse,WhoResponse,ThreeCat_rsq1,ThreeCat_rsq2,ThreeCat_rsq3,ThreeCat_rsq4,ThreeCat_rsq5,ThreeCat_rsq6,ThreeCat_rsq7,ThreeCat_rsq8,ThreeCat_rsq9,ThreeCat_rsq10,ThreeCat_rsq11,ThreeCat_rsq12,ThreeCat_rsq13,ThreeCat_rsq14,ThreeCat_rsq15,ThreeCat_rsq16,ThreeCat_rsq17,ThreeCat_rsq18,ThreeCat_rsq19,ThreeCat_rsq20,ThreeCat_rsq21,ThreeCat_rsq22,ThreeCat_rsq23,ThreeCat_rsq24,ThreeCat_rsq25,ThreeCat_rsq26,ThreeCat_rsq27,ThreeCat_rsq28,ThreeCat_rsq29,ThreeCat_rsq30,ThreeCat_rsq31,ThreeCat_rsq32,ThreeCat_rsq33,ThreeCat_rsq34,ThreeCat_rsq35,ThreeCat_rsq36,ThreeCat_rsq37,ThreeCat_rsq38,ThreeCat_rsq39,ThreeCat_rsq40,ThreeCat_rsq41,ThreeCat_rsq42,ThreeCat_rsq43,ThreeCat_rsq44,ThreeCat_rsq45,ThreeCat_rsq46,ThreeCat_rsq47,ThreeCat_rsq48,ThreeCat_rsq49,ThreeCat_rsq50,ThreeCat_rsq51,ThreeCat_rsq52,ThreeCat_rsq53,ThreeCat_rsq54,ThreeCat_rsq55,ThreeCat_rsq56,ThreeCat_rsq57,ThreeCat_rsq58,ThreeCat_rsq59,ThreeCat_rsq60,ThreeCat_rsq61,ThreeCat_rsq62,ThreeCat_rsq63,ThreeCat_rsq64,ThreeCat_rsq65,ThreeCat_rsq66,ThreeCat_rsq67,ThreeCat_rsq68,ThreeCat_rsq69,ThreeCat_rsq70,ThreeCat_rsq71,ThreeCat_rsq72,ThreeCat_rsq73,ThreeCat_rsq74,ThreeCat_rsq75,ThreeCat_rsq76,ThreeCat_rsq77,ThreeCat_rsq78,ThreeCat_rsq79,ThreeCat_rsq80,ThreeCat_rsq81,ThreeCat_rsq82,ThreeCat_rsq83,ThreeCat_rsq84,ThreeCat_rsq85,ThreeCat_rsq86,ThreeCat_rsq87,ThreeCat_rsq88,ThreeCat_rsq89,ThreeCat_rsq90,NineCat_rsq1,NineCat_rsq2,NineCat_rsq3,NineCat_rsq4,NineCat_rsq5,NineCat_rsq6,NineCat_rsq7,NineCat_rsq8,NineCat_rsq9,NineCat_rsq10,NineCat_rsq11,NineCat_rsq12,NineCat_rsq13,NineCat_rsq14,NineCat_rsq15,NineCat_rsq16,NineCat_rsq17,NineCat_rsq18,NineCat_rsq19,NineCat_rsq20,NineCat_rsq21,NineCat_rsq22,NineCat_rsq23,NineCat_rsq24,NineCat_rsq25,NineCat_rsq26,NineCat_rsq27,NineCat_rsq28,NineCat_rsq29,NineCat_rsq30,NineCat_rsq31,NineCat_rsq32,NineCat_rsq33,NineCat_rsq34,NineCat_rsq35,NineCat_rsq36,NineCat_rsq37,NineCat_rsq38,NineCat_rsq39,NineCat_rsq40,NineCat_rsq41,NineCat_rsq42,NineCat_rsq43,NineCat_rsq44,NineCat_rsq45,NineCat_rsq46,NineCat_rsq47,NineCat_rsq48,NineCat_rsq49,NineCat_rsq50,NineCat_rsq51,NineCat_rsq52,NineCat_rsq53,NineCat_rsq54,NineCat_rsq55,NineCat_rsq56,NineCat_rsq57,NineCat_rsq58,NineCat_rsq59,NineCat_rsq60,NineCat_rsq61,NineCat_rsq62,NineCat_rsq63,NineCat_rsq64,NineCat_rsq65,NineCat_rsq66,NineCat_rsq67,NineCat_rsq68,NineCat_rsq69,NineCat_rsq70,NineCat_rsq71,NineCat_rsq72,NineCat_rsq73,NineCat_rsq74,NineCat_rsq75,NineCat_rsq76,NineCat_rsq77,NineCat_rsq78,NineCat_rsq79,NineCat_rsq80,NineCat_rsq81,NineCat_rsq82,NineCat_rsq83,NineCat_rsq84,NineCat_rsq85,NineCat_rsq86,NineCat_rsq87,NineCat_rsq88,NineCat_rsq89,NineCat_rsq90,PosNeg,SitSimilarity,BBI1,BBI2,BBI3,BBI4,BBI5,BBI6,BBI7,BBI8,BBI9,BBI10,BBI11,BBI12,BBI13,BBI14,BBI15,BBI16,Risk,BFI1,BFI2,BFI3,BFI4,BFI5,BFI6,BFI7,BFI8,BFI9,BFI10,BFI11,BFI12,BFI13,BFI14,BFI15,BFI16,BFI17,BFI18,BFI19,BFI20,BFI21,BFI22,BFI23,BFI24,BFI25,BFI26,BFI27,BFI28,BFI29,BFI30,BFI31,BFI32,BFI33,BFI34,BFI35,BFI36,BFI37,BFI38,BFI39,BFI40,BFI41,BFI42,BFI43,BFI44,BFI45,BFI46,BFI47,BFI48,BFI49,BFI50,BFI51,BFI52,BFI53,BFI54,BFI55,BFI56,BFI57,BFI58,BFI59,BFI60,SWB1,SWB2,SWB3,SWB4,IntHapp1,IntHapp2,IntHapp3,IntHapp4,IntHapp5,IntHapp6,IntHapp7,IntHapp8,IntHapp9,Constru1,Constru2,Constru3,Constru4,Constru5,Constru6,Constru7,Constru8,Constru9,Constru10,Constru11,Constru12,Constru13,Tight1,Tight2,Tight3,Tight4,Tight5,Tight6,ChangeYesNo,ChangeDescribe,ChangeSuccess,Trust1,Trust2,Trust3,Trust4,Trust5,LOT1,LOT2,LOT3,LOT4,LOT5,LOT6,Honest1,Honest2,Honest3,Honest4,Honest5,Honest6,Honest7,Honest8,Honest9,Honest10,Micro1,Micro2,Micro3,Micro4,Micro5,Micro6,Narq1,Narq2,Narq3,Narq4,Narq5,Narq6,ReligionScale1,ReligionScale2,ReligionScale3,ReligionScale4,ReligionScale5,ReligionScale6,ReligionScale7,ReligionScale8,ReligionScale9,ReligionScale10,ReligionScale11,ReligionScale12,ReligionScale13,ReligionScale14,ReligionScale15,ReligionScale16,ReligionScale17', - '"5514232571-something","someString","en-US",32,1,"Special","Tagalog",6,"Big Crater","The Moon",1,1,2,,"15:00","Eating something","Daydreaming","alone",1,2,2,1,2,3,3,1,2,1,2,2,3,2,3,3,2,1,2,3,1,2,1,2,1,2,3,3,2,2,3,2,3,3,2,2,2,2,1,2,1,2,2,1,3,2,2,3,3,2,1,2,1,2,2,2,3,3,2,2,2,1,2,2,3,2,2,1,2,2,3,2,2,2,1,1,2,1,2,3,3,3,1,2,2,2,2,2,1,2,2,6,5,1,4,7,8,6,5,6,6,4,8,3,8,7,4,2,3,7,7,4,4,5,1,6,8,8,4,3,6,6,7,7,3,5,5,5,4,5,6,6,5,2,7,4,4,6,7,5,1,6,2,5,5,4,9,9,4,6,5,3,6,5,7,5,3,2,4,5,8,5,4,5,3,3,4,3,5,9,7,7,3,6,3,4,6,5,2,5,6,4,4,7,7,4,5,3,7,7,3,7,4,4,6,7,4,8,0,2,3,4,2,5,2,5,2,4,1,5,2,4,2,5,1,5,2,4,1,5,1,5,4,3,2,5,5,5,1,4,1,5,1,1,1,2,4,5,1,5,5,1,5,1,5,1,1,5,4,4,2,1,5,1,1,5,5,5,4,7,7,7,6,4,4,1,1,1,1,1,1,1,1,1,2,2,8,2,2,3,2,2,3,2,3,4,3,3,3,2,4,2,,,5,4,3,3,1,3,3,3,4,2,4,1,3,3,2,5,2,4,2,4,1,4,1,3,2,4,,2,4,2,3,3,3,1,2,2,2,2,2,3,4,3,2,4,2,3,3,4,3,3' + 'PID,SID,locale,Age,Gender,Ethnicity,Language,SocialStatus,BirthCity,BirthCountry,Residence,Religion1to10,ReligionYesNo,ReligionFollow,EventTime,WhatResponse,WhereResponse,WhoResponse,ThreeCat_rsq1,ThreeCat_rsq2,ThreeCat_rsq3,ThreeCat_rsq4,ThreeCat_rsq5,ThreeCat_rsq6,ThreeCat_rsq7,ThreeCat_rsq8,ThreeCat_rsq9,ThreeCat_rsq10,ThreeCat_rsq11,ThreeCat_rsq12,ThreeCat_rsq13,ThreeCat_rsq14,ThreeCat_rsq15,ThreeCat_rsq16,ThreeCat_rsq17,ThreeCat_rsq18,ThreeCat_rsq19,ThreeCat_rsq20,ThreeCat_rsq21,ThreeCat_rsq22,ThreeCat_rsq23,ThreeCat_rsq24,ThreeCat_rsq25,ThreeCat_rsq26,ThreeCat_rsq27,ThreeCat_rsq28,ThreeCat_rsq29,ThreeCat_rsq30,ThreeCat_rsq31,ThreeCat_rsq32,ThreeCat_rsq33,ThreeCat_rsq34,ThreeCat_rsq35,ThreeCat_rsq36,ThreeCat_rsq37,ThreeCat_rsq38,ThreeCat_rsq39,ThreeCat_rsq40,ThreeCat_rsq41,ThreeCat_rsq42,ThreeCat_rsq43,ThreeCat_rsq44,ThreeCat_rsq45,ThreeCat_rsq46,ThreeCat_rsq47,ThreeCat_rsq48,ThreeCat_rsq49,ThreeCat_rsq50,ThreeCat_rsq51,ThreeCat_rsq52,ThreeCat_rsq53,ThreeCat_rsq54,ThreeCat_rsq55,ThreeCat_rsq56,ThreeCat_rsq57,ThreeCat_rsq58,ThreeCat_rsq59,ThreeCat_rsq60,ThreeCat_rsq61,ThreeCat_rsq62,ThreeCat_rsq63,ThreeCat_rsq64,ThreeCat_rsq65,ThreeCat_rsq66,ThreeCat_rsq67,ThreeCat_rsq68,ThreeCat_rsq69,ThreeCat_rsq70,ThreeCat_rsq71,ThreeCat_rsq72,ThreeCat_rsq73,ThreeCat_rsq74,ThreeCat_rsq75,ThreeCat_rsq76,ThreeCat_rsq77,ThreeCat_rsq78,ThreeCat_rsq79,ThreeCat_rsq80,ThreeCat_rsq81,ThreeCat_rsq82,ThreeCat_rsq83,ThreeCat_rsq84,ThreeCat_rsq85,ThreeCat_rsq86,ThreeCat_rsq87,ThreeCat_rsq88,ThreeCat_rsq89,ThreeCat_rsq90,NineCat_rsq1,NineCat_rsq2,NineCat_rsq3,NineCat_rsq4,NineCat_rsq5,NineCat_rsq6,NineCat_rsq7,NineCat_rsq8,NineCat_rsq9,NineCat_rsq10,NineCat_rsq11,NineCat_rsq12,NineCat_rsq13,NineCat_rsq14,NineCat_rsq15,NineCat_rsq16,NineCat_rsq17,NineCat_rsq18,NineCat_rsq19,NineCat_rsq20,NineCat_rsq21,NineCat_rsq22,NineCat_rsq23,NineCat_rsq24,NineCat_rsq25,NineCat_rsq26,NineCat_rsq27,NineCat_rsq28,NineCat_rsq29,NineCat_rsq30,NineCat_rsq31,NineCat_rsq32,NineCat_rsq33,NineCat_rsq34,NineCat_rsq35,NineCat_rsq36,NineCat_rsq37,NineCat_rsq38,NineCat_rsq39,NineCat_rsq40,NineCat_rsq41,NineCat_rsq42,NineCat_rsq43,NineCat_rsq44,NineCat_rsq45,NineCat_rsq46,NineCat_rsq47,NineCat_rsq48,NineCat_rsq49,NineCat_rsq50,NineCat_rsq51,NineCat_rsq52,NineCat_rsq53,NineCat_rsq54,NineCat_rsq55,NineCat_rsq56,NineCat_rsq57,NineCat_rsq58,NineCat_rsq59,NineCat_rsq60,NineCat_rsq61,NineCat_rsq62,NineCat_rsq63,NineCat_rsq64,NineCat_rsq65,NineCat_rsq66,NineCat_rsq67,NineCat_rsq68,NineCat_rsq69,NineCat_rsq70,NineCat_rsq71,NineCat_rsq72,NineCat_rsq73,NineCat_rsq74,NineCat_rsq75,NineCat_rsq76,NineCat_rsq77,NineCat_rsq78,NineCat_rsq79,NineCat_rsq80,NineCat_rsq81,NineCat_rsq82,NineCat_rsq83,NineCat_rsq84,NineCat_rsq85,NineCat_rsq86,NineCat_rsq87,NineCat_rsq88,NineCat_rsq89,NineCat_rsq90,PosNeg,SitSimilarity,BBI1,BBI2,BBI3,BBI4,BBI5,BBI6,BBI7,BBI8,BBI9,BBI10,BBI11,BBI12,BBI13,BBI14,BBI15,BBI16,Risk,BFI1,BFI2,BFI3,BFI4,BFI5,BFI6,BFI7,BFI8,BFI9,BFI10,BFI11,BFI12,BFI13,BFI14,BFI15,BFI16,BFI17,BFI18,BFI19,BFI20,BFI21,BFI22,BFI23,BFI24,BFI25,BFI26,BFI27,BFI28,BFI29,BFI30,BFI31,BFI32,BFI33,BFI34,BFI35,BFI36,BFI37,BFI38,BFI39,BFI40,BFI41,BFI42,BFI43,BFI44,BFI45,BFI46,BFI47,BFI48,BFI49,BFI50,BFI51,BFI52,BFI53,BFI54,BFI55,BFI56,BFI57,BFI58,BFI59,BFI60,SWB1,SWB2,SWB3,SWB4,IntHapp1,IntHapp2,IntHapp3,IntHapp4,IntHapp5,IntHapp6,IntHapp7,IntHapp8,IntHapp9,Constru1,Constru2,Constru3,Constru4,Constru5,Constru6,Constru7,Constru8,Constru9,Constru10,Constru11,Constru12,Constru13,Tight1,Tight2,Tight3,Tight4,Tight5,Tight6,ChangeYesNo,ChangeDescribe,ChangeSuccess,Trust1,Trust2,Trust3,Trust4,Trust5,LOT1,LOT2,LOT3,LOT4,LOT5,LOT6,Honest1,Honest2,Honest3,Honest4,Honest5,Honest6,Honest7,Honest8,Honest9,Honest10,Micro1,Micro2,Micro3,Micro4,Micro5,Micro6,Narq1,Narq2,Narq3,Narq4,Narq5,Narq6,ReligionScale1,ReligionScale2,ReligionScale3,ReligionScale4,ReligionScale5,ReligionScale6,ReligionScale7,ReligionScale8,ReligionScale9,ReligionScale10,ReligionScale11,ReligionScale12,ReligionScale13,ReligionScale14,ReligionScale15,ReligionScale16,ReligionScale17,completed', + '"5514232571-something","someString","en-US",32,1,"Special","Tagalog",6,"Big Crater","The Moon",1,1,2,,"15:00","Eating something","Daydreaming","alone",1,2,2,1,2,3,3,1,2,1,2,2,3,2,3,3,2,1,2,3,1,2,1,2,1,2,3,3,2,2,3,2,3,3,2,2,2,2,1,2,1,2,2,1,3,2,2,3,3,2,1,2,1,2,2,2,3,3,2,2,2,1,2,2,3,2,2,1,2,2,3,2,2,2,1,1,2,1,2,3,3,3,1,2,2,2,2,2,1,2,2,6,5,1,4,7,8,6,5,6,6,4,8,3,8,7,4,2,3,7,7,4,4,5,1,6,8,8,4,3,6,6,7,7,3,5,5,5,4,5,6,6,5,2,7,4,4,6,7,5,1,6,2,5,5,4,9,9,4,6,5,3,6,5,7,5,3,2,4,5,8,5,4,5,3,3,4,3,5,9,7,7,3,6,3,4,6,5,2,5,6,4,4,7,7,4,5,3,7,7,3,7,4,4,6,7,4,8,0,2,3,4,2,5,2,5,2,4,1,5,2,4,2,5,1,5,2,4,1,5,1,5,4,3,2,5,5,5,1,4,1,5,1,1,1,2,4,5,1,5,5,1,5,1,5,1,1,5,4,4,2,1,5,1,1,5,5,5,4,7,7,7,6,4,4,1,1,1,1,1,1,1,1,1,2,2,8,2,2,3,2,2,3,2,3,4,3,3,3,2,4,2,,,5,4,3,3,1,3,3,3,4,2,4,1,3,3,2,5,2,4,2,4,1,4,1,3,2,4,,2,4,2,3,3,3,1,2,2,2,2,2,3,4,3,2,4,2,3,3,4,3,3,"2017-10-06T19:33:00.000Z"' ]) ); }); From c13028d9a3233adff42906e1ada4d966eb28f320 Mon Sep 17 00:00:00 2001 From: Addison Schiller Date: Fri, 25 Aug 2017 11:08:43 -0400 Subject: [PATCH 7/8] Don't show data preview if dataset is large Refuse to display preview data if the dataset is larger than 1000 records. Instead show an error message saying "data is too large to preview, please download". The ISP project found that after about 3000 responses, the memory usage of the browser tab was climbing to 1.5Gb. The actual JSON data was much smaller (about ~80Mb), but all the decoration added by Ember in the preview window caused it to explode. [SVCS-434] Closes: #145 --- app/components/export-tool/component.js | 15 +++++++++++++-- app/components/export-tool/template.hbs | 11 +++++++++-- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/app/components/export-tool/component.js b/app/components/export-tool/component.js index adc0c10..0ffef69 100644 --- a/app/components/export-tool/component.js +++ b/app/components/export-tool/component.js @@ -57,8 +57,15 @@ export default Ember.Component.extend({ * @property {null|string} The processed data */ processedData: Ember.computed('data', 'dataFormat', 'mappingFunction', function() { + let data = this.get('data') || []; + var show = true; + var message = ''; + if (data.length > 1000) { + show = false; + message = 'Dataset too large to preview. Please download to view.'; + } if (data.toArray) { data = data.toArray(); } @@ -69,7 +76,11 @@ export default Ember.Component.extend({ if (Ember.isPresent(dataArray)) { const mapped = mappingFunction ? dataArray.map(mappingFunction) : dataArray; - return this.convertToFormat(mapped, dataFormat); + return { + data: this.convertToFormat(mapped, dataFormat), + show: show, + message: message + }; } return null; @@ -173,7 +184,7 @@ export default Ember.Component.extend({ * Creates a file for the user to download */ downloadFile() { - const blob = new window.Blob([this.get('processedData')], { + const blob = new window.Blob([this.get('processedData.data')], { type: 'text/plain;charset=utf-8' }); diff --git a/app/components/export-tool/template.hbs b/app/components/export-tool/template.hbs index 5fbca8a..da1d18d 100644 --- a/app/components/export-tool/template.hbs +++ b/app/components/export-tool/template.hbs @@ -8,8 +8,15 @@ -
- {{textarea value=processedData class="export-tool-textarea"}} +
+ + {{#if processedData.show}} + {{textarea value=processedData.data class="export-tool-textarea"}} + + {{else}} + {{textarea value=processedData.message class="export-tool-textarea"}} + + {{/if}}
From bb021afc10b159271a4d71899b18f40baddb416f Mon Sep 17 00:00:00 2001 From: Fitz Elliott Date: Mon, 9 Oct 2017 12:16:17 -0400 Subject: [PATCH 8/8] bump version & update changelog --- CHANGELOG | 15 +++++++++++++++ package.json | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) create mode 100644 CHANGELOG diff --git a/CHANGELOG b/CHANGELOG new file mode 100644 index 0000000..7a44368 --- /dev/null +++ b/CHANGELOG @@ -0,0 +1,15 @@ +********* +ChangeLog +********* + +0.3.0 (2017-10-09) +================== +- Feature: Don't try to display a data preview if the number of records is too large. ISP +discovered that 3,000+ records can consume more than 1.5Gb of RAM in a single browser tab! Instead, +display a message asking the user to download the data. (Thanks, @AddisonSchiller!) + +- Feature: Add "study completed" timestamps (the date and time the session was last modified) to +output data. (Thanks, @AddisonSchiller!) + +- Docs: Split experiment docs into separate creation and data exploration docs. Add example and +explanation of nesting randomizers. (Thanks, @kimberscott!) diff --git a/package.json b/package.json index 41ae103..9ec00bf 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "experimenter", - "version": "0.2.3", + "version": "0.3.0", "description": "Experimenter: behavioral science research platform admin interface", "private": true, "directories": {