-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[#131733667] Content loader to support Jinja templating #8
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Tests for fields that shouldn't be changed and a failing test for templating section fields.
Templates section.name using service_data as tempalte context.
There's no reason to do any additional processing on the section once we know it has no questions after `.filter` and therefore won't be displayed.
Keep the value as `None` if it is never assigned, otherwise run the value through jinja templating. Renamed the internal `service_data` variable to `context` to better reflect what is being passed into the .filter method.
Replaces multiple descriptions and per-lot selection for section descriptions with description templating.
Accessing section.name and section.description before calling `.filter` could return the template text, which would be displayed to the user as is. This would make catching these errors very hard, since no exception is being raised. To avoid this, we restrict access to templatable section attributes using `__getattribute__`. This raises an exception each time they're accessed on a section object that didn't have `.filter` called.
Some of our tests were accessing our newly-templatable fields before calling `filter` which now raises an Exception. Easy fix is just calling `.filter` on the sections in the tests affected.
While testing with the frontend apps, we found that the `_filtered` attribute wasn't being copied correctly or set correctly when returning a multiquestion as a section. We've since fixed the problem, and now these tests verify that everything works.
Since we're going to be templating question fields, we're going to need to be passing data into the questions themselves so that they can decide what to do with it. This commit doesn't allow any question fields to be templated, but it *does* delegate the decision around filtering questions down to the Question itself. Now a section will call `.filter` on its questions and see what comes back.
Similar to section attributes, run some of the question fields through Jinja.
Adds utils.TemplateField class that parses the field value as Jinja2 template. Creating the template from string is an expensive operation, so we want to do this once when the content file is loaded for the first time, which is why we're changing ContentLoader to replace templatable fields with TemplateField objects. TemplateField stores the original template string to make it possible to compare two instances in tests.
Using the `TEMPLATE_FIELDS` list in each class to decide which fields should be TemplateFields, and then testing that they've been set correctly.
We used to prevent access to templatable fields before the `/filter` method had been called (by checking the `_filtered` flag). Now, instead of rendering the fields when `.filter` is called, we just capture the context and then render the fields when the attribute is accessed. This means that we don't need to iterate over/overwrite field values in the `.filter` method since `__getattribute__` can check the type when fields are accessed. We can also avoid restricting access to templatable fields that don't take any context variables since the templates render without raising an error.
Takes a similar approach to ContentQuestion, rendering templates on attribute access. Since Multiquestions have nested questions we need to pass context to them the same way we do it in `section.filter`.
At the moment, some of the content fields are processed with a markdown filter to make writing question hints and descriptions easier. Since we're introducing Jinja to replay user input in question and section data we need to move the markdown processing before Jinja, since otherwise we might render user input as markdown. We're only applying markdown to multiline fields since that's where it's used by the content repo and we want to avoid wrapping single line strings with paragraph tags. This also introduces a number of issues with Markdown recognising some of the characters in Jinja tags (eg '<', '*', '_') and causing template errors by replacing them with HTML. Our main concern is that we raise these errors early since they don't seem to prevent us from doing common things we want to do with Jinja.
Any HTML tags that come out of our TemplateFields will be escaped in the actual webpage if we don't wrap them in Markup.
To prevent cross-site scripting attacks, we want to make sure that HTML or jinja templating that would be passed in through the context don't end up as valid HTML once the template is rendered. Tests make sure that HTML tags are escaped and that jinja tags are printed literally instead of being interpreted.
[Warning: It is not safe to call yaml.load with any data received from an untrusted source!](http://pyyaml.org/wiki/PyYAMLDocumentation#LoadingYAML)
Also updated CHANGELOG with summary of changes
911bb1f
to
6c85228
Compare
Previous logic would only return a question if it was not previously loaded. Meant it would return the question we wanted once but then never again.
QuestionSummaries were losing their _context and thereby failing to render after getting caught in the `__getattr__` check. Also reordered the `__getattr__` method so that it's easier to read.
In order to remove unsafe uses of `| markdown` filters we need to add TemplateField support to content messages. Unfortunately, unlike sections and questions messages don't have a common list of templatable or HTML-safe attributes. In the existing messages we have markdown fields in nested dictionaries and HTML tags within lists of strings, which means that a predefined list of templatable fields wouldn't really work. Instead, we convert all fields into TemplateFields: string fields, nested dictionary values and strings in list attributes. This also removes support for the unused sub_key arguments since we moved away from using compound keys.
pcraig3
commented
Nov 3, 2016
elif isinstance(item, collections.Sequence): | ||
return [template_all(i) for i in item] | ||
elif isinstance(item, collections.Mapping): | ||
result = {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this be a dictionary comprehension? Would look a bit more like the logic of the previous conditions.
b3f2703
to
960aadf
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note:
⚠️ This needs to come in at the same time as version 4.x.x of the frameworks repo. PR here ⚠️
What changed
Before, all values in YAML files were stored as strings and then printed out
directly by jinja in the frontend apps. Now, we're creating TemplateFields
for specific attibutes of questions and sections. TemplateFields are wrappers
around jinja Templates that are passed an initial string and then are rendered
with a context.
The result of all this is that we can pass variables to our content which
come from briefs or lots or service data or whatever else, either to be
printed as part of the content or evaluated as part of some logic operation.
These are the fields that are turned into TemplateFields when the YAML files are loaded:
name
,description
question
,name
,question_advice
,hint
ContentTemplateErrors are raised if a TemplateField is accessed (ie, rendered) without
the variables it needs. Although we don't have a way to verify that every templated
field gets all of the variables it expects in every case (we would only know
this for sure in the view logic in the frontend apps), we have tests to make sure
HTML and jinja logic is escaped before it is presented to the user.
Nothing that is currently working will break with this update, so the example
below shows how a previously static field can now be given a variable.
Example app change
Old
New
(We have also replaced
yml.load
withyml.safe_load
after reading the dire warning in the PyYAML documentation)Relevant story on Pivotal