Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GSoC work product and contributor docs #2

Merged
merged 3 commits into from
Aug 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions contributor_docs/project_wrapups/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ This folder contains wrapup reports for projects from p5.js related [Google Summ

*Note for contributors: Embedded images and media are welcome. Please host these files externally rather than placing in this repo to avoid adding growing the repository filesize too much.*

### Google Summer of Code 2020
* [p5.js accessibility and canvas descriptions](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/luismn_gsoc_2020.md) by Luis Morales-Navarro, 2020

### Google Summer of Code 2019
* [Search Bar for Sketches in the p5.js Web Editor](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/rachellim_gsoc_2019.md) by Rachel Lim, 2019
Expand Down
53 changes: 53 additions & 0 deletions contributor_docs/project_wrapups/luismn_gsoc_2020.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# p5.js accessibility and canvas descriptions
GSoC 2020 | [Luis Morales-Navarro](https://luismn.com/)

### Overview:
During this Google Summer of Code, I worked with [Kate Hollenbach](https://github.com/kjhollen)
to improve the accessibility features of p5.js. We focused on merging the
text output and table output functionalities of [p5.accessibility](https://github.com/processing/p5.accessibility)
into p5.js and created functions that support p5.js users in writing their own screen reader accessible canvas descriptions.

### Background:
#### p5.js and Accessibility: from an editor feature to an add-on to the library
The work done during this summer is part of the project's [efforts to make p5.js more accessible for persons with dissabilities](https://contributors-zine.p5js.org/#reflection-claire-kearney-volpe).
Early work by Claire Kearney-Volpe, Taeyoon Choi, and Atul Varma identified the need to make p5.js sketches
and the canvas accessible to screen readers and people who are blind. I met Claire in late 2016 when
they were working with Mathura Govindarajan to add accessibility features to the p5.js editor. I joined them
and together with the support of dedicated contributors and advisors (including Cassie Tarajakan, Lauren McCarthy,
Josh Mielle, Sina Bahram, and Chancey Fleet) we implemented three accessible canvas outputs (a text output, a grid
output and a sound output) on the alpha editor.

Later on, through a 2018 Processing Foundation Fellowship Claire, Mathura and I developed p5.accessibility.js a p5.js add-on.
p5.accessibility.js (developed with contributions from Antonio Guimaraes, Elizabeth G Betts, Mithru Vigneshwara, and Yossi Spira)
helped us bring the work we had done with accessible outputs in the editor to any p5.js sketch that included the add-on.
However, the add-on was still an add-on that required users to include an extra file and edit their html.

At the 2019 p5.js Contributors Conference, as a community, we reinforced the project's commitment to access and inclussion.
Together with Claire, Sina, Lauren, Kate, Olivia McKayla Ross and Evelyn Masso we outlined the pathway forward.
Among short-term actions, we identified the need for functions that allow users to write their own descriptions
and the importance of merging the add-on into the p5.js library.

### Contributions:
During the course of Summer of Code, my work focused on creating library generated screen reader accessible outputs
for basic shapes on the canvas and functions to support user-generated screen reader accessible descriptions of canvas content.
I worked on the following PRs:
- [Add describe() and describeElement() #4654](https://github.com/processing/p5.js/pull/4654): This PR adds the functions describe() and describeElement(), tests for these functions, documentation and examples.
- [Merge Accessibility Add-On into p5.js #4703](https://github.com/processing/p5.js/pull/4703): This PR adds the functions textOutput() and gridOutput(), helper functions to create and update, the outputs and tests, documentation and examples. At first the plan was to update the add-on and prepare it for merging it with p5.js in the near future. However, we realized it was more time effective to recreate the functionality of the text output and grid output in p5.js than upgrading the add-on which relied on ["monkey patching," entities and interceptors](https://medium.com/processing-foundation/making-p5-js-accessible-e2ce366e05a0). Now, the outputs are fully integrated to the library.

More information on how these accessibility features work is available in the [web accessibility contributor docs](https://github.com/processing/p5.js/blob/main/contributor_docs/web_accessibility.md).

### Future
- There is a lot of work that can be done to improve the accessibility of p5.js sketches. In the [Web accessibility next steps conversation #4721 Issue](https://github.com/processing/p5.js/issues/4721) we have outlined some ideas and questions.
- The work done during the summer focused on code and code issues but it is important to iteratively test these features with members of the community, particularly novices and learners who are blind. It is also important to create more resources for learning and teaching that support accessibility.
- Immediate next steps include:
- A tutorial on how to describe things on the canvas.
- Changes in the way screen-reader descriptions are created in the reference. Using the describe() function instead of relying on @alt
- Maybe adding describe() to the templates on the website and editor
- Upgrading the tutorial on using p5 with a screen reader
- Changing the way the accessibility settings work on the editor

### Acknowledgements
I am grateful to Kate Hollenbach for their guidance, feedback and assistance, to Lauren McCarthy for their feedback and to Claire Kearney-Volpe for helping me come up with this project. Thanks to Sina Bahram for their input —our conversations at 2019 p5.js Contributors Conference inspired the describe() and describeElement() functions—, and to Akshay Padte for their advice on unit testing. This GSoC project would not have been possible without Chancey Fleet and Claire (who started thinking of ways to make p5.js sketches screen reader accessible in late 2015), the work of Mathura Govindarajan, and of many other contributors and supporters in the p5.js community.

:heart:

12 changes: 6 additions & 6 deletions contributor_docs/web_accessibility.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# p5.js Web Accessibility

This document describes the structure of the web accessibility features of p5.js for contributors and maintainers—and any other interested parties. If you're interested in making your sketches [screen reader](https://en.wikipedia.org/wiki/Screen_reader) accessible, visit the [tutorial]() or if you want to use p5.js with a screen reader visit the [Using p5.js with a Screen Reader tutorial]().
This document describes the structure of the web accessibility features of p5.js for contributors and maintainers—and any other interested parties. If you're interested in making your sketches [screen reader](https://en.wikipedia.org/wiki/Screen_reader) accessible, visit the [tutorial](https://p5js.org/learn) or if you want to use p5.js with a screen reader visit the [Using p5.js with a Screen Reader tutorial](https://p5js.org/learn/p5-screen-reader.html).

## Overview

Expand All @@ -19,10 +19,10 @@ Supported accessible outputs for basic shapes include text and grid outputs.
If a user passes `LABEL` as a parameter in either of these functions, an additional div with the output adjacent to the canvas is created, this is useful for non-screen reader users that might want to display the output outside of the canvas' sub DOM as they code. However, using `LABEL` will create unnecessary redundancy for screen reader users. We recommend using `LABEL` only as part of the development process of a sketch and removing it before publishing or sharing your sketch with screen reader users.

### Outputs structure
Although `textOutput()` and `gridOutput()` are located in [src/accessibility/outputs.js](), the outputs are created and updated using functions distributed across the library. This section details the different functions that support the accessible outputs.
Although `textOutput()` and `gridOutput()` are located in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js), the outputs are created and updated using functions distributed across the library. This section details the different functions that support the accessible outputs.

#### outputs.js
[src/accessibility/outputs.js]() includes the core functions that create the accessible outputs:
[src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js) includes the core functions that create the accessible outputs:
* `textOutput()`: This function activates the text output by setting `this._accessibleOutputs.text` to `true` and calling `_createOutput('textOutput', 'Fallback')`. If `LABEL` is passed as a parameter the function also activates the text output label by setting `this._accessibleOutputs.textLabel` as `true` and calls `_createOutput('textOutput', 'Label')` for the label.
* `gridOutput()`: This function activates the grid output by setting `this._accessibleOutputs.grid` to `true` and calling `_createOutput('gridOutput', 'Fallback')`. If `LABEL` is passed as a parameter the function also activates the grid output label by setting `this._accessibleOutputs.textLabel` as `true` and calls `_createOutput('gridOutput', 'Label')` for the label.
* `_createOutput()`: This function creates the HTML structure for all accessible outputs. Depending on the type and display of the outputs the HTML structure created varies. The function also initializes `this.ingredients` which stores all the data for the outputs including: shapes, colors, and pShapes (which stores a string of the previous shapes of the canvas). It also creates `this.dummyDOM` if it doesn't exist. `this.dummyDOM` stores the HTMLCollection of DOM elements inside of `<body>`.
Expand Down Expand Up @@ -56,23 +56,23 @@ When `this._accessibleOutputs.text` or `this._accessibleOutputs.text` are `true`
* `p5.Renderer2D.prototype.background()`

#### textOutput.js
[src/accessibility/textOutput.js]() contains all functions that update the text output. The main function in this file is `_updateTextOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js]() when `this._accessibleOutputs.text` or `this._accessibleOutputs.textLabel` are `true.`
[src/accessibility/textOutput.js](https://github.com/processing/p5.js/blob/main/src/accessibility/textOutput.js) contains all functions that update the text output. The main function in this file is `_updateTextOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js) when `this._accessibleOutputs.text` or `this._accessibleOutputs.textLabel` are `true.`

`_updateTextOutput()` uses `this.ingredients` to build the content of the text output and text output label which include a summary, a list of shapes, and a table of shapes details. If these are different from the current outputs it updates them. Building the output content is supported by several helper functions in the file that are not part of the prototype:
* `_textSummary()`: Builds the content of the text output summary.
* `_shapeDetails()`: Builds the text output table that contains shape details.
* `_shapeList()`: Builds the list of shapes of the text output.

#### gridOutput.js
[src/accessibility/gridOutput.js]() contains all functions that update the grid output. The main function in this file is `_updateGridOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js]() when `this._accessibleOutputs.grid` or `this._accessibleOutputs.gridLabel` are `true.`
[src/accessibility/gridOutput.js](https://github.com/processing/p5.js/blob/main/src/accessibility/gridOutput.js) contains all functions that update the grid output. The main function in this file is `_updateGridOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js) when `this._accessibleOutputs.grid` or `this._accessibleOutputs.gridLabel` are `true.`

`_updateGridOutput()` uses `this.ingredients` to build the content of the grid output and grid output label which include a summary, a grid that maps the location of shapes and a list of shapes. If these are different from the current outputs it updates them. Building the output content is supported by several helper functions in the file that are not part of the prototype:
* `_gridSummary()`: Builds the content of the grid output summary.
* `_gridMap()`: Builds a grid that maps the location of shapes on the canvas.
* `_gridShapeDetails()`: Builds the list of shapes of the grid output, each line of the list includes details about the shape.

#### color_namer.js
When creating screen reader accessible outputs, naming the colors used in the canvas is important. [src/accessibility/color_namer.js]() contains `_rgbColorName()` a function that receives rgba values and returns a color name. This function is called by `_accsBackground()` and `_accsCanvasColors` in `src/accessibility/outputs.js`.
When creating screen reader accessible outputs, naming the colors used in the canvas is important. [src/accessibility/color_namer.js](https://github.com/processing/p5.js/blob/main/src/accessibility/color_namer.js) contains `_rgbColorName()` a function that receives rgba values and returns a color name. This function is called by `_accsBackground()` and `_accsCanvasColors` in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js).

`_rgbColorName()` uses `color_conversion._rgbaToHSBA()` to get the hsv values of the color and then uses `_calculateColor()` to get the color name. The function `_calculateColor()` in this file comes from [colorNamer.js](https://github.com/MathuraMG/color-namer) which was developed as part of a [2018 Processing Foundation fellowship](https://medium.com/processing-foundation/making-p5-js-accessible-e2ce366e05a0) and in consultation with blind screen reader expert users. This function returns color names by comparing hsv values to those stored in the `colorLookUp` array. The function should be updated as some shades of gray are not named correctly. When updating it, it is also important to ensure contributor readability by including comments that explain what each line of code does.

Expand Down