Skip to content

Commit

Permalink
projects section content first pass
Browse files Browse the repository at this point in the history
  • Loading branch information
brettfiedler committed Jun 19, 2024
1 parent 7f7effd commit e9f93f2
Show file tree
Hide file tree
Showing 5 changed files with 62 additions and 5 deletions.
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Paper Playground is an open-source project for collaboratively creating multimod

While Paper Playground can support many uses, our aim to support a community interested in bringing physical interaction as a means to collaboratively solve problems in codesigning technology.

Paper Playground is based on the [Paper Programs](https://paperprograms.org) open-source project and has been extended to incorporate [SceneryStack](https://github.com/scenerystack), the development stack used by [PhET Interactive Simulations](https://phet.colorado.edu) as a robust 2D scene creator and manager. The project focuses on enabling quick prototyping of web projects based in JavaScript. In creating Paper Playground, we are developing with a particular emphasis on easy addition of multimodal display such as audio features (like sounds and sonifications), speech description (both TTS engines and screen reader descriptions), and other non-visual features that are often difficult to design and develop alongside visual elements in complex web projects.
Paper Playground is based on the [Paper Programs](https://paperprograms.org) open-source project and has been extended to incorporate [SceneryStack](https://github.com/scenerystack), the development stack used by [PhET Interactive Simulations](https://phet.colorado.edu) as a robust 2D scene creator and manager. The project focuses on enabling quick prototyping of web projects based in JavaScript. In creating Paper Playground, we are developing with a particular emphasis on easy addition of multimodal display such as audio features (like sounds and sonifications), speech description (both TTS engines and screen reader descriptions), and other non-visual features that are often difficult to design and develop alongside visual elements in complex web projects. Learn more on the [Multimodal Codesign page](./projects/codesign.md).

### Overview of Paper Playground Components

Expand Down
36 changes: 35 additions & 1 deletion docs/projects/codesign.md
Original file line number Diff line number Diff line change
@@ -1 +1,35 @@
{placeholder}
# Paper Playground as a Tool for Multimodal Interactive Codesign

The power of Paper Playground, augmented with [Creator](../setup/creator.md) code abstraction and [Scenery-based](https://scenerystack.github.io/community/) Display, lies in its ability to combine the physicality of paper and moving real objects, with the scalable affordances of native web technologies to introduce sounds, speech, device communication, and more. Paper Playground lets us quickly prototype interactive and multimodal experiences, which is especially helpful for working with others to iterate quickly on ideas that will go on to become full web experiences (games, simulations, etc).

## But why use Paper Playground to prototype a web project?

### Multimodal Design is Hard

Multimodal design, incorporating visuals, sounds/sonifications, and speech (including spoken descriptions for accessible projects), is the ideal goal of any web experience.
Visual design is hard. Multimodal design is **harder**. Multimodal codesign is ***even harder***. Working with others, helping them to understand the relationships between the interactive objects in your project, while finding the right mapping between how someone interacts and the sounds, speech, or visuals that play/change is a constant challenge for any designer.

We also emphasize that **multimodal design should be done at the start and throughout the process and not after the visual design has already completed**.

### Where are the multimodal design tools?

Tools for visual web design prototypes and interaction design exist on every corner of the web (Figma, Canva, etc.). However, there is no tool to help with multimodal design.

A goal of Paper Playground is to explore the capabilities of a tool that can help highlight the relationships between model (data/logic) components, the interactions that change them, and the ways that (visual/auditory) feedback are provided.

### Get out of the screen

We know that collaboration in the real world is best supported when users can make use of the space around them and use their bodies to gesture and enact their ideas. Codesign practices often make use of paper or craft lo-fi prototyping at the initial stages of an idea. But, as previously emphasized, **sound and speech design should begin as early as possible!** So, why not make it easy to augment lo-fi prototypes with those auditory (and more!) features that are afforded easily by virtual interfaces?

## Follow our research

Follow our research by checking out our [Open Science Framework repository](https://osf.io/6ad5g/)

Check out our [2024 paper at the Interaction Design for Children conference](https://dl.acm.org/doi/10.1145/3628516.3659400) that explores using Paper Playground as a design probe for understanding the needs and desires for a remote multimodal co-design tool as identified by young developers!

## Attributions

This project was developed with the [Craft Tech Lab](https://cucraftlab.org/) and [PhET Interactive Simulations](https://phet.colorado.edu/en/inclusive-design) at University of Colorado, Boulder.

!!! info
This material is based upon work supported by the National Science Foundation under Award [#2119303](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2119303&HistoricalAwards=false).
8 changes: 7 additions & 1 deletion docs/projects/craft.md
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
{placeholder}
# Mixing Paper Crafting with Paper Playground

Paper Playground provides a novel way to integrate craft with STEM exploration. The paper programs can be incorporated into paper sculpture — or other craft-based objects — enabling students to personalize their learning experiences. In addition to building paper controllers (such as the RGB sliders pictured above), students might also create an entire scene or narrative around STEM concepts. This type of tangible interface links the playful affordances of paper with computational elements.

Learn more at the [CU Craft Tech Paper Playground page](https://cucraftlab.org/paper-playground/).

The Paper Playground demo projects using paper crafting are included with Paper Playground by default in *Creator* when using local storage (see `root/server/data/default-data`). Find the templates used in the demos at the [CU Craft Tech website](https://cucraftlab.org/paper-playground-templates/).
20 changes: 19 additions & 1 deletion docs/projects/microcontroller.md
Original file line number Diff line number Diff line change
@@ -1 +1,19 @@
{placeholder}
# Microcontroller Integration with Paper Playground over Bluetooth

## Paper meets circuits

You can now communicate with microcontrollers over bluetooth! This expands the already near infinite capabilities of Paper Playground to enhance the tangible possibilities.

You can create a hybrid tangible interface using paper-based events (paper movement, paper size, markers, paper overlap, etc) from Paper Playground, while triggering actuators connected to your microcontroller. Or, vice versa... take in sensor signals from your microcontroller, pipe a message to Paper Plaground, and trigger animations, sounds, speech, and anything else you can do on the web using Paper Playground!

We currently have the most support for the [BBC micro:bit](), but there are bluetooth service UUIDs available for more generic microcontrollers with bluetooth capabilities.

## Getting Started

To get started, create a `controller` component in *Creator* and select the "Bluetooth" tab. Select the bluetooth service you want to use, and then the characteristic of that service. For example, if you want to write a string to the micro:bit, select the UART service, and the RX characteristic (which is the "read" service on the micro:bit).

You will find demo projects using the micro:bit installed by default in Paper Playground in the *Creator* interface (or look in the directory `root/server/data/default-data`).

## Walkthrough

For a complete walkthrough using the micro:bit microcontroller, see the published [Instructables] (<https://www.instructables.com/Microbit-and-Paper-Playground-Integration-for-Enha/>) or [Hackster.io project](https://www.hackster.io/brfi7385/micro-bit-and-paper-playground-tangible-virtual-interfaces-184685).
1 change: 0 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,6 @@ nav:
- Multimodal Codesign: projects/codesign.md
- Paper Craft: projects/craft.md
- Microcontroller Integration: projects/microcontroller.md
- SceneryStack: projects/scenerystack.md
- Media Gallery: media-gallery.md
- Resources:
- Walkthroughs: use/walkthroughs/walkthroughs.md
Expand Down

0 comments on commit e9f93f2

Please sign in to comment.