Audio Library Documentation and Audio in Pong

docs/api/api-c-audio.md

@@ -0,0 +1,121 @@
+---
+title: Audio API for WebAssembly
+description: twr-wasm provides an audio C API that allows Wasm code to call a subset of the JS Audio API.
+---
+
+# Audio API for WebAssembly
+
+This section describes twr-wasm's C Audio API, which allows audio API functions to be called from WebAssembly.
+
+## Examples
+| Name | View Live Link | Source Link |
+| - | - | -
+| Pong (C++) | [View Pong](/examples/dist/pong/index.html) | [Source for Pong](https://github.com/twiddlingbits/twr-wasm/tree/main/examples/pong) |
+| tests-audio | [View tests-audio](/examples/dist/tests-audio/index.html) | [Source for tests-audio](https://github.com/twiddlingbits/twr-wasm/tree/main/examples/tests-audio) |
+
+## Code Example
+~~~c title="Play Audio"
+#include "twr-audio.h"
+#include <math.h>
+#include <stdlib.h>
+
+#define M_PI 3.14159265358979323846
+
+void play() {
+   twr_audio_play_file("example.mp3"); //plays audio from specified URL
+
+   const long SAMPLE_RATE = 48000; //48,000 samples per second
+   const double DURATION = 10.0; //play for 10 seconds
+   const double freq = 493.883; //Middle B (B4)
+
+   long length = (long)ceil(SAMPLE_RATE*DURATION);
+   //PCM audio data in the form of -1.0 to 1.0
+   float* wave = (float*)malloc(sizeof(float) * length);
+
+   //generate square wave at specified frequency and duration
+   for (long i = 0; i < length; i++) {
+      wave[i] = cos(2*M_PI*freq*(i/(float)sample_rate)) > 0 ? 1 : -1;
+   }
+
+   //creates a mon-audio channel buffer at our given SAMPLE_RATE
+   // and square-wave data we generated
+   long node_id = twr_audio_from_samples(1, SAMPLE_RATE, wave, length);
+
+   //plays the square wave
+   // Can be played multiple times, and is only freed on twr_audio_free
+   twr_audio_play(node_id);
+}
+~~~
+
+## Overview
+The Audio API is part a twr-wasm library that can be accessed via `#include "twr-audio.h"`. It has two main ways to play audio: from raw PCM data and from a URL. 
+
+Raw PCM data can be setup via twr_audio_from_samples or twr_audio_load. twr_audio_from_samples takes in the PCM data as an array of floats between -1.0 and 1.0. twr_audio_load, on the other hand, takes in the URL of an audio file, downloads it, and converts it to PCM data. This method does not stream the audio, so it might take longer on startup. Once the PCM data is setup, it can be played via functions like twr_audio_play, twr_audio_play_range, twr_audio_play_sync, and twr_audio_play_range_sync.
+
+You can also play audio directly from a URL. Unlike twr_audio_load, the url is loaded directly into an HTMLAudioElement which streams the audio and plays it imediately.
+
+In addition to playing audio, there are functions that allow you to query and modify an ongoing playback. These include stopping the playback, getting how long it's been playing, and modifying the pan; volume; or playback rate of the audio.
+
+## Notes
+When playing audio, a playback_id is returned to query or modify the playback. However, the playback is automatically deleted when it ends making the playback_id invalid as it does so. Most functions that take in a playback_id will simply return a warning and return without error. The only slight exception is twr_audio_query_playback_position which will return -1 when given a dead or invalid playback_id.
+
+## Functions
+These are the current Audio APIs available in C:
+
+~~~c
+long twr_audio_from_samples(long num_channels, long sample_rate, float* data, long singleChannelDataLen);
+
+long twr_audio_play(long node_id);
+long twr_audio_play_volume(long node_id, double volume, double pan);
+long twr_audio_play_callback(long node_id, double volume, double pan, int finish_callback);
+
+struct PlayRangeFields {
+   double pan, volume;
+   int loop, finish_callback;
+   long sample_rate;
+};
+struct PlayRangeFields twr_audio_default_play_range();
+long twr_audio_play_range(long node_id, long start_sample, long end_sample);
+long twr_audio_play_range_ex(long node_id, long start_sample, long end_sample, struct PlayRangeFields* fields);
+
+long twr_audio_play_sync(long node_id);
+long twr_audio_play_sync_ex(long node_id, double volume, double pan);
+
+
+struct PlayRangeSyncFields {
+   double pan, volume;
+   int loop;
+   long sample_rate;
+};
+struct PlayRangeSyncFields twr_audio_default_play_range_sync();
+long twr_audio_play_range_sync(long node_id, long start_sample, long end_sample);
+long twr_audio_play_range_sync_ex(long node_id, long start_sample, long end_sample, struct PlayRangeSyncFields* fields);
+
+long twr_audio_load_sync(char* url);
+long twr_audio_load(int event_id, char* url);
+long twr_audio_query_playback_position(long playback_id);
+float* twr_audio_get_samples(long node_id, long* singleChannelDataLenPtr, long* channelPtr);
+void twr_audio_free_id(long node_id);
+
+void twr_audio_stop_playback(long playback_id);
+
+void twr_audio_modify_playback_volume(long node_id, double volume);
+void twr_audio_modify_playback_pan(long node_id, double pan);
+void twr_audio_modify_playback_rate(long node_id, double sample_rate);
+
+long twr_audio_play_file(char* file_url);
+long twr_audio_play_file_ex(char* file_url, double volume, double playback_rate, int loop);
+struct AudioMetadata {
+   long length;
+   long sample_rate;
+   long channels;
+};
+
+void twr_audio_get_metadata(long node_id, struct AudioMetadata* metadata);
+
+
+float* twr_convert_8_bit_pcm(char* pcm, long pcm_len);
+float* twr_convert_16_bit_pcm(short* pcm, long pcm_len);
+float* twr_convert_32_bit_pcm(long* pcm, long pcm_len);
+~~~
+

docs/examples/examples-overview.md

@@ -54,6 +54,7 @@ These examples are a good place to learn how to configure clang and wasm-ld to c
 | tests-user | "cli" for tests using libc++ and `<canvas>` | [tests-user](/examples/dist/tests-user/index.html) |
 | tests-libcxx | Smoke test for libc++.  Shows how to use libc++. | [tests-libcxx](examples-libcxx.md) |
 | tests-d2d | Unit tests for Draw 2D canvas console | [tests-d2d](examples-tests-d2d.md) |
+| tests-audio | Unit tests for the Audio Library | [tests-audio](examples-tests-audio.md) |
 
 ## Running or Building the examples locally
 Online versions of the examples [can be viewed here.](https://twiddlingbits.dev/examples/dist/index.html)  

docs/examples/examples-pong.md

@@ -16,6 +16,7 @@ The Pong example demonstrates
 * Using the twr-wasm canvas.cpp class.
 * A custom typescript library
 * User mouse and keyboard input via events
+* Using the Audio Library to play simple sounds
 
 This example does not use libc++, which results in smaller code size.   For an example that uses libc++ see [tests-libcxx](examples-libcxx.md).
 

docs/examples/examples-tests-audio.md

@@ -0,0 +1,14 @@
+---
+title: Audio API WebAssembly Unit Tests
+description: Simple unit tests of the various Audio functions
+---
+
+# tests-audio - Unit tests for Audio Library
+This is a simple set of Unit Tests for testing the Audio API functions. It includes a test for each Audio function. WARNING: Some of the audio played can be loud, so turn down your volume before playing.
+
+- [view tests-audio example running live](/examples/dist/tests-audio/index.html)
+- [View tests-audio source code](https://github.com/twiddlingbits/twr-wasm/tree/main/examples/tests-audio)
+
+Also see these WebAssembly programs that use this API
+
+- [Pong](examples-pong.md)

examples/pong/Makefile

@@ -7,10 +7,10 @@ CFLAGS_DEBUG := -c -Wall -g -O0  $(TWRCFLAGS) -I $(CPPLIB)
 OBJOUTDIR := out
 $(info $(shell mkdir -p $(OBJOUTDIR)))
 
-HEADERS := pong-menu.h $(CPPLIB)/canvas.h pong.h two-player-pong.h
+HEADERS := pong-menu.h $(CPPLIB)/canvas.h pong.h two-player-pong.h extra.h
 
 # put all objects into a single directory; assumes unqiue filenames
-OBJECTS_RAW := pong-menu.o canvas.o entry-point.o pong.o two-player-pong.o
+OBJECTS_RAW := pong-menu.o canvas.o entry-point.o pong.o two-player-pong.o extra.o
 OBJECTS := $(patsubst %, $(OBJOUTDIR)/%, $(OBJECTS_RAW))
 OBJECTS_ASYNC := $(patsubst %, $(OBJOUTDIR)/async-%, $(OBJECTS_RAW))
 

examples/pong/extra.cpp

@@ -0,0 +1,21 @@
+#include "extra.h"
+#include <math.h>
+#include <stdlib.h>
+
+#define M_PI 3.14159265358979323846
+
+float* generate_square_wave(double frequency, double duration, long sample_rate) {
+   long length = (long)ceil(duration * sample_rate);
+   float* wave = (float*)malloc(sizeof(float) * length);
+   for (long i = 0; i < length; i++) {
+      wave[i] = cos(2*M_PI*frequency*(i/(float)sample_rate)) > 0 ? 1 : -1;
+   }
+   return wave;
+}
+
+long load_square_wave(double frequency, double duration, long sample_rate) {
+   float* wave = generate_square_wave(frequency, duration, sample_rate);
+   long node_id = twr_audio_from_samples(1, sample_rate, wave, (long)ceil(duration * sample_rate));
+   free(wave);
+   return node_id;
+}

examples/pong/extra.h

@@ -0,0 +1,4 @@
+#include "twr-audio.h"
+
+float* generate_square_wave(double frequency, double duration, long sample_rate);
+long load_square_wave(double frequency, double duration, long sample_rate);

examples/pong/pong.cpp

@@ -1,7 +1,11 @@
 #include "pong.h"
+#include "twr-audio.h"
+
+#include "extra.h"
 
 #define M_PI 3.14159265358979323846
 
+
 void Pong::endGame() {
     this->game_running = false;
 }
@@ -32,6 +36,9 @@ Pong::Pong(double width, double height, colorRGB_t border_color, colorRGB_t back
     this->paddle_color = paddle_color;
     this->ball_color = ball_color;
 
+    this->bounce_noise = load_square_wave(493.883, 0.025, 48000);
+    this->lose_noise = load_square_wave(440, 0.25, 48000);
+
     #ifdef ASYNC
     bool image_loaded = d2d_load_image("https://images.pexels.com/photos/235985/pexels-photo-235985.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1", background_image_id);
     assert(image_loaded);
@@ -49,6 +56,9 @@ Pong& Pong::operator=(const Pong& copy) {
     this->paddle_color = copy.paddle_color;
     this->ball_color = copy.ball_color;
 
+    this->bounce_noise = copy.bounce_noise;
+    this->lose_noise = copy.lose_noise;
+
     this->resetGame();
 
     return *this;
@@ -260,17 +270,21 @@ void Pong::tickBall(long delta) {
     if (this->ball_x <= this->border_width) { //left wall
         this->ball_x = this->border_width;
         this->ball_velocity_x *= -1;
+        twr_audio_play(this->bounce_noise);
     } else if (this->ball_x >= this->width - this->ball_size - this->border_width) { //right wall
         this->ball_x = this->width - this->ball_size - this->border_width;
         this->ball_velocity_x *= -1;
+        twr_audio_play(this->bounce_noise);
     } 
 
     //x and y are seperate checks for the corner case
     if (this->ball_y <= border_width) { //top wall
         this->ball_y = this->border_width;
         this->ball_velocity_y *= -1;
+        twr_audio_play(this->bounce_noise);
     } else if (this->ball_y >= this->height - this->ball_size - this->border_width) { //bottom wall, lost game
         this->ball_y = this->height - this->ball_size - this->border_width - 1.0;
+        twr_audio_play(this->lose_noise);
         this->endGame();
     }
 
@@ -310,6 +324,7 @@ void Pong::tickBall(long delta) {
             //set score time
             this->score_time = this->last_timestamp;
         }
+        twr_audio_play(this->bounce_noise);
     }
 }
 void Pong::tickPaddle(long delta) {

examples/pong/pong.h

@@ -69,6 +69,9 @@ class Pong {
     long score_time = 0;
     bool game_running = true;
 
+    long bounce_noise;
+    long lose_noise;
+
     void renderBackground();
     void renderBorder();
     void renderPaddle();

examples/pong/two-player-pong.cpp

@@ -2,6 +2,9 @@
 #include<time.h>
 #include<stdlib.h>
 #include <math.h>
+#include "twr-audio.h"
+
+#include "extra.h"
 
 #define M_PI 3.14159265358979323846
 
@@ -17,6 +20,9 @@ const colorRGB_t BALL_COLOR = 0xFFFFFF;
 const colorRGB_t PADDLE_ONE_COLOR = 0xFFFFFF;
 const colorRGB_t PADDLE_TWO_COLOR = 0xFFFFFF;
 
+
+
+
 TwoPlayerPong::TwoPlayerPong() {
    this->width = 0.0;
    this->height = 0.0;
@@ -27,7 +33,8 @@ TwoPlayerPong::TwoPlayerPong(double width, double height, bool hasAI) {
    this->width = width;
    this->height = height;
    this->hasAI = hasAI;
-
+   this->bounce_noise = load_square_wave(493.883, 0.025, 48000);
+   this->score_noise = load_square_wave(440, 0.05, 48000);
    srand(time(NULL));
 
    this->resetGame();
@@ -37,6 +44,8 @@ TwoPlayerPong& TwoPlayerPong::operator=(const TwoPlayerPong& copy) {
    this->width = copy.width;
    this->height = copy.height;
    this->hasAI = copy.hasAI;
+   this->bounce_noise = copy.bounce_noise;
+   this->score_noise = copy.score_noise;
 
    this->resetGame();
 
@@ -327,7 +336,7 @@ T better_abs(T val) {
 }
 
 
-void paddleCollision(Ball& ball, double& n_x, double& n_y, Paddle& paddle, double paddle_x){
+void paddleCollision(Ball& ball, double& n_x, double& n_y, Paddle& paddle, double paddle_x, long bounce_noise){
 
    double paddle_middle = paddle.y + PADDLE_HEIGHT/2.0;
    double ball_middle = ball.y + BALL_HEIGHT/2.0;
@@ -354,6 +363,8 @@ void paddleCollision(Ball& ball, double& n_x, double& n_y, Paddle& paddle, doubl
          n_y = paddle.y + PADDLE_HEIGHT;
          ball.v_y = better_abs(ball.v_y);
       }
+
+      twr_audio_play(bounce_noise);
 
       //add paddle velocity to ball
       double paddle_vel = get_paddle_vel(paddle);
@@ -385,29 +396,33 @@ void TwoPlayerPong::updateBall(double delta) {
       //interpolate to add the extra y back in th eopposite direction
       n_y = 0 - n_y;
       this->ball.v_y *= -1;
+      twr_audio_play(this->bounce_noise);
    } else if (n_y > max_y) {
       //bounce off bottom wall by flipping y direction
       //interpolate to add the extra y back in the opposite direction
 
       //max_y - (n_y - max_y) = max_y + max_y - n_y = 2*max_y - n_y
       n_y = 2*max_y - n_y;
       this->ball.v_y *= -1;
+      twr_audio_play(this->bounce_noise);
    }
 
    if (n_x < 0) { //hit left
       this->ballScored(false);
+      twr_audio_play(this->score_noise);
       return;
    } else if (n_x > max_x) {
       this->ballScored(true);
+      twr_audio_play(this->score_noise);
       return;
    }
 
 
    //left paddle
-   paddleCollision(this->ball, n_x, n_y, paddleOne, PADDLE_OFFSET);
+   paddleCollision(this->ball, n_x, n_y, paddleOne, PADDLE_OFFSET, this->bounce_noise);
 
    //right paddle
-   paddleCollision(this->ball, n_x, n_y, paddleTwo, this->width - PADDLE_OFFSET - PADDLE_WIDTH);
+   paddleCollision(this->ball, n_x, n_y, paddleTwo, this->width - PADDLE_OFFSET - PADDLE_WIDTH, this->bounce_noise);
 
    this->ball.x = n_x;
    this->ball.y = n_y;

examples/pong/two-player-pong.h

@@ -66,6 +66,9 @@ class TwoPlayerPong {
    Vec2D<double> winner_pos;
    Vec2D<double> reset_pos;
 
+   long bounce_noise;
+   long score_noise;
+
 
 
    void renderBackground();