✨open sourcify toniefy

README.md

@@ -0,0 +1,108 @@
+# toniefy.me - Spotify on your toniebox
+
+This is the main repository for [toniefy](https://toniefy.me) - a simple and seamless
+way to transfer songs from your Spotify premium account to your [toniebox](https://tonies.com).
+
+Please keep in mind that depending on your region this might violate the Terms of Spotify.
+
+Consider it as open-source but don't expect the quality of a well-managed open source project.
+I just don't see any point of hiding the source code as it's a somewhat fancy, creative architecture
+which you won't see a lot. Consider it a tech demo 😊.
+
+## Architecture
+
+### 1. Web App
+
+This is a mostly standard web app using Elixir, the Phoenix Framework, Ecto and LiveView.
+Job handling is done via the awesome [Oban](https://github.com/sorentwo/oban) library.
+
+### 2. Recorder
+
+This is the task where the somewhat fancy architecture comes in. It consists of
+
+* a docker image (which is far from optimised in size - but hey, this doesn't matter here).
+* a custom mix task.
+
+The docker image is an Ubuntu based image with a virtual framebuffer `xfvb` and a `pulseaudio` server.
+This way we are able to run a headful chrome (headless does not work in our case), pipe
+the sound output into a null-sink and record it from there via `parec`. The output is then directly
+encoded to mp3 via the `lame` encoder.
+
+After the recording - or if something unexpected happened - we just kill and delete the container.
+
+The mix task uses [Wallaby](https://github.com/elixir-wallaby/wallaby) for browser automation.
+As soon a it's started it opens up a page from our web app `/record` with a short lived token.
+Upon load the Spotify Web playback SDK takes over and starts the playback of the URI encoded inside of the
+token. Player state updates are propagated in the DOM (via data attributes) and picked up
+by a GenServer which periodically checks for changes in the data attributes.
+
+This way we can easily send messages whenever playback finished, tracks change or the user
+interrupts the playback unexpectedly (which could be the case if the user changes the playback
+device so something else or hits stop in the spotify client).
+
+Status updates are being pushed via a simple `PUT /record/status` call. These are then propagated
+via LiveView to the UI. This enables live updates over the current recording state.
+
+Whenever a track changes, a split mark is being added to be able to split the recorded mp3 file
+into separate songs so that they can be controlled via next/prev options on the toniebox later on.
+
+If everything went well, the songs will be uploaded and from there on handled by the web app.
+
+## Running on you machine
+
+### Prerequisites
+
+1. Docker
+2. Erlang / Elixir
+3. A registered Spotify app.
+
+Fetch the dependencies:
+
+```bash
+mix deps.get
+```
+
+Setup the db
+
+```bash
+mix ecto.setup
+```
+
+Build the docker file
+
+```bash
+cd toniefy-recorder
+mix deps.get
+mix compile
+docker build -t toniex-recorder:1.0.2 .
+```
+
+### Configuration
+
+In `config/dev.exs` configure the URL where your system is hosted.
+
+```elixir
+config :toniex, Toniex.Recorder,
+  url: "https://f5f6db8b4967.eu.ngrok.io", # the public url of the phoenix server
+  docker_image_name: "toniex-recorder:1.0.2" # the docker image name you built
+
+config :ueberauth, Ueberauth.Strategy.Spotify.OAuth,
+  client_id: "your-spotify-client-id",
+  client_secret: "your-spotify-client-secret"
+```
+
+**You need a HTTPS connection, otherwise Spotify Web SDK does not work.**
+
+### Running
+
+```bash
+iex -S mix phx.server
+```
+
+## Contributing
+
+I'm happy if you like to contribute. Just open up a PR or an issue and let's discuss your ideas.
+
+## LICENSE
+
+See [LICENSE](https://github.com/benvp/toniefy/blob/main/LICENSE)

apps/toniefy-recorder/.formatter.exs

@@ -0,0 +1,4 @@
+# Used by "mix format"
+[
+  inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
+]

apps/toniefy-recorder/.gitignore

@@ -0,0 +1,31 @@
+# The directory Mix will write compiled artifacts to.
+/_build/
+
+# If you run "mix test --cover", coverage assets end up here.
+/cover/
+
+# The directory Mix downloads your dependencies sources to.
+/deps/
+
+# Where third-party dependencies like ExDoc output generated docs.
+/doc/
+
+# Ignore .fetch files in case you like to edit your project deps locally.
+/.fetch
+
+# If the VM crashes, it generates a dump, let's ignore it too.
+erl_crash.dump
+
+# Also ignore archive artifacts (built via "mix archive.build").
+*.ez
+
+# Ignore package tarball (built via "mix hex.build").
+toniex_recorder-*.tar
+
+
+# Temporary files for e.g. tests
+/tmp
+
+/screenshots
+
+.DS_Store

apps/toniefy-recorder/Dockerfile

@@ -0,0 +1,28 @@
+FROM hexpm/elixir:1.11.2-erlang-23.2.1-ubuntu-focal-20201008
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV MIX_ENV=prod
+
+WORKDIR /usr/src/app
+
+COPY . .
+
+RUN apt-get update && \
+  apt-get -yq --no-install-recommends install apt-utils && \
+  apt-get -yq --no-install-recommends install gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget x11vnc x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps xvfb && \
+  apt-get -yq install gnupg gnupg1 gnupg2 unzip wget curl apt-utils && \
+  apt-get -yq install pulseaudio pulseaudio-utils lame sox libsox-fmt-mp3 && \
+  wget https://chromedriver.storage.googleapis.com/87.0.4280.88/chromedriver_linux64.zip && \
+  unzip chromedriver_linux64.zip && \
+  mv chromedriver /usr/bin/chromedriver && \
+  chown root:root /usr/bin/chromedriver && \
+  chmod +x /usr/bin/chromedriver && \
+  curl -sSL https://dl.google.com/linux/linux_signing_key.pub | apt-key add - && \
+  echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list && \
+  apt-get update && apt-get install -y google-chrome-stable
+
+RUN adduser root pulse-access
+
+RUN mix local.hex --force && mix local.rebar --force && mix deps.get && mix compile
+
+CMD ["./bin/start.sh"]

apps/toniefy-recorder/bin/run.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+
+# Start the program in the background
+exec "$@" &
+pid1=$!
+
+# Silence warnings from here on
+exec >/dev/null 2>&1
+
+# Read from stdin in the background and
+# kill running program when stdin closes
+exec 0<&0 $(
+  while read; do :; done
+  kill $pid1
+) &
+pid2=$!
+
+# Clean up
+wait $pid1
+ret=$?
+kill $pid2
+exit $ret

apps/toniefy-recorder/bin/start.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+echo "Starting pulseaudio..."
+
+pulseaudio -D --exit-idle-time=-1 --system --disallow-exit
+
+echo "✅ Ready."
+echo "Starting recorder..."
+
+xvfb-run mix record

apps/toniefy-recorder/config/config.exs

@@ -0,0 +1,58 @@
+import Config
+
+config :toniex_recorder,
+  toniex_host: System.get_env("TONIEX_HOST"),
+  record_token: System.get_env("RECORD_TOKEN"),
+  job_id: System.get_env("TONIEX_JOB_ID"),
+  queue: System.get_env("TONIEX_QUEUE")
+
+config :tesla, adapter: Tesla.Adapter.Hackney, recv_timeout: 30_000
+
+config :wallaby,
+  driver: Wallaby.Chrome,
+  screenshot_on_failure: true,
+  chromedriver: [
+    headless: false,
+    capabilities: %{
+      javascriptEnabled: true,
+      loadImages: true,
+      version: "",
+      rotatable: false,
+      takesScreenshot: true,
+      cssSelectorsEnabled: true,
+      nativeEvents: true,
+      platform: "ANY",
+      unhandledPromptBehavior: "accept",
+      loggingPrefs: %{
+        browser: "DEBUG"
+      },
+      chromeOptions: %{
+        args: [
+          "window-size=1280,800",
+          "--no-sandbox",
+          "--disable-setuid-sandbox",
+          "--no-default-browser-check",
+          "--disable-gpu",
+          "--fullscreen",
+          "--no-first-run",
+          "--autoplay-policy=no-user-gesture-required",
+          "--user-agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
+        ]
+      }
+    }
+  ],
+  otp_app: :toniex_recorder
+
+driver_path =
+  case(:os.type()) do
+    {_, :darwin} -> Path.expand("./bin/goon-darwin")
+    {_, :linux} -> Path.expand("./bin/goon-linux")
+  end
+
+# Goon Driver somehoe doesn't work with run.sh. Dunno why
+# but we fall back to the Basic Driver. Does work fine in our case
+# ...Maybe we find some better approach instead of using Porcelain
+config :porcelain, :driver, Porcelain.Driver.Basic
+config :porcelain, goon_driver_path: driver_path
+
+import_config "#{config_env()}.exs"

apps/toniefy-recorder/config/dev.exs

@@ -0,0 +1,6 @@
+import Config
+
+config :toniex_recorder,
+  toniex_host: "http://localhost:4000",
+  record_token:
+    "SFMyNTY.g2gDdAAAAANkAAV0b2tlbmQAA25pbGQAA3VyaWQAA25pbGQAB3VzZXJfaWRtAAAAJDE1ZWFjY2NjLWZiMmYtNGNjYS05M2YyLTBhZDM0MGQ0MGZkZW4GAH_JCj93AWIAAVGA.g35pmyIbwUkAgVWb_jxruVXFDEBzOQKf8qPeGkKayiE"

apps/toniefy-recorder/config/prod.exs

@@ -0,0 +1 @@
+import Config

apps/toniefy-recorder/config/test.exs

@@ -0,0 +1,6 @@
+import Config
+
+config :toniex_recorder,
+  toniex_host: "http://localhost:4000",
+  record_token:
+    "SFMyNTY.g2gDdAAAAAJkAAV0b2tlbm0AAACcQlFDMnV4b25VZFptekFhMnZEVzNFb1l5VzhfaGQ1d002UEQ0V0Zoa1VYRUMzYjVxeFYyWGNxSDY4anlCX29WeEkwRFU3ck5ocHFYdVZjaG9rdTRPLXM0QXVsMWlqS01jWXZtMnBwYWpqclFlc1c1eWdZZUlONWVfbzJUQ2ItM2MyLTNiSTRlQ3B0VDdIVlJlS2I2Nk12cFlXWW1kZAADdXJpbQAAACRzcG90aWZ5OnRyYWNrOjJramxPWjEwWVBLM2RlTU40NWw0YlNuBgCSv8jndgFiAAFRgA.aOldUDINawWIYUfhc1vVooLpShh3pKTkfIjv7tfBd50"