Skip to content

Latest commit

 

History

History
219 lines (161 loc) · 8.51 KB

NOTEBOOK.md

File metadata and controls

219 lines (161 loc) · 8.51 KB
Raw

LOG

Major Issues Encountered

Syncing up clients

One major issue we encountered was syncing playback across clients. We noticed that when clients would call play, pause, or skip, the playback would become unsynchronized.

To tackle this issue, we had to implement server-side code that would keep track of the song frames and ensure that they were being played at the same time across all clients. We introduced a queue on the server that would keep track of the song frames that were being played.

We implemented a mechanism to synchronize the clients' playback every certain amount of time, say every half-second or second through the use of a queue keeping track of the song frames. This was done by having all clients share their current timestamps with the server and get back a current agreed-upon time for playback. If a client was ahead of the agreed-upon time, the server would send a command to pause the playback until the agreed-upon time was reached. Similarly, if a client was behind the agreed-upon time, the server would send a command to skip to the appropriate frame to catch up.

File Upload

Another major issue we encountered was file upload. We noticed that when clients would upload a file, there were many issues that could occur, including:

  1. the file would not be fully uploaded before the client would try to play the file. This would result in the client trying to play a file that was not fully uploaded, resulting in an error. In this case we decided to keep track of a list of files on the server that had been fully uploaded and were ready to be played. If a client tried to play a file that was not fully uploaded, the song would not be played and the server would return an error to the client.
  2. the file already existed on the server. In this case, we decided to keep track of a list of files on the server that had been fully uploaded and were ready to be played. If a client tried to upload a file that already existed on the server, the server would return an error to the client.

A future step for this project would be to introduce hashing of the files, which would allow for the prevention of duplicate files and faster file retrieval. This would help the service scale better and allow for more efficient file storage.

5/7

Bugs resolve:

  • Replication for simultaneously uploading files from multiple clients (broken bc servers only have one internal connection between one another)

  • “File received successfully” prints on input line

    • Queue prints as empty always

  • Each time someone queues a song, this info should be sent to all clients. They drop songs from this queue when they stream it.

  • Upload existing file bug

  • Replication

    • More testing
    • Reconnect udp sockets if server dies
    • Fix reconnect tcp sockets if server dies
    • Fix server-to-server connection (don’t use client tcp sock)

  • Unit tests

    • Test for server_paxos & client_paxos

  • Experiments/analysis

    • How upload time changes with number of clients (all uploading at same time)
    • Printing song frame + unix time to test syncing

5/6

Tasks done

  • Combining in Paxos for our replication and fault tolerance
    • Combined Paxos protocol with existing server code
    • Combined replication for file upload
    • Combined fault tolerance for server failure

TODOs

Problems/Questions:

Resources:

5/5

  • Synchonized fo playback across clients
    • Playback gets unsynchronized when clients call play/pause/skip
    • Added queue to keep track of song frames on the server
      • When the songs are paused, the queues of each of the clients sync up if they are not at the same place
  • Work on Paxos implementation TODOs
  • Combine Paxos implementation with existing server code

5/2

How do we implement synchronization across multiple clients using a centralized server?

  • Every certain amount of time (maybe every half second or second in internal clock time), have all clients share their current timestamps and get back current agreed upon time

  • Connect to UI

    • Use flask routes to connect to UI and call on existing client code

4/27

Tasks done

  • Client/System communication
    • Finished audio file upload from client to server
    • Finished audio streaming directly from server to client
    • Combined both into centralized client and server files
    • Tested concurrent upload and streaming
  • Front end
    • Implemented flask app
  • Paxos/replication

TODOs

Problems/Questions:

Resources:

4/26

Distributed Streaming Service

  • Clients connecting to server
    • Able to Upload Music - Ashley
      • TCP protocol
    • Able to Stream Music - Kevin
      • UDP protocol
    • Each client should be able to queue a song and view current queue
    • Clients should be synchronized on the playback of the current song
      • Leader server broadcasts a stream to all clients
    • Heartbeat process
      • Every <X> ms, the client sends the timestamp it is currently at to the server
      • The server has keeps track of the current timestamp (whichever client is furthest ahead)
  • Server replication
    • Leader election through Paxos - Patrick
    • Save audio files to directory (either as mp4 or bytes)
  • Frontend

Tasks done

  • Brainstorming
    • Distributed audio file storage
    • Replication
    • Paxos consensus protocol
    • Streaming audio from the server
    • Shared audio listening (everyone can control stream)
    • Frontend

TODOs

  1. Commenting
  2. Documentation
  3. Diagramming
  4. Coding
  5. The Project

Problems/Questions: *

Resources:

TODOs

Bugs

  • Client dropping crashes server
  • Upload existing file bug

Need Before Paper

  • Implement Paxos Protocol
  • Unit tests
  • Experiments/analysis?
  • The paper
  • Clean up eng notebook
  • README
    • sudo sysctl -w net.inet.udp.maxdgram=65535

Need Before Demo

  • Can’t queue files that haven’t been fully uploaded (in progress)
  • Implement Audio File Transfer
  • Error handling for incorrect file name
  • Basic replication - Patrick
    • Change server_files/ to be server_files_{port}/
  • Frontend without progress bar - Alex
    • Listing songs from actual server
    • Listing songs in queue from actual server
    • Indicator for when something is playing
    • Current song that is playing
    • Drop down for adding a song to queue
    • Make sure that the songs can actually play through flask
    • Make upload actually work
  • Determine if song has finished - Kevin
  • Interesting details:
    • Now, instead of sending audio all at once, even if a different song is still playing, we wait to start sending chunks once the last song has finished playing
    • We check if the last song has finished playing by checking if there are no more chunks in the queue
  • List available songs - Ashley
  • Fix upload wire protocol - Ashley
  • Presentation
  • Record Demo

Wow!

  • Sync up between clients
    • Consensus protocol for synchronization
    • Sharing timestamps with server
  • Implement features
    • Play
    • Pause
    • Skip (will be in conjunction with client sync up
  • Fix pyaudio on Ashley's laptop >:(

Nice to Have

  • Frame rate
  • Rejoining client(?)
  • Progress bar for uploading
  • Frontend with progress bar
  • Remove .wav from queue input / list