Failure 70 on USB Webcam Raspberry Pi

[davidric]

I'm using USB webcam on my raspberry pi. Tried to run node server-rpi.js . It's successfully run the web server. But when I try to click start video button. It showing the error bellow:
Incomming action 'REQUESTSTREAM' raspivid -t 0 -o - -w 960 -h 540 -fps 12 Failure 70

[TomlDev]

Same here with a fresh install:

pi@raspberrypi:~/player $ uname -a
Linux raspberrypi 5.4.79+ #1373 Mon Nov 23 13:18:15 GMT 2020 armv6l GNU/Linux

pi@raspberrypi:~/player $ node server-rpi.js
New guy
Incomming action 'REQUESTSTREAM'
raspivid -t 0 -o - -w 960 -h 540 -fps 12
stopping client interval
New guy
Incomming action 'REQUESTSTREAM'
raspivid -t 0 -o - -w 960 -h 540 -fps 12
Failure 70

Steps to reproduce:

1. clone repo
2. npm install
3. node server-rpi.js
4. connected to the stream in the browser
5. refreshed page -> Failure 70 when trying to connect to the stream

[Richardn2002]

Failure 70 is due to a second call to the raspivid command while a raspivid process is already running.
In the original logic a string "REQUESTSTREAM" is sent from the client as "Start Video" is pressed, and when the server receives the string it will call raspivid. So if you pressed that button twice, no matter from where, Failure 70 will be generated.
Interestingly, if you open one tab, start video, open another tab, kill the raspivid process from the server, then start video from that second tab, both tabs will be watching the stream.
This makes sense as in _server.js the h.264 data will be send forEach client. I and my roommate are also trying to achieve multiple stream watchers, so we modified the code a bit so that the server will only be starting raspivid once and will not be listening to commands sent from the client side. Now the first to log on is able to watch the stream. However (which confuses us till now), the following clients that connect to the server are unable to display anything, despite that from console.logging we can confirm h.264 data is being sent through websockets to all clients.
My current guess is that late loggers may be missing SPS and PPS frames, which puzzles the h264 decoder (Another fact to support this guess: We call raspivid as the first logger connects.). We are still working on this issue.

[Richardn2002]

After studying @pimterry 's raspivid-stream I wrote a script that can serve h264 streams through websocket to any client that logs on at any time. Now you only need h264-live-player 's client-side code
(Example:

<script type="text/javascript" src="https://rawgit.com/131/h264-live-player/master/vendor/dist/http-live-player.js"></script>
<script>
    var canvas = document.createElement("canvas");
    document.body.appendChild(canvas);

    var wsavc = new WSAvcPlayer(canvas, "webgl");

    wsavc.connect(YOUR_WEBSOCKET_URL);
</script>

)
to handle multiple stream watchers. No more Failure 70.

const WebSocketServer = require('ws').Server;

const raspivid = require('raspivid');
const stream = require('stream');
const Splitter = require('stream-split');
const NALseparator = new Buffer([0, 0, 0, 1]);

const options = {
    width: 640,
    height: 480,
    framerate: 30,
    profile: 'baseline',
    timeout: 0
};

const wsServer = new WebSocketServer({port: 8080});
wsServer.on('connection', (socket) => {
    socket.send(JSON.stringify({
        action: "init",
        width: options.width,
        height: options.height
    }));

    const initialFrames = Buffer.concat([...headerFrames, latestIdrFrame]);
    socket.send(initialFrames, {binary: true});
});
function broadcast(data){
    wsServer.clients.forEach((socket) => {
        socket.send(data, {binary: true});
    });
}

var headerFrames = [];
var latestIdrFrame = null;
const videoStream = raspivid(options)
    .pipe(new Splitter(NALseparator))
    .pipe(new stream.Transform({
        transform: function (chunk, _encoding, callback){
            const chunkType = chunk[0] & 0b11111;
            const chunkWithSeparator = Buffer.concat([NALseparator, chunk]);

            if (chunkType === 7 || chunkType === 8) {
                headerFrames.push(chunkWithSeparator);
            } else {
                if (chunkType === 5) {
                    latestIdrFrame = chunkWithSeparator;
                }
                this.push(chunkWithSeparator);
            }

            callback();
        }
    }));

videoStream.on('data', broadcast);

[sutheim]

After studying @pimterry 's raspivid-stream I wrote a script that can serve h264 streams through websocket to any client that logs on at any time. Now you only need h264-live-player 's client-side code
(Example:

<script type="text/javascript" src="https://rawgit.com/131/h264-live-player/master/vendor/dist/http-live-player.js"></script>
<script>
    var canvas = document.createElement("canvas");
    document.body.appendChild(canvas);

    var wsavc = new WSAvcPlayer(canvas, "webgl");

    wsavc.connect(YOUR_WEBSOCKET_URL);
</script>

)
to handle multiple stream watchers. No more Failure 70.

const WebSocketServer = require('ws').Server;

const raspivid = require('raspivid');
const stream = require('stream');
const Splitter = require('stream-split');
const NALseparator = new Buffer([0, 0, 0, 1]);

const options = {
    width: 640,
    height: 480,
    framerate: 30,
    profile: 'baseline',
    timeout: 0
};

const wsServer = new WebSocketServer({port: 8080});
wsServer.on('connection', (socket) => {
    socket.send(JSON.stringify({
        action: "init",
        width: options.width,
        height: options.height
    }));

    const initialFrames = Buffer.concat([...headerFrames, latestIdrFrame]);
    socket.send(initialFrames, {binary: true});
});
function broadcast(data){
    wsServer.clients.forEach((socket) => {
        socket.send(data, {binary: true});
    });
}

var headerFrames = [];
var latestIdrFrame = null;
const videoStream = raspivid(options)
    .pipe(new Splitter(NALseparator))
    .pipe(new stream.Transform({
        transform: function (chunk, _encoding, callback){
            const chunkType = chunk[0] & 0b11111;
            const chunkWithSeparator = Buffer.concat([NALseparator, chunk]);

            if (chunkType === 7 || chunkType === 8) {
                headerFrames.push(chunkWithSeparator);
            } else {
                if (chunkType === 5) {
                    latestIdrFrame = chunkWithSeparator;
                }
                this.push(chunkWithSeparator);
            }

            callback();
        }
    }));

videoStream.on('data', broadcast);

This looks very promising! Could you elaborate a little bit on how this comes together? Having a hard time incorporating these changes without errors.

[Richardn2002]

Having a hard time incorporating these changes without errors.

Um not quite sure what do you mean here? Having problems using my script? To be short: Install proper dependencies and run my script on a Raspberry Pi with a camera enabled. Then use a browser to open an HTML file like this (suppose the Pi is on 192.168.1.2):

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title></title>
</head>
<title>
</title>
<body>
<script type="text/javascript" src="https://rawgit.com/131/h264-live-player/master/vendor/dist/http-live-player.js"></script>
<script>
    var canvas = document.createElement("canvas");
    document.body.appendChild(canvas);
    var wsavc = new WSAvcPlayer(canvas, "webgl");
    wsavc.connect("ws://192.168.1.2:8080");
</script>
</body>
</html>

And you will be viewing live video streams. The difference between my version of the server side and @131 's version is that my version can handle multiple watchers and late loggers.

Could you elaborate a little bit on how this comes together?

Setting aside the apparent problem that clicking 'start feed' twice will order the server side to spawn two raspivids then Failure 70, the reason why late loggers can not view streams is that the client must receive a few 'headers' to prepare itself and start displaying videos, which late loggers will be missing if the server just mindlessly streams whatever comes out of raspivid. The headers consist of three parts:

1. Data related to @131 's websocket client packaging.
   See line 4496 of the client. The client has to first receive an object to start working:
   {action: "init", width: width, height: height}
2. Data related to h.264 encoding.
   The client has to receive Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) before actual video frames to know how to decode the video stream. SPS and PPS are given as soon as raspivid starts, so I capture and store them and deliver them to new loggers. See here for more info.
3. I frames
   I keep track of the latest IDR frame and deliver it to new loggers so that they will be watching from the latest moment.

And the above is pretty much all the background knowledge for the script. There are still some minor details not mentioned like how to process streams, what is NAL separators... just google it then :P.

I will spin up a repo for the script once I have time...
Repo set up.

[sutheim]

Thanks a lot for the explanation! I think I was having issues because I was running it with some outdated dependencies or conflicts, got it working now with a fresh install of ws, raspivid and stream-split!