ALL twitch.tv issues

[PhiZero]

URL(s) where the issue occurs

twitch.tv

Describe the issue

Embedded video ads are showing again after the fix from #5138 that lasted until now.

Screenshot(s)

Versions

• Browser/version: Chrome 73.0.3683.75
• uBlock Origin version: 1.18.0

Settings

Default settings.

Notes

Fix that was pushed as part of issue 5138 worked until a few hours ago, now getting embedded ads again. No changes made between then and now. Maybe a recent filter change broke this, or maybe Twitch patched this workaround?

[lain566]

The @pixeltris's solution has been broken.

[pixeltris]

The next logical step is removing the ad chunks in the stream. This unfortunately can add quite a delay until the actual stream starts. See https://github.com/instance01/Twitch-HLS-AdBlock

EDIT: If you play around with the params to the access_token request there seem to be more workarounds. Probably futile to do so though? Maybe implement something similar to Twitch-HLS-AdBlock and then add any additional workarounds as they are found?

[stoically]

I've just noticed that the initial request, when entering a stream, to https://usher.ttvnw.net/api/channel/hls/<channelname>.m3u8 (serves the playlists for the different encodings) contains a parameter called show_ads set to true. Might be worth trying to rewrite that one to false.

[TheEnhas]

The platforms (aside from the regular site) are listed here, but I'm not sure what the actual values to use are for each: https://www.twitch.tv/p/platforms/

Might be worth trying though. I think it's odd that platform being undefined worked (for a while) to prevent ads, which means that was probably the default and that the other values are also meant to enable them. On reddit someone (supposedly) doesn't get ads via FireTV so that one may work.

[pixeltris]

@stoically that request includes a "sig" param which looks like a SHA1 based on the length. The json request is probably hashed along with some server side secret so I don't think it's feasible to modify that request.

@TheEnhas there is at least one value which can currently bypass the server_ads thing and avoid ads. But I think with a great certainty if it were added to uBlockOrigin it would get fixed within 24 hours.

[TheEnhas]

@TheEnhas there is at least one value which can currently bypass the server_ads thing and avoid ads. But I think with a great certainty if it were added to uBlockOrigin it would get fixed within 24 hours.

If it's by design on some platforms to save bandwidth or that already have other ads then I'm not sure if they'd "fix it", but I get the feeling it's more likely that they've just not updated those apps yet to work with the new system and they'll get the ads like everyone else once they do.

[stoically]

@pixeltris Although you're probably right and the token needs to match the sig, I wanted to at least give it a try - but it seems that the overwritten fetch in twitch-videoad.js never gets called with the request to usher.ttvnw.net. Could the reason for that be that the call is maybe made inside a ServiceWorker? Is there a way for uBO to inject something into ServiceWorkers?

FWIW, I've also tried to redirect the request using the WebExtensions onBeforeRequest API, but that leads to a Network Error in the Twitch player and it also doesn't show the request at all in the Network log. The only reason I can think of that causes this, is that the fetch request.redirect for that request is set to manual and fails for non-200 status.

[pixeltris]

@stoically https://gist.github.com/pixeltris/e659867516ed0be149bf5e9e22d7d6e0 something like this should work (I have been using it for tests, modifying the response on a locally hosted server so I don't have to deal with the pain of javascript).

[stoically]

That works, thanks. Though rewriting the token params results in the server responding with a 403 error, so you're probably right about the sig.

[pijcab]

Hi guys,
Is there a documentation on how to add the sort of snippet @pixeltris is giving us ?
Im a complete noob when it comes to modifying Chrome/Firefox extensions...
Edit : This Twitch ad thing is getting out of hand, if I'm a Twitch Prime user I dont want to watch ads on top of that (ads which are usually for services like Amazon Video that are meant for Prime subs...)

[stoically]

@pijcab Select advanced in the uBO dashboard, click the appearing cog, replace unset for userResourcesLocation with the raw url to the gist (needs a click on raw on the linked gist). This will break all rules which depend on the default resources that are not included in the gist.

[Jap2-0]

Wouldn't sig= be generated client-side, or no?

[stoically]

Yeah, it looks like the sig is generated in the WASM module. Call stack for the request (version 2.9.2):

[pijcab]

@stoically thank you very much, didn't know that userResourcesLocation was used for that.

Also this might help some people : I found a workaround by using a VPN and connecting to the countries outside of the SureStream available geos (Such as Romania or Hungary).
Didn't get any ads for an hour or two as of now

[TheEnhas]

Some people reportedly had success by changing their browser's user agent to Android while on Twitch. I tried it and ads still got through.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/523c2fa60547046bbd15122e5aa70489f5d8afde/twitch_hls_test.js this is a rough copy/paste of @instance01's https://github.com/instance01/Twitch-HLS-AdBlock for uBlock Origin. Last time I tested his extension I was getting reasonable load times (~5 seconds). Currently everything seems to take like 30+ seconds to load. I haven't looked too much into it yet and had to gut out some code which was causing spam requests.

I tried making a "loading screen" .ts file but failed miserably. If anyone knows how to make a valid .ts file Twitch can load that would be useful. There is a base64 string where the .ts file should go at the bottom of that file. Otherwise the screen will be black for ~30 seconds while ads are displayed.

[TheEnhas]

Is there a value set anywhere that tells Twitch that you've seen an ad recently so as to not play another one? Because (one big thing I hate about the new pre-roll system) you end up behind the stream once the ad finishes and have to pause/play or reset the player to get back to the right latency, and you don't get another ad right away if you do that. Also, even if I reload or change streams after getting an ad I don't tend to get another for some time. This doesn't make a lot of sense because some people report getting an ad every time they load a stream no matter what, but in my case maybe it's setting some value somewhere that expires after like a half hour or whatever amount of stream loads (whichever comes first).

[stoically]

@TheEnhas The ads are at least geo-aware, if not targeted. So it's probably some server-side logic that decides whether to include ads in your m3u8 playlists or not (hence the name SSAI). Something like: Some IP-ranges (or targeted users if you're logged in) are limited to a Pre-roll Ad every 15minutes, others are not - based on the options chosen by advertisers.

Related or maybe even superseding issue: #1905

[CaelThunderwing]

likely Ad blocking may break for me on April 2nd (older Amazon Prime/Twitch Prime and i guess? i was grandfathered into having ads disabled as i have even uBlock disabled and still get squat but i disabled since i made affiliate as to not even tempt to breaking contract,) for when i haveto let it expire briefly to renew w/ a gift of prime... Anyway..

maybe look into how Kodi & the twitch plugin does this because no ads play before or during a stream. I have a media center box in my gaming room setup with Kodi 18.1 and latest twitch plugin outside a few edge cases that streams either crash the box or wont load(a kodi fault/plugin fault) i dont get ads whatso ever, with or without logging into my twitch account.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/0c9f3d988d9cd0ef5f011ad8057a3d50ae23a38c/twitch_hls_test.js updated version of the adapted Twitch-HLS-AdBlock for uBlock Origin. It should load the stream in 5-10 seconds now rather than 30. Not sure about midroll ads though. Actually maybe 10-30 seconds? Pretty bad still.

[TheEnhas]

Is there any way to get the time length of the ad at the start, immediately skip to the end of that length and mark it as being "viewed" by Twitch so that you end up with very little (if any) downtime?

[pixeltris]

@TheEnhas that's essentially what Twitch-HLS-AdBlock / the thing I linked are doing. There is one m3u8 per stream quality, as it polls the m3u8 file, normal stream segments will start showing up. As soon as they are available they are displayed (and all the ad segments removed). Unfortunately there is still quite a delay before the normal stream is available. Maybe you could spam the m3u8 some more but I doubt it would help.

[stoically]

Just realized that the token and sig in the /<channelname>.m3u8 request are the exact return values from the initial call to /access_token, so yeah, @pixeltris's assumption of server-side hashing is correct.

@pixeltris Regarding Pre-rolls, the m3u8 requests for me look like this when loading the stream:

• First Ad:
  • Request time: "Wed, 20 Mar 2019 14:48:47 GMT"
  • m3u8 value: #EXT-X-PROGRAM-DATE-TIME:2019-03-20T14:48:35.822Z
• First non-Ad (after SCTE35-IN):
  • Request time: "Wed, 20 Mar 2019 14:48:53 GMT"
  • m3u8 value: #EXT-X-PROGRAM-DATE-TIME:2019-03-20T14:48:49.822Z

So the Ad plays 14seconds but the first actual stream .ts is served after 6seconds. However, I think that the m3u8 also includes .ts segments which only will be made available in the future and are not directly available when included in the m3u8 (30-60seconds m3u8 segments vs 2-5seconds stream-delay). I guess the twitch player uses the #EXT-X-PROGRAM-DATE-TIME and that could somehow be rewritten accordingly for the first available non-Ad. Here's the raw HAR for the Pre-roll.

Stripping segments completely works for Pre-rolls, since the player didn't start yet and will start correctly with the first non-Ad segment - but it will break for Mid-rolls because the player already started and suddenly doesn't get segments anymore. Also it seems that Mid-rolls don't serve non-Ad segments early. (More information about my Mid-roll findings including attempts to create a placeholder .ts with ffmpeg are here)

Would be interesting to see other HAR's for Pre-rolls and Mid-rolls (here's how to obtain one: Firefox / Chrome).

[pixeltris]

I'm not sure you can completely strip the m3u8 of all segment files as otherwise the twitch player will start spamming the server hard until it finds something to play. That's why I inserted a dummy .ts entry when there are only ads (it would be ideal if this would be an actual .ts file which displays a message about ads being blocked). Does the current solution I linked break when midrolls are played? I haven't had a chance to see a stream that has played midrolls. Is there a streamer who plays them often?

[pijcab]

@pixeltris try Shroud

[stoically]

@pixeltris

I'm not sure you can completely strip the m3u8 of all segment files as otherwise the twitch player will start spamming the server hard until it finds something to play.

It seems as long as the m3u8 can get parsed (request result not empty), it will not result in the "Invalid variant" error and spam, but rather poll normally.

it would be ideal if this would be an actual .ts file which displays a message about ads being blocked

Did you try the linked ffmpeg command?

Does the current solution I linked break when midrolls are played? I haven't had a chance to see a stream that has played midrolls. Is there a streamer who plays them often?

Hadn't a chance to test with Mid-rolls yet. I know that /twitchpresents plays them in breaks and they will go live again tomorrow (3/21 10AM PT). But I'm pretty sure it'll break, since the correct #EXT-X-PROGRAM-DATE-TIME's are missing and the player can't play the (not existing) .ts segment.

[PhiZero]

@pixeltris

I tried your solution with a stream that does frequent mid-rolls, and it broke the feed. After switching tabs it restarted again when I switched back, but I think that's unrelated. It works fine for pre-rolls however.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/ddffe9cfabcf881a028a4a813817ade96f6bad17/twitch_hls_test.js this version should display a message in the top left corner "uBlock Origin is waiting for ads to finish..." and the stream should also now continue after midroll ads finish (though the video feed will freeze until the next stream segment is available, I could possibly modify it to go to black or something?).

I might strip out some of the changes I made to support displaying custom .ts files as it really isn't necessary. But besides from that I think it should be good enough?

@stoically Twitch seems to be very finicky about what it can load, I couldn't get anything from ffmpeg or vlc's convert tool to work.

EDIT: https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/58cf8f1e503f3b5323daf6c12caf986decc86e5e/twitch_hls_test.js this one should be a tiny bit smoother when midrolls play. Live segments take a while to show up during midrolls which is unfortunate. Also I think there is currently some issue when midrolls finish where it skips a couple of segments (possibly due to the sequence modifications). I have noticed that during some hours of the day simply refreshing can result in no ads so it might be possible to spam the access_token request until the result is ad-free (this would require a poll to access_token, followed by the main m3u8, then finally the stream m3u8 to see if ads there or not).

[stoically]

@pixeltris Works fine for Pre-roll ads - have yet to test with Mid-rolls, I'll let you know. Though it might lead to unexpected behavior when switching streams, since self.seq and the other self assigned variables keep their values (related: instance01/Twitch-HLS-AdBlock#12).

To generate playable placeholder .ts segments, it seems necessary to include a valid "timed metadata" (timed_id3) stream. Here's some Pre-roll debug output from ffprobe for 8 initial ad segments and 2 following live segments - including their corresponding m3u8 entries and the raw timed_id3 header.

[pixeltris]

I've found streams under the "Always On" directory are a good resource for midroll ads (they show ads every 10 minutes or so). Is there a suitable place to reset self values?

[Mrcubix]

It seems that the player is starting to resize again, like it would to show side & bottom banner ads, except that there are no ads since they have been blocked.

Versions

• Browser/version: Chrome 124.0.6367.208
• uBlock Origin version: 1.60.0

Screenshots

When an ad is supposed to appear:

Expected: