Proposal: Fallback for the transcript tag

[ryan-lp]

This is both a proposal and an open letter to podcast app developers to please consider supporting a modest fallback to the <transcript> tag that will not deny listeners access to transcripts, even when a hosting platform hasn't supported them yet.

The proposal is simple. When a podcast app loads a transcript:

1. First look for a transcript link in the <podcast:transcript> tag.
2. If one is not found, then look for a transcript link in the description, pattern matching against any URL that ends with a .srt extension. (For the uninformative JSON extensions, see Proposal: replace generic `*.json` extension with specific extensions for each file type #452)

Lagging adoption by hosting platforms and podcaster apathy should not be reasons to deny a listeners access to transcripts in the present, and so we need a pragmatic way to balance the immediate needs of certain listeners with the desire to roll out the namespace standard universally in the long term. Although the march for universal adoption of the namespace should continue on, this proposal is about ensuring comprehension and accessibility for all listeners in the present.

---

Background

Over the past few years, I have been approaching podcasters to ask if they would consider publishing transcripts via this new tag, and the response has almost universally been along these lines:

• Sounds interesting!
• What's a tag, then, and where do I find it?
• My host doesn't have that.
• No I don't want to switch hosts.

"I see. What if you just add a link to the transcript in your episode description? Would be willing to publish transcripts then?"

• Yes, I could do that.

I am hearing definite interest in transcripts and a willingness from podcasters to add a link to transcripts in at least SOME way, whatever they can manage within their current setup, but as yet, there are no apps that actually support this, and so even if they did add a link in this way, the podcast apps would not yet actually take advantage of it and show those transcripts to the listener. In theory, it is a small step from here to there for any developer who wants pick up these links.

I am not going to give up the advocacy work of trying to get more hosting companies to support the namespace, however in the meantime, my job would be a lot easier if my typical dialogue (above) could finish with "Great! If you do that, here are a few podcast apps you can recommend to your listeners that will pick up and display these transcripts."

[nathangathright]

IMO, putting links to SRT files in the show notes feels hacky and very hostile to the listener. But if podcasters are willing to link to a transcript somewhere, I’d love to see transcript providers support some sort of markup on their listener-facing web pages that podcast apps could crawl. For example, if a podcaster links to a public Descript project, those webpages already include:

<meta property="descript:subtitles" content="[link to WebVTT file]">
<meta property="descript:transcript" content="[link to JSON file]">

I’ve built a feature for Steno.fm that can recognize those links, crawl those pages, and display those transcripts alongside the audio. The major downside to this approach is that apps have to develop heuristics for which links to crawl, but I think asking podcasters to include something like a rel=transcript attribute on links is too onerous.

Maybe there should be a <meta property="podcast:transcript"> spec. Or alternatively, more awareness and adoption of existing Schema.org structured data like:

{
   "@context": "https://schema.org/",
   "@type": "PodcastEpisode",
   "url": "http://www.relay.fm/inquisitive/27",
   "name": "#27: Behind the App #1: History",
   "audio": {
     "@type": "AudioObject",
     "contentUrl": "http://www.relay.fm/inquisitive/Inquisitive_027.mp3",
      "caption": {
        "@type": "MediaObject",
        "encodingFormat": "application/x-subrip",
        "contentUrl": "http://www.relay.fm/inquisitive/Inquisitive_027.srt"
      }
   }
}

[ryan-lp]

Thanks for the suggestions and further ideas. I think links in the listener facing page are also feasible.

As a bit of a reality check, the deeper the link is, the less likely it is to be crawled and hence the less impact it will have on the listeners who need those transcripts, and the more metadata or structure we need to place around the link, the less likely it is that the podcaster will be bothered to do it.

• The average podcaster on a platform without native support for transcripts is likely not going to go out of their way to support transcripts by adding complicated metadata structures manually.
• And the average podcast app is likely not going to search 3 levels deep for a transcript link.

But the transcript advocate's job will be more successful if they could just say to the podcast creator: "All you need to do for the transcript to start showing up to your listeners is to place the link anywhere in your episode description, or you could even add it anywhere in the listener facing page for the episode. And if you later choose to switch to a hosting platform that has native support for transcripts, you'll also gain the option to share that transcript link directly with the podcast apps without displaying it in your description." (Thus providing incentives to consider switching hosts without outright requiring it from the start. We have to allow a gradual transition here.)

That approach could be considered the lowest common denominator, because it is easiest for both podcasters to implement and for crawlers to implement. I'm not saying other options are invalid by any means, just that fewer apps will support it the more sophisticated this gets, and fewer podcasters stuck in this predicament will also implement it the more sophisticated this gets.

[ryan-lp]

P.S. The idea of picking up the transcript "wherever" the podcaster is able to manage putting it (if they're on a host that does not support the <podcast:transcript> tag) is similar to the idea of podcast feed verification where in practice we are going to have to pick up the verification token anywhere in the feed, "wherever" the podcaster is able to manage putting it (if they're on a host that does not support the <podcast:txt> tag). It doesn't need to be a standard, it can be considered a sort of meta standard or quasi standard, but it just reflects the reality of needing a gradual path to get from where we are today on adoption to more widespread namespace adoption in the future.

[nathangathright]

I haven’t released this feature yet, but Steno.fm will be able to serve as the host of the transcript file and the link podcasters include in their show notes. Podcasters will be able to upload their own transcripts and get a link to include in their show notes. Once they’ve verified they’re the podcaster by publishing that link in the show notes, the transcript will appear for all visitors.

Ideally, other podcast apps could recognize my steno.fm/transcript links, crawl them, and retrieve the transcript file URL. So I just need to support one of the ways to do that.

[jamescridland]

By offering a shim alternative to "my podcast host can't be bothered to let me link to my transcripts properly", are we not perpetuating the issue that some podcast hosts can't be bothered to let podcasters link to their transcripts properly?

• Sounds interesting!
• What's a tag, then, and where do I find it?
• My host doesn't have that.
• No I don't want to switch hosts.

... then send your host's support team this email, which contains:
a) the specification
b) a suggested simple UX for the specification
c) more in-depth, suggested transcript providers including rev share for the podcast host
d) the legal position of deliberately denying the ability to link to transcripts
e) a list of podcast hosts who support the transcript tag

I'm nervous about producing shims to cover up for podcast hosting company ineptitude. Surely the right thing to do is to pressure podcast hosts to implement at least the simplest version of this tag (which is to paste a URL into a box).

[ryan-lp]

@jamescridland let's assume for a moment that your approach to this is the way to go here. That is, even though it would be possible for apps to support transcripts on all legacy platforms in a matter of weeks via the proposed method, you say that "this group" should actually resist this idea, and effectively withhold listeners' access to transcripts for certain podcasts. You believe that ensuring the podcast namespace is the ONLY way that transcripts are done, at the exclusion of other ways (i.e. us-vs-them), is actually more important than the listener's right to have access to podcast transcripts . OK, so let's go with that assumption for the moment.

Now, pick 3 random podcasts from 3 different countries hosted on popular and free legacy platforms from those respective countries, where I am also waiting in the wings ready to donate transcripts to each of them once the apps can actually crawl them. My question for you is, how many more years (beyond the 3 years I've already been at it) before the listeners are served, if we follow your uncompromising approach? And do you truly think their wait will be worth it if it means that the podcast namespace gets to be the only standard?

Do you truly think that having more than one way to crawl transcripts will be detrimental? Was it detrimental for chapters that Podcast Addict supports your preferred approach and also Spotify's preferred approaches?

Is the mission to have your preferred way be the only way? Or is the mission to actually deliver transcripts, chapters, etc into the hands of listeners in whatever ways we can get the apps to support?

[jamescridland]

I personally don't feel that this is a podcast namespace proposal that this group should support, as I've made very clear. By adding a shim like this, it's my belief that this will delay getting the podcast transcript feature in the namespace fully adopted by the whole podcast ecosystem. I'm entitled to a personal view, and nothing you've said so far has changed my mind.

You're suggesting that I don't care about the listener. That's wrong (and not a little insulting): I want to be in a situation where podcast hosting companies support the ridiculously easy podcast:transcript tag. I don't want to give podcast hosting companies an excuse not to support transcripts, which is explicitly what your suggestion offers them - an easy way out that doesn't deliver the right thing for the listener, which is full support for the transcript feature.

Was it detrimental for chapters that Podcast Addict supports your preferred approach and also Spotify's preferred approaches?

I'm a bit confused about this point - but perhaps it's worthwhile highlighting, again, that this is my personal view. I'm not speaking on behalf of the new podcast namespace - just speaking on behalf of... me.

Since you brought it up - my personal view about chapters is to use ID3 tags, which are widely supported. I don't actually think the chapters JSON file does anything better than the ID3 standard: and specifically does a number of things worse (including support for DAI). I think the new podcast namespace approach to supporting chapters is a mistake. OK?

[ryan-lp]

I'm not suggesting that you don't care about the listeners, only that given the choice between:

1. making the listeners who depend on transcripts wait 6 years, while you convince hosts in all of these different countries to implement your preferred standard, and
2. giving them an option now so they don't have to wait 6 years

you would still choose scenario 1, because you would resist scenario 2 at all costs, which sadly includes the cost of the listener having to wait 6 years. So while you care about listeners' right to transcripts, there's apparently something that you care about even more than listeners' right to transcripts, and that is to resist scenario 2 at all costs. You would not compromise on that for anything because, as you said, you are fundamental about it.

The fact is that listeners who rely on transcripts don't actually care about which approach is used. What they care about is to be able to actually comprehend the podcast with the aid of transcripts. Not in 6 years, but as soon as possible. As someone with mobility issues who struggles with stairs (although not being wheelchair bound), I also often have to deal with the problem of getting into a building, even my own apartment building. There is a way I can get in by walking a long way around the building, and through a corridor, although that is beyond the walking distance I can cope with. Now when I spoke to the building manager, did he give me some resistance and say that I should not be able to easily get access to the building until they save up enough money to do some construction work to build a ramp for this side of the building as well as the back way? Well no, because I proposed him an immediate alternative: give me permission to access the building through the car park entrance. And although it was not a "proper" solution, I was very grateful that I was then able to enter and exit my own apartment building without causing more injuries to my hips and knees. He showed that he cared more about me than about having the perfect solution, because he factored the timescale into it and how the waiting period would actually cause more harm.

Now where does this 6 years come from? Well your response didn't actually address the time frame challenge I put to you. So, I am using my own estimate: Your approach will take 6 years to deliver transcripts into the hands of the listeners of these 3 podcasts from 3 random countries. After 3 years in the English markets we are getting some adoption, although it is still a relatively small amount of adoption. But In many other countries this process of adoption has barely even begun if at all, and is therefore lagging 3 years behind.

Meanwhile, I think that all it takes for option 2 is for at least ONE app to crawl the transcript link wherever it finds it, and then those waiting listeners would have an option available to them that until now has been denied to them. I think that can be realistically be achieved in 1 month or less, not 6 years.

I will also make a prediction that due to the ease of at least one app implementing this compared to the difficulty of all hosts implementing the transcript tag that it's inevitable that both of these approaches are going to coexist until the need for an alternative evaporates. As someone who is only happy with the perfect option, you won't actually be able to prevent people from publishing transcripts in different ways from your preferred way if ultimately people need transcripts however they can manage to get them, and you won't be able to prevent apps from crawling transcripts wherever they appear, if it means we can deliver more transcripts into the hands of listeners. This is the only metric that means anything.

[jamescridland]

I can only repeat what I've said - that a shim/kludge that allows podcast hosts to avoid properly implementing something is not in the best interest of the podcast namespace, and not, ultimately, in the best interest of listeners since it slows down adoption of standards.

I am not, however, anyone other than "another bloke with a point of view". If you're able to carry others with you, whether here or elsewhere, then go for it.

I don't intend to repeat myself further.

[ryan-lp]

You've certainly made your position clear, but you've also not addressed the time frame challenge to your position. Rather than repeat the same points over, one way you could have avoided repeating yourself would be to actually comment on those challenges that you have until now side stepped. But that's OK. You're just an ordinary bloke with a point of view. It's not as though you have a platform where you have the privilege to influence future directions of the industry, such as to publicise this open letter to app developers, for example.