High level tracking of Docs Views with HEAP

[noklam]

Description

While I was trying to find out where is our Architecture diagram, I found that our FAQ pages view has gone down signficantly.

I try to make sense with the data, but I couldn't make HEAP works. I see consistent pattern across configuration, FAQ, data_catalog etc have a view drop in April while we have highest number of unique user of our docs at the same time.

Context

I would like to have at least high level tracking of important themes, particularly to help me understand what are the drivers of the view change.

This is hard to do because HEAP is not very flexible (or maybe I don't know enough) and the fact we change our domain from readtodoc -> kedro.org which makes it hard to match the statistics

• Notebook
• Catalog
• Configuration

This could be useful to understand if we are improving our docs. In addition, it may be useful to track specific features adoption.

Possible Implementation

Possible Alternatives

[tynandebold]

Can you please share the link to that chart? I can help look into this.

[tynandebold]

After digging into this I don't think the issue is with the domain changing. Rather, it's the fact that FAQ doesn't exist as a findable top-level item in the navigation nor on some of the pages themselves.

For example, take this page of the docs: get_started/install.html. Before we saw the drop in traffic to the FAQ pages, a user could find a link to the FAQs with relative ease in a couple of different places:

After some recent documentation changes and updates to the navigation and linking, that's no longer the case:

Perhaps we should reconsider removing FAQs from the navigation if that's a page we want to drive traffic to and think more broadly about how important the navigation seems to be for driving traffic to the pages listed there.

cc @stichbury @astrojuanlu

[astrojuanlu]

Quick comment, slimming down the FAQs was a deliberate choice #1985

If the data catalog page showed a similar downward pattern though, that's something we should look into

[tynandebold]

Further evidence

In March 2023, 21% of people navigated to the FAQ page from get_started/install.html.

---

In May, only 6% did the same, a 71% decrease.

[tynandebold]

Quick comment, slimming down the FAQs was a deliberate choice #1985

If the data catalog page showed a similar downward pattern though, that's something we should look into

Good to know!

[noklam]

Quick comment, slimming down the FAQs was a deliberate choice #1985

If the data catalog page showed a similar downward pattern though, that's something we should look into

I think I understand what @astrojuanlu is saying here. For example, FAQ page have a question "How do I specify parameters at run time?", if you google it, google will directly you to the pointer https://docs.kedro.org/en/stable/configuration/parameters.html#how-to-specify-parameters-at-runtime, which means the traffic won't go to FAQ but the corresponding page.

However, I don't think this is what happening. The FAQ is now harder to find because it was at the Top Level, but now it's hidden in "Resource". If you search in "FAQ" in Docs, you will not find it because it is "Frequently Asked Question" and RTD search doesn't understand semantic. Google Kedro FAQ will find it because the URL contains "faq" I guess?

Using only 1 filter with Path contains "catalog"

Anyway, FAQ is just an example that I found, and I think we can move FAQ back up to the Top level, and maybe alias it as FAQ so it can be searched.

[stichbury]

Perhaps we should reconsider removing FAQs from the navigation if that's a page we want to drive traffic to and think more broadly about how important the navigation seems to be for driving traffic to the pages listed there.

Driving traffic to the FAQ page is only useful if it's of help to the reader in their mode of learning.

The reason we slimmed down the page was because it wasn't useful -- it had marketing content about Kedro (still useful, but not when someone is learning how to use it (docs) rather than learning about Kedro (website)). I'd be interested to see the dwell time on the page from the traffic prior to the changes and compare to after, since I suspect a lot of people bounced off it swiftly when it wasn't targetted.

It would also be useful @tynandebold to see if organic traffic to the website page rose when we did the FAQs move, although the search console stats we saw didn't suggest we were getting that much.

Let's think about our FAQs in terms of the ways people encounter and use the page:

1. They have a specific technical question "how can I specify a parameter to kedro run?". They'll type that into search and navigate via links, whether on google or internal site search. Prior to our changes, when FAQs was a top-level page, they would not have hit the FAQs page from search often because it wasn't answering a technical query (it was a scattered collection of stuff we couldn't quite work out where it went).

Now they have a better chance of it being hit by a technical search. It's still low as that page is limited, but there are distinct queries on the page (alongside the rest of the site content, which is gradually changing in terms of SEO opportunity to answer organic technical search too).

2. When people aren't using search to find queries, they may encounter and use the page through browsing, either following a link elsewhere or browsing for it, or seeing it on nav. You can't really target content for this use case, but you can make it clear what the page is for, so they can either get something from it there and then, or use it in future. A long page of text covering random questions isn't doing that. We've now targetted the page towards technical search.

Traffic for use case 2 is what we have reduced, and TBH that's OK.

The page wasn't particularly useful for them except as a "Oh look, this page is here and tells me about data layers" as it was. We can and should make it more discoverable if we want more traffic, but we're not here to move people about the site but to serve their needs as readers. That page will be hit by queries and move people to useful content to their needs. It's just hit less now because it's targetted properly and isn't encyclopaedic.

TL;DR

@tynandebold's data shows that the drop is because of the change in location & linking. So...

• Yes, perhaps we do need to make the page easier to find for use case 2 (random discovery) and retitle it as "FAQs"
• We don't need to revise the content back to what it was. It'd be great to add more FAQ links though.

To that point...we still need to work out how to provide more FAQ content/knowledgebase content but I'm fairly sure a page in the docs is not the way forward for this, given we have Slack archives, GenAI opportunities and not enough people to wizard and maintain documented FAQs.

[stichbury]

I made some revisions to the FAQs and other pages affected by FAQs (plus some housekeeping for google indexing of slack archive) in #2909 and I'm closing this issue for now. Let's keep an 👁️ on the data for the FAQs page though.