Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce transition page while preserving meta tags for crawlers #60

Merged
merged 8 commits into from
May 13, 2020

Conversation

liangyuanruo
Copy link
Contributor

@liangyuanruo liangyuanruo commented May 4, 2020

Problem

Provide a transition page on redirect for browsers, whilst preserving meta tags for social media crawlers.

Closes #10

Solution

Detecting a crawler

Crawlers tend to have fairly unique user-agent (UA) strings, which are either terse or have the word "bot" embedded (see Telegram, Facebook and Google).

To discern between bots/crawlers and human users, we rely on the popular ua-parser-js library to parse the user-agent string and determine additional metadata such as

  • Operating system
  • Browser
  • Browser engine

which would be readily available for common browsers but not for crawler UAs. The server only serves the transition page if all three pieces of metadata can be determined with confidence. This method leans towards safety - the server should prefer immediately redirecting when operating under uncertainty, rather than risk serving a transition page to a crawler.

TODOs

  • CSS tweaks
  • Mobile view
  • Set a cookie to determine if a browser has been served a transition page recently e.g. last 48h.
  • Adjust design based on user tests (currently 3s automatic transition)

Tests

Also see metatags.io.

Features:

  • Transition page for non-crawler user agents

Improvements:

  • Server-side views moved into src which can take advantage of hot-reloading

Screenshots

Desktop

image

Deploy Notes

New dependencies:

  • ua-parser-js : User-agent string parsing library

New dev dependencies:

  • copyfiles : Used to move server-side EJS template files into /build after Typescript compilation

@liangyuanruo liangyuanruo changed the title Transition page while preserving meta tags Introduce transition page while preserving meta tags for crawlers May 4, 2020
@liangyuanruo liangyuanruo requested review from kylerwsm and yong-jie May 4, 2020 06:05
kylerwsm
kylerwsm previously approved these changes May 4, 2020
Copy link
Contributor

@kylerwsm kylerwsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Added a couple of comments.

src/server/views/transition-page.ejs Outdated Show resolved Hide resolved
src/server/index.ts Show resolved Hide resolved
@liangyuanruo liangyuanruo requested a review from kylerwsm May 13, 2020 17:23
@liangyuanruo liangyuanruo marked this pull request as ready for review May 13, 2020 17:23
@liangyuanruo liangyuanruo merged commit 2312e8b into develop May 13, 2020
@liangyuanruo liangyuanruo deleted the transition-page branch May 13, 2020 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Transition page
3 participants