Page titles should not be updated by versions that are errors

[Mr0grog]

The Version model has an after_create callback that updates its page’s title field:

web-monitoring-db/app/models/version.rb

Lines 107 to 112 in 213dc89

	def sync_page_title
	if title.present?
	most_recent_capture_time = page.latest.capture_time
	page.update(title: title) if most_recent_capture_time.nil? || most_recent_capture_time <= capture_time
	end
	end

However, it shouldn’t update the page title if the version in question represents an error — the page model represents the conceptual page (not the URL) that we are tracking, and it’s unhelpful to rename the page to represent a temporary error, or worse, a “not found” message when the page is permanently removed. It makes it hard to tell what page we’re talking about.

For example, this page: https://monitoring.envirodatagov.org/page/9c85fc4f-c0ef-4327-ac4e-df2b6314917f/ba9b8e97-668d-4c92-bf95-d2ee5ffa46f2..0f4b06b2-6473-4ff6-a5e6-aada11f65cdd
…shows the title “Page Not Found | USDA” when it should show “USDA | Office of the Chief Economist | Climate Change | Effects | Climate Change and Agriculture Report.”

To do this, we should probably add a method on Version named is_error? or similar rather than directly checking the status code so that we can account for things like #468 in the future.

[Mr0grog]

is_error? might be something like:

def is_error?(strict: false)
    return true if status && status >= 400
    return false if strict

    # Use some basic heuristics to determine if a page with an OK
    # status code is actually an error. There's lots of room for
    # improvement here.
    case_title = title.downcase
    (
        case_title.include?('error 404')
        || (case_title.match?(/\b404\b/) && title.include?('not found'))
        || case_title.match?(/(page|file)( was)? not found/)
        || case_title.split(/\s*(?:-+|\|)\s*/).any? {|p| p == 'restricted access'}
    )
end