-
-
Notifications
You must be signed in to change notification settings - Fork 803
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
This is a major rework of the API with a lot of breaking changes. Compared the the old API v0, API v1: - is generated using a Jekyll Generator (see https://jekyllrb.com/docs/plugins/generators/), - is versioned using the api/v1 prefix (#2066). This will make it easier to implement non-backward-compatible changes in API, - feels more "Restful". The API v0 is still generated to give time to users to migrate to API v1. Changes in the "All products" endpoint: - Path has been changed from api/all.json to api/v1/products/ - Response has been changed from a simple array to a JSON document. This made it possible to add endoflife-level data, such as the number of products. - Array elements have been changed from a simple string to a full JSON document. This made it possible to expose new data, such as product category and tags (#2062). Changes in the "Product" endpoint: - Path has been changed from api/<product>.json to api/v1/products/<product>/. - Response has been changed from a simple array to a JSON document. This made it possible to expose product-level data, such as product category and tags (#2062). - Cycles data now always contain all the release cycles properties, even if they are null (example: discontinued, latest, latestReleaseDate, support...). Changes in the "Cycle" endpoint: - Path has been changed from api/<product>/<cycle>.json to api/v1/products/<product>/cycles/<cycle>/. - Cycles data now always contain all the release cycles properties, even if they are null (example: discontinued, latest, latestReleaseDate, support...). - A special /api/v1/products/<product>/cycles/latest/ cycle, containing the same data as the latest cycle, has been added (#2078). New endpoints : - /api/v1/categories/ - list categories used on endoflife.date - /api/v1/categories/<category> - list products having the given category - /api/v1/tags/ - list tags used on endoflife.date - /api/v1/tags/<tag> - list products having the given tag Note that we thought of disabling API generation in development (using JEKYLL_ENV like the Jekyll Feed plugin - see https://github.com/jekyll/jekyll-feed/blob/master/lib/jekyll-feed/generator.rb#L145), but it was finally reverted. It does not work well with Netlify preview, and generate production URL (i.e. https://endoflife.date URLs) in development which makes it difficult to use.
- Loading branch information
1 parent
c5647d4
commit 703c925
Showing
5 changed files
with
341 additions
and
74 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
{{ page.data | jsonify }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,89 +1,226 @@ | ||
#!/usr/bin/env ruby | ||
|
||
# This script creates API files for version 1 of the endoflife.date API. | ||
# | ||
# There are three kind of generated files : | ||
# - all.json: contains the list of all the product names. | ||
# - <product>.json: contains a given product data ()including releases data). | ||
# - <product>/<release>.json: contains a given product release data. | ||
# There are two kind of generated files : | ||
# - all.json: contains endoflife metadata, such as the list of all the products and the product count. | ||
# - <product>.json: contains a given product data (including releases data). | ||
|
||
require 'fileutils' | ||
require 'json' | ||
require 'yaml' | ||
require 'date' | ||
require 'jekyll' | ||
|
||
module ApiV1 | ||
|
||
# This API path | ||
DIR = 'api/v1' | ||
STRIP_HTML_BLOCKS = Regexp.union( | ||
/<script.*?<\/script>/m, | ||
/<!--.*?-->/m, | ||
/<style.*?<\/style>/m | ||
) | ||
STRIP_HTML_TAGS = /<.*?>/m | ||
|
||
# Returns the path of a file inside the API namespace. | ||
def self.file(name, *args) | ||
File.join(DIR, name, *args) | ||
# Remove HTML from a string (such as an LTS label). | ||
# This is the equivalent of Liquid::StandardFilters.strip_html, which cannot be used | ||
# unfortunately. | ||
def self.strip_html(input) | ||
empty = ''.freeze | ||
result = input.to_s.gsub(STRIP_HTML_BLOCKS, empty) | ||
result.gsub!(STRIP_HTML_TAGS, empty) | ||
result | ||
end | ||
|
||
# Holds information about a product. | ||
Product = Class.new do | ||
attr_accessor :data | ||
|
||
# Initializes the product with the given product's markdown file. | ||
# The markdown file is expected to contain a YAML front-matter with the appropriate properties. | ||
# | ||
# Copying the data makes it easier to process it. | ||
def initialize(data) | ||
@data = Hash.new | ||
# The product name is derived from the product's permalink (ex. /debian => debian). | ||
@data["name"] = data['permalink'][1..data['permalink'].length] | ||
@data["title"] = data['title'] | ||
@data["category"] = data['category'] | ||
@data["iconSlug"] = data['iconSlug'] | ||
@data["permalink"] = data['permalink'] | ||
@data["versionCommand"] = data['versionCommand'] | ||
@data["auto"] = data.has_key? 'auto' | ||
@data["releasePolicyLink"] = data['releasePolicyLink'] | ||
@data["releases"] = data['releases'].map do |release| | ||
release_data = Hash.new | ||
release_data["name"] = release['releaseCycle'] | ||
release_data["codename"] = release['codename'] | ||
release_data["releaseDate"] = release['releaseDate'] | ||
release_data["support"] = release['support'] | ||
release_data["eol"] = release['eol'] | ||
release_data["discontinued"] = release['discontinued'] | ||
release_data["lts"] = release['lts'] || false # lts is optional, make sure it always has a value | ||
release_data["latest"] = release['latest'] | ||
release_data["latestReleaseDate"] = release['latestReleaseDate'] | ||
release_data | ||
class ApiGenerator < Jekyll::Generator | ||
safe true | ||
priority :lowest | ||
|
||
TOPIC = "EndOfLife API v1:" | ||
|
||
def generate(site) | ||
@site = site | ||
Jekyll.logger.info TOPIC, "Generating API for products" | ||
|
||
add_index_page(site) | ||
product_pages = add_product_pages(site) | ||
add_all_products_page(site, product_pages) | ||
|
||
add_category_pages(site, product_pages) | ||
add_all_categories_page(site, product_pages) | ||
|
||
add_tag_pages(site, product_pages) | ||
add_all_tags_page(site, product_pages) | ||
end | ||
|
||
private | ||
|
||
def add_index_page(site) | ||
site_url = site.config['url'] | ||
site.pages << JsonPage.new(site, '/', 'index', { | ||
result: [ | ||
{ name: "products", uri: "#{site_url}/api/v1/products/" }, | ||
{ name: "categories", uri: "#{site_url}/api/v1/categories/" }, | ||
{ name: "tags", uri: "#{site_url}/api/v1/tags/" }, | ||
] | ||
}) | ||
end | ||
|
||
def add_product_pages(site) | ||
product_pages = [] | ||
|
||
site.pages.each do |page| | ||
if page.data['layout'] == 'product' | ||
product_pages << page | ||
site.pages << ProductJsonPage.new(site, page) | ||
|
||
site.pages << ProductCycleJsonPage.new(site, page, page.data['releases'][0], 'latest') | ||
page.data['releases'].each do |cycle| | ||
site.pages << ProductCycleJsonPage.new(site, page, cycle) | ||
end | ||
end | ||
end | ||
|
||
return product_pages | ||
end | ||
|
||
def add_all_products_page(site, products) | ||
site.pages << ProductsJsonPage.new(site, '/products/', 'index', products) | ||
end | ||
|
||
def add_category_pages(site, products) | ||
pages_by_category = {} | ||
|
||
products.each do |product| | ||
category = product.data['category'] || 'unknown' | ||
add_to_map(pages_by_category, category, product) | ||
end | ||
|
||
pages_by_category.each do |category, products| | ||
site.pages << ProductsJsonPage.new(site, "/categories/#{category}", 'index', products) | ||
end | ||
end | ||
|
||
def name | ||
data["name"] | ||
def add_all_categories_page(site, products) | ||
site.pages << JsonPage.new(site, '/categories/', 'index', { | ||
result: products | ||
.map { |product| product.data['category'] } | ||
.uniq | ||
.sort | ||
.map { |category| { name: category, uri: "#{site.config['url']}/api/v1/categories/#{category}/" } } | ||
}) | ||
end | ||
|
||
def add_tag_pages(site, products) | ||
products_by_tag = {} | ||
|
||
products.each do |product| | ||
product.data['tags'].each { |tag| add_to_map(products_by_tag, tag, product) } | ||
end | ||
|
||
products_by_tag.each do |tag, products| | ||
site.pages << ProductsJsonPage.new(site, "/tags/#{tag}", 'index', products) | ||
end | ||
end | ||
|
||
def add_all_tags_page(site, products) | ||
site.pages << JsonPage.new(site, '/tags/', 'index', { | ||
result: products | ||
.flat_map { |product| product.data['tags'] } | ||
.uniq | ||
.sort | ||
.map { |tag| { name: tag, uri: "#{site.config['url']}/api/v1/tags/#{tag}/" } } | ||
}) | ||
end | ||
|
||
def add_to_map(map, key, page) | ||
if map.has_key? key | ||
map[key] << page | ||
else | ||
map[key] = [page] | ||
end | ||
end | ||
end | ||
end | ||
|
||
product_names = [] | ||
FileUtils.mkdir_p(ApiV1::file('.')) | ||
|
||
Dir['products/*.md'].each do |file| | ||
# Load and prepare data | ||
raw_data = YAML.safe_load(File.open(file), permitted_classes: [Date]) | ||
product = ApiV1::Product.new(raw_data) | ||
product_names.append(product.name) | ||
|
||
# Write /<product>.json | ||
product_file = ApiV1::file("#{product.name}.json") | ||
File.open(product_file, 'w') { |f| f.puts product.data.to_json } | ||
|
||
# Write all /<product>/<release>.json | ||
FileUtils.mkdir_p(ApiV1::file(product.name)) | ||
product.data["releases"].each do |release| | ||
# Any / characters in the name are replaced with - to avoid file errors. | ||
release_file = ApiV1::file(product.name, "#{release['name'].to_s.tr('/', '-')}.json") | ||
File.open(release_file, 'w') { |f| f.puts release.to_json } | ||
class JsonPage < Jekyll::Page | ||
def initialize(site, path, name, data) | ||
@site = site | ||
@base = site.source | ||
@dir = "api/v1#{path}" | ||
@name = "#{name}.json" | ||
@data = {} | ||
@data['layout'] = 'json' | ||
@data['data'] = data | ||
|
||
self.process(@name) | ||
end | ||
end | ||
|
||
class ProductsJsonPage < JsonPage | ||
def initialize(site, path, name, products) | ||
super(site, path, name, { | ||
total: products.size(), | ||
products: products.map { |product| { | ||
name: product.data['id'], | ||
title: product.data['title'], | ||
category: product.data['category'], | ||
tags: product.data['tags'], | ||
uri: "#{site.config['url']}/api/v1/products/#{product.data['id']}/", | ||
} } | ||
}) | ||
end | ||
end | ||
end | ||
|
||
# Write /all.json | ||
all_products_file = ApiV1::file('all.json') | ||
File.open(all_products_file, 'w') { |f| f.puts product_names.sort.to_json } | ||
class ProductJsonPage < JsonPage | ||
def initialize(site, product) | ||
id = product.data['id'] | ||
super(site, "/products/#{id}", 'index', { | ||
name: id, | ||
label: product.data['title'], | ||
category: product.data['category'], | ||
tags: product.data['tags'], | ||
links: { | ||
icon: product.data['iconUrl'], | ||
html: "#{site.config['url']}/#{id}", | ||
releasePolicyLink: product.data['releasePolicyLink'], | ||
}, | ||
versionCommand: product.data['versionCommand'], | ||
lastModified: product.data['lastModified'], | ||
auto: product.data.has_key?('auto'), | ||
cycles: [ | ||
product.data['releases'] | ||
.map { |cycle| { | ||
name: cycle['releaseCycle'], | ||
codename: cycle['codename'], | ||
label: ApiV1.strip_html(cycle['label']), | ||
date: cycle['releaseDate'], | ||
support: cycle['support'], | ||
lts: cycle['lts'], | ||
eol: cycle['eol'], | ||
discontinued: cycle['discontinued'], | ||
latest: { | ||
version: cycle['latest'], | ||
date: cycle['latestReleaseDate'], | ||
link: cycle['link'], | ||
} | ||
} } | ||
] | ||
}) | ||
end | ||
end | ||
|
||
class ProductCycleJsonPage < JsonPage | ||
def initialize(site, product, cycle, identifier = nil) | ||
name = identifier ? identifier : cycle['id'] | ||
|
||
super(site, "/products/#{product.data['id']}/cycles/#{name}", 'index', { | ||
name: cycle['releaseCycle'], | ||
codename: cycle['codename'], | ||
label: ApiV1.strip_html(cycle['label']), | ||
date: cycle['releaseDate'], | ||
support: cycle['support'], | ||
lts: cycle['lts'], | ||
eol: cycle['eol'], | ||
discontinued: cycle['discontinued'], | ||
latest: { | ||
version: cycle['latest'], | ||
date: cycle['latestReleaseDate'], | ||
link: cycle['link'], | ||
} | ||
}) | ||
end | ||
end | ||
end |
Oops, something went wrong.