This is a site generator project for Apache Camel. It generates static HTML and resources that are to be published.
Tools used to generate the website:
- Git a source code management tool used to fetch document sources from different github repositories.
- Node.js a JavaScript runtime used to build the website. You will need to use Node.js version 10.
- yarn a blazing fast dependency and package manager tool used to download and manage required libraries.
- (installed via yarn) Gulp a task automation tool. Used to build the Camel Antora UI theme.
- (installed via yarn) Hugo a static site generator. Simplified, it takes the
documentation from the
contentdirectory and applies templates fromlayoutsdirectory and together with any resources instaticdirectory generates output in thepublicdirectory. - (installed via yarn) Antora a documentation site generator. It uses Asciidoc documents from different sources in the Camel, Camel K and Camel Quarkus repositories where user manual and component reference documentation resides and renders them for inclusion in this website.
- (optional) Maven a build tool used to run the complete website generating process.
You can build the website locally using the tools Node.js and yarn.
If you can not use these tools on your local machine for some reason you can also build the website using Maven as described in section "Build with Maven".
For windows users, a beginning step to install yarn and nvm on your local system is through installing chocolatey.
An easy step to step guide to install chocolatey on your local system is as follows:
-
Open cmd/powershell and run it as administrator.
-
Install with cmd.exe
> @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin" -
Install with PowerShell.exe
> Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
Make sure that you have Node.js (herein "Node") installed.
$ node --version
If this command fails with an error, you do not have Node installed.
This project requires the Node LTS version 14 (e.g., v14.15.0).
Please make sure to have a suitable version of Node installed. You have several options to install Node on your machine.
- Install using the Node version manager nvm
- Install using Homebrew and Node formulae
- Install from official Node packages
An easy step to step guide to install nvm and install node v14 on your local system is as follows:
$ touch ~/.bash_profile
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.36.0/install.sh | bash
$ source ~/.nvm/nvm.sh
$ nvm install 14
Note - If you have different Node version other than Node LTS version 14 you can use following command to make Node LTS version 14 as default Node version.
$ nvm use 14
Note - The following steps need to be ran on cmd as administrator only.
An easy step to step guide to install nvm and install node v14 on your local system is as follows:
> choco install nvm
> nvm install 14
Note - If you have different Node version other than Node LTS version 14 you can use following command to make Node LTS version 14 as default Node version.
> nvm use 14
Now that you have Node 14 installed, you can proceed with checking the Yarn installation.
Follow the documentation on installing Yarn for your Operating system.
Note: For windows users, run on cmd as administrator and install yarn through chocolatey.
Clone the Apache Camel Website project using git:
$ git clone https://github.com/apache/camel-website.git
The command above clones the Apache Camel Website project. After that you can switch to the new project directory on your filesystem.
Some of the content for the website is derived from the data received from GitHub API and rate limits can cause build failures. For that reason it is necessary to set either the following environment variables:
HUGO_PARAMS_GitHubUsername=<GitHub username>HUGO_PARAMS_GitHubToken=<GitHub token>
Or the following environment variable:
GITHUB_TOKEN=<personal access token>
These values are used by Hugo when building or running in development mode (yarn preview:hugo) or building the
website (yarn build:hugo or yarn build-all) to access GitHub API with a higher rate limit.
NOTE: When you clone the camel-website repository the binaries present in the Yarn PnP cache or in the
.yarn/unplugged directories are built for x86-64 Linux, so if you're running on a different architecture or OS
run yarn workspaces foreach rebuild to rebuild the binaries.
We're using yarn workspaces to build both the theme and the website run build-all script, for example:
$ yarn build-all
That will build the Antora theme (from antora-ui-camel directory) and the website. Result of the build can be seen in the public directory.
The theme sources are located inside Project root directory/antora-ui-camel. So first switch to that directory:
$ cd antora-ui-camel
In that directory execute:
$ yarn build # to perform the ui theme build
You should see the Antora theme bundle generated in in antora-ui-camel/build/ui-bundle.zip.
In case yarn build raises error, run yarn format to format the code and re-run yarn build to build your bundle successfully.
The Camel Antora UI theme should not be a subject to change very frequently. So you might execute this once and never come back.
Building the website requires the built Antora Camel UI theme bundle from above. Please check that the theme bundle exists in antora-ui-camel/build/ui-bundle.zip.
To build the website go to the project root directory and run:
$ yarn build # to perform the build
In case yarn build throws the error: JavaScript heap out of memory, the issue can be resolved by increasing the memory used by node.js by setting NODE_OPTIONS environment variable to include --max_old_space_size, for example to increase the old space to 4GB do:
$ export NODE_OPTIONS="--max_old_space_size=4096"This should fetch doc sources for Camel and Camel K
and generate the website with Hugo. You should see the generated website in the public directory.
You can preview the Apache Camel website on your local machine once you have the generated website available in
the public directory.
Hugo can start a simple web server serving the generated site content so you can view it in your favorite browser.
Simply call
$ yarn preview
and you will be provided with a web server running the site on http://localhost:1313/
Point your favorite browser to http://localhost:1313/ and you will see the Apache Camel website.
Changes that are made to the content managed by Hugo (i.e. content, layouts, config.toml) are applied automatically and reloaded in the browser. To make changes to the content managed by Antora, a rebuild needs to be done. The same is true for the CSS changes in the antora-ui-camel. To rebuild you can run, in another terminal window, from the root directory of the website:
$ (cd antora-ui-camel && yarn build) && yarn antora --clean --fetch antora-playbook.yml
This will build the antora-ui-camel which holds all the CSS and JavaScript, and then rebuild the documentation, resulting in an updated content in the documentation directory.
To iterate quickly, it's easier to make changes directly in the browser tooling and then bring the changes over to the CSS files after the fact.
To build the documentation we pull the content of git repositories (see antora-playbook.yml), so to make changes locally and have them built or previewed without those changes being merged to Camel git repositories you need to adapt the antora-playbook.yml file.
For example to work on the user manual locally change the content, sources to point to HEAD of your local git repository, in this example located in ../camel:
content:
sources:
- url: ../camel
branches: HEAD
start_paths:
# manual
- docs/user-manualNow you can run yarn build:antora or yarn preview and see the locally made changes. More details on this you can find in the Antora documentation.
Typical workflow is to run the yarn preview in one command line session and then rebuild the Antora documentation by running yarn build:antora for the documentation to be refreshed.
TIP: We pull in several git repositories and build several versions (branches) of documentation from them, time can be saved by removing sources for the documentation not worked on. Though be careful about inter-dependencies, for example several documents in the component reference point to the user manual.
Setting the CAMEL_ENV changes the output of the website build slightly, possible values are development (set by default if CAMEL_ENV is unset), production or netlify.
To run the optimizations, which currently consist of running htmlmin to reduce the size of generated HTML documents, set the CAMEL_ENV environment variable to production, for example:
$ CAMEL_ENV=production yarn build
When build is performed on Netlify, we set it to netlify to add the link to Netlify required by Netlify's open source policy.
The Apache Camel website is composed of different sources. So where to add and contribute changes in particular?
The site main menu is defined in the top level configuration config.toml. You can add/change menu items there.
The basic website content is located in content. You can find several different directorys representing different areas of the website:
- docs: Getting started, user manual, component reference
- download: Download Camel artifacts
- blog: Blog posts
- community: Support, contributing, articles, etc.
- projects: Subproject information (e.g. Camel K)
- security: Security information and advisories
- releases: Release notes
Use the blog archetype to create a new markdown content file in content/blog:
$ yarn hugo new --kind blog blog/YYYY/MM/PostName/index.md # replace YYYY with the year, MM with the month and PostName with the actual name
Put a nice featured image in content/blog/YYYY/MM/PostName/featured.png and edit content/blog/YYYY/MM/PostName/index.md filling in the details.
The final generated URL would be something like https://camel.apache.org/blog/2020/05/MyNewPost/.
Don't forget to remove draft: true to publish the blog post.
Use the security-advisory archetype to create a new markdown content file in content/security:
$ yarn run hugo new --kind security-advisory security/CVE-YYYY-NNNNN # replace YYYY-NNNNN with the CVE number
This will create a content/security/CVE-YYYY-NNNNN.md file which you need to edit to and fill in the required parameters.
The content of the created markdown file is added to the Notes section.
Place the signed PGP advisory in plain text as content/security/CVE-YYYY-NNNNN.txt.asc.
Make sure that you set the draft: false property to have the page published.
Use the release-note archetype to create a new markdown content file in content/releases:
$ yarn run hugo new --kind release-note releases/release-x.y.z.md # replace x.y.z with the release version
This will create a content/release-x.y.z.md file which you need to edit to and fill in the required parameters.
The content of the created markdown file is added to the New and Noteworthy section.
Make sure that you set the draft: false property to have the page published.
Use the release-{category} archetype to create a new markdown content file in content/releases/{category}-{version with underscores}/index.md.
For {category} you can use: k k-runtime, ckc, or q.
For example, to create Camel Kafka Connector release note:
$ yarn run hugo new --kind release-ckc releases/ckc/release-x.y.z.md # replace x.y.z with the release version (use underscores)
This will create a content/releases/ckc/release-x.y.z.md file which you need to edit to and fill in the required parameters.
The content of the created markdown file is added to the New and Noteworthy section.
Make sure that you set the draft: false property to have the page published.
Layout related changes go into layout directory where you will find HTML templates that define the common layout of the different page categories including footer and header templates.
The Antora UI theme basically defines the look and feel of the website. You can find the theme sources within this repository in antora-ui-camel.
You need to rebuild the Antora UI theme in order to see your changes reflected locally.
Edits to individual pages can generally be made from the "edit this page" link at the top left of each antora-generated page. However, this will not yet work for Camel component documentation pages.
For more extensive changes, use of the Intellij Asciidoctor plugin is extremely convenient as it provides extensive syntax checking and understands a great deal of Antora-specific asciidoc syntax such as xrefs. The plugin works with all Intellij editors including the free IDEA CE.
Consult the Antora documentation and Asciidoctor documentation for complete information on Antora and AsciidDoc/Asciidoctor.
The Apache Camel website includes documentation sources from other github repositories. Content sources are defined in antora-playbook.yml.
Your changes in these repositories will automatically be included in the website after a site rebuild.
Links to other pages in Antora content are defined using the xref: inline macro. This takes the form xref:<optional #fragment>[]. The default link text for non-fragment links is the target page title.
Antora documentation for page ids is here. A small project showing the effect of all possible forms of page id is here.
The first step is to determine the component, version, and module of both the source and target pages.
To determine the component name and version of a page, look in the antora.yml file next to the "modules" folder the page source is in. The "module" is the folder name under "modules". The "default" module is "ROOT": leaving out the module name will always take you to the ROOT module.
Several repositories and start paths in a repository can contribute to the same component/version. For instance, in the main camel repository, the "components" component has sources under docs/components (dataformats, languages, other, and ROOT (components) modules) and core/camel-core-engine/src/main/docs (eips module). The easiest way to determine the current set of components and their versions is to consult the website and look in the "drawer" at the bottom left, showing the current component/version. When opened it lists the components and for each component the versions. The main camel repository contains both the "components" and "user manual" Antora components. Other components are in the obvious repository.
- To link to another page in the same component/version/module use the absolute path from "pages": xref:path/to/page.adoc[]. For instance, a link from one camel component to another would be xref:jms-component.adoc[]. A link from any camel-quarkus page to a camel-quarkus extension would be xref:reference/extensions/jms.adoc[].
- To link to a page in another module in the same component/version use the module name and absolute path from "pages": xref:module:path/to/page.adoc[]. For instance, a link from a dataformat to a language would be xref:languages:jsonpath-language.adoc[]. A link from a dataformat to an eip would be xref:eips:aggregate-eip.adoc[].
Xref links that do not specify the component or version stay within the same component/version. If only the (Antora) component is specified, the link will be to the "latest" version of the component. This is most likely what you want.
- For instance, a link from camel-quarkus to the latest version of a component would be xref:components::jms-component.adoc[].
- A link from a component, eip, or other "components" module to the user manual would be xref:manual::architecture.adoc[].
- A link from a component, eip, etc, or the user manual to a camel-quarkus extension would be xref:camel-quarkus::reference/extensions/jms.adoc[].
- Note that specifying the same component as the source page will link to the latest version of the target page: e.g. xref:camel-quarkus::reference/extensions/jms.adoc[] from a version 2.0.0 camel-quarkus page will link to the latest jms.adoc version, not the 2.0.0 version.
If you need to link to a specific non-latest version of a page, specify the version in the xref.
- A link from anywhere to a camel-quarkus 2.0.0 page would be xref:2.0.0@camel-quarkus::reference/extensions/jms.adoc[].
To avoid having to install everything on your local machine or if you are a Mac or Windows user, you still have the possibility to launch the yarn commands directly inside a Docker container in which all the required tools have been preinstalled.
Assuming that Docker is properly installed on your local machine and the Docker daemon is started, all you need to do is to call the script yarn-in-docker.sh or yarn-in-docker.bat with the arguments that you would like to pass to the yarn command.
For example to launch yarn with build-all as argument inside a Docker container, the corresponding command is then:
$ ./yarn-in-docker.sh build-all
Equivalent for Windows
> yarn-in-docker.bat build-all
Note - The environment variables GITHUB_TOKEN and CAMEL_ENV locally set are automatically propagated to the Docker container.
The project provides a simple way to build the website sources locally using the build tool Maven.
The Maven build automatically downloads the tool binaries such as node and yarn for you. You do not need to install
those tools on your host then. The binaries are added to the local project sources only and generate the website content.
As the Maven build uses pinned versions of node and yarn that are tested to build the website you most likely avoid
build errors due to incompatible versions of Node.js tooling installed on your machine.
Make sure that you have Maven installed.
$ mvn --version
If this command fails with an error, you do not have Maven installed.
Please install Maven using your favorite package manager (like Homebrew) or from official Maven binaries
When building everything from scratch the build executes following steps:
- Download
yarnandNode.jsbinaries to the local project - Load required libraries to the local project using
yarn - Build the Antora Camel UI theme (antora-ui-camel)
- Fetch the doc sources from Camel and Camel K github reporsitories
- Build the website content using Hugo
You can do all of this with one single command:
$ mvn package
The whole process takes up to five minutes (time to grab some coffee!)
When the build is finished you should see the generated website in the public directory.
When rebuilding the website you can optimize the build process as some of the steps are only required for a fresh build from scratch. You can skip the ui theme rendering (unless you have changes in the theme itself).
$ mvn package -Dskip.theme
This should save you some minutes in the build process. You can find the updated website content in the public directory.
When rebuilding the website the process uses some cached content (e.g. the fetched doc sources for
Camel and Camel K or the Antora ui theme).
If you want to start from scratch for some reason you can simply add the clean operation to the build which removes
all generated sources in the project first.
$ mvn clean package
Of course this then takes some more time than an optimized rebuild (time to grab another coffee!).
The content of the website, as built by the Camel.website job, is served from the asf-site branch and served by ASF Infrastrucuture team.
For the site to be published a number of checks need to pass, these include two levels of link checking:
Antora xref and HTML link checker;
HTML validation. In local development those checks can be run with yarn checks,
to increase the local turnaround time some lengthy checks are not run unless CAMEL_ENV=production environment
variable is set. To run all checks use:
$ CAMEL_ENV=production yarn checks
Publishing the website is done by ASF Jenkins
this includes running all the checks. It is common that a check might fail there that hasn't failed when the
change was made, this could be for any number of reasons, but most commonly there was a change in one of
the subproject's documentation, and most common issue is an introduction of a broken or absolute link towards
camel.apache.org domnain.
The configuration of the HTML validation rules is in .htmlvalidate.json, with exclusions of checks listed in
.htmlvalidateignore file, custom rules are in rules.js for mandating relative links to camel.apache.org domain,
JSON-LD schema validation, and mandating that the HTML title be set.
This website is hosted by Apache Software foundation. Pull request previews and checks are powered by Netlify.
