diff --git a/docs/source/index.rst b/docs/source/index.rst index 6693e71e18..e58f0d7645 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,17 +1,12 @@ -.. paasta_tools documentation master file, created by - sphinx-quickstart on Tue Aug 5 17:43:04 2014. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - -paasta_tools: PaaSTA sauce -====================================== +paasta_tools: PaaSTA Sauce +========================== paasta_tools is a Python library. It is the sauce that combines the individual strands of `PaaSTA `_ -- Jenkins, Docker, Mesos, etc. -- into a cohesive, highly-available, distributed meal. -Directions ----------- +Directions (For PaaSTA Users) +----------------------------- .. toctree:: :maxdepth: 2 @@ -74,6 +69,14 @@ Development upgrading_marathon upgrading_mesos +Installing The PaaSTA Infrastructure +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. toctree:: + :maxdepth: 2 + + installation/getting_started + Indices and tables ------------------ diff --git a/docs/source/installation/getting_started.rst b/docs/source/installation/getting_started.rst new file mode 100644 index 0000000000..34b605622d --- /dev/null +++ b/docs/source/installation/getting_started.rst @@ -0,0 +1,142 @@ +Getting Started +=============== + +**Warning**: PaaSTA is an opinionated way to integrate a collection of open +source components in a holistic way to build a PaaS. It is not optimized to be +simple to deploy for operators. It is optimized to not reinvent the wheel and +utilizes existing solutions to problems where possible. + +PaaSTA has many dependencies. This document provides documentation on +installing some of these dependencies, but some of them are left as an +exercise to the reader. + + +soa-configs +----------- + +soa-configs are the shared configuration storage that PaaSTA uses to hold the +description and configuration of what services exist and how they should be +deployed and monitored. + +This directory needs to be deployed globally in the same location to every +server that runs any PaaSTA component. + +PaaSTA reads particular config files for each service in the soa-configs +directory. There is one folder per service. Here is an example tree:: + + soa-configs + ├── web + │   ├── deploy.yaml + │   ├── marathon-dev.yaml + │   ├── marathon-prod.yaml + │   ├── monitoring.yaml + │   ├── service.yaml + │   └── smartstack.yaml + ├── api + │   ├── deploy.yaml + │   ├── marathon-dev.yaml + │   ├── marathon-prod.yaml + │   ├── monitoring.yaml + │   ├── service.yaml + │   └── smartstack.yaml + ... + +See the `soa-configs documentation `_ for more information +about the structure and contents of these files. + +For more information about why we chose this method of config distribution, +watch `this talk on Yelp's soa-config and how it is used `_. + +Docker and a Docker Registry +---------------------------- + +PaaSTA uses `Docker `_ to build and distribute code for each service. PaaSTA +assumes that a single registry is available and that the associated components +(Docker commands, unix users, mesos slaves, etc) have the correct credentials +to use it. + +The docker registry needs to be defined in a config file in ``/etc/paasta/``. +PaaSTA merges all json files in ``/etc/paasta/`` together, so the actual +filename is irrelevant, but here would be an example +``/etc/paasta/docker.json``:: + + { + "docker_registry": "private-docker-registry.example.com:443" + } + +There are many registries available to use, or you can +`host your own `_. + +Mesos +----- + +PaaSTA uses Mesos to do the heavy lifting of running the actual services on +pools of machines. See the `official documentation `_ +on how to get started with Mesos. + +Marathon +-------- + +PaaSTA uses `Marathon `_ for supervising long-running +services running in Mesos. See the `official documentation `_ +for how to get started with Marathon. +, and then see the `PaaSTA documentation `_ +for how to define Chronos jobs. + +Chronos +------- + +`Chronos `_ is a Marathon framework for +running scheduled tasks. See the `official documentation `_ +for how to get started with Chronos, and then see the `PaaSTA documentation `_ +for how to define Chronos jobs. + +SmartStack +---------- + +`SmartStack `_ is +a dynamic service discovery system that allows clients to find and route to +healthy mesos tasks for a particular service. + +The glue that actually configures SmartStack is `currently not open source `_. + +Other service-discovery mechanisms can be used with PaaSTA, but SmartStack +is currently the only supported method. + +Sensu +----- + +`Sensu `_ is a flexible and scalable monitoring system +that allows clients to send alerts for arbitary events. PaaSTA uses Sensu to allow +individual teams to get alerts for their services. + +The `official documentation ` has +instructions on how to set it up. + +Out of the box Sensu doesn't understand team-centric routing, and must be combined +with handlers that are "team aware" it it is installed in a multi-tenant environment. +We to do that, we have written some `custom Sensu handlers `_ +to do that. + +Sensu is an optional but highly recommended component. + +Jenkins / Build Orchestration +----------------------------- + +Jenkins is the suggested method for orchestrating build pipelines for services, +but it is not a hard requirement. The actual method that Yelp uses to integrate +Jenkins with PaaSTA is not open source. + +In practice, each organization will have to decide how they want to actually +run the ``paasta`` cli tool to kick off the building and deploying of images. +This may be something as simple as a bash script:: + + #!/bin/bash + service=my_service + sha=$(git rev-parse HEAD) + paasta itest --service $service --commit $sha + paasta push-to-registry --service $service --commit $sha + paasta mark-for-deployment --git-url $(git config --get remote.origin.url) --commit $sha --clusterinstance prod.main --service $service + +PaaSTA can integrate with any existing orchestration tool that can execute +commands like this.