Skip to content

Commit

Permalink
Even more getting started docs. Marked which component are optional
Browse files Browse the repository at this point in the history
  • Loading branch information
solarkennedy committed Oct 30, 2015
1 parent 0e05527 commit 8c2c311
Show file tree
Hide file tree
Showing 2 changed files with 70 additions and 19 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ docs:
tox -e docs

test:
# rm -rf .tox
rm -rf .tox
tox

itest: test
Expand Down
87 changes: 69 additions & 18 deletions docs/source/installation/getting_started.rst
Original file line number Diff line number Diff line change
@@ -1,19 +1,28 @@
Getting Started
===============

**Warning**: PaaSTA is an opinionated way to integrate a collection of open source components in a holistic way to build a PaaS. It is not optimized to be simple to deploy for operators. It is optimized to not reinvent the wheel and utilized existing solutions to problems where possible.
**Warning**: PaaSTA is an opinionated way to integrate a collection of open
source components in a holistic way to build a PaaS. It is not optimized to be
simple to deploy for operators. It is optimized to not reinvent the wheel and
utilizes existing solutions to problems where possible.

PaaSTA has many dependencies. This document provides documentation on installing some of these dependencies, but some of them are left up as an exercise to the reader.
PaaSTA has many dependencies. This document provides documentation on
installing some of these dependencies, but some of them are left as an
exercise to the reader.


soa-configs
-----------

soa-configs are the shared configuration storage that PaaSTA uses to hold the description and configuration of what services exist and how they should be deployed and monitored.
soa-configs are the shared configuration storage that PaaSTA uses to hold the
description and configuration of what services exist and how they should be
deployed and monitored.

This directory needs to be deployed globally in the same location to every server that runs any PaaSTA component. We recommend using a git-based distribution mechanism for access control and auditing.
This directory needs to be deployed globally in the same location to every
server that runs any PaaSTA component.

PaaSTA reads particular config files for each service in the soa-configs directory. There is one folder per service. Here is an example tree::
PaaSTA reads particular config files for each service in the soa-configs
directory. There is one folder per service. Here is an example tree::

soa-configs
├── web
Expand All @@ -32,54 +41,95 @@ PaaSTA reads particular config files for each service in the soa-configs directo
│   └── smartstack.yaml
...

For the `soa-configs documentation <yelpsoa-configs>`_ for more information about the structure and contents of these files.
See the `soa-configs documentation <yelpsoa-configs>`_ for more information
about the structure and contents of these files.

For more information about why we chose this method of config distribution, see watch `this talk on Yelp's soa-config and how they are used <https://vimeo.com/141231345>`_.
For more information about why we chose this method of config distribution,
watch `this talk on Yelp's soa-config and how it is used <https://vimeo.com/141231345>`_.

Docker and a Docker Registry
----------------------------

PaaSTA uses Docker to build and distribute code for each service. PaaSTA assumes that a single registry is available and that the associated components (Docker commands, unix users, mesos slaves, etc) have the correct credentials to use it.
PaaSTA uses `Docker <https://www.docker.com/>`_ to build and distribute code for each service. PaaSTA
assumes that a single registry is available and that the associated components
(Docker commands, unix users, mesos slaves, etc) have the correct credentials
to use it.

The docker registry needs to be defined in a config file in ``/etc/paasta/``. PaaSTA merges all json files in ``/etc/paasta/`` together, so the actual filename is irrelevant, but here would be an example ``/etc/paasta/docker.json``::
The docker registry needs to be defined in a config file in ``/etc/paasta/``.
PaaSTA merges all json files in ``/etc/paasta/`` together, so the actual
filename is irrelevant, but here would be an example
``/etc/paasta/docker.json``::

{
"docker_registry": "private-docker-registry.example.com:443"
}

There are many registries available to use, or you can `host your own <https://docs.docker.com/registry/>`_.
There are many registries available to use, or you can
`host your own <https://docs.docker.com/registry/>`_.

Mesos
-----

TBD
PaaSTA uses Mesos to do the heavy lifting of running the actual services on
pools of machines. See the `official documentation <http://mesos.apache.org/gettingstarted/>`_
on how to get started with Mesos.

Marathon
--------

TBD
PaaSTA uses `Marathon <https://mesosphere.github.io/marathon/>`_ for supervising long-running
services running in Mesos. See the `official documentation <https://mesosphere.github.io/marathon/docs/>`_
for how to get started with Marathon.
, and then see the `PaaSTA documentation <yelpsoa_configs.html#marathon-clustername-yaml>`_
for how to define Chronos jobs.

This comment has been minimized.

Copy link
@Rob-Johnson

Rob-Johnson Oct 30, 2015

Contributor

s/Chronos/Marathon/


Chronos
-------

TBD
`Chronos <http://mesos.github.io/chronos/>`_ is a Marathon framework for
running scheduled tasks. See the `official documentation <http://mesos.github.io/chronos/docs/getting-started.html>`_
for how to get started with Chronos, and then see the `PaaSTA documentation <yelpsoa_configs.html#marathon-clustername-yaml>`_
for how to define Chronos jobs.

SmartStack
----------

TBD
`SmartStack <http://nerds.airbnb.com/smartstack-service-discovery-cloud/>`_ is
a dynamic service discovery system that allows clients to find and route to
healthy mesos tasks for a particular service.

The glue that actually configures SmartStack is `currently not open source <https://github.com/Yelp/paasta/issues/13>`_.

Other service-discovery mechanisms can be used with PaaSTA, but SmartStack
is currently the only supported method.

Sensu
-----

TBD
`Sensu <https://sensuapp.org/>`_ is a flexible and scalable monitoring system
that allows clients to send alerts for arbitary events. PaaSTA uses Sensu to allow
individual teams to get alerts for their services.

The `official documentation <https://sensuapp.org/docs/latest/overview>` has
instructions on how to set it up.

Out of the box Sensu doesn't understand team-centric routing, and must be combined
with handlers that are "team aware" it it is installed in a multi-tenant environment.
We to do that, we have written some `custom Sensu handlers <https://github.com/Yelp/sensu_handlers>`_
to do that.

Sensu is an optional but highly recommended component.

This comment has been minimized.

Copy link
@Rob-Johnson

Rob-Johnson Oct 30, 2015

Contributor

I'd make a note here to highlight that sensu or nothing. We don't support other monitoring systems, so if you want to monitor, it has to be with sensu. 'Sensu is optional' makes it seem like we support others.


Jenkins / Build Orchestration
-----------------------------

Jenkins is the suggested method for orchestrating build pipelines for services, but it is not a hard requirement. The actual method that Yelp uses to integrate Jenkins with PaaSTA is not open source.
Jenkins is the suggested method for orchestrating build pipelines for services,
but it is not a hard requirement. The actual method that Yelp uses to integrate
Jenkins with PaaSTA is not open source.

In practice, each organization will have to decide how they want to actually run the ``paasta`` cli tool to kick off the building and deploying of images. This may be something as simple as a bash script::
In practice, each organization will have to decide how they want to actually
run the ``paasta`` cli tool to kick off the building and deploying of images.
This may be something as simple as a bash script::

#!/bin/bash
service=my_service
Expand All @@ -88,4 +138,5 @@ In practice, each organization will have to decide how they want to actually run
paasta push-to-registry --service $service --commit $sha
paasta mark-for-deployment --git-url $(git config --get remote.origin.url) --commit $sha --clusterinstance prod.main --service $service

We designed PaaSTA to use normal command line tools so it could be integrated with opinionated existing workflows.
PaaSTA can integrate with any existing orchestration tool that can execute
commands like this.

0 comments on commit 8c2c311

Please sign in to comment.