Skip to content

Proposal: Model Design

Jump to bottom
Reza Mohammadi edited this page Nov 2, 2015 · 5 revisions

Model

AghaJoon is able to configure machines in several steps. In this proposal, we propose how AghaJoon should give access to the operator to manage the boot process, and we'll go in depth on how this configuration will be stored.

DHCP request

The only management option for the operator in this step will be a flag introduced in issue #1. To have a reliable and stable cluster, we've to store the lease data (the IP assigned to the machine mac) to assign the same IP address on every boot, but this data will not be editable by the operator.

PXE Parameters

AghaJoon passes generated parameters to the kernel of the CoreOS, to pass the address of ignition-config and cloud-config. The only parameter which is going to be configurable in this step is the CoreOS version. We expect that a sub-directory (named 766.4.0 for example) with the corresponding kernel (named coreos_production_pxe.vmlinuz) and initrd (named coreos_production_pxe_image.cpio.gz) be present inside images sub-directory of the workspace directory.

ignition-config and cloud-config

These files are generated by AghaJoon, but AghaJoon isn't aware about the content of these files, except that the generated files will be seemingly valid cloud-config file or ignition-config file. On a request for one of these files, the content of the files in the related sub-directory of the active template (e.x. cloud-config of deis) are yamly merged (sub-trees of write_files of those files are joined together, sub-trees of coreos/units of those files are joined together, ...). Then the generated template is executed against these variables:

Exceptions

  • ssh_authorized_keys list in settings will be added to the ssh_authorized_keys section of the generated cloud-config.

Workspace Directory Structure Example

workspace
   ├─ images
   │   ├─ 766.4.0
   │   |   ├─ coreos_production_pxe.vmlinuz
   │   |   └─ coreos_production_pxe_image.cpio.gz
   │   ├─ 835.1.0
   │   ⋱
   └─ templates
       ├─ kubernetes
       └─ deis

Cluster Template Directory Structure Example

deis
   ├─ cloud-config
   │   ├─ customizations
   │   ├─ deis-components
   │   ├─ docker-and-options
   │   ├─ etcd-client-proxy
   │   ├─ flannel-and-networking
   │   └─ fleet
   └─ ignition-config
       └─ bootstrap-disk

Example content for cluter-name/cloud-config/customizations 

coreos:
  units:
    - name: increase-nf_conntrack-connections.service
      command: start
      content: |
        [Unit]
        Description=Increase the number of connections in nf_conntrack. default is 65536

        [Service]
        Type=oneshot
        ExecStartPre=/usr/sbin/modprobe nf_conntrack
        ExecStart=/bin/sh -c "sysctl -w net.netfilter.nf_conntrack_max=262144"
write_files:
  - path: /etc/environment
    owner: core
    content: |
      public_ipv4={{.IP}}

Etcd2 (data) Structure Example

aghajoon
  ├─ leases
  ├─ nodes
  |   ├─ 382c4ab5f7a3
  |   └─ 382c4ab5f7f0
  └─ settings

Example content for key=aghajoon/settings 

coreos-version: 766.4.0             # Required key
ssh_authorized_keys:
  - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
variables:
  - etcd2-discovery: https://discovery.etcd.io/123456abcdef
  - toolbox-docker-image: alpine

Example content for key=aghajoon/nodes/382c4ab5f7a3 

meta-data: "role=worker,cabinet=two,disk=spinning"   # default=""

Example content for key=aghajoon/leases 

382c4ab5f7a3: 10.0.1.17
382c4ab5f7f0: 10.0.1.18

List of built-in variables

  • IP: e.x. 10.0.1.17
  • MAC: e.x. 382c4ab5f7a3
  • ETCD_SERVERS: e.x. http://10.0.0.101:4001,http://10.0.0.102:4001,http://10.0.0.103:4001
  • FLEET_META_DATA: is read from the node key in etcd
Clone this wiki locally