From 0fb5291dd08946e627f0935918be7010d0812a09 Mon Sep 17 00:00:00 2001 From: Alex Berg Date: Wed, 16 Apr 2014 11:21:41 -0500 Subject: [PATCH 1/5] Update cloud-config.md to include expected file format Clarify root-level keys. Use page structure to indicate expected values. --- Documentation/cloud-config.md | 72 ++++++++++++++++++++++++----------- 1 file changed, 50 insertions(+), 22 deletions(-) diff --git a/Documentation/cloud-config.md b/Documentation/cloud-config.md index 5fe4aaa..cc325da 100644 --- a/Documentation/cloud-config.md +++ b/Documentation/cloud-config.md @@ -1,20 +1,49 @@ -# Customize with Cloud-Config +# Functionality Overview -CoreOS allows you to configure networking, create users, launch systemd units on startup and more. We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms. +CoreOS supports customization of its initialization procedure by specifying values to use when initializing various OS-level items. Such items include network configuration, user accounts, and systemd units to launch on startup. This document defines the full list of initialization items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. -Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS. An example file containing all available options can be found at the bottom of this page. +# Configuration File +The file used by this system initialization program is called a "cloud-config" file, and it aims to implement all of the parameters of the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file, which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Currently, however, only a subset of that file's configuration items are implemented. + +In addition to the configuration items in the cloud-init project's cloud-config file, we added a few additional items which are specific to CoreOS. These CoreOS-specific items include etcd configuration, OEM definition, and systemd units to launch on startup. + +We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms. + +[cloud-init]: https://launchpad.net/cloud-init +[cloud-init-docs]: http://cloudinit.readthedocs.org/en/latest/index.html [cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data -## How to Provide Cloud-Config +## File Format + +The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values. + +A cloud-config file should contain an associative array which has zero or more of the following keys: + +- `coreos` +- `ssh_authorized_keys` +- `hostname` +- `users` +- `write_files` +- `manage_etc_hosts` + +The expected values for these keys are defined in the rest of this document. + +[yaml]: https://en.wikipedia.org/wiki/YAML + +## Providing Cloud-Config with Config-Drive CoreOS tries to conform to each platform's native method to provide user data. Each cloud provider tends to be unique, but this complexity has been abstracted by CoreOS. You can view each platform's instructions on their documentation pages. The most universal way to provide cloud-config is [via config-drive](https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/config-drive.md), which attaches a read-only device to the machine, that contains your cloud-config file. +# Configuration Options + ## CoreOS Parameters -### coreos.etcd +## `coreos` -The `coreos.etcd.*` options are translated to a partial systemd unit acting as an etcd configuration file. +### `etcd` + +The `coreos.etcd.*` options will be translated to a partial systemd unit acting as an etcd configuration file. We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document... ``` @@ -45,10 +74,9 @@ Note that hyphens in the coreos.etcd.* keys are mapped to underscores. [etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md -### coreos.oem +### `oem` -These fields are borrowed from the [os-release spec][os-release] and repurposed -as a way for coreos-cloudinit to know about the OEM partition on this machine: +The `coreos.oem.*` options are the following fields, which are borrowed from the [os-release spec][os-release] and repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine: - **id**: Lowercase string identifying the OEM - **name**: Human-friendly string representing the OEM @@ -84,9 +112,9 @@ BUG_REPORT_URL="https://github.com/coreos/coreos-overlay" [os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html -### coreos.units +### `units` -Arbitrary systemd units may be provided in the `coreos.units` attribute. +The `coreos.units.*` options define arbitrary systemd units to start. `coreos.units` is a list of objects with the following fields: - **name**: String representing unit's name. Required. @@ -135,9 +163,7 @@ coreos: command: start ``` -## Cloud-Config Parameters - -### ssh_authorized_keys +## `ssh_authorized_keys` Provided public SSH keys will be authorized for the `core` user. @@ -151,7 +177,7 @@ ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h... ``` -### hostname +## `hostname` The provided value will be used to set the system's hostname. This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`). @@ -162,7 +188,7 @@ This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.exam hostname: coreos1 ``` -### users +## `users` Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields. Each field is optional and of type string unless otherwise noted. @@ -203,7 +229,7 @@ users: - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h... ``` -#### Generating a password hash +### Generating a password hash If you choose to use a password instead of an SSH key, generating a safe hash is extremely important to the security of your system. Simplified hashes like md5crypt are trivial to crack on modern GPU hardware. Here are a few ways to generate secure hashes: @@ -223,9 +249,11 @@ perl -e 'print crypt("password","\$6\$SALT\$") . "\n"' Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed. On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms. -#### Retrieving ssh authorized keys from a GitHub user +### Retrieving SSH Authorized Keys -Using the field `coreos-ssh-import-github` you can make coreos-cloudinit to add the public ssh keys from a GitHub user as authorized keys to a server. +#### From a GitHub User + +Using the field `coreos-ssh-import-github`, you can make coreos-cloudinit to add the public ssh keys from a GitHub user as authorized keys to a server. ``` #cloud-config @@ -235,7 +263,7 @@ users: coreos-ssh-import-github: elroy ``` -#### Retrieving ssh authorized keys from an http endpoint +#### From an HTTP Endpoint coreos-cloudinit can also pull public SSH keys from any http endpoint that matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user). For example, if you have an installation of GitHub Enterprise, you can provide a complete url with an authentication token: @@ -258,7 +286,7 @@ users: coreos-ssh-import-url: https://example.com/public-keys ``` -### write_files +## `write_files` Inject an arbitrary set of files to the local filesystem. Provide a list of objects with the following attributes: @@ -271,7 +299,7 @@ Provide a list of objects with the following attributes: Explicitly not implemented is the **encoding** attribute. The **content** field must represent exactly what should be written to disk. -### manage_etc_hosts +## `manage_etc_hosts` Have coreos-cloudinit manage your /etc/hosts file for local name resolution. The only supported value is "localhost" which will cause your system's hostname From 4c02e99bc85abae2ff9f6f6911d80d23d3867a9a Mon Sep 17 00:00:00 2001 From: Alex Berg Date: Wed, 16 Apr 2014 16:44:24 -0500 Subject: [PATCH 2/5] Update cloud-config.md option descriptions Re-word a few more things to look more like docs. --- Documentation/cloud-config.md | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/Documentation/cloud-config.md b/Documentation/cloud-config.md index cc325da..3d212b0 100644 --- a/Documentation/cloud-config.md +++ b/Documentation/cloud-config.md @@ -165,10 +165,10 @@ coreos: ## `ssh_authorized_keys` -Provided public SSH keys will be authorized for the `core` user. +The `ssh_authorized_keys` option adds public SSH keys which will be authorized for the `core` user. The keys will be named "coreos-cloudinit" by default. -Override this with the `--ssh-key-name` flag when calling `coreos-cloudinit`. +Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`. ``` #cloud-config @@ -179,7 +179,7 @@ ssh_authorized_keys: ## `hostname` -The provided value will be used to set the system's hostname. +The `hostname` option defines the system's hostname. This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`). ``` @@ -190,8 +190,7 @@ hostname: coreos1 ## `users` -Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields. -Each field is optional and of type string unless otherwise noted. +The `users` option adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted. All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists. - **name**: Required. Login name of user @@ -253,7 +252,7 @@ Using a higher number of rounds will help create more secure passwords, but give #### From a GitHub User -Using the field `coreos-ssh-import-github`, you can make coreos-cloudinit to add the public ssh keys from a GitHub user as authorized keys to a server. +Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server. ``` #cloud-config @@ -265,8 +264,8 @@ users: #### From an HTTP Endpoint -coreos-cloudinit can also pull public SSH keys from any http endpoint that matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user). -For example, if you have an installation of GitHub Enterprise, you can provide a complete url with an authentication token: +We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user). +For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token: ``` #cloud-config @@ -276,7 +275,7 @@ users: coreos-ssh-import-url: https://token:@github-enterprise.example.com/users/elroy/keys ``` -You can also provide any url which response matches that json format for public keys: +You can also specify any URL whose response matches the JSON format for public keys: ``` #cloud-config @@ -288,8 +287,7 @@ users: ## `write_files` -Inject an arbitrary set of files to the local filesystem. -Provide a list of objects with the following attributes: +The `write-file` option defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys: - **path**: Absolute location on disk where contents should be written - **content**: Data to write at the provided `path` @@ -301,8 +299,8 @@ The **content** field must represent exactly what should be written to disk. ## `manage_etc_hosts` -Have coreos-cloudinit manage your /etc/hosts file for local name resolution. -The only supported value is "localhost" which will cause your system's hostname +The `manage_etc_hosts` option configures the contents of the `/etc/hosts` file, which is used for local name resolution. +Currently, the only supported value is "localhost" which will cause your system's hostname to resolve to "127.0.0.1". This is helpful when the host does not have DNS infrastructure in place to resolve its own hostname, for example, when using Vagrant. From 8bee85e63df6ebbe1ffce2e3c43462c5917e0f0c Mon Sep 17 00:00:00 2001 From: Alex Berg Date: Wed, 16 Apr 2014 17:04:35 -0500 Subject: [PATCH 3/5] Update cloud-config.md based on feedback --- Documentation/cloud-config.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/Documentation/cloud-config.md b/Documentation/cloud-config.md index 3d212b0..5b0116c 100644 --- a/Documentation/cloud-config.md +++ b/Documentation/cloud-config.md @@ -1,12 +1,10 @@ # Functionality Overview -CoreOS supports customization of its initialization procedure by specifying values to use when initializing various OS-level items. Such items include network configuration, user accounts, and systemd units to launch on startup. This document defines the full list of initialization items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. +CoreOS allows us to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. # Configuration File -The file used by this system initialization program is called a "cloud-config" file, and it aims to implement all of the parameters of the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file, which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Currently, however, only a subset of that file's configuration items are implemented. - -In addition to the configuration items in the cloud-init project's cloud-config file, we added a few additional items which are specific to CoreOS. These CoreOS-specific items include etcd configuration, OEM definition, and systemd units to launch on startup. +The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units. We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms. @@ -76,7 +74,7 @@ Note that hyphens in the coreos.etcd.* keys are mapped to underscores. ### `oem` -The `coreos.oem.*` options are the following fields, which are borrowed from the [os-release spec][os-release] and repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine: +The `coreos.oem.*` options follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine: - **id**: Lowercase string identifying the OEM - **name**: Human-friendly string representing the OEM From c3b53f24cf3613411362c5fd0c6ba4223aadc1b8 Mon Sep 17 00:00:00 2001 From: Alex Berg Date: Wed, 16 Apr 2014 17:09:54 -0500 Subject: [PATCH 4/5] Update cloud-config.md to use "parameter", not "option" --- Documentation/cloud-config.md | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/Documentation/cloud-config.md b/Documentation/cloud-config.md index 5b0116c..dfb142f 100644 --- a/Documentation/cloud-config.md +++ b/Documentation/cloud-config.md @@ -33,7 +33,7 @@ The expected values for these keys are defined in the rest of this document. CoreOS tries to conform to each platform's native method to provide user data. Each cloud provider tends to be unique, but this complexity has been abstracted by CoreOS. You can view each platform's instructions on their documentation pages. The most universal way to provide cloud-config is [via config-drive](https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/config-drive.md), which attaches a read-only device to the machine, that contains your cloud-config file. -# Configuration Options +# Configuration Parameters ## CoreOS Parameters @@ -41,7 +41,7 @@ CoreOS tries to conform to each platform's native method to provide user data. E ### `etcd` -The `coreos.etcd.*` options will be translated to a partial systemd unit acting as an etcd configuration file. +The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file. We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document... ``` @@ -67,14 +67,14 @@ Environment="ETCD_ADDR=203.0.113.29:4001" Environment="ETCD_PEER_ADDR=192.0.2.13:7001" ``` -For more information about the available configuration options, see the [etcd documentation][etcd-config]. +For more information about the available configuration parameters, see the [etcd documentation][etcd-config]. Note that hyphens in the coreos.etcd.* keys are mapped to underscores. [etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md ### `oem` -The `coreos.oem.*` options follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine: +The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine: - **id**: Lowercase string identifying the OEM - **name**: Human-friendly string representing the OEM @@ -112,8 +112,7 @@ BUG_REPORT_URL="https://github.com/coreos/coreos-overlay" ### `units` -The `coreos.units.*` options define arbitrary systemd units to start. -`coreos.units` is a list of objects with the following fields: +The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields: - **name**: String representing unit's name. Required. - **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` argument to `systemd enable`. Default value is false. @@ -163,7 +162,7 @@ coreos: ## `ssh_authorized_keys` -The `ssh_authorized_keys` option adds public SSH keys which will be authorized for the `core` user. +The `ssh_authorized_keys` parameter adds public SSH keys which will be authorized for the `core` user. The keys will be named "coreos-cloudinit" by default. Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`. @@ -177,7 +176,7 @@ ssh_authorized_keys: ## `hostname` -The `hostname` option defines the system's hostname. +The `hostname` parameter defines the system's hostname. This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`). ``` @@ -188,7 +187,7 @@ hostname: coreos1 ## `users` -The `users` option adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted. +The `users` parameter adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted. All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists. - **name**: Required. Login name of user @@ -285,7 +284,7 @@ users: ## `write_files` -The `write-file` option defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys: +The `write-file` parameter defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys: - **path**: Absolute location on disk where contents should be written - **content**: Data to write at the provided `path` @@ -297,7 +296,7 @@ The **content** field must represent exactly what should be written to disk. ## `manage_etc_hosts` -The `manage_etc_hosts` option configures the contents of the `/etc/hosts` file, which is used for local name resolution. +The `manage_etc_hosts` parameter configures the contents of the `/etc/hosts` file, which is used for local name resolution. Currently, the only supported value is "localhost" which will cause your system's hostname to resolve to "127.0.0.1". This is helpful when the host does not have DNS infrastructure in place to resolve its own hostname, for example, when using Vagrant. From ae350a3b34d252866e801c03fa412b376680ec47 Mon Sep 17 00:00:00 2001 From: Alex Berg Date: Wed, 16 Apr 2014 19:09:12 -0500 Subject: [PATCH 5/5] Update cloud-config.md - use "you" --- Documentation/cloud-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/cloud-config.md b/Documentation/cloud-config.md index dfb142f..9947f80 100644 --- a/Documentation/cloud-config.md +++ b/Documentation/cloud-config.md @@ -1,6 +1,6 @@ # Functionality Overview -CoreOS allows us to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. +CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. # Configuration File