Compare commits

..

1 Commits

Author SHA1 Message Date
Brandon Philips
68202f3c06 feat(Documentation/cloud-config): add section about oem-release
We need to have cloud-config render a file about itself that can be
easily parsed and used by the rest of the system. Essentially just copy
the os-release format.
2014-03-16 16:47:44 -07:00
29 changed files with 599 additions and 1279 deletions

View File

@@ -1,98 +1,147 @@
# Customize with Cloud-Config
# Customize CoreOS with Cloud-Config
CoreOS allows you to configure networking, create users, launch systemd units on startup and more. We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS. An example file containing all available options can be found at the bottom of this page.
CoreOS allows you to configure machine parameters, launch systemd units on startup and more. Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS.
[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
## CoreOS Parameters
## Supported cloud-config Parameters
### coreos.etcd
### ssh_authorized_keys
The `coreos.etcd.*` options are translated to a partial systemd unit acting as an etcd configuration file.
We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
Provided public SSH keys will be authorized for the `core` user.
```
#cloud-config
The keys will be named "coreos-cloudinit" by default.
Override this with the `--ssh-key-name` flag when calling `coreos-cloudinit`.
coreos:
etcd:
name: node001
discovery: https://discovery.etcd.io/3445fa65423d8b04df07f59fb40218f8
addr: $public_ipv4:4001
peer-addr: $private_ipv4:7001
```
### hostname
...will generate a systemd unit drop-in like this:
The provided value will be used to set the system's hostname.
This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
```
[Service]
Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=https://discovery.etcd.io/3445fa65423d8b04df07f59fb40218f8"
Environment="ETCD_ADDR=203.0.113.29:4001"
Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
```
### users
For more information about the available configuration options, see the [etcd documentation][etcd-config].
Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields.
Each field is optional and of type string unless otherwise noted.
All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
- **name**: Required. Login name of user
- **gecos**: GECOS comment of user
- **passwd**: Hash of the password to use for this user
- **homedir**: User's home directory. Defaults to /home/<name>
- **no-create-home**: Boolean. Skip home directory createion.
- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
- **groups**: Add user to these additional groups
- **no-user-group**: Boolean. Skip default group creation.
- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
- **system**: Create the user as a system user. No home directory will be created.
- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
The following fields are not yet implemented:
- **inactive**: Deactivate the user upon creation
- **lock-passwd**: Boolean. Disable password login for user
- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
- **selinux-user**: Corresponding SELinux user
- **ssh-import-id**: Import SSH keys by ID from Launchpad.
##### Generating a password hash
Generating a safe hash is important to the security of your system. Currently with updated tools like [oclhashcat](http://hashcat.net/oclhashcat/) simplified hashes like md5crypt are trivial to crack on modern GPU hardware. You can generate a "safer" hash (read: not safe, never publish your hashes publicly) via:
###### On Debian/Ubuntu (via the package "whois")
mkpasswd --method=SHA-512 --rounds=4096
###### With OpenSSL (note: this will only make md5crypt. While better than plantext it should not be considered fully secure)
openssl passwd -1
###### With Python (change password and salt values)
python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
###### With Perl (change password and salt values)
perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed. On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms.
### write_files
Inject an arbitrary set of files to the local filesystem.
Provide a list of objects with the following attributes:
- **path**: Absolute location on disk where contents should be written
- **content**: Data to write at the provided `path`
- **permissions**: String representing file permissions in octal notation (i.e. '0644')
- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
## Custom cloud-config Parameters
### coreos.oem
These fields are borrowed from the [os-release spec][os-release] and repurposed
as a way for coreos-cloudinit to know about the OEM partition on this machine:
as a way for cloud-init to know about the OEM partition on this machine.
- **id**: Lowercase string identifying the OEM
- **name**: Human-friendly string representing the OEM
- **version-id**: Lowercase string identifying the version of the OEM
- **home-url**: Link to the homepage of the provider or OEM
- **bug-report-url**: Link to a place to file bug reports about this OEM
- **id**: A lower case string identifying the oem.
- **version-id**: A lower case string identifying the version of the OEM. Example: `168.0.0`
- **name**: A name without the version that is suitable for presentation to the user.
- **home-url**: Link to the homepage of the provider or OEM.
- **bug-report-url***: Link to a place to file bug reports about this OEM partition.
coreos-cloudinit renders these fields to `/etc/oem-release`.
If no **id** field is provided, coreos-cloudinit will ignore this section.
For example, the following cloud-config document...
cloudinit must render these fields down to an /etc/oem-release file on disk in the following format:
```
#cloud-config
coreos:
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
```
...would be rendered to the following `/etc/oem-release`:
```
ID="rackspace"
NAME="Rackspace Cloud Servers"
VERSION_ID="168.0.0"
HOME_URL="https://www.rackspace.com/cloud/servers/"
NAME=Rackspace
ID=rackspace
VERSION_ID=168.0.0
PRETTY_NAME="Rackspace Cloud Servers"
HOME_URL="http://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
```
[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html
### coreos.etcd.discovery_url
The value of `coreos.etcd.discovery_url` will be used to discover the instance's etcd peers using the [etcd discovery protocol][disco-proto]. Usage of the [public discovery service][disco-service] is encouraged.
[disco-proto]: https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md
[disco-service]: http://discovery.etcd.io
### coreos.units
Arbitrary systemd units may be provided in the `coreos.units` attribute.
`coreos.units` is a list of objects with the following fields:
- **name**: String representing unit's name. Required.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` argument to `systemd enable`. Default value is false.
- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
- **name**: string representing unit's name
- **runtime**: boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` flag to `systemd enable`.
- **content**: plaintext string representing entire unit file
**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
See docker example below.
##### Examples
## user-data Script
Write a unit to disk, automatically starting it.
Simply set your user-data to a script where the first line is a shebang:
```
#!/bin/bash
echo 'Hello, world!'
```
## Examples
### Inject an SSH key, bootstrap etcd, and start fleet
```
#cloud-config
coreos:
etcd:
discovery_url: https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877
fleet:
autostart: yes
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
```
### Start a docker container on boot
```
#cloud-config
@@ -115,112 +164,33 @@ coreos:
WantedBy=local.target
```
Start the builtin `etcd` and `fleet` services:
```
# cloud-config
coreos:
units:
- name: etcd.service
command: start
- name: fleet.service
command: start
```
## Cloud-Config Parameters
### ssh_authorized_keys
Provided public SSH keys will be authorized for the `core` user.
The keys will be named "coreos-cloudinit" by default.
Override this with the `--ssh-key-name` flag when calling `coreos-cloudinit`.
```
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
```
### hostname
The provided value will be used to set the system's hostname.
This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
```
#cloud-config
hostname: coreos1
```
### users
Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields.
Each field is optional and of type string unless otherwise noted.
All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
- **name**: Required. Login name of user
- **gecos**: GECOS comment of user
- **passwd**: Hash of the password to use for this user
- **homedir**: User's home directory. Defaults to /home/<name>
- **no-create-home**: Boolean. Skip home directory creation.
- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
- **groups**: Add user to these additional groups
- **no-user-group**: Boolean. Skip default group creation.
- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
- **coreos-ssh-import-github**: Authorize SSH keys from Github user
- **system**: Create the user as a system user. No home directory will be created.
- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
The following fields are not yet implemented:
- **inactive**: Deactivate the user upon creation
- **lock-passwd**: Boolean. Disable password login for user
- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
- **selinux-user**: Corresponding SELinux user
- **ssh-import-id**: Import SSH keys by ID from Launchpad.
### Add a user
```
#cloud-config
users:
- name: elroy
passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
groups:
- staff
- docker
ssh-authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
groups:
- staff
- docker
ssh-authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
```
#### Generating a password hash
If you choose to use a password instead of an SSH key, generating a safe hash is extremely important to the security of your system. Simplified hashes like md5crypt are trivial to crack on modern GPU hardware. Here are a few ways to generate secure hashes:
### Inject configuration files
```
# On Debian/Ubuntu (via the package "whois")
mkpasswd --method=SHA-512 --rounds=4096
#cloud-config
# OpenSSL (note: this will only make md5crypt. While better than plantext it should not be considered fully secure)
openssl passwd -1
# Python (change password and salt values)
python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
# Perl (change password and salt values)
perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
write_files:
- path: /etc/hosts
contents: |
127.0.0.1 localhost
192.0.2.211 buildbox
- path: /etc/resolv.conf
contents: |
nameserver 192.0.2.13
nameserver 192.0.2.14
```
Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed. On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms.
### write_files
Inject an arbitrary set of files to the local filesystem.
Provide a list of objects with the following attributes:
- **path**: Absolute location on disk where contents should be written
- **content**: Data to write at the provided `path`
- **permissions**: String representing file permissions in octal notation (i.e. '0644')
- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.

View File

@@ -1,79 +1,9 @@
# coreos-cloudinit
coreos-cloudinit enables a user to customize CoreOS machines by providing either a cloud-config document or an executable script through user-data.
coreos-cloudinit allows a user to customize CoreOS machines by providing either an executable script or a cloud-config document as instance user-data. See below to learn how to use these features.
## Configuration with cloud-config
## Supported Cloud-Config Features
A subset of the [official cloud-config spec][official-cloud-config] is implemented by coreos-cloudinit.
Additionally, several [CoreOS-specific options][custom-cloud-config] have been implemented to support interacting with unit files, bootstrapping etcd clusters, and more.
All supported cloud-config parameters are [documented here][all-cloud-config].
Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS, which are [documented here](https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md).
[official-cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
[custom-cloud-config]: https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/cloud-config.md#coreos-parameters
[all-cloud-config]: https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md
The following is an example cloud-config document:
```
#cloud-config
coreos:
units:
- name: etcd.service
command: start
users:
- name: core
passwd: $1$allJZawX$00S5T756I5PGdQga5qhqv1
write_files:
- path: /etc/resolv.conf
content: |
nameserver 192.0.2.2
nameserver 192.0.2.3
```
## Executing a Script
coreos-cloudinit supports executing user-data as a script instead of parsing it as a cloud-config document.
Make sure the first line of your user-data is a shebang and coreos-cloudinit will attempt to execute it:
```
#!/bin/bash
echo 'Hello, world!'
```
## user-data Field Substitution
coreos-cloudinit will replace the following set of tokens in your user-data with system-generated values.
| Token | Description |
| ------------- | ----------- |
| $public_ipv4 | Public IPv4 address of machine |
| $private_ipv4 | Private IPv4 address of machine |
These values are determined by CoreOS based on the given provider on which your machine is running.
Read more about provider-specific functionality in the [CoreOS OEM documentation][oem-doc].
[oem-doc]: https://coreos.com/docs/sdk-distributors/distributors/notes-for-distributors/
For example, submitting the following user-data...
```
#cloud-config
coreos:
etcd:
addr: $public_ipv4:4001
peer-addr: $private_ipv4:7001
```
...will result in this cloud-config document being executed:
```
#cloud-config
coreos:
etcd:
addr: 203.0.113.29:4001
peer-addr: 192.0.2.13:7001
```
[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data

143
cloudinit/cloud_config.go Normal file
View File

@@ -0,0 +1,143 @@
package cloudinit
import (
"fmt"
"log"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
)
const DefaultSSHKeyName = "coreos-cloudinit"
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd struct{ Discovery_URL string }
Fleet struct{ Autostart bool }
Units []Unit
}
WriteFiles []WriteFile `yaml:"write_files"`
Hostname string
Users []User
}
func NewCloudConfig(contents []byte) (*CloudConfig, error) {
var cfg CloudConfig
err := goyaml.Unmarshal(contents, &cfg)
return &cfg, err
}
func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
func ApplyCloudConfig(cfg CloudConfig, sshKeyName string) error {
if cfg.Hostname != "" {
if err := SetHostname(cfg.Hostname); err != nil {
return err
}
log.Printf("Set hostname to %s", cfg.Hostname)
}
if len(cfg.Users) > 0 {
for _, user := range cfg.Users {
if user.Name == "" {
log.Printf("User object has no 'name' field, skipping")
continue
}
if UserExists(&user) {
log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
if user.PasswordHash != "" {
log.Printf("Setting '%s' user's password", user.Name)
if err := SetUserPassword(user.Name, user.PasswordHash); err != nil {
log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
return err
}
}
} else {
log.Printf("Creating user '%s'", user.Name)
if err := CreateUser(&user); err != nil {
log.Printf("Failed creating user '%s': %v", user.Name, err)
return err
}
}
if len(user.SSHAuthorizedKeys) > 0 {
log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
if err := AuthorizeSSHKeys(user.Name, sshKeyName, user.SSHAuthorizedKeys); err != nil {
return err
}
}
}
}
if len(cfg.SSHAuthorizedKeys) > 0 {
err := AuthorizeSSHKeys("core", sshKeyName, cfg.SSHAuthorizedKeys)
if err == nil {
log.Printf("Authorized SSH keys for core user")
} else {
return err
}
}
if len(cfg.WriteFiles) > 0 {
for _, file := range cfg.WriteFiles {
if err := ProcessWriteFile("/", &file); err != nil {
return err
}
log.Printf("Wrote file %s to filesystem", file.Path)
}
}
if cfg.Coreos.Etcd.Discovery_URL != "" {
err := PersistEtcdDiscoveryURL(cfg.Coreos.Etcd.Discovery_URL)
if err == nil {
log.Printf("Consumed etcd discovery url")
} else {
log.Fatalf("Failed to persist etcd discovery url to filesystem: %v", err)
}
}
if len(cfg.Coreos.Units) > 0 {
for _, unit := range cfg.Coreos.Units {
log.Printf("Placing unit %s on filesystem", unit.Name)
dst, err := PlaceUnit("/", &unit)
if err != nil {
return err
}
log.Printf("Placed unit %s at %s", unit.Name, dst)
if unit.Group() != "network" {
log.Printf("Enabling unit file %s", dst)
if err := EnableUnitFile(dst, unit.Runtime); err != nil {
return err
}
log.Printf("Enabled unit %s", unit.Name)
} else {
log.Printf("Skipping enable for network-like unit %s", unit.Name)
}
}
DaemonReload()
StartUnits(cfg.Coreos.Units)
}
if cfg.Coreos.Fleet.Autostart {
err := StartUnitByName("fleet.service")
if err == nil {
log.Printf("Started fleet service.")
} else {
return err
}
}
return nil
}

View File

@@ -1,4 +1,4 @@
package initialize
package cloudinit
import (
"strings"
@@ -7,7 +7,7 @@ import (
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig("")
cfg, err := NewCloudConfig([]byte{})
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
@@ -17,6 +17,14 @@ func TestCloudConfigEmpty(t *testing.T) {
t.Error("Parsed incorrect number of SSH keys")
}
if cfg.Coreos.Etcd.Discovery_URL != "" {
t.Error("Parsed incorrect value of discovery url")
}
if cfg.Coreos.Fleet.Autostart {
t.Error("Expected AutostartFleet not to be defined")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
@@ -28,10 +36,12 @@ func TestCloudConfigEmpty(t *testing.T) {
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := `
contents := []byte(`
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
discovery_url: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
fleet:
autostart: Yes
units:
- name: 50-eth0.network
runtime: yes
@@ -45,12 +55,6 @@ coreos:
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
@@ -62,7 +66,7 @@ write_files:
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`
`)
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
@@ -77,6 +81,14 @@ hostname: trontastic
t.Error("Expected first SSH key to be 'foobaz'")
}
if cfg.Coreos.Etcd.Discovery_URL != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
t.Error("Failed to parse etcd discovery url")
}
if !cfg.Coreos.Fleet.Autostart {
t.Error("Expected AutostartFleet to be true")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
@@ -87,8 +99,8 @@ hostname: trontastic
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if perm, _ := wf.Permissions(); perm != 0644 {
t.Errorf("WriteFile has incorrect permissions %s", perm)
if wf.Permissions != "0644" {
t.Errorf("WriteFile has incorrect permissions %s", wf.Permissions)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
@@ -122,10 +134,6 @@ Address=10.209.171.177/19
}
}
if cfg.Coreos.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
@@ -133,10 +141,10 @@ Address=10.209.171.177/19
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := `
contents := []byte(`
ssh_authorized_keys:
- foo: bar
`
`)
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
@@ -149,7 +157,7 @@ ssh_authorized_keys:
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig("")
cfg, _ := NewCloudConfig([]byte{})
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
@@ -158,7 +166,7 @@ func TestCloudConfigSerializationHeader(t *testing.T) {
}
func TestCloudConfigUsers(t *testing.T) {
contents := `
contents := []byte(`
users:
- name: elroy
passwd: somehash
@@ -174,7 +182,7 @@ users:
no-user-group: true
system: y
no-log-init: True
`
`)
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
@@ -215,7 +223,7 @@ users:
t.Errorf("Failed to parse no-create-home field")
}
if user.PrimaryGroup != "things" {
if user.PrimaryGroup != "things"{
t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
}

25
cloudinit/etcd.go Normal file
View File

@@ -0,0 +1,25 @@
package cloudinit
import (
"io/ioutil"
"log"
"os"
"path"
)
const (
etcdDiscoveryPath = "/var/run/etcd/bootstrap.disco"
)
func PersistEtcdDiscoveryURL(url string) error {
dir := path.Dir(etcdDiscoveryPath)
if _, err := os.Stat(dir); err != nil {
log.Printf("Creating directory /var/run/etcd")
err := os.MkdirAll(dir, os.FileMode(0644))
if err != nil {
return err
}
}
return ioutil.WriteFile(etcdDiscoveryPath, []byte(url), os.FileMode(0644))
}

View File

@@ -1,4 +1,4 @@
package datasource
package cloudinit
import (
"io/ioutil"
@@ -14,7 +14,7 @@ func NewMetadataService(url string) *metadataService {
return &metadataService{url, http.Client{}}
}
func (ms *metadataService) Fetch() ([]byte, error) {
func (ms *metadataService) UserData() ([]byte, error) {
resp, err := ms.client.Get(ms.url)
if err != nil {
return []byte{}, err
@@ -33,6 +33,4 @@ func (ms *metadataService) Fetch() ([]byte, error) {
return respBytes, nil
}
func (ms *metadataService) Type() string {
return "metadata-service"
}

View File

@@ -1,4 +1,4 @@
package system
package cloudinit
import (
"fmt"

View File

@@ -1,4 +1,4 @@
package system
package cloudinit
import (
"fmt"
@@ -13,15 +13,10 @@ import (
"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
)
// fakeMachineID is placed on non-usr CoreOS images and should
// never be used as a true MachineID
const fakeMachineID = "42000000000000000000000000000042"
type Unit struct {
Name string
Runtime bool
Content string
Command string
}
func (u *Unit) Type() string {
@@ -41,7 +36,7 @@ func (u *Unit) Group() (group string) {
type Script []byte
func PlaceUnit(u *Unit, root string) (string, error) {
func PlaceUnit(root string, u *Unit) (string, error) {
dir := "etc"
if u.Runtime {
dir = "run"
@@ -55,14 +50,7 @@ func PlaceUnit(u *Unit, root string) (string, error) {
}
dst = path.Join(dst, u.Name)
file := File{
Path: dst,
Content: u.Content,
RawFilePermissions: "0644",
}
err := WriteFile(&file)
err := ioutil.WriteFile(dst, []byte(u.Content), os.FileMode(0644))
if err != nil {
return "", err
}
@@ -81,33 +69,34 @@ func EnableUnitFile(file string, runtime bool) error {
return err
}
func RunUnitCommand(command, unit string) (string, error) {
conn, err := dbus.New()
if err != nil {
return "", err
func separateNetworkUnits(units []Unit) ([]Unit, []Unit) {
networkUnits := make([]Unit, 0)
nonNetworkUnits := make([]Unit, 0)
for _, unit := range units {
if unit.Group() == "network" {
networkUnits = append(networkUnits, unit)
} else {
nonNetworkUnits = append(nonNetworkUnits, unit)
}
}
return networkUnits, nonNetworkUnits
}
func StartUnits(units []Unit) error {
networkUnits, nonNetworkUnits := separateNetworkUnits(units)
if len(networkUnits) > 0 {
if err := RestartUnitByName("systemd-networkd.service"); err != nil {
return err
}
}
var fn func(string, string) (string, error)
switch command {
case "start":
fn = conn.StartUnit
case "stop":
fn = conn.StopUnit
case "restart":
fn = conn.RestartUnit
case "reload":
fn = conn.ReloadUnit
case "try-restart":
fn = conn.TryRestartUnit
case "reload-or-restart":
fn = conn.ReloadOrRestartUnit
case "reload-or-try-restart":
fn = conn.ReloadOrTryRestartUnit
default:
return "", fmt.Errorf("Unsupported systemd command %q", command)
for _, unit := range nonNetworkUnits {
if err := RestartUnitByName(unit.Name); err != nil {
return err
}
}
return fn(unit, "replace")
return nil
}
func DaemonReload() error {
@@ -120,6 +109,29 @@ func DaemonReload() error {
return err
}
func RestartUnitByName(name string) error {
log.Printf("Restarting unit %s", name)
conn, err := dbus.New()
if err != nil {
return err
}
output, err := conn.RestartUnit(name, "replace")
log.Printf("Restart completed with '%s'", output)
return err
}
func StartUnitByName(name string) error {
conn, err := dbus.New()
if err != nil {
return err
}
_, err = conn.StartUnit(name, "replace")
return err
}
func ExecuteScript(scriptPath string) (string, error) {
props := []dbus.Property{
dbus.PropDescription("Unit generated and executed by coreos-cloudinit on behalf of user"),
@@ -143,18 +155,3 @@ func ExecuteScript(scriptPath string) (string, error) {
func SetHostname(hostname string) error {
return exec.Command("hostnamectl", "set-hostname", hostname).Run()
}
func Hostname() (string, error) {
return os.Hostname()
}
func MachineID(root string) string {
contents, _ := ioutil.ReadFile(path.Join(root, "etc", "machine-id"))
id := strings.TrimSpace(string(contents))
if id == fakeMachineID {
id = ""
}
return id
}

View File

@@ -1,4 +1,4 @@
package system
package cloudinit
import (
"io/ioutil"
@@ -26,7 +26,7 @@ Address=10.209.171.177/19
}
defer syscall.Rmdir(dir)
if _, err := PlaceUnit(&u, dir); err != nil {
if _, err := PlaceUnit(dir, &u); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
@@ -72,7 +72,7 @@ Where=/media/state
}
defer syscall.Rmdir(dir)
if _, err := PlaceUnit(&u, dir); err != nil {
if _, err := PlaceUnit(dir, &u); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
@@ -100,17 +100,3 @@ Where=/media/state
}
}
func TestMachineID(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007\n"), os.FileMode(0444))
if MachineID(dir) != "node007" {
t.Fatalf("File has incorrect contents")
}
}

View File

@@ -1,4 +1,4 @@
package system
package cloudinit
import (
"fmt"
@@ -9,18 +9,17 @@ import (
)
type User struct {
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
SSHImportGithubUser string `yaml:"coreos-ssh-import-github"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
}
func UserExists(u *User) bool {

30
cloudinit/user_data.go Normal file
View File

@@ -0,0 +1,30 @@
package cloudinit
import (
"bufio"
"bytes"
"fmt"
"log"
"strings"
)
func ParseUserData(contents []byte) (interface{}, error) {
bytereader := bytes.NewReader(contents)
bufreader := bufio.NewReader(bytereader)
header, _ := bufreader.ReadString('\n')
if strings.HasPrefix(header, "#!") {
log.Printf("Parsing user-data as script")
return Script(contents), nil
} else if header == "#cloud-config\n" {
log.Printf("Parsing user-data as cloud-config")
cfg, err := NewCloudConfig(contents)
if err != nil {
log.Fatal(err.Error())
}
return *cfg, nil
} else {
return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
}
}

66
cloudinit/workspace.go Normal file
View File

@@ -0,0 +1,66 @@
package cloudinit
import (
"fmt"
"io/ioutil"
"os"
"path"
)
func PrepWorkspace(workspace string) error {
// Ensure workspace exists and is a directory
info, err := os.Stat(workspace)
if err == nil {
if !info.IsDir() {
return fmt.Errorf("%s is not a directory", workspace)
}
} else {
err = os.MkdirAll(workspace, 0755)
if err != nil {
return err
}
}
// Ensure scripts dir in workspace exists and is a directory
scripts := path.Join(workspace, "scripts")
info, err = os.Stat(scripts)
if err == nil {
if !info.IsDir() {
return fmt.Errorf("%s is not a directory", scripts)
}
} else {
err = os.Mkdir(scripts, 0755)
if err != nil {
return err
}
}
return nil
}
func PersistScriptInWorkspace(script Script, workspace string) (string, error) {
scriptsDir := path.Join(workspace, "scripts")
f, err := ioutil.TempFile(scriptsDir, "")
if err != nil {
return "", err
}
defer f.Close()
f.Chmod(0744)
_, err = f.Write(script)
if err != nil {
return "", err
}
// Ensure script has been written to disk before returning, as the
// next natural thing to do is execute it
f.Sync()
return f.Name(), nil
}
func PersistScriptUnitNameInWorkspace(name string, workspace string) error {
unitPath := path.Join(workspace, "scripts", "unit-name")
return ioutil.WriteFile(unitPath, []byte(name), 0644)
}

46
cloudinit/write_file.go Normal file
View File

@@ -0,0 +1,46 @@
package cloudinit
import (
"errors"
"io/ioutil"
"os"
"os/exec"
"path"
"strconv"
)
type WriteFile struct {
Encoding string
Content string
Owner string
Path string
Permissions string
}
func ProcessWriteFile(base string, wf *WriteFile) error {
fullPath := path.Join(base, wf.Path)
if err := os.MkdirAll(path.Dir(fullPath), os.FileMode(0744)); err != nil {
return err
}
// Parse string representation of file mode as octal
perm, err := strconv.ParseInt(wf.Permissions, 8, 32)
if err != nil {
return errors.New("Unable to parse file permissions as octal integer")
}
if err := ioutil.WriteFile(fullPath, []byte(wf.Content), os.FileMode(perm)); err != nil {
return err
}
if wf.Owner != "" {
// We shell out since we don't have a way to look up unix groups natively
cmd := exec.Command("chown", wf.Owner, fullPath)
if err := cmd.Run(); err != nil {
return err
}
}
return nil
}

View File

@@ -1,4 +1,4 @@
package system
package cloudinit
import (
"io/ioutil"
@@ -9,24 +9,23 @@ import (
)
func TestWriteFileUnencodedContent(t *testing.T) {
wf := WriteFile{
Path: "/tmp/foo",
Content: "bar",
Permissions: "0644",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
fullPath := path.Join(dir, "tmp", "foo")
wf := File{
Path: fullPath,
Content: "bar",
RawFilePermissions: "0644",
}
if err := WriteFile(&wf); err != nil {
if err := ProcessWriteFile(dir, &wf); err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
}
fullPath := path.Join(dir, "tmp", "foo")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
@@ -47,65 +46,36 @@ func TestWriteFileUnencodedContent(t *testing.T) {
}
func TestWriteFileInvalidPermission(t *testing.T) {
wf := WriteFile{
Path: "/tmp/foo",
Content: "bar",
Permissions: "pants",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
wf := File{
Path: path.Join(dir, "tmp", "foo"),
Content: "bar",
RawFilePermissions: "pants",
}
if err := WriteFile(&wf); err == nil {
if err := ProcessWriteFile(dir, &wf); err == nil {
t.Fatalf("Expected error to be raised when writing file with invalid permission")
}
}
func TestWriteFilePermissions(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
fullPath := path.Join(dir, "tmp", "foo")
wf := File{
Path: fullPath,
RawFilePermissions: "0755",
}
if err := WriteFile(&wf); err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
}
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0755) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
}
func TestWriteFileEncodedContent(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
wf := File{
Path: path.Join(dir, "tmp", "foo"),
wf := WriteFile{
Path: "/tmp/foo",
Content: "",
Encoding: "base64",
}
if err := WriteFile(&wf); err == nil {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if err := ProcessWriteFile(dir, &wf); err == nil {
t.Fatalf("Expected error to be raised when writing file with encoding")
}
}

View File

@@ -1,26 +1,24 @@
package main
import (
"flag"
"fmt"
"log"
"flag"
"io/ioutil"
"os"
"strings"
"log"
"github.com/coreos/coreos-cloudinit/datasource"
"github.com/coreos/coreos-cloudinit/initialize"
"github.com/coreos/coreos-cloudinit/system"
"github.com/coreos/coreos-cloudinit/cloudinit"
)
const version = "0.3.0"
const version = "0.1.2+git"
func main() {
var userdata []byte
var err error
var printVersion bool
flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
var ignoreFailure bool
flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
var file string
flag.StringVar(&file, "from-file", "", "Read user-data from provided file")
@@ -31,7 +29,7 @@ func main() {
flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
var sshKeyName string
flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
flag.StringVar(&sshKeyName, "ssh-key-name", cloudinit.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
flag.Parse()
@@ -45,62 +43,49 @@ func main() {
os.Exit(1)
}
var ds datasource.Datasource
if file != "" {
ds = datasource.NewLocalFile(file)
log.Printf("Reading user-data from file: %s", file)
userdata, err = ioutil.ReadFile(file)
if err != nil {
log.Fatal(err.Error())
}
} else if url != "" {
ds = datasource.NewMetadataService(url)
log.Printf("Reading user-data from metadata service")
svc := cloudinit.NewMetadataService(url)
userdata, err = svc.UserData()
if err != nil {
log.Fatal(err.Error())
}
} else {
fmt.Println("Provide one of --from-file or --from-url")
os.Exit(1)
}
log.Printf("Fetching user-data from datasource of type %q", ds.Type())
userdataBytes, err := ds.Fetch()
if err != nil {
log.Printf("Failed fetching user-data from datasource: %v", err)
if ignoreFailure {
os.Exit(0)
} else {
os.Exit(1)
}
}
if len(userdataBytes) == 0 {
if len(userdata) == 0 {
log.Printf("No user data to handle, exiting.")
os.Exit(0)
}
env := initialize.NewEnvironment("/", workspace)
userdata := string(userdataBytes)
userdata = env.Apply(userdata)
parsed, err := ParseUserData(userdata)
parsed, err := cloudinit.ParseUserData(userdata)
if err != nil {
log.Printf("Failed parsing user-data: %v", err)
if ignoreFailure {
os.Exit(0)
} else {
os.Exit(1)
}
log.Fatalf("Failed parsing user-data: %v", err)
}
err = initialize.PrepWorkspace(env.Workspace())
err = cloudinit.PrepWorkspace(workspace)
if err != nil {
log.Fatalf("Failed preparing workspace: %v", err)
}
switch t := parsed.(type) {
case initialize.CloudConfig:
err = initialize.Apply(t, env)
case system.Script:
case cloudinit.CloudConfig:
err = cloudinit.ApplyCloudConfig(t, sshKeyName)
case cloudinit.Script:
var path string
path, err = initialize.PersistScriptInWorkspace(t, env.Workspace())
path, err = cloudinit.PersistScriptInWorkspace(t, workspace)
if err == nil {
var name string
name, err = system.ExecuteScript(path)
initialize.PersistUnitNameInWorkspace(name, workspace)
name, err = cloudinit.ExecuteScript(path)
cloudinit.PersistScriptUnitNameInWorkspace(name, workspace)
}
}
@@ -108,22 +93,3 @@ func main() {
log.Fatalf("Failed resolving user-data: %v", err)
}
}
func ParseUserData(contents string) (interface{}, error) {
header := strings.SplitN(contents, "\n", 2)[0]
if strings.HasPrefix(header, "#!") {
log.Printf("Parsing user-data as script")
return system.Script(contents), nil
} else if header == "#cloud-config" {
log.Printf("Parsing user-data as cloud-config")
cfg, err := initialize.NewCloudConfig(contents)
if err != nil {
log.Fatal(err.Error())
}
return *cfg, nil
} else {
return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
}
}

View File

@@ -1,6 +0,0 @@
package datasource
type Datasource interface {
Fetch() ([]byte, error)
Type() string
}

View File

@@ -1,21 +0,0 @@
package datasource
import (
"io/ioutil"
)
type localFile struct {
path string
}
func NewLocalFile(path string) *localFile {
return &localFile{path}
}
func (self *localFile) Fetch() ([]byte, error) {
return ioutil.ReadFile(self.path)
}
func (self *localFile) Type() string {
return "local-file"
}

View File

@@ -1,169 +0,0 @@
package initialize
import (
"fmt"
"log"
"path"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
"github.com/coreos/coreos-cloudinit/system"
)
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd EtcdEnvironment
Units []system.Unit
OEM OEMRelease
}
WriteFiles []system.File `yaml:"write_files"`
Hostname string
Users []system.User
}
func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig
err := goyaml.Unmarshal([]byte(contents), &cfg)
return &cfg, err
}
func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
func Apply(cfg CloudConfig, env *Environment) error {
if cfg.Hostname != "" {
if err := system.SetHostname(cfg.Hostname); err != nil {
return err
}
log.Printf("Set hostname to %s", cfg.Hostname)
}
if cfg.Coreos.OEM.ID != "" {
if err := WriteOEMRelease(&cfg.Coreos.OEM, env.Root()); err != nil {
return err
}
log.Printf("Wrote /etc/oem-release to filesystem")
}
if len(cfg.Users) > 0 {
for _, user := range cfg.Users {
if user.Name == "" {
log.Printf("User object has no 'name' field, skipping")
continue
}
if system.UserExists(&user) {
log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
if user.PasswordHash != "" {
log.Printf("Setting '%s' user's password", user.Name)
if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
return err
}
}
} else {
log.Printf("Creating user '%s'", user.Name)
if err := system.CreateUser(&user); err != nil {
log.Printf("Failed creating user '%s': %v", user.Name, err)
return err
}
}
if len(user.SSHAuthorizedKeys) > 0 {
log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
if err := system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
return err
}
}
if user.SSHImportGithubUser != "" {
log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", user.SSHImportGithubUser, user.Name)
if err := SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
return err
}
}
}
}
if len(cfg.SSHAuthorizedKeys) > 0 {
err := system.AuthorizeSSHKeys("core", env.SSHKeyName(), cfg.SSHAuthorizedKeys)
if err == nil {
log.Printf("Authorized SSH keys for core user")
} else {
return err
}
}
if len(cfg.WriteFiles) > 0 {
for _, file := range cfg.WriteFiles {
file.Path = path.Join(env.Root(), file.Path)
if err := system.WriteFile(&file); err != nil {
return err
}
log.Printf("Wrote file %s to filesystem", file.Path)
}
}
if len(cfg.Coreos.Etcd) > 0 {
if err := WriteEtcdEnvironment(cfg.Coreos.Etcd, env.Root()); err != nil {
log.Fatalf("Failed to write etcd config to filesystem: %v", err)
}
log.Printf("Wrote etcd config file to filesystem")
}
if len(cfg.Coreos.Units) > 0 {
commands := make(map[string]string, 0)
for _, unit := range cfg.Coreos.Units {
if unit.Content != "" {
log.Printf("Writing unit %s to filesystem", unit.Name)
dst, err := system.PlaceUnit(&unit, env.Root())
if err != nil {
return err
}
log.Printf("Placed unit %s at %s", unit.Name, dst)
if unit.Group() != "network" {
log.Printf("Enabling unit file %s", dst)
if err := system.EnableUnitFile(dst, unit.Runtime); err != nil {
return err
}
log.Printf("Enabled unit %s", unit.Name)
} else {
log.Printf("Skipping enable for network-like unit %s", unit.Name)
}
}
if unit.Group() == "network" {
commands["systemd-networkd.service"] = "restart"
} else {
if unit.Command != "" {
commands[unit.Name] = unit.Command
}
}
}
system.DaemonReload()
for unit, command := range commands {
log.Printf("Calling unit command '%s %s'", command, unit)
res, err := system.RunUnitCommand(command, unit)
if err != nil {
return err
}
log.Printf("Result of '%s %s': %s", command, unit, res)
}
}
return nil
}

View File

@@ -1,47 +0,0 @@
package initialize
import (
"os"
"path"
"strings"
)
const DefaultSSHKeyName = "coreos-cloudinit"
type Environment struct {
root string
workspace string
sshKeyName string
substitutions map[string]string
}
func NewEnvironment(root, workspace string) *Environment {
substitutions := map[string]string{
"$public_ipv4": os.Getenv("COREOS_PUBLIC_IPV4"),
"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
}
return &Environment{root, workspace, DefaultSSHKeyName, substitutions}
}
func (self *Environment) Workspace() string {
return path.Join(self.root, self.workspace)
}
func (self *Environment) Root() string {
return self.root
}
func (self *Environment) SSHKeyName() string {
return self.sshKeyName
}
func (self *Environment) SetSSHKeyName(name string) {
self.sshKeyName = name
}
func (self *Environment) Apply(data string) string {
for key, val := range self.substitutions {
data = strings.Replace(data, key, val, -1)
}
return data
}

View File

@@ -1,27 +0,0 @@
package initialize
import (
"os"
"testing"
)
func TestEnvironmentApply(t *testing.T) {
os.Setenv("COREOS_PUBLIC_IPV4", "192.0.2.3")
os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.203")
env := NewEnvironment("./", "./")
input := `[Service]
ExecStart=/usr/bin/echo "$public_ipv4"
ExecStop=/usr/bin/echo $private_ipv4
ExecStop=/usr/bin/echo $unknown
`
expected := `[Service]
ExecStart=/usr/bin/echo "192.0.2.3"
ExecStop=/usr/bin/echo 192.0.2.203
ExecStop=/usr/bin/echo $unknown
`
output := env.Apply(input)
if output != expected {
t.Fatalf("Environment incorrectly applied.\nOutput:\n%s\nExpected:\n%s", output, expected)
}
}

View File

@@ -1,62 +0,0 @@
package initialize
import (
"errors"
"fmt"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
type EtcdEnvironment map[string]string
func (ec EtcdEnvironment) normalized() map[string]string {
out := make(map[string]string, len(ec))
for key, val := range ec {
key = strings.ToUpper(key)
key = strings.Replace(key, "-", "_", -1)
out[key] = val
}
return out
}
func (ec EtcdEnvironment) String() (out string) {
norm := ec.normalized()
if val, ok := norm["DISCOVERY_URL"]; ok {
delete(norm, "DISCOVERY_URL")
if _, ok := norm["DISCOVERY"]; !ok {
norm["DISCOVERY"] = val
}
}
out += "[Service]\n"
for key, val := range norm {
out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
}
return
}
// Write an EtcdEnvironment to the appropriate path on disk for etcd.service
func WriteEtcdEnvironment(env EtcdEnvironment, root string) error {
if _, ok := env["name"]; !ok {
if machineID := system.MachineID(root); machineID != "" {
env["name"] = machineID
} else if hostname, err := system.Hostname(); err == nil {
env["name"] = hostname
} else {
return errors.New("Unable to determine default etcd name")
}
}
file := system.File{
Path: path.Join(root, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf"),
RawFilePermissions: "0644",
Content: env.String(),
}
return system.WriteFile(&file)
}

View File

@@ -1,139 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"os/exec"
"path"
"syscall"
"testing"
)
func TestEtcdEnvironment(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryURLTranslated(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryOverridesDiscoveryURL(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "ping"
cfg["discovery"] = "pong"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=pong"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
ec := EtcdEnvironment{
"name": "node001",
"discovery": "http://disco.example.com/foobar",
"peer-bind-addr": "127.0.0.1:7002",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if err := WriteEtcdEnvironment(ec, dir); err != nil {
t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
ec := EtcdEnvironment{}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
if err != nil {
t.Fatalf("Failed writing out /etc/machine-id: %v", err)
}
if err := WriteEtcdEnvironment(ec, dir); err != nil {
t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_NAME=node007"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func rmdir(path string) error {
cmd := exec.Command("rm", "-rf", path)
return cmd.Run()
}

View File

@@ -1,52 +0,0 @@
package initialize
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"github.com/coreos/coreos-cloudinit/system"
)
type GithubUserKey struct {
Id int `json:"id"`
Key string `json:"key"`
}
func fetchGithubKeys(github_url string) ([]string, error) {
res, err := http.Get(github_url)
defer res.Body.Close()
if err != nil {
return nil, err
}
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
var data []GithubUserKey
err = json.Unmarshal(body, &data)
if err != nil {
return nil, err
}
keys := make([]string, 0)
for _, key := range data {
keys = append(keys, key.Key)
}
return keys, err
}
func SSHImportGithubUser(system_user string, github_user string) error {
url := fmt.Sprintf("https://api.github.com/users/%s/keys", github_user)
keys, err := fetchGithubKeys(url)
if err != nil {
return err
}
key_name := fmt.Sprintf("github-%s", github_user)
err = system.AuthorizeSSHKeys(system_user, key_name, keys)
if err != nil {
return err
}
return nil
}

View File

@@ -1,71 +0,0 @@
package initialize
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
)
func TestCloudConfigUsersGithubMarshal(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gh_res := `
[
{
"id": 67057,
"key": "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
},
{
"id": 3340477,
"key": "ssh-dss AAAAB3NzaC1kc3MAAACBANxpzIbTzKTeBRaOIdUxwwGwvDasTfU/PonhbNIuhYjc+xFGvBRTumox2F+luVAKKs4WdvA4nJXaY1OFi6DZftk5Bp4E2JaSzp8ulAzHsMexDdv6LGHGEJj/qdHAL1vHk2K89PpwRFSRZI8XRBLjvkr4ZgBKLG5ZILXPJEPP2j3lAAAAFQCtxoTnV8wy0c4grcGrQ+1sCsD7WQAAAIAqZsW2GviMe1RQrbZT0xAZmI64XRPrnLsoLxycHWlS7r6uUln2c6Ae2MB/YF0d4Kd1XZii9GHj7rrypqEo7MW8uSabhu70nmu1J8m2O3Dsr+4oJLeat9vwPsJV92IKO0jQwjKnAOHOiB9JKGeCw+NfXfogbti9/q38Q6XcS+SI5wAAAIEA1803Y2h+tOOpZXAsNIwl9mRfExWzLQ3L7knwJdznQu/6SW1H/1oyoYLebuk187Qj2UFI5qQ6AZNc49DvohWx0Cg6ABcyubNyoaCjZKWIdxVnItHWNbLe//+tyTu0I2eQwJOORsEPK5gMpf599C7wXQ//DzZOWbTWiHEX52gCTmk="
},
{
"id": 5224438,
"key": "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
}
]
`
fmt.Fprintln(w, gh_res)
}))
defer ts.Close()
keys, err := fetchGithubKeys(ts.URL)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
expected := "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
if keys[0] != expected {
t.Fatalf("expected %s, got %s", expected, keys[0])
}
expected = "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
if keys[2] != expected {
t.Fatalf("expected %s, got %s", expected, keys[2])
}
}
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}

View File

@@ -1,39 +0,0 @@
package initialize
import (
"fmt"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
type OEMRelease struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version-id"`
HomeURL string `yaml:"home-url"`
BugReportURL string `yaml:"bug-report-url"`
}
func (oem *OEMRelease) String() string {
fields := []string{
fmt.Sprintf("ID=%q", oem.ID),
fmt.Sprintf("NAME=%q", oem.Name),
fmt.Sprintf("VERSION_ID=%q", oem.VersionID),
fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
}
return strings.Join(fields, "\n") + "\n"
}
func WriteOEMRelease(oem *OEMRelease, root string) error {
file := system.File{
Path: path.Join(root, "etc", "oem-release"),
RawFilePermissions: "0644",
Content: oem.String(),
}
return system.WriteFile(&file)
}

View File

@@ -1,54 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"syscall"
"testing"
)
func TestOEMReleaseWrittenToDisk(t *testing.T) {
oem := OEMRelease{
ID: "rackspace",
Name: "Rackspace Cloud Servers",
VersionID: "168.0.0",
HomeURL: "https://www.rackspace.com/cloud/servers/",
BugReportURL: "https://github.com/coreos/coreos-overlay",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if err := WriteOEMRelease(&oem, dir); err != nil {
t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "etc", "oem-release")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `ID="rackspace"
NAME="Rackspace Cloud Servers"
VERSION_ID="168.0.0"
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

View File

@@ -1,48 +0,0 @@
package initialize
import (
"io/ioutil"
"path"
"github.com/coreos/coreos-cloudinit/system"
)
func PrepWorkspace(workspace string) error {
if err := system.EnsureDirectoryExists(workspace); err != nil {
return err
}
scripts := path.Join(workspace, "scripts")
if err := system.EnsureDirectoryExists(scripts); err != nil {
return err
}
return nil
}
func PersistScriptInWorkspace(script system.Script, workspace string) (string, error) {
scriptsPath := path.Join(workspace, "scripts")
tmp, err := ioutil.TempFile(scriptsPath, "")
if err != nil {
return "", err
}
tmp.Close()
file := system.File{
Path: tmp.Name(),
RawFilePermissions: "0744",
Content: string(script),
}
err = system.WriteFile(&file)
return file.Path, err
}
func PersistUnitNameInWorkspace(name string, workspace string) error {
file := system.File{
Path: path.Join(workspace, "scripts", "unit-name"),
RawFilePermissions: "0644",
Content: name,
}
return system.WriteFile(&file)
}

View File

@@ -1,77 +0,0 @@
package system
import (
"errors"
"fmt"
"io/ioutil"
"os"
"os/exec"
"path"
"strconv"
)
type File struct {
Encoding string
Content string
Owner string
Path string
RawFilePermissions string `yaml:"permissions"`
}
func (f *File) Permissions() (os.FileMode, error) {
if f.RawFilePermissions == "" {
return os.FileMode(0644), nil
}
// Parse string representation of file mode as octal
perm, err := strconv.ParseInt(f.RawFilePermissions, 8, 32)
if err != nil {
return 0, errors.New("Unable to parse file permissions as octal integer")
}
return os.FileMode(perm), nil
}
func WriteFile(f *File) error {
if f.Encoding != "" {
return fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
}
if err := os.MkdirAll(path.Dir(f.Path), os.FileMode(0755)); err != nil {
return err
}
perm, err := f.Permissions()
if err != nil {
return err
}
if err := ioutil.WriteFile(f.Path, []byte(f.Content), perm); err != nil {
return err
}
if f.Owner != "" {
// We shell out since we don't have a way to look up unix groups natively
cmd := exec.Command("chown", f.Owner, f.Path)
if err := cmd.Run(); err != nil {
return err
}
}
return nil
}
func EnsureDirectoryExists(dir string) error {
info, err := os.Stat(dir)
if err == nil {
if !info.IsDir() {
return fmt.Errorf("%s is not a directory", dir)
}
} else {
err = os.MkdirAll(dir, 0755)
if err != nil {
return err
}
}
return nil
}

6
test
View File

@@ -4,7 +4,5 @@ echo "Building bin/coreos-cloudinit"
. build
echo "Running tests..."
for pkg in "./initialize ./system"; do
go test -i $pkg
go test -v $pkg
done
go test -i github.com/coreos/coreos-cloudinit/cloudinit
go test -v github.com/coreos/coreos-cloudinit/cloudinit