chore(release): Bump version to v0.5.0

feat(proc-cmdline): Parse /proc/cmdline for cloud-config-url

Documentation/cloud-config.md

@@ -1,147 +1,128 @@
-# Customize CoreOS with Cloud-Config
+# Using Cloud-Config
 
-CoreOS allows you to configure machine parameters, launch systemd units on startup and more. Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS.
+CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. 
 
+## Configuration File
+
+The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
+
+We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
+
+[cloud-init]: https://launchpad.net/cloud-init
+[cloud-init-docs]: http://cloudinit.readthedocs.org/en/latest/index.html
 [cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
 
-## Supported cloud-config Parameters
+### File Format
 
-### ssh_authorized_keys
+The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values.
 
-Provided public SSH keys will be authorized for the `core` user.
+A cloud-config file should contain an associative array which has zero or more of the following keys:
 
-The keys will be named "coreos-cloudinit" by default.
-Override this with the `--ssh-key-name` flag when calling `coreos-cloudinit`.
+- `coreos`
+- `ssh_authorized_keys`
+- `hostname`
+- `users`
+- `write_files`
+- `manage_etc_hosts`
 
-### hostname
+The expected values for these keys are defined in the rest of this document.
 
-The provided value will be used to set the system's hostname.
-This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
+[yaml]: https://en.wikipedia.org/wiki/YAML
 
-### users
+### Providing Cloud-Config with Config-Drive
 
-Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields.
-Each field is optional and of type string unless otherwise noted.
-All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
+CoreOS tries to conform to each platform's native method to provide user data. Each cloud provider tends to be unique, but this complexity has been abstracted by CoreOS. You can view each platform's instructions on their documentation pages. The most universal way to provide cloud-config is [via config-drive](https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/config-drive.md), which attaches a read-only device to the machine, that contains your cloud-config file.
 
-- **name**: Required. Login name of user
-- **gecos**: GECOS comment of user
-- **passwd**: Hash of the password to use for this user
-- **homedir**: User's home directory. Defaults to /home/<name>
-- **no-create-home**: Boolean. Skip home directory createion.
-- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
-- **groups**: Add user to these additional groups
-- **no-user-group**: Boolean. Skip default group creation.
-- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
-- **system**: Create the user as a system user. No home directory will be created.
-- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
+## Configuration Parameters
 
-The following fields are not yet implemented:
+### coreos
 
-- **inactive**: Deactivate the user upon creation
-- **lock-passwd**: Boolean. Disable password login for user
-- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
-- **selinux-user**: Corresponding SELinux user
-- **ssh-import-id**: Import SSH keys by ID from Launchpad.
+#### etcd
 
-##### Generating a password hash
-
-Generating a safe hash is important to the security of your system.  Currently with updated tools like [oclhashcat](http://hashcat.net/oclhashcat/) simplified hashes like md5crypt are trivial to crack on modern GPU hardware.  You can generate a "safer" hash (read: not safe, never publish your hashes publicly) via:
-
-###### On Debian/Ubuntu (via the package "whois")
-    mkpasswd --method=SHA-512 --rounds=4096
-
-###### With OpenSSL (note: this will only make md5crypt.  While better than plantext it should not be considered fully secure)
-    openssl passwd -1
-
-###### With Python (change password and salt values)
-    python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
-
-###### With Perl (change password and salt values)
-    perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
-
-Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed.  On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms. 
-
-### write_files
-
-Inject an arbitrary set of files to the local filesystem.
-Provide a list of objects with the following attributes:
-
-- **path**: Absolute location on disk where contents should be written
-- **content**: Data to write at the provided `path`
-- **permissions**: String representing file permissions in octal notation (i.e. '0644')
-- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
-
-## Custom cloud-config Parameters
-
-### coreos.oem
-
-These fields are borrowed from the [os-release spec][os-release] and repurposed
-as a way for cloud-init to know about the OEM partition on this machine.
-
-- **id**: A lower case string identifying the oem.
-- **version-id**: A lower case string identifying the version of the OEM. Example: `168.0.0`
-- **name**: A name without the version that is suitable for presentation to the user.
-- **home-url**: Link to the homepage of the provider or OEM.
-- **bug-report-url***: Link to a place to file bug reports about this OEM partition.
-
-cloudinit must render these fields down to an /etc/oem-release file on disk in the following format:
+The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
+We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
+
+```
+#cloud-config
+
+coreos:
+    etcd:
+        name: node001
+	# generate a new token for each unique cluster from https://discovery.etcd.io/new
+        discovery: https://discovery.etcd.io/<token>
+	# multi-region and multi-cloud deployments need to use $public_ipv4
+        addr: $public_ipv4:4001
+        peer-addr: $private_ipv4:7001
+```
+
+...will generate a systemd unit drop-in like this:
+
+```
+[Service]
+Environment="ETCD_NAME=node001"
+Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
+Environment="ETCD_ADDR=203.0.113.29:4001"
+Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
+```
+
+For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
+Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
+
+[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
+
+#### oem
+
+The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine:
+
+- **id**: Lowercase string identifying the OEM
+- **name**: Human-friendly string representing the OEM
+- **version-id**: Lowercase string identifying the version of the OEM
+- **home-url**: Link to the homepage of the provider or OEM
+- **bug-report-url**: Link to a place to file bug reports about this OEM
+
+coreos-cloudinit renders these fields to `/etc/oem-release`.
+If no **id** field is provided, coreos-cloudinit will ignore this section.
+
+For example, the following cloud-config document...
+
+```
+#cloud-config
+coreos:
+  oem:
+    id: rackspace
+    name: Rackspace Cloud Servers
+    version-id: 168.0.0
+    home-url: https://www.rackspace.com/cloud/servers/
+    bug-report-url: https://github.com/coreos/coreos-overlay
+```
+
+...would be rendered to the following `/etc/oem-release`:
 
 ```
-NAME=Rackspace
 ID=rackspace
+NAME="Rackspace Cloud Servers"
 VERSION_ID=168.0.0
-PRETTY_NAME="Rackspace Cloud Servers"
-HOME_URL="http://www.rackspace.com/cloud/servers/"
+HOME_URL="https://www.rackspace.com/cloud/servers/"
 BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
 ```
 
 [os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html
 
-### coreos.etcd.discovery_url
+#### units
 
-The value of `coreos.etcd.discovery_url` will be used to discover the instance's etcd peers using the [etcd discovery protocol][disco-proto]. Usage of the [public discovery service][disco-service] is encouraged.
+The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields:
 
-[disco-proto]: https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md
-[disco-service]: http://discovery.etcd.io
+- **name**: String representing unit's name. Required.
+- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` argument to `systemd enable`. Default value is false.
+- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false.
+- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
+- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
 
-### coreos.units
+**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
 
-Arbitrary systemd units may be provided in the `coreos.units` attribute.
-`coreos.units` is a list of objects with the following fields:
+##### Examples
 
-- **name**: string representing unit's name
-- **runtime**: boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` flag to `systemd enable`.
-- **content**: plaintext string representing entire unit file
-
-See docker example below.
-
-## user-data Script
-
-Simply set your user-data to a script where the first line is a shebang:
-
-```
-#!/bin/bash
-
-echo 'Hello, world!'
-```
-
-## Examples
-
-### Inject an SSH key, bootstrap etcd, and start fleet
-```
-#cloud-config
-
-coreos:
-	etcd:
-		discovery_url: https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877
-    fleet:
-        autostart: yes
-ssh_authorized_keys:
-  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
-```
-
-### Start a docker container on boot
+Write a unit to disk, automatically starting it.
 
 ```
 #cloud-config
@@ -164,33 +145,162 @@ coreos:
           WantedBy=local.target
 ```
 
-### Add a user
+Start the builtin `etcd` and `fleet` services:
+
+```
+# cloud-config
+
+coreos:
+    units:
+      - name: etcd.service
+        command: start
+      - name: fleet.service
+        command: start
+```
+
+### ssh_authorized_keys
+
+The `ssh_authorized_keys` parameter adds public SSH keys which will be authorized for the `core` user.
+
+The keys will be named "coreos-cloudinit" by default.
+Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`.
+
+```
+#cloud-config
+
+ssh_authorized_keys:
+  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
+```
+
+### hostname
+
+The `hostname` parameter defines the system's hostname.
+This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
+
+```
+#cloud-config
+
+hostname: coreos1
+```
+
+### users
+
+The `users` parameter adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted.
+All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
+
+- **name**: Required. Login name of user
+- **gecos**: GECOS comment of user
+- **passwd**: Hash of the password to use for this user
+- **homedir**: User's home directory. Defaults to /home/<name>
+- **no-create-home**: Boolean. Skip home directory creation.
+- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
+- **groups**: Add user to these additional groups
+- **no-user-group**: Boolean. Skip default group creation.
+- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
+- **coreos-ssh-import-github**: Authorize SSH keys from Github user
+- **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint.
+- **system**: Create the user as a system user. No home directory will be created.
+- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
+
+The following fields are not yet implemented:
+
+- **inactive**: Deactivate the user upon creation
+- **lock-passwd**: Boolean. Disable password login for user
+- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
+- **selinux-user**: Corresponding SELinux user
+- **ssh-import-id**: Import SSH keys by ID from Launchpad.
 
 ```
 #cloud-config
 
 users:
   - name: elroy
-	passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
-	groups:
-	  - staff
-	  - docker
-	ssh-authorized-keys:
-	  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
+    passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
+    groups:
+      - sudo
+      - docker
+    ssh-authorized-keys:
+      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
 ```
 
-### Inject configuration files
+#### Generating a password hash
+
+If you choose to use a password instead of an SSH key, generating a safe hash is extremely important to the security of your system. Simplified hashes like md5crypt are trivial to crack on modern GPU hardware. Here are a few ways to generate secure hashes:
+
+```
+# On Debian/Ubuntu (via the package "whois")
+mkpasswd --method=SHA-512 --rounds=4096
+
+# OpenSSL (note: this will only make md5crypt.  While better than plantext it should not be considered fully secure)
+openssl passwd -1
+
+# Python (change password and salt values)
+python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
+
+# Perl (change password and salt values)
+perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
+```
+
+Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed.  On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms. 
+
+#### Retrieving SSH Authorized Keys
+
+##### From a GitHub User
+
+Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server.
 
 ```
 #cloud-config
 
-write_files:
-  - path: /etc/hosts
-    contents: |
-      127.0.0.1    localhost
-      192.0.2.211  buildbox
-  - path: /etc/resolv.conf
-    contents: |
-      nameserver 192.0.2.13
-      nameserver 192.0.2.14
+users:
+  - name: elroy
+    coreos-ssh-import-github: elroy
+```
+
+##### From an HTTP Endpoint
+
+We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
+For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token:
+
+```
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://token:<OAUTH-TOKEN>@github-enterprise.example.com/users/elroy/keys
+```
+
+You can also specify any URL whose response matches the JSON format for public keys:
+
+```
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://example.com/public-keys
+```
+
+### write_files
+
+The `write-file` parameter defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys:
+
+- **path**: Absolute location on disk where contents should be written
+- **content**: Data to write at the provided `path`
+- **permissions**: String representing file permissions in octal notation (i.e. '0644')
+- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
+
+Explicitly not implemented is the **encoding** attribute.
+The **content** field must represent exactly what should be written to disk.
+
+### manage_etc_hosts
+
+The `manage_etc_hosts` parameter configures the contents of the `/etc/hosts` file, which is used for local name resolution.
+Currently, the only supported value is "localhost" which will cause your system's hostname
+to resolve to "127.0.0.1".  This is helpful when the host does not have DNS
+infrastructure in place to resolve its own hostname, for example, when using Vagrant.
+
+```
+#cloud-config
+
+manage_etc_hosts: localhost
 ```

Documentation/config-drive.md

@@ -0,0 +1,30 @@
+# Distribution via Config Drive
+
+CoreOS supports providing configuration data via [config drive][config-drive]
+disk images. Currently only providing a single script or cloud config file is
+supported.
+
+[config-drive]: http://docs.openstack.org/user-guide/content/enable_config_drive.html#config_drive_contents
+
+## Contents and Format
+
+The image should be a single FAT or ISO9660 file system with the label
+`config-2` and the configuration data should be located at
+`openstack/latest/user_data`.
+
+For example, to wrap up a config named `user_data` in a config drive image:
+
+    mkdir -p /tmp/new-drive/openstack/latest
+    cp user_data /tmp/new-drive/openstack/latest/user_data
+    mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
+    rm -r /tmp/new-drive
+
+## QEMU virtfs
+
+One exception to the above, when using QEMU it is possible to skip creating an
+image and use a plain directory containing the same contents:
+
+    qemu-system-x86_64 \
+        -fsdev local,id=conf,security_model=none,readonly,path=/tmp/new-drive \
+        -device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \
+        [usual qemu options here...]

README.md

@@ -1,9 +1,79 @@
 # coreos-cloudinit
 
-coreos-cloudinit allows a user to customize CoreOS machines by providing either an executable script or a cloud-config document as instance user-data. See below to learn how to use these features.
+coreos-cloudinit enables a user to customize CoreOS machines by providing either a cloud-config document or an executable script through user-data.
 
-## Supported Cloud-Config Features
+## Configuration with cloud-config
 
-Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS, which are [documented here](https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md).
+A subset of the [official cloud-config spec][official-cloud-config] is implemented by coreos-cloudinit.
+Additionally, several [CoreOS-specific options][custom-cloud-config] have been implemented to support interacting with unit files, bootstrapping etcd clusters, and more.
+All supported cloud-config parameters are [documented here][all-cloud-config]. 
 
-[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
+[official-cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
+[custom-cloud-config]: https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/cloud-config.md#coreos-parameters
+[all-cloud-config]: https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md
+
+The following is an example cloud-config document:
+
+```
+#cloud-config
+
+coreos:
+    units:
+      - name: etcd.service
+        command: start
+
+users:
+  - name: core
+    passwd: $1$allJZawX$00S5T756I5PGdQga5qhqv1
+
+write_files:
+  - path: /etc/resolv.conf
+    content: |
+        nameserver 192.0.2.2
+        nameserver 192.0.2.3
+```
+
+## Executing a Script
+
+coreos-cloudinit supports executing user-data as a script instead of parsing it as a cloud-config document.
+Make sure the first line of your user-data is a shebang and coreos-cloudinit will attempt to execute it:
+
+```
+#!/bin/bash
+
+echo 'Hello, world!'
+```
+
+## user-data Field Substitution
+
+coreos-cloudinit will replace the following set of tokens in your user-data with system-generated values.
+
+| Token         | Description |
+| ------------- | ----------- |
+| $public_ipv4  | Public IPv4 address of machine |
+| $private_ipv4 | Private IPv4 address of machine |
+
+These values are determined by CoreOS based on the given provider on which your machine is running.
+Read more about provider-specific functionality in the [CoreOS OEM documentation][oem-doc].
+
+[oem-doc]: https://coreos.com/docs/sdk-distributors/distributors/notes-for-distributors/
+
+For example, submitting the following user-data...
+
+```
+#cloud-config
+coreos:
+    etcd:
+        addr: $public_ipv4:4001
+        peer-addr: $private_ipv4:7001
+```
+
+...will result in this cloud-config document being executed:
+
+```
+#cloud-config
+coreos:
+    etcd:
+        addr: 203.0.113.29:4001
+        peer-addr: 192.0.2.13:7001
+```

cloudinit/cloud_config.go

@@ -1,143 +0,0 @@
-package cloudinit
-
-import (
-	"fmt"
-	"log"
-
-	"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
-)
-
-const DefaultSSHKeyName = "coreos-cloudinit"
-
-type CloudConfig struct {
-	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
-	Coreos            struct {
-		Etcd  struct{ Discovery_URL string }
-		Fleet struct{ Autostart bool }
-		Units []Unit
-	}
-	WriteFiles []WriteFile `yaml:"write_files"`
-	Hostname   string
-	Users      []User
-}
-
-func NewCloudConfig(contents []byte) (*CloudConfig, error) {
-	var cfg CloudConfig
-	err := goyaml.Unmarshal(contents, &cfg)
-	return &cfg, err
-}
-
-func (cc CloudConfig) String() string {
-	bytes, err := goyaml.Marshal(cc)
-	if err != nil {
-		return ""
-	}
-
-	stringified := string(bytes)
-	stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
-
-	return stringified
-}
-
-func ApplyCloudConfig(cfg CloudConfig, sshKeyName string) error {
-	if cfg.Hostname != "" {
-		if err := SetHostname(cfg.Hostname); err != nil {
-			return err
-		}
-		log.Printf("Set hostname to %s", cfg.Hostname)
-	}
-
-	if len(cfg.Users) > 0 {
-		for _, user := range cfg.Users {
-			if user.Name == "" {
-				log.Printf("User object has no 'name' field, skipping")
-				continue
-			}
-
-			if UserExists(&user) {
-				log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
-				if user.PasswordHash != "" {
-					log.Printf("Setting '%s' user's password", user.Name)
-					if err := SetUserPassword(user.Name, user.PasswordHash); err != nil {
-						log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
-						return err
-					}
-				}
-			} else {
-				log.Printf("Creating user '%s'", user.Name)
-				if err := CreateUser(&user); err != nil {
-					log.Printf("Failed creating user '%s': %v", user.Name, err)
-					return err
-				}
-			}
-
-			if len(user.SSHAuthorizedKeys) > 0 {
-				log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
-				if err := AuthorizeSSHKeys(user.Name, sshKeyName, user.SSHAuthorizedKeys); err != nil {
-					return err
-				}
-			}
-		}
-	}
-
-	if len(cfg.SSHAuthorizedKeys) > 0 {
-		err := AuthorizeSSHKeys("core", sshKeyName, cfg.SSHAuthorizedKeys)
-		if err == nil {
-			log.Printf("Authorized SSH keys for core user")
-		} else {
-			return err
-		}
-	}
-
-	if len(cfg.WriteFiles) > 0 {
-		for _, file := range cfg.WriteFiles {
-			if err := ProcessWriteFile("/", &file); err != nil {
-				return err
-			}
-			log.Printf("Wrote file %s to filesystem", file.Path)
-		}
-	}
-
-	if cfg.Coreos.Etcd.Discovery_URL != "" {
-		err := PersistEtcdDiscoveryURL(cfg.Coreos.Etcd.Discovery_URL)
-		if err == nil {
-			log.Printf("Consumed etcd discovery url")
-		} else {
-			log.Fatalf("Failed to persist etcd discovery url to filesystem: %v", err)
-		}
-	}
-
-	if len(cfg.Coreos.Units) > 0 {
-		for _, unit := range cfg.Coreos.Units {
-			log.Printf("Placing unit %s on filesystem", unit.Name)
-			dst, err := PlaceUnit("/", &unit)
-			if err != nil {
-				return err
-			}
-			log.Printf("Placed unit %s at %s", unit.Name, dst)
-
-			if unit.Group() != "network" {
-				log.Printf("Enabling unit file %s", dst)
-				if err := EnableUnitFile(dst, unit.Runtime); err != nil {
-					return err
-				}
-				log.Printf("Enabled unit %s", unit.Name)
-			} else {
-				log.Printf("Skipping enable for network-like unit %s", unit.Name)
-			}
-		}
-		DaemonReload()
-		StartUnits(cfg.Coreos.Units)
-	}
-
-	if cfg.Coreos.Fleet.Autostart {
-		err := StartUnitByName("fleet.service")
-		if err == nil {
-			log.Printf("Started fleet service.")
-		} else {
-			return err
-		}
-	}
-
-	return nil
-}

cloudinit/etcd.go

@@ -1,25 +0,0 @@
-package cloudinit
-
-import (
-	"io/ioutil"
-	"log"
-	"os"
-	"path"
-)
-
-const (
-	etcdDiscoveryPath = "/var/run/etcd/bootstrap.disco"
-)
-
-func PersistEtcdDiscoveryURL(url string) error {
-	dir := path.Dir(etcdDiscoveryPath)
-	if _, err := os.Stat(dir); err != nil {
-		log.Printf("Creating directory /var/run/etcd")
-		err := os.MkdirAll(dir, os.FileMode(0644))
-		if err != nil {
-			return err
-		}
-	}
-
-	return ioutil.WriteFile(etcdDiscoveryPath, []byte(url), os.FileMode(0644))
-}

cloudinit/metadata_service.go

@@ -1,36 +0,0 @@
-package cloudinit
-
-import (
-	"io/ioutil"
-	"net/http"
-)
-
-type metadataService struct {
-	url    string
-	client http.Client
-}
-
-func NewMetadataService(url string) *metadataService {
-	return &metadataService{url, http.Client{}}
-}
-
-func (ms *metadataService) UserData() ([]byte, error) {
-	resp, err := ms.client.Get(ms.url)
-	if err != nil {
-		return []byte{}, err
-	}
-	defer resp.Body.Close()
-
-	if resp.StatusCode / 100 != 2 {
-		return []byte{}, nil
-	}
-
-	respBytes, err := ioutil.ReadAll(resp.Body)
-	if err != nil {
-		return nil, err
-	}
-
-	return respBytes, nil
-}
-
-

cloudinit/systemd.go

@@ -1,157 +0,0 @@
-package cloudinit
-
-import (
-	"fmt"
-	"io/ioutil"
-	"log"
-	"os"
-	"os/exec"
-	"path"
-	"path/filepath"
-	"strings"
-
-	"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
-)
-
-type Unit struct {
-	Name    string
-	Runtime bool
-	Content string
-}
-
-func (u *Unit) Type() string {
-	ext := filepath.Ext(u.Name)
-	return strings.TrimLeft(ext, ".")
-}
-
-func (u *Unit) Group() (group string) {
-	t := u.Type()
-	if t == "network" || t == "netdev" || t == "link" {
-		group = "network"
-	} else {
-		group = "system"
-	}
-	return
-}
-
-type Script []byte
-
-func PlaceUnit(root string, u *Unit) (string, error) {
-	dir := "etc"
-	if u.Runtime {
-		dir = "run"
-	}
-
-	dst := path.Join(root, dir, "systemd", u.Group())
-	if _, err := os.Stat(dst); os.IsNotExist(err) {
-		if err := os.MkdirAll(dst, os.FileMode(0755)); err != nil {
-			return "", err
-		}
-	}
-
-	dst = path.Join(dst, u.Name)
-	err := ioutil.WriteFile(dst, []byte(u.Content), os.FileMode(0644))
-	if err != nil {
-		return "", err
-	}
-
-	return dst, nil
-}
-
-func EnableUnitFile(file string, runtime bool) error {
-	conn, err := dbus.New()
-	if err != nil {
-		return err
-	}
-
-	files := []string{file}
-	_, _, err = conn.EnableUnitFiles(files, runtime, true)
-	return err
-}
-
-func separateNetworkUnits(units []Unit) ([]Unit, []Unit) {
-	networkUnits := make([]Unit, 0)
-	nonNetworkUnits := make([]Unit, 0)
-	for _, unit := range units {
-		if unit.Group() == "network" {
-			networkUnits = append(networkUnits, unit)
-		} else {
-			nonNetworkUnits = append(nonNetworkUnits, unit)
-		}
-	}
-	return networkUnits, nonNetworkUnits
-}
-
-func StartUnits(units []Unit) error {
-	networkUnits, nonNetworkUnits := separateNetworkUnits(units)
-	if len(networkUnits) > 0 {
-		if err := RestartUnitByName("systemd-networkd.service"); err != nil {
-			return err
-		}
-	}
-
-	for _, unit := range nonNetworkUnits {
-		if err := RestartUnitByName(unit.Name); err != nil {
-			return err
-		}
-	}
-
-	return nil
-}
-
-func DaemonReload() error {
-	conn, err := dbus.New()
-	if err != nil {
-		return err
-	}
-
-	_, err = conn.Reload()
-	return err
-}
-
-func RestartUnitByName(name string) error {
-	log.Printf("Restarting unit %s", name)
-	conn, err := dbus.New()
-	if err != nil {
-		return err
-	}
-
-	output, err := conn.RestartUnit(name, "replace")
-	log.Printf("Restart completed with '%s'", output)
-
-	return err
-}
-
-func StartUnitByName(name string) error {
-	conn, err := dbus.New()
-	if err != nil {
-		return err
-	}
-
-	_, err = conn.StartUnit(name, "replace")
-	return err
-}
-
-func ExecuteScript(scriptPath string) (string, error) {
-	props := []dbus.Property{
-		dbus.PropDescription("Unit generated and executed by coreos-cloudinit on behalf of user"),
-		dbus.PropExecStart([]string{"/bin/bash", scriptPath}, false),
-	}
-
-	base := path.Base(scriptPath)
-	name := fmt.Sprintf("coreos-cloudinit-%s.service", base)
-
-	log.Printf("Creating transient systemd unit '%s'", name)
-
-	conn, err := dbus.New()
-	if err != nil {
-		return "", err
-	}
-
-	_, err = conn.StartTransientUnit(name, "replace", props...)
-	return name, err
-}
-
-func SetHostname(hostname string) error {
-	return exec.Command("hostnamectl", "set-hostname", hostname).Run()
-}

cloudinit/user_data.go

@@ -1,30 +0,0 @@
-package cloudinit
-
-import (
-	"bufio"
-	"bytes"
-	"fmt"
-	"log"
-	"strings"
-)
-
-func ParseUserData(contents []byte) (interface{}, error) {
-	bytereader := bytes.NewReader(contents)
-	bufreader := bufio.NewReader(bytereader)
-	header, _ := bufreader.ReadString('\n')
-
-	if strings.HasPrefix(header, "#!") {
-		log.Printf("Parsing user-data as script")
-		return Script(contents), nil
-
-	} else if header == "#cloud-config\n" {
-		log.Printf("Parsing user-data as cloud-config")
-		cfg, err := NewCloudConfig(contents)
-		if err != nil {
-			log.Fatal(err.Error())
-		}
-		return *cfg, nil
-	} else {
-		return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
-	}
-}

cloudinit/workspace.go

@@ -1,66 +0,0 @@
-package cloudinit
-
-import (
-	"fmt"
-	"io/ioutil"
-	"os"
-	"path"
-)
-
-func PrepWorkspace(workspace string) error {
-	// Ensure workspace exists and is a directory
-	info, err := os.Stat(workspace)
-	if err == nil {
-		if !info.IsDir() {
-			return fmt.Errorf("%s is not a directory", workspace)
-		}
-	} else {
-		err = os.MkdirAll(workspace, 0755)
-		if err != nil {
-			return err
-		}
-	}
-
-	// Ensure scripts dir in workspace exists and is a directory
-	scripts := path.Join(workspace, "scripts")
-	info, err = os.Stat(scripts)
-	if err == nil {
-		if !info.IsDir() {
-			return fmt.Errorf("%s is not a directory", scripts)
-		}
-	} else {
-		err = os.Mkdir(scripts, 0755)
-		if err != nil {
-			return err
-		}
-	}
-
-	return nil
-}
-
-func PersistScriptInWorkspace(script Script, workspace string) (string, error) {
-	scriptsDir := path.Join(workspace, "scripts")
-	f, err := ioutil.TempFile(scriptsDir, "")
-	if err != nil {
-		return "", err
-	}
-	defer f.Close()
-
-	f.Chmod(0744)
-
-	_, err = f.Write(script)
-	if err != nil {
-		return "", err
-	}
-
-	// Ensure script has been written to disk before returning, as the
-	// next natural thing to do is execute it
-	f.Sync()
-
-	return f.Name(), nil
-}
-
-func PersistScriptUnitNameInWorkspace(name string, workspace string) error {
-	unitPath := path.Join(workspace, "scripts", "unit-name")
-	return ioutil.WriteFile(unitPath, []byte(name), 0644)
-}

cloudinit/write_file.go

@@ -1,46 +0,0 @@
-package cloudinit
-
-import (
-	"errors"
-	"io/ioutil"
-	"os"
-	"os/exec"
-	"path"
-	"strconv"
-)
-
-type WriteFile struct {
-	Encoding    string
-	Content     string
-	Owner       string
-	Path        string
-	Permissions string
-}
-
-func ProcessWriteFile(base string, wf *WriteFile) error {
-	fullPath := path.Join(base, wf.Path)
-
-	if err := os.MkdirAll(path.Dir(fullPath), os.FileMode(0744)); err != nil {
-		return err
-	}
-
-	// Parse string representation of file mode as octal
-	perm, err := strconv.ParseInt(wf.Permissions, 8, 32)
-	if err != nil {
-		return errors.New("Unable to parse file permissions as octal integer")
-	}
-
-	if err := ioutil.WriteFile(fullPath, []byte(wf.Content), os.FileMode(perm)); err != nil {
-		return err
-	}
-
-	if wf.Owner != "" {
-		// We shell out since we don't have a way to look up unix groups natively
-		cmd := exec.Command("chown", wf.Owner, fullPath)
-		if err := cmd.Run(); err != nil {
-			return err
-		}
-	}
-
-	return nil
-}

coreos-cloudinit.go

@@ -1,35 +1,39 @@
 package main
 
 import (
-	"fmt"
 	"flag"
-	"io/ioutil"
-	"os"
+	"fmt"
 	"log"
+	"os"
 
-	"github.com/coreos/coreos-cloudinit/cloudinit"
+	"github.com/coreos/coreos-cloudinit/datasource"
+	"github.com/coreos/coreos-cloudinit/initialize"
+	"github.com/coreos/coreos-cloudinit/system"
 )
 
-const version = "0.1.2+git"
+const version = "0.5.0"
 
 func main() {
-	var userdata []byte
-	var err error
-
 	var printVersion bool
 	flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
 
+	var ignoreFailure bool
+	flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
+
 	var file string
 	flag.StringVar(&file, "from-file", "", "Read user-data from provided file")
 
 	var url string
 	flag.StringVar(&url, "from-url", "", "Download user-data from provided url")
 
+	var useProcCmdline bool
+	flag.BoolVar(&useProcCmdline, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", datasource.ProcCmdlineLocation, datasource.ProcCmdlineCloudConfigFlag))
+
 	var workspace string
 	flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
 
 	var sshKeyName string
-	flag.StringVar(&sshKeyName, "ssh-key-name", cloudinit.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
+	flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
 
 	flag.Parse()
 
@@ -38,54 +42,69 @@ func main() {
 		os.Exit(0)
 	}
 
-	if file != "" && url != "" {
-		fmt.Println("Provide one of --from-file or --from-url")
+	if file != "" && url != "" && !useProcCmdline {
+		fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
 		os.Exit(1)
 	}
 
+	var ds datasource.Datasource
 	if file != "" {
-		log.Printf("Reading user-data from file: %s", file)
-		userdata, err = ioutil.ReadFile(file)
-		if err != nil {
-			log.Fatal(err.Error())
-		}
+		ds = datasource.NewLocalFile(file)
 	} else if url != "" {
-		log.Printf("Reading user-data from metadata service")
-		svc := cloudinit.NewMetadataService(url)
-		userdata, err = svc.UserData()
-		if err != nil {
-			log.Fatal(err.Error())
-		}
+		ds = datasource.NewMetadataService(url)
+	} else if useProcCmdline {
+		ds = datasource.NewProcCmdline()
 	} else {
-		fmt.Println("Provide one of --from-file or --from-url")
+		fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
 		os.Exit(1)
 	}
 
-	if len(userdata) == 0 {
+	log.Printf("Fetching user-data from datasource of type %q", ds.Type())
+	userdataBytes, err := ds.Fetch()
+	if err != nil {
+		log.Printf("Failed fetching user-data from datasource: %v", err)
+		if ignoreFailure {
+			os.Exit(0)
+		} else {
+			os.Exit(1)
+		}
+	}
+
+	if len(userdataBytes) == 0 {
 		log.Printf("No user data to handle, exiting.")
 		os.Exit(0)
 	}
 
-	parsed, err := cloudinit.ParseUserData(userdata)
+	env := initialize.NewEnvironment("/", workspace)
+
+	userdata := string(userdataBytes)
+	userdata = env.Apply(userdata)
+
+	parsed, err := initialize.ParseUserData(userdata)
 	if err != nil {
-		log.Fatalf("Failed parsing user-data: %v", err)
+		log.Printf("Failed parsing user-data: %v", err)
+		if ignoreFailure {
+			os.Exit(0)
+		} else {
+			os.Exit(1)
+		}
 	}
 
-	err = cloudinit.PrepWorkspace(workspace)
+	err = initialize.PrepWorkspace(env.Workspace())
 	if err != nil {
 		log.Fatalf("Failed preparing workspace: %v", err)
 	}
 
 	switch t := parsed.(type) {
-	case cloudinit.CloudConfig:
-		err = cloudinit.ApplyCloudConfig(t, sshKeyName)
-	case cloudinit.Script:
+	case initialize.CloudConfig:
+		err = initialize.Apply(t, env)
+	case system.Script:
 		var path string
-		path, err = cloudinit.PersistScriptInWorkspace(t, workspace)
+		path, err = initialize.PersistScriptInWorkspace(t, env.Workspace())
 		if err == nil {
 			var name string
-			name, err = cloudinit.ExecuteScript(path)
-			cloudinit.PersistScriptUnitNameInWorkspace(name, workspace)
+			name, err = system.ExecuteScript(path)
+			initialize.PersistUnitNameInWorkspace(name, workspace)
 		}
 	}
 
@@ -93,3 +112,4 @@ func main() {
 		log.Fatalf("Failed resolving user-data: %v", err)
 	}
 }
+

datasource/datasource.go

@@ -0,0 +1,31 @@
+package datasource
+
+import (
+	"io/ioutil"
+	"net/http"
+)
+
+type Datasource interface {
+	Fetch() ([]byte, error)
+	Type()  string
+}
+
+func fetchURL(url string) ([]byte, error) {
+	client := http.Client{}
+	resp, err := client.Get(url)
+	if err != nil {
+		return []byte{}, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode / 100 != 2 {
+		return []byte{}, nil
+	}
+
+	respBytes, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return nil, err
+	}
+
+	return respBytes, nil
+}

datasource/file.go

@@ -0,0 +1,21 @@
+package datasource
+
+import (
+	"io/ioutil"
+)
+
+type localFile struct {
+	path string
+}
+
+func NewLocalFile(path string) *localFile {
+	return &localFile{path}
+}
+
+func (self *localFile) Fetch() ([]byte, error) {
+	return ioutil.ReadFile(self.path)
+}
+
+func (self *localFile) Type() string {
+	return "local-file"
+}

datasource/metadata_service.go

@@ -0,0 +1,17 @@
+package datasource
+
+type metadataService struct {
+	url string
+}
+
+func NewMetadataService(url string) *metadataService {
+	return &metadataService{url}
+}
+
+func (ms *metadataService) Fetch() ([]byte, error) {
+	return fetchURL(ms.url)
+}
+
+func (ms *metadataService) Type() string {
+	return "metadata-service"
+}

datasource/proc_cmdline.go

@@ -0,0 +1,66 @@
+package datasource
+
+import (
+	"errors"
+	"io/ioutil"
+	"log"
+	"strings"
+)
+
+const (
+	ProcCmdlineLocation        = "/proc/cmdline"
+	ProcCmdlineCloudConfigFlag = "cloud-config-url"
+)
+
+type procCmdline struct{}
+
+func NewProcCmdline() *procCmdline {
+	return &procCmdline{}
+}
+
+func (self *procCmdline) Fetch() ([]byte, error) {
+	cmdline, err := ioutil.ReadFile(ProcCmdlineLocation)
+	if err != nil {
+		return nil, err
+	}
+
+	url, err := findCloudConfigURL(string(cmdline))
+	if err != nil {
+		return nil, err
+	}
+
+	cfg, err := fetchURL(url)
+	if err != nil {
+		return nil, err
+	}
+
+	return cfg, nil
+}
+
+func (self *procCmdline) Type() string {
+	return "proc-cmdline"
+}
+
+func findCloudConfigURL(input string) (url string, err error) {
+	err = errors.New("cloud-config-url not found")
+	for _, token := range strings.Split(input, " ") {
+		parts := strings.SplitN(token, "=", 2)
+
+		key := parts[0]
+		key = strings.Replace(key, "_", "-", -1)
+
+		if key != "cloud-config-url" {
+			continue
+		}
+
+		if len(parts) != 2 {
+			log.Printf("Found cloud-config-url in /proc/cmdline with no value, ignoring.")
+			continue
+		}
+
+		url = parts[1]
+		err = nil
+	}
+
+	return
+}

datasource/proc_cmdline_test.go

@@ -0,0 +1,47 @@
+package datasource
+
+import (
+	"testing"
+)
+
+func TestParseCmdlineCloudConfigFound(t *testing.T) {
+	tests := []struct {
+		input  string
+		expect string
+	}{
+		{
+			"cloud-config-url=example.com",
+			"example.com",
+		},
+		{
+			"cloud_config_url=example.com",
+			"example.com",
+		},
+		{
+			"cloud-config-url cloud-config-url=example.com",
+			"example.com",
+		},
+		{
+			"cloud-config-url= cloud-config-url=example.com",
+			"example.com",
+		},
+		{
+			"cloud-config-url=one.example.com cloud-config-url=two.example.com",
+			"two.example.com",
+		},
+		{
+			"foo=bar cloud-config-url=example.com ping=pong",
+			"example.com",
+		},
+	}
+
+	for i, tt := range tests {
+		output, err := findCloudConfigURL(tt.input)
+		if output != tt.expect {
+			t.Errorf("Test case %d failed: %s != %s", i, output, tt.expect)
+		}
+		if err != nil {
+			t.Errorf("Test case %d produced error: %v", i, err)
+		}
+	}
+}

initialize/config.go

@@ -0,0 +1,185 @@
+package initialize
+
+import (
+	"fmt"
+	"log"
+	"path"
+
+	"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type CloudConfig struct {
+	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
+	Coreos            struct {
+		Etcd  EtcdEnvironment
+		Units []system.Unit
+		OEM   OEMRelease
+	}
+	WriteFiles     []system.File `yaml:"write_files"`
+	Hostname       string
+	Users          []system.User
+	ManageEtcHosts string `yaml:"manage_etc_hosts"`
+}
+
+func NewCloudConfig(contents string) (*CloudConfig, error) {
+	var cfg CloudConfig
+	err := goyaml.Unmarshal([]byte(contents), &cfg)
+	return &cfg, err
+}
+
+func (cc CloudConfig) String() string {
+	bytes, err := goyaml.Marshal(cc)
+	if err != nil {
+		return ""
+	}
+
+	stringified := string(bytes)
+	stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
+
+	return stringified
+}
+
+func Apply(cfg CloudConfig, env *Environment) error {
+	if cfg.Hostname != "" {
+		if err := system.SetHostname(cfg.Hostname); err != nil {
+			return err
+		}
+		log.Printf("Set hostname to %s", cfg.Hostname)
+	}
+
+	if cfg.Coreos.OEM.ID != "" {
+		if err := WriteOEMRelease(&cfg.Coreos.OEM, env.Root()); err != nil {
+			return err
+		}
+		log.Printf("Wrote /etc/oem-release to filesystem")
+	}
+
+	if len(cfg.Users) > 0 {
+		for _, user := range cfg.Users {
+			if user.Name == "" {
+				log.Printf("User object has no 'name' field, skipping")
+				continue
+			}
+
+			if system.UserExists(&user) {
+				log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
+				if user.PasswordHash != "" {
+					log.Printf("Setting '%s' user's password", user.Name)
+					if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
+						log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
+						return err
+					}
+				}
+			} else {
+				log.Printf("Creating user '%s'", user.Name)
+				if err := system.CreateUser(&user); err != nil {
+					log.Printf("Failed creating user '%s': %v", user.Name, err)
+					return err
+				}
+			}
+
+			if len(user.SSHAuthorizedKeys) > 0 {
+				log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
+				if err := system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
+					return err
+				}
+			}
+			if user.SSHImportGithubUser != "" {
+				log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", user.SSHImportGithubUser, user.Name)
+				if err := SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
+					return err
+				}
+			}
+			if user.SSHImportURL != "" {
+				log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
+				if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
+					return err
+				}
+			}
+		}
+	}
+
+	if len(cfg.SSHAuthorizedKeys) > 0 {
+		err := system.AuthorizeSSHKeys("core", env.SSHKeyName(), cfg.SSHAuthorizedKeys)
+		if err == nil {
+			log.Printf("Authorized SSH keys for core user")
+		} else {
+			return err
+		}
+	}
+
+	if len(cfg.WriteFiles) > 0 {
+		for _, file := range cfg.WriteFiles {
+			file.Path = path.Join(env.Root(), file.Path)
+			if err := system.WriteFile(&file); err != nil {
+				return err
+			}
+			log.Printf("Wrote file %s to filesystem", file.Path)
+		}
+	}
+
+	if len(cfg.Coreos.Etcd) > 0 {
+		if err := WriteEtcdEnvironment(cfg.Coreos.Etcd, env.Root()); err != nil {
+			log.Fatalf("Failed to write etcd config to filesystem: %v", err)
+		}
+
+		log.Printf("Wrote etcd config file to filesystem")
+	}
+
+	if len(cfg.Coreos.Units) > 0 {
+		commands := make(map[string]string, 0)
+		for _, unit := range cfg.Coreos.Units {
+			dst := system.UnitDestination(&unit, env.Root())
+			if unit.Content != "" {
+				log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
+				if err := system.PlaceUnit(&unit, dst); err != nil {
+					return err
+				}
+				log.Printf("Placed unit %s at %s", unit.Name, dst)
+			}
+
+			if unit.Enable {
+				if unit.Group() != "network" {
+					log.Printf("Enabling unit file %s", dst)
+					if err := system.EnableUnitFile(dst, unit.Runtime); err != nil {
+						return err
+					}
+					log.Printf("Enabled unit %s", unit.Name)
+				} else {
+					log.Printf("Skipping enable for network-like unit %s", unit.Name)
+				}
+			}
+
+			if unit.Group() == "network" {
+				commands["systemd-networkd.service"] = "restart"
+			} else {
+				if unit.Command != "" {
+					commands[unit.Name] = unit.Command
+				}
+			}
+		}
+
+		for unit, command := range commands {
+			log.Printf("Calling unit command '%s %s'", command, unit)
+			res, err := system.RunUnitCommand(command, unit)
+			if err != nil {
+				return err
+			}
+			log.Printf("Result of '%s %s': %s", command, unit, res)
+		}
+	}
+
+	if cfg.ManageEtcHosts != "" {
+
+		if err := WriteEtcHosts(cfg.ManageEtcHosts, env.Root()); err != nil {
+			log.Fatalf("Failed to write /etc/hosts to filesystem: %v", err)
+		}
+
+		log.Printf("Wrote /etc/hosts file to filesystem")
+
+	}
+
+	return nil
+}

initialize/config_test.go

@@ -1,4 +1,4 @@
-package cloudinit
+package initialize
 
 import (
 	"strings"
@@ -7,7 +7,7 @@ import (
 
 // Assert that the parsing of a cloud config file "generally works"
 func TestCloudConfigEmpty(t *testing.T) {
-	cfg, err := NewCloudConfig([]byte{})
+	cfg, err := NewCloudConfig("")
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
 	}
@@ -17,14 +17,6 @@ func TestCloudConfigEmpty(t *testing.T) {
 		t.Error("Parsed incorrect number of SSH keys")
 	}
 
-	if cfg.Coreos.Etcd.Discovery_URL != "" {
-		t.Error("Parsed incorrect value of discovery url")
-	}
-
-	if cfg.Coreos.Fleet.Autostart {
-		t.Error("Expected AutostartFleet not to be defined")
-	}
-
 	if len(cfg.WriteFiles) != 0 {
 		t.Error("Expected zero WriteFiles")
 	}
@@ -36,12 +28,10 @@ func TestCloudConfigEmpty(t *testing.T) {
 
 // Assert that the parsing of a cloud config file "generally works"
 func TestCloudConfig(t *testing.T) {
-	contents := []byte(`
+	contents := `
 coreos: 
   etcd:
-    discovery_url: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
-  fleet:
-    autostart: Yes
+    discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
   units:
     - name: 50-eth0.network
       runtime: yes
@@ -55,6 +45,12 @@ coreos:
     Address=10.209.171.177/19
  
 '
+  oem:
+    id: rackspace
+    name: Rackspace Cloud Servers
+    version-id: 168.0.0
+    home-url: https://www.rackspace.com/cloud/servers/
+    bug-report-url: https://github.com/coreos/coreos-overlay
 ssh_authorized_keys:
   - foobar
   - foobaz
@@ -66,7 +62,7 @@ write_files:
     permissions: '0644'
     owner: root:dogepack
 hostname: trontastic
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
@@ -81,14 +77,6 @@ hostname: trontastic
 		t.Error("Expected first SSH key to be 'foobaz'")
 	}
 
-	if cfg.Coreos.Etcd.Discovery_URL != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
-		t.Error("Failed to parse etcd discovery url")
-	}
-
-	if !cfg.Coreos.Fleet.Autostart {
-		t.Error("Expected AutostartFleet to be true")
-	}
-
 	if len(cfg.WriteFiles) != 1 {
 		t.Error("Failed to parse correct number of write_files")
 	} else {
@@ -99,8 +87,8 @@ hostname: trontastic
 		if wf.Encoding != "" {
 			t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
 		}
-		if wf.Permissions != "0644" {
-			t.Errorf("WriteFile has incorrect permissions %s", wf.Permissions)
+		if perm, _ := wf.Permissions(); perm != 0644 {
+			t.Errorf("WriteFile has incorrect permissions %s", perm)
 		}
 		if wf.Path != "/etc/dogepack.conf" {
 			t.Errorf("WriteFile has incorrect path %s", wf.Path)
@@ -134,6 +122,10 @@ Address=10.209.171.177/19
 		}
 	}
 
+	if cfg.Coreos.OEM.ID != "rackspace" {
+		t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
+	}
+
 	if cfg.Hostname != "trontastic" {
 		t.Errorf("Failed to parse hostname")
 	}
@@ -141,10 +133,10 @@ Address=10.209.171.177/19
 
 // Assert that our interface conversion doesn't panic
 func TestCloudConfigKeysNotList(t *testing.T) {
-	contents := []byte(`
+	contents := `
 ssh_authorized_keys:
   - foo: bar
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
@@ -157,7 +149,7 @@ ssh_authorized_keys:
 }
 
 func TestCloudConfigSerializationHeader(t *testing.T) {
-	cfg, _ := NewCloudConfig([]byte{})
+	cfg, _ := NewCloudConfig("")
 	contents := cfg.String()
 	header := strings.SplitN(contents, "\n", 2)[0]
 	if header != "#cloud-config" {
@@ -166,7 +158,7 @@ func TestCloudConfigSerializationHeader(t *testing.T) {
 }
 
 func TestCloudConfigUsers(t *testing.T) {
-	contents := []byte(`
+	contents := `
 users:
   - name: elroy
     passwd: somehash
@@ -182,7 +174,7 @@ users:
     no-user-group: true
     system: y
     no-log-init: True
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error: %v", err)
@@ -223,7 +215,7 @@ users:
 		t.Errorf("Failed to parse no-create-home field")
 	}
 
-	if user.PrimaryGroup != "things"{
+	if user.PrimaryGroup != "things" {
 		t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
 	}
 

initialize/env.go

@@ -0,0 +1,47 @@
+package initialize
+
+import (
+	"os"
+	"path"
+	"strings"
+)
+
+const DefaultSSHKeyName = "coreos-cloudinit"
+
+type Environment struct {
+	root          string
+	workspace     string
+	sshKeyName    string
+	substitutions map[string]string
+}
+
+func NewEnvironment(root, workspace string) *Environment {
+	substitutions := map[string]string{
+		"$public_ipv4":  os.Getenv("COREOS_PUBLIC_IPV4"),
+		"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
+	}
+	return &Environment{root, workspace, DefaultSSHKeyName, substitutions}
+}
+
+func (self *Environment) Workspace() string {
+	return path.Join(self.root, self.workspace)
+}
+
+func (self *Environment) Root() string {
+	return self.root
+}
+
+func (self *Environment) SSHKeyName() string {
+	return self.sshKeyName
+}
+
+func (self *Environment) SetSSHKeyName(name string) {
+	self.sshKeyName = name
+}
+
+func (self *Environment) Apply(data string) string {
+	for key, val := range self.substitutions {
+		data = strings.Replace(data, key, val, -1)
+	}
+	return data
+}

initialize/env_test.go

@@ -0,0 +1,27 @@
+package initialize
+
+import (
+	"os"
+	"testing"
+)
+
+func TestEnvironmentApply(t *testing.T) {
+	os.Setenv("COREOS_PUBLIC_IPV4", "192.0.2.3")
+	os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.203")
+	env := NewEnvironment("./", "./")
+	input := `[Service]
+ExecStart=/usr/bin/echo "$public_ipv4"
+ExecStop=/usr/bin/echo $private_ipv4
+ExecStop=/usr/bin/echo $unknown
+`
+	expected := `[Service]
+ExecStart=/usr/bin/echo "192.0.2.3"
+ExecStop=/usr/bin/echo 192.0.2.203
+ExecStop=/usr/bin/echo $unknown
+`
+
+	output := env.Apply(input)
+	if output != expected {
+		t.Fatalf("Environment incorrectly applied.\nOutput:\n%s\nExpected:\n%s", output, expected)
+	}
+}

initialize/etcd.go

@@ -0,0 +1,62 @@
+package initialize
+
+import (
+	"errors"
+	"fmt"
+	"path"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type EtcdEnvironment map[string]string
+
+func (ec EtcdEnvironment) normalized() map[string]string {
+	out := make(map[string]string, len(ec))
+	for key, val := range ec {
+		key = strings.ToUpper(key)
+		key = strings.Replace(key, "-", "_", -1)
+		out[key] = val
+	}
+	return out
+}
+
+func (ec EtcdEnvironment) String() (out string) {
+	norm := ec.normalized()
+
+	if val, ok := norm["DISCOVERY_URL"]; ok {
+		delete(norm, "DISCOVERY_URL")
+		if _, ok := norm["DISCOVERY"]; !ok {
+			norm["DISCOVERY"] = val
+		}
+	}
+
+	out += "[Service]\n"
+
+	for key, val := range norm {
+		out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
+	}
+
+	return
+}
+
+// Write an EtcdEnvironment to the appropriate path on disk for etcd.service
+func WriteEtcdEnvironment(env EtcdEnvironment, root string) error {
+	if _, ok := env["name"]; !ok {
+		if machineID := system.MachineID(root); machineID != "" {
+			env["name"] = machineID
+		} else if hostname, err := system.Hostname(); err == nil {
+			env["name"] = hostname
+		} else {
+			return errors.New("Unable to determine default etcd name")
+		}
+	}
+
+	file := system.File{
+		Path: path.Join(root, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf"),
+		RawFilePermissions: "0644",
+		Content: env.String(),
+	}
+
+	return system.WriteFile(&file)
+}

initialize/etcd_test.go

@@ -0,0 +1,139 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path"
+	"syscall"
+	"testing"
+)
+
+func TestEtcdEnvironment(t *testing.T) {
+	cfg := make(EtcdEnvironment, 0)
+	cfg["discovery"] = "http://disco.example.com/foobar"
+	cfg["peer-bind-addr"] = "127.0.0.1:7002"
+
+	env := cfg.String()
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestEtcdEnvironmentDiscoveryURLTranslated(t *testing.T) {
+	cfg := make(EtcdEnvironment, 0)
+	cfg["discovery_url"] = "http://disco.example.com/foobar"
+	cfg["peer-bind-addr"] = "127.0.0.1:7002"
+
+	env := cfg.String()
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestEtcdEnvironmentDiscoveryOverridesDiscoveryURL(t *testing.T) {
+	cfg := make(EtcdEnvironment, 0)
+	cfg["discovery_url"] = "ping"
+	cfg["discovery"] = "pong"
+	cfg["peer-bind-addr"] = "127.0.0.1:7002"
+
+	env := cfg.String()
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=pong"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
+	ec := EtcdEnvironment{
+		"name": "node001",
+		"discovery": "http://disco.example.com/foobar",
+		"peer-bind-addr": "127.0.0.1:7002",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if err := WriteEtcdEnvironment(ec, dir); err != nil {
+		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `[Service]
+Environment="ETCD_NAME=node001"
+Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}
+
+func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
+	ec := EtcdEnvironment{}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
+	err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
+	if err != nil {
+		t.Fatalf("Failed writing out /etc/machine-id: %v", err)
+	}
+
+	if err := WriteEtcdEnvironment(ec, dir); err != nil {
+		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `[Service]
+Environment="ETCD_NAME=node007"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}
+
+func rmdir(path string) error {
+    cmd := exec.Command("rm", "-rf", path)
+    return cmd.Run()
+}

initialize/github.go

@@ -0,0 +1,18 @@
+package initialize
+
+import (
+	"fmt"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+func SSHImportGithubUser(system_user string, github_user string) error {
+	url := fmt.Sprintf("https://api.github.com/users/%s/keys", github_user)
+	keys, err := fetchUserKeys(url)
+	if err != nil {
+		return err
+	}
+
+	key_name := fmt.Sprintf("github-%s", github_user)
+	return system.AuthorizeSSHKeys(system_user, key_name, keys)
+}

initialize/github_test.go

@@ -0,0 +1,32 @@
+package initialize
+
+import (
+	"testing"
+)
+
+func TestCloudConfigUsersGithubUser(t *testing.T) {
+
+	contents := `
+users:
+  - name: elroy
+    coreos-ssh-import-github: bcwaldon
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.SSHImportGithubUser != "bcwaldon" {
+		t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
+	}
+}

initialize/manage_etc_hosts.go

@@ -0,0 +1,44 @@
+package initialize
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const DefaultIpv4Address = "127.0.0.1"
+
+func generateEtcHosts(option string) (out string, err error) {
+	if option != "localhost" {
+		return "", errors.New("Invalid option to manage_etc_hosts")
+	}
+
+	// use the operating system hostname
+	hostname, err := os.Hostname()
+	if err != nil {
+		return "", err
+	}
+
+	return fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname), nil
+
+}
+
+// Write an /etc/hosts file
+func WriteEtcHosts(option string, root string) error {
+
+	etcHosts, err := generateEtcHosts(option)
+	if err != nil {
+		return err
+	}
+
+	file := system.File{
+		Path:               path.Join(root, "etc", "hosts"),
+		RawFilePermissions: "0644",
+		Content:            etcHosts,
+	}
+
+	return system.WriteFile(&file)
+}

initialize/manage_etc_hosts_test.go

@@ -0,0 +1,76 @@
+package initialize
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
+	"testing"
+)
+
+func TestCloudConfigManageEtcHosts(t *testing.T) {
+	contents := `
+manage_etc_hosts: localhost
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	manageEtcHosts := cfg.ManageEtcHosts
+
+	if manageEtcHosts != "localhost" {
+		t.Errorf("ManageEtcHosts value is %q, expected 'localhost'", manageEtcHosts)
+	}
+}
+
+func TestManageEtcHostsInvalidValue(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer rmdir(dir)
+
+	if err := WriteEtcHosts("invalid", dir); err == nil {
+		t.Fatalf("WriteEtcHosts succeeded with invalid value: %v", err)
+	}
+}
+
+func TestEtcHostsWrittenToDisk(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer rmdir(dir)
+
+	if err := WriteEtcHosts("localhost", dir); err != nil {
+		t.Fatalf("WriteEtcHosts failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "hosts")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	hostname, err := os.Hostname()
+	if err != nil {
+		t.Fatalf("Unable to read OS hostname: %v", err)
+	}
+
+	expect := fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname)
+
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}

initialize/oem.go

@@ -0,0 +1,39 @@
+package initialize
+
+import (
+	"fmt"
+	"path"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type OEMRelease struct {
+	ID           string `yaml:"id"`
+	Name         string `yaml:"name"`
+	VersionID    string `yaml:"version-id"`
+	HomeURL      string `yaml:"home-url"`
+	BugReportURL string `yaml:"bug-report-url"`
+}
+
+func (oem *OEMRelease) String() string {
+	fields := []string{
+		fmt.Sprintf("ID=%s", oem.ID),
+		fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
+		fmt.Sprintf("NAME=%q", oem.Name),
+		fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
+		fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
+	}
+
+	return strings.Join(fields, "\n") + "\n"
+}
+
+func WriteOEMRelease(oem *OEMRelease, root string) error {
+	file := system.File{
+		Path:               path.Join(root, "etc", "oem-release"),
+		RawFilePermissions: "0644",
+		Content:            oem.String(),
+	}
+
+	return system.WriteFile(&file)
+}

initialize/oem_test.go

@@ -0,0 +1,54 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"syscall"
+	"testing"
+)
+
+func TestOEMReleaseWrittenToDisk(t *testing.T) {
+	oem := OEMRelease{
+		ID:           "rackspace",
+		Name:         "Rackspace Cloud Servers",
+		VersionID:    "168.0.0",
+		HomeURL:      "https://www.rackspace.com/cloud/servers/",
+		BugReportURL: "https://github.com/coreos/coreos-overlay",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if err := WriteOEMRelease(&oem, dir); err != nil {
+		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "oem-release")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `ID=rackspace
+VERSION_ID=168.0.0
+NAME="Rackspace Cloud Servers"
+HOME_URL="https://www.rackspace.com/cloud/servers/"
+BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}

initialize/ssh_keys.go

@@ -0,0 +1,47 @@
+package initialize
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type UserKey struct {
+	ID  int    `json:"id,omitempty"`
+	Key string `json:"key"`
+}
+
+func SSHImportKeysFromURL(system_user string, url string) error {
+	keys, err := fetchUserKeys(url)
+	if err != nil {
+		return err
+	}
+
+	key_name := fmt.Sprintf("coreos-cloudinit-%s", system_user)
+	return system.AuthorizeSSHKeys(system_user, key_name, keys)
+}
+
+func fetchUserKeys(url string) ([]string, error) {
+	res, err := http.Get(url)
+	defer res.Body.Close()
+	if err != nil {
+		return nil, err
+	}
+	body, err := ioutil.ReadAll(res.Body)
+	if err != nil {
+		return nil, err
+	}
+	var data []UserKey
+	err = json.Unmarshal(body, &data)
+	if err != nil {
+		return nil, err
+	}
+	keys := make([]string, 0)
+	for _, key := range data {
+		keys = append(keys, key.Key)
+	}
+	return keys, err
+}

initialize/ssh_keys_test.go

@@ -0,0 +1,69 @@
+package initialize
+
+import (
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+func TestCloudConfigUsersUrlMarshal(t *testing.T) {
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		gh_res := `
+[
+  {
+    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
+  },
+  {
+    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANxpzIbTzKTeBRaOIdUxwwGwvDasTfU/PonhbNIuhYjc+xFGvBRTumox2F+luVAKKs4WdvA4nJXaY1OFi6DZftk5Bp4E2JaSzp8ulAzHsMexDdv6LGHGEJj/qdHAL1vHk2K89PpwRFSRZI8XRBLjvkr4ZgBKLG5ZILXPJEPP2j3lAAAAFQCtxoTnV8wy0c4grcGrQ+1sCsD7WQAAAIAqZsW2GviMe1RQrbZT0xAZmI64XRPrnLsoLxycHWlS7r6uUln2c6Ae2MB/YF0d4Kd1XZii9GHj7rrypqEo7MW8uSabhu70nmu1J8m2O3Dsr+4oJLeat9vwPsJV92IKO0jQwjKnAOHOiB9JKGeCw+NfXfogbti9/q38Q6XcS+SI5wAAAIEA1803Y2h+tOOpZXAsNIwl9mRfExWzLQ3L7knwJdznQu/6SW1H/1oyoYLebuk187Qj2UFI5qQ6AZNc49DvohWx0Cg6ABcyubNyoaCjZKWIdxVnItHWNbLe//+tyTu0I2eQwJOORsEPK5gMpf599C7wXQ//DzZOWbTWiHEX52gCTmk="
+  },
+  {
+    "id": 5224438,
+    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
+  }
+]
+`
+		fmt.Fprintln(w, gh_res)
+	}))
+	defer ts.Close()
+
+	keys, err := fetchUserKeys(ts.URL)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+	expected := "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
+	if keys[0] != expected {
+		t.Fatalf("expected %s, got %s", expected, keys[0])
+	}
+	expected = "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
+	if keys[2] != expected {
+		t.Fatalf("expected %s, got %s", expected, keys[2])
+	}
+
+}
+func TestCloudConfigUsersSSHImportURL(t *testing.T) {
+
+	contents := `
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
+		t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
+	}
+}

initialize/user_data.go

@@ -0,0 +1,33 @@
+package initialize
+
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+func ParseUserData(contents string) (interface{}, error) {
+	header := strings.SplitN(contents, "\n", 2)[0]
+
+	// Explicitly trim the header so we can handle user-data from
+	// non-unix operating systems. The rest of the file is parsed
+	// by goyaml, which correctly handles CRLF.
+	header = strings.TrimSpace(header)
+
+	if strings.HasPrefix(header, "#!") {
+		log.Printf("Parsing user-data as script")
+		return system.Script(contents), nil
+
+	} else if header == "#cloud-config" {
+		log.Printf("Parsing user-data as cloud-config")
+		cfg, err := NewCloudConfig(contents)
+		if err != nil {
+			log.Fatal(err.Error())
+		}
+		return *cfg, nil
+	} else {
+		return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
+	}
+}

initialize/user_data_test.go

@@ -0,0 +1,31 @@
+package initialize
+
+import (
+	"testing"
+)
+
+func TestParseHeaderCRLF(t *testing.T) {
+	configs := []string{
+		"#cloud-config\nfoo: bar",
+		"#cloud-config\r\nfoo: bar",
+	}
+
+	for i, config := range configs {
+		_, err := ParseUserData(config)
+		if err != nil {
+			t.Errorf("Failed parsing config %d: %v", i, err)
+		}
+	}
+
+	scripts := []string{
+		"#!bin/bash\necho foo",
+		"#!bin/bash\r\necho foo",
+	}
+
+	for i, script := range scripts {
+		_, err := ParseUserData(script)
+		if err != nil {
+			t.Errorf("Failed parsing script %d: %v", i, err)
+		}
+	}
+}

initialize/workspace.go

@@ -0,0 +1,48 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"path"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+func PrepWorkspace(workspace string) error {
+	if err := system.EnsureDirectoryExists(workspace); err != nil {
+		return err
+	}
+
+	scripts := path.Join(workspace, "scripts")
+	if err := system.EnsureDirectoryExists(scripts); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func PersistScriptInWorkspace(script system.Script, workspace string) (string, error) {
+	scriptsPath := path.Join(workspace, "scripts")
+	tmp, err := ioutil.TempFile(scriptsPath, "")
+	if err != nil {
+		return "", err
+	}
+	tmp.Close()
+
+	file := system.File{
+		Path: tmp.Name(),
+		RawFilePermissions: "0744",
+		Content: string(script),
+	}
+
+	err = system.WriteFile(&file)
+	return file.Path, err
+}
+
+func PersistUnitNameInWorkspace(name string, workspace string) error {
+	file := system.File{
+		Path: path.Join(workspace, "scripts", "unit-name"),
+		RawFilePermissions: "0644",
+		Content: name,
+	}
+	return system.WriteFile(&file)
+}

system/file.go

@@ -0,0 +1,77 @@
+package system
+
+import (
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path"
+	"strconv"
+)
+
+type File struct {
+	Encoding    string
+	Content     string
+	Owner       string
+	Path        string
+	RawFilePermissions string `yaml:"permissions"`
+}
+
+func (f *File) Permissions() (os.FileMode, error) {
+	if f.RawFilePermissions == "" {
+		return os.FileMode(0644), nil
+	}
+
+	// Parse string representation of file mode as octal
+	perm, err := strconv.ParseInt(f.RawFilePermissions, 8, 32)
+	if err != nil {
+		return 0, errors.New("Unable to parse file permissions as octal integer")
+	}
+	return os.FileMode(perm), nil
+}
+
+
+func WriteFile(f *File) error {
+	if f.Encoding != "" {
+		return fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
+	}
+
+	if err := os.MkdirAll(path.Dir(f.Path), os.FileMode(0755)); err != nil {
+		return err
+	}
+
+	perm, err := f.Permissions()
+	if err != nil {
+		return err
+	}
+
+	if err := ioutil.WriteFile(f.Path, []byte(f.Content), perm); err != nil {
+		return err
+	}
+
+	if f.Owner != "" {
+		// We shell out since we don't have a way to look up unix groups natively
+		cmd := exec.Command("chown", f.Owner, f.Path)
+		if err := cmd.Run(); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func EnsureDirectoryExists(dir string) error {
+	info, err := os.Stat(dir)
+	if err == nil {
+		if !info.IsDir() {
+			return fmt.Errorf("%s is not a directory", dir)
+		}
+	} else {
+		err = os.MkdirAll(dir, 0755)
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}

system/file_test.go

@@ -1,4 +1,4 @@
-package cloudinit
+package system
 
 import (
 	"io/ioutil"
@@ -9,22 +9,23 @@ import (
 )
 
 func TestWriteFileUnencodedContent(t *testing.T) {
-	wf := WriteFile{
-		Path:        "/tmp/foo",
-		Content:     "bar",
-		Permissions: "0644",
-	}
 	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
 	defer syscall.Rmdir(dir)
 
-	if err := ProcessWriteFile(dir, &wf); err != nil {
-		t.Fatalf("Processing of WriteFile failed: %v", err)
+	fullPath := path.Join(dir, "tmp", "foo")
+
+	wf := File{
+		Path:        fullPath,
+		Content:     "bar",
+		RawFilePermissions: "0644",
 	}
 
-	fullPath := path.Join(dir, "tmp", "foo")
+	if err := WriteFile(&wf); err != nil {
+		t.Fatalf("Processing of WriteFile failed: %v", err)
+	}
 
 	fi, err := os.Stat(fullPath)
 	if err != nil {
@@ -46,36 +47,65 @@ func TestWriteFileUnencodedContent(t *testing.T) {
 }
 
 func TestWriteFileInvalidPermission(t *testing.T) {
-	wf := WriteFile{
-		Path:        "/tmp/foo",
-		Content:     "bar",
-		Permissions: "pants",
-	}
 	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
 	defer syscall.Rmdir(dir)
 
-	if err := ProcessWriteFile(dir, &wf); err == nil {
+	wf := File{
+		Path:        path.Join(dir, "tmp", "foo"),
+		Content:     "bar",
+		RawFilePermissions: "pants",
+	}
+
+	if err := WriteFile(&wf); err == nil {
 		t.Fatalf("Expected error to be raised when writing file with invalid permission")
 	}
 }
 
-func TestWriteFileEncodedContent(t *testing.T) {
-	wf := WriteFile{
-		Path: "/tmp/foo",
-		Content: "",
-		Encoding: "base64",
-	}
-
+func TestWriteFilePermissions(t *testing.T) {
 	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
 	defer syscall.Rmdir(dir)
 
-	if err := ProcessWriteFile(dir, &wf); err == nil {
+	fullPath := path.Join(dir, "tmp", "foo")
+
+	wf := File{
+		Path:               fullPath,
+		RawFilePermissions: "0755",
+	}
+
+	if err := WriteFile(&wf); err != nil {
+		t.Fatalf("Processing of WriteFile failed: %v", err)
+	}
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0755) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+}
+
+func TestWriteFileEncodedContent(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	wf := File{
+		Path: path.Join(dir, "tmp", "foo"),
+		Content: "",
+		Encoding: "base64",
+	}
+
+	if err := WriteFile(&wf); err == nil {
 		t.Fatalf("Expected error to be raised when writing file with encoding")
 	}
 }

system/ssh_key.go

@@ -1,4 +1,4 @@
-package cloudinit
+package system
 
 import (
 	"fmt"

system/systemd.go

@@ -0,0 +1,167 @@
+package system
+
+import (
+	"fmt"
+	"io/ioutil"
+	"log"
+	"os"
+	"os/exec"
+	"path"
+	"path/filepath"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
+)
+
+// fakeMachineID is placed on non-usr CoreOS images and should
+// never be used as a true MachineID
+const fakeMachineID = "42000000000000000000000000000042"
+
+type Unit struct {
+	Name    string
+	Enable  bool
+	Runtime bool
+	Content string
+	Command string
+}
+
+func (u *Unit) Type() string {
+	ext := filepath.Ext(u.Name)
+	return strings.TrimLeft(ext, ".")
+}
+
+func (u *Unit) Group() (group string) {
+	t := u.Type()
+	if t == "network" || t == "netdev" || t == "link" {
+		group = "network"
+	} else {
+		group = "system"
+	}
+	return
+}
+
+type Script []byte
+
+// UnitDestination builds the appropriate absolte file path for
+// the given unit. The root argument indicates the effective base
+// directory of the system (similar to a chroot).
+func UnitDestination(u *Unit, root string) string {
+	dir := "etc"
+	if u.Runtime {
+		dir = "run"
+	}
+
+	return path.Join(root, dir, "systemd", u.Group(), u.Name)
+}
+
+// PlaceUnit writes a unit file at the provided destination, creating
+// parent directories as necessary.
+func PlaceUnit(u *Unit, dst string) error {
+	dir := filepath.Dir(dst)
+	if _, err := os.Stat(dir); os.IsNotExist(err) {
+		if err := os.MkdirAll(dir, os.FileMode(0755)); err != nil {
+			return err
+		}
+	}
+
+	file := File{
+		Path: dst,
+		Content: u.Content,
+		RawFilePermissions: "0644",
+	}
+
+	err := WriteFile(&file)
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func EnableUnitFile(file string, runtime bool) error {
+	conn, err := dbus.New()
+	if err != nil {
+		return err
+	}
+
+	files := []string{file}
+	_, _, err = conn.EnableUnitFiles(files, runtime, true)
+	return err
+}
+
+func RunUnitCommand(command, unit string) (string, error) {
+	conn, err := dbus.New()
+	if err != nil {
+		return "", err
+	}
+
+	var fn func(string, string) (string, error)
+	switch command {
+	case "start":
+		fn = conn.StartUnit
+	case "stop":
+		fn = conn.StopUnit
+	case "restart":
+		fn = conn.RestartUnit
+	case "reload":
+		fn = conn.ReloadUnit
+	case "try-restart":
+		fn = conn.TryRestartUnit
+	case "reload-or-restart":
+		fn = conn.ReloadOrRestartUnit
+	case "reload-or-try-restart":
+		fn = conn.ReloadOrTryRestartUnit
+	default:
+		return "", fmt.Errorf("Unsupported systemd command %q", command)
+	}
+
+	return fn(unit, "replace")
+}
+
+func DaemonReload() error {
+	conn, err := dbus.New()
+	if err != nil {
+		return err
+	}
+
+	return conn.Reload()
+}
+
+func ExecuteScript(scriptPath string) (string, error) {
+	props := []dbus.Property{
+		dbus.PropDescription("Unit generated and executed by coreos-cloudinit on behalf of user"),
+		dbus.PropExecStart([]string{"/bin/bash", scriptPath}, false),
+	}
+
+	base := path.Base(scriptPath)
+	name := fmt.Sprintf("coreos-cloudinit-%s.service", base)
+
+	log.Printf("Creating transient systemd unit '%s'", name)
+
+	conn, err := dbus.New()
+	if err != nil {
+		return "", err
+	}
+
+	_, err = conn.StartTransientUnit(name, "replace", props...)
+	return name, err
+}
+
+func SetHostname(hostname string) error {
+	return exec.Command("hostnamectl", "set-hostname", hostname).Run()
+}
+
+func Hostname() (string, error) {
+	return os.Hostname()
+}
+
+func MachineID(root string) string {
+	contents, _ := ioutil.ReadFile(path.Join(root, "etc", "machine-id"))
+	id := strings.TrimSpace(string(contents))
+
+	if id == fakeMachineID {
+		id = ""
+	}
+
+	return id
+}

system/systemd_test.go

@@ -1,4 +1,4 @@
-package cloudinit
+package system
 
 import (
 	"io/ioutil"
@@ -26,12 +26,17 @@ Address=10.209.171.177/19
 	}
 	defer syscall.Rmdir(dir)
 
-	if _, err := PlaceUnit(dir, &u); err != nil {
+	dst := UnitDestination(&u, dir)
+	expectDst := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
+	if dst != expectDst {
+		t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
+	}
+
+	if err := PlaceUnit(&u, dst); err != nil {
 		t.Fatalf("PlaceUnit failed: %v", err)
 	}
 
-	fullPath := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
-	fi, err := os.Stat(fullPath)
+	fi, err := os.Stat(dst)
 	if err != nil {
 		t.Fatalf("Unable to stat file: %v", err)
 	}
@@ -40,19 +45,19 @@ Address=10.209.171.177/19
 		t.Errorf("File has incorrect mode: %v", fi.Mode())
 	}
 
-	contents, err := ioutil.ReadFile(fullPath)
+	contents, err := ioutil.ReadFile(dst)
 	if err != nil {
 		t.Fatalf("Unable to read expected file: %v", err)
 	}
 
-	expect := `[Match]
+	expectContents := `[Match]
 Name=eth47
 
 [Network]
 Address=10.209.171.177/19
 `
-	if string(contents) != expect {
-		t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expect)
+	if string(contents) != expectContents {
+		t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expectContents)
 	}
 }
 
@@ -72,12 +77,17 @@ Where=/media/state
 	}
 	defer syscall.Rmdir(dir)
 
-	if _, err := PlaceUnit(dir, &u); err != nil {
+	dst := UnitDestination(&u, dir)
+	expectDst := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
+	if dst != expectDst {
+		t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
+	}
+
+	if err := PlaceUnit(&u, dst); err != nil {
 		t.Fatalf("PlaceUnit failed: %v", err)
 	}
 
-	fullPath := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
-	fi, err := os.Stat(fullPath)
+	fi, err := os.Stat(dst)
 	if err != nil {
 		t.Fatalf("Unable to stat file: %v", err)
 	}
@@ -86,17 +96,31 @@ Where=/media/state
 		t.Errorf("File has incorrect mode: %v", fi.Mode())
 	}
 
-	contents, err := ioutil.ReadFile(fullPath)
+	contents, err := ioutil.ReadFile(dst)
 	if err != nil {
 		t.Fatalf("Unable to read expected file: %v", err)
 	}
 
-	expect := `[Mount]
+	expectContents := `[Mount]
 What=/dev/sdb1
 Where=/media/state
 `
-	if string(contents) != expect {
-		t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expect)
+	if string(contents) != expectContents {
+		t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expectContents)
 	}
 }
 
+func TestMachineID(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
+	ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007\n"), os.FileMode(0444))
+
+	if MachineID(dir) != "node007" {
+		t.Fatalf("File has incorrect contents")
+	}
+}

system/user.go

@@ -1,4 +1,4 @@
-package cloudinit
+package system
 
 import (
 	"fmt"
@@ -9,17 +9,19 @@ import (
 )
 
 type User struct {
-	Name              string   `yaml:"name"`
-	PasswordHash      string   `yaml:"passwd"`
-	SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
-	GECOS             string   `yaml:"gecos"`
-	Homedir           string   `yaml:"homedir"`
-	NoCreateHome      bool     `yaml:"no-create-home"`
-	PrimaryGroup      string   `yaml:"primary-group"`
-	Groups            []string `yaml:"groups"`
-	NoUserGroup       bool     `yaml:"no-user-group"`
-	System            bool     `yaml:"system"`
-	NoLogInit         bool     `yaml:"no-log-init"`
+	Name                string   `yaml:"name"`
+	PasswordHash        string   `yaml:"passwd"`
+	SSHAuthorizedKeys   []string `yaml:"ssh-authorized-keys"`
+	SSHImportGithubUser string   `yaml:"coreos-ssh-import-github"`
+	SSHImportURL        string   `yaml:"coreos-ssh-import-url"`
+	GECOS               string   `yaml:"gecos"`
+	Homedir             string   `yaml:"homedir"`
+	NoCreateHome        bool     `yaml:"no-create-home"`
+	PrimaryGroup        string   `yaml:"primary-group"`
+	Groups              []string `yaml:"groups"`
+	NoUserGroup         bool     `yaml:"no-user-group"`
+	System              bool     `yaml:"system"`
+	NoLogInit           bool     `yaml:"no-log-init"`
 }
 
 func UserExists(u *User) bool {
@@ -32,6 +34,8 @@ func CreateUser(u *User) error {
 
 	if u.PasswordHash != "" {
 		args = append(args, "--password", u.PasswordHash)
+	} else {
+		args = append(args, "--password", "*")
 	}
 
 	if u.GECOS != "" {

test

@@ -4,5 +4,7 @@ echo "Building bin/coreos-cloudinit"
 . build
 
 echo "Running tests..."
-go test -i github.com/coreos/coreos-cloudinit/cloudinit
-go test -v github.com/coreos/coreos-cloudinit/cloudinit
+for pkg in "./initialize ./system ./datasource"; do
+	go test -i $pkg
+	go test -v $pkg
+done

third_party/github.com/coreos/go-systemd/dbus/dbus.go

@@ -18,6 +18,8 @@ limitations under the License.
 package dbus
 
 import (
+	"os"
+	"strconv"
 	"strings"
 	"sync"
 
@@ -73,7 +75,12 @@ func (c *Conn) initConnection() error {
 		return err
 	}
 
-	err = c.sysconn.Auth(nil)
+	// Only use EXTERNAL method, and hardcode the uid (not username)
+	// to avoid a username lookup (which requires a dynamically linked
+	// libc)
+	methods := []dbus.Auth{dbus.AuthExternal(strconv.Itoa(os.Getuid()))}
+
+	err = c.sysconn.Auth(methods)
 	if err != nil {
 		c.sysconn.Close()
 		return err

third_party/github.com/coreos/go-systemd/dbus/methods.go

@@ -35,6 +35,7 @@ func (c *Conn) jobComplete(signal *dbus.Signal) {
 	out, ok := c.jobListener.jobs[job]
 	if ok {
 		out <- result
+		delete(c.jobListener.jobs, job)
 	}
 	c.jobListener.Unlock()
 }
@@ -137,8 +138,8 @@ func (c *Conn) KillUnit(name string, signal int32) {
 	c.sysobj.Call("org.freedesktop.systemd1.Manager.KillUnit", 0, name, "all", signal).Store()
 }
 
-// GetUnitProperties takes the unit name and returns all of its dbus object properties.
-func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
+// getProperties takes the unit name and returns all of its dbus object properties, for the given dbus interface
+func (c *Conn) getProperties(unit string, dbusInterface string) (map[string]interface{}, error) {
 	var err error
 	var props map[string]dbus.Variant
 
@@ -148,7 +149,7 @@ func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
 	}
 
 	obj := c.sysconn.Object("org.freedesktop.systemd1", path)
-	err = obj.Call("org.freedesktop.DBus.Properties.GetAll", 0, "org.freedesktop.systemd1.Unit").Store(&props)
+	err = obj.Call("org.freedesktop.DBus.Properties.GetAll", 0, dbusInterface).Store(&props)
 	if err != nil {
 		return nil, err
 	}
@@ -161,6 +162,55 @@ func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
 	return out, nil
 }
 
+// GetUnitProperties takes the unit name and returns all of its dbus object properties.
+func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
+	return c.getProperties(unit, "org.freedesktop.systemd1.Unit")
+}
+
+func (c *Conn) getProperty(unit string, dbusInterface string, propertyName string) (*Property, error) {
+	var err error
+	var prop dbus.Variant
+
+	path := ObjectPath("/org/freedesktop/systemd1/unit/" + unit)
+	if !path.IsValid() {
+		return nil, errors.New("invalid unit name: " + unit)
+	}
+
+	obj := c.sysconn.Object("org.freedesktop.systemd1", path)
+	err = obj.Call("org.freedesktop.DBus.Properties.Get", 0, dbusInterface, propertyName).Store(&prop)
+	if err != nil {
+		return nil, err
+	}
+
+	return &Property{Name: propertyName, Value: prop}, nil
+}
+
+func (c *Conn) GetUnitProperty(unit string, propertyName string) (*Property, error) {
+	return c.getProperty(unit, "org.freedesktop.systemd1.Unit", propertyName)
+}
+
+// GetUnitTypeProperties returns the extra properties for a unit, specific to the unit type.
+// Valid values for unitType: Service, Socket, Target, Device, Mount, Automount, Snapshot, Timer, Swap, Path, Slice, Scope
+// return "dbus.Error: Unknown interface" if the unitType is not the correct type of the unit
+func (c *Conn) GetUnitTypeProperties(unit string, unitType string) (map[string]interface{}, error) {
+	return c.getProperties(unit, "org.freedesktop.systemd1."+unitType)
+}
+
+// SetUnitProperties() may be used to modify certain unit properties at runtime.
+// Not all properties may be changed at runtime, but many resource management
+// settings (primarily those in systemd.cgroup(5)) may. The changes are applied
+// instantly, and stored on disk for future boots, unless runtime is true, in which
+// case the settings only apply until the next reboot. name is the name of the unit
+// to modify. properties are the settings to set, encoded as an array of property
+// name and value pairs.
+func (c *Conn) SetUnitProperties(name string, runtime bool, properties ...Property) error {
+	return c.sysobj.Call("SetUnitProperties", 0, name, runtime, properties).Store()
+}
+
+func (c *Conn) GetUnitTypeProperty(unit string, unitType string, propertyName string) (*Property, error) {
+	return c.getProperty(unit, "org.freedesktop.systemd1." + unitType, propertyName)
+}
+
 // ListUnits returns an array with all currently loaded units. Note that
 // units may be known by multiple names at the same time, and hence there might
 // be more unit names loaded than actual units behind them.
@@ -253,8 +303,52 @@ type EnableUnitFileChange struct {
 	Destination string // Destination of the symlink
 }
 
+// DisableUnitFiles() may be used to disable one or more units in the system (by
+// removing symlinks to them from /etc or /run).
+//
+// It takes a list of unit files to disable (either just file names or full
+// absolute paths if the unit files are residing outside the usual unit
+// search paths), and one boolean: whether the unit was enabled for runtime
+// only (true, /run), or persistently (false, /etc).
+//
+// This call returns an array with the changes made. The changes list
+// consists of structures with three strings: the type of the change (one of
+// symlink or unlink), the file name of the symlink and the destination of the
+// symlink.
+func (c *Conn) DisableUnitFiles(files []string, runtime bool) ([]DisableUnitFileChange, error) {
+	result := make([][]interface{}, 0)
+	err := c.sysobj.Call("DisableUnitFiles", 0, files, runtime).Store(&result)
+	if err != nil {
+		return nil, err
+	}
+
+	resultInterface := make([]interface{}, len(result))
+	for i := range result {
+		resultInterface[i] = result[i]
+	}
+
+	changes := make([]DisableUnitFileChange, len(result))
+	changesInterface := make([]interface{}, len(changes))
+	for i := range changes {
+		changesInterface[i] = &changes[i]
+	}
+
+	err = dbus.Store(resultInterface, changesInterface...)
+	if err != nil {
+		return nil, err
+	}
+
+	return changes, nil
+}
+
+type DisableUnitFileChange struct {
+	Type        string // Type of the change (one of symlink or unlink)
+	Filename    string // File name of the symlink
+	Destination string // Destination of the symlink
+}
+
 // Reload instructs systemd to scan for and reload unit files. This is
 // equivalent to a 'systemctl daemon-reload'.
-func (c *Conn) Reload() (string, error) {
-	return c.runJob("org.freedesktop.systemd1.Manager.Reload")
+func (c *Conn) Reload() error {
+	return c.sysobj.Call("org.freedesktop.systemd1.Manager.Reload", 0).Store()
 }

third_party/github.com/coreos/go-systemd/dbus/methods_test.go

@@ -18,9 +18,11 @@ package dbus
 
 import (
 	"fmt"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"math/rand"
 	"os"
 	"path/filepath"
+	"reflect"
 	"testing"
 )
 
@@ -50,13 +52,16 @@ func setupUnit(target string, conn *Conn, t *testing.T) {
 	fixture := []string{abs}
 
 	install, changes, err := conn.EnableUnitFiles(fixture, true, true)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	if install != false {
 		t.Fatal("Install was true")
 	}
 
 	if len(changes) < 1 {
-		t.Fatal("Expected one change, got %v", changes)
+		t.Fatalf("Expected one change, got %v", changes)
 	}
 
 	if changes[0].Filename != targetRun {
@@ -118,6 +123,37 @@ func TestStartStopUnit(t *testing.T) {
 	}
 }
 
+// Enables a unit and then immediately tears it down
+func TestEnableDisableUnit(t *testing.T) {
+	target := "enable-disable.service"
+	conn := setupConn(t)
+
+	setupUnit(target, conn, t)
+
+	abs, err := filepath.Abs("../fixtures/" + target)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	path := filepath.Join("/run/systemd/system/", target)
+
+	// 2. Disable the unit
+	changes, err := conn.DisableUnitFiles([]string{abs}, true)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if len(changes) != 1 {
+		t.Fatalf("Changes should include the path, %v", changes)
+	}
+	if changes[0].Filename != path {
+		t.Fatalf("Change should include correct filename, %+v", changes[0])
+	}
+	if changes[0].Destination != "" {
+		t.Fatalf("Change destination should be empty, %+v", changes[0])
+	}
+}
+
 // TestGetUnitProperties reads the `-.mount` which should exist on all systemd
 // systems and ensures that one of its properties is valid.
 func TestGetUnitProperties(t *testing.T) {
@@ -139,6 +175,20 @@ func TestGetUnitProperties(t *testing.T) {
 	if names[0] != "system.slice" {
 		t.Fatal("unexpected wants for /")
 	}
+
+	prop, err := conn.GetUnitProperty(unit, "Wants")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if prop.Name != "Wants" {
+		t.Fatal("unexpected property name")
+	}
+
+	val := prop.Value.Value().([]string)
+	if !reflect.DeepEqual(val, names) {
+		t.Fatal("unexpected property value")
+	}
 }
 
 // TestGetUnitPropertiesRejectsInvalidName attempts to get the properties for a
@@ -150,10 +200,37 @@ func TestGetUnitPropertiesRejectsInvalidName(t *testing.T) {
 	unit := "//invalid#$^/"
 
 	_, err := conn.GetUnitProperties(unit)
-
 	if err == nil {
 		t.Fatal("Expected an error, got nil")
 	}
+
+	_, err = conn.GetUnitProperty(unit, "Wants")
+	if err == nil {
+		t.Fatal("Expected an error, got nil")
+	}
+}
+
+// TestSetUnitProperties changes a cgroup setting on the `tmp.mount`
+// which should exist on all systemd systems and ensures that the
+// property was set.
+func TestSetUnitProperties(t *testing.T) {
+	conn := setupConn(t)
+
+	unit := "tmp.mount"
+
+	if err := conn.SetUnitProperties(unit, true, Property{"CPUShares", dbus.MakeVariant(uint64(1023))}); err != nil {
+		t.Fatal(err)
+	}
+
+	info, err := conn.GetUnitTypeProperties(unit, "Mount")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	value := info["CPUShares"].(uint64)
+	if value != 1023 {
+		t.Fatal("CPUShares of unit is not 1023, %s", value)
+	}
 }
 
 // Ensure that basic transient unit starting and stopping works.
@@ -211,3 +288,27 @@ func TestStartStopTransientUnit(t *testing.T) {
 		t.Fatalf("Test unit found in list, should be stopped")
 	}
 }
+
+func TestConnJobListener(t *testing.T) {
+	target := "start-stop.service"
+	conn := setupConn(t)
+
+	setupUnit(target, conn, t)
+
+	jobSize := len(conn.jobListener.jobs)
+
+	_, err := conn.StartUnit(target, "replace")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = conn.StopUnit(target, "replace")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	currentJobSize := len(conn.jobListener.jobs)
+	if jobSize != currentJobSize {
+		t.Fatal("JobListener jobs leaked")
+	}
+}

third_party/github.com/coreos/go-systemd/dbus/properties.go

@@ -209,3 +209,12 @@ func PropPropagatesReloadTo(units ...string) Property {
 func PropRequiresMountsFor(units ...string) Property {
 	return propDependency("RequiresMountsFor", units)
 }
+
+// PropSlice sets the Slice unit property.  See
+// http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#Slice=
+func PropSlice(slice string) Property {
+	return Property{
+		Name:  "Slice",
+		Value: dbus.MakeVariant(slice),
+	}
+}

units/90-configdrive.rules

@@ -0,0 +1,11 @@
+# Automatically trigger configdrive mounting.
+
+ACTION!="add|change", GOTO="coreos_configdrive_end"
+
+# A normal config drive. Block device formatted with iso9660 or fat
+SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="iso9660|vfat", ENV{ID_FS_LABEL}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-block.service"
+
+# Addtionally support virtfs from QEMU
+SUBSYSTEM=="virtio", DRIVER=="9pnet_virtio", ATTR{mount_tag}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-virtfs.service"
+
+LABEL="coreos_configdrive_end"

units/configdrive-block.service

@@ -0,0 +1,15 @@
+[Unit]
+Description=Mount config drive
+Conflicts=configdrive-virtfs.service umount.target
+ConditionPathIsMountPoint=!/media/configdrive
+# Only mount config drive block devices automatically in virtual machines
+ConditionVirtualization=vm
+
+# OpenStack defined config drive so they get to stick their name in it
+Wants=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+Before=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=no
+ExecStart=/bin/mount -t auto -o ro,x-mount.mkdir LABEL=config-2 /media/configdrive

units/configdrive-virtfs.service

@@ -0,0 +1,18 @@
+[Unit]
+Description=Mount config drive from virtfs
+Conflicts=configdrive-block.service umount.target
+ConditionPathIsMountPoint=!/media/configdrive
+ConditionVirtualization=vm
+
+# OpenStack defined config drive so they get to stick their name in it
+Wants=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+Before=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+
+# Support old style setup for now
+Wants=addon-run@media-configdrive.service addon-config@media-configdrive.service
+Before=addon-run@media-configdrive.service addon-config@media-configdrive.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=no
+ExecStart=/bin/mount -t 9p -o trans=virtio,version=9p2000.L,x-mount.mkdir config-2 /media/configdrive

units/system-cloudinit@.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=Load cloud-config from %f
+Requires=dbus.service
+After=dbus.service
+Before=system-config.target
+ConditionFileNotEmpty=%f
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

units/system-config.target

@@ -0,0 +1,10 @@
+[Unit]
+Description=Load system-provided cloud configs
+
+# Generate /etc/environment
+Requires=coreos-setup-environment.service
+After=coreos-setup-environment.service
+
+# Load OEM cloud-config.yml
+Requires=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service
+After=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service

units/user-cloudinit@.service

@@ -0,0 +1,12 @@
+[Unit]
+Description=Load cloud-config from %f
+Requires=coreos-setup-environment.service
+After=coreos-setup-environment.service
+Before=user-config.target
+ConditionFileNotEmpty=%f
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+EnvironmentFile=-/etc/environment
+ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

units/user-config.target

@@ -0,0 +1,8 @@
+[Unit]
+Description=Load user-provided cloud configs
+Requires=system-config.target
+After=system-config.target
+
+# Load user_data placed by coreos-install
+Requires=user-cloudinit@var-lib-coreos\x2dinstall-user_data.service
+After=user-cloudinit@var-lib-coreos\x2dinstall-user_data.service