feat(Documentation/cloud-config): add section about oem-release

We need to have cloud-config render a file about itself that can be
easily parsed and used by the rest of the system. Essentially just copy
the os-release format.

Documentation/cloud-config.md

@@ -0,0 +1,196 @@
+# Customize CoreOS with Cloud-Config
+
+CoreOS allows you to configure machine parameters, launch systemd units on startup and more. Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS.
+
+[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
+
+## Supported cloud-config Parameters
+
+### ssh_authorized_keys
+
+Provided public SSH keys will be authorized for the `core` user.
+
+The keys will be named "coreos-cloudinit" by default.
+Override this with the `--ssh-key-name` flag when calling `coreos-cloudinit`.
+
+### hostname
+
+The provided value will be used to set the system's hostname.
+This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
+
+### users
+
+Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields.
+Each field is optional and of type string unless otherwise noted.
+All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
+
+- **name**: Required. Login name of user
+- **gecos**: GECOS comment of user
+- **passwd**: Hash of the password to use for this user
+- **homedir**: User's home directory. Defaults to /home/<name>
+- **no-create-home**: Boolean. Skip home directory createion.
+- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
+- **groups**: Add user to these additional groups
+- **no-user-group**: Boolean. Skip default group creation.
+- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
+- **system**: Create the user as a system user. No home directory will be created.
+- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
+
+The following fields are not yet implemented:
+
+- **inactive**: Deactivate the user upon creation
+- **lock-passwd**: Boolean. Disable password login for user
+- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
+- **selinux-user**: Corresponding SELinux user
+- **ssh-import-id**: Import SSH keys by ID from Launchpad.
+
+##### Generating a password hash
+
+Generating a safe hash is important to the security of your system.  Currently with updated tools like [oclhashcat](http://hashcat.net/oclhashcat/) simplified hashes like md5crypt are trivial to crack on modern GPU hardware.  You can generate a "safer" hash (read: not safe, never publish your hashes publicly) via:
+
+###### On Debian/Ubuntu (via the package "whois")
+    mkpasswd --method=SHA-512 --rounds=4096
+
+###### With OpenSSL (note: this will only make md5crypt.  While better than plantext it should not be considered fully secure)
+    openssl passwd -1
+
+###### With Python (change password and salt values)
+    python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
+
+###### With Perl (change password and salt values)
+    perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
+
+Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed.  On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms. 
+
+### write_files
+
+Inject an arbitrary set of files to the local filesystem.
+Provide a list of objects with the following attributes:
+
+- **path**: Absolute location on disk where contents should be written
+- **content**: Data to write at the provided `path`
+- **permissions**: String representing file permissions in octal notation (i.e. '0644')
+- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
+
+## Custom cloud-config Parameters
+
+### coreos.oem
+
+These fields are borrowed from the [os-release spec][os-release] and repurposed
+as a way for cloud-init to know about the OEM partition on this machine.
+
+- **id**: A lower case string identifying the oem.
+- **version-id**: A lower case string identifying the version of the OEM. Example: `168.0.0`
+- **name**: A name without the version that is suitable for presentation to the user.
+- **home-url**: Link to the homepage of the provider or OEM.
+- **bug-report-url***: Link to a place to file bug reports about this OEM partition.
+
+cloudinit must render these fields down to an /etc/oem-release file on disk in the following format:
+
+```
+NAME=Rackspace
+ID=rackspace
+VERSION_ID=168.0.0
+PRETTY_NAME="Rackspace Cloud Servers"
+HOME_URL="http://www.rackspace.com/cloud/servers/"
+BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
+```
+
+[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html
+
+### coreos.etcd.discovery_url
+
+The value of `coreos.etcd.discovery_url` will be used to discover the instance's etcd peers using the [etcd discovery protocol][disco-proto]. Usage of the [public discovery service][disco-service] is encouraged.
+
+[disco-proto]: https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md
+[disco-service]: http://discovery.etcd.io
+
+### coreos.units
+
+Arbitrary systemd units may be provided in the `coreos.units` attribute.
+`coreos.units` is a list of objects with the following fields:
+
+- **name**: string representing unit's name
+- **runtime**: boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` flag to `systemd enable`.
+- **content**: plaintext string representing entire unit file
+
+See docker example below.
+
+## user-data Script
+
+Simply set your user-data to a script where the first line is a shebang:
+
+```
+#!/bin/bash
+
+echo 'Hello, world!'
+```
+
+## Examples
+
+### Inject an SSH key, bootstrap etcd, and start fleet
+```
+#cloud-config
+
+coreos:
+	etcd:
+		discovery_url: https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877
+    fleet:
+        autostart: yes
+ssh_authorized_keys:
+  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
+```
+
+### Start a docker container on boot
+
+```
+#cloud-config
+
+coreos:
+    units:
+      - name: docker-redis.service
+        content: |
+          [Unit]
+          Description=Redis container
+          Author=Me
+          After=docker.service
+
+          [Service]
+          Restart=always
+          ExecStart=/usr/bin/docker start -a redis_server
+          ExecStop=/usr/bin/docker stop -t 2 redis_server
+          
+          [Install]
+          WantedBy=local.target
+```
+
+### Add a user
+
+```
+#cloud-config
+
+users:
+  - name: elroy
+	passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
+	groups:
+	  - staff
+	  - docker
+	ssh-authorized-keys:
+	  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
+```
+
+### Inject configuration files
+
+```
+#cloud-config
+
+write_files:
+  - path: /etc/hosts
+    contents: |
+      127.0.0.1    localhost
+      192.0.2.211  buildbox
+  - path: /etc/resolv.conf
+    contents: |
+      nameserver 192.0.2.13
+      nameserver 192.0.2.14
+```

README.md

@@ -1,54 +1,9 @@
 # coreos-cloudinit
 
-coreos-cloudinit allows a user to customize CoreOS machiens by providing either an executable script or a cloud-config document as instance user-data. See below to learn how to use these features.
+coreos-cloudinit allows a user to customize CoreOS machines by providing either an executable script or a cloud-config document as instance user-data. See below to learn how to use these features.
 
+## Supported Cloud-Config Features
 
-## cloud-config
+Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS, which are [documented here](https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md).
-
-Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS.
 
 [cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
-
-### Supported cloud-config Parameters
-
-#### ssh_authorized_keys
-
-Provided public SSH keys will be authorized for the `core` user.
-
-### Custom cloud-config Parameters
-
-#### coreos.etcd.discovery_url
-
-The value of `coreos.etcd.discovery_url` will be used to discover the instance's etcd peers using the [etcd discovery protocol][disco-proto]. Usage of the [public discovery service][disco-service] is encouraged.
-
-[disco-proto]: https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md
-[disco-service]: http://discovery.etcd.io
-
-
-## user-data Script
-
-Simply set your user-data to a script where the first line is a shebang:
-
-```
-#!/bin/bash
-
-echo 'Hello, world!'
-```
-
-## Examples
-
-### Inject an SSH key, bootstrap etcd, and start fleet using a cloud-config
-
-```
-#cloud-config
-
-coreos:
-	etcd:
-		discovery_url: https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877
-    fleet:
-        autostart: yes
-
-ssh_authorized_keys:
-  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
-```
-

cloudinit/cloud_config.go

@@ -1,14 +1,24 @@
 package cloudinit
 
 import (
+	"fmt"
 	"log"
 
-	"launchpad.net/goyaml"
+	"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
 )
 
+const DefaultSSHKeyName = "coreos-cloudinit"
+
 type CloudConfig struct {
-	SSH_Authorized_Keys []string
+	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
-	Coreos struct{Etcd struct{ Discovery_URL string }; Fleet struct{ Autostart bool } }
+	Coreos            struct {
+		Etcd  struct{ Discovery_URL string }
+		Fleet struct{ Autostart bool }
+		Units []Unit
+	}
+	WriteFiles []WriteFile `yaml:"write_files"`
+	Hostname   string
+	Users      []User
 }
 
 func NewCloudConfig(contents []byte) (*CloudConfig, error) {
@@ -19,16 +29,59 @@ func NewCloudConfig(contents []byte) (*CloudConfig, error) {
 
 func (cc CloudConfig) String() string {
 	bytes, err := goyaml.Marshal(cc)
-	if err == nil {
+	if err != nil {
-		return string(bytes)
-	} else {
 		return ""
 	}
+
+	stringified := string(bytes)
+	stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
+
+	return stringified
 }
 
-func ResolveCloudConfig(cfg CloudConfig) error {
+func ApplyCloudConfig(cfg CloudConfig, sshKeyName string) error {
-	if len(cfg.SSH_Authorized_Keys) > 0 {
+	if cfg.Hostname != "" {
-		err := AuthorizeSSHKeys(cfg.SSH_Authorized_Keys)
+		if err := SetHostname(cfg.Hostname); err != nil {
+			return err
+		}
+		log.Printf("Set hostname to %s", cfg.Hostname)
+	}
+
+	if len(cfg.Users) > 0 {
+		for _, user := range cfg.Users {
+			if user.Name == "" {
+				log.Printf("User object has no 'name' field, skipping")
+				continue
+			}
+
+			if UserExists(&user) {
+				log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
+				if user.PasswordHash != "" {
+					log.Printf("Setting '%s' user's password", user.Name)
+					if err := SetUserPassword(user.Name, user.PasswordHash); err != nil {
+						log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
+						return err
+					}
+				}
+			} else {
+				log.Printf("Creating user '%s'", user.Name)
+				if err := CreateUser(&user); err != nil {
+					log.Printf("Failed creating user '%s': %v", user.Name, err)
+					return err
+				}
+			}
+
+			if len(user.SSHAuthorizedKeys) > 0 {
+				log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
+				if err := AuthorizeSSHKeys(user.Name, sshKeyName, user.SSHAuthorizedKeys); err != nil {
+					return err
+				}
+			}
+		}
+	}
+
+	if len(cfg.SSHAuthorizedKeys) > 0 {
+		err := AuthorizeSSHKeys("core", sshKeyName, cfg.SSHAuthorizedKeys)
 		if err == nil {
 			log.Printf("Authorized SSH keys for core user")
 		} else {
@@ -36,6 +89,15 @@ func ResolveCloudConfig(cfg CloudConfig) error {
 		}
 	}
 
+	if len(cfg.WriteFiles) > 0 {
+		for _, file := range cfg.WriteFiles {
+			if err := ProcessWriteFile("/", &file); err != nil {
+				return err
+			}
+			log.Printf("Wrote file %s to filesystem", file.Path)
+		}
+	}
+
 	if cfg.Coreos.Etcd.Discovery_URL != "" {
 		err := PersistEtcdDiscoveryURL(cfg.Coreos.Etcd.Discovery_URL)
 		if err == nil {
@@ -45,8 +107,31 @@ func ResolveCloudConfig(cfg CloudConfig) error {
 		}
 	}
 
+	if len(cfg.Coreos.Units) > 0 {
+		for _, unit := range cfg.Coreos.Units {
+			log.Printf("Placing unit %s on filesystem", unit.Name)
+			dst, err := PlaceUnit("/", &unit)
+			if err != nil {
+				return err
+			}
+			log.Printf("Placed unit %s at %s", unit.Name, dst)
+
+			if unit.Group() != "network" {
+				log.Printf("Enabling unit file %s", dst)
+				if err := EnableUnitFile(dst, unit.Runtime); err != nil {
+					return err
+				}
+				log.Printf("Enabled unit %s", unit.Name)
+			} else {
+				log.Printf("Skipping enable for network-like unit %s", unit.Name)
+			}
+		}
+		DaemonReload()
+		StartUnits(cfg.Coreos.Units)
+	}
+
 	if cfg.Coreos.Fleet.Autostart {
-		err := StartUnit("fleet.service")
+		err := StartUnitByName("fleet.service")
 		if err == nil {
 			log.Printf("Started fleet service.")
 		} else {

cloudinit/cloud_config_test.go

@@ -1,6 +1,7 @@
 package cloudinit
 
 import (
+	"strings"
 	"testing"
 )
 
@@ -11,7 +12,7 @@ func TestCloudConfigEmpty(t *testing.T) {
 		t.Fatalf("Encountered unexpected error :%v", err)
 	}
 
-	keys := cfg.SSH_Authorized_Keys
+	keys := cfg.SSHAuthorizedKeys
 	if len(keys) != 0 {
 		t.Error("Parsed incorrect number of SSH keys")
 	}
@@ -23,6 +24,14 @@ func TestCloudConfigEmpty(t *testing.T) {
 	if cfg.Coreos.Fleet.Autostart {
 		t.Error("Expected AutostartFleet not to be defined")
 	}
+
+	if len(cfg.WriteFiles) != 0 {
+		t.Error("Expected zero WriteFiles")
+	}
+
+	if cfg.Hostname != "" {
+		t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
+	}
 }
 
 // Assert that the parsing of a cloud config file "generally works"
@@ -33,16 +42,37 @@ coreos:
     discovery_url: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
   fleet:
     autostart: Yes
+  units:
+    - name: 50-eth0.network
+      runtime: yes
+      content: '[Match]
+ 
+    Name=eth47
+ 
+ 
+    [Network]
+ 
+    Address=10.209.171.177/19
+ 
+'
 ssh_authorized_keys:
   - foobar
   - foobaz
+write_files:
+  - content: |
+      penny
+      elroy
+    path: /etc/dogepack.conf
+    permissions: '0644'
+    owner: root:dogepack
+hostname: trontastic
 `)
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
 	}
 
-	keys := cfg.SSH_Authorized_Keys
+	keys := cfg.SSHAuthorizedKeys
 	if len(keys) != 2 {
 		t.Error("Parsed incorrect number of SSH keys")
 	} else if keys[0] != "foobar" {
@@ -58,6 +88,55 @@ ssh_authorized_keys:
 	if !cfg.Coreos.Fleet.Autostart {
 		t.Error("Expected AutostartFleet to be true")
 	}
+
+	if len(cfg.WriteFiles) != 1 {
+		t.Error("Failed to parse correct number of write_files")
+	} else {
+		wf := cfg.WriteFiles[0]
+		if wf.Content != "penny\nelroy\n" {
+			t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
+		}
+		if wf.Encoding != "" {
+			t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
+		}
+		if wf.Permissions != "0644" {
+			t.Errorf("WriteFile has incorrect permissions %s", wf.Permissions)
+		}
+		if wf.Path != "/etc/dogepack.conf" {
+			t.Errorf("WriteFile has incorrect path %s", wf.Path)
+		}
+		if wf.Owner != "root:dogepack" {
+			t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
+		}
+	}
+
+	if len(cfg.Coreos.Units) != 1 {
+		t.Error("Failed to parse correct number of units")
+	} else {
+		u := cfg.Coreos.Units[0]
+		expect := `[Match]
+Name=eth47
+
+[Network]
+Address=10.209.171.177/19
+`
+		if u.Content != expect {
+			t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
+		}
+		if u.Runtime != true {
+			t.Errorf("Unit has incorrect runtime value")
+		}
+		if u.Name != "50-eth0.network" {
+			t.Errorf("Unit has incorrect name %s", u.Name)
+		}
+		if u.Type() != "network" {
+			t.Errorf("Unit has incorrect type '%s'", u.Type())
+		}
+	}
+
+	if cfg.Hostname != "trontastic" {
+		t.Errorf("Failed to parse hostname")
+	}
 }
 
 // Assert that our interface conversion doesn't panic
@@ -71,8 +150,103 @@ ssh_authorized_keys:
 		t.Fatalf("Encountered unexpected error :%v", err)
 	}
 
-	keys := cfg.SSH_Authorized_Keys
+	keys := cfg.SSHAuthorizedKeys
 	if len(keys) != 0 {
 		t.Error("Parsed incorrect number of SSH keys")
 	}
 }
+
+func TestCloudConfigSerializationHeader(t *testing.T) {
+	cfg, _ := NewCloudConfig([]byte{})
+	contents := cfg.String()
+	header := strings.SplitN(contents, "\n", 2)[0]
+	if header != "#cloud-config" {
+		t.Fatalf("Serialized config did not have expected header")
+	}
+}
+
+func TestCloudConfigUsers(t *testing.T) {
+	contents := []byte(`
+users:
+  - name: elroy
+    passwd: somehash
+    ssh-authorized-keys:
+      - somekey
+    gecos: arbitrary comment
+    homedir: /home/place
+    no-create-home: yes
+    primary-group: things
+    groups:
+      - ping
+      - pong
+    no-user-group: true
+    system: y
+    no-log-init: True
+`)
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.PasswordHash != "somehash" {
+		t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
+	}
+
+	if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
+		t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
+	} else {
+		key := user.SSHAuthorizedKeys[0]
+		if key != "somekey" {
+			t.Errorf("User SSH key is %q, expected 'somekey'", key)
+		}
+	}
+
+	if user.GECOS != "arbitrary comment" {
+		t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
+	}
+
+	if user.Homedir != "/home/place" {
+		t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
+	}
+
+	if !user.NoCreateHome {
+		t.Errorf("Failed to parse no-create-home field")
+	}
+
+	if user.PrimaryGroup != "things"{
+		t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
+	}
+
+	if len(user.Groups) != 2 {
+		t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
+	} else {
+		if user.Groups[0] != "ping" {
+			t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
+		}
+		if user.Groups[1] != "pong" {
+			t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
+		}
+	}
+
+	if !user.NoUserGroup {
+		t.Errorf("Failed to parse no-user-group field")
+	}
+
+	if !user.System {
+		t.Errorf("Failed to parse system field")
+	}
+
+	if !user.NoLogInit {
+		t.Errorf("Failed to parse no-log-init field")
+	}
+}

cloudinit/metadata_service.go

@@ -19,8 +19,12 @@ func (ms *metadataService) UserData() ([]byte, error) {
 	if err != nil {
 		return []byte{}, err
 	}
-
 	defer resp.Body.Close()
+
+	if resp.StatusCode / 100 != 2 {
+		return []byte{}, nil
+	}
+
 	respBytes, err := ioutil.ReadAll(resp.Body)
 	if err != nil {
 		return nil, err

cloudinit/ssh_key.go

@@ -10,7 +10,7 @@ import (
 
 // Add the provide SSH public key to the core user's list of
 // authorized keys
-func AuthorizeSSHKeys(keys []string) error {
+func AuthorizeSSHKeys(user string, keysName string, keys []string) error {
 	for i, key := range keys {
 		keys[i] = strings.TrimSpace(key)
 	}
@@ -19,7 +19,7 @@ func AuthorizeSSHKeys(keys []string) error {
 	// also ends with a newline
 	joined := fmt.Sprintf("%s\n", strings.Join(keys, "\n"))
 
-	cmd := exec.Command("update-ssh-keys", "-u", "core", "-a", "coreos-cloudinit")
+	cmd := exec.Command("update-ssh-keys", "-u", user, "-a", keysName)
 	stdin, err := cmd.StdinPipe()
 	if err != nil {
 		return err

cloudinit/systemd.go

@@ -2,15 +2,127 @@ package cloudinit
 
 import (
 	"fmt"
+	"io/ioutil"
 	"log"
+	"os"
+	"os/exec"
 	"path"
+	"path/filepath"
+	"strings"
 
-	"github.com/coreos/go-systemd/dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
 )
 
+type Unit struct {
+	Name    string
+	Runtime bool
+	Content string
+}
+
+func (u *Unit) Type() string {
+	ext := filepath.Ext(u.Name)
+	return strings.TrimLeft(ext, ".")
+}
+
+func (u *Unit) Group() (group string) {
+	t := u.Type()
+	if t == "network" || t == "netdev" || t == "link" {
+		group = "network"
+	} else {
+		group = "system"
+	}
+	return
+}
+
 type Script []byte
 
-func StartUnit(name string) error {
+func PlaceUnit(root string, u *Unit) (string, error) {
+	dir := "etc"
+	if u.Runtime {
+		dir = "run"
+	}
+
+	dst := path.Join(root, dir, "systemd", u.Group())
+	if _, err := os.Stat(dst); os.IsNotExist(err) {
+		if err := os.MkdirAll(dst, os.FileMode(0755)); err != nil {
+			return "", err
+		}
+	}
+
+	dst = path.Join(dst, u.Name)
+	err := ioutil.WriteFile(dst, []byte(u.Content), os.FileMode(0644))
+	if err != nil {
+		return "", err
+	}
+
+	return dst, nil
+}
+
+func EnableUnitFile(file string, runtime bool) error {
+	conn, err := dbus.New()
+	if err != nil {
+		return err
+	}
+
+	files := []string{file}
+	_, _, err = conn.EnableUnitFiles(files, runtime, true)
+	return err
+}
+
+func separateNetworkUnits(units []Unit) ([]Unit, []Unit) {
+	networkUnits := make([]Unit, 0)
+	nonNetworkUnits := make([]Unit, 0)
+	for _, unit := range units {
+		if unit.Group() == "network" {
+			networkUnits = append(networkUnits, unit)
+		} else {
+			nonNetworkUnits = append(nonNetworkUnits, unit)
+		}
+	}
+	return networkUnits, nonNetworkUnits
+}
+
+func StartUnits(units []Unit) error {
+	networkUnits, nonNetworkUnits := separateNetworkUnits(units)
+	if len(networkUnits) > 0 {
+		if err := RestartUnitByName("systemd-networkd.service"); err != nil {
+			return err
+		}
+	}
+
+	for _, unit := range nonNetworkUnits {
+		if err := RestartUnitByName(unit.Name); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func DaemonReload() error {
+	conn, err := dbus.New()
+	if err != nil {
+		return err
+	}
+
+	_, err = conn.Reload()
+	return err
+}
+
+func RestartUnitByName(name string) error {
+	log.Printf("Restarting unit %s", name)
+	conn, err := dbus.New()
+	if err != nil {
+		return err
+	}
+
+	output, err := conn.RestartUnit(name, "replace")
+	log.Printf("Restart completed with '%s'", output)
+
+	return err
+}
+
+func StartUnitByName(name string) error {
 	conn, err := dbus.New()
 	if err != nil {
 		return err
@@ -39,3 +151,7 @@ func ExecuteScript(scriptPath string) (string, error) {
 	_, err = conn.StartTransientUnit(name, "replace", props...)
 	return name, err
 }
+
+func SetHostname(hostname string) error {
+	return exec.Command("hostnamectl", "set-hostname", hostname).Run()
+}

cloudinit/systemd_test.go

@@ -0,0 +1,102 @@
+package cloudinit
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"syscall"
+	"testing"
+)
+
+func TestPlaceNetworkUnit(t *testing.T) {
+	u := Unit{
+		Name: "50-eth0.network",
+      Runtime: true,
+      Content: `[Match]
+Name=eth47
+
+[Network]
+Address=10.209.171.177/19
+`,
+	}
+
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if _, err := PlaceUnit(dir, &u); err != nil {
+		t.Fatalf("PlaceUnit failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `[Match]
+Name=eth47
+
+[Network]
+Address=10.209.171.177/19
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expect)
+	}
+}
+
+func TestPlaceMountUnit(t *testing.T) {
+	u := Unit{
+		Name: "media-state.mount",
+      Runtime: false,
+      Content: `[Mount]
+What=/dev/sdb1
+Where=/media/state
+`,
+	}
+
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if _, err := PlaceUnit(dir, &u); err != nil {
+		t.Fatalf("PlaceUnit failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `[Mount]
+What=/dev/sdb1
+Where=/media/state
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expect)
+	}
+}
+

cloudinit/user.go

@@ -0,0 +1,106 @@
+package cloudinit
+
+import (
+	"fmt"
+	"log"
+	"os/exec"
+	"os/user"
+	"strings"
+)
+
+type User struct {
+	Name              string   `yaml:"name"`
+	PasswordHash      string   `yaml:"passwd"`
+	SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
+	GECOS             string   `yaml:"gecos"`
+	Homedir           string   `yaml:"homedir"`
+	NoCreateHome      bool     `yaml:"no-create-home"`
+	PrimaryGroup      string   `yaml:"primary-group"`
+	Groups            []string `yaml:"groups"`
+	NoUserGroup       bool     `yaml:"no-user-group"`
+	System            bool     `yaml:"system"`
+	NoLogInit         bool     `yaml:"no-log-init"`
+}
+
+func UserExists(u *User) bool {
+	_, err := user.Lookup(u.Name)
+	return err == nil
+}
+
+func CreateUser(u *User) error {
+	args := []string{}
+
+	if u.PasswordHash != "" {
+		args = append(args, "--password", u.PasswordHash)
+	}
+
+	if u.GECOS != "" {
+		args = append(args, "--comment", fmt.Sprintf("%q", u.GECOS))
+	}
+
+	if u.Homedir != "" {
+		args = append(args, "--home-dir", u.Homedir)
+	}
+
+	if u.NoCreateHome {
+		args = append(args, "--no-create-home")
+	} else {
+		args = append(args, "--create-home")
+	}
+
+	if u.PrimaryGroup != "" {
+		args = append(args, "--primary-group", u.PrimaryGroup)
+	}
+
+	if len(u.Groups) > 0 {
+		args = append(args, "--groups", strings.Join(u.Groups, ","))
+	}
+
+	if u.NoUserGroup {
+		args = append(args, "--no-user-group")
+	}
+
+	if u.System {
+		args = append(args, "--system")
+	}
+
+	if u.NoLogInit {
+		args = append(args, "--no-log-init")
+	}
+
+	args = append(args, u.Name)
+
+	output, err := exec.Command("useradd", args...).CombinedOutput()
+	if err != nil {
+		log.Printf("Command 'useradd %s' failed: %v\n%s", strings.Join(args, " "), err, output)
+	}
+	return err
+}
+
+func SetUserPassword(user, hash string) error {
+	cmd := exec.Command("/usr/sbin/chpasswd", "-e")
+
+	stdin, err := cmd.StdinPipe()
+	if err != nil {
+		return err
+	}
+
+	err = cmd.Start()
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	arg := fmt.Sprintf("%s:%s", user, hash)
+	_, err = stdin.Write([]byte(arg))
+	if err != nil {
+		return err
+	}
+	stdin.Close()
+
+	err = cmd.Wait()
+	if err != nil {
+		return err
+	}
+
+	return nil
+}

cloudinit/workspace.go

@@ -15,7 +15,7 @@ func PrepWorkspace(workspace string) error {
 			return fmt.Errorf("%s is not a directory", workspace)
 		}
 	} else {
-		err = os.MkdirAll(workspace, 0644)
+		err = os.MkdirAll(workspace, 0755)
 		if err != nil {
 			return err
 		}
@@ -29,7 +29,7 @@ func PrepWorkspace(workspace string) error {
 			return fmt.Errorf("%s is not a directory", scripts)
 		}
 	} else {
-		err = os.Mkdir(scripts, 0644)
+		err = os.Mkdir(scripts, 0755)
 		if err != nil {
 			return err
 		}
@@ -62,5 +62,5 @@ func PersistScriptInWorkspace(script Script, workspace string) (string, error) {
 
 func PersistScriptUnitNameInWorkspace(name string, workspace string) error {
 	unitPath := path.Join(workspace, "scripts", "unit-name")
-	return ioutil.WriteFile(unitPath, []byte(name), 644)
+	return ioutil.WriteFile(unitPath, []byte(name), 0644)
 }

cloudinit/write_file.go

@@ -0,0 +1,46 @@
+package cloudinit
+
+import (
+	"errors"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path"
+	"strconv"
+)
+
+type WriteFile struct {
+	Encoding    string
+	Content     string
+	Owner       string
+	Path        string
+	Permissions string
+}
+
+func ProcessWriteFile(base string, wf *WriteFile) error {
+	fullPath := path.Join(base, wf.Path)
+
+	if err := os.MkdirAll(path.Dir(fullPath), os.FileMode(0744)); err != nil {
+		return err
+	}
+
+	// Parse string representation of file mode as octal
+	perm, err := strconv.ParseInt(wf.Permissions, 8, 32)
+	if err != nil {
+		return errors.New("Unable to parse file permissions as octal integer")
+	}
+
+	if err := ioutil.WriteFile(fullPath, []byte(wf.Content), os.FileMode(perm)); err != nil {
+		return err
+	}
+
+	if wf.Owner != "" {
+		// We shell out since we don't have a way to look up unix groups natively
+		cmd := exec.Command("chown", wf.Owner, fullPath)
+		if err := cmd.Run(); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}

cloudinit/write_file_test.go

@@ -0,0 +1,81 @@
+package cloudinit
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"syscall"
+	"testing"
+)
+
+func TestWriteFileUnencodedContent(t *testing.T) {
+	wf := WriteFile{
+		Path:        "/tmp/foo",
+		Content:     "bar",
+		Permissions: "0644",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if err := ProcessWriteFile(dir, &wf); err != nil {
+		t.Fatalf("Processing of WriteFile failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "tmp", "foo")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != "bar" {
+		t.Fatalf("File has incorrect contents")
+	}
+}
+
+func TestWriteFileInvalidPermission(t *testing.T) {
+	wf := WriteFile{
+		Path:        "/tmp/foo",
+		Content:     "bar",
+		Permissions: "pants",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if err := ProcessWriteFile(dir, &wf); err == nil {
+		t.Fatalf("Expected error to be raised when writing file with invalid permission")
+	}
+}
+
+func TestWriteFileEncodedContent(t *testing.T) {
+	wf := WriteFile{
+		Path: "/tmp/foo",
+		Content: "",
+		Encoding: "base64",
+	}
+
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if err := ProcessWriteFile(dir, &wf); err == nil {
+		t.Fatalf("Expected error to be raised when writing file with encoding")
+	}
+}

coreos-cloudinit.go

@@ -10,7 +10,7 @@ import (
 	"github.com/coreos/coreos-cloudinit/cloudinit"
 )
 
-const version = "0.1.0"
+const version = "0.1.2+git"
 
 func main() {
 	var userdata []byte
@@ -28,6 +28,9 @@ func main() {
 	var workspace string
 	flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
 
+	var sshKeyName string
+	flag.StringVar(&sshKeyName, "ssh-key-name", cloudinit.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
+
 	flag.Parse()
 
 	if printVersion == true {
@@ -58,6 +61,11 @@ func main() {
 		os.Exit(1)
 	}
 
+	if len(userdata) == 0 {
+		log.Printf("No user data to handle, exiting.")
+		os.Exit(0)
+	}
+
 	parsed, err := cloudinit.ParseUserData(userdata)
 	if err != nil {
 		log.Fatalf("Failed parsing user-data: %v", err)
@@ -70,7 +78,7 @@ func main() {
 
 	switch t := parsed.(type) {
 	case cloudinit.CloudConfig:
-		err = cloudinit.ResolveCloudConfig(t)
+		err = cloudinit.ApplyCloudConfig(t, sshKeyName)
 	case cloudinit.Script:
 		var path string
 		path, err = cloudinit.PersistScriptInWorkspace(t, workspace)

third_party/github.com/coreos/go-systemd/dbus/dbus.go

@@ -21,7 +21,7 @@ import (
 	"strings"
 	"sync"
 
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 )
 
 const signalBuffer = 100

third_party/github.com/coreos/go-systemd/dbus/methods.go

@@ -18,7 +18,7 @@ package dbus
 
 import (
 	"errors"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 )
 
 func (c *Conn) initJobs() {
@@ -128,10 +128,7 @@ func (c *Conn) ReloadOrTryRestartUnit(name string, mode string) (string, error)
 // unique. mode is the same as in StartUnit(), properties contains properties
 // of the unit.
 func (c *Conn) StartTransientUnit(name string, mode string, properties ...Property) (string, error) {
-	// the dbus interface for this method does not use the last argument and
+	return c.runJob("org.freedesktop.systemd1.Manager.StartTransientUnit", name, mode, properties, make([]PropertyCollection, 0))
-	// should simply be given an empty list.  We use a concrete type here
-	// (instead of the more appropriate interface{}) to satisfy the dbus library.
-	return c.runJob("org.freedesktop.systemd1.Manager.StartTransientUnit", name, mode, properties, make([]string, 0))
 }
 
 // KillUnit takes the unit name and a UNIX signal number to send.  All of the unit's

third_party/github.com/coreos/go-systemd/dbus/methods_test.go

@@ -17,6 +17,8 @@ limitations under the License.
 package dbus
 
 import (
+	"fmt"
+	"math/rand"
 	"os"
 	"path/filepath"
 	"testing"
@@ -153,3 +155,59 @@ func TestGetUnitPropertiesRejectsInvalidName(t *testing.T) {
 		t.Fatal("Expected an error, got nil")
 	}
 }
+
+// Ensure that basic transient unit starting and stopping works.
+func TestStartStopTransientUnit(t *testing.T) {
+	conn := setupConn(t)
+
+	props := []Property{
+		PropExecStart([]string{"/bin/sleep", "400"}, false),
+	}
+	target := fmt.Sprintf("testing-transient-%d.service", rand.Int())
+
+	// Start the unit
+	job, err := conn.StartTransientUnit(target, "replace", props...)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if job != "done" {
+		t.Fatal("Job is not done, %v", job)
+	}
+
+	units, err := conn.ListUnits()
+
+	var unit *UnitStatus
+	for _, u := range units {
+		if u.Name == target {
+			unit = &u
+		}
+	}
+
+	if unit == nil {
+		t.Fatalf("Test unit not found in list")
+	}
+
+	if unit.ActiveState != "active" {
+		t.Fatalf("Test unit not active")
+	}
+
+	// 3. Stop the unit
+	job, err = conn.StopUnit(target, "replace")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	units, err = conn.ListUnits()
+
+	unit = nil
+	for _, u := range units {
+		if u.Name == target {
+			unit = &u
+		}
+	}
+
+	if unit != nil {
+		t.Fatalf("Test unit found in list, should be stopped")
+	}
+}

third_party/github.com/coreos/go-systemd/dbus/properties.go

@@ -17,7 +17,7 @@ limitations under the License.
 package dbus
 
 import (
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 )
 
 // From the systemd docs:
@@ -41,6 +41,11 @@ type Property struct {
 	Value dbus.Variant
 }
 
+type PropertyCollection struct {
+	Name       string
+	Properties []Property
+}
+
 type execStart struct {
 	Path             string   // the binary path to execute
 	Args             []string // an array with all arguments to pass to the executed command, starting with argument 0

third_party/github.com/coreos/go-systemd/dbus/subscription.go

@@ -20,7 +20,7 @@ import (
 	"errors"
 	"time"
 
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 )
 
 const (

third_party/github.com/coreos/go-systemd/examples/activation/activation.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 	"os"
 
-	"github.com/coreos/go-systemd/activation"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/activation"
 )
 
 func fixListenPid() {

third_party/github.com/coreos/go-systemd/examples/activation/httpserver/httpserver.go

@@ -4,7 +4,7 @@ import (
 	"io"
 	"net/http"
 
-	"github.com/coreos/go-systemd/activation"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/activation"
 )
 
 func HelloServer(w http.ResponseWriter, req *http.Request) {

third_party/github.com/coreos/go-systemd/examples/activation/listen.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 	"os"
 
-	"github.com/coreos/go-systemd/activation"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/activation"
 )
 
 func fixListenPid() {

third_party/github.com/guelfey/go.dbus/_examples/eavesdrop.go

@@ -2,7 +2,7 @@ package main
 
 import (
 	"fmt"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"os"
 )
 

third_party/github.com/guelfey/go.dbus/_examples/introspect.go

@@ -2,8 +2,8 @@ package main
 
 import (
 	"encoding/json"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
-	"github.com/guelfey/go.dbus/introspect"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus/introspect"
 	"os"
 )
 

third_party/github.com/guelfey/go.dbus/_examples/list-names.go

@@ -2,7 +2,7 @@ package main
 
 import (
 	"fmt"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"os"
 )
 

third_party/github.com/guelfey/go.dbus/_examples/notification.go

@@ -1,6 +1,6 @@
 package main
 
-import "github.com/guelfey/go.dbus"
+import "github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 
 func main() {
 	conn, err := dbus.SessionBus()

third_party/github.com/guelfey/go.dbus/_examples/prop.go

@@ -2,9 +2,9 @@ package main
 
 import (
 	"fmt"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
-	"github.com/guelfey/go.dbus/introspect"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus/introspect"
-	"github.com/guelfey/go.dbus/prop"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus/prop"
 	"os"
 )
 

third_party/github.com/guelfey/go.dbus/_examples/server.go

@@ -2,8 +2,8 @@ package main
 
 import (
 	"fmt"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
-	"github.com/guelfey/go.dbus/introspect"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus/introspect"
 	"os"
 )
 

third_party/github.com/guelfey/go.dbus/_examples/signal.go

@@ -2,7 +2,7 @@ package main
 
 import (
 	"fmt"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"os"
 )
 

third_party/github.com/guelfey/go.dbus/introspect/call.go

@@ -2,7 +2,7 @@ package introspect
 
 import (
 	"encoding/xml"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"strings"
 )
 

third_party/github.com/guelfey/go.dbus/introspect/introspectable.go

@@ -2,7 +2,7 @@ package introspect
 
 import (
 	"encoding/xml"
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"reflect"
 )
 

third_party/github.com/guelfey/go.dbus/prop/prop.go

@@ -3,8 +3,8 @@
 package prop
 
 import (
-	"github.com/guelfey/go.dbus"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
-	"github.com/guelfey/go.dbus/introspect"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus/introspect"
 	"sync"
 )
 

third_party/launchpad.net/goyaml/decode_test.go

@@ -2,7 +2,7 @@ package goyaml_test
 
 import (
 	. "launchpad.net/gocheck"
-	"launchpad.net/goyaml"
+	"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
 	"math"
 	"reflect"
 )

third_party/launchpad.net/goyaml/encode_test.go

@@ -3,7 +3,7 @@ package goyaml_test
 import (
 	"fmt"
 	. "launchpad.net/gocheck"
-	"launchpad.net/goyaml"
+	"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
 	"math"
 	"strconv"
 	"strings"