chore(coreos-cloudinit): bump to 0.7.1

container and panic fixes

.gitignore

@@ -1,3 +1,4 @@
 *.swp
 bin/
+coverage/
 pkg/

.travis.yml

@@ -0,0 +1,8 @@
+language: go
+go: 1.2
+
+install:
+ - go get code.google.com/p/go.tools/cmd/cover
+
+script:
+ - ./test

Documentation/cloud-config-oem.md

@@ -0,0 +1,37 @@
+## OEM configuration
+
+The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine. Customizing this section is only needed when generating a new OEM of CoreOS from the SDK. The fields include:
+
+- **id**: Lowercase string identifying the OEM
+- **name**: Human-friendly string representing the OEM
+- **version-id**: Lowercase string identifying the version of the OEM
+- **home-url**: Link to the homepage of the provider or OEM
+- **bug-report-url**: Link to a place to file bug reports about this OEM
+
+coreos-cloudinit renders these fields to `/etc/oem-release`.
+If no **id** field is provided, coreos-cloudinit will ignore this section.
+
+For example, the following cloud-config document...
+
+```
+#cloud-config
+coreos:
+  oem:
+    id: rackspace
+    name: Rackspace Cloud Servers
+    version-id: 168.0.0
+    home-url: https://www.rackspace.com/cloud/servers/
+    bug-report-url: https://github.com/coreos/coreos-overlay
+```
+
+...would be rendered to the following `/etc/oem-release`:
+
+```
+ID=rackspace
+NAME="Rackspace Cloud Servers"
+VERSION_ID=168.0.0
+HOME_URL="https://www.rackspace.com/cloud/servers/"
+BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
+```
+
+[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html

Documentation/cloud-config.md

@@ -70,50 +70,52 @@ Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
 
 [etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
 
-#### oem
+#### fleet
 
-The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine:
+The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
+```
+#cloud-config
 
-- **id**: Lowercase string identifying the OEM
+coreos:
-- **name**: Human-friendly string representing the OEM
+    fleet:
-- **version-id**: Lowercase string identifying the version of the OEM
+    	public-ip: $public_ipv4
-- **home-url**: Link to the homepage of the provider or OEM
+	metadata: region=us-west
-- **bug-report-url**: Link to a place to file bug reports about this OEM
+```
 
-coreos-cloudinit renders these fields to `/etc/oem-release`.
+...will generate a systemd unit drop-in like this:
-If no **id** field is provided, coreos-cloudinit will ignore this section.
+```
+[Service]
+Environment="FLEET_PUBLIC_IP=203.0.113.29"
+Environment="FLEET_METADATA=region=us-west"
+```
 
-For example, the following cloud-config document...
+For more information on fleet configuration, see the [fleet documentation][fleet-config].
+
+[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/configuration.md
+
+#### update
+
+The `coreos.update.*` parameters manipulate settings related to how CoreOS instances are updated.
+
+- **reboot-strategy**: One of "reboot", "etcd-lock", "best-effort" or "off" for controlling when reboots are issued after an update is performed.
+  - _reboot_: Reboot immediately after an update is applied.
+  - _etcd-lock_: Reboot after first taking a distributed lock in etcd, this guarantees that only one host will reboot concurrently and that the cluster will remain available during the update.
+  - _best-effort_ - If etcd is running, "etcd-lock", otherwise simply "reboot".
+  - _off_ - Disable rebooting after updates are applied (not recommended).
 
 ```
 #cloud-config
 coreos:
-  oem:
+  update:
-    id: rackspace
+    reboot-strategy: etcd-lock
-    name: Rackspace Cloud Servers
-    version-id: 168.0.0
-    home-url: https://www.rackspace.com/cloud/servers/
-    bug-report-url: https://github.com/coreos/coreos-overlay
 ```
 
-...would be rendered to the following `/etc/oem-release`:
-
-```
-ID=rackspace
-NAME="Rackspace Cloud Servers"
-VERSION_ID=168.0.0
-HOME_URL="https://www.rackspace.com/cloud/servers/"
-BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
-```
-
-[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html
-
 #### units
 
 The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields:
 
 - **name**: String representing unit's name. Required.
-- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` argument to `systemd enable`. Default value is false.
+- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemd enable`. Default value is false.
 - **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false.
 - **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
 - **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
@@ -146,7 +148,7 @@ coreos:
           WantedBy=local.target
 ```
 
-Start the builtin `etcd` and `fleet` services:
+Start the built-in `etcd` and `fleet` services:
 
 ```
 # cloud-config

README.md

@@ -1,4 +1,4 @@
-# coreos-cloudinit
+# coreos-cloudinit [![Build Status](https://travis-ci.org/coreos/coreos-cloudinit.png?branch=master)](https://travis-ci.org/coreos/coreos-cloudinit)
 
 coreos-cloudinit enables a user to customize CoreOS machines by providing either a cloud-config document or an executable script through user-data.
 

build

@@ -1,6 +1,9 @@
 #!/bin/bash -e
 
+ORG_PATH="github.com/coreos"
+REPO_PATH="${ORG_PATH}/coreos-cloudinit"
+
 export GOBIN=${PWD}/bin
 export GOPATH=${PWD}
 
-go build -o bin/coreos-cloudinit github.com/coreos/coreos-cloudinit
+go build -o bin/coreos-cloudinit ${REPO_PATH}

coreos-cloudinit.go

@@ -11,7 +11,7 @@ import (
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
-const version = "0.5.2"
+const version = "0.7.1"
 
 func main() {
 	var printVersion bool
@@ -42,11 +42,6 @@ func main() {
 		os.Exit(0)
 	}
 
-	if file != "" && url != "" && !useProcCmdline {
-		fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
-		os.Exit(1)
-	}
-
 	var ds datasource.Datasource
 	if file != "" {
 		ds = datasource.NewLocalFile(file)

cover

@@ -0,0 +1,27 @@
+#!/bin/bash -e
+#
+# Generate coverage HTML for a package
+# e.g. PKG=./initialize ./cover
+#
+
+if [ -z "$PKG" ]; then
+	echo "cover only works with a single package, sorry"
+	exit 255
+fi
+
+COVEROUT="coverage"
+
+if ! [ -d "$COVEROUT" ]; then
+	mkdir "$COVEROUT"
+fi
+
+# strip out slashes and dots
+COVERPKG=${PKG//\//}
+COVERPKG=${COVERPKG//./}
+
+# generate arg for "go test"
+export COVER="-coverprofile ${COVEROUT}/${COVERPKG}.out"
+
+source ./test
+
+go tool cover -html=${COVEROUT}/${COVERPKG}.out

datasource/datasource.go

@@ -1,31 +1,104 @@
 package datasource
 
 import (
+	"errors"
+	"fmt"
 	"io/ioutil"
+	"log"
+	"math"
+	"net"
 	"net/http"
+	neturl "net/url"
+	"strings"
+	"time"
+)
+
+const (
+	HTTP_2xx = 2
+	HTTP_4xx = 4
+
+	maxTimeout = time.Second * 5
+	maxRetries = 15
 )
 
 type Datasource interface {
 	Fetch() ([]byte, error)
-	Type()  string
+	Type() string
 }
 
-func fetchURL(url string) ([]byte, error) {
+// HTTP client timeout
-	client := http.Client{}
+// This one is low since exponential backoff will kick off too.
-	resp, err := client.Get(url)
+var timeout = time.Duration(2) * time.Second
+
+func dialTimeout(network, addr string) (net.Conn, error) {
+	deadline := time.Now().Add(timeout)
+	c, err := net.DialTimeout(network, addr, timeout)
 	if err != nil {
-		return []byte{}, err
+		return nil, err
 	}
-	defer resp.Body.Close()
+	c.SetDeadline(deadline)
+	return c, nil
+}
 
-	if resp.StatusCode / 100 != 2 {
+// Fetches user-data url with support for exponential backoff and maximum retries
-		return []byte{}, nil
+func fetchURL(rawurl string) ([]byte, error) {
+	if rawurl == "" {
+		return nil, errors.New("user-data URL is empty. Skipping.")
 	}
 
-	respBytes, err := ioutil.ReadAll(resp.Body)
+	url, err := neturl.Parse(rawurl)
 	if err != nil {
 		return nil, err
 	}
 
-	return respBytes, nil
+	// Unfortunately, url.Parse is too generic to throw errors if a URL does not
+	// have a valid HTTP scheme. So, we have to do this extra validation
+	if !strings.HasPrefix(url.Scheme, "http") {
+		return nil, fmt.Errorf("user-data URL %s does not have a valid HTTP scheme. Skipping.", rawurl)
+	}
+
+	userdataURL := url.String()
+
+	// We need to create our own client in order to add timeout support.
+	// TODO(c4milo) Replace it once Go 1.3 is officially used by CoreOS
+	// More info: https://code.google.com/p/go/source/detail?r=ada6f2d5f99f
+	transport := &http.Transport{
+		Dial: dialTimeout,
+	}
+
+	client := &http.Client{
+		Transport: transport,
+	}
+
+	for retry := 1; retry <= maxRetries; retry++ {
+		log.Printf("Fetching user-data from %s. Attempt #%d", userdataURL, retry)
+
+		resp, err := client.Get(userdataURL)
+
+		if err == nil {
+			defer resp.Body.Close()
+			status := resp.StatusCode / 100
+
+			if status == HTTP_2xx {
+				return ioutil.ReadAll(resp.Body)
+			}
+
+			if status == HTTP_4xx {
+				return nil, fmt.Errorf("user-data not found. HTTP status code: %d", resp.StatusCode)
+			}
+
+			log.Printf("user-data not found. HTTP status code: %d", resp.StatusCode)
+		} else {
+			log.Printf("unable to fetch user-data: %s", err.Error())
+		}
+
+		duration := time.Millisecond * time.Duration((math.Pow(float64(2), float64(retry)) * 100))
+		if duration > maxTimeout {
+			duration = maxTimeout
+		}
+
+		time.Sleep(duration)
+	}
+
+	return nil, fmt.Errorf("unable to fetch user-data. Maximum retries reached: %d", maxRetries)
 }

datasource/datasource_test.go

@@ -0,0 +1,119 @@
+package datasource
+
+import (
+	"fmt"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+var expBackoffTests = []struct {
+	count int
+	body  string
+}{
+	{0, "number of attempts: 0"},
+	{1, "number of attempts: 1"},
+	{2, "number of attempts: 2"},
+}
+
+// Test exponential backoff and that it continues retrying if a 5xx response is
+// received
+func TestFetchURLExpBackOff(t *testing.T) {
+	for i, tt := range expBackoffTests {
+		mux := http.NewServeMux()
+		count := 0
+		mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
+			if count == tt.count {
+				io.WriteString(w, fmt.Sprintf("number of attempts: %d", count))
+				return
+			}
+			count++
+			http.Error(w, "", 500)
+		})
+		ts := httptest.NewServer(mux)
+		defer ts.Close()
+
+		data, err := fetchURL(ts.URL)
+		if err != nil {
+			t.Errorf("Test case %d produced error: %v", i, err)
+		}
+
+		if count != tt.count {
+			t.Errorf("Test case %d failed: %d != %d", i, count, tt.count)
+		}
+
+		if string(data) != tt.body {
+			t.Errorf("Test case %d failed: %s != %s", i, tt.body, data)
+		}
+	}
+}
+
+// Test that it stops retrying if a 4xx response comes back
+func TestFetchURL4xx(t *testing.T) {
+	retries := 0
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		retries++
+		http.Error(w, "", 404)
+	}))
+	defer ts.Close()
+
+	_, err := fetchURL(ts.URL)
+	if err == nil {
+		t.Errorf("Incorrect result\ngot:  %s\nwant: %s", err.Error(), "user-data not found. HTTP status code: 404")
+	}
+
+	if retries > 1 {
+		t.Errorf("Number of retries:\n%d\nExpected number of retries:\n%s", retries, 1)
+	}
+}
+
+// Test that it fetches and returns user-data just fine
+func TestFetchURL2xx(t *testing.T) {
+	var cloudcfg = `
+#cloud-config
+coreos: 
+	oem:
+	    id: test
+	    name: CoreOS.box for Test
+	    version-id: %VERSION_ID%+%BUILD_ID%
+	    home-url: https://github.com/coreos/coreos-cloudinit
+	    bug-report-url: https://github.com/coreos/coreos-cloudinit
+	update:
+		reboot-strategy: best-effort
+`
+
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, cloudcfg)
+	}))
+	defer ts.Close()
+
+	data, err := fetchURL(ts.URL)
+	if err != nil {
+		t.Errorf("Incorrect result\ngot:  %v\nwant: %v", err, nil)
+	}
+
+	if string(data) != cloudcfg {
+		t.Errorf("Incorrect result\ngot:  %s\nwant: %s", string(data), cloudcfg)
+	}
+}
+
+// Test attempt to fetching using malformed URL
+func TestFetchURLMalformed(t *testing.T) {
+	var tests = []struct {
+		url  string
+		want string
+	}{
+		{"boo", "user-data URL boo does not have a valid HTTP scheme. Skipping."},
+		{"mailto://boo", "user-data URL mailto://boo does not have a valid HTTP scheme. Skipping."},
+		{"ftp://boo", "user-data URL ftp://boo does not have a valid HTTP scheme. Skipping."},
+		{"", "user-data URL is empty. Skipping."},
+	}
+
+	for _, test := range tests {
+		_, err := fetchURL(test.url)
+		if err == nil || err.Error() != test.want {
+			t.Errorf("Incorrect result\ngot:  %v\nwant: %v", err, test.want)
+		}
+	}
+}

initialize/config.go

@@ -1,6 +1,7 @@
 package initialize
 
 import (
+	"errors"
 	"fmt"
 	"log"
 	"path"
@@ -10,23 +11,122 @@ import (
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
+// CloudConfigFile represents a CoreOS specific configuration option that can generate
+// an associated system.File to be written to disk
+type CloudConfigFile interface {
+	// File should either return (*system.File, error), or (nil, nil) if nothing
+	// needs to be done for this configuration option.
+	File(root string) (*system.File, error)
+}
+
+// CloudConfigUnit represents a CoreOS specific configuration option that can generate
+// an associated system.Unit to be created/enabled appropriately
+type CloudConfigUnit interface {
+	// Unit should either return (*system.Unit, error), or (nil, nil) if nothing
+	// needs to be done for this configuration option.
+	Unit(root string) (*system.Unit, error)
+}
+
+// CloudConfig encapsulates the entire cloud-config configuration file and maps directly to YAML
 type CloudConfig struct {
 	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
 	Coreos            struct {
-		Etcd  EtcdEnvironment
+		Etcd   EtcdEnvironment
-		Units []system.Unit
+		Fleet  FleetEnvironment
-		OEM   OEMRelease
+		OEM    OEMRelease
+		Update UpdateConfig
+		Units  []system.Unit
 	}
 	WriteFiles     []system.File `yaml:"write_files"`
 	Hostname       string
 	Users          []system.User
-	ManageEtcHosts string `yaml:"manage_etc_hosts"`
+	ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
 }
 
+type warner func(format string, v ...interface{})
+
+// warnOnUnrecognizedKeys parses the contents of a cloud-config file and calls
+// warn(msg, key) for every unrecognized key (i.e. those not present in CloudConfig)
+func warnOnUnrecognizedKeys(contents string, warn warner) {
+	// Generate a map of all understood cloud config options
+	var cc map[string]interface{}
+	b, _ := goyaml.Marshal(&CloudConfig{})
+	goyaml.Unmarshal(b, &cc)
+
+	// Now unmarshal the entire provided contents
+	var c map[string]interface{}
+	goyaml.Unmarshal([]byte(contents), &c)
+
+	// Check that every key in the contents exists in the cloud config
+	for k, _ := range c {
+		if _, ok := cc[k]; !ok {
+			warn("Warning: unrecognized key %q in provided cloud config - ignoring section", k)
+		}
+	}
+
+	// Check for unrecognized coreos options, if any are set
+	coreos, ok := c["coreos"]
+	if ok {
+		set := coreos.(map[interface{}]interface{})
+		known := cc["coreos"].(map[interface{}]interface{})
+		for k, _ := range set {
+			key := k.(string)
+			if _, ok := known[key]; !ok {
+				warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", key)
+			}
+		}
+	}
+
+	// Check for any badly-specified users, if any are set
+	users, ok := c["users"]
+	if ok {
+		var known map[string]interface{}
+		b, _ := goyaml.Marshal(&system.User{})
+		goyaml.Unmarshal(b, &known)
+
+		set := users.([]interface{})
+		for _, u := range set {
+			user := u.(map[interface{}]interface{})
+			for k, _ := range user {
+				key := k.(string)
+				if _, ok := known[key]; !ok {
+					warn("Warning: unrecognized key %q in user section of cloud config - ignoring", key)
+				}
+			}
+		}
+	}
+
+	// Check for any badly-specified files, if any are set
+	files, ok := c["write_files"]
+	if ok {
+		var known map[string]interface{}
+		b, _ := goyaml.Marshal(&system.File{})
+		goyaml.Unmarshal(b, &known)
+
+		set := files.([]interface{})
+		for _, f := range set {
+			file := f.(map[interface{}]interface{})
+			for k, _ := range file {
+				key := k.(string)
+				if _, ok := known[key]; !ok {
+					warn("Warning: unrecognized key %q in file section of cloud config - ignoring", key)
+				}
+			}
+		}
+	}
+}
+
+// NewCloudConfig instantiates a new CloudConfig from the given contents (a
+// string of YAML), returning any error encountered. It will ignore unknown
+// fields but log encountering them.
 func NewCloudConfig(contents string) (*CloudConfig, error) {
 	var cfg CloudConfig
 	err := goyaml.Unmarshal([]byte(contents), &cfg)
-	return &cfg, err
+	if err != nil {
+		return &cfg, err
+	}
+	warnOnUnrecognizedKeys(contents, log.Printf)
+	return &cfg, nil
 }
 
 func (cc CloudConfig) String() string {
@@ -41,6 +141,9 @@ func (cc CloudConfig) String() string {
 	return stringified
 }
 
+// Apply renders a CloudConfig to an Environment. This can involve things like
+// configuring the hostname, adding new users, writing various configuration
+// files to disk, and manipulating systemd services.
 func Apply(cfg CloudConfig, env *Environment) error {
 	if cfg.Hostname != "" {
 		if err := system.SetHostname(cfg.Hostname); err != nil {
@@ -49,54 +152,45 @@ func Apply(cfg CloudConfig, env *Environment) error {
 		log.Printf("Set hostname to %s", cfg.Hostname)
 	}
 
-	if cfg.Coreos.OEM.ID != "" {
+	for _, user := range cfg.Users {
-		if err := WriteOEMRelease(&cfg.Coreos.OEM, env.Root()); err != nil {
+		if user.Name == "" {
-			return err
+			log.Printf("User object has no 'name' field, skipping")
+			continue
 		}
-		log.Printf("Wrote /etc/oem-release to filesystem")
-	}
 
-	if len(cfg.Users) > 0 {
+		if system.UserExists(&user) {
-		for _, user := range cfg.Users {
+			log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
-			if user.Name == "" {
+			if user.PasswordHash != "" {
-				log.Printf("User object has no 'name' field, skipping")
+				log.Printf("Setting '%s' user's password", user.Name)
-				continue
+				if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
+					log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
+					return err
+				}
 			}
+		} else {
+			log.Printf("Creating user '%s'", user.Name)
+			if err := system.CreateUser(&user); err != nil {
+				log.Printf("Failed creating user '%s': %v", user.Name, err)
+				return err
+			}
+		}
 
-			if system.UserExists(&user) {
+		if len(user.SSHAuthorizedKeys) > 0 {
-				log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
+			log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
-				if user.PasswordHash != "" {
+			if err := system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
-					log.Printf("Setting '%s' user's password", user.Name)
+				return err
-					if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
-						log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
-						return err
-					}
-				}
-			} else {
-				log.Printf("Creating user '%s'", user.Name)
-				if err := system.CreateUser(&user); err != nil {
-					log.Printf("Failed creating user '%s': %v", user.Name, err)
-					return err
-				}
 			}
-
+		}
-			if len(user.SSHAuthorizedKeys) > 0 {
+		if user.SSHImportGithubUser != "" {
-				log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
+			log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", user.SSHImportGithubUser, user.Name)
-				if err := system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
+			if err := SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
-					return err
+				return err
-				}
 			}
-			if user.SSHImportGithubUser != "" {
+		}
-				log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", user.SSHImportGithubUser, user.Name)
+		if user.SSHImportURL != "" {
-				if err := SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
+			log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
-					return err
+			if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
-				}
+				return err
-			}
-			if user.SSHImportURL != "" {
-				log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
-				if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
-					return err
-				}
 			}
 		}
 	}
@@ -110,79 +204,86 @@ func Apply(cfg CloudConfig, env *Environment) error {
 		}
 	}
 
-	if len(cfg.WriteFiles) > 0 {
+	for _, ccf := range []CloudConfigFile{cfg.Coreos.OEM, cfg.Coreos.Update, cfg.ManageEtcHosts} {
-		for _, file := range cfg.WriteFiles {
+		f, err := ccf.File(env.Root())
-			file.Path = path.Join(env.Root(), file.Path)
+		if err != nil {
-			if err := system.WriteFile(&file); err != nil {
+			return err
+		}
+		if f != nil {
+			cfg.WriteFiles = append(cfg.WriteFiles, *f)
+		}
+	}
+
+	for _, ccu := range []CloudConfigUnit{cfg.Coreos.Etcd, cfg.Coreos.Fleet, cfg.Coreos.Update} {
+		u, err := ccu.Unit(env.Root())
+		if err != nil {
+			return err
+		}
+		if u != nil {
+			cfg.Coreos.Units = append(cfg.Coreos.Units, *u)
+		}
+	}
+
+	for _, file := range cfg.WriteFiles {
+		file.Path = path.Join(env.Root(), file.Path)
+		if err := system.WriteFile(&file); err != nil {
+			return err
+		}
+		log.Printf("Wrote file %s to filesystem", file.Path)
+	}
+
+	commands := make(map[string]string, 0)
+	reload := false
+	for _, unit := range cfg.Coreos.Units {
+		dst := system.UnitDestination(&unit, env.Root())
+		if unit.Content != "" {
+			log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
+			if err := system.PlaceUnit(&unit, dst); err != nil {
 				return err
 			}
-			log.Printf("Wrote file %s to filesystem", file.Path)
+			log.Printf("Placed unit %s at %s", unit.Name, dst)
-		}
+			reload = true
-	}
-
-	if len(cfg.Coreos.Etcd) > 0 {
-		if err := WriteEtcdEnvironment(cfg.Coreos.Etcd, env.Root()); err != nil {
-			log.Fatalf("Failed to write etcd config to filesystem: %v", err)
 		}
 
-		log.Printf("Wrote etcd config file to filesystem")
+		if unit.Mask {
-	}
+			log.Printf("Masking unit file %s", unit.Name)
+			if err := system.MaskUnit(unit.Name, env.Root()); err != nil {
+				return err
+			}
+		}
 
-	if len(cfg.Coreos.Units) > 0 {
+		if unit.Enable {
-		commands := make(map[string]string, 0)
+			if unit.Group() != "network" {
-		for _, unit := range cfg.Coreos.Units {
+				log.Printf("Enabling unit file %s", dst)
-			dst := system.UnitDestination(&unit, env.Root())
+				if err := system.EnableUnitFile(dst, unit.Runtime); err != nil {
-			if unit.Content != "" {
-				log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
-				if err := system.PlaceUnit(&unit, dst); err != nil {
 					return err
 				}
-				log.Printf("Placed unit %s at %s", unit.Name, dst)
+				log.Printf("Enabled unit %s", unit.Name)
-			}
-
-			if unit.Enable {
-				if unit.Group() != "network" {
-					log.Printf("Enabling unit file %s", dst)
-					if err := system.EnableUnitFile(dst, unit.Runtime); err != nil {
-						return err
-					}
-					log.Printf("Enabled unit %s", unit.Name)
-				} else {
-					log.Printf("Skipping enable for network-like unit %s", unit.Name)
-				}
-			}
-
-			if unit.Group() == "network" {
-				commands["systemd-networkd.service"] = "restart"
 			} else {
-				if unit.Command != "" {
+				log.Printf("Skipping enable for network-like unit %s", unit.Name)
-					commands[unit.Name] = unit.Command
-				}
 			}
 		}
 
-		if err := system.DaemonReload(); err != nil {
+		if unit.Group() == "network" {
-			log.Fatalf("Failed systemd daemon-reload: %v", err)
+			commands["systemd-networkd.service"] = "restart"
-		}
+		} else if unit.Command != "" {
-
+			commands[unit.Name] = unit.Command
-		for unit, command := range commands {
-			log.Printf("Calling unit command '%s %s'", command, unit)
-			res, err := system.RunUnitCommand(command, unit)
-			if err != nil {
-				return err
-			}
-			log.Printf("Result of '%s %s': %s", command, unit, res)
 		}
 	}
 
-	if cfg.ManageEtcHosts != "" {
+	if reload {
-
+		if err := system.DaemonReload(); err != nil {
-		if err := WriteEtcHosts(cfg.ManageEtcHosts, env.Root()); err != nil {
+			return errors.New(fmt.Sprintf("failed systemd daemon-reload: %v", err))
-			log.Fatalf("Failed to write /etc/hosts to filesystem: %v", err)
 		}
+	}
 
-		log.Printf("Wrote /etc/hosts file to filesystem")
+	for unit, command := range commands {
-
+		log.Printf("Calling unit command '%s %s'", command, unit)
+		res, err := system.RunUnitCommand(command, unit)
+		if err != nil {
+			return err
+		}
+		log.Printf("Result of '%s %s': %s", command, unit, res)
 	}
 
 	return nil

initialize/config_test.go

@@ -1,10 +1,75 @@
 package initialize
 
 import (
+	"fmt"
 	"strings"
 	"testing"
 )
 
+func TestCloudConfigUnknownKeys(t *testing.T) {
+	contents := `
+coreos: 
+  etcd:
+    discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
+  coreos_unknown:
+    foo: "bar"
+section_unknown:
+  dunno:
+    something
+bare_unknown:
+  bar
+write_files:
+  - content: fun
+    path: /var/party
+    file_unknown: nofun
+users:
+  - name: fry
+    passwd: somehash
+    user_unknown: philip
+hostname:
+  foo
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
+	}
+	if cfg.Hostname != "foo" {
+		t.Fatalf("hostname not correctly set when invalid keys are present")
+	}
+	if len(cfg.Coreos.Etcd) < 1 {
+		t.Fatalf("etcd section not correctly set when invalid keys are present")
+	}
+	if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
+		t.Fatalf("write_files section not correctly set when invalid keys are present")
+	}
+	if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
+		t.Fatalf("users section not correctly set when invalid keys are present")
+	}
+
+	var warnings string
+	catchWarn := func(f string, v ...interface{}) {
+		warnings += fmt.Sprintf(f, v...)
+	}
+
+	warnOnUnrecognizedKeys(contents, catchWarn)
+
+	if !strings.Contains(warnings, "coreos_unknown") {
+		t.Errorf("warnings did not catch unrecognized coreos option coreos_unknown")
+	}
+	if !strings.Contains(warnings, "bare_unknown") {
+		t.Errorf("warnings did not catch unrecognized key bare_unknown")
+	}
+	if !strings.Contains(warnings, "section_unknown") {
+		t.Errorf("warnings did not catch unrecognized key section_unknown")
+	}
+	if !strings.Contains(warnings, "user_unknown") {
+		t.Errorf("warnings did not catch unrecognized user key user_unknown")
+	}
+	if !strings.Contains(warnings, "file_unknown") {
+		t.Errorf("warnings did not catch unrecognized file key file_unknown")
+	}
+}
+
 // Assert that the parsing of a cloud config file "generally works"
 func TestCloudConfigEmpty(t *testing.T) {
 	cfg, err := NewCloudConfig("")
@@ -32,6 +97,8 @@ func TestCloudConfig(t *testing.T) {
 coreos: 
   etcd:
     discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
+  update:
+    reboot-strategy: reboot
   units:
     - name: 50-eth0.network
       runtime: yes
@@ -129,6 +196,9 @@ Address=10.209.171.177/19
 	if cfg.Hostname != "trontastic" {
 		t.Errorf("Failed to parse hostname")
 	}
+	if cfg.Coreos.Update["reboot-strategy"] != "reboot" {
+		t.Errorf("Failed to parse locksmith strategy")
+	}
 }
 
 // Assert that our interface conversion doesn't panic
@@ -139,7 +209,7 @@ ssh_authorized_keys:
 `
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
-		t.Fatalf("Encountered unexpected error :%v", err)
+		t.Fatalf("Encountered unexpected error: %v", err)
 	}
 
 	keys := cfg.SSHAuthorizedKeys
@@ -157,6 +227,26 @@ func TestCloudConfigSerializationHeader(t *testing.T) {
 	}
 }
 
+// TestDropInIgnored asserts that users are unable to set DropIn=True on units
+func TestDropInIgnored(t *testing.T) {
+	contents := `
+coreos:
+  units:
+    - name: test
+      dropin: true
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil || len(cfg.Coreos.Units) != 1 {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+	if len(cfg.Coreos.Units) != 1 || cfg.Coreos.Units[0].Name != "test" {
+		t.Fatalf("Expected 1 unit, but got %d: %v", len(cfg.Coreos.Units), cfg.Coreos.Units)
+	}
+	if cfg.Coreos.Units[0].DropIn {
+		t.Errorf("dropin option on unit in cloud-config was not ignored!")
+	}
+}
+
 func TestCloudConfigUsers(t *testing.T) {
 	contents := `
 users:

initialize/env.go

@@ -45,3 +45,16 @@ func (self *Environment) Apply(data string) string {
 	}
 	return data
 }
+
+// normalizeSvcEnv standardizes the keys of the map (environment variables for a service)
+// by replacing any dashes with underscores and ensuring they are entirely upper case.
+// For example, "some-env" --> "SOME_ENV"
+func normalizeSvcEnv(m map[string]string) map[string]string {
+	out := make(map[string]string, len(m))
+	for key, val := range m {
+		key = strings.ToUpper(key)
+		key = strings.Replace(key, "-", "_", -1)
+		out[key] = val
+	}
+	return out
+}

initialize/etcd.go

@@ -3,26 +3,14 @@ package initialize
 import (
 	"errors"
 	"fmt"
-	"path"
-	"strings"
 
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
 type EtcdEnvironment map[string]string
 
-func (ec EtcdEnvironment) normalized() map[string]string {
+func (ee EtcdEnvironment) String() (out string) {
-	out := make(map[string]string, len(ec))
+	norm := normalizeSvcEnv(ee)
-	for key, val := range ec {
-		key = strings.ToUpper(key)
-		key = strings.Replace(key, "-", "_", -1)
-		out[key] = val
-	}
-	return out
-}
-
-func (ec EtcdEnvironment) String() (out string) {
-	norm := ec.normalized()
 
 	if val, ok := norm["DISCOVERY_URL"]; ok {
 		delete(norm, "DISCOVERY_URL")
@@ -40,23 +28,27 @@ func (ec EtcdEnvironment) String() (out string) {
 	return
 }
 
-// Write an EtcdEnvironment to the appropriate path on disk for etcd.service
+// Unit creates a Unit file drop-in for etcd, using any configured
-func WriteEtcdEnvironment(env EtcdEnvironment, root string) error {
+// options and adding a default MachineID if unset.
-	if _, ok := env["name"]; !ok {
+func (ee EtcdEnvironment) Unit(root string) (*system.Unit, error) {
+	if ee == nil {
+		return nil, nil
+	}
+
+	if _, ok := ee["name"]; !ok {
 		if machineID := system.MachineID(root); machineID != "" {
-			env["name"] = machineID
+			ee["name"] = machineID
 		} else if hostname, err := system.Hostname(); err == nil {
-			env["name"] = hostname
+			ee["name"] = hostname
 		} else {
-			return errors.New("Unable to determine default etcd name")
+			return nil, errors.New("Unable to determine default etcd name")
 		}
 	}
 
-	file := system.File{
+	return &system.Unit{
-		Path: path.Join(root, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf"),
+		Name:    "etcd.service",
-		RawFilePermissions: "0644",
+		Runtime: true,
-		Content: env.String(),
+		DropIn:  true,
-	}
+		Content: ee.String(),
-
+	}, nil
-	return system.WriteFile(&file)
 }

initialize/etcd_test.go

@@ -3,10 +3,10 @@ package initialize
 import (
 	"io/ioutil"
 	"os"
-	"os/exec"
 	"path"
-	"syscall"
 	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
 )
 
 func TestEtcdEnvironment(t *testing.T) {
@@ -60,18 +60,28 @@ Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
 
 func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
 	ec := EtcdEnvironment{
-		"name": "node001",
+		"name":           "node001",
-		"discovery": "http://disco.example.com/foobar",
+		"discovery":      "http://disco.example.com/foobar",
 		"peer-bind-addr": "127.0.0.1:7002",
 	}
 	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
-	if err := WriteEtcdEnvironment(ec, dir); err != nil {
+	u, err := ec.Unit(dir)
-		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	if err != nil {
+		t.Fatalf("Generating etcd unit failed: %v", err)
+	}
+	if u == nil {
+		t.Fatalf("Returned nil etcd unit unexpectedly")
+	}
+
+	dst := system.UnitDestination(u, dir)
+	os.Stderr.WriteString("writing to " + dir + "\n")
+	if err := system.PlaceUnit(u, dst); err != nil {
+		t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
 	}
 
 	fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
@@ -101,12 +111,12 @@ Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
 }
 
 func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
-	ec := EtcdEnvironment{}
+	ee := EtcdEnvironment{}
 	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
 	os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
 	err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
@@ -114,8 +124,18 @@ func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
 		t.Fatalf("Failed writing out /etc/machine-id: %v", err)
 	}
 
-	if err := WriteEtcdEnvironment(ec, dir); err != nil {
+	u, err := ee.Unit(dir)
-		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	if err != nil {
+		t.Fatalf("Generating etcd unit failed: %v", err)
+	}
+	if u == nil {
+		t.Fatalf("Returned nil etcd unit unexpectedly")
+	}
+
+	dst := system.UnitDestination(u, dir)
+	os.Stderr.WriteString("writing to " + dir + "\n")
+	if err := system.PlaceUnit(u, dst); err != nil {
+		t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
 	}
 
 	fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
@@ -133,7 +153,14 @@ Environment="ETCD_NAME=node007"
 	}
 }
 
-func rmdir(path string) error {
+func TestEtcdEnvironmentWhenNil(t *testing.T) {
-    cmd := exec.Command("rm", "-rf", path)
+	// EtcdEnvironment will be a nil map if it wasn't in the yaml
-    return cmd.Run()
+	var ee EtcdEnvironment
+	if ee != nil {
+		t.Fatalf("EtcdEnvironment is not nil")
+	}
+	u, err := ee.Unit("")
+	if u != nil || err != nil {
+		t.Fatalf("Unit returned a non-nil value for nil input")
+	}
 }

initialize/fleet.go

@@ -0,0 +1,34 @@
+package initialize
+
+import (
+	"fmt"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type FleetEnvironment map[string]string
+
+func (fe FleetEnvironment) String() (out string) {
+	norm := normalizeSvcEnv(fe)
+	out += "[Service]\n"
+
+	for key, val := range norm {
+		out += fmt.Sprintf("Environment=\"FLEET_%s=%s\"\n", key, val)
+	}
+
+	return
+}
+
+// Unit generates a Unit file drop-in for fleet, if any fleet options were
+// configured in cloud-config
+func (fe FleetEnvironment) Unit(root string) (*system.Unit, error) {
+	if len(fe) < 1 {
+		return nil, nil
+	}
+	return &system.Unit{
+		Name:    "fleet.service",
+		Runtime: true,
+		DropIn:  true,
+		Content: fe.String(),
+	}, nil
+}

initialize/fleet_test.go

@@ -0,0 +1,42 @@
+package initialize
+
+import "testing"
+
+func TestFleetEnvironment(t *testing.T) {
+	cfg := make(FleetEnvironment, 0)
+	cfg["public-ip"] = "12.34.56.78"
+
+	env := cfg.String()
+
+	expect := `[Service]
+Environment="FLEET_PUBLIC_IP=12.34.56.78"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestFleetUnit(t *testing.T) {
+	cfg := make(FleetEnvironment, 0)
+	u, err := cfg.Unit("/")
+	if u != nil {
+		t.Errorf("unexpectedly generated unit with empty FleetEnvironment")
+	}
+
+	cfg["public-ip"] = "12.34.56.78"
+
+	u, err = cfg.Unit("/")
+	if err != nil {
+		t.Errorf("error generating fleet unit: %v", err)
+	}
+	if u == nil {
+		t.Fatalf("unexpectedly got nil unit generating fleet unit!")
+	}
+	if !u.Runtime {
+		t.Errorf("bad Runtime for generated fleet unit!")
+	}
+	if !u.DropIn {
+		t.Errorf("bad DropIn for generated fleet unit!")
+	}
+}

initialize/manage_etc_hosts.go

@@ -11,8 +11,10 @@ import (
 
 const DefaultIpv4Address = "127.0.0.1"
 
-func generateEtcHosts(option string) (out string, err error) {
+type EtcHosts string
-	if option != "localhost" {
+
+func (eh EtcHosts) generateEtcHosts() (out string, err error) {
+	if eh != "localhost" {
 		return "", errors.New("Invalid option to manage_etc_hosts")
 	}
 
@@ -26,19 +28,19 @@ func generateEtcHosts(option string) (out string, err error) {
 
 }
 
-// Write an /etc/hosts file
+func (eh EtcHosts) File(root string) (*system.File, error) {
-func WriteEtcHosts(option string, root string) error {
+	if eh == "" {
-
+		return nil, nil
-	etcHosts, err := generateEtcHosts(option)
-	if err != nil {
-		return err
 	}
 
-	file := system.File{
+	etcHosts, err := eh.generateEtcHosts()
-		Path:               path.Join(root, "etc", "hosts"),
+	if err != nil {
+		return nil, err
+	}
+
+	return &system.File{
+		Path:               path.Join("etc", "hosts"),
 		RawFilePermissions: "0644",
 		Content:            etcHosts,
-	}
+	}, nil
-
-	return system.WriteFile(&file)
 }

initialize/manage_etc_hosts_test.go

@@ -6,6 +6,8 @@ import (
 	"os"
 	"path"
 	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
 )
 
 func TestCloudConfigManageEtcHosts(t *testing.T) {
@@ -25,14 +27,9 @@ manage_etc_hosts: localhost
 }
 
 func TestManageEtcHostsInvalidValue(t *testing.T) {
-	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	eh := EtcHosts("invalid")
-	if err != nil {
+	if f, err := eh.File(""); err == nil || f != nil {
-		t.Fatalf("Unable to create tempdir: %v", err)
+		t.Fatalf("EtcHosts File succeeded with invalid value!")
-	}
-	defer rmdir(dir)
-
-	if err := WriteEtcHosts("invalid", dir); err == nil {
-		t.Fatalf("WriteEtcHosts succeeded with invalid value: %v", err)
 	}
 }
 
@@ -41,10 +38,22 @@ func TestEtcHostsWrittenToDisk(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer rmdir(dir)
+	defer os.RemoveAll(dir)
 
-	if err := WriteEtcHosts("localhost", dir); err != nil {
+	eh := EtcHosts("localhost")
-		t.Fatalf("WriteEtcHosts failed: %v", err)
+
+	f, err := eh.File(dir)
+	if err != nil {
+		t.Fatalf("Error calling File on EtcHosts: %v", err)
+	}
+	if f == nil {
+		t.Fatalf("manageEtcHosts returned nil file unexpectedly")
+	}
+
+	f.Path = path.Join(dir, f.Path)
+
+	if err := system.WriteFile(f); err != nil {
+		t.Fatalf("Error writing EtcHosts: %v", err)
 	}
 
 	fullPath := path.Join(dir, "etc", "hosts")

initialize/oem.go

@@ -16,7 +16,7 @@ type OEMRelease struct {
 	BugReportURL string `yaml:"bug-report-url"`
 }
 
-func (oem *OEMRelease) String() string {
+func (oem OEMRelease) String() string {
 	fields := []string{
 		fmt.Sprintf("ID=%s", oem.ID),
 		fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
@@ -28,12 +28,14 @@ func (oem *OEMRelease) String() string {
 	return strings.Join(fields, "\n") + "\n"
 }
 
-func WriteOEMRelease(oem *OEMRelease, root string) error {
+func (oem OEMRelease) File(root string) (*system.File, error) {
-	file := system.File{
+	if oem.ID == "" {
-		Path:               path.Join(root, "etc", "oem-release"),
+		return nil, nil
-		RawFilePermissions: "0644",
-		Content:            oem.String(),
 	}
 
-	return system.WriteFile(&file)
+	return &system.File{
+		Path:               path.Join("etc", "oem-release"),
+		RawFilePermissions: "0644",
+		Content:            oem.String(),
+	}, nil
 }

initialize/oem_test.go

@@ -4,8 +4,9 @@ import (
 	"io/ioutil"
 	"os"
 	"path"
-	"syscall"
 	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
 )
 
 func TestOEMReleaseWrittenToDisk(t *testing.T) {
@@ -20,10 +21,19 @@ func TestOEMReleaseWrittenToDisk(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
-	if err := WriteOEMRelease(&oem, dir); err != nil {
+	f, err := oem.File(dir)
-		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	if err != nil {
+		t.Fatalf("Processing of OEMRelease failed: %v", err)
+	}
+	if f == nil {
+		t.Fatalf("OEMRelease returned nil file unexpectedly")
+	}
+
+	f.Path = path.Join(dir, f.Path)
+	if err := system.WriteFile(f); err != nil {
+		t.Fatalf("Writing of OEMRelease failed: %v", err)
 	}
 
 	fullPath := path.Join(dir, "etc", "oem-release")

initialize/ssh_keys.go

@@ -26,10 +26,10 @@ func SSHImportKeysFromURL(system_user string, url string) error {
 
 func fetchUserKeys(url string) ([]string, error) {
 	res, err := http.Get(url)
-	defer res.Body.Close()
 	if err != nil {
 		return nil, err
 	}
+	defer res.Body.Close()
 	body, err := ioutil.ReadAll(res.Body)
 	if err != nil {
 		return nil, err

initialize/update.go

@@ -0,0 +1,151 @@
+package initialize
+
+import (
+	"bufio"
+	"errors"
+	"fmt"
+	"os"
+	"path"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const (
+	locksmithUnit = "locksmithd.service"
+)
+
+// updateOption represents a configurable update option, which, if set, will be
+// written into update.conf, replacing any existing value for the option
+type updateOption struct {
+	key    string   // key used to configure this option in cloud-config
+	valid  []string // valid values for the option
+	prefix string   // prefix for the option in the update.conf file
+	value  string   // used to store the new value in update.conf (including prefix)
+	seen   bool     // whether the option has been seen in any existing update.conf
+}
+
+// updateOptions defines the update options understood by cloud-config.
+// The keys represent the string used in cloud-config to configure the option.
+var updateOptions = []*updateOption{
+	&updateOption{
+		key:    "reboot-strategy",
+		prefix: "REBOOT_STRATEGY=",
+		valid:  []string{"best-effort", "etcd-lock", "reboot", "off"},
+	},
+	&updateOption{
+		key:    "group",
+		prefix: "GROUP=",
+		valid:  []string{"master", "beta", "alpha", "stable"},
+	},
+	&updateOption{
+		key:    "server",
+		prefix: "SERVER=",
+	},
+}
+
+// isValid checks whether a supplied value is valid for this option
+func (uo updateOption) isValid(val string) bool {
+	if len(uo.valid) == 0 {
+		return true
+	}
+	for _, v := range uo.valid {
+		if val == v {
+			return true
+		}
+	}
+	return false
+}
+
+type UpdateConfig map[string]string
+
+// File generates an `/etc/coreos/update.conf` file (if any update
+// configuration options are set in cloud-config) by either rewriting the
+// existing file on disk, or starting from `/usr/share/coreos/update.conf`
+func (uc UpdateConfig) File(root string) (*system.File, error) {
+	if len(uc) < 1 {
+		return nil, nil
+	}
+
+	var out string
+
+	// Generate the list of possible substitutions to be performed based on the options that are configured
+	subs := make([]*updateOption, 0)
+	for _, uo := range updateOptions {
+		val, ok := uc[uo.key]
+		if !ok {
+			continue
+		}
+		if !uo.isValid(val) {
+			return nil, errors.New(fmt.Sprintf("invalid value %v for option %v (valid options: %v)", val, uo.key, uo.valid))
+		}
+		uo.value = uo.prefix + val
+		subs = append(subs, uo)
+	}
+
+	etcUpdate := path.Join(root, "etc", "coreos", "update.conf")
+	usrUpdate := path.Join(root, "usr", "share", "coreos", "update.conf")
+
+	conf, err := os.Open(etcUpdate)
+	if os.IsNotExist(err) {
+		conf, err = os.Open(usrUpdate)
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	scanner := bufio.NewScanner(conf)
+
+	for scanner.Scan() {
+		line := scanner.Text()
+		for _, s := range subs {
+			if strings.HasPrefix(line, s.prefix) {
+				line = s.value
+				s.seen = true
+				break
+			}
+		}
+		out += line
+		out += "\n"
+		if err := scanner.Err(); err != nil {
+			return nil, err
+		}
+	}
+
+	for _, s := range subs {
+		if !s.seen {
+			out += s.value
+			out += "\n"
+		}
+	}
+
+	return &system.File{
+		Path:               path.Join("etc", "coreos", "update.conf"),
+		RawFilePermissions: "0644",
+		Content:            out,
+	}, nil
+}
+
+// GetUnit generates a locksmith system.Unit, if reboot-strategy was set in
+// cloud-config, for the cloud-init initializer to act on appropriately
+func (uc UpdateConfig) Unit(root string) (*system.Unit, error) {
+	strategy, ok := uc["reboot-strategy"]
+	if !ok {
+		return nil, nil
+	}
+
+	u := &system.Unit{
+		Name:    locksmithUnit,
+		Enable:  true,
+		Command: "restart",
+		Mask:    false,
+	}
+
+	if strategy == "off" {
+		u.Enable = false
+		u.Command = "stop"
+		u.Mask = true
+	}
+
+	return u, nil
+}

initialize/update_test.go

@@ -0,0 +1,217 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"sort"
+	"strings"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const (
+	base = `SERVER=https://example.com
+GROUP=thegroupc`
+	configured = base + `
+REBOOT_STRATEGY=awesome
+`
+	expected = base + `
+REBOOT_STRATEGY=etcd-lock
+`
+)
+
+func setupFixtures(dir string) {
+	os.MkdirAll(path.Join(dir, "usr", "share", "coreos"), 0755)
+	os.MkdirAll(path.Join(dir, "run", "systemd", "system"), 0755)
+
+	ioutil.WriteFile(path.Join(dir, "usr", "share", "coreos", "update.conf"), []byte(base), 0644)
+}
+
+func TestEmptyUpdateConfig(t *testing.T) {
+	uc := &UpdateConfig{}
+	f, err := uc.File("")
+	if err != nil {
+		t.Error("unexpected error getting file from empty UpdateConfig")
+	}
+	if f != nil {
+		t.Errorf("getting file from empty UpdateConfig should have returned nil, got %v", f)
+	}
+	u, err := uc.Unit("")
+	if err != nil {
+		t.Error("unexpected error getting unit from empty UpdateConfig")
+	}
+	if u != nil {
+		t.Errorf("getting unit from empty UpdateConfig should have returned nil, got %v", u)
+	}
+}
+
+func TestInvalidUpdateOptions(t *testing.T) {
+	uon := &updateOption{
+		key:    "numbers",
+		prefix: "numero_",
+		valid:  []string{"one", "two"},
+	}
+	uoa := &updateOption{
+		key:    "any_will_do",
+		prefix: "any_",
+	}
+
+	if !uon.isValid("one") {
+		t.Error("update option did not accept valid option \"one\"")
+	}
+	if uon.isValid("three") {
+		t.Error("update option accepted invalid option \"three\"")
+	}
+	for _, s := range []string{"one", "asdf", "foobarbaz"} {
+		if !uoa.isValid(s) {
+			t.Errorf("update option with no \"valid\" field did not accept %q", s)
+		}
+	}
+
+	uc := &UpdateConfig{"reboot-strategy": "wizzlewazzle"}
+	f, err := uc.File("")
+	if err == nil {
+		t.Errorf("File did not give an error on invalid UpdateOption")
+	}
+	if f != nil {
+		t.Errorf("File did not return a nil file on invalid UpdateOption")
+	}
+}
+
+func TestServerGroupOptions(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	setupFixtures(dir)
+	u := &UpdateConfig{"group": "master", "server": "http://foo.com"}
+
+	want := `
+GROUP=master
+SERVER=http://foo.com`
+
+	f, err := u.File(dir)
+	if err != nil {
+		t.Errorf("unexpected error getting file from UpdateConfig: %v", err)
+	} else if f == nil {
+		t.Error("unexpectedly got empty file from UpdateConfig")
+	} else {
+		out := strings.Split(f.Content, "\n")
+		sort.Strings(out)
+		got := strings.Join(out, "\n")
+		if got != want {
+			t.Errorf("File has incorrect contents, got %v, want %v", got, want)
+		}
+	}
+}
+
+func TestRebootStrategies(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	setupFixtures(dir)
+	strategies := []struct {
+		name     string
+		line     string
+		uMask    bool
+		uCommand string
+	}{
+		{"best-effort", "REBOOT_STRATEGY=best-effort", false, "restart"},
+		{"etcd-lock", "REBOOT_STRATEGY=etcd-lock", false, "restart"},
+		{"reboot", "REBOOT_STRATEGY=reboot", false, "restart"},
+		{"off", "REBOOT_STRATEGY=off", true, "stop"},
+	}
+	for _, s := range strategies {
+		uc := &UpdateConfig{"reboot-strategy": s.name}
+		f, err := uc.File(dir)
+		if err != nil {
+			t.Errorf("update failed to generate file for reboot-strategy=%v: %v", s.name, err)
+		} else if f == nil {
+			t.Errorf("generated empty file for reboot-strategy=%v", s.name)
+		} else {
+			seen := false
+			for _, line := range strings.Split(f.Content, "\n") {
+				if line == s.line {
+					seen = true
+					break
+				}
+			}
+			if !seen {
+				t.Errorf("couldn't find expected line %v for reboot-strategy=%v", s.line)
+			}
+		}
+		u, err := uc.Unit(dir)
+		if err != nil {
+			t.Errorf("failed to generate unit for reboot-strategy=%v!", s.name)
+		} else if u == nil {
+			t.Errorf("generated empty unit for reboot-strategy=%v", s.name)
+		} else {
+			if u.Name != locksmithUnit {
+				t.Errorf("unit generated for reboot strategy=%v had bad name: %v", s.name, u.Name)
+			}
+			if u.Mask != s.uMask {
+				t.Errorf("unit generated for reboot strategy=%v had bad mask: %t", s.name, u.Mask)
+			}
+			if u.Command != s.uCommand {
+				t.Errorf("unit generated for reboot strategy=%v had bad command: %v", s.name, u.Command)
+			}
+		}
+	}
+
+}
+
+func TestUpdateConfWrittenToDisk(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	setupFixtures(dir)
+
+	for i := 0; i < 2; i++ {
+		if i == 1 {
+			err = ioutil.WriteFile(path.Join(dir, "etc", "coreos", "update.conf"), []byte(configured), 0644)
+			if err != nil {
+				t.Fatal(err)
+			}
+		}
+		uc := &UpdateConfig{"reboot-strategy": "etcd-lock"}
+
+		f, err := uc.File(dir)
+		if err != nil {
+			t.Fatalf("Processing UpdateConfig failed: %v", err)
+		} else if f == nil {
+			t.Fatal("Unexpectedly got nil updateconfig file")
+		}
+
+		f.Path = path.Join(dir, f.Path)
+		if err := system.WriteFile(f); err != nil {
+			t.Fatalf("Error writing update config: %v", err)
+		}
+
+		fullPath := path.Join(dir, "etc", "coreos", "update.conf")
+
+		fi, err := os.Stat(fullPath)
+		if err != nil {
+			t.Fatalf("Unable to stat file: %v", err)
+		}
+
+		if fi.Mode() != os.FileMode(0644) {
+			t.Errorf("File has incorrect mode: %v", fi.Mode())
+		}
+
+		contents, err := ioutil.ReadFile(fullPath)
+		if err != nil {
+			t.Fatalf("Unable to read expected file: %v", err)
+		}
+
+		if string(contents) != expected {
+			t.Fatalf("File has incorrect contents, got %v, wanted %v", string(contents), expected)
+		}
+	}
+}

system/file.go

@@ -11,10 +11,10 @@ import (
 )
 
 type File struct {
-	Encoding    string
+	Encoding           string
-	Content     string
+	Content            string
-	Owner       string
+	Owner              string
-	Path        string
+	Path               string
 	RawFilePermissions string `yaml:"permissions"`
 }
 
@@ -31,7 +31,6 @@ func (f *File) Permissions() (os.FileMode, error) {
 	return os.FileMode(perm), nil
 }
 
-
 func WriteFile(f *File) error {
 	if f.Encoding != "" {
 		return fmt.Errorf("Unable to write file with encoding %s", f.Encoding)

system/systemd.go

@@ -17,12 +17,21 @@ import (
 // never be used as a true MachineID
 const fakeMachineID = "42000000000000000000000000000042"
 
+// Name for drop-in service configuration files created by cloudconfig
+const cloudConfigDropIn = "20-cloudinit.conf"
+
 type Unit struct {
 	Name    string
+	Mask    bool
 	Enable  bool
 	Runtime bool
 	Content string
 	Command string
+
+	// For drop-in units, a cloudinit.conf is generated.
+	// This is currently unbound in YAML (and hence unsettable in cloud-config files)
+	// until the correct behaviour for multiple drop-in units is determined.
+	DropIn bool `yaml:"-"`
 }
 
 func (u *Unit) Type() string {
@@ -42,8 +51,8 @@ func (u *Unit) Group() (group string) {
 
 type Script []byte
 
-// UnitDestination builds the appropriate absolte file path for
+// UnitDestination builds the appropriate absolute file path for
-// the given unit. The root argument indicates the effective base
+// the given Unit. The root argument indicates the effective base
 // directory of the system (similar to a chroot).
 func UnitDestination(u *Unit, root string) string {
 	dir := "etc"
@@ -51,7 +60,11 @@ func UnitDestination(u *Unit, root string) string {
 		dir = "run"
 	}
 
-	return path.Join(root, dir, "systemd", u.Group(), u.Name)
+	if u.DropIn {
+		return path.Join(root, dir, "systemd", u.Group(), fmt.Sprintf("%s.d", u.Name), cloudConfigDropIn)
+	} else {
+		return path.Join(root, dir, "systemd", u.Group(), u.Name)
+	}
 }
 
 // PlaceUnit writes a unit file at the provided destination, creating
@@ -65,8 +78,8 @@ func PlaceUnit(u *Unit, dst string) error {
 	}
 
 	file := File{
-		Path: dst,
+		Path:               dst,
-		Content: u.Content,
+		Content:            u.Content,
 		RawFilePermissions: "0644",
 	}
 
@@ -165,3 +178,11 @@ func MachineID(root string) string {
 
 	return id
 }
+
+func MaskUnit(unit string, root string) error {
+	masked := path.Join(root, "etc", "systemd", "system", unit)
+	if err := os.MkdirAll(path.Dir(masked), os.FileMode(0755)); err != nil {
+		return err
+	}
+	return os.Symlink("/dev/null", masked)
+}

system/systemd_test.go

@@ -4,15 +4,14 @@ import (
 	"io/ioutil"
 	"os"
 	"path"
-	"syscall"
 	"testing"
 )
 
 func TestPlaceNetworkUnit(t *testing.T) {
 	u := Unit{
-		Name: "50-eth0.network",
+		Name:    "50-eth0.network",
-      Runtime: true,
+		Runtime: true,
-      Content: `[Match]
+		Content: `[Match]
 Name=eth47
 
 [Network]
@@ -24,7 +23,7 @@ Address=10.209.171.177/19
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
 	dst := UnitDestination(&u, dir)
 	expectDst := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
@@ -61,11 +60,35 @@ Address=10.209.171.177/19
 	}
 }
 
+func TestUnitDestination(t *testing.T) {
+	dir := "/some/dir"
+	name := "foobar.service"
+
+	u := Unit{
+		Name:   name,
+		DropIn: false,
+	}
+
+	dst := UnitDestination(&u, dir)
+	expectDst := path.Join(dir, "etc", "systemd", "system", "foobar.service")
+	if dst != expectDst {
+		t.Errorf("UnitDestination returned %s, expected %s", dst, expectDst)
+	}
+
+	u.DropIn = true
+
+	dst = UnitDestination(&u, dir)
+	expectDst = path.Join(dir, "etc", "systemd", "system", "foobar.service.d", cloudConfigDropIn)
+	if dst != expectDst {
+		t.Errorf("UnitDestination returned %s, expected %s", dst, expectDst)
+	}
+}
+
 func TestPlaceMountUnit(t *testing.T) {
 	u := Unit{
-		Name: "media-state.mount",
+		Name:    "media-state.mount",
-      Runtime: false,
+		Runtime: false,
-      Content: `[Mount]
+		Content: `[Mount]
 What=/dev/sdb1
 Where=/media/state
 `,
@@ -75,7 +98,7 @@ Where=/media/state
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
 	dst := UnitDestination(&u, dir)
 	expectDst := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
@@ -115,7 +138,7 @@ func TestMachineID(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
 	os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
 	ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007\n"), os.FileMode(0444))
@@ -124,3 +147,23 @@ func TestMachineID(t *testing.T) {
 		t.Fatalf("File has incorrect contents")
 	}
 }
+
+func TestMaskUnit(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	if err := MaskUnit("foo.service", dir); err != nil {
+		t.Fatalf("Unable to mask unit: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "systemd", "system", "foo.service")
+	target, err := os.Readlink(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read link", err)
+	}
+	if target != "/dev/null" {
+		t.Fatalf("unit not masked, got unit target", target)
+	}
+}

system/user.go

@@ -53,7 +53,7 @@ func CreateUser(u *User) error {
 	}
 
 	if u.PrimaryGroup != "" {
-		args = append(args, "--primary-group", u.PrimaryGroup)
+		args = append(args, "--gid", u.PrimaryGroup)
 	}
 
 	if len(u.Groups) > 0 {

test

@@ -1,10 +1,37 @@
 #!/bin/bash -e
+#
+# Run all coreos-cloudinit tests
+#   ./test
+#   ./test -v
+#
+# Run tests for one package
+#   PKG=initialize ./test
+#
 
-echo "Building bin/coreos-cloudinit"
+# Invoke ./cover for HTML output
-. build
+COVER=${COVER:-"-cover"}
+
+source ./build
+
+declare -a TESTPKGS=(initialize system datasource)
+
+if [ -z "$PKG" ]; then
+	GOFMTPATH="$TESTPKGS coreos-cloudinit.go"
+	# prepend repo path to each package
+	TESTPKGS=${TESTPKGS[@]/#/${REPO_PATH}/}
+else
+	GOFMTPATH="$TESTPKGS"
+	# strip out slashes and dots from PKG=./foo/
+	TESTPKGS=${PKG//\//}
+	TESTPKGS=${TESTPKGS//./}
+	TESTPKGS=${TESTPKGS/#/${REPO_PATH}/}
+fi
 
 echo "Running tests..."
-for pkg in "./initialize ./system ./datasource"; do
+go test -i ${TESTPKGS}
-	go test -i $pkg
+go test ${COVER} $@ ${TESTPKGS}
-	go test -v $pkg
+
-done
+echo "Checking gofmt..."
+fmtRes=$(gofmt -l $GOFMTPATH)
+
+echo "Success"

units/configdrive-block.service

@@ -5,10 +5,6 @@ ConditionPathIsMountPoint=!/media/configdrive
 # Only mount config drive block devices automatically in virtual machines
 ConditionVirtualization=vm
 
-# OpenStack defined config drive so they get to stick their name in it
-Wants=user-cloudinit@media-configdrive-openstack-latest-user_data.service
-Before=user-cloudinit@media-configdrive-openstack-latest-user_data.service
-
 [Service]
 Type=oneshot
 RemainAfterExit=no

units/configdrive-virtfs.service

@@ -4,10 +4,6 @@ Conflicts=configdrive-block.service umount.target
 ConditionPathIsMountPoint=!/media/configdrive
 ConditionVirtualization=vm
 
-# OpenStack defined config drive so they get to stick their name in it
-Wants=user-cloudinit@media-configdrive-openstack-latest-user_data.service
-Before=user-cloudinit@media-configdrive-openstack-latest-user_data.service
-
 # Support old style setup for now
 Wants=addon-run@media-configdrive.service addon-config@media-configdrive.service
 Before=addon-run@media-configdrive.service addon-config@media-configdrive.service

units/user-cloudinit@.path

@@ -0,0 +1,5 @@
+[Unit]
+Description=Watch for a cloud-config at %f
+
+[Path]
+PathExists=%f

units/user-config.target

@@ -3,9 +3,11 @@ Description=Load user-provided cloud configs
 Requires=system-config.target
 After=system-config.target
 
-# Load user_data placed by coreos-install
+# Watch for configs at a couple common paths
-Requires=user-cloudinit@var-lib-coreos\x2dinstall-user_data.service
+Requires=user-cloudinit@media-configdrive-openstack-latest-user_data.path
-After=user-cloudinit@var-lib-coreos\x2dinstall-user_data.service
+After=user-cloudinit@media-configdrive-openstack-latest-user_data.path
+Requires=user-cloudinit@var-lib-coreos\x2dinstall-user_data.path
+After=user-cloudinit@var-lib-coreos\x2dinstall-user_data.path
 
 Requires=user-cloudinit-proc-cmdline.service
 After=user-cloudinit-proc-cmdline.service