coreos-cloudinit: bump to 0.8.9

env_file: fix broken test cases

.gitignore

@@ -1,4 +1,4 @@
 *.swp
 bin/
 coverage/
-pkg/
+gopath/

Documentation/cloud-config.md

@@ -16,7 +16,7 @@ We've designed our implementation to allow the same cloud-config file to work ac
 
 The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values.
 
-A cloud-config file should contain an associative array which has zero or more of the following keys:
+A cloud-config file should contain `#cloud-config`, followed by an associative array which has zero or more of the following keys:
 
 - `coreos`
 - `ssh_authorized_keys`
@@ -73,6 +73,7 @@ Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
 #### fleet
 
 The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
+
 ```
 #cloud-config
 
@@ -83,6 +84,7 @@ coreos:
 ```
 
 ...will generate a systemd unit drop-in like this:
+
 ```
 [Service]
 Environment="FLEET_PUBLIC_IP=203.0.113.29"
@@ -97,11 +99,20 @@ For more information on fleet configuration, see the [fleet documentation][fleet
 
 The `coreos.update.*` parameters manipulate settings related to how CoreOS instances are updated.
 
+These fields will be written out to and replace `/etc/coreos/update.conf`. If only one of the parameters is given it will only overwrite the given field. 
+The `reboot-strategy` parameter also affects the behaviour of [locksmith](https://github.com/coreos/locksmith). 
+
 - **reboot-strategy**: One of "reboot", "etcd-lock", "best-effort" or "off" for controlling when reboots are issued after an update is performed.
   - _reboot_: Reboot immediately after an update is applied.
   - _etcd-lock_: Reboot after first taking a distributed lock in etcd, this guarantees that only one host will reboot concurrently and that the cluster will remain available during the update.
   - _best-effort_ - If etcd is running, "etcd-lock", otherwise simply "reboot".
   - _off_ - Disable rebooting after updates are applied (not recommended).
+- **server**: is the omaha endpoint URL which will be queried for updates.
+- **group**:  signifies the channel which should be used for automatic updates.  This value defaults to the version of the image initially downloaded. (one of "master", "alpha", "beta", "stable")
+
+*Note: cloudinit will only manipulate the locksmith unit file in the systemd runtime directory (`/run/systemd/system/locksmithd.service`). If any manual modifications are made to an overriding unit configuration file (e.g. `/etc/systemd/system/locksmithd.service`), cloudinit will no longer be able to control the locksmith service unit.*
+
+##### Example
 
 ```
 #cloud-config
@@ -115,10 +126,11 @@ coreos:
 The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields:
 
 - **name**: String representing unit's name. Required.
-- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemd enable`. Default value is false.
+- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. Default value is false.
 - **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false.
 - **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
 - **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
+- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. Default value is false.
 
 **NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
 
@@ -143,15 +155,12 @@ coreos:
           Restart=always
           ExecStart=/usr/bin/docker start -a redis_server
           ExecStop=/usr/bin/docker stop -t 2 redis_server
-          
-          [Install]
-          WantedBy=local.target
 ```
 
 Start the built-in `etcd` and `fleet` services:
 
 ```
-# cloud-config
+#cloud-config
 
 coreos:
     units:
@@ -270,7 +279,7 @@ For example, if you have an installation of GitHub Enterprise, you can provide a
 
 users:
   - name: elroy
-    coreos-ssh-import-url: https://token:<OAUTH-TOKEN>@github-enterprise.example.com/users/elroy/keys
+    coreos-ssh-import-url: https://github-enterprise.example.com/api/v3/users/elroy/keys?access_token=<TOKEN>
 ```
 
 You can also specify any URL whose response matches the JSON format for public keys:

Documentation/debian-interfaces.md

@@ -0,0 +1,27 @@
+#Debian Interfaces#
+**WARNING**: This option is EXPERIMENTAL and may change or be removed at any
+point.  
+There is basic support for converting from a Debian network configuration to
+networkd unit files. The -convert-netconf=debian option is used to activate
+this feature.
+
+#convert-netconf#
+Default: ""  
+Read the network config provided in cloud-drive and translate it from the
+specified format into networkd unit files (requires the -from-configdrive
+flag). Currently only supports "debian" which provides support for a small
+subset of the [Debian network configuration]
+(https://wiki.debian.org/NetworkConfiguration). These options include:
+
+- interface config methods
+	- static
+		- address/netmask
+		- gateway
+		- hwaddress
+		- dns-nameservers
+	- dhcp
+		- hwaddress
+	- manual
+	- loopback
+- vlan_raw_device
+- bond-slaves

build

@@ -3,7 +3,12 @@
 ORG_PATH="github.com/coreos"
 REPO_PATH="${ORG_PATH}/coreos-cloudinit"
 
+if [ ! -h gopath/src/${REPO_PATH} ]; then
+	mkdir -p gopath/src/${ORG_PATH}
+	ln -s ../../../.. gopath/src/${REPO_PATH} || exit 255
+fi
+
 export GOBIN=${PWD}/bin
-export GOPATH=${PWD}
+export GOPATH=${PWD}/gopath
 
 go build -o bin/coreos-cloudinit ${REPO_PATH}

coreos-cloudinit.go

@@ -3,107 +3,279 @@ package main
 import (
 	"flag"
 	"fmt"
-	"log"
 	"os"
+	"sync"
+	"time"
 
 	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/initialize"
+	"github.com/coreos/coreos-cloudinit/pkg"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
-const version = "0.7.1"
+const (
+	version               = "0.8.9"
+	datasourceInterval    = 100 * time.Millisecond
+	datasourceMaxInterval = 30 * time.Second
+	datasourceTimeout     = 5 * time.Minute
+)
+
+var (
+	printVersion  bool
+	ignoreFailure bool
+	sources       struct {
+		file            string
+		configDrive     string
+		metadataService bool
+		url             string
+		procCmdLine     bool
+	}
+	convertNetconf string
+	workspace      string
+	sshKeyName     string
+)
+
+func init() {
+	flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
+	flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
+	flag.StringVar(&sources.file, "from-file", "", "Read user-data from provided file")
+	flag.StringVar(&sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory")
+	flag.BoolVar(&sources.metadataService, "from-metadata-service", false, "Download data from metadata service")
+	flag.StringVar(&sources.url, "from-url", "", "Download user-data from provided url")
+	flag.BoolVar(&sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", datasource.ProcCmdlineLocation, datasource.ProcCmdlineCloudConfigFlag))
+	flag.StringVar(&convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files (requires the -from-configdrive flag)")
+	flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
+	flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
+}
 
 func main() {
-	var printVersion bool
-	flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
-
-	var ignoreFailure bool
-	flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
-
-	var file string
-	flag.StringVar(&file, "from-file", "", "Read user-data from provided file")
-
-	var url string
-	flag.StringVar(&url, "from-url", "", "Download user-data from provided url")
-
-	var useProcCmdline bool
-	flag.BoolVar(&useProcCmdline, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", datasource.ProcCmdlineLocation, datasource.ProcCmdlineCloudConfigFlag))
-
-	var workspace string
-	flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
-
-	var sshKeyName string
-	flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
-
 	flag.Parse()
 
+	die := func() {
+		if ignoreFailure {
+			os.Exit(0)
+		}
+		os.Exit(1)
+	}
+
 	if printVersion == true {
 		fmt.Printf("coreos-cloudinit version %s\n", version)
 		os.Exit(0)
 	}
 
-	var ds datasource.Datasource
-	if file != "" {
-		ds = datasource.NewLocalFile(file)
-	} else if url != "" {
-		ds = datasource.NewMetadataService(url)
-	} else if useProcCmdline {
-		ds = datasource.NewProcCmdline()
-	} else {
-		fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
+	if convertNetconf != "" && sources.configDrive == "" {
+		fmt.Println("-convert-netconf flag requires -from-configdrive")
 		os.Exit(1)
 	}
 
-	log.Printf("Fetching user-data from datasource of type %q", ds.Type())
-	userdataBytes, err := ds.Fetch()
-	if err != nil {
-		log.Printf("Failed fetching user-data from datasource: %v", err)
-		if ignoreFailure {
-			os.Exit(0)
-		} else {
+	switch convertNetconf {
+	case "":
+	case "debian":
+	default:
+		fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian'\n", convertNetconf)
 		os.Exit(1)
 	}
-	}
 
-	if len(userdataBytes) == 0 {
-		log.Printf("No user data to handle, exiting.")
-		os.Exit(0)
-	}
-
-	env := initialize.NewEnvironment("/", workspace)
-
-	userdata := string(userdataBytes)
-	userdata = env.Apply(userdata)
-
-	parsed, err := initialize.ParseUserData(userdata)
-	if err != nil {
-		log.Printf("Failed parsing user-data: %v", err)
-		if ignoreFailure {
-			os.Exit(0)
-		} else {
+	dss := getDatasources()
+	if len(dss) == 0 {
+		fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-metadata-service, --from-url or --from-proc-cmdline")
 		os.Exit(1)
 	}
+
+	ds := selectDatasource(dss)
+	if ds == nil {
+		fmt.Println("No datasources available in time")
+		die()
 	}
 
-	err = initialize.PrepWorkspace(env.Workspace())
+	fmt.Printf("Fetching user-data from datasource of type %q\n", ds.Type())
+	userdataBytes, err := ds.FetchUserdata()
 	if err != nil {
-		log.Fatalf("Failed preparing workspace: %v", err)
+		fmt.Printf("Failed fetching user-data from datasource: %v\n", err)
+		die()
 	}
 
-	switch t := parsed.(type) {
-	case initialize.CloudConfig:
-		err = initialize.Apply(t, env)
+	fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
+	metadataBytes, err := ds.FetchMetadata()
+	if err != nil {
+		fmt.Printf("Failed fetching meta-data from datasource: %v\n", err)
+		die()
+	}
+
+	// Extract IPv4 addresses from metadata if possible
+	var subs map[string]string
+	if len(metadataBytes) > 0 {
+		subs, err = initialize.ExtractIPsFromMetadata(metadataBytes)
+		if err != nil {
+			fmt.Printf("Failed extracting IPs from meta-data: %v\n", err)
+			die()
+		}
+	}
+
+	// Apply environment to user-data
+	env := initialize.NewEnvironment("/", ds.ConfigRoot(), workspace, convertNetconf, sshKeyName, subs)
+	userdata := env.Apply(string(userdataBytes))
+
+	var ccm, ccu *initialize.CloudConfig
+	var script *system.Script
+	if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil {
+		fmt.Printf("Failed to parse meta-data: %v\n", err)
+		die()
+	}
+	if ud, err := initialize.ParseUserData(userdata); err != nil {
+		fmt.Printf("Failed to parse user-data: %v\n", err)
+		die()
+	} else {
+		switch t := ud.(type) {
+		case *initialize.CloudConfig:
+			ccu = t
 		case system.Script:
-		var path string
-		path, err = initialize.PersistScriptInWorkspace(t, env.Workspace())
+			script = &t
+		}
+	}
+
+	var cc *initialize.CloudConfig
+	if ccm != nil && ccu != nil {
+		fmt.Println("Merging cloud-config from meta-data and user-data")
+		merged := mergeCloudConfig(*ccm, *ccu)
+		cc = &merged
+	} else if ccm != nil && ccu == nil {
+		fmt.Println("Processing cloud-config from meta-data")
+		cc = ccm
+	} else if ccm == nil && ccu != nil {
+		fmt.Println("Processing cloud-config from user-data")
+		cc = ccu
+	} else {
+		fmt.Println("No cloud-config data to handle.")
+	}
+
+	if cc != nil {
+		if err = initialize.Apply(*cc, env); err != nil {
+			fmt.Printf("Failed to apply cloud-config: %v\n", err)
+			die()
+		}
+	}
+
+	if script != nil {
+		if err = runScript(*script, env); err != nil {
+			fmt.Printf("Failed to run script: %v\n", err)
+			die()
+		}
+	}
+}
+
+// mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from
+// meta-data) onto udcc (a CloudConfig derived from user-data), if they are
+// not already set on udcc (i.e. user-data always takes precedence)
+// NB: This needs to be kept in sync with ParseMetadata so that it tracks all
+// elements of a CloudConfig which that function can populate.
+func mergeCloudConfig(mdcc, udcc initialize.CloudConfig) (cc initialize.CloudConfig) {
+	if mdcc.Hostname != "" {
+		if udcc.Hostname != "" {
+			fmt.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)", udcc.Hostname, mdcc.Hostname)
+		} else {
+			udcc.Hostname = mdcc.Hostname
+		}
+
+	}
+	for _, key := range mdcc.SSHAuthorizedKeys {
+		udcc.SSHAuthorizedKeys = append(udcc.SSHAuthorizedKeys, key)
+	}
+	if mdcc.NetworkConfigPath != "" {
+		if udcc.NetworkConfigPath != "" {
+			fmt.Printf("Warning: user-data NetworkConfigPath %s overrides metadata NetworkConfigPath %s", udcc.NetworkConfigPath, mdcc.NetworkConfigPath)
+		} else {
+			udcc.NetworkConfigPath = mdcc.NetworkConfigPath
+		}
+	}
+	return udcc
+}
+
+// getDatasources creates a slice of possible Datasources for cloudinit based
+// on the different source command-line flags.
+func getDatasources() []datasource.Datasource {
+	dss := make([]datasource.Datasource, 0, 5)
+	if sources.file != "" {
+		dss = append(dss, datasource.NewLocalFile(sources.file))
+	}
+	if sources.url != "" {
+		dss = append(dss, datasource.NewRemoteFile(sources.url))
+	}
+	if sources.configDrive != "" {
+		dss = append(dss, datasource.NewConfigDrive(sources.configDrive))
+	}
+	if sources.metadataService {
+		dss = append(dss, datasource.NewMetadataService())
+	}
+	if sources.procCmdLine {
+		dss = append(dss, datasource.NewProcCmdline())
+	}
+	return dss
+}
+
+// selectDatasource attempts to choose a valid Datasource to use based on its
+// current availability. The first Datasource to report to be available is
+// returned. Datasources will be retried if possible if they are not
+// immediately available. If all Datasources are permanently unavailable or
+// datasourceTimeout is reached before one becomes available, nil is returned.
+func selectDatasource(sources []datasource.Datasource) datasource.Datasource {
+	ds := make(chan datasource.Datasource)
+	stop := make(chan struct{})
+	var wg sync.WaitGroup
+
+	for _, s := range sources {
+		wg.Add(1)
+		go func(s datasource.Datasource) {
+			defer wg.Done()
+
+			duration := datasourceInterval
+			for {
+				fmt.Printf("Checking availability of %q\n", s.Type())
+				if s.IsAvailable() {
+					ds <- s
+					return
+				} else if !s.AvailabilityChanges() {
+					return
+				}
+				select {
+				case <-stop:
+					return
+				case <-time.Tick(duration):
+					duration = pkg.ExpBackoff(duration, datasourceMaxInterval)
+				}
+			}
+		}(s)
+	}
+
+	done := make(chan struct{})
+	go func() {
+		wg.Wait()
+		close(done)
+	}()
+
+	var s datasource.Datasource
+	select {
+	case s = <-ds:
+	case <-done:
+	case <-time.Tick(datasourceTimeout):
+	}
+
+	close(stop)
+	return s
+}
+
+// TODO(jonboulle): this should probably be refactored and moved into a different module
+func runScript(script system.Script, env *initialize.Environment) error {
+	err := initialize.PrepWorkspace(env.Workspace())
+	if err != nil {
+		fmt.Printf("Failed preparing workspace: %v\n", err)
+		return err
+	}
+	path, err := initialize.PersistScriptInWorkspace(script, env.Workspace())
 	if err == nil {
 		var name string
 		name, err = system.ExecuteScript(path)
-			initialize.PersistUnitNameInWorkspace(name, workspace)
-		}
-	}
-
-	if err != nil {
-		log.Fatalf("Failed resolving user-data: %v", err)
+		initialize.PersistUnitNameInWorkspace(name, env.Workspace())
 	}
+	return err
 }

coreos-cloudinit_test.go

@@ -0,0 +1,110 @@
+package main
+
+import (
+	"reflect"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/initialize"
+)
+
+func TestMergeCloudConfig(t *testing.T) {
+	simplecc := initialize.CloudConfig{
+		SSHAuthorizedKeys: []string{"abc", "def"},
+		Hostname:          "foobar",
+		NetworkConfigPath: "/path/somewhere",
+	}
+	for i, tt := range []struct {
+		udcc initialize.CloudConfig
+		mdcc initialize.CloudConfig
+		want initialize.CloudConfig
+	}{
+		{
+			// If mdcc is empty, udcc should be returned unchanged
+			simplecc,
+			initialize.CloudConfig{},
+			simplecc,
+		},
+		{
+			// If udcc is empty, mdcc should be returned unchanged(overridden)
+			initialize.CloudConfig{},
+			simplecc,
+			simplecc,
+		},
+		{
+			// user-data should override completely in the case of conflicts
+			simplecc,
+			initialize.CloudConfig{
+				Hostname:          "meta-hostname",
+				NetworkConfigPath: "/path/meta",
+			},
+			simplecc,
+		},
+		{
+			// Mixed merge should succeed
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"abc", "def"},
+				Hostname:          "user-hostname",
+				NetworkConfigPath: "/path/somewhere",
+			},
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"woof", "qux"},
+				Hostname:          "meta-hostname",
+			},
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"},
+				Hostname:          "user-hostname",
+				NetworkConfigPath: "/path/somewhere",
+			},
+		},
+		{
+			// Completely non-conflicting merge should be fine
+			initialize.CloudConfig{
+				Hostname: "supercool",
+			},
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
+				NetworkConfigPath: "/dev/fun",
+			},
+			initialize.CloudConfig{
+				Hostname:          "supercool",
+				SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
+				NetworkConfigPath: "/dev/fun",
+			},
+		},
+		{
+			// Non-mergeable settings in user-data should not be affected
+			initialize.CloudConfig{
+				Hostname:       "mememe",
+				ManageEtcHosts: initialize.EtcHosts("lolz"),
+			},
+			initialize.CloudConfig{
+				Hostname:          "youyouyou",
+				NetworkConfigPath: "meta-meta-yo",
+			},
+			initialize.CloudConfig{
+				Hostname:          "mememe",
+				ManageEtcHosts:    initialize.EtcHosts("lolz"),
+				NetworkConfigPath: "meta-meta-yo",
+			},
+		},
+		{
+			// Non-mergeable (unexpected) settings in meta-data are ignored
+			initialize.CloudConfig{
+				Hostname: "mememe",
+			},
+			initialize.CloudConfig{
+				ManageEtcHosts:    initialize.EtcHosts("lolz"),
+				NetworkConfigPath: "meta-meta-yo",
+			},
+			initialize.CloudConfig{
+				Hostname:          "mememe",
+				NetworkConfigPath: "meta-meta-yo",
+			},
+		},
+	} {
+		got := mergeCloudConfig(tt.mdcc, tt.udcc)
+		if !reflect.DeepEqual(got, tt.want) {
+			t.Errorf("case #%d: mergeCloudConfig mutated CloudConfig unexpectedly:\ngot:\n%s\nwant:\n%s", i, got, tt.want)
+		}
+	}
+}

datasource/configdrive.go

@@ -0,0 +1,48 @@
+package datasource
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+)
+
+type configDrive struct {
+	root string
+}
+
+func NewConfigDrive(root string) *configDrive {
+	return &configDrive{path.Join(root, "openstack")}
+}
+
+func (cd *configDrive) IsAvailable() bool {
+	_, err := os.Stat(cd.root)
+	return !os.IsNotExist(err)
+}
+
+func (cd *configDrive) AvailabilityChanges() bool {
+	return true
+}
+
+func (cd *configDrive) ConfigRoot() string {
+	return cd.root
+}
+
+func (cd *configDrive) FetchMetadata() ([]byte, error) {
+	return cd.readFile("meta_data.json")
+}
+
+func (cd *configDrive) FetchUserdata() ([]byte, error) {
+	return cd.readFile("user_data")
+}
+
+func (cd *configDrive) Type() string {
+	return "cloud-drive"
+}
+
+func (cd *configDrive) readFile(filename string) ([]byte, error) {
+	data, err := ioutil.ReadFile(path.Join(cd.root, "latest", filename))
+	if os.IsNotExist(err) {
+		err = nil
+	}
+	return data, err
+}

datasource/datasource.go

@@ -1,104 +1,10 @@
 package datasource
 
-import (
-	"errors"
-	"fmt"
-	"io/ioutil"
-	"log"
-	"math"
-	"net"
-	"net/http"
-	neturl "net/url"
-	"strings"
-	"time"
-)
-
-const (
-	HTTP_2xx = 2
-	HTTP_4xx = 4
-
-	maxTimeout = time.Second * 5
-	maxRetries = 15
-)
-
 type Datasource interface {
-	Fetch() ([]byte, error)
+	IsAvailable() bool
+	AvailabilityChanges() bool
+	ConfigRoot() string
+	FetchMetadata() ([]byte, error)
+	FetchUserdata() ([]byte, error)
 	Type() string
 }
-
-// HTTP client timeout
-// This one is low since exponential backoff will kick off too.
-var timeout = time.Duration(2) * time.Second
-
-func dialTimeout(network, addr string) (net.Conn, error) {
-	deadline := time.Now().Add(timeout)
-	c, err := net.DialTimeout(network, addr, timeout)
-	if err != nil {
-		return nil, err
-	}
-	c.SetDeadline(deadline)
-	return c, nil
-}
-
-// Fetches user-data url with support for exponential backoff and maximum retries
-func fetchURL(rawurl string) ([]byte, error) {
-	if rawurl == "" {
-		return nil, errors.New("user-data URL is empty. Skipping.")
-	}
-
-	url, err := neturl.Parse(rawurl)
-	if err != nil {
-		return nil, err
-	}
-
-	// Unfortunately, url.Parse is too generic to throw errors if a URL does not
-	// have a valid HTTP scheme. So, we have to do this extra validation
-	if !strings.HasPrefix(url.Scheme, "http") {
-		return nil, fmt.Errorf("user-data URL %s does not have a valid HTTP scheme. Skipping.", rawurl)
-	}
-
-	userdataURL := url.String()
-
-	// We need to create our own client in order to add timeout support.
-	// TODO(c4milo) Replace it once Go 1.3 is officially used by CoreOS
-	// More info: https://code.google.com/p/go/source/detail?r=ada6f2d5f99f
-	transport := &http.Transport{
-		Dial: dialTimeout,
-	}
-
-	client := &http.Client{
-		Transport: transport,
-	}
-
-	for retry := 1; retry <= maxRetries; retry++ {
-		log.Printf("Fetching user-data from %s. Attempt #%d", userdataURL, retry)
-
-		resp, err := client.Get(userdataURL)
-
-		if err == nil {
-			defer resp.Body.Close()
-			status := resp.StatusCode / 100
-
-			if status == HTTP_2xx {
-				return ioutil.ReadAll(resp.Body)
-			}
-
-			if status == HTTP_4xx {
-				return nil, fmt.Errorf("user-data not found. HTTP status code: %d", resp.StatusCode)
-			}
-
-			log.Printf("user-data not found. HTTP status code: %d", resp.StatusCode)
-		} else {
-			log.Printf("unable to fetch user-data: %s", err.Error())
-		}
-
-		duration := time.Millisecond * time.Duration((math.Pow(float64(2), float64(retry)) * 100))
-		if duration > maxTimeout {
-			duration = maxTimeout
-		}
-
-		time.Sleep(duration)
-	}
-
-	return nil, fmt.Errorf("unable to fetch user-data. Maximum retries reached: %d", maxRetries)
-}

datasource/file.go

@@ -2,6 +2,7 @@ package datasource
 
 import (
 	"io/ioutil"
+	"os"
 )
 
 type localFile struct {
@@ -12,10 +13,27 @@ func NewLocalFile(path string) *localFile {
 	return &localFile{path}
 }
 
-func (self *localFile) Fetch() ([]byte, error) {
-	return ioutil.ReadFile(self.path)
+func (f *localFile) IsAvailable() bool {
+	_, err := os.Stat(f.path)
+	return !os.IsNotExist(err)
 }
 
-func (self *localFile) Type() string {
+func (f *localFile) AvailabilityChanges() bool {
+	return true
+}
+
+func (f *localFile) ConfigRoot() string {
+	return ""
+}
+
+func (f *localFile) FetchMetadata() ([]byte, error) {
+	return []byte{}, nil
+}
+
+func (f *localFile) FetchUserdata() ([]byte, error) {
+	return ioutil.ReadFile(f.path)
+}
+
+func (f *localFile) Type() string {
 	return "local-file"
 }

datasource/metadata_service.go

@@ -1,17 +1,155 @@
 package datasource
 
-type metadataService struct {
-	url string
+import (
+	"bufio"
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/pkg"
+)
+
+// metadataService retrieves metadata from either an OpenStack[1] (2012-08-10)
+// or EC2[2] (2009-04-04) compatible endpoint. It will first attempt to
+// directly retrieve a JSON blob from the OpenStack endpoint. If that fails
+// with a 404, it then attempts to retrieve metadata bit-by-bit from the EC2
+// endpoint, and populates that into an equivalent JSON blob. metadataService
+// also checks for userdata from EC2 and, if that fails with a 404, OpenStack.
+//
+// [1] http://docs.openstack.org/grizzly/openstack-compute/admin/content/metadata-service.html
+// [2] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html#instancedata-data-categories
+
+const (
+	BaseUrl              = "http://169.254.169.254/"
+	Ec2ApiVersion        = "2009-04-04"
+	Ec2UserdataUrl       = BaseUrl + Ec2ApiVersion + "/user-data"
+	Ec2MetadataUrl       = BaseUrl + Ec2ApiVersion + "/meta-data"
+	OpenstackApiVersion  = "openstack/2012-08-10"
+	OpenstackUserdataUrl = BaseUrl + OpenstackApiVersion + "/user_data"
+)
+
+type metadataService struct{}
+
+type getter interface {
+	GetRetry(string) ([]byte, error)
 }
 
-func NewMetadataService(url string) *metadataService {
-	return &metadataService{url}
+func NewMetadataService() *metadataService {
+	return &metadataService{}
 }
 
-func (ms *metadataService) Fetch() ([]byte, error) {
-	return fetchURL(ms.url)
+func (ms *metadataService) IsAvailable() bool {
+	client := pkg.NewHttpClient()
+	_, err := client.Get(BaseUrl)
+	return (err == nil)
+}
+
+func (ms *metadataService) AvailabilityChanges() bool {
+	return true
+}
+
+func (ms *metadataService) ConfigRoot() string {
+	return ""
+}
+
+func (ms *metadataService) FetchMetadata() ([]byte, error) {
+	return fetchMetadata(pkg.NewHttpClient())
+}
+
+func (ms *metadataService) FetchUserdata() ([]byte, error) {
+	client := pkg.NewHttpClient()
+	if data, err := client.GetRetry(Ec2UserdataUrl); err == nil {
+		return data, err
+	} else if _, ok := err.(pkg.ErrTimeout); ok {
+		return data, err
+	}
+
+	if data, err := client.GetRetry(OpenstackUserdataUrl); err == nil {
+		return data, err
+	} else if _, ok := err.(pkg.ErrNotFound); ok {
+		return []byte{}, nil
+	} else {
+		return data, err
+	}
 }
 
 func (ms *metadataService) Type() string {
 	return "metadata-service"
 }
+
+func fetchMetadata(client getter) ([]byte, error) {
+	attrs := make(map[string]interface{})
+	if keynames, err := fetchAttributes(client, fmt.Sprintf("%s/public-keys", Ec2MetadataUrl)); err == nil {
+		keyIDs := make(map[string]string)
+		for _, keyname := range keynames {
+			tokens := strings.SplitN(keyname, "=", 2)
+			if len(tokens) != 2 {
+				return nil, fmt.Errorf("malformed public key: %q\n", keyname)
+			}
+			keyIDs[tokens[1]] = tokens[0]
+		}
+
+		keys := make(map[string]string)
+		for name, id := range keyIDs {
+			sshkey, err := fetchAttribute(client, fmt.Sprintf("%s/public-keys/%s/openssh-key", Ec2MetadataUrl, id))
+			if err != nil {
+				return nil, err
+			}
+			keys[name] = sshkey
+			fmt.Printf("Found SSH key for %q\n", name)
+		}
+		attrs["public_keys"] = keys
+	} else if _, ok := err.(pkg.ErrNotFound); !ok {
+		return nil, err
+	}
+
+	if hostname, err := fetchAttribute(client, fmt.Sprintf("%s/hostname", Ec2MetadataUrl)); err == nil {
+		attrs["hostname"] = hostname
+	} else if _, ok := err.(pkg.ErrNotFound); !ok {
+		return nil, err
+	}
+
+	if localAddr, err := fetchAttribute(client, fmt.Sprintf("%s/local-ipv4", Ec2MetadataUrl)); err == nil {
+		attrs["local-ipv4"] = localAddr
+	} else if _, ok := err.(pkg.ErrNotFound); !ok {
+		return nil, err
+	}
+
+	if publicAddr, err := fetchAttribute(client, fmt.Sprintf("%s/public-ipv4", Ec2MetadataUrl)); err == nil {
+		attrs["public-ipv4"] = publicAddr
+	} else if _, ok := err.(pkg.ErrNotFound); !ok {
+		return nil, err
+	}
+
+	if content_path, err := fetchAttribute(client, fmt.Sprintf("%s/network_config/content_path", Ec2MetadataUrl)); err == nil {
+		attrs["network_config"] = map[string]string{
+			"content_path": content_path,
+		}
+	} else if _, ok := err.(pkg.ErrNotFound); !ok {
+		return nil, err
+	}
+
+	return json.Marshal(attrs)
+}
+
+func fetchAttributes(client getter, url string) ([]string, error) {
+	resp, err := client.GetRetry(url)
+	if err != nil {
+		return nil, err
+	}
+	scanner := bufio.NewScanner(bytes.NewBuffer(resp))
+	data := make([]string, 0)
+	for scanner.Scan() {
+		data = append(data, scanner.Text())
+	}
+	return data, scanner.Err()
+}
+
+func fetchAttribute(client getter, url string) (string, error) {
+	if attrs, err := fetchAttributes(client, url); err == nil && len(attrs) > 0 {
+		return attrs[0], nil
+	} else {
+		return "", err
+	}
+}

datasource/metadata_service_test.go

@@ -0,0 +1,159 @@
+package datasource
+
+import (
+	"bytes"
+	"fmt"
+	"reflect"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/pkg"
+)
+
+type TestHttpClient struct {
+	metadata map[string]string
+	err      error
+}
+
+func (t *TestHttpClient) GetRetry(url string) ([]byte, error) {
+	if t.err != nil {
+		return nil, t.err
+	}
+	if val, ok := t.metadata[url]; ok {
+		return []byte(val), nil
+	} else {
+		return nil, pkg.ErrNotFound{fmt.Errorf("not found: %q", url)}
+	}
+}
+
+func TestFetchAttributes(t *testing.T) {
+	for _, s := range []struct {
+		metadata map[string]string
+		err      error
+		tests    []struct {
+			path string
+			val  []string
+		}
+	}{
+		{
+			metadata: map[string]string{
+				"/":      "a\nb\nc/",
+				"/c/":    "d\ne/",
+				"/c/e/":  "f",
+				"/a":     "1",
+				"/b":     "2",
+				"/c/d":   "3",
+				"/c/e/f": "4",
+			},
+			tests: []struct {
+				path string
+				val  []string
+			}{
+				{"/", []string{"a", "b", "c/"}},
+				{"/b", []string{"2"}},
+				{"/c/d", []string{"3"}},
+				{"/c/e/", []string{"f"}},
+			},
+		},
+		{
+			err: pkg.ErrNotFound{fmt.Errorf("test error")},
+			tests: []struct {
+				path string
+				val  []string
+			}{
+				{"", nil},
+			},
+		},
+	} {
+		client := &TestHttpClient{s.metadata, s.err}
+		for _, tt := range s.tests {
+			attrs, err := fetchAttributes(client, tt.path)
+			if err != s.err {
+				t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.metadata, s.err, err)
+			}
+			if !reflect.DeepEqual(attrs, tt.val) {
+				t.Fatalf("bad fetch for %q (%q): want %q, got %q", tt.path, s.metadata, tt.val, attrs)
+			}
+		}
+	}
+}
+
+func TestFetchAttribute(t *testing.T) {
+	for _, s := range []struct {
+		metadata map[string]string
+		err      error
+		tests    []struct {
+			path string
+			val  string
+		}
+	}{
+		{
+			metadata: map[string]string{
+				"/":      "a\nb\nc/",
+				"/c/":    "d\ne/",
+				"/c/e/":  "f",
+				"/a":     "1",
+				"/b":     "2",
+				"/c/d":   "3",
+				"/c/e/f": "4",
+			},
+			tests: []struct {
+				path string
+				val  string
+			}{
+				{"/a", "1"},
+				{"/b", "2"},
+				{"/c/d", "3"},
+				{"/c/e/f", "4"},
+			},
+		},
+		{
+			err: pkg.ErrNotFound{fmt.Errorf("test error")},
+			tests: []struct {
+				path string
+				val  string
+			}{
+				{"", ""},
+			},
+		},
+	} {
+		client := &TestHttpClient{s.metadata, s.err}
+		for _, tt := range s.tests {
+			attr, err := fetchAttribute(client, tt.path)
+			if err != s.err {
+				t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.metadata, s.err, err)
+			}
+			if attr != tt.val {
+				t.Fatalf("bad fetch for %q (%q): want %q, got %q", tt.path, s.metadata, tt.val, attr)
+			}
+		}
+	}
+}
+
+func TestFetchMetadata(t *testing.T) {
+	for _, tt := range []struct {
+		metadata map[string]string
+		err      error
+		expect   []byte
+	}{
+		{
+			metadata: map[string]string{
+				"http://169.254.169.254/2009-04-04/meta-data/hostname":                    "host",
+				"http://169.254.169.254/2009-04-04/meta-data/public-keys":                 "0=test1\n",
+				"http://169.254.169.254/2009-04-04/meta-data/public-keys/0":               "openssh-key",
+				"http://169.254.169.254/2009-04-04/meta-data/public-keys/0/openssh-key":   "key",
+				"http://169.254.169.254/2009-04-04/meta-data/network_config/content_path": "path",
+			},
+			expect: []byte(`{"hostname":"host","network_config":{"content_path":"path"},"public_keys":{"test1":"key"}}`),
+		},
+		{err: pkg.ErrTimeout{fmt.Errorf("test error")}},
+	} {
+		client := &TestHttpClient{tt.metadata, tt.err}
+		metadata, err := fetchMetadata(client)
+		if err != tt.err {
+			t.Fatalf("bad error (%q): want %q, got %q", tt.metadata, tt.err, err)
+		}
+		if !bytes.Equal(metadata, tt.expect) {
+			t.Fatalf("bad fetch (%q): want %q, got %q", tt.metadata, tt.expect, metadata)
+		}
+	}
+}

datasource/proc_cmdline.go

@@ -5,6 +5,8 @@ import (
 	"io/ioutil"
 	"log"
 	"strings"
+
+	"github.com/coreos/coreos-cloudinit/pkg"
 )
 
 const (
@@ -12,24 +14,51 @@ const (
 	ProcCmdlineCloudConfigFlag = "cloud-config-url"
 )
 
-type procCmdline struct{}
-
-func NewProcCmdline() *procCmdline {
-	return &procCmdline{}
+type procCmdline struct {
+	Location string
 }
 
-func (self *procCmdline) Fetch() ([]byte, error) {
-	cmdline, err := ioutil.ReadFile(ProcCmdlineLocation)
+func NewProcCmdline() *procCmdline {
+	return &procCmdline{Location: ProcCmdlineLocation}
+}
+
+func (c *procCmdline) IsAvailable() bool {
+	contents, err := ioutil.ReadFile(c.Location)
+	if err != nil {
+		return false
+	}
+
+	cmdline := strings.TrimSpace(string(contents))
+	_, err = findCloudConfigURL(cmdline)
+	return (err == nil)
+}
+
+func (c *procCmdline) AvailabilityChanges() bool {
+	return false
+}
+
+func (c *procCmdline) ConfigRoot() string {
+	return ""
+}
+
+func (c *procCmdline) FetchMetadata() ([]byte, error) {
+	return []byte{}, nil
+}
+
+func (c *procCmdline) FetchUserdata() ([]byte, error) {
+	contents, err := ioutil.ReadFile(c.Location)
 	if err != nil {
 		return nil, err
 	}
 
-	url, err := findCloudConfigURL(string(cmdline))
+	cmdline := strings.TrimSpace(string(contents))
+	url, err := findCloudConfigURL(cmdline)
 	if err != nil {
 		return nil, err
 	}
 
-	cfg, err := fetchURL(url)
+	client := pkg.NewHttpClient()
+	cfg, err := client.GetRetry(url)
 	if err != nil {
 		return nil, err
 	}
@@ -37,7 +66,7 @@ func (self *procCmdline) Fetch() ([]byte, error) {
 	return cfg, nil
 }
 
-func (self *procCmdline) Type() string {
+func (c *procCmdline) Type() string {
 	return "proc-cmdline"
 }
 

datasource/proc_cmdline_test.go

@@ -1,6 +1,11 @@
 package datasource
 
 import (
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"os"
 	"testing"
 )
 
@@ -45,3 +50,39 @@ func TestParseCmdlineCloudConfigFound(t *testing.T) {
 		}
 	}
 }
+
+func TestProcCmdlineAndFetchConfig(t *testing.T) {
+
+	var (
+		ProcCmdlineTmpl    = "foo=bar cloud-config-url=%s/config\n"
+		CloudConfigContent = "#cloud-config\n"
+	)
+
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.Method == "GET" && r.RequestURI == "/config" {
+			fmt.Fprint(w, CloudConfigContent)
+		}
+	}))
+	defer ts.Close()
+
+	file, err := ioutil.TempFile(os.TempDir(), "test_proc_cmdline")
+	defer os.Remove(file.Name())
+	if err != nil {
+		t.Errorf("Test produced error: %v", err)
+	}
+	_, err = file.Write([]byte(fmt.Sprintf(ProcCmdlineTmpl, ts.URL)))
+	if err != nil {
+		t.Errorf("Test produced error: %v", err)
+	}
+
+	p := NewProcCmdline()
+	p.Location = file.Name()
+	cfg, err := p.FetchUserdata()
+	if err != nil {
+		t.Errorf("Test produced error: %v", err)
+	}
+
+	if string(cfg) != CloudConfigContent {
+		t.Errorf("Test failed, response body: %s != %s", cfg, CloudConfigContent)
+	}
+}

datasource/url.go

@@ -0,0 +1,38 @@
+package datasource
+
+import "github.com/coreos/coreos-cloudinit/pkg"
+
+type remoteFile struct {
+	url string
+}
+
+func NewRemoteFile(url string) *remoteFile {
+	return &remoteFile{url}
+}
+
+func (f *remoteFile) IsAvailable() bool {
+	client := pkg.NewHttpClient()
+	_, err := client.Get(f.url)
+	return (err == nil)
+}
+
+func (f *remoteFile) AvailabilityChanges() bool {
+	return true
+}
+
+func (f *remoteFile) ConfigRoot() string {
+	return ""
+}
+
+func (f *remoteFile) FetchMetadata() ([]byte, error) {
+	return []byte{}, nil
+}
+
+func (f *remoteFile) FetchUserdata() ([]byte, error) {
+	client := pkg.NewHttpClient()
+	return client.GetRetry(f.url)
+}
+
+func (f *remoteFile) Type() string {
+	return "url"
+}

initialize/config.go

@@ -3,11 +3,13 @@ package initialize
 import (
 	"errors"
 	"fmt"
+	"io/ioutil"
 	"log"
 	"path"
 
 	"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
 
+	"github.com/coreos/coreos-cloudinit/network"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -20,11 +22,9 @@ type CloudConfigFile interface {
 }
 
 // CloudConfigUnit represents a CoreOS specific configuration option that can generate
-// an associated system.Unit to be created/enabled appropriately
+// associated system.Units to be created/enabled appropriately
 type CloudConfigUnit interface {
-	// Unit should either return (*system.Unit, error), or (nil, nil) if nothing
-	// needs to be done for this configuration option.
-	Unit(root string) (*system.Unit, error)
+	Units(root string) ([]system.Unit, error)
 }
 
 // CloudConfig encapsulates the entire cloud-config configuration file and maps directly to YAML
@@ -41,6 +41,7 @@ type CloudConfig struct {
 	Hostname          string
 	Users             []system.User
 	ManageEtcHosts    EtcHosts `yaml:"manage_etc_hosts"`
+	NetworkConfigPath string
 }
 
 type warner func(format string, v ...interface{})
@@ -215,27 +216,66 @@ func Apply(cfg CloudConfig, env *Environment) error {
 	}
 
 	for _, ccu := range []CloudConfigUnit{cfg.Coreos.Etcd, cfg.Coreos.Fleet, cfg.Coreos.Update} {
-		u, err := ccu.Unit(env.Root())
+		u, err := ccu.Units(env.Root())
 		if err != nil {
 			return err
 		}
-		if u != nil {
-			cfg.Coreos.Units = append(cfg.Coreos.Units, *u)
+		cfg.Coreos.Units = append(cfg.Coreos.Units, u...)
+	}
+
+	wroteEnvironment := false
+	for _, file := range cfg.WriteFiles {
+		fullPath, err := system.WriteFile(&file, env.Root())
+		if err != nil {
+			return err
+		}
+		if path.Clean(file.Path) == "/etc/environment" {
+			wroteEnvironment = true
+		}
+		log.Printf("Wrote file %s to filesystem", fullPath)
+	}
+
+	if !wroteEnvironment {
+		ef := env.DefaultEnvironmentFile()
+		if ef != nil {
+			err := system.WriteEnvFile(ef, env.Root())
+			if err != nil {
+				return err
+			}
+			log.Printf("Updated /etc/environment")
 		}
 	}
 
-	for _, file := range cfg.WriteFiles {
-		file.Path = path.Join(env.Root(), file.Path)
-		if err := system.WriteFile(&file); err != nil {
+	if env.NetconfType() != "" {
+		netconfBytes, err := ioutil.ReadFile(path.Join(env.ConfigRoot(), cfg.NetworkConfigPath))
+		if err != nil {
+			return err
+		}
+
+		var interfaces []network.InterfaceGenerator
+		switch env.NetconfType() {
+		case "debian":
+			interfaces, err = network.ProcessDebianNetconf(string(netconfBytes))
+		default:
+			return fmt.Errorf("Unsupported network config format %q", env.NetconfType())
+		}
+
+		if err != nil {
+			return err
+		}
+
+		if err := system.WriteNetworkdConfigs(interfaces); err != nil {
+			return err
+		}
+		if err := system.RestartNetwork(interfaces); err != nil {
 			return err
 		}
-		log.Printf("Wrote file %s to filesystem", file.Path)
 	}
 
 	commands := make(map[string]string, 0)
 	reload := false
 	for _, unit := range cfg.Coreos.Units {
-		dst := system.UnitDestination(&unit, env.Root())
+		dst := unit.Destination(env.Root())
 		if unit.Content != "" {
 			log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
 			if err := system.PlaceUnit(&unit, dst); err != nil {
@@ -247,15 +287,20 @@ func Apply(cfg CloudConfig, env *Environment) error {
 
 		if unit.Mask {
 			log.Printf("Masking unit file %s", unit.Name)
-			if err := system.MaskUnit(unit.Name, env.Root()); err != nil {
+			if err := system.MaskUnit(&unit, env.Root()); err != nil {
+				return err
+			}
+		} else if unit.Runtime {
+			log.Printf("Ensuring runtime unit file %s is unmasked", unit.Name)
+			if err := system.UnmaskUnit(&unit, env.Root()); err != nil {
 				return err
 			}
 		}
 
 		if unit.Enable {
 			if unit.Group() != "network" {
-				log.Printf("Enabling unit file %s", dst)
-				if err := system.EnableUnitFile(dst, unit.Runtime); err != nil {
+				log.Printf("Enabling unit file %s", unit.Name)
+				if err := system.EnableUnitFile(unit.Name, unit.Runtime); err != nil {
 					return err
 				}
 				log.Printf("Enabled unit %s", unit.Name)

initialize/env.go

@@ -4,48 +4,89 @@ import (
 	"os"
 	"path"
 	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
 )
 
 const DefaultSSHKeyName = "coreos-cloudinit"
 
 type Environment struct {
 	root          string
+	configRoot    string
 	workspace     string
+	netconfType   string
 	sshKeyName    string
 	substitutions map[string]string
 }
 
-func NewEnvironment(root, workspace string) *Environment {
-	substitutions := map[string]string{
+// TODO(jonboulle): this is getting unwieldy, should be able to simplify the interface somehow
+func NewEnvironment(root, configRoot, workspace, netconfType, sshKeyName string, substitutions map[string]string) *Environment {
+	if substitutions == nil {
+		substitutions = make(map[string]string)
+	}
+	// If certain values are not in the supplied substitution, fall back to retrieving them from the environment
+	for k, v := range map[string]string{
 		"$public_ipv4":  os.Getenv("COREOS_PUBLIC_IPV4"),
 		"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
+	} {
+		if _, ok := substitutions[k]; !ok {
+			substitutions[k] = v
 		}
-	return &Environment{root, workspace, DefaultSSHKeyName, substitutions}
+	}
+	return &Environment{root, configRoot, workspace, netconfType, sshKeyName, substitutions}
 }
 
-func (self *Environment) Workspace() string {
-	return path.Join(self.root, self.workspace)
+func (e *Environment) Workspace() string {
+	return path.Join(e.root, e.workspace)
 }
 
-func (self *Environment) Root() string {
-	return self.root
+func (e *Environment) Root() string {
+	return e.root
 }
 
-func (self *Environment) SSHKeyName() string {
-	return self.sshKeyName
+func (e *Environment) ConfigRoot() string {
+	return e.configRoot
 }
 
-func (self *Environment) SetSSHKeyName(name string) {
-	self.sshKeyName = name
+func (e *Environment) NetconfType() string {
+	return e.netconfType
 }
 
-func (self *Environment) Apply(data string) string {
-	for key, val := range self.substitutions {
+func (e *Environment) SSHKeyName() string {
+	return e.sshKeyName
+}
+
+func (e *Environment) SetSSHKeyName(name string) {
+	e.sshKeyName = name
+}
+
+func (e *Environment) Apply(data string) string {
+	for key, val := range e.substitutions {
 		data = strings.Replace(data, key, val, -1)
 	}
 	return data
 }
 
+func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
+	ef := system.EnvFile{
+		File: &system.File{
+			Path: "/etc/environment",
+		},
+		Vars: map[string]string{},
+	}
+	if ip, ok := e.substitutions["$public_ipv4"]; ok && len(ip) > 0 {
+		ef.Vars["COREOS_PUBLIC_IPV4"] = ip
+	}
+	if ip, ok := e.substitutions["$private_ipv4"]; ok && len(ip) > 0 {
+		ef.Vars["COREOS_PRIVATE_IPV4"] = ip
+	}
+	if len(ef.Vars) == 0 {
+		return nil
+	} else {
+		return &ef
+	}
+}
+
 // normalizeSvcEnv standardizes the keys of the map (environment variables for a service)
 // by replacing any dashes with underscores and ensuring they are entirely upper case.
 // For example, "some-env" --> "SOME_ENV"

initialize/env_test.go

@@ -1,27 +1,106 @@
 package initialize
 
 import (
+	"io/ioutil"
 	"os"
+	"path"
 	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
 )
 
 func TestEnvironmentApply(t *testing.T) {
-	os.Setenv("COREOS_PUBLIC_IPV4", "192.0.2.3")
-	os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.203")
-	env := NewEnvironment("./", "./")
-	input := `[Service]
+	os.Setenv("COREOS_PUBLIC_IPV4", "1.2.3.4")
+	os.Setenv("COREOS_PRIVATE_IPV4", "5.6.7.8")
+	for _, tt := range []struct {
+		subs  map[string]string
+		input string
+		out   string
+	}{
+		{
+			// Substituting both values directly should always take precedence
+			// over environment variables
+			map[string]string{
+				"$public_ipv4":  "192.0.2.3",
+				"$private_ipv4": "192.0.2.203",
+			},
+			`[Service]
 ExecStart=/usr/bin/echo "$public_ipv4"
 ExecStop=/usr/bin/echo $private_ipv4
-ExecStop=/usr/bin/echo $unknown
-`
-	expected := `[Service]
+ExecStop=/usr/bin/echo $unknown`,
+			`[Service]
 ExecStart=/usr/bin/echo "192.0.2.3"
 ExecStop=/usr/bin/echo 192.0.2.203
-ExecStop=/usr/bin/echo $unknown
-`
+ExecStop=/usr/bin/echo $unknown`,
+		},
+		{
+			// Substituting one value directly while falling back with the other
+			map[string]string{"$private_ipv4": "127.0.0.1"},
+			"$private_ipv4\n$public_ipv4",
+			"127.0.0.1\n1.2.3.4",
+		},
+		{
+			// Falling back to environment variables for both values
+			map[string]string{"foo": "bar"},
+			"$private_ipv4\n$public_ipv4",
+			"5.6.7.8\n1.2.3.4",
+		},
+		{
+			// No substitutions
+			nil,
+			"$private_ipv4\nfoobar",
+			"5.6.7.8\nfoobar",
+		},
+	} {
 
-	output := env.Apply(input)
-	if output != expected {
-		t.Fatalf("Environment incorrectly applied.\nOutput:\n%s\nExpected:\n%s", output, expected)
+		env := NewEnvironment("./", "./", "./", "", "", tt.subs)
+		got := env.Apply(tt.input)
+		if got != tt.out {
+			t.Fatalf("Environment incorrectly applied.\ngot:\n%s\nwant:\n%s", got, tt.out)
+		}
+	}
+}
+
+func TestEnvironmentFile(t *testing.T) {
+	subs := map[string]string{
+		"$public_ipv4":  "1.2.3.4",
+		"$private_ipv4": "5.6.7.8",
+	}
+	expect := "COREOS_PUBLIC_IPV4=1.2.3.4\nCOREOS_PRIVATE_IPV4=5.6.7.8\n"
+
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	env := NewEnvironment("./", "./", "./", "", "", subs)
+	ef := env.DefaultEnvironmentFile()
+	err = system.WriteEnvFile(ef, dir)
+	if err != nil {
+		t.Fatalf("WriteEnvFile failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "environment")
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+}
+
+func TestEnvironmentFileNil(t *testing.T) {
+	subs := map[string]string{
+		"$public_ipv4":  "",
+		"$private_ipv4": "",
+	}
+
+	env := NewEnvironment("./", "./", "./", "", "", subs)
+	ef := env.DefaultEnvironmentFile()
+	if ef != nil {
+		t.Fatalf("Environment file not nil: %v", ef)
 	}
 }

initialize/etcd.go

@@ -3,6 +3,7 @@ package initialize
 import (
 	"errors"
 	"fmt"
+	"sort"
 
 	"github.com/coreos/coreos-cloudinit/system"
 )
@@ -19,18 +20,25 @@ func (ee EtcdEnvironment) String() (out string) {
 		}
 	}
 
+	var sorted sort.StringSlice
+	for k, _ := range norm {
+		sorted = append(sorted, k)
+	}
+	sorted.Sort()
+
 	out += "[Service]\n"
 
-	for key, val := range norm {
+	for _, key := range sorted {
+		val := norm[key]
 		out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
 	}
 
 	return
 }
 
-// Unit creates a Unit file drop-in for etcd, using any configured
+// Units creates a Unit file drop-in for etcd, using any configured
 // options and adding a default MachineID if unset.
-func (ee EtcdEnvironment) Unit(root string) (*system.Unit, error) {
+func (ee EtcdEnvironment) Units(root string) ([]system.Unit, error) {
 	if ee == nil {
 		return nil, nil
 	}
@@ -45,10 +53,11 @@ func (ee EtcdEnvironment) Unit(root string) (*system.Unit, error) {
 		}
 	}
 
-	return &system.Unit{
+	etcd := system.Unit{
 		Name:    "etcd.service",
 		Runtime: true,
 		DropIn:  true,
 		Content: ee.String(),
-	}, nil
+	}
+	return []system.Unit{etcd}, nil
 }

initialize/etcd_test.go

@@ -59,7 +59,7 @@ Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
 }
 
 func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
-	ec := EtcdEnvironment{
+	ee := EtcdEnvironment{
 		"name":           "node001",
 		"discovery":      "http://disco.example.com/foobar",
 		"peer-bind-addr": "127.0.0.1:7002",
@@ -70,17 +70,18 @@ func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
 	}
 	defer os.RemoveAll(dir)
 
-	u, err := ec.Unit(dir)
+	uu, err := ee.Units(dir)
 	if err != nil {
 		t.Fatalf("Generating etcd unit failed: %v", err)
 	}
-	if u == nil {
-		t.Fatalf("Returned nil etcd unit unexpectedly")
+	if len(uu) != 1 {
+		t.Fatalf("Expected 1 unit to be returned, got %d", len(uu))
 	}
+	u := uu[0]
 
-	dst := system.UnitDestination(u, dir)
+	dst := u.Destination(dir)
 	os.Stderr.WriteString("writing to " + dir + "\n")
-	if err := system.PlaceUnit(u, dst); err != nil {
+	if err := system.PlaceUnit(&u, dst); err != nil {
 		t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
 	}
 
@@ -101,8 +102,8 @@ func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
 	}
 
 	expect := `[Service]
-Environment="ETCD_NAME=node001"
 Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_NAME=node001"
 Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
 `
 	if string(contents) != expect {
@@ -124,17 +125,18 @@ func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
 		t.Fatalf("Failed writing out /etc/machine-id: %v", err)
 	}
 
-	u, err := ee.Unit(dir)
+	uu, err := ee.Units(dir)
 	if err != nil {
 		t.Fatalf("Generating etcd unit failed: %v", err)
 	}
-	if u == nil {
-		t.Fatalf("Returned nil etcd unit unexpectedly")
+	if len(uu) == 0 {
+		t.Fatalf("Returned empty etcd units unexpectedly")
 	}
+	u := uu[0]
 
-	dst := system.UnitDestination(u, dir)
+	dst := u.Destination(dir)
 	os.Stderr.WriteString("writing to " + dir + "\n")
-	if err := system.PlaceUnit(u, dst); err != nil {
+	if err := system.PlaceUnit(&u, dst); err != nil {
 		t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
 	}
 
@@ -159,8 +161,8 @@ func TestEtcdEnvironmentWhenNil(t *testing.T) {
 	if ee != nil {
 		t.Fatalf("EtcdEnvironment is not nil")
 	}
-	u, err := ee.Unit("")
-	if u != nil || err != nil {
-		t.Fatalf("Unit returned a non-nil value for nil input")
+	uu, err := ee.Units("")
+	if len(uu) != 0 || err != nil {
+		t.Fatalf("Units returned value for nil input")
 	}
 }

initialize/fleet.go

@@ -19,16 +19,17 @@ func (fe FleetEnvironment) String() (out string) {
 	return
 }
 
-// Unit generates a Unit file drop-in for fleet, if any fleet options were
+// Units generates a Unit file drop-in for fleet, if any fleet options were
 // configured in cloud-config
-func (fe FleetEnvironment) Unit(root string) (*system.Unit, error) {
+func (fe FleetEnvironment) Units(root string) ([]system.Unit, error) {
 	if len(fe) < 1 {
 		return nil, nil
 	}
-	return &system.Unit{
+	fleet := system.Unit{
 		Name:    "fleet.service",
 		Runtime: true,
 		DropIn:  true,
 		Content: fe.String(),
-	}, nil
+	}
+	return []system.Unit{fleet}, nil
 }

initialize/fleet_test.go

@@ -19,20 +19,21 @@ Environment="FLEET_PUBLIC_IP=12.34.56.78"
 
 func TestFleetUnit(t *testing.T) {
 	cfg := make(FleetEnvironment, 0)
-	u, err := cfg.Unit("/")
-	if u != nil {
+	uu, err := cfg.Units("/")
+	if len(uu) != 0 {
 		t.Errorf("unexpectedly generated unit with empty FleetEnvironment")
 	}
 
 	cfg["public-ip"] = "12.34.56.78"
 
-	u, err = cfg.Unit("/")
+	uu, err = cfg.Units("/")
 	if err != nil {
 		t.Errorf("error generating fleet unit: %v", err)
 	}
-	if u == nil {
-		t.Fatalf("unexpectedly got nil unit generating fleet unit!")
+	if len(uu) != 1 {
+		t.Fatalf("expected 1 unit generated, got %d", len(uu))
 	}
+	u := uu[0]
 	if !u.Runtime {
 		t.Errorf("bad Runtime for generated fleet unit!")
 	}

initialize/manage_etc_hosts_test.go

@@ -50,9 +50,7 @@ func TestEtcHostsWrittenToDisk(t *testing.T) {
 		t.Fatalf("manageEtcHosts returned nil file unexpectedly")
 	}
 
-	f.Path = path.Join(dir, f.Path)
-
-	if err := system.WriteFile(f); err != nil {
+	if _, err := system.WriteFile(f, dir); err != nil {
 		t.Fatalf("Error writing EtcHosts: %v", err)
 	}
 

initialize/meta_data.go

@@ -0,0 +1,52 @@
+package initialize
+
+import "encoding/json"
+
+// ParseMetaData parses a JSON blob in the OpenStack metadata service format, and
+// converts it to a partially hydrated CloudConfig
+func ParseMetaData(contents string) (*CloudConfig, error) {
+	if len(contents) == 0 {
+		return nil, nil
+	}
+	var metadata struct {
+		SSHAuthorizedKeyMap map[string]string `json:"public_keys"`
+		Hostname            string            `json:"hostname"`
+		NetworkConfig       struct {
+			ContentPath string `json:"content_path"`
+		} `json:"network_config"`
+	}
+	if err := json.Unmarshal([]byte(contents), &metadata); err != nil {
+		return nil, err
+	}
+
+	var cfg CloudConfig
+	if len(metadata.SSHAuthorizedKeyMap) > 0 {
+		cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap))
+		for _, key := range metadata.SSHAuthorizedKeyMap {
+			cfg.SSHAuthorizedKeys = append(cfg.SSHAuthorizedKeys, key)
+		}
+	}
+	cfg.Hostname = metadata.Hostname
+	cfg.NetworkConfigPath = metadata.NetworkConfig.ContentPath
+	return &cfg, nil
+}
+
+// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service format,
+// and returns a substitution map possibly containing private_ipv4 and public_ipv4 addresses
+func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) {
+	var ips struct {
+		Public  string `json:"public-ipv4"`
+		Private string `json:"local-ipv4"`
+	}
+	if err := json.Unmarshal(contents, &ips); err != nil {
+		return nil, err
+	}
+	m := make(map[string]string)
+	if ips.Private != "" {
+		m["$private_ipv4"] = ips.Private
+	}
+	if ips.Public != "" {
+		m["$public_ipv4"] = ips.Public
+	}
+	return m, nil
+}

initialize/meta_data_test.go

@@ -0,0 +1,69 @@
+package initialize
+
+import "reflect"
+import "testing"
+
+func TestParseMetadata(t *testing.T) {
+	for i, tt := range []struct {
+		in   string
+		want *CloudConfig
+		err  bool
+	}{
+		{"", nil, false},
+		{`garbage, invalid json`, nil, true},
+		{`{"foo": "bar"}`, &CloudConfig{}, false},
+		{`{"network_config": {"content_path": "asdf"}}`, &CloudConfig{NetworkConfigPath: "asdf"}, false},
+		{`{"hostname": "turkleton"}`, &CloudConfig{Hostname: "turkleton"}, false},
+		{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"jill", "alice"}}, false},
+		{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
+	} {
+		got, err := ParseMetaData(tt.in)
+		if tt.err != (err != nil) {
+			t.Errorf("case #%d: bad error state: got %t, want %t (err=%v)", i, (err != nil), tt.err, err)
+		}
+		if got == nil {
+			if tt.want != nil {
+				t.Errorf("case #%d: unexpected nil output", i)
+			}
+		} else if tt.want == nil {
+			t.Errorf("case #%d: unexpected non-nil output", i)
+		} else {
+			if !reflect.DeepEqual(*got, *tt.want) {
+				t.Errorf("case #%d: bad output:\ngot\n%v\nwant\n%v", i, *got, *tt.want)
+			}
+		}
+	}
+
+}
+
+func TestExtractIPsFromMetadata(t *testing.T) {
+	for i, tt := range []struct {
+		in  []byte
+		err bool
+		out map[string]string
+	}{
+		{
+			[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4"}`),
+			false,
+			map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4"},
+		},
+		{
+			[]byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`),
+			false,
+			map[string]string{"$private_ipv4": "127.0.0.1"},
+		},
+		{
+			[]byte(`garbage`),
+			true,
+			nil,
+		},
+	} {
+		got, err := ExtractIPsFromMetadata(tt.in)
+		if (err != nil) != tt.err {
+			t.Errorf("bad error state (got %t, want %t)", err != nil, tt.err)
+		}
+		if !reflect.DeepEqual(got, tt.out) {
+			t.Errorf("case %d: got %s, want %s", i, got, tt.out)
+		}
+	}
+}

initialize/oem_test.go

@@ -31,8 +31,7 @@ func TestOEMReleaseWrittenToDisk(t *testing.T) {
 		t.Fatalf("OEMRelease returned nil file unexpectedly")
 	}
 
-	f.Path = path.Join(dir, f.Path)
-	if err := system.WriteFile(f); err != nil {
+	if _, err := system.WriteFile(f, dir); err != nil {
 		t.Fatalf("Writing of OEMRelease failed: %v", err)
 	}
 

initialize/ssh_keys.go

@@ -3,9 +3,8 @@ package initialize
 import (
 	"encoding/json"
 	"fmt"
-	"io/ioutil"
-	"net/http"
 
+	"github.com/coreos/coreos-cloudinit/pkg"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -25,22 +24,19 @@ func SSHImportKeysFromURL(system_user string, url string) error {
 }
 
 func fetchUserKeys(url string) ([]string, error) {
-	res, err := http.Get(url)
+	client := pkg.NewHttpClient()
+	data, err := client.GetRetry(url)
 	if err != nil {
 		return nil, err
 	}
-	defer res.Body.Close()
-	body, err := ioutil.ReadAll(res.Body)
-	if err != nil {
-		return nil, err
-	}
-	var data []UserKey
-	err = json.Unmarshal(body, &data)
+
+	var userKeys []UserKey
+	err = json.Unmarshal(data, &userKeys)
 	if err != nil {
 		return nil, err
 	}
 	keys := make([]string, 0)
-	for _, key := range data {
+	for _, key := range userKeys {
 		keys = append(keys, key.Key)
 	}
 	return keys, err

initialize/update.go

@@ -13,6 +13,7 @@ import (
 
 const (
 	locksmithUnit    = "locksmithd.service"
+	updateEngineUnit = "update-engine.service"
 )
 
 // updateOption represents a configurable update option, which, if set, will be
@@ -36,7 +37,6 @@ var updateOptions = []*updateOption{
 	&updateOption{
 		key:    "group",
 		prefix: "GROUP=",
-		valid:  []string{"master", "beta", "alpha", "stable"},
 	},
 	&updateOption{
 		key:    "server",
@@ -126,26 +126,40 @@ func (uc UpdateConfig) File(root string) (*system.File, error) {
 	}, nil
 }
 
-// GetUnit generates a locksmith system.Unit, if reboot-strategy was set in
-// cloud-config, for the cloud-init initializer to act on appropriately
-func (uc UpdateConfig) Unit(root string) (*system.Unit, error) {
-	strategy, ok := uc["reboot-strategy"]
-	if !ok {
-		return nil, nil
-	}
-
-	u := &system.Unit{
+// Units generates units for the cloud-init initializer to act on:
+// - a locksmith system.Unit, if "reboot-strategy" was set in cloud-config
+// - an update_engine system.Unit, if "group" was set in cloud-config
+func (uc UpdateConfig) Units(root string) ([]system.Unit, error) {
+	var units []system.Unit
+	if strategy, ok := uc["reboot-strategy"]; ok {
+		ls := &system.Unit{
 			Name:    locksmithUnit,
-		Enable:  true,
 			Command: "restart",
 			Mask:    false,
+			Runtime: true,
 		}
 
 		if strategy == "off" {
-		u.Enable = false
-		u.Command = "stop"
-		u.Mask = true
+			ls.Command = "stop"
+			ls.Mask = true
+		}
+		units = append(units, *ls)
 	}
 
-	return u, nil
+	rue := false
+	if _, ok := uc["group"]; ok {
+		rue = true
+	}
+	if _, ok := uc["server"]; ok {
+		rue = true
+	}
+	if rue {
+		ue := system.Unit{
+			Name:    updateEngineUnit,
+			Command: "restart",
+		}
+		units = append(units, ue)
+	}
+
+	return units, nil
 }

initialize/update_test.go

@@ -38,12 +38,12 @@ func TestEmptyUpdateConfig(t *testing.T) {
 	if f != nil {
 		t.Errorf("getting file from empty UpdateConfig should have returned nil, got %v", f)
 	}
-	u, err := uc.Unit("")
+	uu, err := uc.Units("")
 	if err != nil {
 		t.Error("unexpected error getting unit from empty UpdateConfig")
 	}
-	if u != nil {
-		t.Errorf("getting unit from empty UpdateConfig should have returned nil, got %v", u)
+	if len(uu) != 0 {
+		t.Errorf("getting unit from empty UpdateConfig should have returned zero units, got %d", len(uu))
 	}
 }
 
@@ -106,6 +106,21 @@ SERVER=http://foo.com`
 			t.Errorf("File has incorrect contents, got %v, want %v", got, want)
 		}
 	}
+
+	uu, err := u.Units(dir)
+	if err != nil {
+		t.Errorf("unexpected error getting units from UpdateConfig: %v", err)
+	} else if len(uu) != 1 {
+		t.Errorf("unexpected number of files returned from UpdateConfig: want 1, got %d", len(uu))
+	} else {
+		unit := uu[0]
+		if unit.Name != "update-engine.service" {
+			t.Errorf("bad name for generated unit: want update-engine.service, got %s", unit.Name)
+		}
+		if unit.Command != "restart" {
+			t.Errorf("bad command for generated unit: want restart, got %s", unit.Command)
+		}
+	}
 }
 
 func TestRebootStrategies(t *testing.T) {
@@ -145,12 +160,13 @@ func TestRebootStrategies(t *testing.T) {
 				t.Errorf("couldn't find expected line %v for reboot-strategy=%v", s.line)
 			}
 		}
-		u, err := uc.Unit(dir)
+		uu, err := uc.Units(dir)
 		if err != nil {
 			t.Errorf("failed to generate unit for reboot-strategy=%v!", s.name)
-		} else if u == nil {
-			t.Errorf("generated empty unit for reboot-strategy=%v", s.name)
+		} else if len(uu) != 1 {
+			t.Errorf("unexpected number of units for reboot-strategy=%v: %d", s.name, len(uu))
 		} else {
+			u := uu[0]
 			if u.Name != locksmithUnit {
 				t.Errorf("unit generated for reboot strategy=%v had bad name: %v", s.name, u.Name)
 			}
@@ -189,8 +205,7 @@ func TestUpdateConfWrittenToDisk(t *testing.T) {
 			t.Fatal("Unexpectedly got nil updateconfig file")
 		}
 
-		f.Path = path.Join(dir, f.Path)
-		if err := system.WriteFile(f); err != nil {
+		if _, err := system.WriteFile(f, dir); err != nil {
 			t.Fatalf("Error writing update config: %v", err)
 		}
 

initialize/user_data.go

@@ -9,6 +9,9 @@ import (
 )
 
 func ParseUserData(contents string) (interface{}, error) {
+	if len(contents) == 0 {
+		return nil, nil
+	}
 	header := strings.SplitN(contents, "\n", 2)[0]
 
 	// Explicitly trim the header so we can handle user-data from
@@ -19,14 +22,9 @@ func ParseUserData(contents string) (interface{}, error) {
 	if strings.HasPrefix(header, "#!") {
 		log.Printf("Parsing user-data as script")
 		return system.Script(contents), nil
-
 	} else if header == "#cloud-config" {
 		log.Printf("Parsing user-data as cloud-config")
-		cfg, err := NewCloudConfig(contents)
-		if err != nil {
-			log.Fatal(err.Error())
-		}
-		return *cfg, nil
+		return NewCloudConfig(contents)
 	} else {
 		return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
 	}

initialize/user_data_test.go

@@ -37,7 +37,7 @@ func TestParseConfigCRLF(t *testing.T) {
 		t.Fatalf("Failed parsing config: %v", err)
 	}
 
-	cfg := ud.(CloudConfig)
+	cfg := ud.(*CloudConfig)
 
 	if cfg.Hostname != "foo" {
 		t.Error("Failed parsing hostname from config")
@@ -47,3 +47,12 @@ func TestParseConfigCRLF(t *testing.T) {
 		t.Error("Parsed incorrect number of SSH keys")
 	}
 }
+
+func TestParseConfigEmpty(t *testing.T) {
+	i, e := ParseUserData(``)
+	if i != nil {
+		t.Error("ParseUserData of empty string returned non-nil unexpectedly")
+	} else if e != nil {
+		t.Error("ParseUserData of empty string returned error unexpectedly")
+	}
+}

initialize/workspace.go

@@ -3,6 +3,7 @@ package initialize
 import (
 	"io/ioutil"
 	"path"
+	"strings"
 
 	"github.com/coreos/coreos-cloudinit/system"
 )
@@ -28,21 +29,23 @@ func PersistScriptInWorkspace(script system.Script, workspace string) (string, e
 	}
 	tmp.Close()
 
+	relpath := strings.TrimPrefix(tmp.Name(), workspace)
+
 	file := system.File{
-		Path: tmp.Name(),
+		Path:               relpath,
 		RawFilePermissions: "0744",
 		Content:            string(script),
 	}
 
-	err = system.WriteFile(&file)
-	return file.Path, err
+	return system.WriteFile(&file, workspace)
 }
 
 func PersistUnitNameInWorkspace(name string, workspace string) error {
 	file := system.File{
-		Path: path.Join(workspace, "scripts", "unit-name"),
+		Path:               path.Join("scripts", "unit-name"),
 		RawFilePermissions: "0644",
 		Content:            name,
 	}
-	return system.WriteFile(&file)
+	_, err := system.WriteFile(&file, workspace)
+	return err
 }

network/interface.go

@@ -0,0 +1,253 @@
+package network
+
+import (
+	"fmt"
+	"strconv"
+)
+
+type InterfaceGenerator interface {
+	Name() string
+	Filename() string
+	Netdev() string
+	Link() string
+	Network() string
+}
+
+type networkInterface interface {
+	InterfaceGenerator
+	Children() []networkInterface
+	setConfigDepth(int)
+}
+
+type logicalInterface struct {
+	name        string
+	config      configMethod
+	children    []networkInterface
+	configDepth int
+}
+
+func (i *logicalInterface) Network() string {
+	config := fmt.Sprintf("[Match]\nName=%s\n\n[Network]\n", i.name)
+
+	for _, child := range i.children {
+		switch iface := child.(type) {
+		case *vlanInterface:
+			config += fmt.Sprintf("VLAN=%s\n", iface.name)
+		case *bondInterface:
+			config += fmt.Sprintf("Bond=%s\n", iface.name)
+		}
+	}
+
+	switch conf := i.config.(type) {
+	case configMethodStatic:
+		for _, nameserver := range conf.nameservers {
+			config += fmt.Sprintf("DNS=%s\n", nameserver)
+		}
+		if conf.address.IP != nil {
+			config += fmt.Sprintf("\n[Address]\nAddress=%s\n", conf.address.String())
+		}
+		for _, route := range conf.routes {
+			config += fmt.Sprintf("\n[Route]\nDestination=%s\nGateway=%s\n", route.destination.String(), route.gateway)
+		}
+	case configMethodDHCP:
+		config += "DHCP=true\n"
+	}
+
+	return config
+}
+
+func (i *logicalInterface) Link() string {
+	return ""
+}
+
+func (i *logicalInterface) Filename() string {
+	return fmt.Sprintf("%02x-%s", i.configDepth, i.name)
+}
+
+func (i *logicalInterface) Children() []networkInterface {
+	return i.children
+}
+
+func (i *logicalInterface) setConfigDepth(depth int) {
+	i.configDepth = depth
+}
+
+type physicalInterface struct {
+	logicalInterface
+}
+
+func (p *physicalInterface) Name() string {
+	return p.name
+}
+
+func (p *physicalInterface) Netdev() string {
+	return ""
+}
+
+type bondInterface struct {
+	logicalInterface
+	slaves []string
+}
+
+func (b *bondInterface) Name() string {
+	return b.name
+}
+
+func (b *bondInterface) Netdev() string {
+	return fmt.Sprintf("[NetDev]\nKind=bond\nName=%s\n", b.name)
+}
+
+type vlanInterface struct {
+	logicalInterface
+	id        int
+	rawDevice string
+}
+
+func (v *vlanInterface) Name() string {
+	return v.name
+}
+
+func (v *vlanInterface) Netdev() string {
+	config := fmt.Sprintf("[NetDev]\nKind=vlan\nName=%s\n", v.name)
+	switch c := v.config.(type) {
+	case configMethodStatic:
+		if c.hwaddress != nil {
+			config += fmt.Sprintf("MACAddress=%s\n", c.hwaddress)
+		}
+	case configMethodDHCP:
+		if c.hwaddress != nil {
+			config += fmt.Sprintf("MACAddress=%s\n", c.hwaddress)
+		}
+	}
+	config += fmt.Sprintf("\n[VLAN]\nId=%d\n", v.id)
+	return config
+}
+
+func buildInterfaces(stanzas []*stanzaInterface) []InterfaceGenerator {
+	interfaceMap := createInterfaces(stanzas)
+	linkAncestors(interfaceMap)
+	markConfigDepths(interfaceMap)
+
+	interfaces := make([]InterfaceGenerator, 0, len(interfaceMap))
+	for _, iface := range interfaceMap {
+		interfaces = append(interfaces, iface)
+	}
+
+	return interfaces
+}
+
+func createInterfaces(stanzas []*stanzaInterface) map[string]networkInterface {
+	interfaceMap := make(map[string]networkInterface)
+	for _, iface := range stanzas {
+		switch iface.kind {
+		case interfaceBond:
+			interfaceMap[iface.name] = &bondInterface{
+				logicalInterface{
+					name:     iface.name,
+					config:   iface.configMethod,
+					children: []networkInterface{},
+				},
+				iface.options["slaves"],
+			}
+			for _, slave := range iface.options["slaves"] {
+				if _, ok := interfaceMap[slave]; !ok {
+					interfaceMap[slave] = &physicalInterface{
+						logicalInterface{
+							name:     slave,
+							config:   configMethodManual{},
+							children: []networkInterface{},
+						},
+					}
+				}
+			}
+
+		case interfacePhysical:
+			if _, ok := iface.configMethod.(configMethodLoopback); ok {
+				continue
+			}
+			interfaceMap[iface.name] = &physicalInterface{
+				logicalInterface{
+					name:     iface.name,
+					config:   iface.configMethod,
+					children: []networkInterface{},
+				},
+			}
+
+		case interfaceVLAN:
+			var rawDevice string
+			id, _ := strconv.Atoi(iface.options["id"][0])
+			if device := iface.options["raw_device"]; len(device) == 1 {
+				rawDevice = device[0]
+				if _, ok := interfaceMap[rawDevice]; !ok {
+					interfaceMap[rawDevice] = &physicalInterface{
+						logicalInterface{
+							name:     rawDevice,
+							config:   configMethodManual{},
+							children: []networkInterface{},
+						},
+					}
+				}
+			}
+			interfaceMap[iface.name] = &vlanInterface{
+				logicalInterface{
+					name:     iface.name,
+					config:   iface.configMethod,
+					children: []networkInterface{},
+				},
+				id,
+				rawDevice,
+			}
+		}
+	}
+	return interfaceMap
+}
+
+func linkAncestors(interfaceMap map[string]networkInterface) {
+	for _, iface := range interfaceMap {
+		switch i := iface.(type) {
+		case *vlanInterface:
+			if parent, ok := interfaceMap[i.rawDevice]; ok {
+				switch p := parent.(type) {
+				case *physicalInterface:
+					p.children = append(p.children, iface)
+				case *bondInterface:
+					p.children = append(p.children, iface)
+				}
+			}
+		case *bondInterface:
+			for _, slave := range i.slaves {
+				if parent, ok := interfaceMap[slave]; ok {
+					switch p := parent.(type) {
+					case *physicalInterface:
+						p.children = append(p.children, iface)
+					case *bondInterface:
+						p.children = append(p.children, iface)
+					}
+				}
+			}
+		}
+	}
+}
+
+func markConfigDepths(interfaceMap map[string]networkInterface) {
+	rootInterfaceMap := make(map[string]networkInterface)
+	for k, v := range interfaceMap {
+		rootInterfaceMap[k] = v
+	}
+
+	for _, iface := range interfaceMap {
+		for _, child := range iface.Children() {
+			delete(rootInterfaceMap, child.Name())
+		}
+	}
+	for _, iface := range rootInterfaceMap {
+		setDepth(iface, 0)
+	}
+}
+
+func setDepth(iface networkInterface, depth int) {
+	iface.setConfigDepth(depth)
+	for _, child := range iface.Children() {
+		setDepth(child, depth+1)
+	}
+}

network/interface_test.go

@@ -0,0 +1,426 @@
+package network
+
+import (
+	"net"
+	"reflect"
+	"testing"
+)
+
+func TestPhysicalInterfaceName(t *testing.T) {
+	p := physicalInterface{logicalInterface{name: "testname"}}
+	if p.Name() != "testname" {
+		t.FailNow()
+	}
+}
+
+func TestPhysicalInterfaceNetdev(t *testing.T) {
+	p := physicalInterface{}
+	if p.Netdev() != "" {
+		t.FailNow()
+	}
+}
+
+func TestPhysicalInterfaceLink(t *testing.T) {
+	p := physicalInterface{}
+	if p.Link() != "" {
+		t.FailNow()
+	}
+}
+
+func TestPhysicalInterfaceNetwork(t *testing.T) {
+	p := physicalInterface{logicalInterface{
+		name: "testname",
+		children: []networkInterface{
+			&bondInterface{
+				logicalInterface{
+					name: "testbond1",
+				},
+				nil,
+			},
+			&vlanInterface{
+				logicalInterface{
+					name: "testvlan1",
+				},
+				1,
+				"",
+			},
+			&vlanInterface{
+				logicalInterface{
+					name: "testvlan2",
+				},
+				1,
+				"",
+			},
+		},
+	}}
+	network := `[Match]
+Name=testname
+
+[Network]
+Bond=testbond1
+VLAN=testvlan1
+VLAN=testvlan2
+`
+	if p.Network() != network {
+		t.FailNow()
+	}
+}
+
+func TestBondInterfaceName(t *testing.T) {
+	b := bondInterface{logicalInterface{name: "testname"}, nil}
+	if b.Name() != "testname" {
+		t.FailNow()
+	}
+}
+
+func TestBondInterfaceNetdev(t *testing.T) {
+	b := bondInterface{logicalInterface{name: "testname"}, nil}
+	netdev := `[NetDev]
+Kind=bond
+Name=testname
+`
+	if b.Netdev() != netdev {
+		t.FailNow()
+	}
+}
+
+func TestBondInterfaceLink(t *testing.T) {
+	b := bondInterface{}
+	if b.Link() != "" {
+		t.FailNow()
+	}
+}
+
+func TestBondInterfaceNetwork(t *testing.T) {
+	b := bondInterface{
+		logicalInterface{
+			name:   "testname",
+			config: configMethodDHCP{},
+			children: []networkInterface{
+				&bondInterface{
+					logicalInterface{
+						name: "testbond1",
+					},
+					nil,
+				},
+				&vlanInterface{
+					logicalInterface{
+						name: "testvlan1",
+					},
+					1,
+					"",
+				},
+				&vlanInterface{
+					logicalInterface{
+						name: "testvlan2",
+					},
+					1,
+					"",
+				},
+			},
+		},
+		nil,
+	}
+	network := `[Match]
+Name=testname
+
+[Network]
+Bond=testbond1
+VLAN=testvlan1
+VLAN=testvlan2
+DHCP=true
+`
+	if b.Network() != network {
+		t.FailNow()
+	}
+}
+
+func TestVLANInterfaceName(t *testing.T) {
+	v := vlanInterface{logicalInterface{name: "testname"}, 1, ""}
+	if v.Name() != "testname" {
+		t.FailNow()
+	}
+}
+
+func TestVLANInterfaceNetdev(t *testing.T) {
+	for _, tt := range []struct {
+		i vlanInterface
+		l string
+	}{
+		{
+			vlanInterface{logicalInterface{name: "testname"}, 1, ""},
+			"[NetDev]\nKind=vlan\nName=testname\n\n[VLAN]\nId=1\n",
+		},
+		{
+			vlanInterface{logicalInterface{name: "testname", config: configMethodStatic{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""},
+			"[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n",
+		},
+		{
+			vlanInterface{logicalInterface{name: "testname", config: configMethodDHCP{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""},
+			"[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n",
+		},
+	} {
+		if tt.i.Netdev() != tt.l {
+			t.Fatalf("bad netdev config (%q): got %q, want %q", tt.i, tt.i.Netdev(), tt.l)
+		}
+	}
+}
+
+func TestVLANInterfaceLink(t *testing.T) {
+	v := vlanInterface{}
+	if v.Link() != "" {
+		t.FailNow()
+	}
+}
+
+func TestVLANInterfaceNetwork(t *testing.T) {
+	v := vlanInterface{
+		logicalInterface{
+			name: "testname",
+			config: configMethodStatic{
+				address: net.IPNet{
+					IP:   []byte{192, 168, 1, 100},
+					Mask: []byte{255, 255, 255, 0},
+				},
+				nameservers: []net.IP{
+					[]byte{8, 8, 8, 8},
+				},
+				routes: []route{
+					route{
+						destination: net.IPNet{
+							IP:   []byte{0, 0, 0, 0},
+							Mask: []byte{0, 0, 0, 0},
+						},
+						gateway: []byte{1, 2, 3, 4},
+					},
+				},
+			},
+		},
+		0,
+		"",
+	}
+	network := `[Match]
+Name=testname
+
+[Network]
+DNS=8.8.8.8
+
+[Address]
+Address=192.168.1.100/24
+
+[Route]
+Destination=0.0.0.0/0
+Gateway=1.2.3.4
+`
+	if v.Network() != network {
+		t.Log(v.Network())
+		t.FailNow()
+	}
+}
+
+func TestBuildInterfacesLo(t *testing.T) {
+	stanzas := []*stanzaInterface{
+		&stanzaInterface{
+			name:         "lo",
+			kind:         interfacePhysical,
+			auto:         false,
+			configMethod: configMethodLoopback{},
+			options:      map[string][]string{},
+		},
+	}
+	interfaces := buildInterfaces(stanzas)
+	if len(interfaces) != 0 {
+		t.FailNow()
+	}
+}
+
+func TestBuildInterfacesBlindBond(t *testing.T) {
+	stanzas := []*stanzaInterface{
+		{
+			name:         "bond0",
+			kind:         interfaceBond,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options: map[string][]string{
+				"slaves": []string{"eth0"},
+			},
+		},
+	}
+	interfaces := buildInterfaces(stanzas)
+	bond0 := &bondInterface{
+		logicalInterface{
+			name:        "bond0",
+			config:      configMethodManual{},
+			children:    []networkInterface{},
+			configDepth: 1,
+		},
+		[]string{"eth0"},
+	}
+	eth0 := &physicalInterface{
+		logicalInterface{
+			name:        "eth0",
+			config:      configMethodManual{},
+			children:    []networkInterface{bond0},
+			configDepth: 0,
+		},
+	}
+	expect := []InterfaceGenerator{bond0, eth0}
+	if !reflect.DeepEqual(interfaces, expect) {
+		t.FailNow()
+	}
+}
+
+func TestBuildInterfacesBlindVLAN(t *testing.T) {
+	stanzas := []*stanzaInterface{
+		{
+			name:         "vlan0",
+			kind:         interfaceVLAN,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options: map[string][]string{
+				"id":         []string{"0"},
+				"raw_device": []string{"eth0"},
+			},
+		},
+	}
+	interfaces := buildInterfaces(stanzas)
+	vlan0 := &vlanInterface{
+		logicalInterface{
+			name:        "vlan0",
+			config:      configMethodManual{},
+			children:    []networkInterface{},
+			configDepth: 1,
+		},
+		0,
+		"eth0",
+	}
+	eth0 := &physicalInterface{
+		logicalInterface{
+			name:        "eth0",
+			config:      configMethodManual{},
+			children:    []networkInterface{vlan0},
+			configDepth: 0,
+		},
+	}
+	expect := []InterfaceGenerator{eth0, vlan0}
+	if !reflect.DeepEqual(interfaces, expect) {
+		t.FailNow()
+	}
+}
+
+func TestBuildInterfaces(t *testing.T) {
+	stanzas := []*stanzaInterface{
+		&stanzaInterface{
+			name:         "eth0",
+			kind:         interfacePhysical,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options:      map[string][]string{},
+		},
+		&stanzaInterface{
+			name:         "bond0",
+			kind:         interfaceBond,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options: map[string][]string{
+				"slaves": []string{"eth0"},
+			},
+		},
+		&stanzaInterface{
+			name:         "bond1",
+			kind:         interfaceBond,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options: map[string][]string{
+				"slaves": []string{"bond0"},
+			},
+		},
+		&stanzaInterface{
+			name:         "vlan0",
+			kind:         interfaceVLAN,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options: map[string][]string{
+				"id":         []string{"0"},
+				"raw_device": []string{"eth0"},
+			},
+		},
+		&stanzaInterface{
+			name:         "vlan1",
+			kind:         interfaceVLAN,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options: map[string][]string{
+				"id":         []string{"1"},
+				"raw_device": []string{"bond0"},
+			},
+		},
+	}
+	interfaces := buildInterfaces(stanzas)
+	vlan1 := &vlanInterface{
+		logicalInterface{
+			name:        "vlan1",
+			config:      configMethodManual{},
+			children:    []networkInterface{},
+			configDepth: 2,
+		},
+		1,
+		"bond0",
+	}
+	vlan0 := &vlanInterface{
+		logicalInterface{
+			name:        "vlan0",
+			config:      configMethodManual{},
+			children:    []networkInterface{},
+			configDepth: 1,
+		},
+		0,
+		"eth0",
+	}
+	bond1 := &bondInterface{
+		logicalInterface{
+			name:        "bond1",
+			config:      configMethodManual{},
+			children:    []networkInterface{},
+			configDepth: 2,
+		},
+		[]string{"bond0"},
+	}
+	bond0 := &bondInterface{
+		logicalInterface{
+			name:        "bond0",
+			config:      configMethodManual{},
+			children:    []networkInterface{bond1, vlan1},
+			configDepth: 1,
+		},
+		[]string{"eth0"},
+	}
+	eth0 := &physicalInterface{
+		logicalInterface{
+			name:        "eth0",
+			config:      configMethodManual{},
+			children:    []networkInterface{bond0, vlan0},
+			configDepth: 0,
+		},
+	}
+	expect := []InterfaceGenerator{eth0, bond0, bond1, vlan0, vlan1}
+	if !reflect.DeepEqual(interfaces, expect) {
+		t.FailNow()
+	}
+}
+
+func TestFilename(t *testing.T) {
+	for _, tt := range []struct {
+		i logicalInterface
+		f string
+	}{
+		{logicalInterface{name: "iface", configDepth: 0}, "00-iface"},
+		{logicalInterface{name: "iface", configDepth: 9}, "09-iface"},
+		{logicalInterface{name: "iface", configDepth: 10}, "0a-iface"},
+		{logicalInterface{name: "iface", configDepth: 53}, "35-iface"},
+	} {
+		if tt.i.Filename() != tt.f {
+			t.Fatalf("bad filename (%q): got %q, want %q", tt.i, tt.i.Filename(), tt.f)
+		}
+	}
+}

network/network.go

@@ -0,0 +1,45 @@
+package network
+
+import (
+	"strings"
+)
+
+func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) {
+	lines := formatConfig(config)
+	stanzas, err := parseStanzas(lines)
+	if err != nil {
+		return nil, err
+	}
+
+	interfaces := make([]*stanzaInterface, 0, len(stanzas))
+	for _, stanza := range stanzas {
+		switch s := stanza.(type) {
+		case *stanzaInterface:
+			interfaces = append(interfaces, s)
+		}
+	}
+
+	return buildInterfaces(interfaces), nil
+}
+
+func formatConfig(config string) []string {
+	lines := []string{}
+	config = strings.Replace(config, "\\\n", "", -1)
+	for config != "" {
+		split := strings.SplitN(config, "\n", 2)
+		line := strings.TrimSpace(split[0])
+
+		if len(split) == 2 {
+			config = split[1]
+		} else {
+			config = ""
+		}
+
+		if strings.HasPrefix(line, "#") || line == "" {
+			continue
+		}
+
+		lines = append(lines, line)
+	}
+	return lines
+}

network/network_test.go

@@ -0,0 +1,42 @@
+package network
+
+import (
+	"testing"
+)
+
+func TestFormatConfigs(t *testing.T) {
+	for in, n := range map[string]int{
+		"":                                                    0,
+		"line1\\\nis long":                                    1,
+		"#comment":                                            0,
+		"#comment\\\ncomment":                                 0,
+		"  #comment \\\n comment\nline 1\nline 2\\\n is long": 2,
+	} {
+		lines := formatConfig(in)
+		if len(lines) != n {
+			t.Fatalf("bad number of lines for config %q: got %d, want %d", in, len(lines), n)
+		}
+	}
+}
+
+func TestProcessDebianNetconf(t *testing.T) {
+	for _, tt := range []struct {
+		in   string
+		fail bool
+		n    int
+	}{
+		{"", false, 0},
+		{"iface", true, -1},
+		{"auto eth1\nauto eth2", false, 0},
+		{"iface eth1 inet manual", false, 1},
+	} {
+		interfaces, err := ProcessDebianNetconf(tt.in)
+		failed := err != nil
+		if tt.fail != failed {
+			t.Fatalf("bad failure state for %q: got %b, want %b", failed, tt.fail)
+		}
+		if tt.n != -1 && tt.n != len(interfaces) {
+			t.Fatalf("bad number of interfaces for %q: got %d, want %q", tt.in, len(interfaces), tt.n)
+		}
+	}
+}

network/stanza.go

@@ -0,0 +1,322 @@
+package network
+
+import (
+	"fmt"
+	"net"
+	"strconv"
+	"strings"
+)
+
+type stanza interface{}
+
+type stanzaAuto struct {
+	interfaces []string
+}
+
+type stanzaInterface struct {
+	name         string
+	kind         interfaceKind
+	auto         bool
+	configMethod configMethod
+	options      map[string][]string
+}
+
+type interfaceKind int
+
+const (
+	interfaceBond = interfaceKind(iota)
+	interfacePhysical
+	interfaceVLAN
+)
+
+type route struct {
+	destination net.IPNet
+	gateway     net.IP
+}
+
+type configMethod interface{}
+
+type configMethodStatic struct {
+	address     net.IPNet
+	nameservers []net.IP
+	routes      []route
+	hwaddress   net.HardwareAddr
+}
+
+type configMethodLoopback struct{}
+
+type configMethodManual struct{}
+
+type configMethodDHCP struct {
+	hwaddress net.HardwareAddr
+}
+
+func parseStanzas(lines []string) (stanzas []stanza, err error) {
+	rawStanzas, err := splitStanzas(lines)
+	if err != nil {
+		return nil, err
+	}
+
+	stanzas = make([]stanza, 0, len(rawStanzas))
+	for _, rawStanza := range rawStanzas {
+		if stanza, err := parseStanza(rawStanza); err == nil {
+			stanzas = append(stanzas, stanza)
+		} else {
+			return nil, err
+		}
+	}
+
+	autos := make([]string, 0)
+	interfaceMap := make(map[string]*stanzaInterface)
+	for _, stanza := range stanzas {
+		switch c := stanza.(type) {
+		case *stanzaAuto:
+			autos = append(autos, c.interfaces...)
+		case *stanzaInterface:
+			interfaceMap[c.name] = c
+		}
+	}
+
+	// Apply the auto attribute
+	for _, auto := range autos {
+		if iface, ok := interfaceMap[auto]; ok {
+			iface.auto = true
+		}
+	}
+
+	return stanzas, nil
+}
+
+func splitStanzas(lines []string) ([][]string, error) {
+	var curStanza []string
+	stanzas := make([][]string, 0)
+	for _, line := range lines {
+		if isStanzaStart(line) {
+			if curStanza != nil {
+				stanzas = append(stanzas, curStanza)
+			}
+			curStanza = []string{line}
+		} else if curStanza != nil {
+			curStanza = append(curStanza, line)
+		} else {
+			return nil, fmt.Errorf("missing stanza start %q", line)
+		}
+	}
+
+	if curStanza != nil {
+		stanzas = append(stanzas, curStanza)
+	}
+
+	return stanzas, nil
+}
+
+func isStanzaStart(line string) bool {
+	switch strings.Split(line, " ")[0] {
+	case "auto":
+		fallthrough
+	case "iface":
+		fallthrough
+	case "mapping":
+		return true
+	}
+
+	if strings.HasPrefix(line, "allow-") {
+		return true
+	}
+
+	return false
+}
+
+func parseStanza(rawStanza []string) (stanza, error) {
+	if len(rawStanza) == 0 {
+		panic("empty stanza")
+	}
+	tokens := strings.Fields(rawStanza[0])
+	if len(tokens) < 2 {
+		return nil, fmt.Errorf("malformed stanza start %q", rawStanza[0])
+	}
+
+	kind := tokens[0]
+	attributes := tokens[1:]
+
+	switch kind {
+	case "auto":
+		return parseAutoStanza(attributes, rawStanza[1:])
+	case "iface":
+		return parseInterfaceStanza(attributes, rawStanza[1:])
+	default:
+		return nil, fmt.Errorf("unknown stanza %q", kind)
+	}
+}
+
+func parseAutoStanza(attributes []string, options []string) (*stanzaAuto, error) {
+	return &stanzaAuto{interfaces: attributes}, nil
+}
+
+func parseInterfaceStanza(attributes []string, options []string) (*stanzaInterface, error) {
+	if len(attributes) != 3 {
+		return nil, fmt.Errorf("incorrect number of attributes")
+	}
+
+	iface := attributes[0]
+	confMethod := attributes[2]
+
+	optionMap := make(map[string][]string, 0)
+	for _, option := range options {
+		if strings.HasPrefix(option, "post-up") {
+			tokens := strings.SplitAfterN(option, " ", 2)
+			if len(tokens) != 2 {
+				continue
+			}
+			if v, ok := optionMap["post-up"]; ok {
+				optionMap["post-up"] = append(v, tokens[1])
+			} else {
+				optionMap["post-up"] = []string{tokens[1]}
+			}
+		} else if strings.HasPrefix(option, "pre-down") {
+			tokens := strings.SplitAfterN(option, " ", 2)
+			if len(tokens) != 2 {
+				continue
+			}
+			if v, ok := optionMap["pre-down"]; ok {
+				optionMap["pre-down"] = append(v, tokens[1])
+			} else {
+				optionMap["pre-down"] = []string{tokens[1]}
+			}
+		} else {
+			tokens := strings.Fields(option)
+			optionMap[tokens[0]] = tokens[1:]
+		}
+	}
+
+	var conf configMethod
+	switch confMethod {
+	case "static":
+		config := configMethodStatic{
+			routes:      make([]route, 0),
+			nameservers: make([]net.IP, 0),
+		}
+		if addresses, ok := optionMap["address"]; ok {
+			if len(addresses) == 1 {
+				config.address.IP = net.ParseIP(addresses[0])
+			}
+		}
+		if netmasks, ok := optionMap["netmask"]; ok {
+			if len(netmasks) == 1 {
+				config.address.Mask = net.IPMask(net.ParseIP(netmasks[0]).To4())
+			}
+		}
+		if config.address.IP == nil || config.address.Mask == nil {
+			return nil, fmt.Errorf("malformed static network config for %q", iface)
+		}
+		if gateways, ok := optionMap["gateway"]; ok {
+			if len(gateways) == 1 {
+				config.routes = append(config.routes, route{
+					destination: net.IPNet{
+						IP:   net.IPv4(0, 0, 0, 0),
+						Mask: net.IPv4Mask(0, 0, 0, 0),
+					},
+					gateway: net.ParseIP(gateways[0]),
+				})
+			}
+		}
+		if hwaddress, err := parseHwaddress(optionMap, iface); err == nil {
+			config.hwaddress = hwaddress
+		} else {
+			return nil, err
+		}
+		for _, nameserver := range optionMap["dns-nameservers"] {
+			config.nameservers = append(config.nameservers, net.ParseIP(nameserver))
+		}
+		for _, postup := range optionMap["post-up"] {
+			if strings.HasPrefix(postup, "route add") {
+				route := route{}
+				fields := strings.Fields(postup)
+				for i, field := range fields[:len(fields)-1] {
+					switch field {
+					case "-net":
+						route.destination.IP = net.ParseIP(fields[i+1])
+					case "netmask":
+						route.destination.Mask = net.IPMask(net.ParseIP(fields[i+1]).To4())
+					case "gw":
+						route.gateway = net.ParseIP(fields[i+1])
+					}
+				}
+				if route.destination.IP != nil && route.destination.Mask != nil && route.gateway != nil {
+					config.routes = append(config.routes, route)
+				}
+			}
+		}
+		conf = config
+	case "loopback":
+		conf = configMethodLoopback{}
+	case "manual":
+		conf = configMethodManual{}
+	case "dhcp":
+		config := configMethodDHCP{}
+		if hwaddress, err := parseHwaddress(optionMap, iface); err == nil {
+			config.hwaddress = hwaddress
+		} else {
+			return nil, err
+		}
+		conf = config
+	default:
+		return nil, fmt.Errorf("invalid config method %q", confMethod)
+	}
+
+	if _, ok := optionMap["vlan_raw_device"]; ok {
+		return parseVLANStanza(iface, conf, attributes, optionMap)
+	}
+
+	if strings.Contains(iface, ".") {
+		return parseVLANStanza(iface, conf, attributes, optionMap)
+	}
+
+	if _, ok := optionMap["bond-slaves"]; ok {
+		return parseBondStanza(iface, conf, attributes, optionMap)
+	}
+
+	return parsePhysicalStanza(iface, conf, attributes, optionMap)
+}
+
+func parseHwaddress(options map[string][]string, iface string) (net.HardwareAddr, error) {
+	if hwaddress, ok := options["hwaddress"]; ok && len(hwaddress) == 2 {
+		switch hwaddress[0] {
+		case "ether":
+			if address, err := net.ParseMAC(hwaddress[1]); err == nil {
+				return address, nil
+			}
+			return nil, fmt.Errorf("malformed hwaddress option for %q", iface)
+		}
+	}
+	return nil, nil
+}
+
+func parseBondStanza(iface string, conf configMethod, attributes []string, options map[string][]string) (*stanzaInterface, error) {
+	options["slaves"] = options["bond-slaves"]
+	return &stanzaInterface{name: iface, kind: interfaceBond, configMethod: conf, options: options}, nil
+}
+
+func parsePhysicalStanza(iface string, conf configMethod, attributes []string, options map[string][]string) (*stanzaInterface, error) {
+	return &stanzaInterface{name: iface, kind: interfacePhysical, configMethod: conf, options: options}, nil
+}
+
+func parseVLANStanza(iface string, conf configMethod, attributes []string, options map[string][]string) (*stanzaInterface, error) {
+	var id string
+	if strings.Contains(iface, ".") {
+		tokens := strings.Split(iface, ".")
+		id = tokens[len(tokens)-1]
+	} else if strings.HasPrefix(iface, "vlan") {
+		id = strings.TrimPrefix(iface, "vlan")
+	} else {
+		return nil, fmt.Errorf("malformed vlan name %q", iface)
+	}
+
+	if _, err := strconv.Atoi(id); err != nil {
+		return nil, fmt.Errorf("malformed vlan name %q", iface)
+	}
+	options["id"] = []string{id}
+	options["raw_device"] = options["vlan_raw_device"]
+
+	return &stanzaInterface{name: iface, kind: interfaceVLAN, configMethod: conf, options: options}, nil
+}

network/stanza_test.go

@@ -0,0 +1,543 @@
+package network
+
+import (
+	"net"
+	"reflect"
+	"strings"
+	"testing"
+)
+
+func TestSplitStanzasNoParent(t *testing.T) {
+	in := []string{"test"}
+	e := "missing stanza start"
+	_, err := splitStanzas(in)
+	if err == nil || !strings.HasPrefix(err.Error(), e) {
+		t.Fatalf("bad error for splitStanzas(%q): got %q, want %q", in, err, e)
+	}
+}
+
+func TestBadParseStanzas(t *testing.T) {
+	for in, e := range map[string]string{
+		"":                 "missing stanza start",
+		"iface":            "malformed stanza start",
+		"allow-?? unknown": "unknown stanza",
+	} {
+		_, err := parseStanzas([]string{in})
+		if err == nil || !strings.HasPrefix(err.Error(), e) {
+			t.Fatalf("bad error for parseStanzas(%q): got %q, want %q", in, err, e)
+		}
+
+	}
+}
+
+func TestBadParseInterfaceStanza(t *testing.T) {
+	for _, tt := range []struct {
+		in   []string
+		opts []string
+		e    string
+	}{
+		{[]string{}, nil, "incorrect number of attributes"},
+		{[]string{"eth", "inet", "invalid"}, nil, "invalid config method"},
+		{[]string{"eth", "inet", "static"}, []string{"address 192.168.1.100"}, "malformed static network config"},
+		{[]string{"eth", "inet", "static"}, []string{"netmask 255.255.255.0"}, "malformed static network config"},
+		{[]string{"eth", "inet", "static"}, []string{"address invalid", "netmask 255.255.255.0"}, "malformed static network config"},
+		{[]string{"eth", "inet", "static"}, []string{"address 192.168.1.100", "netmask invalid"}, "malformed static network config"},
+		{[]string{"eth", "inet", "static"}, []string{"address 192.168.1.100", "netmask 255.255.255.0", "hwaddress ether NotAnAddress"}, "malformed hwaddress option"},
+		{[]string{"eth", "inet", "dhcp"}, []string{"hwaddress ether NotAnAddress"}, "malformed hwaddress option"},
+	} {
+		_, err := parseInterfaceStanza(tt.in, tt.opts)
+		if err == nil || !strings.HasPrefix(err.Error(), tt.e) {
+			t.Fatalf("bad error parsing interface stanza %q: got %q, want %q", tt.in, err.Error(), tt.e)
+		}
+	}
+}
+
+func TestBadParseVLANStanzas(t *testing.T) {
+	conf := configMethodManual{}
+	options := map[string][]string{}
+	for _, in := range []string{"myvlan", "eth.vlan"} {
+		_, err := parseVLANStanza(in, conf, nil, options)
+		if err == nil || !strings.HasPrefix(err.Error(), "malformed vlan name") {
+			t.Fatalf("did not error on bad vlan %q", in)
+		}
+	}
+}
+
+func TestSplitStanzas(t *testing.T) {
+	expect := [][]string{
+		{"auto lo"},
+		{"iface eth1", "option: 1"},
+		{"mapping"},
+		{"allow-"},
+	}
+	lines := make([]string, 0, 5)
+	for _, stanza := range expect {
+		for _, line := range stanza {
+			lines = append(lines, line)
+		}
+	}
+
+	stanzas, err := splitStanzas(lines)
+	if err != nil {
+		t.FailNow()
+	}
+	for i, stanza := range stanzas {
+		if len(stanza) != len(expect[i]) {
+			t.FailNow()
+		}
+		for j, line := range stanza {
+			if line != expect[i][j] {
+				t.FailNow()
+			}
+		}
+	}
+}
+
+func TestParseStanzaNil(t *testing.T) {
+	defer func() {
+		if r := recover(); r == nil {
+			t.Fatal("parseStanza(nil) did not panic")
+		}
+	}()
+	parseStanza(nil)
+}
+
+func TestParseStanzaSuccess(t *testing.T) {
+	for _, in := range []string{
+		"auto a",
+		"iface a inet manual",
+	} {
+		if _, err := parseStanza([]string{in}); err != nil {
+			t.Fatalf("unexpected error parsing stanza %q: %s", in, err)
+		}
+	}
+}
+
+func TestParseAutoStanza(t *testing.T) {
+	interfaces := []string{"test", "attribute"}
+	stanza, err := parseAutoStanza(interfaces, nil)
+	if err != nil {
+		t.Fatalf("unexpected error parsing auto stanza %q: %s", interfaces, err)
+	}
+	if !reflect.DeepEqual(stanza.interfaces, interfaces) {
+		t.FailNow()
+	}
+}
+
+func TestParseBondStanzaNoSlaves(t *testing.T) {
+	bond, err := parseBondStanza("", nil, nil, map[string][]string{})
+	if err != nil {
+		t.FailNow()
+	}
+	if bond.options["slaves"] != nil {
+		t.FailNow()
+	}
+}
+
+func TestParseBondStanza(t *testing.T) {
+	conf := configMethodManual{}
+	options := map[string][]string{
+		"bond-slaves": []string{"1", "2"},
+	}
+	bond, err := parseBondStanza("test", conf, nil, options)
+	if err != nil {
+		t.FailNow()
+	}
+	if bond.name != "test" {
+		t.FailNow()
+	}
+	if bond.kind != interfaceBond {
+		t.FailNow()
+	}
+	if bond.configMethod != conf {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(bond.options["slaves"], options["bond-slaves"]) {
+		t.FailNow()
+	}
+}
+
+func TestParsePhysicalStanza(t *testing.T) {
+	conf := configMethodManual{}
+	options := map[string][]string{
+		"a": []string{"1", "2"},
+		"b": []string{"1"},
+	}
+	physical, err := parsePhysicalStanza("test", conf, nil, options)
+	if err != nil {
+		t.FailNow()
+	}
+	if physical.name != "test" {
+		t.FailNow()
+	}
+	if physical.kind != interfacePhysical {
+		t.FailNow()
+	}
+	if physical.configMethod != conf {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(physical.options, options) {
+		t.FailNow()
+	}
+}
+
+func TestParseVLANStanzas(t *testing.T) {
+	conf := configMethodManual{}
+	options := map[string][]string{}
+	for _, in := range []string{"vlan25", "eth.25"} {
+		vlan, err := parseVLANStanza(in, conf, nil, options)
+		if err != nil {
+			t.Fatalf("unexpected error from parseVLANStanza(%q): %s", in, err)
+		}
+		if !reflect.DeepEqual(vlan.options["id"], []string{"25"}) {
+			t.FailNow()
+		}
+	}
+}
+
+func TestParseInterfaceStanzaStaticAddress(t *testing.T) {
+	options := []string{"address 192.168.1.100", "netmask 255.255.255.0"}
+	expect := net.IPNet{
+		IP:   net.IPv4(192, 168, 1, 100),
+		Mask: net.IPv4Mask(255, 255, 255, 0),
+	}
+
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	static, ok := iface.configMethod.(configMethodStatic)
+	if !ok {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(static.address, expect) {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaStaticGateway(t *testing.T) {
+	options := []string{"address 192.168.1.100", "netmask 255.255.255.0", "gateway 192.168.1.1"}
+	expect := []route{
+		{
+			destination: net.IPNet{
+				IP:   net.IPv4(0, 0, 0, 0),
+				Mask: net.IPv4Mask(0, 0, 0, 0),
+			},
+			gateway: net.IPv4(192, 168, 1, 1),
+		},
+	}
+
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	static, ok := iface.configMethod.(configMethodStatic)
+	if !ok {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(static.routes, expect) {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaStaticDNS(t *testing.T) {
+	options := []string{"address 192.168.1.100", "netmask 255.255.255.0", "dns-nameservers 192.168.1.10 192.168.1.11 192.168.1.12"}
+	expect := []net.IP{
+		net.IPv4(192, 168, 1, 10),
+		net.IPv4(192, 168, 1, 11),
+		net.IPv4(192, 168, 1, 12),
+	}
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	static, ok := iface.configMethod.(configMethodStatic)
+	if !ok {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(static.nameservers, expect) {
+		t.FailNow()
+	}
+}
+
+func TestBadParseInterfaceStanzasStaticPostUp(t *testing.T) {
+	for _, in := range []string{
+		"post-up invalid",
+		"post-up route add",
+		"post-up route add -net",
+		"post-up route add gw",
+		"post-up route add netmask",
+		"gateway",
+		"gateway 192.168.1.1 192.168.1.2",
+	} {
+		options := []string{"address 192.168.1.100", "netmask 255.255.255.0", in}
+		iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
+		if err != nil {
+			t.Fatalf("parseInterfaceStanza with options %s got unexpected error", options)
+		}
+		static, ok := iface.configMethod.(configMethodStatic)
+		if !ok {
+			t.Fatalf("parseInterfaceStanza with options %s did not return configMethodStatic", options)
+		}
+		if len(static.routes) != 0 {
+			t.Fatalf("parseInterfaceStanza with options %s did not return zero-length static routes", options)
+		}
+	}
+}
+
+func TestParseInterfaceStanzaStaticPostUp(t *testing.T) {
+	options := []string{
+		"address 192.168.1.100",
+		"netmask 255.255.255.0",
+		"post-up route add gw 192.168.1.1 -net 192.168.1.0 netmask 255.255.255.0",
+	}
+	expect := []route{
+		{
+			destination: net.IPNet{
+				IP:   net.IPv4(192, 168, 1, 0),
+				Mask: net.IPv4Mask(255, 255, 255, 0),
+			},
+			gateway: net.IPv4(192, 168, 1, 1),
+		},
+	}
+
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	static, ok := iface.configMethod.(configMethodStatic)
+	if !ok {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(static.routes, expect) {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaLoopback(t *testing.T) {
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "loopback"}, nil)
+	if err != nil {
+		t.FailNow()
+	}
+	if _, ok := iface.configMethod.(configMethodLoopback); !ok {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaManual(t *testing.T) {
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, nil)
+	if err != nil {
+		t.FailNow()
+	}
+	if _, ok := iface.configMethod.(configMethodManual); !ok {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaDHCP(t *testing.T) {
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "dhcp"}, nil)
+	if err != nil {
+		t.FailNow()
+	}
+	if _, ok := iface.configMethod.(configMethodDHCP); !ok {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaPostUpOption(t *testing.T) {
+	options := []string{
+		"post-up",
+		"post-up 1 2",
+		"post-up 3 4",
+	}
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(iface.options["post-up"], []string{"1 2", "3 4"}) {
+		t.Log(iface.options["post-up"])
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaPreDownOption(t *testing.T) {
+	options := []string{
+		"pre-down",
+		"pre-down 3",
+		"pre-down 4",
+	}
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(iface.options["pre-down"], []string{"3", "4"}) {
+		t.Log(iface.options["pre-down"])
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaEmptyOption(t *testing.T) {
+	options := []string{
+		"test",
+	}
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(iface.options["test"], []string{}) {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaOptions(t *testing.T) {
+	options := []string{
+		"test1 1",
+		"test2 2 3",
+		"test1 5 6",
+	}
+	iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
+	if err != nil {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(iface.options["test1"], []string{"5", "6"}) {
+		t.Log(iface.options["test1"])
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(iface.options["test2"], []string{"2", "3"}) {
+		t.Log(iface.options["test2"])
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaHwaddress(t *testing.T) {
+	for _, tt := range []struct {
+		attr []string
+		opt  []string
+		hw   net.HardwareAddr
+	}{
+		{
+			[]string{"mybond", "inet", "dhcp"},
+			[]string{},
+			nil,
+		},
+		{
+			[]string{"mybond", "inet", "dhcp"},
+			[]string{"hwaddress ether 00:01:02:03:04:05"},
+			net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5}),
+		},
+		{
+			[]string{"mybond", "inet", "static"},
+			[]string{"hwaddress ether 00:01:02:03:04:05", "address 192.168.1.100", "netmask 255.255.255.0"},
+			net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5}),
+		},
+	} {
+		iface, err := parseInterfaceStanza(tt.attr, tt.opt)
+		if err != nil {
+			t.Fatalf("error in parseInterfaceStanza (%q, %q): %q", tt.attr, tt.opt, err)
+		}
+		switch c := iface.configMethod.(type) {
+		case configMethodStatic:
+			if !reflect.DeepEqual(c.hwaddress, tt.hw) {
+				t.Fatalf("bad hwaddress (%q, %q): got %q, want %q", tt.attr, tt.opt, c.hwaddress, tt.hw)
+			}
+		case configMethodDHCP:
+			if !reflect.DeepEqual(c.hwaddress, tt.hw) {
+				t.Fatalf("bad hwaddress (%q, %q): got %q, want %q", tt.attr, tt.opt, c.hwaddress, tt.hw)
+			}
+		}
+	}
+}
+
+func TestParseInterfaceStanzaBond(t *testing.T) {
+	iface, err := parseInterfaceStanza([]string{"mybond", "inet", "manual"}, []string{"bond-slaves eth"})
+	if err != nil {
+		t.FailNow()
+	}
+	if iface.kind != interfaceBond {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaVLANName(t *testing.T) {
+	iface, err := parseInterfaceStanza([]string{"eth0.1", "inet", "manual"}, nil)
+	if err != nil {
+		t.FailNow()
+	}
+	if iface.kind != interfaceVLAN {
+		t.FailNow()
+	}
+}
+
+func TestParseInterfaceStanzaVLANOption(t *testing.T) {
+	iface, err := parseInterfaceStanza([]string{"vlan1", "inet", "manual"}, []string{"vlan_raw_device eth"})
+	if err != nil {
+		t.FailNow()
+	}
+	if iface.kind != interfaceVLAN {
+		t.FailNow()
+	}
+}
+
+func TestParseStanzasNone(t *testing.T) {
+	stanzas, err := parseStanzas(nil)
+	if err != nil {
+		t.FailNow()
+	}
+	if len(stanzas) != 0 {
+		t.FailNow()
+	}
+}
+
+func TestParseStanzas(t *testing.T) {
+	lines := []string{
+		"auto lo",
+		"iface lo inet loopback",
+		"iface eth1 inet manual",
+		"iface eth2 inet manual",
+		"iface eth3 inet manual",
+		"auto eth1 eth3",
+	}
+	expect := []stanza{
+		&stanzaAuto{
+			interfaces: []string{"lo"},
+		},
+		&stanzaInterface{
+			name:         "lo",
+			kind:         interfacePhysical,
+			auto:         true,
+			configMethod: configMethodLoopback{},
+			options:      map[string][]string{},
+		},
+		&stanzaInterface{
+			name:         "eth1",
+			kind:         interfacePhysical,
+			auto:         true,
+			configMethod: configMethodManual{},
+			options:      map[string][]string{},
+		},
+		&stanzaInterface{
+			name:         "eth2",
+			kind:         interfacePhysical,
+			auto:         false,
+			configMethod: configMethodManual{},
+			options:      map[string][]string{},
+		},
+		&stanzaInterface{
+			name:         "eth3",
+			kind:         interfacePhysical,
+			auto:         true,
+			configMethod: configMethodManual{},
+			options:      map[string][]string{},
+		},
+		&stanzaAuto{
+			interfaces: []string{"eth1", "eth3"},
+		},
+	}
+	stanzas, err := parseStanzas(lines)
+	if err != err {
+		t.FailNow()
+	}
+	if !reflect.DeepEqual(stanzas, expect) {
+		t.FailNow()
+	}
+}

pkg/http_client.go

@@ -0,0 +1,156 @@
+package pkg
+
+import (
+	"crypto/tls"
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"net"
+	"net/http"
+	neturl "net/url"
+	"strings"
+	"time"
+)
+
+const (
+	HTTP_2xx = 2
+	HTTP_4xx = 4
+)
+
+type Err error
+
+type ErrTimeout struct {
+	Err
+}
+
+type ErrNotFound struct {
+	Err
+}
+
+type ErrInvalid struct {
+	Err
+}
+
+type ErrServer struct {
+	Err
+}
+
+type ErrNetwork struct {
+	Err
+}
+
+type HttpClient struct {
+	// Maximum exp backoff duration. Defaults to 5 seconds
+	MaxBackoff time.Duration
+
+	// Maximum number of connection retries. Defaults to 15
+	MaxRetries int
+
+	// HTTP client timeout, this is suggested to be low since exponential
+	// backoff will kick off too. Defaults to 2 seconds
+	Timeout time.Duration
+
+	// Whether or not to skip TLS verification. Defaults to false
+	SkipTLS bool
+
+	client *http.Client
+}
+
+func NewHttpClient() *HttpClient {
+	hc := &HttpClient{
+		MaxBackoff: time.Second * 5,
+		MaxRetries: 15,
+		Timeout:    time.Duration(2) * time.Second,
+		SkipTLS:    false,
+	}
+
+	// We need to create our own client in order to add timeout support.
+	// TODO(c4milo) Replace it once Go 1.3 is officially used by CoreOS
+	// More info: https://code.google.com/p/go/source/detail?r=ada6f2d5f99f
+	hc.client = &http.Client{
+		Transport: &http.Transport{
+			TLSClientConfig: &tls.Config{
+				InsecureSkipVerify: hc.SkipTLS,
+			},
+			Dial: func(network, addr string) (net.Conn, error) {
+				deadline := time.Now().Add(hc.Timeout)
+				c, err := net.DialTimeout(network, addr, hc.Timeout)
+				if err != nil {
+					return nil, err
+				}
+				c.SetDeadline(deadline)
+				return c, nil
+			},
+		},
+	}
+
+	return hc
+}
+
+func ExpBackoff(interval, max time.Duration) time.Duration {
+	interval = interval * 2
+	if interval > max {
+		interval = max
+	}
+	return interval
+}
+
+// GetRetry fetches a given URL with support for exponential backoff and maximum retries
+func (h *HttpClient) GetRetry(rawurl string) ([]byte, error) {
+	if rawurl == "" {
+		return nil, ErrInvalid{errors.New("URL is empty. Skipping.")}
+	}
+
+	url, err := neturl.Parse(rawurl)
+	if err != nil {
+		return nil, ErrInvalid{err}
+	}
+
+	// Unfortunately, url.Parse is too generic to throw errors if a URL does not
+	// have a valid HTTP scheme. So, we have to do this extra validation
+	if !strings.HasPrefix(url.Scheme, "http") {
+		return nil, ErrInvalid{fmt.Errorf("URL %s does not have a valid HTTP scheme. Skipping.", rawurl)}
+	}
+
+	dataURL := url.String()
+
+	duration := 50 * time.Millisecond
+	for retry := 1; retry <= h.MaxRetries; retry++ {
+		log.Printf("Fetching data from %s. Attempt #%d", dataURL, retry)
+
+		data, err := h.Get(dataURL)
+		switch err.(type) {
+		case ErrNetwork:
+			log.Printf(err.Error())
+		case ErrServer:
+			log.Printf(err.Error())
+		case ErrNotFound:
+			return data, err
+		default:
+			return data, err
+		}
+
+		duration = ExpBackoff(duration, h.MaxBackoff)
+		log.Printf("Sleeping for %v...", duration)
+		time.Sleep(duration)
+	}
+
+	return nil, ErrTimeout{fmt.Errorf("Unable to fetch data. Maximum retries reached: %d", h.MaxRetries)}
+}
+
+func (h *HttpClient) Get(dataURL string) ([]byte, error) {
+	if resp, err := h.client.Get(dataURL); err == nil {
+		defer resp.Body.Close()
+		switch resp.StatusCode / 100 {
+		case HTTP_2xx:
+			return ioutil.ReadAll(resp.Body)
+		case HTTP_4xx:
+			return nil, ErrNotFound{fmt.Errorf("Not found. HTTP status code: %d", resp.StatusCode)}
+		default:
+			return nil, ErrServer{fmt.Errorf("Server error. HTTP status code: %d", resp.StatusCode)}
+		}
+	} else {
+		return nil, ErrNetwork{fmt.Errorf("Unable to fetch data: %s", err.Error())}
+	}
+}

pkg/http_client_test.go

@@ -1,25 +1,42 @@
-package datasource
+package pkg
 
 import (
 	"fmt"
 	"io"
+	"math"
 	"net/http"
 	"net/http/httptest"
 	"testing"
+	"time"
 )
 
-var expBackoffTests = []struct {
-	count int
-	body  string
-}{
-	{0, "number of attempts: 0"},
-	{1, "number of attempts: 1"},
-	{2, "number of attempts: 2"},
+func TestExpBackoff(t *testing.T) {
+	duration := time.Millisecond
+	max := time.Hour
+	for i := 0; i < math.MaxUint16; i++ {
+		duration = ExpBackoff(duration, max)
+		if duration < 0 {
+			t.Fatalf("duration too small: %v %v", duration, i)
+		}
+		if duration > max {
+			t.Fatalf("duration too large: %v %v", duration, i)
+		}
+	}
 }
 
 // Test exponential backoff and that it continues retrying if a 5xx response is
 // received
-func TestFetchURLExpBackOff(t *testing.T) {
+func TestGetURLExpBackOff(t *testing.T) {
+	var expBackoffTests = []struct {
+		count int
+		body  string
+	}{
+		{0, "number of attempts: 0"},
+		{1, "number of attempts: 1"},
+		{2, "number of attempts: 2"},
+	}
+	client := NewHttpClient()
+
 	for i, tt := range expBackoffTests {
 		mux := http.NewServeMux()
 		count := 0
@@ -34,7 +51,7 @@ func TestFetchURLExpBackOff(t *testing.T) {
 		ts := httptest.NewServer(mux)
 		defer ts.Close()
 
-		data, err := fetchURL(ts.URL)
+		data, err := client.GetRetry(ts.URL)
 		if err != nil {
 			t.Errorf("Test case %d produced error: %v", i, err)
 		}
@@ -50,7 +67,8 @@ func TestFetchURLExpBackOff(t *testing.T) {
 }
 
 // Test that it stops retrying if a 4xx response comes back
-func TestFetchURL4xx(t *testing.T) {
+func TestGetURL4xx(t *testing.T) {
+	client := NewHttpClient()
 	retries := 0
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		retries++
@@ -58,9 +76,9 @@ func TestFetchURL4xx(t *testing.T) {
 	}))
 	defer ts.Close()
 
-	_, err := fetchURL(ts.URL)
+	_, err := client.GetRetry(ts.URL)
 	if err == nil {
-		t.Errorf("Incorrect result\ngot:  %s\nwant: %s", err.Error(), "user-data not found. HTTP status code: 404")
+		t.Errorf("Incorrect result\ngot:  %s\nwant: %s", err.Error(), "Not found. HTTP status code: 404")
 	}
 
 	if retries > 1 {
@@ -69,7 +87,7 @@ func TestFetchURL4xx(t *testing.T) {
 }
 
 // Test that it fetches and returns user-data just fine
-func TestFetchURL2xx(t *testing.T) {
+func TestGetURL2xx(t *testing.T) {
 	var cloudcfg = `
 #cloud-config
 coreos: 
@@ -83,12 +101,13 @@ coreos:
 		reboot-strategy: best-effort
 `
 
+	client := NewHttpClient()
 	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		fmt.Fprint(w, cloudcfg)
 	}))
 	defer ts.Close()
 
-	data, err := fetchURL(ts.URL)
+	data, err := client.GetRetry(ts.URL)
 	if err != nil {
 		t.Errorf("Incorrect result\ngot:  %v\nwant: %v", err, nil)
 	}
@@ -99,19 +118,21 @@ coreos:
 }
 
 // Test attempt to fetching using malformed URL
-func TestFetchURLMalformed(t *testing.T) {
+func TestGetMalformedURL(t *testing.T) {
+	client := NewHttpClient()
+
 	var tests = []struct {
 		url  string
 		want string
 	}{
-		{"boo", "user-data URL boo does not have a valid HTTP scheme. Skipping."},
-		{"mailto://boo", "user-data URL mailto://boo does not have a valid HTTP scheme. Skipping."},
-		{"ftp://boo", "user-data URL ftp://boo does not have a valid HTTP scheme. Skipping."},
-		{"", "user-data URL is empty. Skipping."},
+		{"boo", "URL boo does not have a valid HTTP scheme. Skipping."},
+		{"mailto://boo", "URL mailto://boo does not have a valid HTTP scheme. Skipping."},
+		{"ftp://boo", "URL ftp://boo does not have a valid HTTP scheme. Skipping."},
+		{"", "URL is empty. Skipping."},
 	}
 
 	for _, test := range tests {
-		_, err := fetchURL(test.url)
+		_, err := client.GetRetry(test.url)
 		if err == nil || err.Error() != test.want {
 			t.Errorf("Incorrect result\ngot:  %v\nwant: %v", err, test.want)
 		}

src/github.com/coreos/coreos-cloudinit

@@ -1 +0,0 @@
-../../../

system/env_file.go

@@ -0,0 +1,89 @@
+package system
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
+	"regexp"
+)
+
+type EnvFile struct {
+	Vars map[string]string
+	// mask File.Content, it shouldn't be used.
+	Content interface{} `json:"-" yaml:"-"`
+	*File
+}
+
+// only allow sh compatible identifiers
+var validKey = regexp.MustCompile(`^[a-zA-Z0-9_]+$`)
+
+// match each line, optionally capturing valid identifiers, discarding dos line endings
+var lineLexer = regexp.MustCompile(`(?m)^((?:([a-zA-Z0-9_]+)=)?.*?)\r?\n`)
+
+// mergeEnvContents: Update the existing file contents with new values,
+// preserving variable ordering and all content this code doesn't understand.
+// All new values are appended to the bottom of the old.
+func mergeEnvContents(old []byte, pending map[string]string) []byte {
+	var buf bytes.Buffer
+	var match [][]byte
+
+	// it is awkward for the regex to handle a missing newline gracefully
+	if len(old) != 0 && !bytes.HasSuffix(old, []byte{'\n'}) {
+		old = append(old, byte('\n'))
+	}
+
+	for _, match = range lineLexer.FindAllSubmatch(old, -1) {
+		key := string(match[2])
+		if value, ok := pending[key]; ok {
+			fmt.Fprintf(&buf, "%s=%s\n", key, value)
+			delete(pending, key)
+		} else {
+			fmt.Fprintf(&buf, "%s\n", match[1])
+		}
+	}
+
+	for key, value := range pending {
+		fmt.Fprintf(&buf, "%s=%s\n", key, value)
+	}
+
+	return buf.Bytes()
+}
+
+// WriteEnvFile updates an existing env `KEY=value` formated file with
+// new values provided in EnvFile.Vars; File.Content is ignored.
+// Existing ordering and any unknown formatting such as comments are
+// preserved. If no changes are required the file is untouched.
+func WriteEnvFile(ef *EnvFile, root string) error {
+	// validate new keys, mergeEnvContents uses pending to track writes
+	pending := make(map[string]string, len(ef.Vars))
+	for key, value := range ef.Vars {
+		if !validKey.MatchString(key) {
+			return fmt.Errorf("Invalid name %q for %s", key, ef.Path)
+		}
+		pending[key] = value
+	}
+
+	if len(pending) == 0 {
+		return nil
+	}
+
+	oldContent, err := ioutil.ReadFile(path.Join(root, ef.Path))
+	if err != nil {
+		if os.IsNotExist(err) {
+			oldContent = []byte{}
+		} else {
+			return err
+		}
+	}
+
+	newContent := mergeEnvContents(oldContent, pending)
+	if bytes.Equal(oldContent, newContent) {
+		return nil
+	}
+
+	ef.File.Content = string(newContent)
+	_, err = WriteFile(ef.File, root)
+	return err
+}

system/env_file_test.go

@@ -0,0 +1,426 @@
+package system
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"strings"
+	"syscall"
+	"testing"
+)
+
+const (
+	base          = "# a file\nFOO=base\n\nBAR= hi there\n"
+	baseNoNewline = "# a file\nFOO=base\n\nBAR= hi there"
+	baseDos       = "# a file\r\nFOO=base\r\n\r\nBAR= hi there\r\n"
+	expectUpdate  = "# a file\nFOO=test\n\nBAR= hi there\nNEW=a value\n"
+	expectCreate  = "FOO=test\nNEW=a value\n"
+)
+
+var (
+	valueUpdate = map[string]string{
+		"FOO": "test",
+		"NEW": "a value",
+	}
+	valueNoop = map[string]string{
+		"FOO": "base",
+	}
+	valueEmpty   = map[string]string{}
+	valueInvalid = map[string]string{
+		"FOO-X": "test",
+	}
+)
+
+func TestWriteEnvFileUpdate(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(base), 0644)
+
+	oldStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueUpdate,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != expectUpdate {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+
+	newStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("File was not replaced: %s", fullPath)
+	}
+}
+
+func TestWriteEnvFileUpdateNoNewline(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(baseNoNewline), 0644)
+
+	oldStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueUpdate,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != expectUpdate {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+
+	newStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("File was not replaced: %s", fullPath)
+	}
+}
+
+func TestWriteEnvFileCreate(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueUpdate,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != expectCreate {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+}
+
+func TestWriteEnvFileNoop(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(base), 0644)
+
+	oldStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueNoop,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != base {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+
+	newStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	if oldStat.Sys().(*syscall.Stat_t).Ino != newStat.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("File was replaced: %s", fullPath)
+	}
+}
+
+func TestWriteEnvFileUpdateDos(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(baseDos), 0644)
+
+	oldStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueUpdate,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != expectUpdate {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+
+	newStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("File was not replaced: %s", fullPath)
+	}
+}
+
+// A middle ground noop, values are unchanged but we did have a value.
+// Seems reasonable to rewrite the file in Unix format anyway.
+func TestWriteEnvFileDos2Unix(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(baseDos), 0644)
+
+	oldStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueNoop,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != base {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+
+	newStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("File was not replaced: %s", fullPath)
+	}
+}
+
+// If it really is a noop (structure is empty) don't even do dos2unix
+func TestWriteEnvFileEmpty(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(baseDos), 0644)
+
+	oldStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueEmpty,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	if string(contents) != baseDos {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	}
+
+	newStat, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatal("Unable to stat file: %v", err)
+	}
+
+	if oldStat.Sys().(*syscall.Stat_t).Ino != newStat.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("File was replaced: %s", fullPath)
+	}
+}
+
+// no point in creating empty files
+func TestWriteEnvFileEmptyNoCreate(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueEmpty,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err != nil {
+		t.Fatalf("WriteFile failed: %v", err)
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err == nil {
+		t.Fatalf("File has incorrect contents: %q", contents)
+	} else if !os.IsNotExist(err) {
+		t.Fatalf("Unexpected error while reading file: %v", err)
+	}
+}
+
+func TestWriteEnvFilePermFailure(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+	fullPath := path.Join(dir, name)
+	ioutil.WriteFile(fullPath, []byte(base), 0000)
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueUpdate,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if !os.IsPermission(err) {
+		t.Fatalf("Not a pemission denied error: %v", err)
+	}
+}
+
+func TestWriteEnvFileNameFailure(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	name := "foo.conf"
+
+	ef := EnvFile{
+		File: &File{
+			Path: name,
+		},
+		Vars: valueInvalid,
+	}
+
+	err = WriteEnvFile(&ef, dir)
+	if err == nil || !strings.HasPrefix(err.Error(), "Invalid name") {
+		t.Fatalf("Not an invalid name error: %v", err)
+	}
+}

system/file.go

@@ -31,33 +31,55 @@ func (f *File) Permissions() (os.FileMode, error) {
 	return os.FileMode(perm), nil
 }
 
-func WriteFile(f *File) error {
+func WriteFile(f *File, root string) (string, error) {
 	if f.Encoding != "" {
-		return fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
+		return "", fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
 	}
 
-	if err := os.MkdirAll(path.Dir(f.Path), os.FileMode(0755)); err != nil {
-		return err
+	fullpath := path.Join(root, f.Path)
+	dir := path.Dir(fullpath)
+
+	if err := EnsureDirectoryExists(dir); err != nil {
+		return "", err
 	}
 
 	perm, err := f.Permissions()
 	if err != nil {
-		return err
+		return "", err
 	}
 
-	if err := ioutil.WriteFile(f.Path, []byte(f.Content), perm); err != nil {
-		return err
+	var tmp *os.File
+	// Create a temporary file in the same directory to ensure it's on the same filesystem
+	if tmp, err = ioutil.TempFile(dir, "cloudinit-temp"); err != nil {
+		return "", err
+	}
+
+	if err := ioutil.WriteFile(tmp.Name(), []byte(f.Content), perm); err != nil {
+		return "", err
+	}
+
+	if err := tmp.Close(); err != nil {
+		return "", err
+	}
+
+	// Ensure the permissions are as requested (since WriteFile can be affected by sticky bit)
+	if err := os.Chmod(tmp.Name(), perm); err != nil {
+		return "", err
 	}
 
 	if f.Owner != "" {
 		// We shell out since we don't have a way to look up unix groups natively
-		cmd := exec.Command("chown", f.Owner, f.Path)
+		cmd := exec.Command("chown", f.Owner, tmp.Name())
 		if err := cmd.Run(); err != nil {
-			return err
+			return "", err
 		}
 	}
 
-	return nil
+	if err := os.Rename(tmp.Name(), fullpath); err != nil {
+		return "", err
+	}
+
+	return fullpath, nil
 }
 
 func EnsureDirectoryExists(dir string) error {

system/file_test.go

@@ -4,7 +4,6 @@ import (
 	"io/ioutil"
 	"os"
 	"path"
-	"syscall"
 	"testing"
 )
 
@@ -13,18 +12,22 @@ func TestWriteFileUnencodedContent(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
-	fullPath := path.Join(dir, "tmp", "foo")
+	fn := "foo"
+	fullPath := path.Join(dir, fn)
 
 	wf := File{
-		Path:        fullPath,
+		Path:               fn,
 		Content:            "bar",
 		RawFilePermissions: "0644",
 	}
 
-	if err := WriteFile(&wf); err != nil {
+	path, err := WriteFile(&wf, dir)
+	if err != nil {
 		t.Fatalf("Processing of WriteFile failed: %v", err)
+	} else if path != fullPath {
+		t.Fatalf("WriteFile returned bad path: want %s, got %s", fullPath, path)
 	}
 
 	fi, err := os.Stat(fullPath)
@@ -51,7 +54,7 @@ func TestWriteFileInvalidPermission(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
 	wf := File{
 		Path:               path.Join(dir, "tmp", "foo"),
@@ -59,7 +62,7 @@ func TestWriteFileInvalidPermission(t *testing.T) {
 		RawFilePermissions: "pants",
 	}
 
-	if err := WriteFile(&wf); err == nil {
+	if _, err := WriteFile(&wf, dir); err == nil {
 		t.Fatalf("Expected error to be raised when writing file with invalid permission")
 	}
 }
@@ -69,17 +72,21 @@ func TestWriteFilePermissions(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
-	fullPath := path.Join(dir, "tmp", "foo")
+	fn := "foo"
+	fullPath := path.Join(dir, fn)
 
 	wf := File{
-		Path:               fullPath,
+		Path:               fn,
 		RawFilePermissions: "0755",
 	}
 
-	if err := WriteFile(&wf); err != nil {
+	path, err := WriteFile(&wf, dir)
+	if err != nil {
 		t.Fatalf("Processing of WriteFile failed: %v", err)
+	} else if path != fullPath {
+		t.Fatalf("WriteFile returned bad path: want %s, got %s", fullPath, path)
 	}
 
 	fi, err := os.Stat(fullPath)
@@ -97,7 +104,7 @@ func TestWriteFileEncodedContent(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
-	defer syscall.Rmdir(dir)
+	defer os.RemoveAll(dir)
 
 	wf := File{
 		Path:     path.Join(dir, "tmp", "foo"),
@@ -105,7 +112,7 @@ func TestWriteFileEncodedContent(t *testing.T) {
 		Encoding: "base64",
 	}
 
-	if err := WriteFile(&wf); err == nil {
+	if _, err := WriteFile(&wf, dir); err == nil {
 		t.Fatalf("Expected error to be raised when writing file with encoding")
 	}
 }

system/networkd.go

@@ -0,0 +1,91 @@
+package system
+
+import (
+	"fmt"
+	"io/ioutil"
+	"net"
+	"os/exec"
+	"path"
+
+	"github.com/coreos/coreos-cloudinit/network"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/dotcloud/docker/pkg/netlink"
+)
+
+const (
+	runtimeNetworkPath = "/run/systemd/network"
+)
+
+func RestartNetwork(interfaces []network.InterfaceGenerator) (err error) {
+	defer func() {
+		if e := restartNetworkd(); e != nil {
+			err = e
+		}
+	}()
+
+	if err = downNetworkInterfaces(interfaces); err != nil {
+		return
+	}
+
+	if err = probe8012q(); err != nil {
+		return
+	}
+	return
+}
+
+func downNetworkInterfaces(interfaces []network.InterfaceGenerator) error {
+	sysInterfaceMap := make(map[string]*net.Interface)
+	if systemInterfaces, err := net.Interfaces(); err == nil {
+		for _, iface := range systemInterfaces {
+			// Need a copy of the interface so we can take the address
+			temp := iface
+			sysInterfaceMap[temp.Name] = &temp
+		}
+	} else {
+		return err
+	}
+
+	for _, iface := range interfaces {
+		if systemInterface, ok := sysInterfaceMap[iface.Name()]; ok {
+			if err := netlink.NetworkLinkDown(systemInterface); err != nil {
+				fmt.Printf("Error while downing interface %q (%s). Continuing...\n", systemInterface.Name, err)
+			}
+		}
+	}
+
+	return nil
+}
+
+func probe8012q() error {
+	return exec.Command("modprobe", "8021q").Run()
+}
+
+func restartNetworkd() error {
+	_, err := RunUnitCommand("restart", "systemd-networkd.service")
+	return err
+}
+
+func WriteNetworkdConfigs(interfaces []network.InterfaceGenerator) error {
+	for _, iface := range interfaces {
+		filename := path.Join(runtimeNetworkPath, fmt.Sprintf("%s.netdev", iface.Filename()))
+		if err := writeConfig(filename, iface.Netdev()); err != nil {
+			return err
+		}
+		filename = path.Join(runtimeNetworkPath, fmt.Sprintf("%s.link", iface.Filename()))
+		if err := writeConfig(filename, iface.Link()); err != nil {
+			return err
+		}
+		filename = path.Join(runtimeNetworkPath, fmt.Sprintf("%s.network", iface.Filename()))
+		if err := writeConfig(filename, iface.Network()); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func writeConfig(filename string, config string) error {
+	if config == "" {
+		return nil
+	}
+
+	return ioutil.WriteFile(filename, []byte(config), 0444)
+}

system/systemd.go

@@ -51,10 +51,10 @@ func (u *Unit) Group() (group string) {
 
 type Script []byte
 
-// UnitDestination builds the appropriate absolute file path for
-// the given Unit. The root argument indicates the effective base
+// Destination builds the appropriate absolute file path for
+// the Unit. The root argument indicates the effective base
 // directory of the system (similar to a chroot).
-func UnitDestination(u *Unit, root string) string {
+func (u *Unit) Destination(root string) string {
 	dir := "etc"
 	if u.Runtime {
 		dir = "run"
@@ -78,12 +78,12 @@ func PlaceUnit(u *Unit, dst string) error {
 	}
 
 	file := File{
-		Path:               dst,
+		Path:               filepath.Base(dst),
 		Content:            u.Content,
 		RawFilePermissions: "0644",
 	}
 
-	err := WriteFile(&file)
+	_, err := WriteFile(&file, dir)
 	if err != nil {
 		return err
 	}
@@ -91,14 +91,14 @@ func PlaceUnit(u *Unit, dst string) error {
 	return nil
 }
 
-func EnableUnitFile(file string, runtime bool) error {
+func EnableUnitFile(unit string, runtime bool) error {
 	conn, err := dbus.New()
 	if err != nil {
 		return err
 	}
 
-	files := []string{file}
-	_, _, err = conn.EnableUnitFiles(files, runtime, true)
+	units := []string{unit}
+	_, _, err = conn.EnableUnitFiles(units, runtime, true)
 	return err
 }
 
@@ -179,10 +179,53 @@ func MachineID(root string) string {
 	return id
 }
 
-func MaskUnit(unit string, root string) error {
-	masked := path.Join(root, "etc", "systemd", "system", unit)
+// MaskUnit masks the given Unit by symlinking its unit file to
+// /dev/null, analogous to `systemctl mask`.
+// N.B.: Unlike `systemctl mask`, this function will *remove any existing unit
+// file at the location*, to ensure that the mask will succeed.
+func MaskUnit(unit *Unit, root string) error {
+	masked := unit.Destination(root)
+	if _, err := os.Stat(masked); os.IsNotExist(err) {
 		if err := os.MkdirAll(path.Dir(masked), os.FileMode(0755)); err != nil {
 			return err
 		}
+	} else if err := os.Remove(masked); err != nil {
+		return err
+	}
 	return os.Symlink("/dev/null", masked)
 }
+
+// UnmaskUnit is analogous to systemd's unit_file_unmask. If the file
+// associated with the given Unit is empty or appears to be a symlink to
+// /dev/null, it is removed.
+func UnmaskUnit(unit *Unit, root string) error {
+	masked := unit.Destination(root)
+	ne, err := nullOrEmpty(masked)
+	if os.IsNotExist(err) {
+		return nil
+	} else if err != nil {
+		return err
+	}
+	if !ne {
+		log.Printf("%s is not null or empty, refusing to unmask", masked)
+		return nil
+	}
+	return os.Remove(masked)
+}
+
+// nullOrEmpty checks whether a given path appears to be an empty regular file
+// or a symlink to /dev/null
+func nullOrEmpty(path string) (bool, error) {
+	fi, err := os.Stat(path)
+	if err != nil {
+		return false, err
+	}
+	m := fi.Mode()
+	if m.IsRegular() && fi.Size() <= 0 {
+		return true, nil
+	}
+	if m&os.ModeCharDevice > 0 {
+		return true, nil
+	}
+	return false, nil
+}

system/systemd_test.go

@@ -25,10 +25,10 @@ Address=10.209.171.177/19
 	}
 	defer os.RemoveAll(dir)
 
-	dst := UnitDestination(&u, dir)
+	dst := u.Destination(dir)
 	expectDst := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
 	if dst != expectDst {
-		t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
+		t.Fatalf("unit.Destination returned %s, expected %s", dst, expectDst)
 	}
 
 	if err := PlaceUnit(&u, dst); err != nil {
@@ -69,18 +69,18 @@ func TestUnitDestination(t *testing.T) {
 		DropIn: false,
 	}
 
-	dst := UnitDestination(&u, dir)
+	dst := u.Destination(dir)
 	expectDst := path.Join(dir, "etc", "systemd", "system", "foobar.service")
 	if dst != expectDst {
-		t.Errorf("UnitDestination returned %s, expected %s", dst, expectDst)
+		t.Errorf("unit.Destination returned %s, expected %s", dst, expectDst)
 	}
 
 	u.DropIn = true
 
-	dst = UnitDestination(&u, dir)
+	dst = u.Destination(dir)
 	expectDst = path.Join(dir, "etc", "systemd", "system", "foobar.service.d", cloudConfigDropIn)
 	if dst != expectDst {
-		t.Errorf("UnitDestination returned %s, expected %s", dst, expectDst)
+		t.Errorf("unit.Destination returned %s, expected %s", dst, expectDst)
 	}
 }
 
@@ -100,10 +100,10 @@ Where=/media/state
 	}
 	defer os.RemoveAll(dir)
 
-	dst := UnitDestination(&u, dir)
+	dst := u.Destination(dir)
 	expectDst := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
 	if dst != expectDst {
-		t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
+		t.Fatalf("unit.Destination returned %s, expected %s", dst, expectDst)
 	}
 
 	if err := PlaceUnit(&u, dst); err != nil {
@@ -154,16 +154,126 @@ func TestMaskUnit(t *testing.T) {
 		t.Fatalf("Unable to create tempdir: %v", err)
 	}
 	defer os.RemoveAll(dir)
-	if err := MaskUnit("foo.service", dir); err != nil {
-		t.Fatalf("Unable to mask unit: %v", err)
-	}
 
-	fullPath := path.Join(dir, "etc", "systemd", "system", "foo.service")
-	target, err := os.Readlink(fullPath)
+	// Ensure mask works with units that do not currently exist
+	uf := &Unit{Name: "foo.service"}
+	if err := MaskUnit(uf, dir); err != nil {
+		t.Fatalf("Unable to mask new unit: %v", err)
+	}
+	fooPath := path.Join(dir, "etc", "systemd", "system", "foo.service")
+	fooTgt, err := os.Readlink(fooPath)
 	if err != nil {
 		t.Fatalf("Unable to read link", err)
 	}
-	if target != "/dev/null" {
-		t.Fatalf("unit not masked, got unit target", target)
+	if fooTgt != "/dev/null" {
+		t.Fatalf("unit not masked, got unit target", fooTgt)
+	}
+
+	// Ensure mask works with unit files that already exist
+	ub := &Unit{Name: "bar.service"}
+	barPath := path.Join(dir, "etc", "systemd", "system", "bar.service")
+	if _, err := os.Create(barPath); err != nil {
+		t.Fatalf("Error creating new unit file: %v", err)
+	}
+	if err := MaskUnit(ub, dir); err != nil {
+		t.Fatalf("Unable to mask existing unit: %v", err)
+	}
+	barTgt, err := os.Readlink(barPath)
+	if err != nil {
+		t.Fatalf("Unable to read link", err)
+	}
+	if barTgt != "/dev/null" {
+		t.Fatalf("unit not masked, got unit target", barTgt)
 	}
 }
+
+func TestUnmaskUnit(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	nilUnit := &Unit{Name: "null.service"}
+	if err := UnmaskUnit(nilUnit, dir); err != nil {
+		t.Errorf("unexpected error from unmasking nonexistent unit: %v", err)
+	}
+
+	uf := &Unit{Name: "foo.service", Content: "[Service]\nExecStart=/bin/true"}
+	dst := uf.Destination(dir)
+	if err := os.MkdirAll(path.Dir(dst), os.FileMode(0755)); err != nil {
+		t.Fatalf("Unable to create unit directory: %v", err)
+	}
+	if _, err := os.Create(dst); err != nil {
+		t.Fatalf("Unable to write unit file: %v", err)
+	}
+
+	if err := ioutil.WriteFile(dst, []byte(uf.Content), 700); err != nil {
+		t.Fatalf("Unable to write unit file: %v", err)
+	}
+	if err := UnmaskUnit(uf, dir); err != nil {
+		t.Errorf("unmask of non-empty unit returned unexpected error: %v", err)
+	}
+	got, _ := ioutil.ReadFile(dst)
+	if string(got) != uf.Content {
+		t.Errorf("unmask of non-empty unit mutated unit contents unexpectedly")
+	}
+
+	ub := &Unit{Name: "bar.service"}
+	dst = ub.Destination(dir)
+	if err := os.Symlink("/dev/null", dst); err != nil {
+		t.Fatalf("Unable to create masked unit: %v", err)
+	}
+	if err := UnmaskUnit(ub, dir); err != nil {
+		t.Errorf("unmask of unit returned unexpected error: %v", err)
+	}
+	if _, err := os.Stat(dst); !os.IsNotExist(err) {
+		t.Errorf("expected %s to not exist after unmask, but got err: %s", err)
+	}
+}
+
+func TestNullOrEmpty(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	non := path.Join(dir, "does_not_exist")
+	ne, err := nullOrEmpty(non)
+	if !os.IsNotExist(err) {
+		t.Errorf("nullOrEmpty on nonexistent file returned bad error: %v", err)
+	}
+	if ne {
+		t.Errorf("nullOrEmpty returned true unxpectedly")
+	}
+
+	regEmpty := path.Join(dir, "regular_empty_file")
+	_, err = os.Create(regEmpty)
+	if err != nil {
+		t.Fatalf("Unable to create tempfile: %v", err)
+	}
+	gotNe, gotErr := nullOrEmpty(regEmpty)
+	if !gotNe || gotErr != nil {
+		t.Errorf("nullOrEmpty of regular empty file returned %t, %v - want true, nil", gotNe, gotErr)
+	}
+
+	reg := path.Join(dir, "regular_file")
+	if err := ioutil.WriteFile(reg, []byte("asdf"), 700); err != nil {
+		t.Fatalf("Unable to create tempfile: %v", err)
+	}
+	gotNe, gotErr = nullOrEmpty(reg)
+	if gotNe || gotErr != nil {
+		t.Errorf("nullOrEmpty of regular file returned %t, %v - want false, nil", gotNe, gotErr)
+	}
+
+	null := path.Join(dir, "null")
+	if err := os.Symlink(os.DevNull, null); err != nil {
+		t.Fatalf("Unable to create /dev/null link: %s", err)
+	}
+	gotNe, gotErr = nullOrEmpty(null)
+	if !gotNe || gotErr != nil {
+		t.Errorf("nullOrEmpty of null symlink returned %t, %v - want true, nil", gotNe, gotErr)
+	}
+
+}

test

@@ -13,12 +13,12 @@ COVER=${COVER:-"-cover"}
 
 source ./build
 
-declare -a TESTPKGS=(initialize system datasource)
+declare -a TESTPKGS=(initialize system datasource pkg network)
 
 if [ -z "$PKG" ]; then
 	GOFMTPATH="$TESTPKGS coreos-cloudinit.go"
 	# prepend repo path to each package
-	TESTPKGS=${TESTPKGS[@]/#/${REPO_PATH}/}
+	TESTPKGS="${TESTPKGS[@]/#/${REPO_PATH}/} ./"
 else
 	GOFMTPATH="$TESTPKGS"
 	# strip out slashes and dots from PKG=./foo/

third_party/github.com/dotcloud/docker/pkg/netlink/MAINTAINERS

@@ -0,0 +1,2 @@
+Michael Crosby <michael@crosbymichael.com> (@crosbymichael)
+Guillaume J. Charmes <guillaume@docker.com> (@creack)

third_party/github.com/dotcloud/docker/pkg/netlink/netlink.go

@@ -0,0 +1,23 @@
+// Packet netlink provide access to low level Netlink sockets and messages.
+//
+// Actual implementations are in:
+// netlink_linux.go
+// netlink_darwin.go
+package netlink
+
+import (
+	"errors"
+	"net"
+)
+
+var (
+	ErrWrongSockType = errors.New("Wrong socket type")
+	ErrShortResponse = errors.New("Got short response from netlink")
+)
+
+// A Route is a subnet associated with the interface to reach it.
+type Route struct {
+	*net.IPNet
+	Iface   *net.Interface
+	Default bool
+}

third_party/github.com/dotcloud/docker/pkg/netlink/netlink_linux.go

@@ -0,0 +1,891 @@
+// +build amd64
+
+package netlink
+
+import (
+	"encoding/binary"
+	"fmt"
+	"math/rand"
+	"net"
+	"syscall"
+	"unsafe"
+)
+
+const (
+	IFNAMSIZ       = 16
+	DEFAULT_CHANGE = 0xFFFFFFFF
+	IFLA_INFO_KIND = 1
+	IFLA_INFO_DATA = 2
+	VETH_INFO_PEER = 1
+	IFLA_NET_NS_FD = 28
+	SIOC_BRADDBR   = 0x89a0
+	SIOC_BRADDIF   = 0x89a2
+)
+
+var nextSeqNr int
+
+type ifreqHwaddr struct {
+	IfrnName   [16]byte
+	IfruHwaddr syscall.RawSockaddr
+}
+
+type ifreqIndex struct {
+	IfrnName  [16]byte
+	IfruIndex int32
+}
+
+func nativeEndian() binary.ByteOrder {
+	var x uint32 = 0x01020304
+	if *(*byte)(unsafe.Pointer(&x)) == 0x01 {
+		return binary.BigEndian
+	}
+	return binary.LittleEndian
+}
+
+func getSeq() int {
+	nextSeqNr = nextSeqNr + 1
+	return nextSeqNr
+}
+
+func getIpFamily(ip net.IP) int {
+	if len(ip) <= net.IPv4len {
+		return syscall.AF_INET
+	}
+	if ip.To4() != nil {
+		return syscall.AF_INET
+	}
+	return syscall.AF_INET6
+}
+
+type NetlinkRequestData interface {
+	Len() int
+	ToWireFormat() []byte
+}
+
+type IfInfomsg struct {
+	syscall.IfInfomsg
+}
+
+func newIfInfomsg(family int) *IfInfomsg {
+	return &IfInfomsg{
+		IfInfomsg: syscall.IfInfomsg{
+			Family: uint8(family),
+		},
+	}
+}
+
+func newIfInfomsgChild(parent *RtAttr, family int) *IfInfomsg {
+	msg := newIfInfomsg(family)
+	parent.children = append(parent.children, msg)
+	return msg
+}
+
+func (msg *IfInfomsg) ToWireFormat() []byte {
+	native := nativeEndian()
+
+	length := syscall.SizeofIfInfomsg
+	b := make([]byte, length)
+	b[0] = msg.Family
+	b[1] = 0
+	native.PutUint16(b[2:4], msg.Type)
+	native.PutUint32(b[4:8], uint32(msg.Index))
+	native.PutUint32(b[8:12], msg.Flags)
+	native.PutUint32(b[12:16], msg.Change)
+	return b
+}
+
+func (msg *IfInfomsg) Len() int {
+	return syscall.SizeofIfInfomsg
+}
+
+type IfAddrmsg struct {
+	syscall.IfAddrmsg
+}
+
+func newIfAddrmsg(family int) *IfAddrmsg {
+	return &IfAddrmsg{
+		IfAddrmsg: syscall.IfAddrmsg{
+			Family: uint8(family),
+		},
+	}
+}
+
+func (msg *IfAddrmsg) ToWireFormat() []byte {
+	native := nativeEndian()
+
+	length := syscall.SizeofIfAddrmsg
+	b := make([]byte, length)
+	b[0] = msg.Family
+	b[1] = msg.Prefixlen
+	b[2] = msg.Flags
+	b[3] = msg.Scope
+	native.PutUint32(b[4:8], msg.Index)
+	return b
+}
+
+func (msg *IfAddrmsg) Len() int {
+	return syscall.SizeofIfAddrmsg
+}
+
+type RtMsg struct {
+	syscall.RtMsg
+}
+
+func newRtMsg(family int) *RtMsg {
+	return &RtMsg{
+		RtMsg: syscall.RtMsg{
+			Family:   uint8(family),
+			Table:    syscall.RT_TABLE_MAIN,
+			Scope:    syscall.RT_SCOPE_UNIVERSE,
+			Protocol: syscall.RTPROT_BOOT,
+			Type:     syscall.RTN_UNICAST,
+		},
+	}
+}
+
+func (msg *RtMsg) ToWireFormat() []byte {
+	native := nativeEndian()
+
+	length := syscall.SizeofRtMsg
+	b := make([]byte, length)
+	b[0] = msg.Family
+	b[1] = msg.Dst_len
+	b[2] = msg.Src_len
+	b[3] = msg.Tos
+	b[4] = msg.Table
+	b[5] = msg.Protocol
+	b[6] = msg.Scope
+	b[7] = msg.Type
+	native.PutUint32(b[8:12], msg.Flags)
+	return b
+}
+
+func (msg *RtMsg) Len() int {
+	return syscall.SizeofRtMsg
+}
+
+func rtaAlignOf(attrlen int) int {
+	return (attrlen + syscall.RTA_ALIGNTO - 1) & ^(syscall.RTA_ALIGNTO - 1)
+}
+
+type RtAttr struct {
+	syscall.RtAttr
+	Data     []byte
+	children []NetlinkRequestData
+}
+
+func newRtAttr(attrType int, data []byte) *RtAttr {
+	return &RtAttr{
+		RtAttr: syscall.RtAttr{
+			Type: uint16(attrType),
+		},
+		children: []NetlinkRequestData{},
+		Data:     data,
+	}
+}
+
+func newRtAttrChild(parent *RtAttr, attrType int, data []byte) *RtAttr {
+	attr := newRtAttr(attrType, data)
+	parent.children = append(parent.children, attr)
+	return attr
+}
+
+func (a *RtAttr) Len() int {
+	l := 0
+	for _, child := range a.children {
+		l += child.Len() + syscall.SizeofRtAttr
+	}
+	if l == 0 {
+		l++
+	}
+	return rtaAlignOf(l + len(a.Data))
+}
+
+func (a *RtAttr) ToWireFormat() []byte {
+	native := nativeEndian()
+
+	length := a.Len()
+	buf := make([]byte, rtaAlignOf(length+syscall.SizeofRtAttr))
+
+	if a.Data != nil {
+		copy(buf[4:], a.Data)
+	} else {
+		next := 4
+		for _, child := range a.children {
+			childBuf := child.ToWireFormat()
+			copy(buf[next:], childBuf)
+			next += rtaAlignOf(len(childBuf))
+		}
+	}
+
+	if l := uint16(rtaAlignOf(length)); l != 0 {
+		native.PutUint16(buf[0:2], l+1)
+	}
+	native.PutUint16(buf[2:4], a.Type)
+
+	return buf
+}
+
+type NetlinkRequest struct {
+	syscall.NlMsghdr
+	Data []NetlinkRequestData
+}
+
+func (rr *NetlinkRequest) ToWireFormat() []byte {
+	native := nativeEndian()
+
+	length := rr.Len
+	dataBytes := make([][]byte, len(rr.Data))
+	for i, data := range rr.Data {
+		dataBytes[i] = data.ToWireFormat()
+		length += uint32(len(dataBytes[i]))
+	}
+	b := make([]byte, length)
+	native.PutUint32(b[0:4], length)
+	native.PutUint16(b[4:6], rr.Type)
+	native.PutUint16(b[6:8], rr.Flags)
+	native.PutUint32(b[8:12], rr.Seq)
+	native.PutUint32(b[12:16], rr.Pid)
+
+	next := 16
+	for _, data := range dataBytes {
+		copy(b[next:], data)
+		next += len(data)
+	}
+	return b
+}
+
+func (rr *NetlinkRequest) AddData(data NetlinkRequestData) {
+	if data != nil {
+		rr.Data = append(rr.Data, data)
+	}
+}
+
+func newNetlinkRequest(proto, flags int) *NetlinkRequest {
+	return &NetlinkRequest{
+		NlMsghdr: syscall.NlMsghdr{
+			Len:   uint32(syscall.NLMSG_HDRLEN),
+			Type:  uint16(proto),
+			Flags: syscall.NLM_F_REQUEST | uint16(flags),
+			Seq:   uint32(getSeq()),
+		},
+	}
+}
+
+type NetlinkSocket struct {
+	fd  int
+	lsa syscall.SockaddrNetlink
+}
+
+func getNetlinkSocket() (*NetlinkSocket, error) {
+	fd, err := syscall.Socket(syscall.AF_NETLINK, syscall.SOCK_RAW, syscall.NETLINK_ROUTE)
+	if err != nil {
+		return nil, err
+	}
+	s := &NetlinkSocket{
+		fd: fd,
+	}
+	s.lsa.Family = syscall.AF_NETLINK
+	if err := syscall.Bind(fd, &s.lsa); err != nil {
+		syscall.Close(fd)
+		return nil, err
+	}
+
+	return s, nil
+}
+
+func (s *NetlinkSocket) Close() {
+	syscall.Close(s.fd)
+}
+
+func (s *NetlinkSocket) Send(request *NetlinkRequest) error {
+	if err := syscall.Sendto(s.fd, request.ToWireFormat(), 0, &s.lsa); err != nil {
+		return err
+	}
+	return nil
+}
+
+func (s *NetlinkSocket) Receive() ([]syscall.NetlinkMessage, error) {
+	rb := make([]byte, syscall.Getpagesize())
+	nr, _, err := syscall.Recvfrom(s.fd, rb, 0)
+	if err != nil {
+		return nil, err
+	}
+	if nr < syscall.NLMSG_HDRLEN {
+		return nil, ErrShortResponse
+	}
+	rb = rb[:nr]
+	return syscall.ParseNetlinkMessage(rb)
+}
+
+func (s *NetlinkSocket) GetPid() (uint32, error) {
+	lsa, err := syscall.Getsockname(s.fd)
+	if err != nil {
+		return 0, err
+	}
+	switch v := lsa.(type) {
+	case *syscall.SockaddrNetlink:
+		return v.Pid, nil
+	}
+	return 0, ErrWrongSockType
+}
+
+func (s *NetlinkSocket) HandleAck(seq uint32) error {
+	native := nativeEndian()
+
+	pid, err := s.GetPid()
+	if err != nil {
+		return err
+	}
+
+done:
+	for {
+		msgs, err := s.Receive()
+		if err != nil {
+			return err
+		}
+		for _, m := range msgs {
+			if m.Header.Seq != seq {
+				return fmt.Errorf("Wrong Seq nr %d, expected %d", m.Header.Seq, seq)
+			}
+			if m.Header.Pid != pid {
+				return fmt.Errorf("Wrong pid %d, expected %d", m.Header.Pid, pid)
+			}
+			if m.Header.Type == syscall.NLMSG_DONE {
+				break done
+			}
+			if m.Header.Type == syscall.NLMSG_ERROR {
+				error := int32(native.Uint32(m.Data[0:4]))
+				if error == 0 {
+					break done
+				}
+				return syscall.Errno(-error)
+			}
+		}
+	}
+
+	return nil
+}
+
+// Add a new default gateway. Identical to:
+// ip route add default via $ip
+func AddDefaultGw(ip net.IP) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	family := getIpFamily(ip)
+
+	wb := newNetlinkRequest(syscall.RTM_NEWROUTE, syscall.NLM_F_CREATE|syscall.NLM_F_EXCL|syscall.NLM_F_ACK)
+
+	msg := newRtMsg(family)
+	wb.AddData(msg)
+
+	var ipData []byte
+	if family == syscall.AF_INET {
+		ipData = ip.To4()
+	} else {
+		ipData = ip.To16()
+	}
+
+	gateway := newRtAttr(syscall.RTA_GATEWAY, ipData)
+
+	wb.AddData(gateway)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+// Bring up a particular network interface
+func NetworkLinkUp(iface *net.Interface) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_NEWLINK, syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	msg.Change = syscall.IFF_UP
+	msg.Flags = syscall.IFF_UP
+	msg.Index = int32(iface.Index)
+	wb.AddData(msg)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+func NetworkLinkDown(iface *net.Interface) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_NEWLINK, syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	msg.Change = syscall.IFF_UP
+	msg.Flags = 0 & ^syscall.IFF_UP
+	msg.Index = int32(iface.Index)
+	wb.AddData(msg)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+func NetworkSetMTU(iface *net.Interface, mtu int) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	msg.Type = syscall.RTM_SETLINK
+	msg.Flags = syscall.NLM_F_REQUEST
+	msg.Index = int32(iface.Index)
+	msg.Change = DEFAULT_CHANGE
+	wb.AddData(msg)
+
+	var (
+		b      = make([]byte, 4)
+		native = nativeEndian()
+	)
+	native.PutUint32(b, uint32(mtu))
+
+	data := newRtAttr(syscall.IFLA_MTU, b)
+	wb.AddData(data)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+	return s.HandleAck(wb.Seq)
+}
+
+// same as ip link set $name master $master
+func NetworkSetMaster(iface, master *net.Interface) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	msg.Type = syscall.RTM_SETLINK
+	msg.Flags = syscall.NLM_F_REQUEST
+	msg.Index = int32(iface.Index)
+	msg.Change = DEFAULT_CHANGE
+	wb.AddData(msg)
+
+	var (
+		b      = make([]byte, 4)
+		native = nativeEndian()
+	)
+	native.PutUint32(b, uint32(master.Index))
+
+	data := newRtAttr(syscall.IFLA_MASTER, b)
+	wb.AddData(data)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+func NetworkSetNsPid(iface *net.Interface, nspid int) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	msg.Type = syscall.RTM_SETLINK
+	msg.Flags = syscall.NLM_F_REQUEST
+	msg.Index = int32(iface.Index)
+	msg.Change = DEFAULT_CHANGE
+	wb.AddData(msg)
+
+	var (
+		b      = make([]byte, 4)
+		native = nativeEndian()
+	)
+	native.PutUint32(b, uint32(nspid))
+
+	data := newRtAttr(syscall.IFLA_NET_NS_PID, b)
+	wb.AddData(data)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+func NetworkSetNsFd(iface *net.Interface, fd int) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_SETLINK, syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	msg.Type = syscall.RTM_SETLINK
+	msg.Flags = syscall.NLM_F_REQUEST
+	msg.Index = int32(iface.Index)
+	msg.Change = DEFAULT_CHANGE
+	wb.AddData(msg)
+
+	var (
+		b      = make([]byte, 4)
+		native = nativeEndian()
+	)
+	native.PutUint32(b, uint32(fd))
+
+	data := newRtAttr(IFLA_NET_NS_FD, b)
+	wb.AddData(data)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+// Add an Ip address to an interface. This is identical to:
+// ip addr add $ip/$ipNet dev $iface
+func NetworkLinkAddIp(iface *net.Interface, ip net.IP, ipNet *net.IPNet) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	family := getIpFamily(ip)
+
+	wb := newNetlinkRequest(syscall.RTM_NEWADDR, syscall.NLM_F_CREATE|syscall.NLM_F_EXCL|syscall.NLM_F_ACK)
+
+	msg := newIfAddrmsg(family)
+	msg.Index = uint32(iface.Index)
+	prefixLen, _ := ipNet.Mask.Size()
+	msg.Prefixlen = uint8(prefixLen)
+	wb.AddData(msg)
+
+	var ipData []byte
+	if family == syscall.AF_INET {
+		ipData = ip.To4()
+	} else {
+		ipData = ip.To16()
+	}
+
+	localData := newRtAttr(syscall.IFA_LOCAL, ipData)
+	wb.AddData(localData)
+
+	addrData := newRtAttr(syscall.IFA_ADDRESS, ipData)
+	wb.AddData(addrData)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+func zeroTerminated(s string) []byte {
+	return []byte(s + "\000")
+}
+
+func nonZeroTerminated(s string) []byte {
+	return []byte(s)
+}
+
+// Add a new network link of a specified type. This is identical to
+// running: ip add link $name type $linkType
+func NetworkLinkAdd(name string, linkType string) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_NEWLINK, syscall.NLM_F_CREATE|syscall.NLM_F_EXCL|syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	wb.AddData(msg)
+
+	if name != "" {
+		nameData := newRtAttr(syscall.IFLA_IFNAME, zeroTerminated(name))
+		wb.AddData(nameData)
+	}
+
+	kindData := newRtAttr(IFLA_INFO_KIND, nonZeroTerminated(linkType))
+
+	infoData := newRtAttr(syscall.IFLA_LINKINFO, kindData.ToWireFormat())
+	wb.AddData(infoData)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+
+	return s.HandleAck(wb.Seq)
+}
+
+// Returns an array of IPNet for all the currently routed subnets on ipv4
+// This is similar to the first column of "ip route" output
+func NetworkGetRoutes() ([]Route, error) {
+	native := nativeEndian()
+
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return nil, err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_GETROUTE, syscall.NLM_F_DUMP)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	wb.AddData(msg)
+
+	if err := s.Send(wb); err != nil {
+		return nil, err
+	}
+
+	pid, err := s.GetPid()
+	if err != nil {
+		return nil, err
+	}
+
+	res := make([]Route, 0)
+
+done:
+	for {
+		msgs, err := s.Receive()
+		if err != nil {
+			return nil, err
+		}
+		for _, m := range msgs {
+			if m.Header.Seq != wb.Seq {
+				return nil, fmt.Errorf("Wrong Seq nr %d, expected 1", m.Header.Seq)
+			}
+			if m.Header.Pid != pid {
+				return nil, fmt.Errorf("Wrong pid %d, expected %d", m.Header.Pid, pid)
+			}
+			if m.Header.Type == syscall.NLMSG_DONE {
+				break done
+			}
+			if m.Header.Type == syscall.NLMSG_ERROR {
+				error := int32(native.Uint32(m.Data[0:4]))
+				if error == 0 {
+					break done
+				}
+				return nil, syscall.Errno(-error)
+			}
+			if m.Header.Type != syscall.RTM_NEWROUTE {
+				continue
+			}
+
+			var r Route
+
+			msg := (*RtMsg)(unsafe.Pointer(&m.Data[0:syscall.SizeofRtMsg][0]))
+
+			if msg.Flags&syscall.RTM_F_CLONED != 0 {
+				// Ignore cloned routes
+				continue
+			}
+
+			if msg.Table != syscall.RT_TABLE_MAIN {
+				// Ignore non-main tables
+				continue
+			}
+
+			if msg.Family != syscall.AF_INET {
+				// Ignore non-ipv4 routes
+				continue
+			}
+
+			if msg.Dst_len == 0 {
+				// Default routes
+				r.Default = true
+			}
+
+			attrs, err := syscall.ParseNetlinkRouteAttr(&m)
+			if err != nil {
+				return nil, err
+			}
+			for _, attr := range attrs {
+				switch attr.Attr.Type {
+				case syscall.RTA_DST:
+					ip := attr.Value
+					r.IPNet = &net.IPNet{
+						IP:   ip,
+						Mask: net.CIDRMask(int(msg.Dst_len), 8*len(ip)),
+					}
+				case syscall.RTA_OIF:
+					index := int(native.Uint32(attr.Value[0:4]))
+					r.Iface, _ = net.InterfaceByIndex(index)
+				}
+			}
+			if r.Default || r.IPNet != nil {
+				res = append(res, r)
+			}
+		}
+	}
+
+	return res, nil
+}
+
+func getIfSocket() (fd int, err error) {
+	for _, socket := range []int{
+		syscall.AF_INET,
+		syscall.AF_PACKET,
+		syscall.AF_INET6,
+	} {
+		if fd, err = syscall.Socket(socket, syscall.SOCK_DGRAM, 0); err == nil {
+			break
+		}
+	}
+	if err == nil {
+		return fd, nil
+	}
+	return -1, err
+}
+
+func NetworkChangeName(iface *net.Interface, newName string) error {
+	fd, err := getIfSocket()
+	if err != nil {
+		return err
+	}
+	defer syscall.Close(fd)
+
+	data := [IFNAMSIZ * 2]byte{}
+	// the "-1"s here are very important for ensuring we get proper null
+	// termination of our new C strings
+	copy(data[:IFNAMSIZ-1], iface.Name)
+	copy(data[IFNAMSIZ:IFNAMSIZ*2-1], newName)
+
+	if _, _, errno := syscall.Syscall(syscall.SYS_IOCTL, uintptr(fd), syscall.SIOCSIFNAME, uintptr(unsafe.Pointer(&data[0]))); errno != 0 {
+		return errno
+	}
+	return nil
+}
+
+func NetworkCreateVethPair(name1, name2 string) error {
+	s, err := getNetlinkSocket()
+	if err != nil {
+		return err
+	}
+	defer s.Close()
+
+	wb := newNetlinkRequest(syscall.RTM_NEWLINK, syscall.NLM_F_CREATE|syscall.NLM_F_EXCL|syscall.NLM_F_ACK)
+
+	msg := newIfInfomsg(syscall.AF_UNSPEC)
+	wb.AddData(msg)
+
+	nameData := newRtAttr(syscall.IFLA_IFNAME, zeroTerminated(name1))
+	wb.AddData(nameData)
+
+	nest1 := newRtAttr(syscall.IFLA_LINKINFO, nil)
+	newRtAttrChild(nest1, IFLA_INFO_KIND, zeroTerminated("veth"))
+	nest2 := newRtAttrChild(nest1, IFLA_INFO_DATA, nil)
+	nest3 := newRtAttrChild(nest2, VETH_INFO_PEER, nil)
+
+	newIfInfomsgChild(nest3, syscall.AF_UNSPEC)
+	newRtAttrChild(nest3, syscall.IFLA_IFNAME, zeroTerminated(name2))
+
+	wb.AddData(nest1)
+
+	if err := s.Send(wb); err != nil {
+		return err
+	}
+	return s.HandleAck(wb.Seq)
+}
+
+// Create the actual bridge device.  This is more backward-compatible than
+// netlink.NetworkLinkAdd and works on RHEL 6.
+func CreateBridge(name string, setMacAddr bool) error {
+	s, err := syscall.Socket(syscall.AF_INET6, syscall.SOCK_STREAM, syscall.IPPROTO_IP)
+	if err != nil {
+		// ipv6 issue, creating with ipv4
+		s, err = syscall.Socket(syscall.AF_INET, syscall.SOCK_STREAM, syscall.IPPROTO_IP)
+		if err != nil {
+			return err
+		}
+	}
+	defer syscall.Close(s)
+
+	nameBytePtr, err := syscall.BytePtrFromString(name)
+	if err != nil {
+		return err
+	}
+	if _, _, err := syscall.Syscall(syscall.SYS_IOCTL, uintptr(s), SIOC_BRADDBR, uintptr(unsafe.Pointer(nameBytePtr))); err != 0 {
+		return err
+	}
+	if setMacAddr {
+		return setBridgeMacAddress(s, name)
+	}
+	return nil
+}
+
+// Add a slave to abridge device.  This is more backward-compatible than
+// netlink.NetworkSetMaster and works on RHEL 6.
+func AddToBridge(iface, master *net.Interface) error {
+	s, err := syscall.Socket(syscall.AF_INET6, syscall.SOCK_STREAM, syscall.IPPROTO_IP)
+	if err != nil {
+		// ipv6 issue, creating with ipv4
+		s, err = syscall.Socket(syscall.AF_INET, syscall.SOCK_STREAM, syscall.IPPROTO_IP)
+		if err != nil {
+			return err
+		}
+	}
+	defer syscall.Close(s)
+
+	ifr := ifreqIndex{}
+	copy(ifr.IfrnName[:], master.Name)
+	ifr.IfruIndex = int32(iface.Index)
+
+	if _, _, err := syscall.Syscall(syscall.SYS_IOCTL, uintptr(s), SIOC_BRADDIF, uintptr(unsafe.Pointer(&ifr))); err != 0 {
+		return err
+	}
+
+	return nil
+}
+
+func setBridgeMacAddress(s int, name string) error {
+	ifr := ifreqHwaddr{}
+	ifr.IfruHwaddr.Family = syscall.ARPHRD_ETHER
+	copy(ifr.IfrnName[:], name)
+
+	for i := 0; i < 6; i++ {
+		ifr.IfruHwaddr.Data[i] = int8(rand.Intn(255))
+	}
+
+	ifr.IfruHwaddr.Data[0] &^= 0x1 // clear multicast bit
+	ifr.IfruHwaddr.Data[0] |= 0x2  // set local assignment bit (IEEE802)
+
+	if _, _, err := syscall.Syscall(syscall.SYS_IOCTL, uintptr(s), syscall.SIOCSIFHWADDR, uintptr(unsafe.Pointer(&ifr))); err != 0 {
+		return err
+	}
+	return nil
+}

third_party/github.com/dotcloud/docker/pkg/netlink/netlink_unsupported.go

@@ -0,0 +1,69 @@
+// +build !linux !amd64
+
+package netlink
+
+import (
+	"errors"
+	"net"
+)
+
+var (
+	ErrNotImplemented = errors.New("not implemented")
+)
+
+func NetworkGetRoutes() ([]Route, error) {
+	return nil, ErrNotImplemented
+}
+
+func NetworkLinkAdd(name string, linkType string) error {
+	return ErrNotImplemented
+}
+
+func NetworkLinkUp(iface *net.Interface) error {
+	return ErrNotImplemented
+}
+
+func NetworkLinkAddIp(iface *net.Interface, ip net.IP, ipNet *net.IPNet) error {
+	return ErrNotImplemented
+}
+
+func AddDefaultGw(ip net.IP) error {
+	return ErrNotImplemented
+
+}
+
+func NetworkSetMTU(iface *net.Interface, mtu int) error {
+	return ErrNotImplemented
+}
+
+func NetworkCreateVethPair(name1, name2 string) error {
+	return ErrNotImplemented
+}
+
+func NetworkChangeName(iface *net.Interface, newName string) error {
+	return ErrNotImplemented
+}
+
+func NetworkSetNsFd(iface *net.Interface, fd int) error {
+	return ErrNotImplemented
+}
+
+func NetworkSetNsPid(iface *net.Interface, nspid int) error {
+	return ErrNotImplemented
+}
+
+func NetworkSetMaster(iface, master *net.Interface) error {
+	return ErrNotImplemented
+}
+
+func NetworkLinkDown(iface *net.Interface) error {
+	return ErrNotImplemented
+}
+
+func CreateBridge(name string, setMacAddr bool) error {
+	return ErrNotImplemented
+}
+
+func AddToBridge(iface, master *net.Interface) error {
+	return ErrNotImplemented
+}

units/90-configdrive.rules

@@ -3,9 +3,9 @@
 ACTION!="add|change", GOTO="coreos_configdrive_end"
 
 # A normal config drive. Block device formatted with iso9660 or fat
-SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="iso9660|vfat", ENV{ID_FS_LABEL}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-block.service"
+SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="iso9660|vfat", ENV{ID_FS_LABEL}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="media-configdrive.mount"
 
 # Addtionally support virtfs from QEMU
-SUBSYSTEM=="virtio", DRIVER=="9pnet_virtio", ATTR{mount_tag}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-virtfs.service"
+SUBSYSTEM=="virtio", DRIVER=="9pnet_virtio", ATTR{mount_tag}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="media-configvirtfs.mount"
 
 LABEL="coreos_configdrive_end"

units/configdrive-block.service

@@ -1,11 +0,0 @@
-[Unit]
-Description=Mount config drive
-Conflicts=configdrive-virtfs.service umount.target
-ConditionPathIsMountPoint=!/media/configdrive
-# Only mount config drive block devices automatically in virtual machines
-ConditionVirtualization=vm
-
-[Service]
-Type=oneshot
-RemainAfterExit=no
-ExecStart=/bin/mount -t auto -o ro,x-mount.mkdir LABEL=config-2 /media/configdrive

units/configdrive-virtfs.service

@@ -1,14 +0,0 @@
-[Unit]
-Description=Mount config drive from virtfs
-Conflicts=configdrive-block.service umount.target
-ConditionPathIsMountPoint=!/media/configdrive
-ConditionVirtualization=vm
-
-# Support old style setup for now
-Wants=addon-run@media-configdrive.service addon-config@media-configdrive.service
-Before=addon-run@media-configdrive.service addon-config@media-configdrive.service
-
-[Service]
-Type=oneshot
-RemainAfterExit=no
-ExecStart=/bin/mount -t 9p -o trans=virtio,version=9p2000.L,x-mount.mkdir config-2 /media/configdrive

units/media-configdrive.mount

@@ -0,0 +1,13 @@
+[Unit]
+Wants=user-configdrive.service
+Before=user-configdrive.service
+# Only mount config drive block devices automatically in virtual machines
+# or any host that has it explicitly enabled and not explicitly disabled.
+ConditionVirtualization=|vm
+ConditionKernelCommandLine=|coreos.configdrive=1
+ConditionKernelCommandLine=!coreos.configdrive=0
+
+[Mount]
+What=LABEL=config-2
+Where=/media/configdrive
+Options=ro

units/media-configvirtfs.mount

@@ -0,0 +1,18 @@
+[Unit]
+Wants=user-configvirtfs.service
+Before=user-configvirtfs.service
+# Only mount config drive block devices automatically in virtual machines
+# or any host that has it explicitly enabled and not explicitly disabled.
+ConditionVirtualization=|vm
+ConditionKernelCommandLine=|coreos.configdrive=1
+ConditionKernelCommandLine=!coreos.configdrive=0
+
+# Support old style setup for now
+Wants=addon-run@media-configvirtfs.service addon-config@media-configvirtfs.service
+Before=addon-run@media-configvirtfs.service addon-config@media-configvirtfs.service
+
+[Mount]
+What=config-2
+Where=/media/configvirtfs
+Options=ro,trans=virtio,version=9p2000.L
+Type=9p

units/user-config.target

@@ -4,8 +4,8 @@ Requires=system-config.target
 After=system-config.target
 
 # Watch for configs at a couple common paths
-Requires=user-cloudinit@media-configdrive-openstack-latest-user_data.path
-After=user-cloudinit@media-configdrive-openstack-latest-user_data.path
+Requires=user-configdrive.path
+After=user-configdrive.path
 Requires=user-cloudinit@var-lib-coreos\x2dinstall-user_data.path
 After=user-cloudinit@var-lib-coreos\x2dinstall-user_data.path
 

units/user-configdrive.path

@@ -0,0 +1,10 @@
+[Unit]
+Description=Watch for a cloud-config at /media/configdrive
+
+# Note: This unit is essentially just here as a fall-back mechanism to
+# trigger cloudinit if it isn't triggered explicitly by other means
+# such as by a Wants= in the mount unit. This ensures we handle the
+# case where /media/configdrive is provided to a CoreOS container.
+
+[Path]
+DirectoryNotEmpty=/media/configdrive

units/user-configdrive.service

@@ -1,9 +1,8 @@
 [Unit]
-Description=Load cloud-config from %f
+Description=Load cloud-config from /media/configdrive
 Requires=coreos-setup-environment.service
-After=coreos-setup-environment.service
+After=coreos-setup-environment.service system-config.target
 Before=user-config.target
-ConditionFileNotEmpty=%f
 
 # HACK: work around ordering between config drive and ec2 metadata It is
 # possible for OpenStack style systems to provide both the metadata service
@@ -14,11 +13,10 @@ ConditionFileNotEmpty=%f
 # systemd knows about the ordering as early as possible.
 # coreos-cloudinit could implement a simple lock but that cannot be used
 # until after the systemd dbus calls are made non-blocking.
-After=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service
 After=ec2-cloudinit.service
 
 [Service]
 Type=oneshot
 RemainAfterExit=yes
 EnvironmentFile=-/etc/environment
-ExecStart=/usr/bin/coreos-cloudinit --from-file=%f
+ExecStart=/usr/bin/coreos-cloudinit --from-configdrive=/media/configdrive

units/user-configvirtfs.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=Load cloud-config from /media/configvirtfs
+Requires=coreos-setup-environment.service
+After=coreos-setup-environment.service
+Before=user-config.target
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+EnvironmentFile=-/etc/environment
+ExecStart=/usr/bin/coreos-cloudinit --from-configdrive=/media/configvirtfs