coreos-cloudinit: bump to v1.4.1

docs: fix typo with etcd2 env vars

Documentation/cloud-config.md

@@ -39,7 +39,7 @@ CoreOS tries to conform to each platform's native method to provide user data. E
 
 ### coreos
 
-#### etcd
+#### etcd (deprecated. see etcd2)
 
 The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
 If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
@@ -49,15 +49,15 @@ If the platform environment supports the templating feature of coreos-cloudinit
 
 coreos:
   etcd:
-      name: node001
-      # generate a new token for each unique cluster from https://discovery.etcd.io/new
-      discovery: https://discovery.etcd.io/<token>
-      # multi-region and multi-cloud deployments need to use $public_ipv4
-      addr: $public_ipv4:4001
-      peer-addr: $private_ipv4:7001
+    name: node001
+    # generate a new token for each unique cluster from https://discovery.etcd.io/new
+    discovery: https://discovery.etcd.io/<token>
+    # multi-region and multi-cloud deployments need to use $public_ipv4
+    addr: $public_ipv4:4001
+    peer-addr: $private_ipv4:7001
 ```
 
-...will generate a systemd unit drop-in like this:
+...will generate a systemd unit drop-in for etcd.service with the following contents:
 
 ```yaml
 [Service]
@@ -71,11 +71,51 @@ For more information about the available configuration parameters, see the [etcd
 
 _Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
 
-[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
+[etcd-config]: https://github.com/coreos/etcd/blob/9fa3bea5a22265151f0d5063ce38a79c5b5d0271/Documentation/configuration.md
+
+#### etcd2
+
+The `coreos.etcd2.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
+If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. When generating a [discovery token](https://discovery.etcd.io/new?size=3), set the `size` parameter, since etcd uses this to determine if all members have joined the cluster. After the cluster is bootstrapped, it can grow or shrink from this configured size.
+
+For example, the following cloud-config document...
+
+```yaml
+#cloud-config
+
+coreos:
+  etcd2:
+    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
+    discovery: https://discovery.etcd.io/<token>
+    # multi-region and multi-cloud deployments need to use $public_ipv4
+    advertise-client-urls: http://$public_ipv4:2379
+    initial-advertise-peer-urls: http://$private_ipv4:2380
+    # listen on both the official ports and the legacy ports
+    # legacy ports can be omitted if your application doesn't depend on them
+    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
+    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
+```
+
+...will generate a systemd unit drop-in for etcd2.service with the following contents:
+
+```yaml
+[Service]
+Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
+Environment="ETCD_ADVERTISE_CLIENT_URLS=http://203.0.113.29:2379"
+Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=http://192.0.2.13:2380"
+Environment="ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001"
+Environment="ETCD_LISTEN_PEER_URLS=http://192.0.2.13:2380,http://192.0.2.13:7001"
+```
+
+For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
+
+_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
+
+[etcd-config]: https://github.com/coreos/etcd/blob/86e616c6e974828fc9119c1eb0f6439577a9ce0b/Documentation/configuration.md
 
 #### fleet
 
-The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
+The `coreos.fleet.*` parameters work very similarly to `coreos.etcd2.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
 
 ```yaml
 #cloud-config
@@ -100,7 +140,7 @@ For more information on fleet configuration, see the [fleet documentation][fleet
 
 #### flannel
 
-The `coreos.flannel.*` parameters also work very similarly to `coreos.etcd.*`
+The `coreos.flannel.*` parameters also work very similarly to `coreos.etcd2.*`
 and `coreos.fleet.*`. They can be used to set environment variables for
 flanneld. For example, the following cloud-config...
 
@@ -109,7 +149,7 @@ flanneld. For example, the following cloud-config...
 
 coreos:
   flannel:
-      etcd-prefix: /coreos.com/network2
+      etcd_prefix: /coreos.com/network2
 ```
 
 ...will generate a systemd unit drop-in like so:
@@ -119,7 +159,16 @@ coreos:
 Environment="FLANNELD_ETCD_PREFIX=/coreos.com/network2"
 ```
 
-For the complete list of flannel configuraion parameters, see the [flannel documentation][flannel-readme].
+List of flannel configuration parameters:
+
+- **etcd_endpoints**: Comma separated list of etcd endpoints
+- **etcd_cafile**: Path to CA file used for TLS communication with etcd
+- **etcd_certfile**: Path to certificate file used for TLS communication with etcd
+- **etcd_keyfile**: Path to private key file used for TLS communication with etcd
+- **etcd_prefix**: etcd prefix path to be used for flannel keys
+- **ip_masq**: Install IP masquerade rules for traffic outside of flannel subnet
+- **subnet_file**: Path to flannel subnet file to write out
+- **interface**: Interface (name or IP) that should be used for inter-host communication
 
 [flannel-readme]: https://github.com/coreos/flannel/blob/master/README.md
 
@@ -143,7 +192,7 @@ coreos:
 Environment="LOCKSMITHD_ENDPOINT=example.com:4001"
 ```
 
-For the complete list of locksmith configuraion parameters, see the [locksmith documentation][locksmith-readme].
+For the complete list of locksmith configuration parameters, see the [locksmith documentation][locksmith-readme].
 
 [locksmith-readme]: https://github.com/coreos/locksmith/blob/master/README.md
 
@@ -159,9 +208,12 @@ The `reboot-strategy` parameter also affects the behaviour of [locksmith](https:
   - _etcd-lock_: Reboot after first taking a distributed lock in etcd, this guarantees that only one host will reboot concurrently and that the cluster will remain available during the update.
   - _best-effort_ - If etcd is running, "etcd-lock", otherwise simply "reboot".
   - _off_ - Disable rebooting after updates are applied (not recommended).
-- **server**: is the omaha endpoint URL which will be queried for updates.
+- **server**: The location of the [CoreUpdate][coreupdate] server which will be queried for updates. Also known as the [omaha][omaha-docs] server endpoint.
 - **group**:  signifies the channel which should be used for automatic updates.  This value defaults to the version of the image initially downloaded. (one of "master", "alpha", "beta", "stable")
 
+[coreupdate]: https://coreos.com/products/coreupdate
+[omaha-docs]: https://coreos.com/docs/coreupdate/custom-apps/coreupdate-protocol/
+
 *Note: cloudinit will only manipulate the locksmith unit file in the systemd runtime directory (`/run/systemd/system/locksmithd.service`). If any manual modifications are made to an overriding unit configuration file (e.g. `/etc/systemd/system/locksmithd.service`), cloudinit will no longer be able to control the locksmith service unit.*
 
 ##### Example
@@ -230,14 +282,14 @@ coreos:
             Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
 ```
 
-Start the built-in `etcd` and `fleet` services:
+Start the built-in `etcd2` and `fleet` services:
 
 ```yaml
 #cloud-config
 
 coreos:
   units:
-    - name: etcd.service
+    - name: etcd2.service
       command: start
     - name: fleet.service
       command: start
@@ -282,10 +334,12 @@ All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the use
 - **groups**: Add user to these additional groups
 - **no-user-group**: Boolean. Skip default group creation.
 - **ssh-authorized-keys**: List of public SSH keys to authorize for this user
-- **coreos-ssh-import-github**: Authorize SSH keys from Github user
+- **coreos-ssh-import-github**: Authorize SSH keys from GitHub user
+- **coreos-ssh-import-github-users**: Authorize SSH keys from a list of GitHub users
 - **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint.
 - **system**: Create the user as a system user. No home directory will be created.
 - **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
+- **shell**: User's login shell.
 
 The following fields are not yet implemented:
 

Godeps/Godeps.json

@@ -1,6 +1,6 @@
 {
 	"ImportPath": "github.com/coreos/coreos-cloudinit",
-	"GoVersion": "go1.3.1",
+	"GoVersion": "go1.3.3",
 	"Packages": [
 		"./..."
 	],
@@ -13,6 +13,10 @@
 			"ImportPath": "github.com/coreos/go-systemd/dbus",
 			"Rev": "4fbc5060a317b142e6c7bfbedb65596d5f0ab99b"
 		},
+		{
+			"ImportPath": "github.com/coreos/yaml",
+			"Rev": "6b16a5714269b2f70720a45406b1babd947a17ef"
+		},
 		{
 			"ImportPath": "github.com/dotcloud/docker/pkg/netlink",
 			"Comment": "v0.11.1-359-g55d41c3e21e1",
@@ -25,10 +29,6 @@
 		{
 			"ImportPath": "github.com/tarm/goserial",
 			"Rev": "cdabc8d44e8e84f58f18074ae44337e1f2f375b9"
-		},
-		{
-			"ImportPath": "gopkg.in/yaml.v1",
-			"Rev": "9f9df34309c04878acc86042b16630b0f696e1de"
 		}
 	]
 }

Godeps/_workspace/src/github.com/coreos/yaml/README.md

@@ -1,3 +1,6 @@
+Note: This is a fork of https://github.com/go-yaml/yaml. The following README
+doesn't necessarily apply to this fork.
+
 # YAML support for the Go language
 
 Introduction

Godeps/_workspace/src/github.com/coreos/yaml/decode.go

@@ -30,13 +30,15 @@ type node struct {
 // Parser, produces a node tree out of a libyaml event stream.
 
 type parser struct {
-	parser yaml_parser_t
-	event  yaml_event_t
-	doc    *node
+	parser    yaml_parser_t
+	event     yaml_event_t
+	doc       *node
+	transform transformString
 }
 
-func newParser(b []byte) *parser {
-	p := parser{}
+func newParser(b []byte, t transformString) *parser {
+	p := parser{transform: t}
+
 	if !yaml_parser_initialize(&p.parser) {
 		panic("Failed to initialize YAML emitter")
 	}
@@ -175,7 +177,10 @@ func (p *parser) mapping() *node {
 	p.anchor(n, p.event.anchor)
 	p.skip()
 	for p.event.typ != yaml_MAPPING_END_EVENT {
-		n.children = append(n.children, p.parse(), p.parse())
+		key := p.parse()
+		key.value = p.transform(key.value)
+		value := p.parse()
+		n.children = append(n.children, key, value)
 	}
 	p.skip()
 	return n

Godeps/_workspace/src/github.com/coreos/yaml/decode_test.go

@@ -1,8 +1,8 @@
 package yaml_test
 
 import (
+	"github.com/coreos/yaml"
 	. "gopkg.in/check.v1"
-	"gopkg.in/yaml.v1"
 	"math"
 	"reflect"
 	"strings"
@@ -557,6 +557,23 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
 	c.Assert(m["ghi"].value, Equals, 3)
 }
 
+func (s *S) TestUnmarshalWithTransform(c *C) {
+	data := `{a_b: 1, c-d: 2, e-f_g: 3, h_i-j: 4}`
+	expect := map[string]int{
+		"a_b":   1,
+		"c_d":   2,
+		"e_f_g": 3,
+		"h_i_j": 4,
+	}
+	m := map[string]int{}
+	yaml.UnmarshalMappingKeyTransform = func(i string) string {
+		return strings.Replace(i, "-", "_", -1)
+	}
+	err := yaml.Unmarshal([]byte(data), m)
+	c.Assert(err, IsNil)
+	c.Assert(m, DeepEquals, expect)
+}
+
 // From http://yaml.org/type/merge.html
 var mergeTests = `
 anchors:

Godeps/_workspace/src/github.com/coreos/yaml/encode_test.go

@@ -7,8 +7,8 @@ import (
 	"strings"
 	"time"
 
+	"github.com/coreos/yaml"
 	. "gopkg.in/check.v1"
-	"gopkg.in/yaml.v1"
 )
 
 var marshalIntTest = 123

Godeps/_workspace/src/github.com/coreos/yaml/yaml.go

@@ -84,7 +84,7 @@ type Getter interface {
 func Unmarshal(in []byte, out interface{}) (err error) {
 	defer handleErr(&err)
 	d := newDecoder()
-	p := newParser(in)
+	p := newParser(in, UnmarshalMappingKeyTransform)
 	defer p.destroy()
 	node := p.parse()
 	if node != nil {
@@ -146,6 +146,17 @@ func Marshal(in interface{}) (out []byte, err error) {
 	return
 }
 
+// UnmarshalMappingKeyTransform is a string transformation that is applied to
+// each mapping key in a YAML document before it is unmarshalled. By default,
+// UnmarshalMappingKeyTransform is an identity transform (no modification).
+var UnmarshalMappingKeyTransform transformString = identityTransform
+
+type transformString func(in string) (out string)
+
+func identityTransform(in string) (out string) {
+	return in
+}
+
 // --------------------------------------------------------------------------
 // Maintain a mapping of keys to structure field indexes
 

config/config.go

@@ -1,27 +1,26 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
 import (
 	"fmt"
 	"reflect"
+	"regexp"
 	"strings"
 
-	"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/gopkg.in/yaml.v1"
+	"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/coreos/yaml"
 )
 
 // CloudConfig encapsulates the entire cloud-config configuration file and maps
@@ -29,21 +28,22 @@ import (
 // used for internal use) have the YAML tag '-' so that they aren't marshalled.
 type CloudConfig struct {
 	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
-	Coreos            struct {
-		Etcd      Etcd      `yaml:"etcd"`
-		Flannel   Flannel   `yaml:"flannel"`
-		Fleet     Fleet     `yaml:"fleet"`
-		Locksmith Locksmith `yaml:"locksmith"`
-		OEM       OEM       `yaml:"oem"`
-		Update    Update    `yaml:"update"`
-		Units     []Unit    `yaml:"units"`
-	} `yaml:"coreos"`
+	CoreOS            CoreOS   `yaml:"coreos"`
 	WriteFiles        []File   `yaml:"write_files"`
 	Hostname          string   `yaml:"hostname"`
 	Users             []User   `yaml:"users"`
 	ManageEtcHosts    EtcHosts `yaml:"manage_etc_hosts"`
-	NetworkConfigPath string   `yaml:"-"`
-	NetworkConfig     string   `yaml:"-"`
+}
+
+type CoreOS struct {
+	Etcd      Etcd      `yaml:"etcd"`
+	Etcd2     Etcd2     `yaml:"etcd2"`
+	Flannel   Flannel   `yaml:"flannel"`
+	Fleet     Fleet     `yaml:"fleet"`
+	Locksmith Locksmith `yaml:"locksmith"`
+	OEM       OEM       `yaml:"oem"`
+	Update    Update    `yaml:"update"`
+	Units     []Unit    `yaml:"units"`
 }
 
 func IsCloudConfig(userdata string) bool {
@@ -61,15 +61,12 @@ func IsCloudConfig(userdata string) bool {
 // string of YAML), returning any error encountered. It will ignore unknown
 // fields but log encountering them.
 func NewCloudConfig(contents string) (*CloudConfig, error) {
+	yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
+		return strings.Replace(nameIn, "-", "_", -1)
+	}
 	var cfg CloudConfig
-	ncontents, err := normalizeConfig(contents)
-	if err != nil {
-		return &cfg, err
-	}
-	if err = yaml.Unmarshal(ncontents, &cfg); err != nil {
-		return &cfg, err
-	}
-	return &cfg, nil
+	err := yaml.Unmarshal([]byte(contents), &cfg)
+	return &cfg, err
 }
 
 func (cc CloudConfig) String() string {
@@ -92,7 +89,7 @@ func IsZero(c interface{}) bool {
 
 type ErrorValid struct {
 	Value string
-	Valid []string
+	Valid string
 	Field string
 }
 
@@ -126,16 +123,15 @@ func AssertValid(value reflect.Value, valid string) *ErrorValid {
 	if valid == "" || isZero(value) {
 		return nil
 	}
+
 	vs := fmt.Sprintf("%v", value.Interface())
-	valids := strings.Split(valid, ",")
-	for _, valid := range valids {
-		if vs == valid {
-			return nil
-		}
+	if m, _ := regexp.MatchString(valid, vs); m {
+		return nil
 	}
+
 	return &ErrorValid{
 		Value: vs,
-		Valid: valids,
+		Valid: valid,
 	}
 }
 
@@ -157,31 +153,3 @@ func isZero(v reflect.Value) bool {
 func isFieldExported(f reflect.StructField) bool {
 	return f.PkgPath == ""
 }
-
-func normalizeConfig(config string) ([]byte, error) {
-	var cfg map[interface{}]interface{}
-	if err := yaml.Unmarshal([]byte(config), &cfg); err != nil {
-		return nil, err
-	}
-	return yaml.Marshal(normalizeKeys(cfg))
-}
-
-func normalizeKeys(m map[interface{}]interface{}) map[interface{}]interface{} {
-	for k, v := range m {
-		if m, ok := m[k].(map[interface{}]interface{}); ok {
-			normalizeKeys(m)
-		}
-
-		if s, ok := m[k].([]interface{}); ok {
-			for _, e := range s {
-				if m, ok := e.(map[interface{}]interface{}); ok {
-					normalizeKeys(m)
-				}
-			}
-		}
-
-		delete(m, k)
-		m[strings.Replace(fmt.Sprint(k), "-", "_", -1)] = v
-	}
-	return m
-}

config/config_test.go

@@ -1,30 +1,82 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
 import (
 	"reflect"
+	"regexp"
 	"strings"
 	"testing"
 )
 
+func TestNewCloudConfig(t *testing.T) {
+	tests := []struct {
+		contents string
+
+		config CloudConfig
+	}{
+		{},
+		{
+			contents: "#cloud-config\nwrite_files:\n  - path: underscore",
+			config:   CloudConfig{WriteFiles: []File{File{Path: "underscore"}}},
+		},
+		{
+			contents: "#cloud-config\nwrite-files:\n  - path: hyphen",
+			config:   CloudConfig{WriteFiles: []File{File{Path: "hyphen"}}},
+		},
+		{
+			contents: "#cloud-config\ncoreos:\n  update:\n    reboot-strategy: off",
+			config:   CloudConfig{CoreOS: CoreOS{Update: Update{RebootStrategy: "off"}}},
+		},
+		{
+			contents: "#cloud-config\ncoreos:\n  update:\n    reboot-strategy: false",
+			config:   CloudConfig{CoreOS: CoreOS{Update: Update{RebootStrategy: "false"}}},
+		},
+		{
+			contents: "#cloud-config\nwrite_files:\n  - permissions: 0744",
+			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "0744"}}},
+		},
+		{
+			contents: "#cloud-config\nwrite_files:\n  - permissions: 744",
+			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "744"}}},
+		},
+		{
+			contents: "#cloud-config\nwrite_files:\n  - permissions: '0744'",
+			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "0744"}}},
+		},
+		{
+			contents: "#cloud-config\nwrite_files:\n  - permissions: '744'",
+			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "744"}}},
+		},
+	}
+
+	for i, tt := range tests {
+		config, err := NewCloudConfig(tt.contents)
+		if err != nil {
+			t.Errorf("bad error (test case #%d): want %v, got %s", i, nil, err)
+		}
+		if !reflect.DeepEqual(&tt.config, config) {
+			t.Errorf("bad config (test case #%d): want %#v, got %#v", i, tt.config, config)
+		}
+	}
+}
+
 func TestIsZero(t *testing.T) {
-	for _, tt := range []struct {
-		c     interface{}
+	tests := []struct {
+		c interface{}
+
 		empty bool
 	}{
 		{struct{}{}, true},
@@ -34,7 +86,9 @@ func TestIsZero(t *testing.T) {
 		{struct{ A string }{A: "hello"}, false},
 		{struct{ A int }{}, true},
 		{struct{ A int }{A: 1}, false},
-	} {
+	}
+
+	for _, tt := range tests {
 		if empty := IsZero(tt.c); tt.empty != empty {
 			t.Errorf("bad result (%q): want %t, got %t", tt.c, tt.empty, empty)
 		}
@@ -42,66 +96,68 @@ func TestIsZero(t *testing.T) {
 }
 
 func TestAssertStructValid(t *testing.T) {
-	for _, tt := range []struct {
-		c   interface{}
+	tests := []struct {
+		c interface{}
+
 		err error
 	}{
 		{struct{}{}, nil},
 		{struct {
-			A, b string `valid:"1,2"`
+			A, b string `valid:"^1|2$"`
 		}{}, nil},
 		{struct {
-			A, b string `valid:"1,2"`
+			A, b string `valid:"^1|2$"`
 		}{A: "1", b: "2"}, nil},
 		{struct {
-			A, b string `valid:"1,2"`
+			A, b string `valid:"^1|2$"`
 		}{A: "1", b: "hello"}, nil},
 		{struct {
-			A, b string `valid:"1,2"`
-		}{A: "hello", b: "2"}, &ErrorValid{Value: "hello", Field: "A", Valid: []string{"1", "2"}}},
+			A, b string `valid:"^1|2$"`
+		}{A: "hello", b: "2"}, &ErrorValid{Value: "hello", Field: "A", Valid: "^1|2$"}},
 		{struct {
-			A, b int `valid:"1,2"`
+			A, b int `valid:"^1|2$"`
 		}{}, nil},
 		{struct {
-			A, b int `valid:"1,2"`
+			A, b int `valid:"^1|2$"`
 		}{A: 1, b: 2}, nil},
 		{struct {
-			A, b int `valid:"1,2"`
+			A, b int `valid:"^1|2$"`
 		}{A: 1, b: 9}, nil},
 		{struct {
-			A, b int `valid:"1,2"`
-		}{A: 9, b: 2}, &ErrorValid{Value: "9", Field: "A", Valid: []string{"1", "2"}}},
-	} {
+			A, b int `valid:"^1|2$"`
+		}{A: 9, b: 2}, &ErrorValid{Value: "9", Field: "A", Valid: "^1|2$"}},
+	}
+
+	for _, tt := range tests {
 		if err := AssertStructValid(tt.c); !reflect.DeepEqual(tt.err, err) {
 			t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.err, err)
 		}
 	}
 }
 
-func TestCloudConfigInvalidKeys(t *testing.T) {
-	defer func() {
-		if r := recover(); r != nil {
-			t.Fatalf("panic while instantiating CloudConfig with nil keys: %v", r)
-		}
-	}()
+func TestConfigCompile(t *testing.T) {
+	tests := []interface{}{
+		Etcd{},
+		File{},
+		Flannel{},
+		Fleet{},
+		Locksmith{},
+		OEM{},
+		Unit{},
+		Update{},
+	}
 
-	for _, tt := range []struct {
-		contents string
-	}{
-		{"coreos:"},
-		{"ssh_authorized_keys:"},
-		{"ssh_authorized_keys:\n  -"},
-		{"ssh_authorized_keys:\n  - 0:"},
-		{"write_files:"},
-		{"write_files:\n  -"},
-		{"write_files:\n  - 0:"},
-		{"users:"},
-		{"users:\n  -"},
-		{"users:\n  - 0:"},
-	} {
-		_, err := NewCloudConfig(tt.contents)
-		if err != nil {
-			t.Fatalf("error instantiating CloudConfig with invalid keys: %v", err)
+	for _, tt := range tests {
+		ttt := reflect.TypeOf(tt)
+		for i := 0; i < ttt.NumField(); i++ {
+			ft := ttt.Field(i)
+			if !isFieldExported(ft) {
+				continue
+			}
+
+			if _, err := regexp.Compile(ft.Tag.Get("valid")); err != nil {
+				t.Errorf("bad regexp(%s.%s): want %v, got %s", ttt.Name(), ft.Name, nil, err)
+			}
 		}
 	}
 }
@@ -136,7 +192,7 @@ hostname:
 	if cfg.Hostname != "foo" {
 		t.Fatalf("hostname not correctly set when invalid keys are present")
 	}
-	if cfg.Coreos.Etcd.Discovery != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
+	if cfg.CoreOS.Etcd.Discovery != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
 		t.Fatalf("etcd section not correctly set when invalid keys are present")
 	}
 	if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
@@ -242,10 +298,10 @@ hostname: trontastic
 		}
 	}
 
-	if len(cfg.Coreos.Units) != 1 {
+	if len(cfg.CoreOS.Units) != 1 {
 		t.Error("Failed to parse correct number of units")
 	} else {
-		u := cfg.Coreos.Units[0]
+		u := cfg.CoreOS.Units[0]
 		expect := `[Match]
 Name=eth47
 
@@ -263,50 +319,16 @@ Address=10.209.171.177/19
 		}
 	}
 
-	if cfg.Coreos.OEM.ID != "rackspace" {
-		t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
+	if cfg.CoreOS.OEM.ID != "rackspace" {
+		t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.CoreOS.OEM.ID)
 	}
 
 	if cfg.Hostname != "trontastic" {
 		t.Errorf("Failed to parse hostname")
 	}
-	if cfg.Coreos.Update.RebootStrategy != "reboot" {
+	if cfg.CoreOS.Update.RebootStrategy != "reboot" {
 		t.Errorf("Failed to parse locksmith strategy")
 	}
-
-	contents = `
-coreos:
-write_files:
-  - path: /home/me/notes
-    permissions: 0744
-`
-	cfg, err = NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error :%v", err)
-	}
-
-	if len(cfg.WriteFiles) != 1 {
-		t.Error("Failed to parse correct number of write_files")
-	} else {
-		wf := cfg.WriteFiles[0]
-		if wf.Content != "" {
-			t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
-		}
-		if wf.Encoding != "" {
-			t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
-		}
-		// Verify that the normalization of the config converted 0744 to its decimal
-		// representation, 484.
-		if wf.RawFilePermissions != "484" {
-			t.Errorf("WriteFile has incorrect permissions %s", wf.RawFilePermissions)
-		}
-		if wf.Path != "/home/me/notes" {
-			t.Errorf("WriteFile has incorrect path %s", wf.Path)
-		}
-		if wf.Owner != "" {
-			t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
-		}
-	}
 }
 
 // Assert that our interface conversion doesn't panic
@@ -352,6 +374,7 @@ users:
     no_user_group: true
     system: y
     no_log_init: True
+    shell: /bin/sh
 `
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
@@ -419,6 +442,10 @@ users:
 	if !user.NoLogInit {
 		t.Errorf("Failed to parse no_log_init field")
 	}
+
+	if user.Shell != "/bin/sh" {
+		t.Errorf("Failed to parse shell field, got %q", user.Shell)
+	}
 }
 
 func TestCloudConfigUsersGithubUser(t *testing.T) {
@@ -473,31 +500,3 @@ users:
 		t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
 	}
 }
-
-func TestNormalizeKeys(t *testing.T) {
-	for _, tt := range []struct {
-		in  string
-		out string
-	}{
-		{"my_key_name: the-value\n", "my_key_name: the-value\n"},
-		{"my-key_name: the-value\n", "my_key_name: the-value\n"},
-		{"my-key-name: the-value\n", "my_key_name: the-value\n"},
-
-		{"a:\n- key_name: the-value\n", "a:\n- key_name: the-value\n"},
-		{"a:\n- key-name: the-value\n", "a:\n- key_name: the-value\n"},
-
-		{"a:\n  b:\n  - key_name: the-value\n", "a:\n  b:\n  - key_name: the-value\n"},
-		{"a:\n  b:\n  - key-name: the-value\n", "a:\n  b:\n  - key_name: the-value\n"},
-
-		{"coreos:\n  update:\n    reboot-strategy: off\n", "coreos:\n  update:\n    reboot_strategy: false\n"},
-		{"coreos:\n  update:\n    reboot-strategy: 'off'\n", "coreos:\n  update:\n    reboot_strategy: \"off\"\n"},
-	} {
-		out, err := normalizeConfig(tt.in)
-		if err != nil {
-			t.Fatalf("bad error (%q): want nil, got %s", tt.in, err)
-		}
-		if string(out) != tt.out {
-			t.Fatalf("bad normalization (%q): want %q, got %q", tt.in, tt.out, out)
-		}
-	}
-}

config/decode.go

@@ -0,0 +1,56 @@
+package config
+
+import (
+	"bytes"
+	"compress/gzip"
+	"encoding/base64"
+	"fmt"
+)
+
+func DecodeBase64Content(content string) ([]byte, error) {
+	output, err := base64.StdEncoding.DecodeString(content)
+
+	if err != nil {
+		return nil, fmt.Errorf("Unable to decode base64: %q", err)
+	}
+
+	return output, nil
+}
+
+func DecodeGzipContent(content string) ([]byte, error) {
+	gzr, err := gzip.NewReader(bytes.NewReader([]byte(content)))
+
+	if err != nil {
+		return nil, fmt.Errorf("Unable to decode gzip: %q", err)
+	}
+	defer gzr.Close()
+
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(gzr)
+
+	return buf.Bytes(), nil
+}
+
+func DecodeContent(content string, encoding string) ([]byte, error) {
+	switch encoding {
+	case "":
+		return []byte(content), nil
+
+	case "b64", "base64":
+		return DecodeBase64Content(content)
+
+	case "gz", "gzip":
+		return DecodeGzipContent(content)
+
+	case "gz+base64", "gzip+base64", "gz+b64", "gzip+b64":
+		gz, err := DecodeBase64Content(content)
+
+		if err != nil {
+			return nil, err
+		}
+
+		return DecodeGzipContent(string(gz))
+	}
+
+	return nil, fmt.Errorf("Unsupported encoding %q", encoding)
+}

config/etc_hosts.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 

config/etcd.go

@@ -1,53 +1,67 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
 type Etcd struct {
-	Addr                  string  `yaml:"addr"                    env:"ETCD_ADDR"`
-	BindAddr              string  `yaml:"bind_addr"               env:"ETCD_BIND_ADDR"`
-	CAFile                string  `yaml:"ca_file"                 env:"ETCD_CA_FILE"`
-	CertFile              string  `yaml:"cert_file"               env:"ETCD_CERT_FILE"`
-	ClusterActiveSize     int     `yaml:"cluster_active_size"     env:"ETCD_CLUSTER_ACTIVE_SIZE"`
-	ClusterRemoveDelay    float64 `yaml:"cluster_remove_delay"    env:"ETCD_CLUSTER_REMOVE_DELAY"`
-	ClusterSyncInterval   float64 `yaml:"cluster_sync_interval"   env:"ETCD_CLUSTER_SYNC_INTERVAL"`
-	CorsOrigins           string  `yaml:"cors"                    env:"ETCD_CORS"`
-	DataDir               string  `yaml:"data_dir"                env:"ETCD_DATA_DIR"`
-	Discovery             string  `yaml:"discovery"               env:"ETCD_DISCOVERY"`
-	GraphiteHost          string  `yaml:"graphite_host"           env:"ETCD_GRAPHITE_HOST"`
-	HTTPReadTimeout       float64 `yaml:"http_read_timeout"       env:"ETCD_HTTP_READ_TIMEOUT"`
-	HTTPWriteTimeout      float64 `yaml:"http_write_timeout"      env:"ETCD_HTTP_WRITE_TIMEOUT"`
-	KeyFile               string  `yaml:"key_file"                env:"ETCD_KEY_FILE"`
-	MaxResultBuffer       int     `yaml:"max_result_buffer"       env:"ETCD_MAX_RESULT_BUFFER"`
-	MaxRetryAttempts      int     `yaml:"max_retry_attempts"      env:"ETCD_MAX_RETRY_ATTEMPTS"`
-	Name                  string  `yaml:"name"                    env:"ETCD_NAME"`
-	PeerAddr              string  `yaml:"peer_addr"               env:"ETCD_PEER_ADDR"`
-	PeerBindAddr          string  `yaml:"peer_bind_addr"          env:"ETCD_PEER_BIND_ADDR"`
-	PeerCAFile            string  `yaml:"peer_ca_file"            env:"ETCD_PEER_CA_FILE"`
-	PeerCertFile          string  `yaml:"peer_cert_file"          env:"ETCD_PEER_CERT_FILE"`
-	PeerElectionTimeout   int     `yaml:"peer_election_timeout"   env:"ETCD_PEER_ELECTION_TIMEOUT"`
-	PeerHeartbeatInterval int     `yaml:"peer_heartbeat_interval" env:"ETCD_PEER_HEARTBEAT_INTERVAL"`
-	PeerKeyFile           string  `yaml:"peer_key_file"           env:"ETCD_PEER_KEY_FILE"`
-	Peers                 string  `yaml:"peers"                   env:"ETCD_PEERS"`
-	PeersFile             string  `yaml:"peers_file"              env:"ETCD_PEERS_FILE"`
-	RetryInterval         float64 `yaml:"retry_interval"          env:"ETCD_RETRY_INTERVAL"`
-	Snapshot              bool    `yaml:"snapshot"                env:"ETCD_SNAPSHOT"`
-	SnapshotCount         int     `yaml:"snapshot_count"          env:"ETCD_SNAPSHOTCOUNT"`
-	StrTrace              string  `yaml:"trace"                   env:"ETCD_TRACE"`
-	Verbose               bool    `yaml:"verbose"                 env:"ETCD_VERBOSE"`
-	VeryVerbose           bool    `yaml:"very_verbose"            env:"ETCD_VERY_VERBOSE"`
-	VeryVeryVerbose       bool    `yaml:"very_very_verbose"       env:"ETCD_VERY_VERY_VERBOSE"`
+	Addr                     string  `yaml:"addr"                          env:"ETCD_ADDR"`
+	AdvertiseClientURLs      string  `yaml:"advertise_client_urls"         env:"ETCD_ADVERTISE_CLIENT_URLS"       deprecated:"etcd2 options no longer work for etcd"`
+	BindAddr                 string  `yaml:"bind_addr"                     env:"ETCD_BIND_ADDR"`
+	CAFile                   string  `yaml:"ca_file"                       env:"ETCD_CA_FILE"`
+	CertFile                 string  `yaml:"cert_file"                     env:"ETCD_CERT_FILE"`
+	ClusterActiveSize        int     `yaml:"cluster_active_size"           env:"ETCD_CLUSTER_ACTIVE_SIZE"`
+	ClusterRemoveDelay       float64 `yaml:"cluster_remove_delay"          env:"ETCD_CLUSTER_REMOVE_DELAY"`
+	ClusterSyncInterval      float64 `yaml:"cluster_sync_interval"         env:"ETCD_CLUSTER_SYNC_INTERVAL"`
+	CorsOrigins              string  `yaml:"cors"                          env:"ETCD_CORS"`
+	DataDir                  string  `yaml:"data_dir"                      env:"ETCD_DATA_DIR"`
+	Discovery                string  `yaml:"discovery"                     env:"ETCD_DISCOVERY"`
+	DiscoveryFallback        string  `yaml:"discovery_fallback"            env:"ETCD_DISCOVERY_FALLBACK"          deprecated:"etcd2 options no longer work for etcd"`
+	DiscoverySRV             string  `yaml:"discovery_srv"                 env:"ETCD_DISCOVERY_SRV"               deprecated:"etcd2 options no longer work for etcd"`
+	DiscoveryProxy           string  `yaml:"discovery_proxy"               env:"ETCD_DISCOVERY_PROXY"             deprecated:"etcd2 options no longer work for etcd"`
+	ElectionTimeout          int     `yaml:"election_timeout"              env:"ETCD_ELECTION_TIMEOUT"            deprecated:"etcd2 options no longer work for etcd"`
+	ForceNewCluster          bool    `yaml:"force_new_cluster"             env:"ETCD_FORCE_NEW_CLUSTER"           deprecated:"etcd2 options no longer work for etcd"`
+	GraphiteHost             string  `yaml:"graphite_host"                 env:"ETCD_GRAPHITE_HOST"`
+	HeartbeatInterval        int     `yaml:"heartbeat_interval"            env:"ETCD_HEARTBEAT_INTERVAL"          deprecated:"etcd2 options no longer work for etcd"`
+	HTTPReadTimeout          float64 `yaml:"http_read_timeout"             env:"ETCD_HTTP_READ_TIMEOUT"`
+	HTTPWriteTimeout         float64 `yaml:"http_write_timeout"            env:"ETCD_HTTP_WRITE_TIMEOUT"`
+	InitialAdvertisePeerURLs string  `yaml:"initial_advertise_peer_urls"   env:"ETCD_INITIAL_ADVERTISE_PEER_URLS" deprecated:"etcd2 options no longer work for etcd"`
+	InitialCluster           string  `yaml:"initial_cluster"               env:"ETCD_INITIAL_CLUSTER"             deprecated:"etcd2 options no longer work for etcd"`
+	InitialClusterState      string  `yaml:"initial_cluster_state"         env:"ETCD_INITIAL_CLUSTER_STATE"       deprecated:"etcd2 options no longer work for etcd"`
+	InitialClusterToken      string  `yaml:"initial_cluster_token"         env:"ETCD_INITIAL_CLUSTER_TOKEN"       deprecated:"etcd2 options no longer work for etcd"`
+	KeyFile                  string  `yaml:"key_file"                      env:"ETCD_KEY_FILE"`
+	ListenClientURLs         string  `yaml:"listen_client_urls"            env:"ETCD_LISTEN_CLIENT_URLS"          deprecated:"etcd2 options no longer work for etcd"`
+	ListenPeerURLs           string  `yaml:"listen_peer_urls"              env:"ETCD_LISTEN_PEER_URLS"            deprecated:"etcd2 options no longer work for etcd"`
+	MaxResultBuffer          int     `yaml:"max_result_buffer"             env:"ETCD_MAX_RESULT_BUFFER"`
+	MaxRetryAttempts         int     `yaml:"max_retry_attempts"            env:"ETCD_MAX_RETRY_ATTEMPTS"`
+	MaxSnapshots             int     `yaml:"max_snapshots"                 env:"ETCD_MAX_SNAPSHOTS"               deprecated:"etcd2 options no longer work for etcd"`
+	MaxWALs                  int     `yaml:"max_wals"                      env:"ETCD_MAX_WALS"                    deprecated:"etcd2 options no longer work for etcd"`
+	Name                     string  `yaml:"name"                          env:"ETCD_NAME"`
+	PeerAddr                 string  `yaml:"peer_addr"                     env:"ETCD_PEER_ADDR"`
+	PeerBindAddr             string  `yaml:"peer_bind_addr"                env:"ETCD_PEER_BIND_ADDR"`
+	PeerCAFile               string  `yaml:"peer_ca_file"                  env:"ETCD_PEER_CA_FILE"`
+	PeerCertFile             string  `yaml:"peer_cert_file"                env:"ETCD_PEER_CERT_FILE"`
+	PeerElectionTimeout      int     `yaml:"peer_election_timeout"         env:"ETCD_PEER_ELECTION_TIMEOUT"`
+	PeerHeartbeatInterval    int     `yaml:"peer_heartbeat_interval"       env:"ETCD_PEER_HEARTBEAT_INTERVAL"`
+	PeerKeyFile              string  `yaml:"peer_key_file"                 env:"ETCD_PEER_KEY_FILE"`
+	Peers                    string  `yaml:"peers"                         env:"ETCD_PEERS"`
+	PeersFile                string  `yaml:"peers_file"                    env:"ETCD_PEERS_FILE"`
+	Proxy                    string  `yaml:"proxy"                         env:"ETCD_PROXY"                       deprecated:"etcd2 options no longer work for etcd"`
+	RetryInterval            float64 `yaml:"retry_interval"                env:"ETCD_RETRY_INTERVAL"`
+	Snapshot                 bool    `yaml:"snapshot"                      env:"ETCD_SNAPSHOT"`
+	SnapshotCount            int     `yaml:"snapshot_count"                env:"ETCD_SNAPSHOTCOUNT"`
+	StrTrace                 string  `yaml:"trace"                         env:"ETCD_TRACE"`
+	Verbose                  bool    `yaml:"verbose"                       env:"ETCD_VERBOSE"`
+	VeryVerbose              bool    `yaml:"very_verbose"                  env:"ETCD_VERY_VERBOSE"`
+	VeryVeryVerbose          bool    `yaml:"very_very_verbose"             env:"ETCD_VERY_VERY_VERBOSE"`
 }

config/etcd2.go

@@ -0,0 +1,44 @@
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package config
+
+type Etcd2 struct {
+	AdvertiseClientURLs      string `yaml:"advertise_client_urls"         env:"ETCD_ADVERTISE_CLIENT_URLS"`
+	CAFile                   string `yaml:"ca_file"                       env:"ETCD_CA_FILE"`
+	CertFile                 string `yaml:"cert_file"                     env:"ETCD_CERT_FILE"`
+	CorsOrigins              string `yaml:"cors"                          env:"ETCD_CORS"`
+	DataDir                  string `yaml:"data_dir"                      env:"ETCD_DATA_DIR"`
+	Discovery                string `yaml:"discovery"                     env:"ETCD_DISCOVERY"`
+	DiscoveryFallback        string `yaml:"discovery_fallback"            env:"ETCD_DISCOVERY_FALLBACK"`
+	DiscoverySRV             string `yaml:"discovery_srv"                 env:"ETCD_DISCOVERY_SRV"`
+	DiscoveryProxy           string `yaml:"discovery_proxy"               env:"ETCD_DISCOVERY_PROXY"`
+	ElectionTimeout          int    `yaml:"election_timeout"              env:"ETCD_ELECTION_TIMEOUT"`
+	HeartbeatInterval        int    `yaml:"heartbeat_interval"            env:"ETCD_HEARTBEAT_INTERVAL"`
+	InitialAdvertisePeerURLs string `yaml:"initial_advertise_peer_urls"   env:"ETCD_INITIAL_ADVERTISE_PEER_URLS"`
+	InitialCluster           string `yaml:"initial_cluster"               env:"ETCD_INITIAL_CLUSTER"`
+	InitialClusterState      string `yaml:"initial_cluster_state"         env:"ETCD_INITIAL_CLUSTER_STATE"`
+	InitialClusterToken      string `yaml:"initial_cluster_token"         env:"ETCD_INITIAL_CLUSTER_TOKEN"`
+	KeyFile                  string `yaml:"key_file"                      env:"ETCD_KEY_FILE"`
+	ListenClientURLs         string `yaml:"listen_client_urls"            env:"ETCD_LISTEN_CLIENT_URLS"`
+	ListenPeerURLs           string `yaml:"listen_peer_urls"              env:"ETCD_LISTEN_PEER_URLS"`
+	MaxSnapshots             int    `yaml:"max_snapshots"                 env:"ETCD_MAX_SNAPSHOTS"`
+	MaxWALs                  int    `yaml:"max_wals"                      env:"ETCD_MAX_WALS"`
+	Name                     string `yaml:"name"                          env:"ETCD_NAME"`
+	PeerCAFile               string `yaml:"peer_ca_file"                  env:"ETCD_PEER_CA_FILE"`
+	PeerCertFile             string `yaml:"peer_cert_file"                env:"ETCD_PEER_CERT_FILE"`
+	PeerKeyFile              string `yaml:"peer_key_file"                 env:"ETCD_PEER_KEY_FILE"`
+	Proxy                    string `yaml:"proxy"                         env:"ETCD_PROXY"`
+	SnapshotCount            int    `yaml:"snapshot_count"                env:"ETCD_SNAPSHOTCOUNT"`
+}

config/file.go

@@ -1,25 +1,23 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
 type File struct {
-	Encoding           string `yaml:"encoding" valid:"base64,b64,gz,gzip,gz+base64,gzip+base64,gz+b64,gzip+b64"`
+	Encoding           string `yaml:"encoding" valid:"^(base64|b64|gz|gzip|gz\\+base64|gzip\\+base64|gz\\+b64|gzip\\+b64)$"`
 	Content            string `yaml:"content"`
 	Owner              string `yaml:"owner"`
 	Path               string `yaml:"path"`
-	RawFilePermissions string `yaml:"permissions"`
+	RawFilePermissions string `yaml:"permissions" valid:"^0?[0-7]{3,4}$"`
 }

config/file_test.go

@@ -0,0 +1,71 @@
+/*
+   Copyright 2014 CoreOS, Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+*/
+
+package config
+
+import (
+	"testing"
+)
+
+func TestEncodingValid(t *testing.T) {
+	tests := []struct {
+		value string
+
+		isValid bool
+	}{
+		{value: "base64", isValid: true},
+		{value: "b64", isValid: true},
+		{value: "gz", isValid: true},
+		{value: "gzip", isValid: true},
+		{value: "gz+base64", isValid: true},
+		{value: "gzip+base64", isValid: true},
+		{value: "gz+b64", isValid: true},
+		{value: "gzip+b64", isValid: true},
+		{value: "gzzzzbase64", isValid: false},
+		{value: "gzipppbase64", isValid: false},
+		{value: "unknown", isValid: false},
+	}
+
+	for _, tt := range tests {
+		isValid := (nil == AssertStructValid(File{Encoding: tt.value}))
+		if tt.isValid != isValid {
+			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
+		}
+	}
+}
+
+func TestRawFilePermissionsValid(t *testing.T) {
+	tests := []struct {
+		value string
+
+		isValid bool
+	}{
+		{value: "744", isValid: true},
+		{value: "0744", isValid: true},
+		{value: "1744", isValid: true},
+		{value: "01744", isValid: true},
+		{value: "11744", isValid: false},
+		{value: "rwxr--r--", isValid: false},
+		{value: "800", isValid: false},
+	}
+
+	for _, tt := range tests {
+		isValid := (nil == AssertStructValid(File{RawFilePermissions: tt.value}))
+		if tt.isValid != isValid {
+			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
+		}
+	}
+}

config/flannel.go

@@ -1,9 +1,26 @@
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package config
 
 type Flannel struct {
-	EtcdEndpoint string `yaml:"etcd_endpoint" env:"FLANNELD_ETCD_ENDPOINT"`
-	EtcdPrefix   string `yaml:"etcd_prefix"   env:"FLANNELD_ETCD_PREFIX"`
-	IPMasq       string `yaml:"ip_masq"       env:"FLANNELD_IP_MASQ"`
-	SubnetFile   string `yaml:"subnet_file"   env:"FLANNELD_SUBNET_FILE"`
-	Iface        string `yaml:"interface"     env:"FLANNELD_IFACE"`
+	EtcdEndpoints string `yaml:"etcd_endpoints" env:"FLANNELD_ETCD_ENDPOINTS"`
+	EtcdCAFile    string `yaml:"etcd_cafile"    env:"FLANNELD_ETCD_CAFILE"`
+	EtcdCertFile  string `yaml:"etcd_certfile"  env:"FLANNELD_ETCD_CERTFILE"`
+	EtcdKeyFile   string `yaml:"etcd_keyfile"   env:"FLANNELD_ETCD_KEYFILE"`
+	EtcdPrefix    string `yaml:"etcd_prefix"    env:"FLANNELD_ETCD_PREFIX"`
+	IPMasq        string `yaml:"ip_masq"        env:"FLANNELD_IP_MASQ"`
+	SubnetFile    string `yaml:"subnet_file"    env:"FLANNELD_SUBNET_FILE"`
+	Iface         string `yaml:"interface"      env:"FLANNELD_IFACE"`
 }

config/fleet.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 

config/locksmith.go

@@ -1,3 +1,17 @@
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package config
 
 type Locksmith struct {

config/oem.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 

config/script.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
@@ -27,6 +25,7 @@ func IsScript(userdata string) bool {
 	return strings.HasPrefix(header, "#!")
 }
 
-func NewScript(userdata string) (Script, error) {
-	return Script(userdata), nil
+func NewScript(userdata string) (*Script, error) {
+	s := Script(userdata)
+	return &s, nil
 }

config/unit.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
@@ -22,7 +20,7 @@ type Unit struct {
 	Enable  bool         `yaml:"enable"`
 	Runtime bool         `yaml:"runtime"`
 	Content string       `yaml:"content"`
-	Command string       `yaml:"command" valid:"start,stop,restart,reload,try-restart,reload-or-restart,reload-or-try-restart"`
+	Command string       `yaml:"command" valid:"^(start|stop|restart|reload|try-restart|reload-or-restart|reload-or-try-restart)$"`
 	DropIns []UnitDropIn `yaml:"drop_ins"`
 }
 

config/unit_test.go

@@ -0,0 +1,46 @@
+/*
+   Copyright 2014 CoreOS, Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+*/
+
+package config
+
+import (
+	"testing"
+)
+
+func TestCommandValid(t *testing.T) {
+	tests := []struct {
+		value string
+
+		isValid bool
+	}{
+		{value: "start", isValid: true},
+		{value: "stop", isValid: true},
+		{value: "restart", isValid: true},
+		{value: "reload", isValid: true},
+		{value: "try-restart", isValid: true},
+		{value: "reload-or-restart", isValid: true},
+		{value: "reload-or-try-restart", isValid: true},
+		{value: "tryrestart", isValid: false},
+		{value: "unknown", isValid: false},
+	}
+
+	for _, tt := range tests {
+		isValid := (nil == AssertStructValid(Unit{Command: tt.value}))
+		if tt.isValid != isValid {
+			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
+		}
+	}
+}

config/update.go

@@ -1,23 +1,21 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
 type Update struct {
-	RebootStrategy string `yaml:"reboot_strategy" env:"REBOOT_STRATEGY" valid:"best-effort,etcd-lock,reboot,off,false"`
+	RebootStrategy string `yaml:"reboot_strategy" env:"REBOOT_STRATEGY" valid:"^(best-effort|etcd-lock|reboot|off)$"`
 	Group          string `yaml:"group"           env:"GROUP"`
 	Server         string `yaml:"server"          env:"SERVER"`
 }

config/update_test.go

@@ -0,0 +1,43 @@
+/*
+   Copyright 2014 CoreOS, Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+*/
+
+package config
+
+import (
+	"testing"
+)
+
+func TestRebootStrategyValid(t *testing.T) {
+	tests := []struct {
+		value string
+
+		isValid bool
+	}{
+		{value: "best-effort", isValid: true},
+		{value: "etcd-lock", isValid: true},
+		{value: "reboot", isValid: true},
+		{value: "off", isValid: true},
+		{value: "besteffort", isValid: false},
+		{value: "unknown", isValid: false},
+	}
+
+	for _, tt := range tests {
+		isValid := (nil == AssertStructValid(Update{RebootStrategy: tt.value}))
+		if tt.isValid != isValid {
+			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
+		}
+	}
+}

config/user.go

@@ -1,33 +1,33 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package config
 
 type User struct {
-	Name                string   `yaml:"name"`
-	PasswordHash        string   `yaml:"passwd"`
-	SSHAuthorizedKeys   []string `yaml:"ssh_authorized_keys"`
-	SSHImportGithubUser string   `yaml:"coreos_ssh_import_github"`
-	SSHImportURL        string   `yaml:"coreos_ssh_import_url"`
-	GECOS               string   `yaml:"gecos"`
-	Homedir             string   `yaml:"homedir"`
-	NoCreateHome        bool     `yaml:"no_create_home"`
-	PrimaryGroup        string   `yaml:"primary_group"`
-	Groups              []string `yaml:"groups"`
-	NoUserGroup         bool     `yaml:"no_user_group"`
-	System              bool     `yaml:"system"`
-	NoLogInit           bool     `yaml:"no_log_init"`
+	Name                 string   `yaml:"name"`
+	PasswordHash         string   `yaml:"passwd"`
+	SSHAuthorizedKeys    []string `yaml:"ssh_authorized_keys"`
+	SSHImportGithubUser  string   `yaml:"coreos_ssh_import_github"       deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
+	SSHImportGithubUsers []string `yaml:"coreos_ssh_import_github_users" deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
+	SSHImportURL         string   `yaml:"coreos_ssh_import_url"          deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
+	GECOS                string   `yaml:"gecos"`
+	Homedir              string   `yaml:"homedir"`
+	NoCreateHome         bool     `yaml:"no_create_home"`
+	PrimaryGroup         string   `yaml:"primary_group"`
+	Groups               []string `yaml:"groups"`
+	NoUserGroup          bool     `yaml:"no_user_group"`
+	System               bool     `yaml:"system"`
+	NoLogInit            bool     `yaml:"no_log_init"`
+	Shell                string   `yaml:"shell"`
 }

config/validate/context.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 

config/validate/context_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 

config/validate/node.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 

config/validate/node_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 

config/validate/report.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 

config/validate/report_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 

config/validate/rules.go

@@ -1,24 +1,25 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 
 import (
 	"fmt"
+	"net/url"
+	"path"
 	"reflect"
+	"strings"
 
 	"github.com/coreos/coreos-cloudinit/config"
 )
@@ -27,8 +28,40 @@ type rule func(config node, report *Report)
 
 // Rules contains all of the validation rules.
 var Rules []rule = []rule{
+	checkDiscoveryUrl,
+	checkEncoding,
 	checkStructure,
 	checkValidity,
+	checkWriteFiles,
+	checkWriteFilesUnderCoreos,
+}
+
+// checkDiscoveryUrl verifies that the string is a valid url.
+func checkDiscoveryUrl(cfg node, report *Report) {
+	c := cfg.Child("coreos").Child("etcd").Child("discovery")
+	if !c.IsValid() {
+		return
+	}
+
+	if _, err := url.ParseRequestURI(c.String()); err != nil {
+		report.Warning(c.line, "discovery URL is not valid")
+	}
+}
+
+// checkEncoding validates that, for each file under 'write_files', the
+// content can be decoded given the specified encoding.
+func checkEncoding(cfg node, report *Report) {
+	for _, f := range cfg.Child("write_files").children {
+		e := f.Child("encoding")
+		if !e.IsValid() {
+			continue
+		}
+
+		c := f.Child("content")
+		if _, err := config.DecodeContent(c.String(), e.String()); err != nil {
+			report.Error(c.line, fmt.Sprintf("content cannot be decoded as %q", e.String()))
+		}
+	}
 }
 
 // checkStructure compares the provided config to the empty config.CloudConfig
@@ -49,6 +82,9 @@ func checkNodeStructure(n, g node, r *Report) {
 	case reflect.Struct:
 		for _, cn := range n.children {
 			if cg := g.Child(cn.name); cg.IsValid() {
+				if msg := cg.field.Tag.Get("deprecated"); msg != "" {
+					r.Warning(cn.line, fmt.Sprintf("deprecated key %q (%s)", cn.name, msg))
+				}
 				checkNodeStructure(cn, cg, r)
 			} else {
 				r.Warning(cn.line, fmt.Sprintf("unrecognized key %q", cn.name))
@@ -67,6 +103,24 @@ func checkNodeStructure(n, g node, r *Report) {
 	}
 }
 
+// isCompatible determines if the type of kind n can be converted to the type
+// of kind g in the context of YAML. This is not an exhaustive list, but its
+// enough for the purposes of cloud-config validation.
+func isCompatible(n, g reflect.Kind) bool {
+	switch g {
+	case reflect.String:
+		return n == reflect.String || n == reflect.Int || n == reflect.Float64 || n == reflect.Bool
+	case reflect.Struct:
+		return n == reflect.Struct || n == reflect.Map
+	case reflect.Float64:
+		return n == reflect.Float64 || n == reflect.Int
+	case reflect.Bool, reflect.Slice, reflect.Int:
+		return n == g
+	default:
+		panic(fmt.Sprintf("isCompatible(): unhandled kind %s", g))
+	}
+}
+
 // checkValidity checks the value of every node in the provided config by
 // running config.AssertValid() on it.
 func checkValidity(cfg node, report *Report) {
@@ -76,7 +130,7 @@ func checkValidity(cfg node, report *Report) {
 
 func checkNodeValidity(n, g node, r *Report) {
 	if err := config.AssertValid(n.Value, g.field.Tag.Get("valid")); err != nil {
-		r.Error(n.line, fmt.Sprintf("invalid value %v", n.Value))
+		r.Error(n.line, fmt.Sprintf("invalid value %v", n.Value.Interface()))
 	}
 	switch g.Kind() {
 	case reflect.Struct:
@@ -98,18 +152,29 @@ func checkNodeValidity(n, g node, r *Report) {
 	}
 }
 
-// isCompatible determines if the type of kind n can be converted to the type
-// of kind g in the context of YAML. This is not an exhaustive list, but its
-// enough for the purposes of cloud-config validation.
-func isCompatible(n, g reflect.Kind) bool {
-	switch g {
-	case reflect.String:
-		return n == reflect.String || n == reflect.Int || n == reflect.Float64 || n == reflect.Bool
-	case reflect.Struct:
-		return n == reflect.Struct || n == reflect.Map
-	case reflect.Bool, reflect.Slice, reflect.Int, reflect.Float64:
-		return n == g
-	default:
-		panic(fmt.Sprintf("isCompatible(): unhandled kind %s", g))
+// checkWriteFiles checks to make sure that the target file can actually be
+// written. Note that this check is approximate (it only checks to see if the file
+// is under /usr).
+func checkWriteFiles(cfg node, report *Report) {
+	for _, f := range cfg.Child("write_files").children {
+		c := f.Child("path")
+		if !c.IsValid() {
+			continue
+		}
+
+		d := path.Dir(c.String())
+		switch {
+		case strings.HasPrefix(d, "/usr"):
+			report.Error(c.line, "file cannot be written to a read-only filesystem")
+		}
+	}
+}
+
+// checkWriteFilesUnderCoreos checks to see if the 'write_files' node is a
+// child of 'coreos' (it shouldn't be).
+func checkWriteFilesUnderCoreos(cfg node, report *Report) {
+	c := cfg.Child("coreos").Child("write_files")
+	if c.IsValid() {
+		report.Info(c.line, "write_files doesn't belong under coreos")
 	}
 }

config/validate/rules_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 
@@ -21,6 +19,85 @@ import (
 	"testing"
 )
 
+func TestCheckDiscoveryUrl(t *testing.T) {
+	tests := []struct {
+		config string
+
+		entries []Entry
+	}{
+		{},
+		{
+			config: "coreos:\n  etcd:\n    discovery: https://discovery.etcd.io/00000000000000000000000000000000",
+		},
+		{
+			config: "coreos:\n  etcd:\n    discovery: http://custom.domain/mytoken",
+		},
+		{
+			config:  "coreos:\n  etcd:\n    discovery: disco",
+			entries: []Entry{{entryWarning, "discovery URL is not valid", 3}},
+		},
+	}
+
+	for i, tt := range tests {
+		r := Report{}
+		n, err := parseCloudConfig([]byte(tt.config), &r)
+		if err != nil {
+			panic(err)
+		}
+		checkDiscoveryUrl(n, &r)
+
+		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
+			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
+		}
+	}
+}
+
+func TestCheckEncoding(t *testing.T) {
+	tests := []struct {
+		config string
+
+		entries []Entry
+	}{
+		{},
+		{
+			config: "write_files:\n  - encoding: base64\n    content: aGVsbG8K",
+		},
+		{
+			config: "write_files:\n  - content: !!binary aGVsbG8K",
+		},
+		{
+			config:  "write_files:\n  - encoding: base64\n    content: !!binary aGVsbG8K",
+			entries: []Entry{{entryError, `content cannot be decoded as "base64"`, 3}},
+		},
+		{
+			config: "write_files:\n  - encoding: base64\n    content: !!binary YUdWc2JHOEsK",
+		},
+		{
+			config: "write_files:\n  - encoding: gzip\n    content: !!binary H4sIAOC3tVQAA8tIzcnJ5wIAIDA6NgYAAAA=",
+		},
+		{
+			config: "write_files:\n  - encoding: gzip+base64\n    content: H4sIAOC3tVQAA8tIzcnJ5wIAIDA6NgYAAAA=",
+		},
+		{
+			config:  "write_files:\n  - encoding: custom\n    content: hello",
+			entries: []Entry{{entryError, `content cannot be decoded as "custom"`, 3}},
+		},
+	}
+
+	for i, tt := range tests {
+		r := Report{}
+		n, err := parseCloudConfig([]byte(tt.config), &r)
+		if err != nil {
+			panic(err)
+		}
+		checkEncoding(n, &r)
+
+		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
+			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
+		}
+	}
+}
+
 func TestCheckStructure(t *testing.T) {
 	tests := []struct {
 		config string
@@ -42,6 +119,15 @@ func TestCheckStructure(t *testing.T) {
 			config: "coreos:\n  etcd:\n    discovery: good",
 		},
 
+		// Test for deprecated keys
+		{
+			config: "coreos:\n  etcd:\n    addr: hi",
+		},
+		{
+			config:  "coreos:\n  etcd:\n    proxy: hi",
+			entries: []Entry{{entryWarning, "deprecated key \"proxy\" (etcd2 options no longer work for etcd)", 3}},
+		},
+
 		// Test for error on list of nodes
 		{
 			config: "coreos:\n  units:\n    - hello\n    - goodbye",
@@ -249,3 +335,74 @@ func TestCheckValidity(t *testing.T) {
 		}
 	}
 }
+
+func TestCheckWriteFiles(t *testing.T) {
+	tests := []struct {
+		config string
+
+		entries []Entry
+	}{
+		{},
+		{
+			config: "write_files:\n  - path: /valid",
+		},
+		{
+			config: "write_files:\n  - path: /tmp/usr/valid",
+		},
+		{
+			config:  "write_files:\n  - path: /usr/invalid",
+			entries: []Entry{{entryError, "file cannot be written to a read-only filesystem", 2}},
+		},
+		{
+			config:  "write-files:\n  - path: /tmp/../usr/invalid",
+			entries: []Entry{{entryError, "file cannot be written to a read-only filesystem", 2}},
+		},
+	}
+
+	for i, tt := range tests {
+		r := Report{}
+		n, err := parseCloudConfig([]byte(tt.config), &r)
+		if err != nil {
+			panic(err)
+		}
+		checkWriteFiles(n, &r)
+
+		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
+			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
+		}
+	}
+}
+
+func TestCheckWriteFilesUnderCoreos(t *testing.T) {
+	tests := []struct {
+		config string
+
+		entries []Entry
+	}{
+		{},
+		{
+			config: "write_files:\n  - path: /hi",
+		},
+		{
+			config:  "coreos:\n  write_files:\n    - path: /hi",
+			entries: []Entry{{entryInfo, "write_files doesn't belong under coreos", 2}},
+		},
+		{
+			config:  "coreos:\n  write-files:\n    - path: /hyphen",
+			entries: []Entry{{entryInfo, "write_files doesn't belong under coreos", 2}},
+		},
+	}
+
+	for i, tt := range tests {
+		r := Report{}
+		n, err := parseCloudConfig([]byte(tt.config), &r)
+		if err != nil {
+			panic(err)
+		}
+		checkWriteFilesUnderCoreos(n, &r)
+
+		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
+			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
+		}
+	}
+}

config/validate/validate.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 
@@ -25,7 +23,7 @@ import (
 
 	"github.com/coreos/coreos-cloudinit/config"
 
-	"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/gopkg.in/yaml.v1"
+	"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/coreos/yaml"
 )
 
 var (
@@ -65,7 +63,6 @@ func validateCloudConfig(config []byte, rules []rule) (report Report, err error)
 		return report, err
 	}
 
-	c = normalizeNodeNames(c, &report)
 	for _, r := range rules {
 		r(c, &report)
 	}
@@ -75,30 +72,79 @@ func validateCloudConfig(config []byte, rules []rule) (report Report, err error)
 // parseCloudConfig parses the provided config into a node structure and logs
 // any parsing issues into the provided report. Unrecoverable errors are
 // returned as an error.
-func parseCloudConfig(config []byte, report *Report) (n node, err error) {
-	var raw map[interface{}]interface{}
-	if err := yaml.Unmarshal(config, &raw); err != nil {
+func parseCloudConfig(cfg []byte, report *Report) (node, error) {
+	yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
+		return nameIn
+	}
+	// unmarshal the config into an implicitly-typed form. The yaml library
+	// will implicitly convert types into their normalized form
+	// (e.g. 0744 -> 484, off -> false).
+	var weak map[interface{}]interface{}
+	if err := yaml.Unmarshal(cfg, &weak); err != nil {
 		matches := yamlLineError.FindStringSubmatch(err.Error())
 		if len(matches) == 3 {
 			line, err := strconv.Atoi(matches[1])
 			if err != nil {
-				return n, err
+				return node{}, err
 			}
 			msg := matches[2]
 			report.Error(line, msg)
-			return n, nil
+			return node{}, nil
 		}
 
 		matches = yamlError.FindStringSubmatch(err.Error())
 		if len(matches) == 2 {
 			report.Error(1, matches[1])
-			return n, nil
+			return node{}, nil
 		}
 
-		return n, errors.New("couldn't parse yaml error")
+		return node{}, errors.New("couldn't parse yaml error")
+	}
+	w := NewNode(weak, NewContext(cfg))
+	w = normalizeNodeNames(w, report)
+
+	// unmarshal the config into the explicitly-typed form.
+	yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
+		return strings.Replace(nameIn, "-", "_", -1)
+	}
+	var strong config.CloudConfig
+	if err := yaml.Unmarshal([]byte(cfg), &strong); err != nil {
+		return node{}, err
+	}
+	s := NewNode(strong, NewContext(cfg))
+
+	// coerceNodes weak nodes and strong nodes. strong nodes replace weak nodes
+	// if they are compatible types (this happens when the yaml library
+	// converts the input).
+	// (e.g. weak 484 is replaced by strong 0744, weak 4 is not replaced by
+	// strong false)
+	return coerceNodes(w, s), nil
+}
+
+// coerceNodes recursively evaluates two nodes, returning a new node containing
+// either the weak or strong node's value and its recursively processed
+// children. The strong node's value is used if the two nodes are leafs, are
+// both valid, and are compatible types (defined by isCompatible()). The weak
+// node is returned in all other cases. coerceNodes is used to counteract the
+// effects of yaml's automatic type conversion. The weak node is the one
+// resulting from unmarshalling into an empty interface{} (the type is
+// inferred). The strong node is the one resulting from unmarshalling into a
+// struct. If the two nodes are of compatible types, the yaml library correctly
+// parsed the value into the strongly typed unmarshalling. In this case, we
+// prefer the strong node because its actually the type we are expecting.
+func coerceNodes(w, s node) node {
+	n := w
+	n.children = nil
+	if len(w.children) == 0 && len(s.children) == 0 &&
+		w.IsValid() && s.IsValid() &&
+		isCompatible(w.Kind(), s.Kind()) {
+		n.Value = s.Value
 	}
 
-	return NewNode(raw, NewContext(config)), nil
+	for _, cw := range w.children {
+		n.children = append(n.children, coerceNodes(cw, s.Child(cw.name)))
+	}
+	return n
 }
 
 // normalizeNodeNames replaces all occurences of '-' with '_' within key names

config/validate/validate_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package validate
 
@@ -65,6 +63,31 @@ func TestValidateCloudConfig(t *testing.T) {
 			rules: []rule{func(_ node, _ *Report) { panic("something happened") }},
 			err:   errors.New("something happened"),
 		},
+		{
+			config: "write_files:\n  - permissions: 0744",
+			rules:  Rules,
+		},
+		{
+			config: "write_files:\n  - permissions: '0744'",
+			rules:  Rules,
+		},
+		{
+			config: "write_files:\n  - permissions: 744",
+			rules:  Rules,
+		},
+		{
+			config: "write_files:\n  - permissions: '744'",
+			rules:  Rules,
+		},
+		{
+			config: "coreos:\n  update:\n    reboot-strategy: off",
+			rules:  Rules,
+		},
+		{
+			config: "coreos:\n  update:\n    reboot-strategy: false",
+			rules:  Rules,
+			report: Report{entries: []Entry{{entryError, "invalid value false", 3}}},
+		},
 	}
 
 	for _, tt := range tests {

coreos-cloudinit.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package main
 
@@ -35,12 +33,13 @@ import (
 	"github.com/coreos/coreos-cloudinit/datasource/url"
 	"github.com/coreos/coreos-cloudinit/datasource/waagent"
 	"github.com/coreos/coreos-cloudinit/initialize"
+	"github.com/coreos/coreos-cloudinit/network"
 	"github.com/coreos/coreos-cloudinit/pkg"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
 const (
-	version               = "1.0.2"
+	version               = "1.4.1"
 	datasourceInterval    = 100 * time.Millisecond
 	datasourceMaxInterval = 30 * time.Second
 	datasourceTimeout     = 5 * time.Minute
@@ -107,6 +106,9 @@ var (
 		"azure": oemConfig{
 			"from-waagent": "/var/lib/waagent",
 		},
+		"cloudsigma": oemConfig{
+			"from-cloudsigma-metadata": "true",
+		},
 	}
 )
 
@@ -178,43 +180,18 @@ func main() {
 	}
 
 	fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
-	metadataBytes, err := ds.FetchMetadata()
+	metadata, err := ds.FetchMetadata()
 	if err != nil {
 		fmt.Printf("Failed fetching meta-data from datasource: %v\n", err)
 		os.Exit(1)
 	}
 
-	// Extract IPv4 addresses from metadata if possible
-	var subs map[string]string
-	if len(metadataBytes) > 0 {
-		subs, err = initialize.ExtractIPsFromMetadata(metadataBytes)
-		if err != nil {
-			fmt.Printf("Failed extracting IPs from meta-data: %v\n", err)
-			os.Exit(1)
-		}
-	}
-
 	// Apply environment to user-data
-	env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.convertNetconf, flags.sshKeyName, subs)
+	env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.sshKeyName, metadata)
 	userdata := env.Apply(string(userdataBytes))
 
-	var ccm, ccu *config.CloudConfig
+	var ccu *config.CloudConfig
 	var script *config.Script
-	if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil {
-		fmt.Printf("Failed to parse meta-data: %v\n", err)
-		os.Exit(1)
-	}
-
-	if ccm != nil && flags.convertNetconf != "" {
-		fmt.Printf("Fetching network config from datasource of type %q\n", ds.Type())
-		netconfBytes, err := ds.FetchNetworkConfig(ccm.NetworkConfigPath)
-		if err != nil {
-			fmt.Printf("Failed fetching network config from datasource: %v\n", err)
-			os.Exit(1)
-		}
-		ccm.NetworkConfig = string(netconfBytes)
-	}
-
 	if ud, err := initialize.ParseUserData(userdata); err != nil {
 		fmt.Printf("Failed to parse user-data: %v\nContinuing...\n", err)
 		failure = true
@@ -222,33 +199,36 @@ func main() {
 		switch t := ud.(type) {
 		case *config.CloudConfig:
 			ccu = t
-		case config.Script:
-			script = &t
+		case *config.Script:
+			script = t
 		}
 	}
 
-	var cc *config.CloudConfig
-	if ccm != nil && ccu != nil {
-		fmt.Println("Merging cloud-config from meta-data and user-data")
-		merged := mergeCloudConfig(*ccm, *ccu)
-		cc = &merged
-	} else if ccm != nil && ccu == nil {
-		fmt.Println("Processing cloud-config from meta-data")
-		cc = ccm
-	} else if ccm == nil && ccu != nil {
-		fmt.Println("Processing cloud-config from user-data")
-		cc = ccu
-	} else {
-		fmt.Println("No cloud-config data to handle.")
-	}
+	fmt.Println("Merging cloud-config from meta-data and user-data")
+	cc := mergeConfigs(ccu, metadata)
 
-	if cc != nil {
-		if err = initialize.Apply(*cc, env); err != nil {
-			fmt.Printf("Failed to apply cloud-config: %v\n", err)
+	var ifaces []network.InterfaceGenerator
+	if flags.convertNetconf != "" {
+		var err error
+		switch flags.convertNetconf {
+		case "debian":
+			ifaces, err = network.ProcessDebianNetconf(metadata.NetworkConfig)
+		case "digitalocean":
+			ifaces, err = network.ProcessDigitalOceanNetconf(metadata.NetworkConfig)
+		default:
+			err = fmt.Errorf("Unsupported network config format %q", flags.convertNetconf)
+		}
+		if err != nil {
+			fmt.Printf("Failed to generate interfaces: %v\n", err)
 			os.Exit(1)
 		}
 	}
 
+	if err = initialize.Apply(cc, ifaces, env); err != nil {
+		fmt.Printf("Failed to apply cloud-config: %v\n", err)
+		os.Exit(1)
+	}
+
 	if script != nil {
 		if err = runScript(*script, env); err != nil {
 			fmt.Printf("Failed to run script: %v\n", err)
@@ -261,38 +241,25 @@ func main() {
 	}
 }
 
-// mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from
-// meta-data) onto udcc (a CloudConfig derived from user-data), if they are
-// not already set on udcc (i.e. user-data always takes precedence)
-// NB: This needs to be kept in sync with ParseMetadata so that it tracks all
-// elements of a CloudConfig which that function can populate.
-func mergeCloudConfig(mdcc, udcc config.CloudConfig) (cc config.CloudConfig) {
-	if mdcc.Hostname != "" {
-		if udcc.Hostname != "" {
-			fmt.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)\n", udcc.Hostname, mdcc.Hostname)
-		} else {
-			udcc.Hostname = mdcc.Hostname
-		}
+// mergeConfigs merges certain options from md (meta-data from the datasource)
+// onto cc (a CloudConfig derived from user-data), if they are not already set
+// on cc (i.e. user-data always takes precedence)
+func mergeConfigs(cc *config.CloudConfig, md datasource.Metadata) (out config.CloudConfig) {
+	if cc != nil {
+		out = *cc
+	}
 
-	}
-	for _, key := range mdcc.SSHAuthorizedKeys {
-		udcc.SSHAuthorizedKeys = append(udcc.SSHAuthorizedKeys, key)
-	}
-	if mdcc.NetworkConfigPath != "" {
-		if udcc.NetworkConfigPath != "" {
-			fmt.Printf("Warning: user-data NetworkConfigPath %s overrides metadata NetworkConfigPath %s\n", udcc.NetworkConfigPath, mdcc.NetworkConfigPath)
+	if md.Hostname != "" {
+		if out.Hostname != "" {
+			fmt.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)\n", out.Hostname, md.Hostname)
 		} else {
-			udcc.NetworkConfigPath = mdcc.NetworkConfigPath
+			out.Hostname = md.Hostname
 		}
 	}
-	if mdcc.NetworkConfig != "" {
-		if udcc.NetworkConfig != "" {
-			fmt.Printf("Warning: user-data NetworkConfig %s overrides metadata NetworkConfig %s\n", udcc.NetworkConfig, mdcc.NetworkConfig)
-		} else {
-			udcc.NetworkConfig = mdcc.NetworkConfig
-		}
+	for _, key := range md.SSHPublicKeys {
+		out.SSHAuthorizedKeys = append(out.SSHAuthorizedKeys, key)
 	}
-	return udcc
+	return
 }
 
 // getDatasources creates a slice of possible Datasources for cloudinit based

coreos-cloudinit_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package main
 
@@ -21,116 +19,71 @@ import (
 	"testing"
 
 	"github.com/coreos/coreos-cloudinit/config"
+	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
-func TestMergeCloudConfig(t *testing.T) {
-	simplecc := config.CloudConfig{
-		SSHAuthorizedKeys: []string{"abc", "def"},
-		Hostname:          "foobar",
-		NetworkConfigPath: "/path/somewhere",
-		NetworkConfig:     `{}`,
-	}
-	for i, tt := range []struct {
-		udcc config.CloudConfig
-		mdcc config.CloudConfig
-		want config.CloudConfig
+func TestMergeConfigs(t *testing.T) {
+	tests := []struct {
+		cc *config.CloudConfig
+		md datasource.Metadata
+
+		out config.CloudConfig
 	}{
 		{
-			// If mdcc is empty, udcc should be returned unchanged
-			simplecc,
-			config.CloudConfig{},
-			simplecc,
+			// If md is empty and cc is nil, result should be empty
+			out: config.CloudConfig{},
 		},
 		{
-			// If udcc is empty, mdcc should be returned unchanged(overridden)
-			config.CloudConfig{},
-			simplecc,
-			simplecc,
+			// If md and cc are empty, result should be empty
+			cc:  &config.CloudConfig{},
+			out: config.CloudConfig{},
+		},
+		{
+			// If cc is empty, cc should be returned unchanged
+			cc:  &config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+			out: config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+		},
+		{
+			// If cc is empty, cc should be returned unchanged(overridden)
+			cc:  &config.CloudConfig{},
+			md:  datasource.Metadata{Hostname: "md-host", SSHPublicKeys: map[string]string{"key": "ghi"}},
+			out: config.CloudConfig{SSHAuthorizedKeys: []string{"ghi"}, Hostname: "md-host"},
+		},
+		{
+			// If cc is nil, cc should be returned unchanged(overridden)
+			md:  datasource.Metadata{Hostname: "md-host", SSHPublicKeys: map[string]string{"key": "ghi"}},
+			out: config.CloudConfig{SSHAuthorizedKeys: []string{"ghi"}, Hostname: "md-host"},
 		},
 		{
 			// user-data should override completely in the case of conflicts
-			simplecc,
-			config.CloudConfig{
-				Hostname:          "meta-hostname",
-				NetworkConfigPath: "/path/meta",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
-			simplecc,
+			cc:  &config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+			md:  datasource.Metadata{Hostname: "md-host"},
+			out: config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
 		},
 		{
 			// Mixed merge should succeed
-			config.CloudConfig{
-				SSHAuthorizedKeys: []string{"abc", "def"},
-				Hostname:          "user-hostname",
-				NetworkConfigPath: "/path/somewhere",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
-			config.CloudConfig{
-				SSHAuthorizedKeys: []string{"woof", "qux"},
-				Hostname:          "meta-hostname",
-			},
-			config.CloudConfig{
-				SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"},
-				Hostname:          "user-hostname",
-				NetworkConfigPath: "/path/somewhere",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
+			cc:  &config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+			md:  datasource.Metadata{Hostname: "md-host", SSHPublicKeys: map[string]string{"key": "ghi"}},
+			out: config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def", "ghi"}, Hostname: "cc-host"},
 		},
 		{
 			// Completely non-conflicting merge should be fine
-			config.CloudConfig{
-				Hostname: "supercool",
-			},
-			config.CloudConfig{
-				SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
-				NetworkConfigPath: "/dev/fun",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
-			config.CloudConfig{
-				Hostname:          "supercool",
-				SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
-				NetworkConfigPath: "/dev/fun",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
+			cc:  &config.CloudConfig{Hostname: "cc-host"},
+			md:  datasource.Metadata{SSHPublicKeys: map[string]string{"zaphod": "beeblebrox"}},
+			out: config.CloudConfig{Hostname: "cc-host", SSHAuthorizedKeys: []string{"beeblebrox"}},
 		},
 		{
 			// Non-mergeable settings in user-data should not be affected
-			config.CloudConfig{
-				Hostname:       "mememe",
-				ManageEtcHosts: config.EtcHosts("lolz"),
-			},
-			config.CloudConfig{
-				Hostname:          "youyouyou",
-				NetworkConfigPath: "meta-meta-yo",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
-			config.CloudConfig{
-				Hostname:          "mememe",
-				ManageEtcHosts:    config.EtcHosts("lolz"),
-				NetworkConfigPath: "meta-meta-yo",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
+			cc:  &config.CloudConfig{Hostname: "cc-host", ManageEtcHosts: config.EtcHosts("lolz")},
+			md:  datasource.Metadata{Hostname: "md-host"},
+			out: config.CloudConfig{Hostname: "cc-host", ManageEtcHosts: config.EtcHosts("lolz")},
 		},
-		{
-			// Non-mergeable (unexpected) settings in meta-data are ignored
-			config.CloudConfig{
-				Hostname: "mememe",
-			},
-			config.CloudConfig{
-				ManageEtcHosts:    config.EtcHosts("lolz"),
-				NetworkConfigPath: "meta-meta-yo",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
-			config.CloudConfig{
-				Hostname:          "mememe",
-				NetworkConfigPath: "meta-meta-yo",
-				NetworkConfig:     `{"hostname":"test"}`,
-			},
-		},
-	} {
-		got := mergeCloudConfig(tt.mdcc, tt.udcc)
-		if !reflect.DeepEqual(got, tt.want) {
-			t.Errorf("case #%d: mergeCloudConfig mutated CloudConfig unexpectedly:\ngot:\n%s\nwant:\n%s", i, got, tt.want)
+	}
+
+	for i, tt := range tests {
+		out := mergeConfigs(tt.cc, tt.md)
+		if !reflect.DeepEqual(tt.out, out) {
+			t.Errorf("bad config (%d): want %#v, got %#v", i, tt.out, out)
 		}
 	}
 }

datasource/configdrive/configdrive.go

@@ -1,26 +1,27 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package configdrive
 
 import (
+	"encoding/json"
 	"fmt"
 	"io/ioutil"
 	"os"
 	"path"
+
+	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
 const (
@@ -49,21 +50,36 @@ func (cd *configDrive) ConfigRoot() string {
 	return cd.openstackRoot()
 }
 
-func (cd *configDrive) FetchMetadata() ([]byte, error) {
-	return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "meta_data.json"))
+func (cd *configDrive) FetchMetadata() (metadata datasource.Metadata, err error) {
+	var data []byte
+	var m struct {
+		SSHAuthorizedKeyMap map[string]string `json:"public_keys"`
+		Hostname            string            `json:"hostname"`
+		NetworkConfig       struct {
+			ContentPath string `json:"content_path"`
+		} `json:"network_config"`
+	}
+
+	if data, err = cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "meta_data.json")); err != nil || len(data) == 0 {
+		return
+	}
+	if err = json.Unmarshal([]byte(data), &m); err != nil {
+		return
+	}
+
+	metadata.SSHPublicKeys = m.SSHAuthorizedKeyMap
+	metadata.Hostname = m.Hostname
+	if m.NetworkConfig.ContentPath != "" {
+		metadata.NetworkConfig, err = cd.tryReadFile(path.Join(cd.openstackRoot(), m.NetworkConfig.ContentPath))
+	}
+
+	return
 }
 
 func (cd *configDrive) FetchUserdata() ([]byte, error) {
 	return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "user_data"))
 }
 
-func (cd *configDrive) FetchNetworkConfig(filename string) ([]byte, error) {
-	if filename == "" {
-		return []byte{}, nil
-	}
-	return cd.tryReadFile(path.Join(cd.openstackRoot(), filename))
-}
-
 func (cd *configDrive) Type() string {
 	return "cloud-drive"
 }

datasource/configdrive/configdrive_test.go

@@ -1,99 +1,103 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package configdrive
 
 import (
-	"os"
+	"reflect"
 	"testing"
+
+	"github.com/coreos/coreos-cloudinit/datasource"
+	"github.com/coreos/coreos-cloudinit/datasource/test"
 )
 
-type mockFilesystem []string
-
-func (m mockFilesystem) readFile(filename string) ([]byte, error) {
-	for _, file := range m {
-		if file == filename {
-			return []byte(filename), nil
-		}
-	}
-	return nil, os.ErrNotExist
-}
-
 func TestFetchMetadata(t *testing.T) {
 	for _, tt := range []struct {
-		root     string
-		filename string
-		files    mockFilesystem
+		root  string
+		files test.MockFilesystem
+
+		metadata datasource.Metadata
 	}{
 		{
-			"/",
-			"",
-			mockFilesystem{},
+			root:  "/",
+			files: test.NewMockFilesystem(test.File{Path: "/openstack/latest/meta_data.json", Contents: ""}),
 		},
 		{
-			"/",
-			"/openstack/latest/meta_data.json",
-			mockFilesystem([]string{"/openstack/latest/meta_data.json"}),
+			root:  "/",
+			files: test.NewMockFilesystem(test.File{Path: "/openstack/latest/meta_data.json", Contents: `{"ignore": "me"}`}),
 		},
 		{
-			"/media/configdrive",
-			"/media/configdrive/openstack/latest/meta_data.json",
-			mockFilesystem([]string{"/media/configdrive/openstack/latest/meta_data.json"}),
+			root:     "/",
+			files:    test.NewMockFilesystem(test.File{Path: "/openstack/latest/meta_data.json", Contents: `{"hostname": "host"}`}),
+			metadata: datasource.Metadata{Hostname: "host"},
+		},
+		{
+			root: "/media/configdrive",
+			files: test.NewMockFilesystem(test.File{Path: "/media/configdrive/openstack/latest/meta_data.json", Contents: `{"hostname": "host", "network_config": {"content_path": "config_file.json"}, "public_keys":{"1": "key1", "2": "key2"}}`},
+				test.File{Path: "/media/configdrive/openstack/config_file.json", Contents: "make it work"},
+			),
+			metadata: datasource.Metadata{
+				Hostname:      "host",
+				NetworkConfig: []byte("make it work"),
+				SSHPublicKeys: map[string]string{
+					"1": "key1",
+					"2": "key2",
+				},
+			},
 		},
 	} {
-		cd := configDrive{tt.root, tt.files.readFile}
-		filename, err := cd.FetchMetadata()
+		cd := configDrive{tt.root, tt.files.ReadFile}
+		metadata, err := cd.FetchMetadata()
 		if err != nil {
-			t.Fatalf("bad error for %q: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
 		}
-		if string(filename) != tt.filename {
-			t.Fatalf("bad path for %q: want %q, got %q", tt, tt.filename, filename)
+		if !reflect.DeepEqual(tt.metadata, metadata) {
+			t.Fatalf("bad metadata for %+v: want %#v, got %#v", tt, tt.metadata, metadata)
 		}
 	}
 }
 
 func TestFetchUserdata(t *testing.T) {
 	for _, tt := range []struct {
-		root     string
-		filename string
-		files    mockFilesystem
+		root  string
+		files test.MockFilesystem
+
+		userdata string
 	}{
 		{
 			"/",
+			test.NewMockFilesystem(),
 			"",
-			mockFilesystem{},
 		},
 		{
 			"/",
-			"/openstack/latest/user_data",
-			mockFilesystem([]string{"/openstack/latest/user_data"}),
+			test.NewMockFilesystem(test.File{Path: "/openstack/latest/user_data", Contents: "userdata"}),
+			"userdata",
 		},
 		{
 			"/media/configdrive",
-			"/media/configdrive/openstack/latest/user_data",
-			mockFilesystem([]string{"/media/configdrive/openstack/latest/user_data"}),
+			test.NewMockFilesystem(test.File{Path: "/media/configdrive/openstack/latest/user_data", Contents: "userdata"}),
+			"userdata",
 		},
 	} {
-		cd := configDrive{tt.root, tt.files.readFile}
-		filename, err := cd.FetchUserdata()
+		cd := configDrive{tt.root, tt.files.ReadFile}
+		userdata, err := cd.FetchUserdata()
 		if err != nil {
-			t.Fatalf("bad error for %q: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
 		}
-		if string(filename) != tt.filename {
-			t.Fatalf("bad path for %q: want %q, got %q", tt, tt.filename, filename)
+		if string(userdata) != tt.userdata {
+			t.Fatalf("bad userdata for %+v: want %q, got %q", tt, tt.userdata, userdata)
 		}
 	}
 }

datasource/datasource.go

@@ -1,27 +1,38 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package datasource
 
+import (
+	"net"
+)
+
 type Datasource interface {
 	IsAvailable() bool
 	AvailabilityChanges() bool
 	ConfigRoot() string
-	FetchMetadata() ([]byte, error)
+	FetchMetadata() (Metadata, error)
 	FetchUserdata() ([]byte, error)
-	FetchNetworkConfig(string) ([]byte, error)
 	Type() string
 }
+
+type Metadata struct {
+	PublicIPv4    net.IP
+	PublicIPv6    net.IP
+	PrivateIPv4   net.IP
+	PrivateIPv6   net.IP
+	Hostname      string
+	SSHPublicKeys map[string]string
+	NetworkConfig []byte
+}

datasource/file/file.go

@@ -1,24 +1,24 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package file
 
 import (
 	"io/ioutil"
 	"os"
+
+	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
 type localFile struct {
@@ -42,18 +42,14 @@ func (f *localFile) ConfigRoot() string {
 	return ""
 }
 
-func (f *localFile) FetchMetadata() ([]byte, error) {
-	return []byte{}, nil
+func (f *localFile) FetchMetadata() (datasource.Metadata, error) {
+	return datasource.Metadata{}, nil
 }
 
 func (f *localFile) FetchUserdata() ([]byte, error) {
 	return ioutil.ReadFile(f.path)
 }
 
-func (f *localFile) FetchNetworkConfig(filename string) ([]byte, error) {
-	return nil, nil
-}
-
 func (f *localFile) Type() string {
 	return "local-file"
 }

datasource/metadata/cloudsigma/server_context.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package cloudsigma
 
@@ -26,6 +24,8 @@ import (
 	"os"
 	"strings"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
+
 	"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/cloudsigma/cepgo"
 )
 
@@ -71,7 +71,7 @@ func (_ *serverContextService) Type() string {
 	return "server-context"
 }
 
-func (scs *serverContextService) FetchMetadata() ([]byte, error) {
+func (scs *serverContextService) FetchMetadata() (metadata datasource.Metadata, err error) {
 	var (
 		inputMetadata struct {
 			Name string            `json:"name"`
@@ -90,48 +90,43 @@ func (scs *serverContextService) FetchMetadata() ([]byte, error) {
 				} `json:"vlan"`
 			} `json:"nics"`
 		}
-		outputMetadata struct {
-			Hostname   string            `json:"name"`
-			PublicKeys map[string]string `json:"public_keys"`
-			LocalIPv4  string            `json:"local-ipv4"`
-			PublicIPv4 string            `json:"public-ipv4"`
-		}
+		rawMetadata []byte
 	)
 
-	rawMetadata, err := scs.client.FetchRaw("")
-	if err != nil {
-		return []byte{}, err
+	if rawMetadata, err = scs.client.FetchRaw(""); err != nil {
+		return
 	}
 
-	err = json.Unmarshal(rawMetadata, &inputMetadata)
-	if err != nil {
-		return []byte{}, err
+	if err = json.Unmarshal(rawMetadata, &inputMetadata); err != nil {
+		return
 	}
 
 	if inputMetadata.Name != "" {
-		outputMetadata.Hostname = inputMetadata.Name
+		metadata.Hostname = inputMetadata.Name
 	} else {
-		outputMetadata.Hostname = inputMetadata.UUID
+		metadata.Hostname = inputMetadata.UUID
 	}
 
-	if key, ok := inputMetadata.Meta["ssh_public_key"]; ok {
+	metadata.SSHPublicKeys = map[string]string{}
+	// CloudSigma uses an empty string, rather than no string,
+	// to represent the lack of a SSH key
+	if key, _ := inputMetadata.Meta["ssh_public_key"]; len(key) > 0 {
 		splitted := strings.Split(key, " ")
-		outputMetadata.PublicKeys = make(map[string]string)
-		outputMetadata.PublicKeys[splitted[len(splitted)-1]] = key
+		metadata.SSHPublicKeys[splitted[len(splitted)-1]] = key
 	}
 
 	for _, nic := range inputMetadata.Nics {
 		if nic.IPv4Conf.IP.UUID != "" {
-			outputMetadata.PublicIPv4 = nic.IPv4Conf.IP.UUID
+			metadata.PublicIPv4 = net.ParseIP(nic.IPv4Conf.IP.UUID)
 		}
 		if nic.VLAN.UUID != "" {
 			if localIP, err := scs.findLocalIP(nic.Mac); err == nil {
-				outputMetadata.LocalIPv4 = localIP
+				metadata.PrivateIPv4 = localIP
 			}
 		}
 	}
 
-	return json.Marshal(outputMetadata)
+	return
 }
 
 func (scs *serverContextService) FetchUserdata() ([]byte, error) {
@@ -152,18 +147,14 @@ func (scs *serverContextService) FetchUserdata() ([]byte, error) {
 	return []byte(userData), nil
 }
 
-func (scs *serverContextService) FetchNetworkConfig(a string) ([]byte, error) {
-	return nil, nil
-}
-
-func (scs *serverContextService) findLocalIP(mac string) (string, error) {
+func (scs *serverContextService) findLocalIP(mac string) (net.IP, error) {
 	ifaces, err := net.Interfaces()
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 	ifaceMac, err := net.ParseMAC(mac)
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 	for _, iface := range ifaces {
 		if !bytes.Equal(iface.HardwareAddr, ifaceMac) {
@@ -178,12 +169,12 @@ func (scs *serverContextService) findLocalIP(mac string) (string, error) {
 			switch ip := addr.(type) {
 			case *net.IPNet:
 				if ip.IP.To4() != nil {
-					return ip.IP.To4().String(), nil
+					return ip.IP.To4(), nil
 				}
 			}
 		}
 	}
-	return "", errors.New("Local IP not found")
+	return nil, errors.New("Local IP not found")
 }
 
 func isBase64Encoded(field string, userdata map[string]string) bool {

datasource/metadata/cloudsigma/server_context_test.go

@@ -1,23 +1,21 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package cloudsigma
 
 import (
-	"encoding/json"
+	"net"
 	"reflect"
 	"testing"
 )
@@ -45,13 +43,28 @@ func (f *fakeCepgoClient) FetchRaw(key string) ([]byte, error) {
 	return f.raw, f.err
 }
 
-func TestServerContextFetchMetadata(t *testing.T) {
-	var metadata struct {
-		Hostname   string            `json:"name"`
-		PublicKeys map[string]string `json:"public_keys"`
-		LocalIPv4  string            `json:"local-ipv4"`
-		PublicIPv4 string            `json:"public-ipv4"`
+func TestServerContextWithEmptyPublicSSHKey(t *testing.T) {
+	client := new(fakeCepgoClient)
+	scs := NewServerContextService()
+	scs.client = client
+	client.raw = []byte(`{
+		"meta": {
+			"base64_fields": "cloudinit-user-data",
+			"cloudinit-user-data": "I2Nsb3VkLWNvbmZpZwoKaG9zdG5hbWU6IGNvcmVvczE=",
+			"ssh_public_key": ""
+		}
+	}`)
+	metadata, err := scs.FetchMetadata()
+	if err != nil {
+		t.Error(err.Error())
 	}
+
+	if len(metadata.SSHPublicKeys) != 0 {
+		t.Error("There should be no Public SSH Keys provided")
+	}
+}
+
+func TestServerContextFetchMetadata(t *testing.T) {
 	client := new(fakeCepgoClient)
 	scs := NewServerContextService()
 	scs.client = client
@@ -116,24 +129,20 @@ func TestServerContextFetchMetadata(t *testing.T) {
 		"uuid": "20a0059b-041e-4d0c-bcc6-9b2852de48b3"
 	}`)
 
-	metadataBytes, err := scs.FetchMetadata()
+	metadata, err := scs.FetchMetadata()
 	if err != nil {
 		t.Error(err.Error())
 	}
 
-	if err := json.Unmarshal(metadataBytes, &metadata); err != nil {
-		t.Error(err.Error())
-	}
-
 	if metadata.Hostname != "coreos" {
 		t.Errorf("Hostname is not 'coreos' but %s instead", metadata.Hostname)
 	}
 
-	if metadata.PublicKeys["john@doe"] != "ssh-rsa AAAAB3NzaC1yc2E.../hQ5D5 john@doe" {
+	if metadata.SSHPublicKeys["john@doe"] != "ssh-rsa AAAAB3NzaC1yc2E.../hQ5D5 john@doe" {
 		t.Error("Public SSH Keys are not being read properly")
 	}
 
-	if metadata.PublicIPv4 != "31.171.251.74" {
+	if !metadata.PublicIPv4.Equal(net.ParseIP("31.171.251.74")) {
 		t.Errorf("Public IP is not 31.171.251.74 but %s instead", metadata.PublicIPv4)
 	}
 }

datasource/metadata/digitalocean/metadata.go

@@ -1,25 +1,25 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package digitalocean
 
 import (
 	"encoding/json"
+	"net"
 	"strconv"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 )
 
@@ -61,8 +61,6 @@ type Metadata struct {
 }
 
 type metadataService struct {
-	interfaces Interfaces
-	dns        DNS
 	metadata.MetadataService
 }
 
@@ -70,52 +68,41 @@ func NewDatasource(root string) *metadataService {
 	return &metadataService{MetadataService: metadata.NewDatasource(root, apiVersion, userdataUrl, metadataPath)}
 }
 
-func (ms *metadataService) FetchMetadata() ([]byte, error) {
-	data, err := ms.FetchData(ms.MetadataUrl())
-	if err != nil || len(data) == 0 {
-		return []byte{}, err
+func (ms *metadataService) FetchMetadata() (metadata datasource.Metadata, err error) {
+	var data []byte
+	var m Metadata
+
+	if data, err = ms.FetchData(ms.MetadataUrl()); err != nil || len(data) == 0 {
+		return
+	}
+	if err = json.Unmarshal(data, &m); err != nil {
+		return
 	}
 
-	var metadata Metadata
-	if err := json.Unmarshal(data, &metadata); err != nil {
-		return []byte{}, err
-	}
-
-	ms.interfaces = metadata.Interfaces
-	ms.dns = metadata.DNS
-
-	attrs := make(map[string]interface{})
-	if len(metadata.Interfaces.Public) > 0 {
-		if metadata.Interfaces.Public[0].IPv4 != nil {
-			attrs["public-ipv4"] = metadata.Interfaces.Public[0].IPv4.IPAddress
+	if len(m.Interfaces.Public) > 0 {
+		if m.Interfaces.Public[0].IPv4 != nil {
+			metadata.PublicIPv4 = net.ParseIP(m.Interfaces.Public[0].IPv4.IPAddress)
 		}
-		if metadata.Interfaces.Public[0].IPv6 != nil {
-			attrs["public-ipv6"] = metadata.Interfaces.Public[0].IPv6.IPAddress
+		if m.Interfaces.Public[0].IPv6 != nil {
+			metadata.PublicIPv6 = net.ParseIP(m.Interfaces.Public[0].IPv6.IPAddress)
 		}
 	}
-	if len(metadata.Interfaces.Private) > 0 {
-		if metadata.Interfaces.Private[0].IPv4 != nil {
-			attrs["local-ipv4"] = metadata.Interfaces.Private[0].IPv4.IPAddress
+	if len(m.Interfaces.Private) > 0 {
+		if m.Interfaces.Private[0].IPv4 != nil {
+			metadata.PrivateIPv4 = net.ParseIP(m.Interfaces.Private[0].IPv4.IPAddress)
 		}
-		if metadata.Interfaces.Private[0].IPv6 != nil {
-			attrs["local-ipv6"] = metadata.Interfaces.Private[0].IPv6.IPAddress
+		if m.Interfaces.Private[0].IPv6 != nil {
+			metadata.PrivateIPv6 = net.ParseIP(m.Interfaces.Private[0].IPv6.IPAddress)
 		}
 	}
-	attrs["hostname"] = metadata.Hostname
-	keys := make(map[string]string)
-	for i, key := range metadata.PublicKeys {
-		keys[strconv.Itoa(i)] = key
+	metadata.Hostname = m.Hostname
+	metadata.SSHPublicKeys = map[string]string{}
+	for i, key := range m.PublicKeys {
+		metadata.SSHPublicKeys[strconv.Itoa(i)] = key
 	}
-	attrs["public_keys"] = keys
+	metadata.NetworkConfig = data
 
-	return json.Marshal(attrs)
-}
-
-func (ms metadataService) FetchNetworkConfig(filename string) ([]byte, error) {
-	return json.Marshal(Metadata{
-		Interfaces: ms.interfaces,
-		DNS:        ms.dns,
-	})
+	return
 }
 
 func (ms metadataService) Type() string {

datasource/metadata/digitalocean/metadata_test.go

@@ -1,26 +1,26 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package digitalocean
 
 import (
-	"bytes"
 	"fmt"
+	"net"
+	"reflect"
 	"testing"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
 	"github.com/coreos/coreos-cloudinit/pkg"
@@ -38,7 +38,7 @@ func TestFetchMetadata(t *testing.T) {
 		root         string
 		metadataPath string
 		resources    map[string]string
-		expect       []byte
+		expect       datasource.Metadata
 		clientErr    error
 		expectErr    error
 	}{
@@ -83,7 +83,42 @@ func TestFetchMetadata(t *testing.T) {
   }
 }`,
 			},
-			expect: []byte(`{"hostname":"","public-ipv4":"192.168.1.2","public-ipv6":"fe00::","public_keys":{"0":"publickey1","1":"publickey2"}}`),
+			expect: datasource.Metadata{
+				PublicIPv4: net.ParseIP("192.168.1.2"),
+				PublicIPv6: net.ParseIP("fe00::"),
+				SSHPublicKeys: map[string]string{
+					"0": "publickey1",
+					"1": "publickey2",
+				},
+				NetworkConfig: []byte(`{
+  "droplet_id": 1,
+  "user_data": "hello",
+  "vendor_data": "hello",
+  "public_keys": [
+    "publickey1",
+    "publickey2"
+  ],
+  "region": "nyc2",
+  "interfaces": {
+    "public": [
+      {
+        "ipv4": {
+          "ip_address": "192.168.1.2",
+          "netmask": "255.255.255.0",
+          "gateway": "192.168.1.1"
+        },
+        "ipv6": {
+          "ip_address": "fe00::",
+          "cidr": 126,
+          "gateway": "fe00::"
+        },
+        "mac": "ab:cd:ef:gh:ij",
+        "type": "public"
+      }
+    ]
+  }
+}`),
+			},
 		},
 		{
 			clientErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
@@ -101,8 +136,8 @@ func TestFetchMetadata(t *testing.T) {
 		if Error(err) != Error(tt.expectErr) {
 			t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
 		}
-		if !bytes.Equal(metadata, tt.expect) {
-			t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
+		if !reflect.DeepEqual(tt.expect, metadata) {
+			t.Fatalf("bad fetch (%q): want %#q, got %#q", tt.resources, tt.expect, metadata)
 		}
 	}
 }

datasource/metadata/ec2/metadata.go

@@ -1,28 +1,27 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package ec2
 
 import (
 	"bufio"
 	"bytes"
-	"encoding/json"
 	"fmt"
+	"net"
 	"strings"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 	"github.com/coreos/coreos-cloudinit/pkg"
 )
@@ -42,59 +41,51 @@ func NewDatasource(root string) *metadataService {
 	return &metadataService{metadata.NewDatasource(root, apiVersion, userdataPath, metadataPath)}
 }
 
-func (ms metadataService) FetchMetadata() ([]byte, error) {
-	attrs := make(map[string]interface{})
+func (ms metadataService) FetchMetadata() (datasource.Metadata, error) {
+	metadata := datasource.Metadata{}
+
 	if keynames, err := ms.fetchAttributes(fmt.Sprintf("%s/public-keys", ms.MetadataUrl())); err == nil {
 		keyIDs := make(map[string]string)
 		for _, keyname := range keynames {
 			tokens := strings.SplitN(keyname, "=", 2)
 			if len(tokens) != 2 {
-				return nil, fmt.Errorf("malformed public key: %q", keyname)
+				return metadata, fmt.Errorf("malformed public key: %q", keyname)
 			}
 			keyIDs[tokens[1]] = tokens[0]
 		}
 
-		keys := make(map[string]string)
+		metadata.SSHPublicKeys = map[string]string{}
 		for name, id := range keyIDs {
 			sshkey, err := ms.fetchAttribute(fmt.Sprintf("%s/public-keys/%s/openssh-key", ms.MetadataUrl(), id))
 			if err != nil {
-				return nil, err
+				return metadata, err
 			}
-			keys[name] = sshkey
+			metadata.SSHPublicKeys[name] = sshkey
 			fmt.Printf("Found SSH key for %q\n", name)
 		}
-		attrs["public_keys"] = keys
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return nil, err
+		return metadata, err
 	}
 
 	if hostname, err := ms.fetchAttribute(fmt.Sprintf("%s/hostname", ms.MetadataUrl())); err == nil {
-		attrs["hostname"] = hostname
+		metadata.Hostname = strings.Split(hostname, " ")[0]
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return nil, err
+		return metadata, err
 	}
 
 	if localAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/local-ipv4", ms.MetadataUrl())); err == nil {
-		attrs["local-ipv4"] = localAddr
+		metadata.PrivateIPv4 = net.ParseIP(localAddr)
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return nil, err
+		return metadata, err
 	}
 
 	if publicAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/public-ipv4", ms.MetadataUrl())); err == nil {
-		attrs["public-ipv4"] = publicAddr
+		metadata.PublicIPv4 = net.ParseIP(publicAddr)
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return nil, err
+		return metadata, err
 	}
 
-	if content_path, err := ms.fetchAttribute(fmt.Sprintf("%s/network_config/content_path", ms.MetadataUrl())); err == nil {
-		attrs["network_config"] = map[string]string{
-			"content_path": content_path,
-		}
-	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return nil, err
-	}
-
-	return json.Marshal(attrs)
+	return metadata, nil
 }
 
 func (ms metadataService) Type() string {

datasource/metadata/ec2/metadata_test.go

@@ -1,27 +1,26 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package ec2
 
 import (
-	"bytes"
 	"fmt"
+	"net"
 	"reflect"
 	"testing"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
 	"github.com/coreos/coreos-cloudinit/pkg"
@@ -147,7 +146,7 @@ func TestFetchMetadata(t *testing.T) {
 		root         string
 		metadataPath string
 		resources    map[string]string
-		expect       []byte
+		expect       datasource.Metadata
 		clientErr    error
 		expectErr    error
 	}{
@@ -163,15 +162,37 @@ func TestFetchMetadata(t *testing.T) {
 			root:         "/",
 			metadataPath: "2009-04-04/meta-data",
 			resources: map[string]string{
-				"/2009-04-04/meta-data/hostname":                    "host",
-				"/2009-04-04/meta-data/local-ipv4":                  "1.2.3.4",
-				"/2009-04-04/meta-data/public-ipv4":                 "5.6.7.8",
-				"/2009-04-04/meta-data/public-keys":                 "0=test1\n",
-				"/2009-04-04/meta-data/public-keys/0":               "openssh-key",
-				"/2009-04-04/meta-data/public-keys/0/openssh-key":   "key",
-				"/2009-04-04/meta-data/network_config/content_path": "path",
+				"/2009-04-04/meta-data/hostname":                  "host",
+				"/2009-04-04/meta-data/local-ipv4":                "1.2.3.4",
+				"/2009-04-04/meta-data/public-ipv4":               "5.6.7.8",
+				"/2009-04-04/meta-data/public-keys":               "0=test1\n",
+				"/2009-04-04/meta-data/public-keys/0":             "openssh-key",
+				"/2009-04-04/meta-data/public-keys/0/openssh-key": "key",
+			},
+			expect: datasource.Metadata{
+				Hostname:      "host",
+				PrivateIPv4:   net.ParseIP("1.2.3.4"),
+				PublicIPv4:    net.ParseIP("5.6.7.8"),
+				SSHPublicKeys: map[string]string{"test1": "key"},
+			},
+		},
+		{
+			root:         "/",
+			metadataPath: "2009-04-04/meta-data",
+			resources: map[string]string{
+				"/2009-04-04/meta-data/hostname":                  "host domain another_domain",
+				"/2009-04-04/meta-data/local-ipv4":                "1.2.3.4",
+				"/2009-04-04/meta-data/public-ipv4":               "5.6.7.8",
+				"/2009-04-04/meta-data/public-keys":               "0=test1\n",
+				"/2009-04-04/meta-data/public-keys/0":             "openssh-key",
+				"/2009-04-04/meta-data/public-keys/0/openssh-key": "key",
+			},
+			expect: datasource.Metadata{
+				Hostname:      "host",
+				PrivateIPv4:   net.ParseIP("1.2.3.4"),
+				PublicIPv4:    net.ParseIP("5.6.7.8"),
+				SSHPublicKeys: map[string]string{"test1": "key"},
 			},
-			expect: []byte(`{"hostname":"host","local-ipv4":"1.2.3.4","network_config":{"content_path":"path"},"public-ipv4":"5.6.7.8","public_keys":{"test1":"key"}}`),
 		},
 		{
 			clientErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
@@ -187,8 +208,8 @@ func TestFetchMetadata(t *testing.T) {
 		if Error(err) != Error(tt.expectErr) {
 			t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
 		}
-		if !bytes.Equal(metadata, tt.expect) {
-			t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
+		if !reflect.DeepEqual(tt.expect, metadata) {
+			t.Fatalf("bad fetch (%q): want %#v, got %#v", tt.resources, tt.expect, metadata)
 		}
 	}
 }

datasource/metadata/metadata.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package metadata
 
@@ -54,10 +52,6 @@ func (ms MetadataService) FetchUserdata() ([]byte, error) {
 	return ms.FetchData(ms.UserdataUrl())
 }
 
-func (ms MetadataService) FetchNetworkConfig(filename string) ([]byte, error) {
-	return nil, nil
-}
-
 func (ms MetadataService) FetchData(url string) ([]byte, error) {
 	if data, err := ms.Client.GetRetry(url); err == nil {
 		return data, err

datasource/metadata/metadata_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package metadata
 

datasource/metadata/test/test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package test
 

datasource/proc_cmdline/proc_cmdline.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package proc_cmdline
 
@@ -22,6 +20,7 @@ import (
 	"log"
 	"strings"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/pkg"
 )
 
@@ -57,8 +56,8 @@ func (c *procCmdline) ConfigRoot() string {
 	return ""
 }
 
-func (c *procCmdline) FetchMetadata() ([]byte, error) {
-	return []byte{}, nil
+func (c *procCmdline) FetchMetadata() (datasource.Metadata, error) {
+	return datasource.Metadata{}, nil
 }
 
 func (c *procCmdline) FetchUserdata() ([]byte, error) {
@@ -82,10 +81,6 @@ func (c *procCmdline) FetchUserdata() ([]byte, error) {
 	return cfg, nil
 }
 
-func (c *procCmdline) FetchNetworkConfig(filename string) ([]byte, error) {
-	return nil, nil
-}
-
 func (c *procCmdline) Type() string {
 	return "proc-cmdline"
 }

datasource/proc_cmdline/proc_cmdline_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package proc_cmdline
 

datasource/test/filesystem.go

@@ -0,0 +1,57 @@
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package test
+
+import (
+	"fmt"
+	"os"
+	"path"
+)
+
+type MockFilesystem map[string]File
+
+type File struct {
+	Path      string
+	Contents  string
+	Directory bool
+}
+
+func (m MockFilesystem) ReadFile(filename string) ([]byte, error) {
+	if f, ok := m[path.Clean(filename)]; ok {
+		if f.Directory {
+			return nil, fmt.Errorf("read %s: is a directory", filename)
+		}
+		return []byte(f.Contents), nil
+	}
+	return nil, os.ErrNotExist
+}
+
+func NewMockFilesystem(files ...File) MockFilesystem {
+	fs := MockFilesystem{}
+	for _, file := range files {
+		fs[file.Path] = file
+
+		// Create the directories leading up to the file
+		p := path.Dir(file.Path)
+		for p != "/" && p != "." {
+			if f, ok := fs[p]; ok && !f.Directory {
+				panic(fmt.Sprintf("%q already exists and is not a directory (%#v)", p, f))
+			}
+			fs[p] = File{Path: p, Directory: true}
+			p = path.Dir(p)
+		}
+	}
+	return fs
+}

datasource/test/filesystem_test.go

@@ -0,0 +1,115 @@
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package test
+
+import (
+	"errors"
+	"os"
+	"reflect"
+	"testing"
+)
+
+func TestReadFile(t *testing.T) {
+	tests := []struct {
+		filesystem MockFilesystem
+
+		filename string
+		contents string
+		err      error
+	}{
+		{
+			filename: "dne",
+			err:      os.ErrNotExist,
+		},
+		{
+			filesystem: MockFilesystem{
+				"exists": File{Contents: "hi"},
+			},
+			filename: "exists",
+			contents: "hi",
+		},
+		{
+			filesystem: MockFilesystem{
+				"dir": File{Directory: true},
+			},
+			filename: "dir",
+			err:      errors.New("read dir: is a directory"),
+		},
+	}
+
+	for i, tt := range tests {
+		contents, err := tt.filesystem.ReadFile(tt.filename)
+		if tt.contents != string(contents) {
+			t.Errorf("bad contents (test %d): want %q, got %q", i, tt.contents, string(contents))
+		}
+		if !reflect.DeepEqual(tt.err, err) {
+			t.Errorf("bad error (test %d): want %v, got %v", i, tt.err, err)
+		}
+	}
+}
+
+func TestNewMockFilesystem(t *testing.T) {
+	tests := []struct {
+		files []File
+
+		filesystem MockFilesystem
+	}{
+		{
+			filesystem: MockFilesystem{},
+		},
+		{
+			files: []File{File{Path: "file"}},
+			filesystem: MockFilesystem{
+				"file": File{Path: "file"},
+			},
+		},
+		{
+			files: []File{File{Path: "/file"}},
+			filesystem: MockFilesystem{
+				"/file": File{Path: "/file"},
+			},
+		},
+		{
+			files: []File{File{Path: "/dir/file"}},
+			filesystem: MockFilesystem{
+				"/dir":      File{Path: "/dir", Directory: true},
+				"/dir/file": File{Path: "/dir/file"},
+			},
+		},
+		{
+			files: []File{File{Path: "/dir/dir/file"}},
+			filesystem: MockFilesystem{
+				"/dir":          File{Path: "/dir", Directory: true},
+				"/dir/dir":      File{Path: "/dir/dir", Directory: true},
+				"/dir/dir/file": File{Path: "/dir/dir/file"},
+			},
+		},
+		{
+			files: []File{File{Path: "/dir/dir/dir", Directory: true}},
+			filesystem: MockFilesystem{
+				"/dir":         File{Path: "/dir", Directory: true},
+				"/dir/dir":     File{Path: "/dir/dir", Directory: true},
+				"/dir/dir/dir": File{Path: "/dir/dir/dir", Directory: true},
+			},
+		},
+	}
+
+	for i, tt := range tests {
+		filesystem := NewMockFilesystem(tt.files...)
+		if !reflect.DeepEqual(tt.filesystem, filesystem) {
+			t.Errorf("bad filesystem (test %d): want %#v, got %#v", i, tt.filesystem, filesystem)
+		}
+	}
+}

datasource/url/url.go

@@ -1,22 +1,21 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package url
 
 import (
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/pkg"
 )
 
@@ -42,8 +41,8 @@ func (f *remoteFile) ConfigRoot() string {
 	return ""
 }
 
-func (f *remoteFile) FetchMetadata() ([]byte, error) {
-	return []byte{}, nil
+func (f *remoteFile) FetchMetadata() (datasource.Metadata, error) {
+	return datasource.Metadata{}, nil
 }
 
 func (f *remoteFile) FetchUserdata() ([]byte, error) {
@@ -51,10 +50,6 @@ func (f *remoteFile) FetchUserdata() ([]byte, error) {
 	return client.GetRetry(f.url)
 }
 
-func (f *remoteFile) FetchNetworkConfig(filename string) ([]byte, error) {
-	return nil, nil
-}
-
 func (f *remoteFile) Type() string {
 	return "url"
 }

datasource/waagent/waagent.go

@@ -1,29 +1,28 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package waagent
 
 import (
-	"encoding/json"
 	"encoding/xml"
 	"fmt"
 	"io/ioutil"
 	"net"
 	"os"
 	"path"
+
+	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
 type waagent struct {
@@ -48,13 +47,13 @@ func (a *waagent) ConfigRoot() string {
 	return a.root
 }
 
-func (a *waagent) FetchMetadata() ([]byte, error) {
-	metadataBytes, err := a.tryReadFile(path.Join(a.root, "SharedConfig.xml"))
-	if err != nil {
-		return nil, err
+func (a *waagent) FetchMetadata() (metadata datasource.Metadata, err error) {
+	var metadataBytes []byte
+	if metadataBytes, err = a.tryReadFile(path.Join(a.root, "SharedConfig.xml")); err != nil {
+		return
 	}
 	if len(metadataBytes) == 0 {
-		return metadataBytes, nil
+		return
 	}
 
 	type Instance struct {
@@ -76,40 +75,34 @@ func (a *waagent) FetchMetadata() ([]byte, error) {
 		}
 	}
 
-	var metadata SharedConfig
-	if err := xml.Unmarshal(metadataBytes, &metadata); err != nil {
-		return nil, err
+	var m SharedConfig
+	if err = xml.Unmarshal(metadataBytes, &m); err != nil {
+		return
 	}
 
 	var instance Instance
-	for _, i := range metadata.Instances.Instances {
-		if i.Id == metadata.Incarnation.Instance {
+	for _, i := range m.Instances.Instances {
+		if i.Id == m.Incarnation.Instance {
 			instance = i
 			break
 		}
 	}
 
-	attrs := map[string]string{
-		"local-ipv4": instance.Address,
-	}
+	metadata.PrivateIPv4 = net.ParseIP(instance.Address)
 	for _, e := range instance.InputEndpoints.Endpoints {
 		host, _, err := net.SplitHostPort(e.LoadBalancedPublicAddress)
 		if err == nil {
-			attrs["public-ipv4"] = host
+			metadata.PublicIPv4 = net.ParseIP(host)
 			break
 		}
 	}
-	return json.Marshal(attrs)
+	return
 }
 
 func (a *waagent) FetchUserdata() ([]byte, error) {
 	return a.tryReadFile(path.Join(a.root, "CustomData"))
 }
 
-func (a *waagent) FetchNetworkConfig(filename string) ([]byte, error) {
-	return nil, nil
-}
-
 func (a *waagent) Type() string {
 	return "waagent"
 }

datasource/waagent/waagent_test.go

@@ -1,61 +1,49 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package waagent
 
 import (
-	"encoding/json"
-	"os"
+	"net"
 	"reflect"
 	"testing"
+
+	"github.com/coreos/coreos-cloudinit/datasource"
+	"github.com/coreos/coreos-cloudinit/datasource/test"
 )
 
-type mockFilesystem map[string][]byte
-
-func (m mockFilesystem) readFile(filename string) ([]byte, error) {
-	if contents := m[filename]; contents != nil {
-		return contents, nil
-	}
-	return nil, os.ErrNotExist
-}
-
 func TestFetchMetadata(t *testing.T) {
 	for _, tt := range []struct {
 		root     string
-		files    mockFilesystem
-		metadata map[string]string
+		files    test.MockFilesystem
+		metadata datasource.Metadata
 	}{
 		{
-			"/",
-			mockFilesystem{},
-			nil,
+			root:  "/",
+			files: test.NewMockFilesystem(),
 		},
 		{
-			"/",
-			mockFilesystem{"/SharedConfig.xml": []byte("")},
-			nil,
+			root:  "/",
+			files: test.NewMockFilesystem(test.File{Path: "/SharedConfig.xml", Contents: ""}),
 		},
 		{
-			"/var/lib/waagent",
-			mockFilesystem{"/var/lib/waagent/SharedConfig.xml": []byte("")},
-			nil,
+			root:  "/var/lib/waagent",
+			files: test.NewMockFilesystem(test.File{Path: "/var/lib/waagent/SharedConfig.xml", Contents: ""}),
 		},
 		{
-			"/var/lib/waagent",
-			mockFilesystem{"/var/lib/waagent/SharedConfig.xml": []byte(`<?xml version="1.0" encoding="utf-8"?>
+			root: "/var/lib/waagent",
+			files: test.NewMockFilesystem(test.File{Path: "/var/lib/waagent/SharedConfig.xml", Contents: `<?xml version="1.0" encoding="utf-8"?>
 <SharedConfig version="1.0.0.0" goalStateIncarnation="1">
   <Deployment name="c8f9e4c9c18948e1bebf57c5685da756" guid="{1d10394f-c741-4a1a-a6bb-278f213c5a5e}" incarnation="0" isNonCancellableTopologyChangeEnabled="false">
     <Service name="core-test-1" guid="{00000000-0000-0000-0000-000000000000}" />
@@ -91,26 +79,20 @@ func TestFetchMetadata(t *testing.T) {
       </InputEndpoints>
     </Instance>
   </Instances>
-</SharedConfig>`)},
-			map[string]string{
-				"local-ipv4":  "100.73.202.64",
-				"public-ipv4": "191.239.39.77",
+</SharedConfig>`}),
+			metadata: datasource.Metadata{
+				PrivateIPv4: net.ParseIP("100.73.202.64"),
+				PublicIPv4:  net.ParseIP("191.239.39.77"),
 			},
 		},
 	} {
-		a := waagent{tt.root, tt.files.readFile}
-		metadataBytes, err := a.FetchMetadata()
+		a := waagent{tt.root, tt.files.ReadFile}
+		metadata, err := a.FetchMetadata()
 		if err != nil {
-			t.Fatalf("bad error for %q: want %v, got %q", tt, nil, err)
-		}
-		var metadata map[string]string
-		if len(metadataBytes) > 0 {
-			if err := json.Unmarshal(metadataBytes, &metadata); err != nil {
-				panic(err)
-			}
+			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
 		}
 		if !reflect.DeepEqual(tt.metadata, metadata) {
-			t.Fatalf("bad metadata for %q: want %q, got %q", tt, tt.metadata, metadata)
+			t.Fatalf("bad metadata for %+v: want %#v, got %#v", tt, tt.metadata, metadata)
 		}
 	}
 }
@@ -118,25 +100,25 @@ func TestFetchMetadata(t *testing.T) {
 func TestFetchUserdata(t *testing.T) {
 	for _, tt := range []struct {
 		root  string
-		files mockFilesystem
+		files test.MockFilesystem
 	}{
 		{
 			"/",
-			mockFilesystem{},
+			test.NewMockFilesystem(),
 		},
 		{
 			"/",
-			mockFilesystem{"/CustomData": []byte{}},
+			test.NewMockFilesystem(test.File{Path: "/CustomData", Contents: ""}),
 		},
 		{
 			"/var/lib/waagent/",
-			mockFilesystem{"/var/lib/waagent/CustomData": []byte{}},
+			test.NewMockFilesystem(test.File{Path: "/var/lib/waagent/CustomData", Contents: ""}),
 		},
 	} {
-		a := waagent{tt.root, tt.files.readFile}
+		a := waagent{tt.root, tt.files.ReadFile}
 		_, err := a.FetchUserdata()
 		if err != nil {
-			t.Fatalf("bad error for %q: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
 		}
 	}
 }

initialize/config.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 
@@ -44,7 +42,7 @@ type CloudConfigUnit interface {
 // Apply renders a CloudConfig to an Environment. This can involve things like
 // configuring the hostname, adding new users, writing various configuration
 // files to disk, and manipulating systemd services.
-func Apply(cfg config.CloudConfig, env *Environment) error {
+func Apply(cfg config.CloudConfig, ifaces []network.InterfaceGenerator, env *Environment) error {
 	if cfg.Hostname != "" {
 		if err := system.SetHostname(cfg.Hostname); err != nil {
 			return err
@@ -87,6 +85,12 @@ func Apply(cfg config.CloudConfig, env *Environment) error {
 				return err
 			}
 		}
+		for _, u := range user.SSHImportGithubUsers {
+			log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", u, user.Name)
+			if err := SSHImportGithubUser(user.Name, u); err != nil {
+				return err
+			}
+		}
 		if user.SSHImportURL != "" {
 			log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
 			if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
@@ -110,9 +114,10 @@ func Apply(cfg config.CloudConfig, env *Environment) error {
 	}
 
 	for _, ccf := range []CloudConfigFile{
-		system.OEM{OEM: cfg.Coreos.OEM},
-		system.Update{Update: cfg.Coreos.Update, ReadConfig: system.DefaultReadConfig},
+		system.OEM{OEM: cfg.CoreOS.OEM},
+		system.Update{Update: cfg.CoreOS.Update, ReadConfig: system.DefaultReadConfig},
 		system.EtcHosts{EtcHosts: cfg.ManageEtcHosts},
+		system.Flannel{Flannel: cfg.CoreOS.Flannel},
 	} {
 		f, err := ccf.File()
 		if err != nil {
@@ -124,16 +129,16 @@ func Apply(cfg config.CloudConfig, env *Environment) error {
 	}
 
 	var units []system.Unit
-	for _, u := range cfg.Coreos.Units {
+	for _, u := range cfg.CoreOS.Units {
 		units = append(units, system.Unit{Unit: u})
 	}
 
 	for _, ccu := range []CloudConfigUnit{
-		system.Etcd{Etcd: cfg.Coreos.Etcd},
-		system.Fleet{Fleet: cfg.Coreos.Fleet},
-		system.Locksmith{Locksmith: cfg.Coreos.Locksmith},
-		system.Flannel{Flannel: cfg.Coreos.Flannel},
-		system.Update{Update: cfg.Coreos.Update, ReadConfig: system.DefaultReadConfig},
+		system.Etcd{Etcd: cfg.CoreOS.Etcd},
+		system.Etcd2{Etcd2: cfg.CoreOS.Etcd2},
+		system.Fleet{Fleet: cfg.CoreOS.Fleet},
+		system.Locksmith{Locksmith: cfg.CoreOS.Locksmith},
+		system.Update{Update: cfg.CoreOS.Update, ReadConfig: system.DefaultReadConfig},
 	} {
 		units = append(units, ccu.Units()...)
 	}
@@ -161,23 +166,9 @@ func Apply(cfg config.CloudConfig, env *Environment) error {
 		}
 	}
 
-	if env.NetconfType() != "" {
-		var interfaces []network.InterfaceGenerator
-		var err error
-		switch env.NetconfType() {
-		case "debian":
-			interfaces, err = network.ProcessDebianNetconf(cfg.NetworkConfig)
-		case "digitalocean":
-			interfaces, err = network.ProcessDigitalOceanNetconf(cfg.NetworkConfig)
-		default:
-			err = fmt.Errorf("Unsupported network config format %q", env.NetconfType())
-		}
-		if err != nil {
-			return err
-		}
-
-		units = append(units, createNetworkingUnits(interfaces)...)
-		if err := system.RestartNetwork(interfaces); err != nil {
+	if len(ifaces) > 0 {
+		units = append(units, createNetworkingUnits(ifaces)...)
+		if err := system.RestartNetwork(ifaces); err != nil {
 			return err
 		}
 	}

initialize/config_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

initialize/env.go

@@ -1,28 +1,28 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 
 import (
+	"net"
 	"os"
 	"path"
 	"regexp"
 	"strings"
 
 	"github.com/coreos/coreos-cloudinit/config"
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -32,28 +32,25 @@ type Environment struct {
 	root          string
 	configRoot    string
 	workspace     string
-	netconfType   string
 	sshKeyName    string
 	substitutions map[string]string
 }
 
 // TODO(jonboulle): this is getting unwieldy, should be able to simplify the interface somehow
-func NewEnvironment(root, configRoot, workspace, netconfType, sshKeyName string, substitutions map[string]string) *Environment {
-	if substitutions == nil {
-		substitutions = make(map[string]string)
-	}
-	// If certain values are not in the supplied substitution, fall back to retrieving them from the environment
-	for k, v := range map[string]string{
-		"$public_ipv4":  os.Getenv("COREOS_PUBLIC_IPV4"),
-		"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
-		"$public_ipv6":  os.Getenv("COREOS_PUBLIC_IPV6"),
-		"$private_ipv6": os.Getenv("COREOS_PRIVATE_IPV6"),
-	} {
-		if _, ok := substitutions[k]; !ok {
-			substitutions[k] = v
+func NewEnvironment(root, configRoot, workspace, sshKeyName string, metadata datasource.Metadata) *Environment {
+	firstNonNull := func(ip net.IP, env string) string {
+		if ip == nil {
+			return env
 		}
+		return ip.String()
 	}
-	return &Environment{root, configRoot, workspace, netconfType, sshKeyName, substitutions}
+	substitutions := map[string]string{
+		"$public_ipv4":  firstNonNull(metadata.PublicIPv4, os.Getenv("COREOS_PUBLIC_IPV4")),
+		"$private_ipv4": firstNonNull(metadata.PrivateIPv4, os.Getenv("COREOS_PRIVATE_IPV4")),
+		"$public_ipv6":  firstNonNull(metadata.PublicIPv6, os.Getenv("COREOS_PUBLIC_IPV6")),
+		"$private_ipv6": firstNonNull(metadata.PrivateIPv6, os.Getenv("COREOS_PRIVATE_IPV6")),
+	}
+	return &Environment{root, configRoot, workspace, sshKeyName, substitutions}
 }
 
 func (e *Environment) Workspace() string {
@@ -68,10 +65,6 @@ func (e *Environment) ConfigRoot() string {
 	return e.configRoot
 }
 
-func (e *Environment) NetconfType() string {
-	return e.netconfType
-}
-
 func (e *Environment) SSHKeyName() string {
 	return e.sshKeyName
 }

initialize/env_test.go

@@ -1,27 +1,27 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 
 import (
 	"io/ioutil"
+	"net"
 	"os"
 	"path"
 	"testing"
 
+	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -31,18 +31,18 @@ func TestEnvironmentApply(t *testing.T) {
 	os.Setenv("COREOS_PUBLIC_IPV6", "1234::")
 	os.Setenv("COREOS_PRIVATE_IPV6", "5678::")
 	for _, tt := range []struct {
-		subs  map[string]string
-		input string
-		out   string
+		metadata datasource.Metadata
+		input    string
+		out      string
 	}{
 		{
 			// Substituting both values directly should always take precedence
 			// over environment variables
-			map[string]string{
-				"$public_ipv4":  "192.0.2.3",
-				"$private_ipv4": "192.0.2.203",
-				"$public_ipv6":  "fe00:1234::",
-				"$private_ipv6": "fe00:5678::",
+			datasource.Metadata{
+				PublicIPv4:  net.ParseIP("192.0.2.3"),
+				PrivateIPv4: net.ParseIP("192.0.2.203"),
+				PublicIPv6:  net.ParseIP("fe00:1234::"),
+				PrivateIPv6: net.ParseIP("fe00:5678::"),
 			},
 			`[Service]
 ExecStart=/usr/bin/echo "$public_ipv4 $public_ipv6"
@@ -55,25 +55,29 @@ ExecStop=/usr/bin/echo $unknown`,
 		},
 		{
 			// Substituting one value directly while falling back with the other
-			map[string]string{"$private_ipv4": "127.0.0.1"},
+			datasource.Metadata{
+				PrivateIPv4: net.ParseIP("127.0.0.1"),
+			},
 			"$private_ipv4\n$public_ipv4",
 			"127.0.0.1\n1.2.3.4",
 		},
 		{
 			// Falling back to environment variables for both values
-			map[string]string{"foo": "bar"},
+			datasource.Metadata{},
 			"$private_ipv4\n$public_ipv4",
 			"5.6.7.8\n1.2.3.4",
 		},
 		{
 			// No substitutions
-			nil,
+			datasource.Metadata{},
 			"$private_ipv4\nfoobar",
 			"5.6.7.8\nfoobar",
 		},
 		{
 			// Escaping substitutions
-			map[string]string{"$private_ipv4": "127.0.0.1"},
+			datasource.Metadata{
+				PrivateIPv4: net.ParseIP("127.0.0.1"),
+			},
 			`\$private_ipv4
 $private_ipv4
 addr: \$private_ipv4
@@ -85,13 +89,13 @@ addr: $private_ipv4
 		},
 		{
 			// No substitutions with escaping
-			nil,
+			datasource.Metadata{},
 			"\\$test\n$test",
 			"\\$test\n$test",
 		},
 	} {
 
-		env := NewEnvironment("./", "./", "./", "", "", tt.subs)
+		env := NewEnvironment("./", "./", "./", "", tt.metadata)
 		got := env.Apply(tt.input)
 		if got != tt.out {
 			t.Fatalf("Environment incorrectly applied.\ngot:\n%s\nwant:\n%s", got, tt.out)
@@ -100,11 +104,11 @@ addr: $private_ipv4
 }
 
 func TestEnvironmentFile(t *testing.T) {
-	subs := map[string]string{
-		"$public_ipv4":  "1.2.3.4",
-		"$private_ipv4": "5.6.7.8",
-		"$public_ipv6":  "1234::",
-		"$private_ipv6": "5678::",
+	metadata := datasource.Metadata{
+		PublicIPv4:  net.ParseIP("1.2.3.4"),
+		PrivateIPv4: net.ParseIP("5.6.7.8"),
+		PublicIPv6:  net.ParseIP("1234::"),
+		PrivateIPv6: net.ParseIP("5678::"),
 	}
 	expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PRIVATE_IPV6=5678::\nCOREOS_PUBLIC_IPV4=1.2.3.4\nCOREOS_PUBLIC_IPV6=1234::\n"
 
@@ -114,7 +118,7 @@ func TestEnvironmentFile(t *testing.T) {
 	}
 	defer os.RemoveAll(dir)
 
-	env := NewEnvironment("./", "./", "./", "", "", subs)
+	env := NewEnvironment("./", "./", "./", "", metadata)
 	ef := env.DefaultEnvironmentFile()
 	err = system.WriteEnvFile(ef, dir)
 	if err != nil {
@@ -133,14 +137,10 @@ func TestEnvironmentFile(t *testing.T) {
 }
 
 func TestEnvironmentFileNil(t *testing.T) {
-	subs := map[string]string{
-		"$public_ipv4":  "",
-		"$private_ipv4": "",
-		"$public_ipv6":  "",
-		"$private_ipv6": "",
-	}
+	os.Clearenv()
+	metadata := datasource.Metadata{}
 
-	env := NewEnvironment("./", "./", "./", "", "", subs)
+	env := NewEnvironment("./", "./", "./", "", metadata)
 	ef := env.DefaultEnvironmentFile()
 	if ef != nil {
 		t.Fatalf("Environment file not nil: %v", ef)

initialize/github.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

initialize/meta_data.go

@@ -1,90 +0,0 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
-
-package initialize
-
-import (
-	"encoding/json"
-	"sort"
-
-	"github.com/coreos/coreos-cloudinit/config"
-)
-
-// ParseMetaData parses a JSON blob in the OpenStack metadata service format,
-// and converts it to a partially hydrated CloudConfig.
-func ParseMetaData(contents string) (*config.CloudConfig, error) {
-	if len(contents) == 0 {
-		return nil, nil
-	}
-	var metadata struct {
-		SSHAuthorizedKeyMap map[string]string `json:"public_keys"`
-		Hostname            string            `json:"hostname"`
-		NetworkConfig       struct {
-			ContentPath string `json:"content_path"`
-		} `json:"network_config"`
-	}
-	if err := json.Unmarshal([]byte(contents), &metadata); err != nil {
-		return nil, err
-	}
-
-	var cfg config.CloudConfig
-	if len(metadata.SSHAuthorizedKeyMap) > 0 {
-		cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap))
-		for _, name := range sortedKeys(metadata.SSHAuthorizedKeyMap) {
-			cfg.SSHAuthorizedKeys = append(cfg.SSHAuthorizedKeys, metadata.SSHAuthorizedKeyMap[name])
-		}
-	}
-	cfg.Hostname = metadata.Hostname
-	cfg.NetworkConfigPath = metadata.NetworkConfig.ContentPath
-	return &cfg, nil
-}
-
-// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service
-// format and returns a substitution map possibly containing private_ipv4,
-// public_ipv4, private_ipv6, and public_ipv6 addresses.
-func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) {
-	var ips struct {
-		PublicIPv4  string `json:"public-ipv4"`
-		PrivateIPv4 string `json:"local-ipv4"`
-		PublicIPv6  string `json:"public-ipv6"`
-		PrivateIPv6 string `json:"local-ipv6"`
-	}
-	if err := json.Unmarshal(contents, &ips); err != nil {
-		return nil, err
-	}
-	m := make(map[string]string)
-	if ips.PrivateIPv4 != "" {
-		m["$private_ipv4"] = ips.PrivateIPv4
-	}
-	if ips.PublicIPv4 != "" {
-		m["$public_ipv4"] = ips.PublicIPv4
-	}
-	if ips.PrivateIPv6 != "" {
-		m["$private_ipv6"] = ips.PrivateIPv6
-	}
-	if ips.PublicIPv6 != "" {
-		m["$public_ipv6"] = ips.PublicIPv6
-	}
-	return m, nil
-}
-
-func sortedKeys(m map[string]string) (keys []string) {
-	for key := range m {
-		keys = append(keys, key)
-	}
-	sort.Strings(keys)
-	return
-}

initialize/meta_data_test.go

@@ -1,89 +0,0 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
-
-package initialize
-
-import (
-	"reflect"
-	"testing"
-
-	"github.com/coreos/coreos-cloudinit/config"
-)
-
-func TestParseMetadata(t *testing.T) {
-	for i, tt := range []struct {
-		in   string
-		want *config.CloudConfig
-		err  bool
-	}{
-		{"", nil, false},
-		{`garbage, invalid json`, nil, true},
-		{`{"foo": "bar"}`, &config.CloudConfig{}, false},
-		{`{"network_config": {"content_path": "asdf"}}`, &config.CloudConfig{NetworkConfigPath: "asdf"}, false},
-		{`{"hostname": "turkleton"}`, &config.CloudConfig{Hostname: "turkleton"}, false},
-		{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &config.CloudConfig{SSHAuthorizedKeys: []string{"alice", "jill"}}, false},
-		{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &config.CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
-	} {
-		got, err := ParseMetaData(tt.in)
-		if tt.err != (err != nil) {
-			t.Errorf("case #%d: bad error state: got %t, want %t (err=%v)", i, (err != nil), tt.err, err)
-		}
-		if got == nil {
-			if tt.want != nil {
-				t.Errorf("case #%d: unexpected nil output", i)
-			}
-		} else if tt.want == nil {
-			t.Errorf("case #%d: unexpected non-nil output", i)
-		} else {
-			if !reflect.DeepEqual(*got, *tt.want) {
-				t.Errorf("case #%d: bad output:\ngot\n%v\nwant\n%v", i, *got, *tt.want)
-			}
-		}
-	}
-
-}
-
-func TestExtractIPsFromMetadata(t *testing.T) {
-	for i, tt := range []struct {
-		in  []byte
-		err bool
-		out map[string]string
-	}{
-		{
-			[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4", "public-ipv6": "1234::", "local-ipv6": "5678::"}`),
-			false,
-			map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4", "$public_ipv6": "1234::", "$private_ipv6": "5678::"},
-		},
-		{
-			[]byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`),
-			false,
-			map[string]string{"$private_ipv4": "127.0.0.1"},
-		},
-		{
-			[]byte(`garbage`),
-			true,
-			nil,
-		},
-	} {
-		got, err := ExtractIPsFromMetadata(tt.in)
-		if (err != nil) != tt.err {
-			t.Errorf("bad error state (got %t, want %t)", err != nil, tt.err)
-		}
-		if !reflect.DeepEqual(got, tt.out) {
-			t.Errorf("case %d: got %s, want %s", i, got, tt.out)
-		}
-	}
-}

initialize/ssh_keys.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

initialize/ssh_keys_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

initialize/user_data.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

initialize/user_data_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

initialize/workspace.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package initialize
 

network/debian.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 
@@ -21,9 +19,9 @@ import (
 	"strings"
 )
 
-func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) {
+func ProcessDebianNetconf(config []byte) ([]InterfaceGenerator, error) {
 	log.Println("Processing Debian network config")
-	lines := formatConfig(config)
+	lines := formatConfig(string(config))
 	stanzas, err := parseStanzas(lines)
 	if err != nil {
 		return nil, err

network/debian_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 
@@ -46,7 +44,7 @@ func TestProcessDebianNetconf(t *testing.T) {
 		{"auto eth1\nauto eth2", false, 0},
 		{"iface eth1 inet manual", false, 1},
 	} {
-		interfaces, err := ProcessDebianNetconf(tt.in)
+		interfaces, err := ProcessDebianNetconf([]byte(tt.in))
 		failed := err != nil
 		if tt.fail != failed {
 			t.Fatalf("bad failure state for %q: got %t, want %t", tt.in, failed, tt.fail)

network/digitalocean.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 
@@ -25,14 +23,14 @@ import (
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
 )
 
-func ProcessDigitalOceanNetconf(config string) ([]InterfaceGenerator, error) {
+func ProcessDigitalOceanNetconf(config []byte) ([]InterfaceGenerator, error) {
 	log.Println("Processing DigitalOcean network config")
-	if config == "" {
+	if len(config) == 0 {
 		return nil, nil
 	}
 
 	var cfg digitalocean.Metadata
-	if err := json.Unmarshal([]byte(config), &cfg); err != nil {
+	if err := json.Unmarshal(config, &cfg); err != nil {
 		return nil, err
 	}
 

network/digitalocean_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 
@@ -380,7 +378,7 @@ func TestProcessDigitalOceanNetconf(t *testing.T) {
 			ifaces: []InterfaceGenerator{},
 		},
 	} {
-		ifaces, err := ProcessDigitalOceanNetconf(tt.cfg)
+		ifaces, err := ProcessDigitalOceanNetconf([]byte(tt.cfg))
 		if !errorsEqual(tt.err, err) {
 			t.Fatalf("bad error (%q): want %q, got %q", tt.cfg, tt.err, err)
 		}

network/interface.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 

network/interface_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 

network/stanza.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 

network/stanza_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package network
 

pkg/http_client.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package pkg
 

pkg/http_client_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package pkg
 

system/env.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package system
 
@@ -23,22 +21,32 @@ import (
 	"github.com/coreos/coreos-cloudinit/config"
 )
 
-// dropinContents generates the contents for a drop-in unit given the config.
+// serviceContents generates the contents for a drop-in unit given the config.
 // The argument must be a struct from the 'config' package.
 func serviceContents(e interface{}) string {
+	vars := getEnvVars(e)
+	if len(vars) == 0 {
+		return ""
+	}
+
+	out := "[Service]\n"
+	for _, v := range vars {
+		out += fmt.Sprintf("Environment=\"%s\"\n", v)
+	}
+	return out
+}
+
+func getEnvVars(e interface{}) []string {
 	et := reflect.TypeOf(e)
 	ev := reflect.ValueOf(e)
 
-	var out string
+	vars := []string{}
 	for i := 0; i < et.NumField(); i++ {
 		if val := ev.Field(i).Interface(); !config.IsZero(val) {
 			key := et.Field(i).Tag.Get("env")
-			out += fmt.Sprintf("Environment=\"%s=%v\"\n", key, val)
+			vars = append(vars, fmt.Sprintf("%s=%v", key, val))
 		}
 	}
 
-	if out == "" {
-		return ""
-	}
-	return "[Service]\n" + out
+	return vars
 }

system/env_file.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package system
 

system/env_file_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package system
 

system/env_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package system
 
 import (

system/etc_hosts.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package system
 

system/etc_hosts_test.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package system
 

system/etcd.go

@@ -1,18 +1,16 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
+// Copyright 2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
 
 package system