coreos-cloudinit: bump to 0.10.6

This reverts commit cdfc94f4e9, reversing
changes made to 2051cd3e1c.

Conflicts:
	initialize/config.go
	system/etcd.go
	system/etcd_test.go
	system/fleet.go
	system/fleet_test.go
	system/update.go
	system/update_test.go
	test

.travis.yml

@@ -1,12 +1,10 @@
 language: go
-sudo: false
+go:
-matrix:
+  - 1.3
-  include:
+  - 1.2
-    - go: 1.4
+
-      install:
+install:
-        - go get golang.org/x/tools/cmd/cover
+ - go get code.google.com/p/go.tools/cmd/cover
-        - go get golang.org/x/tools/cmd/vet
-    - go: 1.5
 
 script:
  - ./test

Documentation/cloud-config-deprecated.md

@@ -1,38 +0,0 @@
-# Deprecated Cloud-Config Features
-
-## Retrieving SSH Authorized Keys
-
-### From a GitHub User
-
-Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server.
-
-```yaml
-#cloud-config
-
-users:
-  - name: elroy
-    coreos-ssh-import-github: elroy
-```
-
-### From an HTTP Endpoint
-
-We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
-For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token:
-
-```yaml
-#cloud-config
-
-users:
-  - name: elroy
-    coreos-ssh-import-url: https://github-enterprise.example.com/api/v3/users/elroy/keys?access_token=<TOKEN>
-```
-
-You can also specify any URL whose response matches the JSON format for public keys:
-
-```yaml
-#cloud-config
-
-users:
-  - name: elroy
-    coreos-ssh-import-url: https://example.com/public-keys
-```

Documentation/cloud-config-locations.md

@@ -1,26 +0,0 @@
-# Cloud-Config Locations
-
-On every boot, coreos-cloudinit looks for a config file to configure your host. Here is a list of locations which are used by the Cloud-Config utility, depending on your CoreOS platform:
-
-Location | Description
---- | --- | ---
-|`/media/configvirtfs/openstack/latest/user_data`|`/media/configvirtfs` mount point with [config-2](/os/docs/latest/config-drive.html#contents-and-format) label. It should contain a `openstack/latest/user_data` relative path. Usually used by cloud providers or in VM installations.|
-|`/media/configdrive/openstack/latest/user_data`|FAT or ISO9660 filesystem with [config-2](/os/docs/latest/config-drive.html#qemu-virtfs) label and `/media/configdrive/` mount point. It should also contain a `openstack/latest/user_data` relative path. Usually used in installations which are configured by USB Flash sticks or CDROM media.|
-|Kernel command line: `cloud-config-url=http://example.com/user_data`.| You can find this string using this command `cat /proc/cmdline`. Usually used in [PXE](/os/docs/latest/booting-with-pxe.html) or [iPXE](/os/docs/latest/booting-with-ipxe.html) boots.|
-|`/var/lib/coreos-install/user_data`| When you install CoreOS manually using the [coreos-install](/os/docs/latest/installing-to-disk.html) tool. Usually used in bare metal installations.|
-|`/usr/share/oem/cloud-config.yml`| Path for OEM images.|
-|`/var/lib/coreos-vagrant/vagrantfile-user-data`| Vagrant OEM scripts automatically store Cloud-Config into this path. |
-|`/var/lib/waagent/CustomData`| Azure platform uses OEM path for first Cloud-Config initialization and then `/var/lib/waagent/CustomData` to apply your settings.|
-|`http://169.254.169.254/metadata/v1/user-data` `http://169.254.169.254/2009-04-04/user-data` `https://metadata.packet.net/userdata`|DigitalOcean, EC2 and Packet cloud providers correspondingly use these URLs to download Cloud-Config.|
-|`/usr/share/oem/bin/vmtoolsd --cmd "info-get guestinfo.coreos.config.data"`|Cloud-Config provided by [VMware Guestinfo][VMware Guestinfo]|
-|`/usr/share/oem/bin/vmtoolsd --cmd "info-get guestinfo.coreos.config.url"`|Cloud-Config URL provided by [VMware Guestinfo][VMware Guestinfo]|
-
-[VMware Guestinfo]: vmware-guestinfo.md
-
-You can also run the `coreos-cloudinit` tool manually and provide a path to your custom Cloud-Config file:
-
-```sh
-sudo coreos-cloudinit --from-file=/home/core/cloud-config.yaml
-```
-
-This command will apply your custom cloud-config.

Documentation/cloud-config-oem.md

@@ -17,11 +17,11 @@ For example, the following cloud-config document...
 #cloud-config
 coreos:
   oem:
-    id: "rackspace"
+    id: rackspace
-    name: "Rackspace Cloud Servers"
+    name: Rackspace Cloud Servers
-    version-id: "168.0.0"
+    version-id: 168.0.0
-    home-url: "https://www.rackspace.com/cloud/servers/"
+    home-url: https://www.rackspace.com/cloud/servers/
-    bug-report-url: "https://github.com/coreos/coreos-overlay"
+    bug-report-url: https://github.com/coreos/coreos-overlay
 ```
 
 ...would be rendered to the following `/etc/oem-release`:

Documentation/cloud-config.md

@@ -1,16 +1,6 @@
 # Using Cloud-Config
 
-CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime.
+CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. Your cloud-config is processed during each boot.
-
-Your cloud-config is processed during each boot. Invalid cloud-config won't be processed but will be logged in the journal. You can validate your cloud-config with the [CoreOS validator]({{site.url}}/validate) or by running `coreos-cloudinit -validate`.
-
-In addition to `coreos-cloudinit -validate` command and https://coreos.com/validate/ online service you can debug `coreos-cloudinit` system output through the `journalctl` tool:
-
-```sh
-journalctl _EXE=/usr/bin/coreos-cloudinit
-```
-
-It will show `coreos-cloudinit` run output which was triggered by system boot.
 
 ## Configuration File
 
@@ -26,7 +16,7 @@ We've designed our implementation to allow the same cloud-config file to work ac
 
 The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values.
 
-A cloud-config file must contain a header: either `#cloud-config` for processing as cloud-config (suggested) or `#!` for processing as a shell script (advanced). If cloud-config has #cloud-config header, it should followed by an associative array which has zero or more of the following keys:
+A cloud-config file should contain `#cloud-config`, followed by an associative array which has zero or more of the following keys:
 
 - `coreos`
 - `ssh_authorized_keys`
@@ -37,8 +27,6 @@ A cloud-config file must contain a header: either `#cloud-config` for processing
 
 The expected values for these keys are defined in the rest of this document.
 
-If cloud-config header starts on `#!` then coreos-cloudinit will recognize it as shell script which is interpreted by bash and run it as transient systemd service.
-
 [yaml]: https://en.wikipedia.org/wiki/YAML
 
 ### Providing Cloud-Config with Config-Drive
@@ -49,7 +37,7 @@ CoreOS tries to conform to each platform's native method to provide user data. E
 
 ### coreos
 
-#### etcd (deprecated. see etcd2)
+#### etcd
 
 The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
 If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
@@ -58,16 +46,16 @@ If the platform environment supports the templating feature of coreos-cloudinit
 #cloud-config
 
 coreos:
-  etcd:
+    etcd:
-    name: "node001"
+        name: node001
-    # generate a new token for each unique cluster from https://discovery.etcd.io/new
+	# generate a new token for each unique cluster from https://discovery.etcd.io/new
-    discovery: "https://discovery.etcd.io/<token>"
+        discovery: https://discovery.etcd.io/<token>
-    # multi-region and multi-cloud deployments need to use $public_ipv4
+	# multi-region and multi-cloud deployments need to use $public_ipv4
-    addr: "$public_ipv4:4001"
+        addr: $public_ipv4:4001
-    peer-addr: "$private_ipv4:7001"
+        peer-addr: $private_ipv4:7001
 ```
 
-...will generate a systemd unit drop-in for etcd.service with the following contents:
+...will generate a systemd unit drop-in like this:
 
 ```yaml
 [Service]
@@ -78,62 +66,23 @@ Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
 ```
 
 For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
+Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
 
 _Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
 
-[etcd-config]: https://github.com/coreos/etcd/blob/release-0.4/Documentation/configuration.md
+[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
-
-#### etcd2
-
-The `coreos.etcd2.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
-If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. When generating a [discovery token](https://discovery.etcd.io/new?size=3), set the `size` parameter, since etcd uses this to determine if all members have joined the cluster. After the cluster is bootstrapped, it can grow or shrink from this configured size.
-
-For example, the following cloud-config document...
-
-```yaml
-#cloud-config
-
-coreos:
-  etcd2:
-    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
-    discovery: "https://discovery.etcd.io/<token>"
-    # multi-region and multi-cloud deployments need to use $public_ipv4
-    advertise-client-urls: "http://$public_ipv4:2379"
-    initial-advertise-peer-urls: "http://$private_ipv4:2380"
-    # listen on both the official ports and the legacy ports
-    # legacy ports can be omitted if your application doesn't depend on them
-    listen-client-urls: "http://0.0.0.0:2379,http://0.0.0.0:4001"
-    listen-peer-urls: "http://$private_ipv4:2380,http://$private_ipv4:7001"
-```
-
-...will generate a systemd unit drop-in for etcd2.service with the following contents:
-
-```yaml
-[Service]
-Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
-Environment="ETCD_ADVERTISE_CLIENT_URLS=http://203.0.113.29:2379"
-Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=http://192.0.2.13:2380"
-Environment="ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001"
-Environment="ETCD_LISTEN_PEER_URLS=http://192.0.2.13:2380,http://192.0.2.13:7001"
-```
-
-For more information about the available configuration parameters, see the [etcd2 documentation][etcd2-config].
-
-_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
-
-[etcd2-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
 
 #### fleet
 
-The `coreos.fleet.*` parameters work very similarly to `coreos.etcd2.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
+The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
 
 ```yaml
 #cloud-config
 
 coreos:
-  fleet:
+    fleet:
-      public-ip: "$public_ipv4"
+        public-ip: $public_ipv4
-      metadata: "region=us-west"
+        metadata: region=us-west
 ```
 
 ...will generate a systemd unit drop-in like this:
@@ -144,92 +93,10 @@ Environment="FLEET_PUBLIC_IP=203.0.113.29"
 Environment="FLEET_METADATA=region=us-west"
 ```
 
-List of fleet configuration parameters:
-
-- **agent_ttl**: An Agent will be considered dead if it exceeds this amount of time to communicate with the Registry
-- **engine_reconcile_interval**: Interval in seconds at which the engine should reconcile the cluster schedule in etcd
-- **etcd_cafile**: Path to CA file used for TLS communication with etcd
-- **etcd_certfile**: Provide TLS configuration when SSL certificate authentication is enabled in etcd endpoints
-- **etcd_keyfile**: Path to private key file used for TLS communication with etcd
-- **etcd_key_prefix**: etcd prefix path to be used for fleet keys
-- **etcd_request_timeout**: Amount of time in seconds to allow a single etcd request before considering it failed
-- **etcd_servers**: Comma separated list of etcd endpoints
-- **metadata**: Comma separated key/value pairs that are published with the local to the fleet registry
-- **public_ip**: IP accessible by other nodes for inter-host communication
-- **verbosity**: Enable debug logging by setting this to an integer value greater than zero
-
 For more information on fleet configuration, see the [fleet documentation][fleet-config].
 
 [fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md#configuration
 
-#### flannel
-
-The `coreos.flannel.*` parameters also work very similarly to `coreos.etcd2.*`
-and `coreos.fleet.*`. They can be used to set environment variables for
-flanneld. For example, the following cloud-config...
-
-```yaml
-#cloud-config
-
-coreos:
-  flannel:
-      etcd_prefix: "/coreos.com/network2"
-```
-
-...will generate a systemd unit drop-in like so:
-
-```
-[Service]
-Environment="FLANNELD_ETCD_PREFIX=/coreos.com/network2"
-```
-
-List of flannel configuration parameters:
-
-- **etcd_endpoints**: Comma separated list of etcd endpoints
-- **etcd_cafile**: Path to CA file used for TLS communication with etcd
-- **etcd_certfile**: Path to certificate file used for TLS communication with etcd
-- **etcd_keyfile**: Path to private key file used for TLS communication with etcd
-- **etcd_prefix**: etcd prefix path to be used for flannel keys
-- **ip_masq**: Install IP masquerade rules for traffic outside of flannel subnet
-- **subnet_file**: Path to flannel subnet file to write out
-- **interface**: Interface (name or IP) that should be used for inter-host communication
-- **public_ip**: IP accessible by other nodes for inter-host communication
-
-For more information on flannel configuration, see the [flannel documentation][flannel-readme].
-
-[flannel-readme]: https://github.com/coreos/flannel/blob/master/README.md
-
-#### locksmith
-
-The `coreos.locksmith.*` parameters can be used to set environment variables
-for locksmith. For example, the following cloud-config...
-
-```yaml
-#cloud-config
-
-coreos:
-  locksmith:
-      endpoint: "http://example.com:2379"
-```
-
-...will generate a systemd unit drop-in like so:
-
-```
-[Service]
-Environment="LOCKSMITHD_ENDPOINT=http://example.com:2379"
-```
-
-List of locksmith configuration parameters:
-
-- **endpoint**: Comma separated list of etcd endpoints
-- **etcd_cafile**: Path to CA file used for TLS communication with etcd
-- **etcd_certfile**: Path to certificate file used for TLS communication with etcd
-- **etcd_keyfile**: Path to private key file used for TLS communication with etcd
-
-For the complete list of locksmith configuration parameters, see the [locksmith documentation][locksmith-readme].
-
-[locksmith-readme]: https://github.com/coreos/locksmith/blob/master/README.md
-
 #### update
 
 The `coreos.update.*` parameters manipulate settings related to how CoreOS instances are updated.
@@ -242,12 +109,9 @@ The `reboot-strategy` parameter also affects the behaviour of [locksmith](https:
   - _etcd-lock_: Reboot after first taking a distributed lock in etcd, this guarantees that only one host will reboot concurrently and that the cluster will remain available during the update.
   - _best-effort_ - If etcd is running, "etcd-lock", otherwise simply "reboot".
   - _off_ - Disable rebooting after updates are applied (not recommended).
-- **server**: The location of the [CoreUpdate][coreupdate] server which will be queried for updates. Also known as the [omaha][omaha-docs] server endpoint.
+- **server**: is the omaha endpoint URL which will be queried for updates.
 - **group**:  signifies the channel which should be used for automatic updates.  This value defaults to the version of the image initially downloaded. (one of "master", "alpha", "beta", "stable")
 
-[coreupdate]: https://coreos.com/products/coreupdate
-[omaha-docs]: https://coreos.com/docs/coreupdate/custom-apps/coreupdate-protocol/
-
 *Note: cloudinit will only manipulate the locksmith unit file in the systemd runtime directory (`/run/systemd/system/locksmithd.service`). If any manual modifications are made to an overriding unit configuration file (e.g. `/etc/systemd/system/locksmithd.service`), cloudinit will no longer be able to control the locksmith service unit.*
 
 ##### Example
@@ -256,7 +120,7 @@ The `reboot-strategy` parameter also affects the behaviour of [locksmith](https:
 #cloud-config
 coreos:
   update:
-    reboot-strategy: "etcd-lock"
+    reboot-strategy: etcd-lock
 ```
 
 #### units
@@ -271,10 +135,6 @@ Each item is an object with the following fields:
 - **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
 - **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. The default behavior is to not execute any commands.
 - **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. The default value is false.
-- **drop-ins**: A list of unit drop-ins with the following fields:
-  - **name**: String representing unit's name. Required.
-  - **content**: Plaintext string representing entire file. Required.
-
 
 **NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
 
@@ -286,47 +146,32 @@ Write a unit to disk, automatically starting it.
 #cloud-config
 
 coreos:
-  units:
+    units:
-    - name: "docker-redis.service"
+      - name: docker-redis.service
-      command: "start"
+        command: start
-      content: |
+        content: |
-        [Unit]
+          [Unit]
-        Description=Redis container
+          Description=Redis container
-        Author=Me
+          Author=Me
-        After=docker.service
+          After=docker.service
 
-        [Service]
+          [Service]
-        Restart=always
+          Restart=always
-        ExecStart=/usr/bin/docker start -a redis_server
+          ExecStart=/usr/bin/docker start -a redis_server
-        ExecStop=/usr/bin/docker stop -t 2 redis_server
+          ExecStop=/usr/bin/docker stop -t 2 redis_server
 ```
 
-Add the DOCKER_OPTS environment variable to docker.service.
+Start the built-in `etcd` and `fleet` services:
 
 ```yaml
 #cloud-config
 
 coreos:
-  units:
+    units:
-    - name: "docker.service"
+      - name: etcd.service
-      drop-ins:
+        command: start
-        - name: "50-insecure-registry.conf"
+      - name: fleet.service
-          content: |
+        command: start
-            [Service]
-            Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
-```
-
-Start the built-in `etcd2` and `fleet` services:
-
-```yaml
-#cloud-config
-
-coreos:
-  units:
-    - name: "etcd2.service"
-      command: "start"
-    - name: "fleet.service"
-      command: "start"
 ```
 
 ### ssh_authorized_keys
@@ -340,7 +185,7 @@ Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`
 #cloud-config
 
 ssh_authorized_keys:
-  - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
+  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
 ```
 
 ### hostname
@@ -351,7 +196,7 @@ This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.exam
 ```yaml
 #cloud-config
 
-hostname: "coreos1"
+hostname: coreos1
 ```
 
 ### users
@@ -368,12 +213,10 @@ All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the use
 - **groups**: Add user to these additional groups
 - **no-user-group**: Boolean. Skip default group creation.
 - **ssh-authorized-keys**: List of public SSH keys to authorize for this user
-- **coreos-ssh-import-github** [DEPRECATED]: Authorize SSH keys from GitHub user
+- **coreos-ssh-import-github**: Authorize SSH keys from Github user
-- **coreos-ssh-import-github-users** [DEPRECATED]: Authorize SSH keys from a list of GitHub users
+- **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint.
-- **coreos-ssh-import-url** [DEPRECATED]: Authorize SSH keys imported from a url endpoint.
 - **system**: Create the user as a system user. No home directory will be created.
 - **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
-- **shell**: User's login shell.
 
 The following fields are not yet implemented:
 
@@ -387,13 +230,13 @@ The following fields are not yet implemented:
 #cloud-config
 
 users:
-  - name: "elroy"
+  - name: elroy
-    passwd: "$6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm..."
+    passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
     groups:
-      - "sudo"
+      - sudo
-      - "docker"
+      - docker
     ssh-authorized-keys:
-      - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
+      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
 ```
 
 #### Generating a password hash
@@ -416,6 +259,43 @@ perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
 
 Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed.  On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms. 
 
+#### Retrieving SSH Authorized Keys
+
+##### From a GitHub User
+
+Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server.
+
+```yaml
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-github: elroy
+```
+
+##### From an HTTP Endpoint
+
+We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
+For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token:
+
+```yaml
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://github-enterprise.example.com/api/v3/users/elroy/keys?access_token=<TOKEN>
+```
+
+You can also specify any URL whose response matches the JSON format for public keys:
+
+```yaml
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://example.com/public-keys
+```
+
 ### write_files
 
 The `write_files` directive defines a set of files to create on the local filesystem.
@@ -423,45 +303,25 @@ Each item in the list may have the following keys:
 
 - **path**: Absolute location on disk where contents should be written
 - **content**: Data to write at the provided `path`
-- **permissions**: Integer representing file permissions, typically in octal notation (i.e. 0644)
+- **permissions**: String representing file permissions in octal notation (i.e. '0644')
 - **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
-- **encoding**: Optional. The encoding of the data in content. If not specified this defaults to the yaml document encoding (usually utf-8). Supported encoding types are:
-    - **b64, base64**: Base64 encoded content
-    - **gz, gzip**: gzip encoded content, for use with the !!binary tag
-    - **gz+b64, gz+base64, gzip+b64, gzip+base64**: Base64 encoded gzip content
 
+Explicitly not implemented is the **encoding** attribute.
+The **content** field must represent exactly what should be written to disk.
 
 ```yaml
 #cloud-config
 write_files:
-  - path: "/etc/resolv.conf"
+  - path: /etc/resolv.conf
-    permissions: "0644"
+    permissions: 0644
-    owner: "root"
+    owner: root
     content: |
       nameserver 8.8.8.8
-  - path: "/etc/motd"
+  - path: /etc/motd
-    permissions: "0644"
+    permissions: 0644
-    owner: "root"
+    owner: root
     content: |
       Good news, everyone!
-  - path: "/tmp/like_this"
-    permissions: "0644"
-    owner: "root"
-    encoding: "gzip"
-    content: !!binary |
-      H4sIAKgdh1QAAwtITM5WyK1USMqvUCjPLMlQSMssS1VIya9KzVPIySwszS9SyCpNLwYARQFQ5CcAAAA=
-  - path: "/tmp/or_like_this"
-    permissions: "0644"
-    owner: "root"
-    encoding: "gzip+base64"
-    content: |
-      H4sIAKgdh1QAAwtITM5WyK1USMqvUCjPLMlQSMssS1VIya9KzVPIySwszS9SyCpNLwYARQFQ5CcAAAA=
-  - path: "/tmp/todolist"
-    permissions: "0644"
-    owner: "root"
-    encoding: "base64"
-    content: |
-      UGFjayBteSBib3ggd2l0aCBmaXZlIGRvemVuIGxpcXVvciBqdWdz
 ```
 
 ### manage_etc_hosts
@@ -474,5 +334,5 @@ infrastructure in place to resolve its own hostname, for example, when using Vag
 ```yaml
 #cloud-config
 
-manage_etc_hosts: "localhost"
+manage_etc_hosts: localhost
 ```

Documentation/config-drive.md

@@ -4,7 +4,7 @@ CoreOS supports providing configuration data via [config drive][config-drive]
 disk images. Currently only providing a single script or cloud config file is
 supported.
 
-[config-drive]: http://docs.openstack.org/user-guide/cli_config_drive.html
+[config-drive]: http://docs.openstack.org/user-guide/content/enable_config_drive.html#config_drive_contents
 
 ## Contents and Format
 
@@ -21,12 +21,6 @@ mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
 rm -r /tmp/new-drive
 ```
 
-If on OS X, replace the `mkisofs` invocation with:
-
-```sh
-hdiutil makehybrid -iso -joliet -default-volume-name config-2 -o configdrive.iso /tmp/new-drive
-```
-
 ## QEMU virtfs
 
 One exception to the above, when using QEMU it is possible to skip creating an

Documentation/vmware-guestinfo.md

@@ -1,35 +0,0 @@
-# VMWare Guestinfo Interface
-
-## Cloud-Config VMWare Guestinfo Variables
-
-coreos-cloudinit accepts configuration from the VMware RPC API's *guestinfo*
-facility. This datasource can be enabled with the `--from-vmware-guestinfo`
-flag to coreos-cloudinit.
-
-The following guestinfo variables are recognized and processed by cloudinit
-when passed from the hypervisor to the virtual machine at boot time. Note that
-property names are prefixed with `guestinfo.` in the VMX, e.g., `guestinfo.hostname`.
-
-|            guestinfo variable             |              type               |
-|:--------------------------------------|:--------------------------------|
-| `hostname`                            | `hostname`                      |
-| `interface.<n>.name`                  | `string`                        |
-| `interface.<n>.mac`                   | `MAC address`                   |
-| `interface.<n>.dhcp`                  | `{"yes", "no"}`                 |
-| `interface.<n>.role`                  | `{"public", "private"}`         |
-| `interface.<n>.ip.<m>.address`        | `CIDR IP address`               |
-| `interface.<n>.route.<l>.gateway`     | `IP address`                    |
-| `interface.<n>.route.<l>.destination` | `CIDR IP address`               |
-| `dns.server.<x>`                      | `IP address`                    |
-| `coreos.config.data`                  | `string`                        |
-| `coreos.config.data.encoding`         | `{"", "base64", "gzip+base64"}` |
-| `coreos.config.url`                   | `URL`                           |
-
-Note: "n", "m", "l", and "x" are 0-indexed, incrementing integers. The
-identifier for an `interface` does not correspond to anything outside of this
-configuration; it serves only to distinguish between multiple `interface`s.
-
-The guide to [booting on VMWare][bootvmware] is the starting point for more
-information about configuring and running CoreOS on VMWare.
-
-[bootvmware]: https://github.com/coreos/docs/blob/master/os/booting-on-vmware.md

README.md

@@ -76,11 +76,4 @@ coreos:
     etcd:
         addr: 203.0.113.29:4001
         peer-addr: 192.0.2.13:7001
-```
+```
-
-## Bugs
-
-Please use the [CoreOS issue tracker][bugs] to report all bugs, issues, and feature requests.
-
-[bugs]: https://github.com/coreos/bugs/issues/new?labels=component/cloud-init
-

build

@@ -1,37 +1,14 @@
-#!/bin/bash -x
+#!/bin/bash -e
 
 ORG_PATH="github.com/coreos"
 REPO_PATH="${ORG_PATH}/coreos-cloudinit"
-VERSION=$(git describe --tags)
-GLDFLAGS="-X main.version=${VERSION}"
 
-rm -rf bin tmp
+if [ ! -h gopath/src/${REPO_PATH} ]; then
+	mkdir -p gopath/src/${ORG_PATH}
+	ln -s ../../../.. gopath/src/${REPO_PATH} || exit 255
+fi
 
-export GO15VENDOREXPERIMENT=1
 export GOBIN=${PWD}/bin
 export GOPATH=${PWD}/gopath
-mkdir -p $GOBIN
-mkdir -p $GOPATH
-mkdir -p bin tmp
 
-which go 2>/dev/null
+go build -o bin/coreos-cloudinit ${REPO_PATH}
-
-if [ "x$?" != "x0" ]; then
-    export GOROOT=$(pwd)/goroot
-    export PATH=$GOROOT/bin:$PATH
-    mkdir -p $GOROOT
-    wget https://storage.googleapis.com/golang/go1.5.linux-amd64.tar.gz -O tmp/go.tar.gz
-    tar --strip-components=1 -C $GOROOT -xf tmp/go.tar.gz
-fi
-
-if [ ! -h $GOPATH/src/${REPO_PATH} ]; then
-        mkdir -p $GOPATH/src/${ORG_PATH}
-        ln -s ../../../.. $GOPATH/src/${REPO_PATH} || echo "exit 255"
-fi
-
-set -e
-
-for os in linux freebsd netbsd openbsd windows; do
-    GOOS=${os} go build -x -ldflags "${GLDFLAGS}" -tags netgo -o bin/cloudinit-${os}-x86_64 ${REPO_PATH}
-    GOOS=${os} GOARCH=386 go build -x -ldflags "${GLDFLAGS}" -tags netgo -o bin/cloudinit-${os}-x86_32 ${REPO_PATH}
-done

config/config.go

@@ -1,164 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-import (
-	"fmt"
-	"reflect"
-	"regexp"
-	"strings"
-	"unicode"
-
-	yaml "gopkg.in/yaml.v2"
-)
-
-// CloudConfig encapsulates the entire cloud-config configuration file and maps
-// directly to YAML. Fields that cannot be set in the cloud-config (fields
-// used for internal use) have the YAML tag '-' so that they aren't marshalled.
-type CloudConfig struct {
-	SSHAuthorizedKeys []string   `yaml:"ssh_authorized_keys"`
-	SSHFingerprints   bool       `yaml:"no_ssh_fingerprints"`
-	Debug             bool       `yaml:"debug"`
-	RunCMD            []string   `yaml:"runcmd"`
-	NetworkConfigPath string     `yaml:"-"`
-	NetworkConfig     string     `yaml:"-"`
-	Bootstrap         string     `yaml:"-"`
-	SystemInfo        SystemInfo `yaml:"system_info"`
-	DisableRoot       bool       `yaml:"disable_root"`
-	SSHPasswdAuth     bool       `yaml:"ssh_pwauth"`
-	ResizeRootfs      bool       `yaml:"resize_rootfs"`
-	CoreOS            CoreOS     `yaml:"coreos"`
-	WriteFiles        []File     `yaml:"write_files"`
-	Hostname          string     `yaml:"hostname"`
-	Users             []User     `yaml:"users"`
-	ManageEtcHosts    EtcHosts   `yaml:"manage_etc_hosts"`
-}
-
-type CoreOS struct {
-	Etcd      Etcd      `yaml:"etcd"`
-	Etcd2     Etcd2     `yaml:"etcd2"`
-	Flannel   Flannel   `yaml:"flannel"`
-	Fleet     Fleet     `yaml:"fleet"`
-	Locksmith Locksmith `yaml:"locksmith"`
-	OEM       OEM       `yaml:"oem"`
-	Update    Update    `yaml:"update"`
-	Units     []Unit    `yaml:"units"`
-}
-
-func IsCloudConfig(userdata string) bool {
-	header := strings.SplitN(userdata, "\n", 2)[0]
-
-	// Trim trailing whitespaces
-	header = strings.TrimRightFunc(header, unicode.IsSpace)
-
-	return (header == "#cloud-config")
-}
-
-// NewCloudConfig instantiates a new CloudConfig from the given contents (a
-// string of YAML), returning any error encountered. It will ignore unknown
-// fields but log encountering them.
-func NewCloudConfig(contents string) (*CloudConfig, error) {
-	//	yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
-	//		return strings.Replace(nameIn, "-", "_", -1)
-	//	}
-	var cfg CloudConfig
-	err := yaml.Unmarshal([]byte(contents), &cfg)
-	return &cfg, err
-}
-
-func (cc CloudConfig) String() string {
-	bytes, err := yaml.Marshal(cc)
-	if err != nil {
-		return ""
-	}
-
-	stringified := string(bytes)
-	stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
-
-	return stringified
-}
-
-// IsZero returns whether or not the parameter is the zero value for its type.
-// If the parameter is a struct, only the exported fields are considered.
-func IsZero(c interface{}) bool {
-	return isZero(reflect.ValueOf(c))
-}
-
-type ErrorValid struct {
-	Value string
-	Valid string
-	Field string
-}
-
-func (e ErrorValid) Error() string {
-	return fmt.Sprintf("invalid value %q for option %q (valid options: %q)", e.Value, e.Field, e.Valid)
-}
-
-// AssertStructValid checks the fields in the structure and makes sure that
-// they contain valid values as specified by the 'valid' flag. Empty fields are
-// implicitly valid.
-func AssertStructValid(c interface{}) error {
-	ct := reflect.TypeOf(c)
-	cv := reflect.ValueOf(c)
-	for i := 0; i < ct.NumField(); i++ {
-		ft := ct.Field(i)
-		if !isFieldExported(ft) {
-			continue
-		}
-
-		if err := AssertValid(cv.Field(i), ft.Tag.Get("valid")); err != nil {
-			err.Field = ft.Name
-			return err
-		}
-	}
-	return nil
-}
-
-// AssertValid checks to make sure that the given value is in the list of
-// valid values. Zero values are implicitly valid.
-func AssertValid(value reflect.Value, valid string) *ErrorValid {
-	if valid == "" || isZero(value) {
-		return nil
-	}
-
-	vs := fmt.Sprintf("%v", value.Interface())
-	if m, _ := regexp.MatchString(valid, vs); m {
-		return nil
-	}
-
-	return &ErrorValid{
-		Value: vs,
-		Valid: valid,
-	}
-}
-
-func isZero(v reflect.Value) bool {
-	switch v.Kind() {
-	case reflect.Struct:
-		vt := v.Type()
-		for i := 0; i < v.NumField(); i++ {
-			if isFieldExported(vt.Field(i)) && !isZero(v.Field(i)) {
-				return false
-			}
-		}
-		return true
-	default:
-		return v.Interface() == reflect.Zero(v.Type()).Interface()
-	}
-}
-
-func isFieldExported(f reflect.StructField) bool {
-	return f.PkgPath == ""
-}

config/config_test.go

@@ -1,503 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-import (
-	"reflect"
-	"regexp"
-	"strings"
-	"testing"
-)
-
-func TestNewCloudConfig(t *testing.T) {
-	tests := []struct {
-		contents string
-
-		config CloudConfig
-	}{
-		{},
-		{
-			contents: "#cloud-config\nwrite_files:\n  - path: underscore",
-			config:   CloudConfig{WriteFiles: []File{File{Path: "underscore"}}},
-		},
-		{
-			contents: "#cloud-config\nwrite-files:\n  - path: hyphen",
-			config:   CloudConfig{WriteFiles: []File{File{Path: "hyphen"}}},
-		},
-		{
-			contents: "#cloud-config\ncoreos:\n  update:\n    reboot-strategy: off",
-			config:   CloudConfig{CoreOS: CoreOS{Update: Update{RebootStrategy: "off"}}},
-		},
-		{
-			contents: "#cloud-config\ncoreos:\n  update:\n    reboot-strategy: false",
-			config:   CloudConfig{CoreOS: CoreOS{Update: Update{RebootStrategy: "false"}}},
-		},
-		{
-			contents: "#cloud-config\nwrite_files:\n  - permissions: 0744",
-			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "0744"}}},
-		},
-		{
-			contents: "#cloud-config\nwrite_files:\n  - permissions: 744",
-			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "744"}}},
-		},
-		{
-			contents: "#cloud-config\nwrite_files:\n  - permissions: '0744'",
-			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "0744"}}},
-		},
-		{
-			contents: "#cloud-config\nwrite_files:\n  - permissions: '744'",
-			config:   CloudConfig{WriteFiles: []File{File{RawFilePermissions: "744"}}},
-		},
-	}
-
-	for i, tt := range tests {
-		config, err := NewCloudConfig(tt.contents)
-		if err != nil {
-			t.Errorf("bad error (test case #%d): want %v, got %s", i, nil, err)
-		}
-		if !reflect.DeepEqual(&tt.config, config) {
-			t.Errorf("bad config (test case #%d): want %#v, got %#v", i, tt.config, config)
-		}
-	}
-}
-
-func TestIsZero(t *testing.T) {
-	tests := []struct {
-		c interface{}
-
-		empty bool
-	}{
-		{struct{}{}, true},
-		{struct{ a, b string }{}, true},
-		{struct{ A, b string }{}, true},
-		{struct{ A, B string }{}, true},
-		{struct{ A string }{A: "hello"}, false},
-		{struct{ A int }{}, true},
-		{struct{ A int }{A: 1}, false},
-	}
-
-	for _, tt := range tests {
-		if empty := IsZero(tt.c); tt.empty != empty {
-			t.Errorf("bad result (%q): want %t, got %t", tt.c, tt.empty, empty)
-		}
-	}
-}
-
-func TestAssertStructValid(t *testing.T) {
-	tests := []struct {
-		c interface{}
-
-		err error
-	}{
-		{struct{}{}, nil},
-		{struct {
-			A, b string `valid:"^1|2$"`
-		}{}, nil},
-		{struct {
-			A, b string `valid:"^1|2$"`
-		}{A: "1", b: "2"}, nil},
-		{struct {
-			A, b string `valid:"^1|2$"`
-		}{A: "1", b: "hello"}, nil},
-		{struct {
-			A, b string `valid:"^1|2$"`
-		}{A: "hello", b: "2"}, &ErrorValid{Value: "hello", Field: "A", Valid: "^1|2$"}},
-		{struct {
-			A, b int `valid:"^1|2$"`
-		}{}, nil},
-		{struct {
-			A, b int `valid:"^1|2$"`
-		}{A: 1, b: 2}, nil},
-		{struct {
-			A, b int `valid:"^1|2$"`
-		}{A: 1, b: 9}, nil},
-		{struct {
-			A, b int `valid:"^1|2$"`
-		}{A: 9, b: 2}, &ErrorValid{Value: "9", Field: "A", Valid: "^1|2$"}},
-	}
-
-	for _, tt := range tests {
-		if err := AssertStructValid(tt.c); !reflect.DeepEqual(tt.err, err) {
-			t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.err, err)
-		}
-	}
-}
-
-func TestConfigCompile(t *testing.T) {
-	tests := []interface{}{
-		Etcd{},
-		File{},
-		Flannel{},
-		Fleet{},
-		Locksmith{},
-		OEM{},
-		Unit{},
-		Update{},
-	}
-
-	for _, tt := range tests {
-		ttt := reflect.TypeOf(tt)
-		for i := 0; i < ttt.NumField(); i++ {
-			ft := ttt.Field(i)
-			if !isFieldExported(ft) {
-				continue
-			}
-
-			if _, err := regexp.Compile(ft.Tag.Get("valid")); err != nil {
-				t.Errorf("bad regexp(%s.%s): want %v, got %s", ttt.Name(), ft.Name, nil, err)
-			}
-		}
-	}
-}
-
-func TestCloudConfigUnknownKeys(t *testing.T) {
-	contents := `
-coreos: 
-  etcd:
-    discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
-  coreos_unknown:
-    foo: "bar"
-section_unknown:
-  dunno:
-    something
-bare_unknown:
-  bar
-write_files:
-  - content: fun
-    path: /var/party
-    file_unknown: nofun
-users:
-  - name: fry
-    passwd: somehash
-    user_unknown: philip
-hostname:
-  foo
-`
-	cfg, err := NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
-	}
-	if cfg.Hostname != "foo" {
-		t.Fatalf("hostname not correctly set when invalid keys are present")
-	}
-	if cfg.CoreOS.Etcd.Discovery != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
-		t.Fatalf("etcd section not correctly set when invalid keys are present")
-	}
-	if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
-		t.Fatalf("write_files section not correctly set when invalid keys are present")
-	}
-	if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
-		t.Fatalf("users section not correctly set when invalid keys are present")
-	}
-}
-
-// Assert that the parsing of a cloud config file "generally works"
-func TestCloudConfigEmpty(t *testing.T) {
-	cfg, err := NewCloudConfig("")
-	if err != nil {
-		t.Fatalf("Encountered unexpected error :%v", err)
-	}
-
-	keys := cfg.SSHAuthorizedKeys
-	if len(keys) != 0 {
-		t.Error("Parsed incorrect number of SSH keys")
-	}
-
-	if len(cfg.WriteFiles) != 0 {
-		t.Error("Expected zero WriteFiles")
-	}
-
-	if cfg.Hostname != "" {
-		t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
-	}
-}
-
-// Assert that the parsing of a cloud config file "generally works"
-func TestCloudConfig(t *testing.T) {
-	contents := `
-coreos: 
-  etcd:
-    discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
-  update:
-    reboot_strategy: reboot
-  units:
-    - name: 50-eth0.network
-      runtime: yes
-      content: '[Match]
- 
-    Name=eth47
- 
- 
-    [Network]
- 
-    Address=10.209.171.177/19
- 
-'
-  oem:
-    id: rackspace
-    name: Rackspace Cloud Servers
-    version_id: 168.0.0
-    home_url: https://www.rackspace.com/cloud/servers/
-    bug_report_url: https://github.com/coreos/coreos-overlay
-ssh_authorized_keys:
-  - foobar
-  - foobaz
-write_files:
-  - content: |
-      penny
-      elroy
-    path: /etc/dogepack.conf
-    permissions: '0644'
-    owner: root:dogepack
-hostname: trontastic
-`
-	cfg, err := NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error :%v", err)
-	}
-
-	keys := cfg.SSHAuthorizedKeys
-	if len(keys) != 2 {
-		t.Error("Parsed incorrect number of SSH keys")
-	} else if keys[0] != "foobar" {
-		t.Error("Expected first SSH key to be 'foobar'")
-	} else if keys[1] != "foobaz" {
-		t.Error("Expected first SSH key to be 'foobaz'")
-	}
-
-	if len(cfg.WriteFiles) != 1 {
-		t.Error("Failed to parse correct number of write_files")
-	} else {
-		wf := cfg.WriteFiles[0]
-		if wf.Content != "penny\nelroy\n" {
-			t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
-		}
-		if wf.Encoding != "" {
-			t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
-		}
-		if wf.RawFilePermissions != "0644" {
-			t.Errorf("WriteFile has incorrect permissions %s", wf.RawFilePermissions)
-		}
-		if wf.Path != "/etc/dogepack.conf" {
-			t.Errorf("WriteFile has incorrect path %s", wf.Path)
-		}
-		if wf.Owner != "root:dogepack" {
-			t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
-		}
-	}
-
-	if len(cfg.CoreOS.Units) != 1 {
-		t.Error("Failed to parse correct number of units")
-	} else {
-		u := cfg.CoreOS.Units[0]
-		expect := `[Match]
-Name=eth47
-
-[Network]
-Address=10.209.171.177/19
-`
-		if u.Content != expect {
-			t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
-		}
-		if u.Runtime != true {
-			t.Errorf("Unit has incorrect runtime value")
-		}
-		if u.Name != "50-eth0.network" {
-			t.Errorf("Unit has incorrect name %s", u.Name)
-		}
-	}
-
-	if cfg.CoreOS.OEM.ID != "rackspace" {
-		t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.CoreOS.OEM.ID)
-	}
-
-	if cfg.Hostname != "trontastic" {
-		t.Errorf("Failed to parse hostname")
-	}
-	if cfg.CoreOS.Update.RebootStrategy != "reboot" {
-		t.Errorf("Failed to parse locksmith strategy")
-	}
-}
-
-// Assert that our interface conversion doesn't panic
-func TestCloudConfigKeysNotList(t *testing.T) {
-	contents := `
-ssh_authorized_keys:
-  - foo: bar
-`
-	cfg, err := NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error: %v", err)
-	}
-
-	keys := cfg.SSHAuthorizedKeys
-	if len(keys) != 0 {
-		t.Error("Parsed incorrect number of SSH keys")
-	}
-}
-
-func TestCloudConfigSerializationHeader(t *testing.T) {
-	cfg, _ := NewCloudConfig("")
-	contents := cfg.String()
-	header := strings.SplitN(contents, "\n", 2)[0]
-	if header != "#cloud-config" {
-		t.Fatalf("Serialized config did not have expected header")
-	}
-}
-
-func TestCloudConfigUsers(t *testing.T) {
-	contents := `
-users:
-  - name: elroy
-    passwd: somehash
-    ssh_authorized_keys:
-      - somekey
-    gecos: arbitrary comment
-    homedir: /home/place
-    no_create_home: yes
-    lock_passwd: false
-    primary_group: things
-    groups:
-      - ping
-      - pong
-    no_user_group: true
-    system: y
-    no_log_init: True
-    shell: /bin/sh
-`
-	cfg, err := NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error: %v", err)
-	}
-
-	if len(cfg.Users) != 1 {
-		t.Fatalf("Parsed %d users, expected 1", len(cfg.Users))
-	}
-
-	user := cfg.Users[0]
-
-	if user.Name != "elroy" {
-		t.Errorf("User name is %q, expected 'elroy'", user.Name)
-	}
-
-	if user.PasswordHash != "somehash" {
-		t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
-	}
-
-	if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
-		t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
-	} else {
-		key := user.SSHAuthorizedKeys[0]
-		if key != "somekey" {
-			t.Errorf("User SSH key is %q, expected 'somekey'", key)
-		}
-	}
-
-	if user.GECOS != "arbitrary comment" {
-		t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
-	}
-
-	if user.Homedir != "/home/place" {
-		t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
-	}
-
-	if !user.NoCreateHome {
-		t.Errorf("Failed to parse no_create_home field")
-	}
-
-	if user.PrimaryGroup != "things" {
-		t.Errorf("Failed to parse primary_group field, got %q", user.PrimaryGroup)
-	}
-
-	if len(user.Groups) != 2 {
-		t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
-	} else {
-		if user.Groups[0] != "ping" {
-			t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
-		}
-		if user.Groups[1] != "pong" {
-			t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
-		}
-	}
-
-	if !user.NoUserGroup {
-		t.Errorf("Failed to parse no_user_group field")
-	}
-
-	if !user.System {
-		t.Errorf("Failed to parse system field")
-	}
-
-	if !user.NoLogInit {
-		t.Errorf("Failed to parse no_log_init field")
-	}
-
-	if user.Shell != "/bin/sh" {
-		t.Errorf("Failed to parse shell field, got %q", user.Shell)
-	}
-}
-
-func TestCloudConfigUsersGithubUser(t *testing.T) {
-
-	contents := `
-users:
-  - name: elroy
-    coreos_ssh_import_github: bcwaldon
-`
-	cfg, err := NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error: %v", err)
-	}
-
-	if len(cfg.Users) != 1 {
-		t.Fatalf("Parsed %d users, expected 1", len(cfg.Users))
-	}
-
-	user := cfg.Users[0]
-
-	if user.Name != "elroy" {
-		t.Errorf("User name is %q, expected 'elroy'", user.Name)
-	}
-
-	if user.SSHImportGithubUser != "bcwaldon" {
-		t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
-	}
-}
-
-func TestCloudConfigUsersSSHImportURL(t *testing.T) {
-	contents := `
-users:
-  - name: elroy
-    coreos_ssh_import_url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
-`
-	cfg, err := NewCloudConfig(contents)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error: %v", err)
-	}
-
-	if len(cfg.Users) != 1 {
-		t.Fatalf("Parsed %d users, expected 1", len(cfg.Users))
-	}
-
-	user := cfg.Users[0]
-
-	if user.Name != "elroy" {
-		t.Errorf("User name is %q, expected 'elroy'", user.Name)
-	}
-
-	if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
-		t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
-	}
-}

config/decode.go

@@ -1,56 +0,0 @@
-package config
-
-import (
-	"bytes"
-	"compress/gzip"
-	"encoding/base64"
-	"fmt"
-)
-
-func DecodeBase64Content(content string) ([]byte, error) {
-	output, err := base64.StdEncoding.DecodeString(content)
-
-	if err != nil {
-		return nil, fmt.Errorf("Unable to decode base64: %q", err)
-	}
-
-	return output, nil
-}
-
-func DecodeGzipContent(content string) ([]byte, error) {
-	gzr, err := gzip.NewReader(bytes.NewReader([]byte(content)))
-
-	if err != nil {
-		return nil, fmt.Errorf("Unable to decode gzip: %q", err)
-	}
-	defer gzr.Close()
-
-	buf := new(bytes.Buffer)
-	buf.ReadFrom(gzr)
-
-	return buf.Bytes(), nil
-}
-
-func DecodeContent(content string, encoding string) ([]byte, error) {
-	switch encoding {
-	case "":
-		return []byte(content), nil
-
-	case "b64", "base64":
-		return DecodeBase64Content(content)
-
-	case "gz", "gzip":
-		return DecodeGzipContent(content)
-
-	case "gz+base64", "gzip+base64", "gz+b64", "gzip+b64":
-		gz, err := DecodeBase64Content(content)
-
-		if err != nil {
-			return nil, err
-		}
-
-		return DecodeGzipContent(string(gz))
-	}
-
-	return nil, fmt.Errorf("Unsupported encoding %q", encoding)
-}

config/etc_hosts.go

@@ -1,17 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type EtcHosts string

config/etcd.go

@@ -1,67 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Etcd struct {
-	Addr                     string  `yaml:"addr"                          env:"ETCD_ADDR"`
-	AdvertiseClientURLs      string  `yaml:"advertise_client_urls"         env:"ETCD_ADVERTISE_CLIENT_URLS"       deprecated:"etcd2 options no longer work for etcd"`
-	BindAddr                 string  `yaml:"bind_addr"                     env:"ETCD_BIND_ADDR"`
-	CAFile                   string  `yaml:"ca_file"                       env:"ETCD_CA_FILE"`
-	CertFile                 string  `yaml:"cert_file"                     env:"ETCD_CERT_FILE"`
-	ClusterActiveSize        int     `yaml:"cluster_active_size"           env:"ETCD_CLUSTER_ACTIVE_SIZE"`
-	ClusterRemoveDelay       float64 `yaml:"cluster_remove_delay"          env:"ETCD_CLUSTER_REMOVE_DELAY"`
-	ClusterSyncInterval      float64 `yaml:"cluster_sync_interval"         env:"ETCD_CLUSTER_SYNC_INTERVAL"`
-	CorsOrigins              string  `yaml:"cors"                          env:"ETCD_CORS"`
-	DataDir                  string  `yaml:"data_dir"                      env:"ETCD_DATA_DIR"`
-	Discovery                string  `yaml:"discovery"                     env:"ETCD_DISCOVERY"`
-	DiscoveryFallback        string  `yaml:"discovery_fallback"            env:"ETCD_DISCOVERY_FALLBACK"          deprecated:"etcd2 options no longer work for etcd"`
-	DiscoverySRV             string  `yaml:"discovery_srv"                 env:"ETCD_DISCOVERY_SRV"               deprecated:"etcd2 options no longer work for etcd"`
-	DiscoveryProxy           string  `yaml:"discovery_proxy"               env:"ETCD_DISCOVERY_PROXY"             deprecated:"etcd2 options no longer work for etcd"`
-	ElectionTimeout          int     `yaml:"election_timeout"              env:"ETCD_ELECTION_TIMEOUT"            deprecated:"etcd2 options no longer work for etcd"`
-	ForceNewCluster          bool    `yaml:"force_new_cluster"             env:"ETCD_FORCE_NEW_CLUSTER"           deprecated:"etcd2 options no longer work for etcd"`
-	GraphiteHost             string  `yaml:"graphite_host"                 env:"ETCD_GRAPHITE_HOST"`
-	HeartbeatInterval        int     `yaml:"heartbeat_interval"            env:"ETCD_HEARTBEAT_INTERVAL"          deprecated:"etcd2 options no longer work for etcd"`
-	HTTPReadTimeout          float64 `yaml:"http_read_timeout"             env:"ETCD_HTTP_READ_TIMEOUT"`
-	HTTPWriteTimeout         float64 `yaml:"http_write_timeout"            env:"ETCD_HTTP_WRITE_TIMEOUT"`
-	InitialAdvertisePeerURLs string  `yaml:"initial_advertise_peer_urls"   env:"ETCD_INITIAL_ADVERTISE_PEER_URLS" deprecated:"etcd2 options no longer work for etcd"`
-	InitialCluster           string  `yaml:"initial_cluster"               env:"ETCD_INITIAL_CLUSTER"             deprecated:"etcd2 options no longer work for etcd"`
-	InitialClusterState      string  `yaml:"initial_cluster_state"         env:"ETCD_INITIAL_CLUSTER_STATE"       deprecated:"etcd2 options no longer work for etcd"`
-	InitialClusterToken      string  `yaml:"initial_cluster_token"         env:"ETCD_INITIAL_CLUSTER_TOKEN"       deprecated:"etcd2 options no longer work for etcd"`
-	KeyFile                  string  `yaml:"key_file"                      env:"ETCD_KEY_FILE"`
-	ListenClientURLs         string  `yaml:"listen_client_urls"            env:"ETCD_LISTEN_CLIENT_URLS"          deprecated:"etcd2 options no longer work for etcd"`
-	ListenPeerURLs           string  `yaml:"listen_peer_urls"              env:"ETCD_LISTEN_PEER_URLS"            deprecated:"etcd2 options no longer work for etcd"`
-	MaxResultBuffer          int     `yaml:"max_result_buffer"             env:"ETCD_MAX_RESULT_BUFFER"`
-	MaxRetryAttempts         int     `yaml:"max_retry_attempts"            env:"ETCD_MAX_RETRY_ATTEMPTS"`
-	MaxSnapshots             int     `yaml:"max_snapshots"                 env:"ETCD_MAX_SNAPSHOTS"               deprecated:"etcd2 options no longer work for etcd"`
-	MaxWALs                  int     `yaml:"max_wals"                      env:"ETCD_MAX_WALS"                    deprecated:"etcd2 options no longer work for etcd"`
-	Name                     string  `yaml:"name"                          env:"ETCD_NAME"`
-	PeerAddr                 string  `yaml:"peer_addr"                     env:"ETCD_PEER_ADDR"`
-	PeerBindAddr             string  `yaml:"peer_bind_addr"                env:"ETCD_PEER_BIND_ADDR"`
-	PeerCAFile               string  `yaml:"peer_ca_file"                  env:"ETCD_PEER_CA_FILE"`
-	PeerCertFile             string  `yaml:"peer_cert_file"                env:"ETCD_PEER_CERT_FILE"`
-	PeerElectionTimeout      int     `yaml:"peer_election_timeout"         env:"ETCD_PEER_ELECTION_TIMEOUT"`
-	PeerHeartbeatInterval    int     `yaml:"peer_heartbeat_interval"       env:"ETCD_PEER_HEARTBEAT_INTERVAL"`
-	PeerKeyFile              string  `yaml:"peer_key_file"                 env:"ETCD_PEER_KEY_FILE"`
-	Peers                    string  `yaml:"peers"                         env:"ETCD_PEERS"`
-	PeersFile                string  `yaml:"peers_file"                    env:"ETCD_PEERS_FILE"`
-	Proxy                    string  `yaml:"proxy"                         env:"ETCD_PROXY"                       deprecated:"etcd2 options no longer work for etcd"`
-	RetryInterval            float64 `yaml:"retry_interval"                env:"ETCD_RETRY_INTERVAL"`
-	Snapshot                 bool    `yaml:"snapshot"                      env:"ETCD_SNAPSHOT"`
-	SnapshotCount            int     `yaml:"snapshot_count"                env:"ETCD_SNAPSHOTCOUNT"`
-	StrTrace                 string  `yaml:"trace"                         env:"ETCD_TRACE"`
-	Verbose                  bool    `yaml:"verbose"                       env:"ETCD_VERBOSE"`
-	VeryVerbose              bool    `yaml:"very_verbose"                  env:"ETCD_VERY_VERBOSE"`
-	VeryVeryVerbose          bool    `yaml:"very_very_verbose"             env:"ETCD_VERY_VERY_VERBOSE"`
-}

config/etcd2.go

@@ -1,57 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Etcd2 struct {
-	AdvertiseClientURLs      string `yaml:"advertise_client_urls"         env:"ETCD_ADVERTISE_CLIENT_URLS"`
-	CAFile                   string `yaml:"ca_file"                       env:"ETCD_CA_FILE"                     deprecated:"ca_file obsoleted by trusted_ca_file and client_cert_auth"`
-	CertFile                 string `yaml:"cert_file"                     env:"ETCD_CERT_FILE"`
-	ClientCertAuth           bool   `yaml:"client_cert_auth"              env:"ETCD_CLIENT_CERT_AUTH"`
-	CorsOrigins              string `yaml:"cors"                          env:"ETCD_CORS"`
-	DataDir                  string `yaml:"data_dir"                      env:"ETCD_DATA_DIR"`
-	Debug                    bool   `yaml:"debug"                         env:"ETCD_DEBUG"`
-	Discovery                string `yaml:"discovery"                     env:"ETCD_DISCOVERY"`
-	DiscoveryFallback        string `yaml:"discovery_fallback"            env:"ETCD_DISCOVERY_FALLBACK"`
-	DiscoverySRV             string `yaml:"discovery_srv"                 env:"ETCD_DISCOVERY_SRV"`
-	DiscoveryProxy           string `yaml:"discovery_proxy"               env:"ETCD_DISCOVERY_PROXY"`
-	ElectionTimeout          int    `yaml:"election_timeout"              env:"ETCD_ELECTION_TIMEOUT"`
-	ForceNewCluster          bool   `yaml:"force_new_cluster"             env:"ETCD_FORCE_NEW_CLUSTER"`
-	HeartbeatInterval        int    `yaml:"heartbeat_interval"            env:"ETCD_HEARTBEAT_INTERVAL"`
-	InitialAdvertisePeerURLs string `yaml:"initial_advertise_peer_urls"   env:"ETCD_INITIAL_ADVERTISE_PEER_URLS"`
-	InitialCluster           string `yaml:"initial_cluster"               env:"ETCD_INITIAL_CLUSTER"`
-	InitialClusterState      string `yaml:"initial_cluster_state"         env:"ETCD_INITIAL_CLUSTER_STATE"`
-	InitialClusterToken      string `yaml:"initial_cluster_token"         env:"ETCD_INITIAL_CLUSTER_TOKEN"`
-	KeyFile                  string `yaml:"key_file"                      env:"ETCD_KEY_FILE"`
-	ListenClientURLs         string `yaml:"listen_client_urls"            env:"ETCD_LISTEN_CLIENT_URLS"`
-	ListenPeerURLs           string `yaml:"listen_peer_urls"              env:"ETCD_LISTEN_PEER_URLS"`
-	LogPackageLevels         string `yaml:"log_package_levels"            env:"ETCD_LOG_PACKAGE_LEVELS"`
-	MaxSnapshots             int    `yaml:"max_snapshots"                 env:"ETCD_MAX_SNAPSHOTS"`
-	MaxWALs                  int    `yaml:"max_wals"                      env:"ETCD_MAX_WALS"`
-	Name                     string `yaml:"name"                          env:"ETCD_NAME"`
-	PeerCAFile               string `yaml:"peer_ca_file"                  env:"ETCD_PEER_CA_FILE"                deprecated:"peer_ca_file obsoleted peer_trusted_ca_file and peer_client_cert_auth"`
-	PeerCertFile             string `yaml:"peer_cert_file"                env:"ETCD_PEER_CERT_FILE"`
-	PeerKeyFile              string `yaml:"peer_key_file"                 env:"ETCD_PEER_KEY_FILE"`
-	PeerClientCertAuth       bool   `yaml:"peer_client_cert_auth"         env:"ETCD_PEER_CLIENT_CERT_AUTH"`
-	PeerTrustedCAFile        string `yaml:"peer_trusted_ca_file"          env:"ETCD_PEER_TRUSTED_CA_FILE"`
-	Proxy                    string `yaml:"proxy"                         env:"ETCD_PROXY"                       valid:"^(on|off|readonly)$"`
-	ProxyDialTimeout         int    `yaml:"proxy_dial_timeout"            env:"ETCD_PROXY_DIAL_TIMEOUT"`
-	ProxyFailureWait         int    `yaml:"proxy_failure_wait"            env:"ETCD_PROXY_FAILURE_WAIT"`
-	ProxyReadTimeout         int    `yaml:"proxy_read_timeout"            env:"ETCD_PROXY_READ_TIMEOUT"`
-	ProxyRefreshInterval     int    `yaml:"proxy_refresh_interval"        env:"ETCD_PROXY_REFRESH_INTERVAL"`
-	ProxyWriteTimeout        int    `yaml:"proxy_write_timeout"           env:"ETCD_PROXY_WRITE_TIMEOUT"`
-	SnapshotCount            int    `yaml:"snapshot_count"                env:"ETCD_SNAPSHOT_COUNT"`
-	TrustedCAFile            string `yaml:"trusted_ca_file"               env:"ETCD_TRUSTED_CA_FILE"`
-	WalDir                   string `yaml:"wal_dir"                       env:"ETCD_WAL_DIR"`
-}

config/file.go

@@ -1,23 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type File struct {
-	Encoding           string `yaml:"encoding" valid:"^(base64|b64|gz|gzip|gz\\+base64|gzip\\+base64|gz\\+b64|gzip\\+b64)$"`
-	Content            string `yaml:"content"`
-	Owner              string `yaml:"owner"`
-	Path               string `yaml:"path"`
-	RawFilePermissions string `yaml:"permissions" valid:"^0?[0-7]{3,4}$"`
-}

config/file_test.go

@@ -1,69 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-import (
-	"testing"
-)
-
-func TestEncodingValid(t *testing.T) {
-	tests := []struct {
-		value string
-
-		isValid bool
-	}{
-		{value: "base64", isValid: true},
-		{value: "b64", isValid: true},
-		{value: "gz", isValid: true},
-		{value: "gzip", isValid: true},
-		{value: "gz+base64", isValid: true},
-		{value: "gzip+base64", isValid: true},
-		{value: "gz+b64", isValid: true},
-		{value: "gzip+b64", isValid: true},
-		{value: "gzzzzbase64", isValid: false},
-		{value: "gzipppbase64", isValid: false},
-		{value: "unknown", isValid: false},
-	}
-
-	for _, tt := range tests {
-		isValid := (nil == AssertStructValid(File{Encoding: tt.value}))
-		if tt.isValid != isValid {
-			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
-		}
-	}
-}
-
-func TestRawFilePermissionsValid(t *testing.T) {
-	tests := []struct {
-		value string
-
-		isValid bool
-	}{
-		{value: "744", isValid: true},
-		{value: "0744", isValid: true},
-		{value: "1744", isValid: true},
-		{value: "01744", isValid: true},
-		{value: "11744", isValid: false},
-		{value: "rwxr--r--", isValid: false},
-		{value: "800", isValid: false},
-	}
-
-	for _, tt := range tests {
-		isValid := (nil == AssertStructValid(File{RawFilePermissions: tt.value}))
-		if tt.isValid != isValid {
-			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
-		}
-	}
-}

config/flannel.go

@@ -1,27 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Flannel struct {
-	EtcdEndpoints string `yaml:"etcd_endpoints" env:"FLANNELD_ETCD_ENDPOINTS"`
-	EtcdCAFile    string `yaml:"etcd_cafile"    env:"FLANNELD_ETCD_CAFILE"`
-	EtcdCertFile  string `yaml:"etcd_certfile"  env:"FLANNELD_ETCD_CERTFILE"`
-	EtcdKeyFile   string `yaml:"etcd_keyfile"   env:"FLANNELD_ETCD_KEYFILE"`
-	EtcdPrefix    string `yaml:"etcd_prefix"    env:"FLANNELD_ETCD_PREFIX"`
-	IPMasq        string `yaml:"ip_masq"        env:"FLANNELD_IP_MASQ"`
-	SubnetFile    string `yaml:"subnet_file"    env:"FLANNELD_SUBNET_FILE"`
-	Iface         string `yaml:"interface"      env:"FLANNELD_IFACE"`
-	PublicIP      string `yaml:"public_ip"      env:"FLANNELD_PUBLIC_IP"`
-}

config/fleet.go

@@ -1,33 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Fleet struct {
-	AgentTTL                string  `yaml:"agent_ttl"                 env:"FLEET_AGENT_TTL"`
-	AuthorizedKeysFile      string  `yaml:"authorized_keys_file"      env:"FLEET_AUTHORIZED_KEYS_FILE"`
-	DisableEngine           bool    `yaml:"disable_engine"            env:"FLEET_DISABLE_ENGINE"`
-	EngineReconcileInterval float64 `yaml:"engine_reconcile_interval" env:"FLEET_ENGINE_RECONCILE_INTERVAL"`
-	EtcdCAFile              string  `yaml:"etcd_cafile"               env:"FLEET_ETCD_CAFILE"`
-	EtcdCertFile            string  `yaml:"etcd_certfile"             env:"FLEET_ETCD_CERTFILE"`
-	EtcdKeyFile             string  `yaml:"etcd_keyfile"              env:"FLEET_ETCD_KEYFILE"`
-	EtcdKeyPrefix           string  `yaml:"etcd_key_prefix"           env:"FLEET_ETCD_KEY_PREFIX"`
-	EtcdRequestTimeout      float64 `yaml:"etcd_request_timeout"      env:"FLEET_ETCD_REQUEST_TIMEOUT"`
-	EtcdServers             string  `yaml:"etcd_servers"              env:"FLEET_ETCD_SERVERS"`
-	Metadata                string  `yaml:"metadata"                  env:"FLEET_METADATA"`
-	PublicIP                string  `yaml:"public_ip"                 env:"FLEET_PUBLIC_IP"`
-	TokenLimit              int     `yaml:"token_limit"               env:"FLEET_TOKEN_LIMIT"`
-	Verbosity               int     `yaml:"verbosity"                 env:"FLEET_VERBOSITY"`
-	VerifyUnits             bool    `yaml:"verify_units"              env:"FLEET_VERIFY_UNITS"`
-}

config/ignition.go

@@ -1,26 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-import (
-	"encoding/json"
-)
-
-func IsIgnitionConfig(userdata string) bool {
-	var cfg struct {
-		Version *int `json:"ignitionVersion" yaml:"ignition_version"`
-	}
-	return (json.Unmarshal([]byte(userdata), &cfg) == nil && cfg.Version != nil)
-}

config/locksmith.go

@@ -1,25 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Locksmith struct {
-	Endpoint           string `yaml:"endpoint"      env:"LOCKSMITHD_ENDPOINT"`
-	EtcdCAFile         string `yaml:"etcd_cafile"   env:"LOCKSMITHD_ETCD_CAFILE"`
-	EtcdCertFile       string `yaml:"etcd_certfile" env:"LOCKSMITHD_ETCD_CERTFILE"`
-	EtcdKeyFile        string `yaml:"etcd_keyfile"  env:"LOCKSMITHD_ETCD_KEYFILE"`
-	Group              string `yaml:"group"         env:"LOCKSMITHD_GROUP"`
-	RebootWindowStart  string `yaml:"window_start"  env:"REBOOT_WINDOW_START"  valid:"^((?i:sun|mon|tue|wed|thu|fri|sat|sun) )?0*([0-9]|1[0-9]|2[0-3]):0*([0-9]|[1-5][0-9])$"`
-	RebootWindowLength string `yaml:"window_length" env:"REBOOT_WINDOW_LENGTH" valid:"^[-+]?([0-9]*(\\.[0-9]*)?[a-z]+)+$"`
-}

config/locksmith_test.go

@@ -1,76 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-import (
-	"testing"
-)
-
-func TestRebootWindowStart(t *testing.T) {
-	tests := []struct {
-		value string
-
-		isValid bool
-	}{
-		{value: "Sun 0:0", isValid: true},
-		{value: "Sun 00:00", isValid: true},
-		{value: "sUn 23:59", isValid: true},
-		{value: "mon 0:0", isValid: true},
-		{value: "tue 0:0", isValid: true},
-		{value: "tues 0:0", isValid: false},
-		{value: "wed 0:0", isValid: true},
-		{value: "thu 0:0", isValid: true},
-		{value: "thur 0:0", isValid: false},
-		{value: "fri 0:0", isValid: true},
-		{value: "sat 0:0", isValid: true},
-		{value: "sat00:00", isValid: false},
-		{value: "00:00", isValid: true},
-		{value: "10:10", isValid: true},
-		{value: "20:20", isValid: true},
-		{value: "20:30", isValid: true},
-		{value: "20:40", isValid: true},
-		{value: "20:50", isValid: true},
-		{value: "20:60", isValid: false},
-		{value: "24:00", isValid: false},
-	}
-
-	for _, tt := range tests {
-		isValid := (nil == AssertStructValid(Locksmith{RebootWindowStart: tt.value}))
-		if tt.isValid != isValid {
-			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
-		}
-	}
-}
-
-func TestRebootWindowLength(t *testing.T) {
-	tests := []struct {
-		value string
-
-		isValid bool
-	}{
-		{value: "1h", isValid: true},
-		{value: "1d", isValid: true},
-		{value: "0d", isValid: true},
-		{value: "0.5h", isValid: true},
-		{value: "0.5.0h", isValid: false},
-	}
-
-	for _, tt := range tests {
-		isValid := (nil == AssertStructValid(Locksmith{RebootWindowLength: tt.value}))
-		if tt.isValid != isValid {
-			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
-		}
-	}
-}

config/oem.go

@@ -1,23 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type OEM struct {
-	ID           string `yaml:"id"`
-	Name         string `yaml:"name"`
-	VersionID    string `yaml:"version_id"`
-	HomeURL      string `yaml:"home_url"`
-	BugReportURL string `yaml:"bug_report_url"`
-}

config/script.go

@@ -1,31 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-import (
-	"strings"
-)
-
-type Script []byte
-
-func IsScript(userdata string) bool {
-	header := strings.SplitN(userdata, "\n", 2)[0]
-	return strings.HasPrefix(header, "#!")
-}
-
-func NewScript(userdata string) (*Script, error) {
-	s := Script(userdata)
-	return &s, nil
-}

config/system_info.go

@@ -1,7 +0,0 @@
-package config
-
-type SystemInfo struct {
-	DefaultUser struct {
-		Name string `yaml:"name"`
-	} `yaml:"default_user"`
-}

config/unit.go

@@ -1,30 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Unit struct {
-	Name    string       `yaml:"name"`
-	Mask    bool         `yaml:"mask"`
-	Enable  bool         `yaml:"enable"`
-	Runtime bool         `yaml:"runtime"`
-	Content string       `yaml:"content"`
-	Command string       `yaml:"command" valid:"^(start|stop|restart|reload|try-restart|reload-or-restart|reload-or-try-restart)$"`
-	DropIns []UnitDropIn `yaml:"drop_ins"`
-}
-
-type UnitDropIn struct {
-	Name    string `yaml:"name"`
-	Content string `yaml:"content"`
-}

config/unit_test.go

@@ -1,46 +0,0 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
-
-package config
-
-import (
-	"testing"
-)
-
-func TestCommandValid(t *testing.T) {
-	tests := []struct {
-		value string
-
-		isValid bool
-	}{
-		{value: "start", isValid: true},
-		{value: "stop", isValid: true},
-		{value: "restart", isValid: true},
-		{value: "reload", isValid: true},
-		{value: "try-restart", isValid: true},
-		{value: "reload-or-restart", isValid: true},
-		{value: "reload-or-try-restart", isValid: true},
-		{value: "tryrestart", isValid: false},
-		{value: "unknown", isValid: false},
-	}
-
-	for _, tt := range tests {
-		isValid := (nil == AssertStructValid(Unit{Command: tt.value}))
-		if tt.isValid != isValid {
-			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
-		}
-	}
-}

config/update.go

@@ -1,21 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type Update struct {
-	RebootStrategy string `yaml:"reboot_strategy" env:"REBOOT_STRATEGY" valid:"^(best-effort|etcd-lock|reboot|off)$"`
-	Group          string `yaml:"group"           env:"GROUP"`
-	Server         string `yaml:"server"          env:"SERVER"`
-}

config/update_test.go

@@ -1,43 +0,0 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
-
-package config
-
-import (
-	"testing"
-)
-
-func TestRebootStrategyValid(t *testing.T) {
-	tests := []struct {
-		value string
-
-		isValid bool
-	}{
-		{value: "best-effort", isValid: true},
-		{value: "etcd-lock", isValid: true},
-		{value: "reboot", isValid: true},
-		{value: "off", isValid: true},
-		{value: "besteffort", isValid: false},
-		{value: "unknown", isValid: false},
-	}
-
-	for _, tt := range tests {
-		isValid := (nil == AssertStructValid(Update{RebootStrategy: tt.value}))
-		if tt.isValid != isValid {
-			t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
-		}
-	}
-}

config/user.go

@@ -1,34 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package config
-
-type User struct {
-	Name                 string   `yaml:"name"`
-	PasswordHash         string   `yaml:"passwd"`
-	SSHAuthorizedKeys    []string `yaml:"ssh_authorized_keys"`
-	SSHImportGithubUser  string   `yaml:"coreos_ssh_import_github"       deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
-	SSHImportGithubUsers []string `yaml:"coreos_ssh_import_github_users" deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
-	SSHImportURL         string   `yaml:"coreos_ssh_import_url"          deprecated:"trying to fetch from a remote endpoint introduces too many intermittent errors"`
-	GECOS                string   `yaml:"gecos"`
-	Homedir              string   `yaml:"homedir"`
-	NoCreateHome         bool     `yaml:"no_create_home"`
-	PrimaryGroup         string   `yaml:"primary_group"`
-	Groups               []string `yaml:"groups"`
-	NoUserGroup          bool     `yaml:"no_user_group"`
-	System               bool     `yaml:"system"`
-	NoLogInit            bool     `yaml:"no_log_init"`
-	LockPasswd           bool     `yaml:"lock_passwd"`
-	Shell                string   `yaml:"shell"`
-}

config/validate/context.go

@@ -1,52 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"strings"
-)
-
-// context represents the current position within a newline-delimited string.
-// Each line is loaded, one by one, into currentLine (newline omitted) and
-// lineNumber keeps track of its position within the original string.
-type context struct {
-	currentLine    string
-	remainingLines string
-	lineNumber     int
-}
-
-// Increment moves the context to the next line (if available).
-func (c *context) Increment() {
-	if c.currentLine == "" && c.remainingLines == "" {
-		return
-	}
-
-	lines := strings.SplitN(c.remainingLines, "\n", 2)
-	c.currentLine = lines[0]
-	if len(lines) == 2 {
-		c.remainingLines = lines[1]
-	} else {
-		c.remainingLines = ""
-	}
-	c.lineNumber++
-}
-
-// NewContext creates a context from the provided data. It strips out all
-// carriage returns and moves to the first line (if available).
-func NewContext(content []byte) context {
-	c := context{remainingLines: strings.Replace(string(content), "\r", "", -1)}
-	c.Increment()
-	return c
-}

config/validate/context_test.go

@@ -1,131 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"reflect"
-	"testing"
-)
-
-func TestNewContext(t *testing.T) {
-	tests := []struct {
-		in string
-
-		out context
-	}{
-		{
-			out: context{
-				currentLine:    "",
-				remainingLines: "",
-				lineNumber:     0,
-			},
-		},
-		{
-			in: "this\r\nis\r\na\r\ntest",
-			out: context{
-				currentLine:    "this",
-				remainingLines: "is\na\ntest",
-				lineNumber:     1,
-			},
-		},
-	}
-
-	for _, tt := range tests {
-		if out := NewContext([]byte(tt.in)); !reflect.DeepEqual(tt.out, out) {
-			t.Errorf("bad context (%q): want %#v, got %#v", tt.in, tt.out, out)
-		}
-	}
-}
-
-func TestIncrement(t *testing.T) {
-	tests := []struct {
-		init context
-		op   func(c *context)
-
-		res context
-	}{
-		{
-			init: context{
-				currentLine:    "",
-				remainingLines: "",
-				lineNumber:     0,
-			},
-			res: context{
-				currentLine:    "",
-				remainingLines: "",
-				lineNumber:     0,
-			},
-			op: func(c *context) {
-				c.Increment()
-			},
-		},
-		{
-			init: context{
-				currentLine:    "test",
-				remainingLines: "",
-				lineNumber:     1,
-			},
-			res: context{
-				currentLine:    "",
-				remainingLines: "",
-				lineNumber:     2,
-			},
-			op: func(c *context) {
-				c.Increment()
-				c.Increment()
-				c.Increment()
-			},
-		},
-		{
-			init: context{
-				currentLine:    "this",
-				remainingLines: "is\na\ntest",
-				lineNumber:     1,
-			},
-			res: context{
-				currentLine:    "is",
-				remainingLines: "a\ntest",
-				lineNumber:     2,
-			},
-			op: func(c *context) {
-				c.Increment()
-			},
-		},
-		{
-			init: context{
-				currentLine:    "this",
-				remainingLines: "is\na\ntest",
-				lineNumber:     1,
-			},
-			res: context{
-				currentLine:    "test",
-				remainingLines: "",
-				lineNumber:     4,
-			},
-			op: func(c *context) {
-				c.Increment()
-				c.Increment()
-				c.Increment()
-			},
-		},
-	}
-
-	for i, tt := range tests {
-		res := tt.init
-		if tt.op(&res); !reflect.DeepEqual(tt.res, res) {
-			t.Errorf("bad context (%d, %#v): want %#v, got %#v", i, tt.init, tt.res, res)
-		}
-	}
-}

config/validate/node.go

@@ -1,157 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"fmt"
-	"reflect"
-	"regexp"
-)
-
-var (
-	yamlKey  = regexp.MustCompile(`^ *-? ?(?P<key>.*?):`)
-	yamlElem = regexp.MustCompile(`^ *-`)
-)
-
-type node struct {
-	name     string
-	line     int
-	children []node
-	field    reflect.StructField
-	reflect.Value
-}
-
-// Child attempts to find the child with the given name in the node's list of
-// children. If no such child is found, an invalid node is returned.
-func (n node) Child(name string) node {
-	for _, c := range n.children {
-		if c.name == name {
-			return c
-		}
-	}
-	return node{}
-}
-
-// HumanType returns the human-consumable string representation of the type of
-// the node.
-func (n node) HumanType() string {
-	switch k := n.Kind(); k {
-	case reflect.Slice:
-		c := n.Type().Elem()
-		return "[]" + node{Value: reflect.New(c).Elem()}.HumanType()
-	default:
-		return k.String()
-	}
-}
-
-// NewNode returns the node representation of the given value. The context
-// will be used in an attempt to determine line numbers for the given value.
-func NewNode(value interface{}, context context) node {
-	var n node
-	toNode(value, context, &n)
-	return n
-}
-
-// toNode converts the given value into a node and then recursively processes
-// each of the nodes components (e.g. fields, array elements, keys).
-func toNode(v interface{}, c context, n *node) {
-	vv := reflect.ValueOf(v)
-	if !vv.IsValid() {
-		return
-	}
-
-	n.Value = vv
-	switch vv.Kind() {
-	case reflect.Struct:
-		// Walk over each field in the structure, skipping unexported fields,
-		// and create a node for it.
-		for i := 0; i < vv.Type().NumField(); i++ {
-			ft := vv.Type().Field(i)
-			k := ft.Tag.Get("yaml")
-			if k == "-" || k == "" {
-				continue
-			}
-
-			cn := node{name: k, field: ft}
-			c, ok := findKey(cn.name, c)
-			if ok {
-				cn.line = c.lineNumber
-			}
-			toNode(vv.Field(i).Interface(), c, &cn)
-			n.children = append(n.children, cn)
-		}
-	case reflect.Map:
-		// Walk over each key in the map and create a node for it.
-		v := v.(map[interface{}]interface{})
-		for k, cv := range v {
-			cn := node{name: fmt.Sprintf("%s", k)}
-			c, ok := findKey(cn.name, c)
-			if ok {
-				cn.line = c.lineNumber
-			}
-			toNode(cv, c, &cn)
-			n.children = append(n.children, cn)
-		}
-	case reflect.Slice:
-		// Walk over each element in the slice and create a node for it.
-		// While iterating over the slice, preserve the context after it
-		// is modified. This allows the line numbers to reflect the current
-		// element instead of the first.
-		for i := 0; i < vv.Len(); i++ {
-			cn := node{
-				name:  fmt.Sprintf("%s[%d]", n.name, i),
-				field: n.field,
-			}
-			var ok bool
-			c, ok = findElem(c)
-			if ok {
-				cn.line = c.lineNumber
-			}
-			toNode(vv.Index(i).Interface(), c, &cn)
-			n.children = append(n.children, cn)
-			c.Increment()
-		}
-	case reflect.String, reflect.Int, reflect.Bool, reflect.Float64:
-	default:
-		panic(fmt.Sprintf("toNode(): unhandled kind %s", vv.Kind()))
-	}
-}
-
-// findKey attempts to find the requested key within the provided context.
-// A modified copy of the context is returned with every line up to the key
-// incremented past. A boolean, true if the key was found, is also returned.
-func findKey(key string, context context) (context, bool) {
-	return find(yamlKey, key, context)
-}
-
-// findElem attempts to find an array element within the provided context.
-// A modified copy of the context is returned with every line up to the array
-// element incremented past. A boolean, true if the key was found, is also
-// returned.
-func findElem(context context) (context, bool) {
-	return find(yamlElem, "", context)
-}
-
-func find(exp *regexp.Regexp, key string, context context) (context, bool) {
-	for len(context.currentLine) > 0 || len(context.remainingLines) > 0 {
-		matches := exp.FindStringSubmatch(context.currentLine)
-		if len(matches) > 0 && (key == "" || matches[1] == key) {
-			return context, true
-		}
-
-		context.Increment()
-	}
-	return context, false
-}

config/validate/node_test.go

@@ -1,284 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"reflect"
-	"testing"
-)
-
-func TestChild(t *testing.T) {
-	tests := []struct {
-		parent node
-		name   string
-
-		child node
-	}{
-		{},
-		{
-			name: "c1",
-		},
-		{
-			parent: node{
-				children: []node{
-					node{name: "c1"},
-					node{name: "c2"},
-					node{name: "c3"},
-				},
-			},
-		},
-		{
-			parent: node{
-				children: []node{
-					node{name: "c1"},
-					node{name: "c2"},
-					node{name: "c3"},
-				},
-			},
-			name:  "c2",
-			child: node{name: "c2"},
-		},
-	}
-
-	for _, tt := range tests {
-		if child := tt.parent.Child(tt.name); !reflect.DeepEqual(tt.child, child) {
-			t.Errorf("bad child (%q): want %#v, got %#v", tt.name, tt.child, child)
-		}
-	}
-}
-
-func TestHumanType(t *testing.T) {
-	tests := []struct {
-		node node
-
-		humanType string
-	}{
-		{
-			humanType: "invalid",
-		},
-		{
-			node:      node{Value: reflect.ValueOf("hello")},
-			humanType: "string",
-		},
-		{
-			node: node{
-				Value: reflect.ValueOf([]int{1, 2}),
-				children: []node{
-					node{Value: reflect.ValueOf(1)},
-					node{Value: reflect.ValueOf(2)},
-				}},
-			humanType: "[]int",
-		},
-	}
-
-	for _, tt := range tests {
-		if humanType := tt.node.HumanType(); tt.humanType != humanType {
-			t.Errorf("bad type (%q): want %q, got %q", tt.node, tt.humanType, humanType)
-		}
-	}
-}
-
-func TestToNode(t *testing.T) {
-	tests := []struct {
-		value   interface{}
-		context context
-
-		node node
-	}{
-		{},
-		{
-			value: struct{}{},
-			node:  node{Value: reflect.ValueOf(struct{}{})},
-		},
-		{
-			value: struct {
-				A int `yaml:"a"`
-			}{},
-			node: node{
-				children: []node{
-					node{
-						name: "a",
-						field: reflect.TypeOf(struct {
-							A int `yaml:"a"`
-						}{}).Field(0),
-					},
-				},
-			},
-		},
-		{
-			value: struct {
-				A []int `yaml:"a"`
-			}{},
-			node: node{
-				children: []node{
-					node{
-						name: "a",
-						field: reflect.TypeOf(struct {
-							A []int `yaml:"a"`
-						}{}).Field(0),
-					},
-				},
-			},
-		},
-		{
-			value: map[interface{}]interface{}{
-				"a": map[interface{}]interface{}{
-					"b": 2,
-				},
-			},
-			context: NewContext([]byte("a:\n  b: 2")),
-			node: node{
-				children: []node{
-					node{
-						line: 1,
-						name: "a",
-						children: []node{
-							node{name: "b", line: 2},
-						},
-					},
-				},
-			},
-		},
-		{
-			value: struct {
-				A struct {
-					Jon bool `yaml:"b"`
-				} `yaml:"a"`
-			}{},
-			node: node{
-				children: []node{
-					node{
-						name: "a",
-						children: []node{
-							node{
-								name: "b",
-								field: reflect.TypeOf(struct {
-									Jon bool `yaml:"b"`
-								}{}).Field(0),
-								Value: reflect.ValueOf(false),
-							},
-						},
-						field: reflect.TypeOf(struct {
-							A struct {
-								Jon bool `yaml:"b"`
-							} `yaml:"a"`
-						}{}).Field(0),
-						Value: reflect.ValueOf(struct {
-							Jon bool `yaml:"b"`
-						}{}),
-					},
-				},
-				Value: reflect.ValueOf(struct {
-					A struct {
-						Jon bool `yaml:"b"`
-					} `yaml:"a"`
-				}{}),
-			},
-		},
-	}
-
-	for _, tt := range tests {
-		var node node
-		toNode(tt.value, tt.context, &node)
-		if !nodesEqual(tt.node, node) {
-			t.Errorf("bad node (%#v): want %#v, got %#v", tt.value, tt.node, node)
-		}
-	}
-}
-
-func TestFindKey(t *testing.T) {
-	tests := []struct {
-		key     string
-		context context
-
-		found bool
-	}{
-		{},
-		{
-			key:     "key1",
-			context: NewContext([]byte("key1: hi")),
-			found:   true,
-		},
-		{
-			key:     "key2",
-			context: NewContext([]byte("key1: hi")),
-			found:   false,
-		},
-		{
-			key:     "key3",
-			context: NewContext([]byte("key1:\n  key2:\n    key3: hi")),
-			found:   true,
-		},
-		{
-			key:     "key4",
-			context: NewContext([]byte("key1:\n  - key4: hi")),
-			found:   true,
-		},
-		{
-			key:     "key5",
-			context: NewContext([]byte("#key5")),
-			found:   false,
-		},
-	}
-
-	for _, tt := range tests {
-		if _, found := findKey(tt.key, tt.context); tt.found != found {
-			t.Errorf("bad find (%q): want %t, got %t", tt.key, tt.found, found)
-		}
-	}
-}
-
-func TestFindElem(t *testing.T) {
-	tests := []struct {
-		context context
-
-		found bool
-	}{
-		{},
-		{
-			context: NewContext([]byte("test: hi")),
-			found:   false,
-		},
-		{
-			context: NewContext([]byte("test:\n  - a\n  -b")),
-			found:   true,
-		},
-		{
-			context: NewContext([]byte("test:\n  -\n    a")),
-			found:   true,
-		},
-	}
-
-	for _, tt := range tests {
-		if _, found := findElem(tt.context); tt.found != found {
-			t.Errorf("bad find (%q): want %t, got %t", tt.context, tt.found, found)
-		}
-	}
-}
-
-func nodesEqual(a, b node) bool {
-	if a.name != b.name ||
-		a.line != b.line ||
-		!reflect.DeepEqual(a.field, b.field) ||
-		len(a.children) != len(b.children) {
-		return false
-	}
-	for i := 0; i < len(a.children); i++ {
-		if !nodesEqual(a.children[i], b.children[i]) {
-			return false
-		}
-	}
-	return true
-}

config/validate/report.go

@@ -1,88 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"encoding/json"
-	"fmt"
-)
-
-// Report represents the list of entries resulting from validation.
-type Report struct {
-	entries []Entry
-}
-
-// Error adds an error entry to the report.
-func (r *Report) Error(line int, message string) {
-	r.entries = append(r.entries, Entry{entryError, message, line})
-}
-
-// Warning adds a warning entry to the report.
-func (r *Report) Warning(line int, message string) {
-	r.entries = append(r.entries, Entry{entryWarning, message, line})
-}
-
-// Info adds an info entry to the report.
-func (r *Report) Info(line int, message string) {
-	r.entries = append(r.entries, Entry{entryInfo, message, line})
-}
-
-// Entries returns the list of entries in the report.
-func (r *Report) Entries() []Entry {
-	return r.entries
-}
-
-// Entry represents a single generic item in the report.
-type Entry struct {
-	kind    entryKind
-	message string
-	line    int
-}
-
-// String returns a human-readable representation of the entry.
-func (e Entry) String() string {
-	return fmt.Sprintf("line %d: %s: %s", e.line, e.kind, e.message)
-}
-
-// MarshalJSON satisfies the json.Marshaler interface, returning the entry
-// encoded as a JSON object.
-func (e Entry) MarshalJSON() ([]byte, error) {
-	return json.Marshal(map[string]interface{}{
-		"kind":    e.kind.String(),
-		"message": e.message,
-		"line":    e.line,
-	})
-}
-
-type entryKind int
-
-const (
-	entryError entryKind = iota
-	entryWarning
-	entryInfo
-)
-
-func (k entryKind) String() string {
-	switch k {
-	case entryError:
-		return "error"
-	case entryWarning:
-		return "warning"
-	case entryInfo:
-		return "info"
-	default:
-		panic(fmt.Sprintf("invalid kind %d", k))
-	}
-}

config/validate/report_test.go

@@ -1,96 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"bytes"
-	"reflect"
-	"testing"
-)
-
-func TestEntry(t *testing.T) {
-	tests := []struct {
-		entry Entry
-
-		str  string
-		json []byte
-	}{
-		{
-			Entry{entryInfo, "test info", 1},
-			"line 1: info: test info",
-			[]byte(`{"kind":"info","line":1,"message":"test info"}`),
-		},
-		{
-			Entry{entryWarning, "test warning", 1},
-			"line 1: warning: test warning",
-			[]byte(`{"kind":"warning","line":1,"message":"test warning"}`),
-		},
-		{
-			Entry{entryError, "test error", 2},
-			"line 2: error: test error",
-			[]byte(`{"kind":"error","line":2,"message":"test error"}`),
-		},
-	}
-
-	for _, tt := range tests {
-		if str := tt.entry.String(); tt.str != str {
-			t.Errorf("bad string (%q): want %q, got %q", tt.entry, tt.str, str)
-		}
-		json, err := tt.entry.MarshalJSON()
-		if err != nil {
-			t.Errorf("bad error (%q): want %v, got %q", tt.entry, nil, err)
-		}
-		if !bytes.Equal(tt.json, json) {
-			t.Errorf("bad JSON (%q): want %q, got %q", tt.entry, tt.json, json)
-		}
-	}
-}
-
-func TestReport(t *testing.T) {
-	type reportFunc struct {
-		fn      func(*Report, int, string)
-		line    int
-		message string
-	}
-
-	tests := []struct {
-		fs []reportFunc
-
-		es []Entry
-	}{
-		{
-			[]reportFunc{
-				{(*Report).Warning, 1, "test warning 1"},
-				{(*Report).Error, 2, "test error 2"},
-				{(*Report).Info, 10, "test info 10"},
-			},
-			[]Entry{
-				Entry{entryWarning, "test warning 1", 1},
-				Entry{entryError, "test error 2", 2},
-				Entry{entryInfo, "test info 10", 10},
-			},
-		},
-	}
-
-	for _, tt := range tests {
-		r := Report{}
-		for _, f := range tt.fs {
-			f.fn(&r, f.line, f.message)
-		}
-		if es := r.Entries(); !reflect.DeepEqual(tt.es, es) {
-			t.Errorf("bad entries (%v): want %#v, got %#v", tt.fs, tt.es, es)
-		}
-	}
-}

config/validate/rules.go

@@ -1,180 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"fmt"
-	"net/url"
-	"path"
-	"reflect"
-	"strings"
-
-	"github.com/coreos/coreos-cloudinit/config"
-)
-
-type rule func(config node, report *Report)
-
-// Rules contains all of the validation rules.
-var Rules []rule = []rule{
-	checkDiscoveryUrl,
-	checkEncoding,
-	checkStructure,
-	checkValidity,
-	checkWriteFiles,
-	checkWriteFilesUnderCoreos,
-}
-
-// checkDiscoveryUrl verifies that the string is a valid url.
-func checkDiscoveryUrl(cfg node, report *Report) {
-	c := cfg.Child("coreos").Child("etcd").Child("discovery")
-	if !c.IsValid() {
-		return
-	}
-
-	if _, err := url.ParseRequestURI(c.String()); err != nil {
-		report.Warning(c.line, "discovery URL is not valid")
-	}
-}
-
-// checkEncoding validates that, for each file under 'write_files', the
-// content can be decoded given the specified encoding.
-func checkEncoding(cfg node, report *Report) {
-	for _, f := range cfg.Child("write_files").children {
-		e := f.Child("encoding")
-		if !e.IsValid() {
-			continue
-		}
-
-		c := f.Child("content")
-		if _, err := config.DecodeContent(c.String(), e.String()); err != nil {
-			report.Error(c.line, fmt.Sprintf("content cannot be decoded as %q", e.String()))
-		}
-	}
-}
-
-// checkStructure compares the provided config to the empty config.CloudConfig
-// structure. Each node is checked to make sure that it exists in the known
-// structure and that its type is compatible.
-func checkStructure(cfg node, report *Report) {
-	g := NewNode(config.CloudConfig{}, NewContext([]byte{}))
-	checkNodeStructure(cfg, g, report)
-}
-
-func checkNodeStructure(n, g node, r *Report) {
-	if !isCompatible(n.Kind(), g.Kind()) {
-		r.Warning(n.line, fmt.Sprintf("incorrect type for %q (want %s)", n.name, g.HumanType()))
-		return
-	}
-
-	switch g.Kind() {
-	case reflect.Struct:
-		for _, cn := range n.children {
-			if cg := g.Child(cn.name); cg.IsValid() {
-				if msg := cg.field.Tag.Get("deprecated"); msg != "" {
-					r.Warning(cn.line, fmt.Sprintf("deprecated key %q (%s)", cn.name, msg))
-				}
-				checkNodeStructure(cn, cg, r)
-			} else {
-				r.Warning(cn.line, fmt.Sprintf("unrecognized key %q", cn.name))
-			}
-		}
-	case reflect.Slice:
-		for _, cn := range n.children {
-			var cg node
-			c := g.Type().Elem()
-			toNode(reflect.New(c).Elem().Interface(), context{}, &cg)
-			checkNodeStructure(cn, cg, r)
-		}
-	case reflect.String, reflect.Int, reflect.Float64, reflect.Bool:
-	default:
-		panic(fmt.Sprintf("checkNodeStructure(): unhandled kind %s", g.Kind()))
-	}
-}
-
-// isCompatible determines if the type of kind n can be converted to the type
-// of kind g in the context of YAML. This is not an exhaustive list, but its
-// enough for the purposes of cloud-config validation.
-func isCompatible(n, g reflect.Kind) bool {
-	switch g {
-	case reflect.String:
-		return n == reflect.String || n == reflect.Int || n == reflect.Float64 || n == reflect.Bool
-	case reflect.Struct:
-		return n == reflect.Struct || n == reflect.Map
-	case reflect.Float64:
-		return n == reflect.Float64 || n == reflect.Int
-	case reflect.Bool, reflect.Slice, reflect.Int:
-		return n == g
-	default:
-		panic(fmt.Sprintf("isCompatible(): unhandled kind %s", g))
-	}
-}
-
-// checkValidity checks the value of every node in the provided config by
-// running config.AssertValid() on it.
-func checkValidity(cfg node, report *Report) {
-	g := NewNode(config.CloudConfig{}, NewContext([]byte{}))
-	checkNodeValidity(cfg, g, report)
-}
-
-func checkNodeValidity(n, g node, r *Report) {
-	if err := config.AssertValid(n.Value, g.field.Tag.Get("valid")); err != nil {
-		r.Error(n.line, fmt.Sprintf("invalid value %v", n.Value.Interface()))
-	}
-	switch g.Kind() {
-	case reflect.Struct:
-		for _, cn := range n.children {
-			if cg := g.Child(cn.name); cg.IsValid() {
-				checkNodeValidity(cn, cg, r)
-			}
-		}
-	case reflect.Slice:
-		for _, cn := range n.children {
-			var cg node
-			c := g.Type().Elem()
-			toNode(reflect.New(c).Elem().Interface(), context{}, &cg)
-			checkNodeValidity(cn, cg, r)
-		}
-	case reflect.String, reflect.Int, reflect.Float64, reflect.Bool:
-	default:
-		panic(fmt.Sprintf("checkNodeValidity(): unhandled kind %s", g.Kind()))
-	}
-}
-
-// checkWriteFiles checks to make sure that the target file can actually be
-// written. Note that this check is approximate (it only checks to see if the file
-// is under /usr).
-func checkWriteFiles(cfg node, report *Report) {
-	for _, f := range cfg.Child("write_files").children {
-		c := f.Child("path")
-		if !c.IsValid() {
-			continue
-		}
-
-		d := path.Dir(c.String())
-		switch {
-		case strings.HasPrefix(d, "/usr"):
-			report.Error(c.line, "file cannot be written to a read-only filesystem")
-		}
-	}
-}
-
-// checkWriteFilesUnderCoreos checks to see if the 'write_files' node is a
-// child of 'coreos' (it shouldn't be).
-func checkWriteFilesUnderCoreos(cfg node, report *Report) {
-	c := cfg.Child("coreos").Child("write_files")
-	if c.IsValid() {
-		report.Info(c.line, "write_files doesn't belong under coreos")
-	}
-}

config/validate/rules_test.go

@@ -1,408 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"reflect"
-	"testing"
-)
-
-func TestCheckDiscoveryUrl(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		{},
-		{
-			config: "coreos:\n  etcd:\n    discovery: https://discovery.etcd.io/00000000000000000000000000000000",
-		},
-		{
-			config: "coreos:\n  etcd:\n    discovery: http://custom.domain/mytoken",
-		},
-		{
-			config:  "coreos:\n  etcd:\n    discovery: disco",
-			entries: []Entry{{entryWarning, "discovery URL is not valid", 3}},
-		},
-	}
-
-	for i, tt := range tests {
-		r := Report{}
-		n, err := parseCloudConfig([]byte(tt.config), &r)
-		if err != nil {
-			panic(err)
-		}
-		checkDiscoveryUrl(n, &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
-		}
-	}
-}
-
-func TestCheckEncoding(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		{},
-		{
-			config: "write_files:\n  - encoding: base64\n    content: aGVsbG8K",
-		},
-		{
-			config: "write_files:\n  - content: !!binary aGVsbG8K",
-		},
-		{
-			config:  "write_files:\n  - encoding: base64\n    content: !!binary aGVsbG8K",
-			entries: []Entry{{entryError, `content cannot be decoded as "base64"`, 3}},
-		},
-		{
-			config: "write_files:\n  - encoding: base64\n    content: !!binary YUdWc2JHOEsK",
-		},
-		{
-			config: "write_files:\n  - encoding: gzip\n    content: !!binary H4sIAOC3tVQAA8tIzcnJ5wIAIDA6NgYAAAA=",
-		},
-		{
-			config: "write_files:\n  - encoding: gzip+base64\n    content: H4sIAOC3tVQAA8tIzcnJ5wIAIDA6NgYAAAA=",
-		},
-		{
-			config:  "write_files:\n  - encoding: custom\n    content: hello",
-			entries: []Entry{{entryError, `content cannot be decoded as "custom"`, 3}},
-		},
-	}
-
-	for i, tt := range tests {
-		r := Report{}
-		n, err := parseCloudConfig([]byte(tt.config), &r)
-		if err != nil {
-			panic(err)
-		}
-		checkEncoding(n, &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
-		}
-	}
-}
-
-func TestCheckStructure(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		{},
-
-		// Test for unrecognized keys
-		{
-			config:  "test:",
-			entries: []Entry{{entryWarning, "unrecognized key \"test\"", 1}},
-		},
-		{
-			config:  "coreos:\n  etcd:\n    bad:",
-			entries: []Entry{{entryWarning, "unrecognized key \"bad\"", 3}},
-		},
-		{
-			config: "coreos:\n  etcd:\n    discovery: good",
-		},
-
-		// Test for deprecated keys
-		{
-			config: "coreos:\n  etcd:\n    addr: hi",
-		},
-		{
-			config:  "coreos:\n  etcd:\n    proxy: hi",
-			entries: []Entry{{entryWarning, "deprecated key \"proxy\" (etcd2 options no longer work for etcd)", 3}},
-		},
-
-		// Test for error on list of nodes
-		{
-			config: "coreos:\n  units:\n    - hello\n    - goodbye",
-			entries: []Entry{
-				{entryWarning, "incorrect type for \"units[0]\" (want struct)", 3},
-				{entryWarning, "incorrect type for \"units[1]\" (want struct)", 4},
-			},
-		},
-
-		// Test for incorrect types
-		// Want boolean
-		{
-			config: "coreos:\n  units:\n    - enable: true",
-		},
-		{
-			config:  "coreos:\n  units:\n    - enable: 4",
-			entries: []Entry{{entryWarning, "incorrect type for \"enable\" (want bool)", 3}},
-		},
-		{
-			config:  "coreos:\n  units:\n    - enable: bad",
-			entries: []Entry{{entryWarning, "incorrect type for \"enable\" (want bool)", 3}},
-		},
-		{
-			config:  "coreos:\n  units:\n    - enable:\n        bad:",
-			entries: []Entry{{entryWarning, "incorrect type for \"enable\" (want bool)", 3}},
-		},
-		{
-			config:  "coreos:\n  units:\n    - enable:\n      - bad",
-			entries: []Entry{{entryWarning, "incorrect type for \"enable\" (want bool)", 3}},
-		},
-		// Want string
-		{
-			config: "hostname: true",
-		},
-		{
-			config: "hostname: 4",
-		},
-		{
-			config: "hostname: host",
-		},
-		{
-			config:  "hostname:\n  name:",
-			entries: []Entry{{entryWarning, "incorrect type for \"hostname\" (want string)", 1}},
-		},
-		{
-			config:  "hostname:\n  - name",
-			entries: []Entry{{entryWarning, "incorrect type for \"hostname\" (want string)", 1}},
-		},
-		// Want struct
-		{
-			config:  "coreos: true",
-			entries: []Entry{{entryWarning, "incorrect type for \"coreos\" (want struct)", 1}},
-		},
-		{
-			config:  "coreos: 4",
-			entries: []Entry{{entryWarning, "incorrect type for \"coreos\" (want struct)", 1}},
-		},
-		{
-			config:  "coreos: hello",
-			entries: []Entry{{entryWarning, "incorrect type for \"coreos\" (want struct)", 1}},
-		},
-		{
-			config: "coreos:\n  etcd:\n    discovery: fire in the disco",
-		},
-		{
-			config:  "coreos:\n  - hello",
-			entries: []Entry{{entryWarning, "incorrect type for \"coreos\" (want struct)", 1}},
-		},
-		// Want []string
-		{
-			config:  "ssh_authorized_keys: true",
-			entries: []Entry{{entryWarning, "incorrect type for \"ssh_authorized_keys\" (want []string)", 1}},
-		},
-		{
-			config:  "ssh_authorized_keys: 4",
-			entries: []Entry{{entryWarning, "incorrect type for \"ssh_authorized_keys\" (want []string)", 1}},
-		},
-		{
-			config:  "ssh_authorized_keys: key",
-			entries: []Entry{{entryWarning, "incorrect type for \"ssh_authorized_keys\" (want []string)", 1}},
-		},
-		{
-			config:  "ssh_authorized_keys:\n  key: value",
-			entries: []Entry{{entryWarning, "incorrect type for \"ssh_authorized_keys\" (want []string)", 1}},
-		},
-		{
-			config: "ssh_authorized_keys:\n  - key",
-		},
-		{
-			config:  "ssh_authorized_keys:\n  - key: value",
-			entries: []Entry{{entryWarning, "incorrect type for \"ssh_authorized_keys[0]\" (want string)", 2}},
-		},
-		// Want []struct
-		{
-			config:  "users:\n  true",
-			entries: []Entry{{entryWarning, "incorrect type for \"users\" (want []struct)", 1}},
-		},
-		{
-			config:  "users:\n  4",
-			entries: []Entry{{entryWarning, "incorrect type for \"users\" (want []struct)", 1}},
-		},
-		{
-			config:  "users:\n  bad",
-			entries: []Entry{{entryWarning, "incorrect type for \"users\" (want []struct)", 1}},
-		},
-		{
-			config:  "users:\n  bad:",
-			entries: []Entry{{entryWarning, "incorrect type for \"users\" (want []struct)", 1}},
-		},
-		{
-			config: "users:\n  - name: good",
-		},
-		// Want struct within array
-		{
-			config:  "users:\n  - true",
-			entries: []Entry{{entryWarning, "incorrect type for \"users[0]\" (want struct)", 2}},
-		},
-		{
-			config:  "users:\n  - name: hi\n  - true",
-			entries: []Entry{{entryWarning, "incorrect type for \"users[1]\" (want struct)", 3}},
-		},
-		{
-			config:  "users:\n  - 4",
-			entries: []Entry{{entryWarning, "incorrect type for \"users[0]\" (want struct)", 2}},
-		},
-		{
-			config:  "users:\n  - bad",
-			entries: []Entry{{entryWarning, "incorrect type for \"users[0]\" (want struct)", 2}},
-		},
-		{
-			config:  "users:\n  - - bad",
-			entries: []Entry{{entryWarning, "incorrect type for \"users[0]\" (want struct)", 2}},
-		},
-	}
-
-	for i, tt := range tests {
-		r := Report{}
-		n, err := parseCloudConfig([]byte(tt.config), &r)
-		if err != nil {
-			panic(err)
-		}
-		checkStructure(n, &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
-		}
-	}
-}
-
-func TestCheckValidity(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		// string
-		{
-			config: "hostname: test",
-		},
-
-		// int
-		{
-			config: "coreos:\n  fleet:\n    verbosity: 2",
-		},
-
-		// bool
-		{
-			config: "coreos:\n  units:\n    - enable: true",
-		},
-
-		// slice
-		{
-			config: "coreos:\n  units:\n    - command: start\n    - name: stop",
-		},
-		{
-			config:  "coreos:\n  units:\n    - command: lol",
-			entries: []Entry{{entryError, "invalid value lol", 3}},
-		},
-
-		// struct
-		{
-			config: "coreos:\n  update:\n    reboot_strategy: off",
-		},
-		{
-			config:  "coreos:\n  update:\n    reboot_strategy: always",
-			entries: []Entry{{entryError, "invalid value always", 3}},
-		},
-
-		// unknown
-		{
-			config: "unknown: hi",
-		},
-	}
-
-	for i, tt := range tests {
-		r := Report{}
-		n, err := parseCloudConfig([]byte(tt.config), &r)
-		if err != nil {
-			panic(err)
-		}
-		checkValidity(n, &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
-		}
-	}
-}
-
-func TestCheckWriteFiles(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		{},
-		{
-			config: "write_files:\n  - path: /valid",
-		},
-		{
-			config: "write_files:\n  - path: /tmp/usr/valid",
-		},
-		{
-			config:  "write_files:\n  - path: /usr/invalid",
-			entries: []Entry{{entryError, "file cannot be written to a read-only filesystem", 2}},
-		},
-		{
-			config:  "write-files:\n  - path: /tmp/../usr/invalid",
-			entries: []Entry{{entryError, "file cannot be written to a read-only filesystem", 2}},
-		},
-	}
-
-	for i, tt := range tests {
-		r := Report{}
-		n, err := parseCloudConfig([]byte(tt.config), &r)
-		if err != nil {
-			panic(err)
-		}
-		checkWriteFiles(n, &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
-		}
-	}
-}
-
-func TestCheckWriteFilesUnderCoreos(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		{},
-		{
-			config: "write_files:\n  - path: /hi",
-		},
-		{
-			config:  "coreos:\n  write_files:\n    - path: /hi",
-			entries: []Entry{{entryInfo, "write_files doesn't belong under coreos", 2}},
-		},
-		{
-			config:  "coreos:\n  write-files:\n    - path: /hyphen",
-			entries: []Entry{{entryInfo, "write_files doesn't belong under coreos", 2}},
-		},
-	}
-
-	for i, tt := range tests {
-		r := Report{}
-		n, err := parseCloudConfig([]byte(tt.config), &r)
-		if err != nil {
-			panic(err)
-		}
-		checkWriteFilesUnderCoreos(n, &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%d, %q): want %#v, got %#v", i, tt.config, tt.entries, e)
-		}
-	}
-}

config/validate/validate.go

@@ -1,164 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"errors"
-	"fmt"
-	"regexp"
-	"strconv"
-	"strings"
-
-	"github.com/coreos/coreos-cloudinit/config"
-
-	yaml "gopkg.in/yaml.v2"
-)
-
-var (
-	yamlLineError = regexp.MustCompile(`^YAML error: line (?P<line>[[:digit:]]+): (?P<msg>.*)$`)
-	yamlError     = regexp.MustCompile(`^YAML error: (?P<msg>.*)$`)
-)
-
-// Validate runs a series of validation tests against the given userdata and
-// returns a report detailing all of the issues. Presently, only cloud-configs
-// can be validated.
-func Validate(userdataBytes []byte) (Report, error) {
-	switch {
-	case len(userdataBytes) == 0:
-		return Report{}, nil
-	case config.IsScript(string(userdataBytes)):
-		return Report{}, nil
-	case config.IsIgnitionConfig(string(userdataBytes)):
-		return Report{}, nil
-	case config.IsCloudConfig(string(userdataBytes)):
-		return validateCloudConfig(userdataBytes, Rules)
-	default:
-		return Report{entries: []Entry{
-			Entry{kind: entryError, message: `must be "#cloud-config" or begin with "#!"`, line: 1},
-		}}, nil
-	}
-}
-
-// validateCloudConfig runs all of the validation rules in Rules and returns
-// the resulting report and any errors encountered.
-func validateCloudConfig(config []byte, rules []rule) (report Report, err error) {
-	defer func() {
-		if r := recover(); r != nil {
-			err = fmt.Errorf("%v", r)
-		}
-	}()
-
-	c, err := parseCloudConfig(config, &report)
-	if err != nil {
-		return report, err
-	}
-
-	for _, r := range rules {
-		r(c, &report)
-	}
-	return report, nil
-}
-
-// parseCloudConfig parses the provided config into a node structure and logs
-// any parsing issues into the provided report. Unrecoverable errors are
-// returned as an error.
-func parseCloudConfig(cfg []byte, report *Report) (node, error) {
-	//	yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
-	//		return nameIn
-	//	}
-	// unmarshal the config into an implicitly-typed form. The yaml library
-	// will implicitly convert types into their normalized form
-	// (e.g. 0744 -> 484, off -> false).
-	var weak map[interface{}]interface{}
-	if err := yaml.Unmarshal(cfg, &weak); err != nil {
-		matches := yamlLineError.FindStringSubmatch(err.Error())
-		if len(matches) == 3 {
-			line, err := strconv.Atoi(matches[1])
-			if err != nil {
-				return node{}, err
-			}
-			msg := matches[2]
-			report.Error(line, msg)
-			return node{}, nil
-		}
-
-		matches = yamlError.FindStringSubmatch(err.Error())
-		if len(matches) == 2 {
-			report.Error(1, matches[1])
-			return node{}, nil
-		}
-
-		return node{}, errors.New("couldn't parse yaml error")
-	}
-	w := NewNode(weak, NewContext(cfg))
-	w = normalizeNodeNames(w, report)
-
-	// unmarshal the config into the explicitly-typed form.
-	//	yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
-	//		return strings.Replace(nameIn, "-", "_", -1)
-	//	}
-	var strong config.CloudConfig
-	if err := yaml.Unmarshal([]byte(cfg), &strong); err != nil {
-		return node{}, err
-	}
-	s := NewNode(strong, NewContext(cfg))
-
-	// coerceNodes weak nodes and strong nodes. strong nodes replace weak nodes
-	// if they are compatible types (this happens when the yaml library
-	// converts the input).
-	// (e.g. weak 484 is replaced by strong 0744, weak 4 is not replaced by
-	// strong false)
-	return coerceNodes(w, s), nil
-}
-
-// coerceNodes recursively evaluates two nodes, returning a new node containing
-// either the weak or strong node's value and its recursively processed
-// children. The strong node's value is used if the two nodes are leafs, are
-// both valid, and are compatible types (defined by isCompatible()). The weak
-// node is returned in all other cases. coerceNodes is used to counteract the
-// effects of yaml's automatic type conversion. The weak node is the one
-// resulting from unmarshalling into an empty interface{} (the type is
-// inferred). The strong node is the one resulting from unmarshalling into a
-// struct. If the two nodes are of compatible types, the yaml library correctly
-// parsed the value into the strongly typed unmarshalling. In this case, we
-// prefer the strong node because its actually the type we are expecting.
-func coerceNodes(w, s node) node {
-	n := w
-	n.children = nil
-	if len(w.children) == 0 && len(s.children) == 0 &&
-		w.IsValid() && s.IsValid() &&
-		isCompatible(w.Kind(), s.Kind()) {
-		n.Value = s.Value
-	}
-
-	for _, cw := range w.children {
-		n.children = append(n.children, coerceNodes(cw, s.Child(cw.name)))
-	}
-	return n
-}
-
-// normalizeNodeNames replaces all occurences of '-' with '_' within key names
-// and makes a note of each replacement in the report.
-func normalizeNodeNames(node node, report *Report) node {
-	if strings.Contains(node.name, "-") {
-		// TODO(crawford): Enable this message once the new validator hits stable.
-		//report.Info(node.line, fmt.Sprintf("%q uses '-' instead of '_'", node.name))
-		node.name = strings.Replace(node.name, "-", "_", -1)
-	}
-	for i := range node.children {
-		node.children[i] = normalizeNodeNames(node.children[i], report)
-	}
-	return node
-}

config/validate/validate_test.go

@@ -1,177 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package validate
-
-import (
-	"errors"
-	"reflect"
-	"testing"
-)
-
-func TestParseCloudConfig(t *testing.T) {
-	tests := []struct {
-		config string
-
-		entries []Entry
-	}{
-		{},
-		{
-			config: "	",
-			entries: []Entry{{entryError, "found character that cannot start any token", 1}},
-		},
-		{
-			config:  "a:\na",
-			entries: []Entry{{entryError, "could not find expected ':'", 2}},
-		},
-		{
-			config:  "#hello\na:\na",
-			entries: []Entry{{entryError, "could not find expected ':'", 3}},
-		},
-	}
-
-	for _, tt := range tests {
-		r := Report{}
-		parseCloudConfig([]byte(tt.config), &r)
-
-		if e := r.Entries(); !reflect.DeepEqual(tt.entries, e) {
-			t.Errorf("bad report (%s): want %#v, got %#v", tt.config, tt.entries, e)
-		}
-	}
-}
-
-func TestValidateCloudConfig(t *testing.T) {
-	tests := []struct {
-		config string
-		rules  []rule
-
-		report Report
-		err    error
-	}{
-		{
-			rules: []rule{func(_ node, _ *Report) { panic("something happened") }},
-			err:   errors.New("something happened"),
-		},
-		{
-			config: "write_files:\n  - permissions: 0744",
-			rules:  Rules,
-		},
-		{
-			config: "write_files:\n  - permissions: '0744'",
-			rules:  Rules,
-		},
-		{
-			config: "write_files:\n  - permissions: 744",
-			rules:  Rules,
-		},
-		{
-			config: "write_files:\n  - permissions: '744'",
-			rules:  Rules,
-		},
-		{
-			config: "coreos:\n  update:\n    reboot-strategy: off",
-			rules:  Rules,
-		},
-		{
-			config: "coreos:\n  update:\n    reboot-strategy: false",
-			rules:  Rules,
-			report: Report{entries: []Entry{{entryError, "invalid value false", 3}}},
-		},
-	}
-
-	for _, tt := range tests {
-		r, err := validateCloudConfig([]byte(tt.config), tt.rules)
-		if !reflect.DeepEqual(tt.err, err) {
-			t.Errorf("bad error (%s): want %v, got %v", tt.config, tt.err, err)
-		}
-		if !reflect.DeepEqual(tt.report, r) {
-			t.Errorf("bad report (%s): want %+v, got %+v", tt.config, tt.report, r)
-		}
-	}
-}
-
-func TestValidate(t *testing.T) {
-	tests := []struct {
-		config string
-
-		report Report
-	}{
-		{},
-		{
-			config: "#!/bin/bash\necho hey",
-		},
-		{
-			config: "{}",
-			report: Report{entries: []Entry{{entryError, `must be "#cloud-config" or begin with "#!"`, 1}}},
-		},
-		{
-			config: `{"ignitionVersion":0}`,
-		},
-		{
-			config: `{"ignitionVersion":1}`,
-		},
-	}
-
-	for i, tt := range tests {
-		r, err := Validate([]byte(tt.config))
-		if err != nil {
-			t.Errorf("bad error (case #%d): want %v, got %v", i, nil, err)
-		}
-		if !reflect.DeepEqual(tt.report, r) {
-			t.Errorf("bad report (case #%d): want %+v, got %+v", i, tt.report, r)
-		}
-	}
-}
-
-func BenchmarkValidate(b *testing.B) {
-	config := `#cloud-config
-hostname: test
-
-coreos:
-  etcd:
-    name:      node001
-    discovery: https://discovery.etcd.io/disco
-    addr:      $public_ipv4:4001
-    peer-addr: $private_ipv4:7001
-  fleet:
-    verbosity: 2
-    metadata:  "hi"
-  update:
-    reboot-strategy: off
-  units:
-    - name:    hi.service
-      command: start
-      enable:  true
-    - name:    bye.service
-      command: stop
-
-ssh_authorized_keys:
-  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
-  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
-
-users:
-  - name: me
-
-write_files:
-  - path: /etc/yes
-    content: "Hi"
-
-manage_etc_hosts: localhost`
-
-	for i := 0; i < b.N; i++ {
-		if _, err := Validate([]byte(config)); err != nil {
-			panic(err)
-		}
-	}
-}

coreos-cloudinit.go

@@ -1,52 +1,28 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package main
 
 import (
-	"bytes"
-	"compress/gzip"
 	"flag"
 	"fmt"
-	"io/ioutil"
-	"log"
 	"os"
-	"runtime"
 	"sync"
 	"time"
 
-	"github.com/coreos/coreos-cloudinit/config"
-	"github.com/coreos/coreos-cloudinit/config/validate"
 	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/configdrive"
 	"github.com/coreos/coreos-cloudinit/datasource/file"
-	"github.com/coreos/coreos-cloudinit/datasource/metadata/openstack"
+	"github.com/coreos/coreos-cloudinit/datasource/metadata/cloudsigma"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/ec2"
-	"github.com/coreos/coreos-cloudinit/datasource/metadata/packet"
 	"github.com/coreos/coreos-cloudinit/datasource/proc_cmdline"
 	"github.com/coreos/coreos-cloudinit/datasource/url"
-
-	//	"github.com/coreos/coreos-cloudinit/datasource/vmware"
 	"github.com/coreos/coreos-cloudinit/datasource/waagent"
 	"github.com/coreos/coreos-cloudinit/initialize"
-	"github.com/coreos/coreos-cloudinit/network"
 	"github.com/coreos/coreos-cloudinit/pkg"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
-var (
+const (
+	version               = "0.10.6"
 	datasourceInterval    = 100 * time.Millisecond
 	datasourceMaxInterval = 30 * time.Second
 	datasourceTimeout     = 5 * time.Minute
@@ -57,28 +33,21 @@ var (
 		printVersion  bool
 		ignoreFailure bool
 		sources       struct {
-			file               string
+			file                        string
-			configDrive        string
+			configDrive                 string
-			waagent            string
+			waagent                     string
-			metadataService    bool
+			metadataService             bool
-			ec2MetadataService string
+			ec2MetadataService          string
-			openstackMetadataService string
+			cloudSigmaMetadataService   bool
-			//			cloudSigmaMetadataService   bool
 			digitalOceanMetadataService string
-			packetMetadataService       string
 			url                         string
 			procCmdLine                 bool
-			//			vmware                      bool
 		}
 		convertNetconf string
 		workspace      string
 		sshKeyName     string
 		oem            string
-		validate       bool
-		timeout        string
-		dstimeout      string
 	}{}
-	version = "was not built properly"
 )
 
 func init() {
@@ -87,21 +56,16 @@ func init() {
 	flag.StringVar(&flags.sources.file, "from-file", "", "Read user-data from provided file")
 	flag.StringVar(&flags.sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory")
 	flag.StringVar(&flags.sources.waagent, "from-waagent", "", "Read data from provided waagent directory")
+	flag.BoolVar(&flags.sources.metadataService, "from-metadata-service", false, "[DEPRECATED - Use -from-ec2-metadata] Download data from metadata service")
 	flag.StringVar(&flags.sources.ec2MetadataService, "from-ec2-metadata", "", "Download EC2 data from the provided url")
-	//	flag.BoolVar(&flags.sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context")
+	flag.BoolVar(&flags.sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context")
 	flag.StringVar(&flags.sources.digitalOceanMetadataService, "from-digitalocean-metadata", "", "Download DigitalOcean data from the provided url")
-	flag.StringVar(&flags.sources.openstackMetadataService, "from-openstack-metadata", "", "Download OpenStack data from the provided url")
-	flag.StringVar(&flags.sources.packetMetadataService, "from-packet-metadata", "", "Download Packet data from metadata service")
 	flag.StringVar(&flags.sources.url, "from-url", "", "Download user-data from provided url")
 	flag.BoolVar(&flags.sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", proc_cmdline.ProcCmdlineLocation, proc_cmdline.ProcCmdlineCloudConfigFlag))
-	//	flag.BoolVar(&flags.sources.vmware, "from-vmware-guestinfo", false, "Read data from VMware guestinfo")
 	flag.StringVar(&flags.oem, "oem", "", "Use the settings specific to the provided OEM")
 	flag.StringVar(&flags.convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files")
-	flag.StringVar(&flags.workspace, "workspace", "/var/lib/cloudinit", "Base directory where cloudinit should use to store data")
+	flag.StringVar(&flags.workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
 	flag.StringVar(&flags.sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
-	flag.BoolVar(&flags.validate, "validate", false, "[EXPERIMENTAL] Validate the user-data but do not apply it to the system")
-	flag.StringVar(&flags.timeout, "timeout", "60s", "Timeout to wait for all datasource metadata")
-	flag.StringVar(&flags.dstimeout, "dstimeout", "10s", "Timeout to wait for single datasource metadata")
 }
 
 type oemConfig map[string]string
@@ -123,29 +87,12 @@ var (
 		"azure": oemConfig{
 			"from-waagent": "/var/lib/waagent",
 		},
-		//		"cloudsigma": oemConfig{
-		//			"from-cloudsigma-metadata": "true",
-		//		},
-		"packet": oemConfig{
-			"from-packet-metadata": "https://metadata.packet.net/",
-		},
-		//		"vmware": oemConfig{
-		//			"from-vmware-guestinfo": "true",
-		//			"convert-netconf":       "vmware",
-		//		},
 	}
 )
 
 func main() {
-	var err error
 	failure := false
 
-	// Conservative Go 1.5 upgrade strategy:
-	// keep GOMAXPROCS' default at 1 for now.
-	if os.Getenv("GOMAXPROCS") == "" {
-		runtime.GOMAXPROCS(1)
-	}
-
 	flag.Parse()
 
 	if c, ok := oemConfigs[flags.oem]; ok {
@@ -157,138 +104,118 @@ func main() {
 		for k := range oemConfigs {
 			oems = append(oems, k)
 		}
-		fmt.Printf("Invalid option to -oem: %q. Supported options: %q\n", flags.oem, oems)
+		fmt.Printf("Invalid option to --oem: %q. Supported options: %q\n", flags.oem, oems)
 		os.Exit(2)
 	}
 
 	if flags.printVersion == true {
-		fmt.Printf("coreos-cloudinit %s\n", version)
+		fmt.Printf("coreos-cloudinit version %s\n", version)
 		os.Exit(0)
 	}
 
-	datasourceTimeout, err = time.ParseDuration(flags.timeout)
-	if err != nil {
-		fmt.Printf("Invalid value to --timeout: %q\n", err)
-		os.Exit(1)
-	}
-	datasourceMaxInterval, err = time.ParseDuration(flags.dstimeout)
-	if err != nil {
-		fmt.Printf("Invalid value to --dstimeout: %q\n", err)
-		os.Exit(1)
-	}
-
 	switch flags.convertNetconf {
 	case "":
 	case "debian":
 	case "digitalocean":
-	case "packet":
-		//	case "vmware":
 	default:
-		fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian, digitalocean, packet, vmware'\n", flags.convertNetconf)
+		fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian, digitalocean'\n", flags.convertNetconf)
 		os.Exit(2)
 	}
 
 	dss := getDatasources()
 	if len(dss) == 0 {
-		fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-openstack-metadata, --from-ec2-metadata, --from-cloudsigma-metadata, --from-packet-metadata, --from-digitalocean-metadata, --from-vmware-guestinfo, --from-waagent, --from-url or --from-proc-cmdline")
+		fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-ec2-metadata, --from-cloudsigma-metadata, --from-url or --from-proc-cmdline")
 		os.Exit(2)
 	}
-  fmt.Printf("%#+v\n", dss)
+
 	ds := selectDatasource(dss)
 	if ds == nil {
-		log.Println("No datasources available in time")
+		fmt.Println("No datasources available in time")
 		os.Exit(1)
 	}
 
-	log.Printf("Fetching user-data from datasource of type %q\n", ds.Type())
+	fmt.Printf("Fetching user-data from datasource of type %q\n", ds.Type())
 	userdataBytes, err := ds.FetchUserdata()
 	if err != nil {
-		log.Printf("Failed fetching user-data from datasource: %v. Continuing...\n", err)
+		fmt.Printf("Failed fetching user-data from datasource: %v\nContinuing...\n", err)
-		failure = true
-	}
-	userdataBytes, err = decompressIfGzip(userdataBytes)
-	if err != nil {
-		log.Printf("Failed decompressing user-data from datasource: %v. Continuing...\n", err)
 		failure = true
 	}
 
-	if report, err := validate.Validate(userdataBytes); err == nil {
+	fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
-		ret := 0
+	metadataBytes, err := ds.FetchMetadata()
-		for _, e := range report.Entries() {
+	if err != nil {
-			log.Println(e)
+		fmt.Printf("Failed fetching meta-data from datasource: %v\n", err)
-			ret = 1
+		os.Exit(1)
-		}
+	}
-		if flags.validate {
+
-			os.Exit(ret)
+	// Extract IPv4 addresses from metadata if possible
-		}
+	var subs map[string]string
-	} else {
+	if len(metadataBytes) > 0 {
-		log.Printf("Failed while validating user_data (%q)\n", err)
+		subs, err = initialize.ExtractIPsFromMetadata(metadataBytes)
-		if flags.validate {
+		if err != nil {
+			fmt.Printf("Failed extracting IPs from meta-data: %v\n", err)
 			os.Exit(1)
 		}
 	}
 
-	log.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
-	metadata, err := ds.FetchMetadata()
-	if err != nil {
-		log.Printf("Failed fetching meta-data from datasource: %v\n", err)
-		os.Exit(1)
-	}
-
 	// Apply environment to user-data
-	env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.sshKeyName, metadata)
+	env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.convertNetconf, flags.sshKeyName, subs)
 	userdata := env.Apply(string(userdataBytes))
 
-	var ccu *config.CloudConfig
+	var ccm, ccu *initialize.CloudConfig
-	var script *config.Script
+	var script *system.Script
-	switch ud, err := initialize.ParseUserData(userdata); err {
+	if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil {
-	case initialize.ErrIgnitionConfig:
+		fmt.Printf("Failed to parse meta-data: %v\n", err)
-		fmt.Printf("Detected an Ignition config. Exiting...")
+		os.Exit(1)
-		os.Exit(0)
-	case nil:
-		switch t := ud.(type) {
-		case *config.CloudConfig:
-			ccu = t
-		case *config.Script:
-			script = t
-		}
-	default:
-		fmt.Printf("Failed to parse user-data: %v\nContinuing...\n", err)
-		failure = true
 	}
 
-	log.Println("Merging cloud-config from meta-data and user-data")
+	if ccm != nil && flags.convertNetconf != "" {
-	cc := mergeConfigs(ccu, metadata)
+		fmt.Printf("Fetching network config from datasource of type %q\n", ds.Type())
-
+		netconfBytes, err := ds.FetchNetworkConfig(ccm.NetworkConfigPath)
-	var ifaces []network.InterfaceGenerator
-	if flags.convertNetconf != "" {
-		var err error
-		switch flags.convertNetconf {
-		case "debian":
-			ifaces, err = network.ProcessDebianNetconf(metadata.NetworkConfig.([]byte))
-		case "digitalocean":
-			ifaces, err = network.ProcessDigitalOceanNetconf(metadata.NetworkConfig.(digitalocean.Metadata))
-		case "packet":
-			ifaces, err = network.ProcessPacketNetconf(metadata.NetworkConfig.(packet.NetworkData))
-			//		case "vmware":
-			//			ifaces, err = network.ProcessVMwareNetconf(metadata.NetworkConfig.(map[string]string))
-		default:
-			err = fmt.Errorf("Unsupported network config format %q", flags.convertNetconf)
-		}
 		if err != nil {
-			log.Printf("Failed to generate interfaces: %v\n", err)
+			fmt.Printf("Failed fetching network config from datasource: %v\n", err)
 			os.Exit(1)
 		}
+		ccm.NetworkConfig = string(netconfBytes)
 	}
 
-	if err = initialize.Apply(cc, ifaces, env); err != nil {
+	if ud, err := initialize.ParseUserData(userdata); err != nil {
-		log.Printf("Failed to apply cloud-config: %v\n", err)
+		fmt.Printf("Failed to parse user-data: %v\nContinuing...\n", err)
-		os.Exit(1)
+		failure = true
+	} else {
+		switch t := ud.(type) {
+		case *initialize.CloudConfig:
+			ccu = t
+		case system.Script:
+			script = &t
+		}
+	}
+
+	var cc *initialize.CloudConfig
+	if ccm != nil && ccu != nil {
+		fmt.Println("Merging cloud-config from meta-data and user-data")
+		merged := mergeCloudConfig(*ccm, *ccu)
+		cc = &merged
+	} else if ccm != nil && ccu == nil {
+		fmt.Println("Processing cloud-config from meta-data")
+		cc = ccm
+	} else if ccm == nil && ccu != nil {
+		fmt.Println("Processing cloud-config from user-data")
+		cc = ccu
+	} else {
+		fmt.Println("No cloud-config data to handle.")
+	}
+
+	if cc != nil {
+		if err = initialize.Apply(*cc, env); err != nil {
+			fmt.Printf("Failed to apply cloud-config: %v\n", err)
+			os.Exit(1)
+		}
 	}
 
 	if script != nil {
 		if err = runScript(*script, env); err != nil {
-			log.Printf("Failed to run script: %v\n", err)
+			fmt.Printf("Failed to run script: %v\n", err)
 			os.Exit(1)
 		}
 	}
@@ -298,25 +225,38 @@ func main() {
 	}
 }
 
-// mergeConfigs merges certain options from md (meta-data from the datasource)
+// mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from
-// onto cc (a CloudConfig derived from user-data), if they are not already set
+// meta-data) onto udcc (a CloudConfig derived from user-data), if they are
-// on cc (i.e. user-data always takes precedence)
+// not already set on udcc (i.e. user-data always takes precedence)
-func mergeConfigs(cc *config.CloudConfig, md datasource.Metadata) (out config.CloudConfig) {
+// NB: This needs to be kept in sync with ParseMetadata so that it tracks all
-	if cc != nil {
+// elements of a CloudConfig which that function can populate.
-		out = *cc
+func mergeCloudConfig(mdcc, udcc initialize.CloudConfig) (cc initialize.CloudConfig) {
-	}
+	if mdcc.Hostname != "" {
-
+		if udcc.Hostname != "" {
-	if md.Hostname != "" {
+			fmt.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)\n", udcc.Hostname, mdcc.Hostname)
-		if out.Hostname != "" {
-			log.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)\n", out.Hostname, md.Hostname)
 		} else {
-			out.Hostname = md.Hostname
+			udcc.Hostname = mdcc.Hostname
+		}
+
+	}
+	for _, key := range mdcc.SSHAuthorizedKeys {
+		udcc.SSHAuthorizedKeys = append(udcc.SSHAuthorizedKeys, key)
+	}
+	if mdcc.NetworkConfigPath != "" {
+		if udcc.NetworkConfigPath != "" {
+			fmt.Printf("Warning: user-data NetworkConfigPath %s overrides metadata NetworkConfigPath %s\n", udcc.NetworkConfigPath, mdcc.NetworkConfigPath)
+		} else {
+			udcc.NetworkConfigPath = mdcc.NetworkConfigPath
 		}
 	}
-	for _, key := range md.SSHPublicKeys {
+	if mdcc.NetworkConfig != "" {
-		out.SSHAuthorizedKeys = append(out.SSHAuthorizedKeys, key)
+		if udcc.NetworkConfig != "" {
+			fmt.Printf("Warning: user-data NetworkConfig %s overrides metadata NetworkConfig %s\n", udcc.NetworkConfig, mdcc.NetworkConfig)
+		} else {
+			udcc.NetworkConfig = mdcc.NetworkConfig
+		}
 	}
-	return
+	return udcc
 }
 
 // getDatasources creates a slice of possible Datasources for cloudinit based
@@ -335,30 +275,21 @@ func getDatasources() []datasource.Datasource {
 	if flags.sources.metadataService {
 		dss = append(dss, ec2.NewDatasource(ec2.DefaultAddress))
 	}
-	if flags.sources.openstackMetadataService != "" {
-	  dss = append(dss, openstack.NewDatasource(flags.sources.openstackMetadataService))
-  }
 	if flags.sources.ec2MetadataService != "" {
 		dss = append(dss, ec2.NewDatasource(flags.sources.ec2MetadataService))
 	}
-	//	if flags.sources.cloudSigmaMetadataService {
+	if flags.sources.cloudSigmaMetadataService {
-	//		dss = append(dss, cloudsigma.NewServerContextService())
+		dss = append(dss, cloudsigma.NewServerContextService())
-	//	}
+	}
 	if flags.sources.digitalOceanMetadataService != "" {
 		dss = append(dss, digitalocean.NewDatasource(flags.sources.digitalOceanMetadataService))
 	}
 	if flags.sources.waagent != "" {
 		dss = append(dss, waagent.NewDatasource(flags.sources.waagent))
 	}
-	if flags.sources.packetMetadataService != "" {
-		dss = append(dss, packet.NewDatasource(flags.sources.packetMetadataService))
-	}
 	if flags.sources.procCmdLine {
 		dss = append(dss, proc_cmdline.NewDatasource())
 	}
-	//	if flags.sources.vmware {
-	//		dss = append(dss, vmware.NewDatasource())
-	//	}
 	return dss
 }
 
@@ -379,7 +310,7 @@ func selectDatasource(sources []datasource.Datasource) datasource.Datasource {
 
 			duration := datasourceInterval
 			for {
-				log.Printf("Checking availability of %q\n", s.Type())
+				fmt.Printf("Checking availability of %q\n", s.Type())
 				if s.IsAvailable() {
 					ds <- s
 					return
@@ -414,10 +345,10 @@ func selectDatasource(sources []datasource.Datasource) datasource.Datasource {
 }
 
 // TODO(jonboulle): this should probably be refactored and moved into a different module
-func runScript(script config.Script, env *initialize.Environment) error {
+func runScript(script system.Script, env *initialize.Environment) error {
 	err := initialize.PrepWorkspace(env.Workspace())
 	if err != nil {
-		log.Printf("Failed preparing workspace: %v\n", err)
+		fmt.Printf("Failed preparing workspace: %v\n", err)
 		return err
 	}
 	path, err := initialize.PersistScriptInWorkspace(script, env.Workspace())
@@ -428,17 +359,3 @@ func runScript(script config.Script, env *initialize.Environment) error {
 	}
 	return err
 }
-
-const gzipMagicBytes = "\x1f\x8b"
-
-func decompressIfGzip(userdataBytes []byte) ([]byte, error) {
-	if !bytes.HasPrefix(userdataBytes, []byte(gzipMagicBytes)) {
-		return userdataBytes, nil
-	}
-	gzr, err := gzip.NewReader(bytes.NewReader(userdataBytes))
-	if err != nil {
-		return nil, err
-	}
-	defer gzr.Close()
-	return ioutil.ReadAll(gzr)
-}

coreos-cloudinit_test.go

@@ -1,147 +1,120 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package main
 
 import (
-	"bytes"
-	"encoding/base64"
-	"errors"
 	"reflect"
 	"testing"
 
-	"github.com/coreos/coreos-cloudinit/config"
+	"github.com/coreos/coreos-cloudinit/initialize"
-	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
-func TestMergeConfigs(t *testing.T) {
+func TestMergeCloudConfig(t *testing.T) {
-	tests := []struct {
+	simplecc := initialize.CloudConfig{
-		cc *config.CloudConfig
+		SSHAuthorizedKeys: []string{"abc", "def"},
-		md datasource.Metadata
+		Hostname:          "foobar",
-
+		NetworkConfigPath: "/path/somewhere",
-		out config.CloudConfig
+		NetworkConfig:     `{}`,
+	}
+	for i, tt := range []struct {
+		udcc initialize.CloudConfig
+		mdcc initialize.CloudConfig
+		want initialize.CloudConfig
 	}{
 		{
-			// If md is empty and cc is nil, result should be empty
+			// If mdcc is empty, udcc should be returned unchanged
-			out: config.CloudConfig{},
+			simplecc,
+			initialize.CloudConfig{},
+			simplecc,
 		},
 		{
-			// If md and cc are empty, result should be empty
+			// If udcc is empty, mdcc should be returned unchanged(overridden)
-			cc:  &config.CloudConfig{},
+			initialize.CloudConfig{},
-			out: config.CloudConfig{},
+			simplecc,
-		},
+			simplecc,
-		{
-			// If cc is empty, cc should be returned unchanged
-			cc:  &config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
-			out: config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
-		},
-		{
-			// If cc is empty, cc should be returned unchanged(overridden)
-			cc:  &config.CloudConfig{},
-			md:  datasource.Metadata{Hostname: "md-host", SSHPublicKeys: map[string]string{"key": "ghi"}},
-			out: config.CloudConfig{SSHAuthorizedKeys: []string{"ghi"}, Hostname: "md-host"},
-		},
-		{
-			// If cc is nil, cc should be returned unchanged(overridden)
-			md:  datasource.Metadata{Hostname: "md-host", SSHPublicKeys: map[string]string{"key": "ghi"}},
-			out: config.CloudConfig{SSHAuthorizedKeys: []string{"ghi"}, Hostname: "md-host"},
 		},
 		{
 			// user-data should override completely in the case of conflicts
-			cc:  &config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+			simplecc,
-			md:  datasource.Metadata{Hostname: "md-host"},
+			initialize.CloudConfig{
-			out: config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+				Hostname:          "meta-hostname",
+				NetworkConfigPath: "/path/meta",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
+			simplecc,
 		},
 		{
 			// Mixed merge should succeed
-			cc:  &config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def"}, Hostname: "cc-host"},
+			initialize.CloudConfig{
-			md:  datasource.Metadata{Hostname: "md-host", SSHPublicKeys: map[string]string{"key": "ghi"}},
+				SSHAuthorizedKeys: []string{"abc", "def"},
-			out: config.CloudConfig{SSHAuthorizedKeys: []string{"abc", "def", "ghi"}, Hostname: "cc-host"},
+				Hostname:          "user-hostname",
+				NetworkConfigPath: "/path/somewhere",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"woof", "qux"},
+				Hostname:          "meta-hostname",
+			},
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"},
+				Hostname:          "user-hostname",
+				NetworkConfigPath: "/path/somewhere",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
 		},
 		{
 			// Completely non-conflicting merge should be fine
-			cc:  &config.CloudConfig{Hostname: "cc-host"},
+			initialize.CloudConfig{
-			md:  datasource.Metadata{SSHPublicKeys: map[string]string{"zaphod": "beeblebrox"}},
+				Hostname: "supercool",
-			out: config.CloudConfig{Hostname: "cc-host", SSHAuthorizedKeys: []string{"beeblebrox"}},
+			},
+			initialize.CloudConfig{
+				SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
+				NetworkConfigPath: "/dev/fun",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
+			initialize.CloudConfig{
+				Hostname:          "supercool",
+				SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
+				NetworkConfigPath: "/dev/fun",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
 		},
 		{
 			// Non-mergeable settings in user-data should not be affected
-			cc:  &config.CloudConfig{Hostname: "cc-host", ManageEtcHosts: config.EtcHosts("lolz")},
+			initialize.CloudConfig{
-			md:  datasource.Metadata{Hostname: "md-host"},
+				Hostname:       "mememe",
-			out: config.CloudConfig{Hostname: "cc-host", ManageEtcHosts: config.EtcHosts("lolz")},
+				ManageEtcHosts: initialize.EtcHosts("lolz"),
+			},
+			initialize.CloudConfig{
+				Hostname:          "youyouyou",
+				NetworkConfigPath: "meta-meta-yo",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
+			initialize.CloudConfig{
+				Hostname:          "mememe",
+				ManageEtcHosts:    initialize.EtcHosts("lolz"),
+				NetworkConfigPath: "meta-meta-yo",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
 		},
-	}
+		{
-
+			// Non-mergeable (unexpected) settings in meta-data are ignored
-	for i, tt := range tests {
+			initialize.CloudConfig{
-		out := mergeConfigs(tt.cc, tt.md)
+				Hostname: "mememe",
-		if !reflect.DeepEqual(tt.out, out) {
+			},
-			t.Errorf("bad config (%d): want %#v, got %#v", i, tt.out, out)
+			initialize.CloudConfig{
+				ManageEtcHosts:    initialize.EtcHosts("lolz"),
+				NetworkConfigPath: "meta-meta-yo",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
+			initialize.CloudConfig{
+				Hostname:          "mememe",
+				NetworkConfigPath: "meta-meta-yo",
+				NetworkConfig:     `{"hostname":"test"}`,
+			},
+		},
+	} {
+		got := mergeCloudConfig(tt.mdcc, tt.udcc)
+		if !reflect.DeepEqual(got, tt.want) {
+			t.Errorf("case #%d: mergeCloudConfig mutated CloudConfig unexpectedly:\ngot:\n%s\nwant:\n%s", i, got, tt.want)
 		}
 	}
 }
-
-func mustDecode(in string) []byte {
-	out, err := base64.StdEncoding.DecodeString(in)
-	if err != nil {
-		panic(err)
-	}
-	return out
-}
-
-func TestDecompressIfGzip(t *testing.T) {
-	tests := []struct {
-		in []byte
-
-		out []byte
-		err error
-	}{
-		{
-			in: nil,
-
-			out: nil,
-			err: nil,
-		},
-		{
-			in: []byte{},
-
-			out: []byte{},
-			err: nil,
-		},
-		{
-			in: mustDecode("H4sIAJWV/VUAA1NOzskvTdFNzs9Ly0wHABt6mQENAAAA"),
-
-			out: []byte("#cloud-config"),
-			err: nil,
-		},
-		{
-			in: []byte("#cloud-config"),
-
-			out: []byte("#cloud-config"),
-			err: nil,
-		},
-		{
-			in: mustDecode("H4sCORRUPT=="),
-
-			out: nil,
-			err: errors.New("any error"),
-		},
-	}
-	for i, tt := range tests {
-		out, err := decompressIfGzip(tt.in)
-		if !bytes.Equal(out, tt.out) || (tt.err != nil && err == nil) {
-			t.Errorf("bad gzip (%d): want (%s, %#v), got (%s, %#v)", i, string(tt.out), tt.err, string(out), err)
-		}
-	}
-
-}

datasource/configdrive/configdrive.go

@@ -1,27 +1,10 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package configdrive
 
 import (
-	"encoding/json"
+	"fmt"
 	"io/ioutil"
-	"log"
 	"os"
 	"path"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
 const (
@@ -50,36 +33,21 @@ func (cd *configDrive) ConfigRoot() string {
 	return cd.openstackRoot()
 }
 
-func (cd *configDrive) FetchMetadata() (metadata datasource.Metadata, err error) {
+func (cd *configDrive) FetchMetadata() ([]byte, error) {
-	var data []byte
+	return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "meta_data.json"))
-	var m struct {
-		SSHAuthorizedKeyMap map[string]string `json:"public_keys"`
-		Hostname            string            `json:"hostname"`
-		NetworkConfig       struct {
-			ContentPath string `json:"content_path"`
-		} `json:"network_config"`
-	}
-
-	if data, err = cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "meta_data.json")); err != nil || len(data) == 0 {
-		return
-	}
-	if err = json.Unmarshal([]byte(data), &m); err != nil {
-		return
-	}
-
-	metadata.SSHPublicKeys = m.SSHAuthorizedKeyMap
-	metadata.Hostname = m.Hostname
-	if m.NetworkConfig.ContentPath != "" {
-		metadata.NetworkConfig, err = cd.tryReadFile(path.Join(cd.openstackRoot(), m.NetworkConfig.ContentPath))
-	}
-
-	return
 }
 
 func (cd *configDrive) FetchUserdata() ([]byte, error) {
 	return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "user_data"))
 }
 
+func (cd *configDrive) FetchNetworkConfig(filename string) ([]byte, error) {
+	if filename == "" {
+		return []byte{}, nil
+	}
+	return cd.tryReadFile(path.Join(cd.openstackRoot(), filename))
+}
+
 func (cd *configDrive) Type() string {
 	return "cloud-drive"
 }
@@ -93,7 +61,7 @@ func (cd *configDrive) openstackVersionRoot() string {
 }
 
 func (cd *configDrive) tryReadFile(filename string) ([]byte, error) {
-	log.Printf("Attempting to read from %q\n", filename)
+	fmt.Printf("Attempting to read from %q\n", filename)
 	data, err := cd.readFile(filename)
 	if os.IsNotExist(err) {
 		err = nil

datasource/configdrive/configdrive_test.go

@@ -1,103 +1,83 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package configdrive
 
 import (
-	"reflect"
+	"os"
 	"testing"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
-	"github.com/coreos/coreos-cloudinit/datasource/test"
 )
 
+type mockFilesystem []string
+
+func (m mockFilesystem) readFile(filename string) ([]byte, error) {
+	for _, file := range m {
+		if file == filename {
+			return []byte(filename), nil
+		}
+	}
+	return nil, os.ErrNotExist
+}
+
 func TestFetchMetadata(t *testing.T) {
 	for _, tt := range []struct {
-		root  string
+		root     string
-		files test.MockFilesystem
+		filename string
-
+		files    mockFilesystem
-		metadata datasource.Metadata
 	}{
 		{
-			root:  "/",
+			"/",
-			files: test.NewMockFilesystem(test.File{Path: "/openstack/latest/meta_data.json", Contents: ""}),
+			"",
+			mockFilesystem{},
 		},
 		{
-			root:  "/",
+			"/",
-			files: test.NewMockFilesystem(test.File{Path: "/openstack/latest/meta_data.json", Contents: `{"ignore": "me"}`}),
+			"/openstack/latest/meta_data.json",
+			mockFilesystem([]string{"/openstack/latest/meta_data.json"}),
 		},
 		{
-			root:     "/",
+			"/media/configdrive",
-			files:    test.NewMockFilesystem(test.File{Path: "/openstack/latest/meta_data.json", Contents: `{"hostname": "host"}`}),
+			"/media/configdrive/openstack/latest/meta_data.json",
-			metadata: datasource.Metadata{Hostname: "host"},
+			mockFilesystem([]string{"/media/configdrive/openstack/latest/meta_data.json"}),
-		},
-		{
-			root: "/media/configdrive",
-			files: test.NewMockFilesystem(test.File{Path: "/media/configdrive/openstack/latest/meta_data.json", Contents: `{"hostname": "host", "network_config": {"content_path": "config_file.json"}, "public_keys":{"1": "key1", "2": "key2"}}`},
-				test.File{Path: "/media/configdrive/openstack/config_file.json", Contents: "make it work"},
-			),
-			metadata: datasource.Metadata{
-				Hostname:      "host",
-				NetworkConfig: []byte("make it work"),
-				SSHPublicKeys: map[string]string{
-					"1": "key1",
-					"2": "key2",
-				},
-			},
 		},
 	} {
-		cd := configDrive{tt.root, tt.files.ReadFile}
+		cd := configDrive{tt.root, tt.files.readFile}
-		metadata, err := cd.FetchMetadata()
+		filename, err := cd.FetchMetadata()
 		if err != nil {
-			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %q: want %q, got %q", tt, nil, err)
 		}
-		if !reflect.DeepEqual(tt.metadata, metadata) {
+		if string(filename) != tt.filename {
-			t.Fatalf("bad metadata for %+v: want %#v, got %#v", tt, tt.metadata, metadata)
+			t.Fatalf("bad path for %q: want %q, got %q", tt, tt.filename, filename)
 		}
 	}
 }
 
 func TestFetchUserdata(t *testing.T) {
 	for _, tt := range []struct {
-		root  string
+		root     string
-		files test.MockFilesystem
+		filename string
-
+		files    mockFilesystem
-		userdata string
 	}{
 		{
 			"/",
-			test.NewMockFilesystem(),
 			"",
+			mockFilesystem{},
 		},
 		{
 			"/",
-			test.NewMockFilesystem(test.File{Path: "/openstack/latest/user_data", Contents: "userdata"}),
+			"/openstack/latest/user_data",
-			"userdata",
+			mockFilesystem([]string{"/openstack/latest/user_data"}),
 		},
 		{
 			"/media/configdrive",
-			test.NewMockFilesystem(test.File{Path: "/media/configdrive/openstack/latest/user_data", Contents: "userdata"}),
+			"/media/configdrive/openstack/latest/user_data",
-			"userdata",
+			mockFilesystem([]string{"/media/configdrive/openstack/latest/user_data"}),
 		},
 	} {
-		cd := configDrive{tt.root, tt.files.ReadFile}
+		cd := configDrive{tt.root, tt.files.readFile}
-		userdata, err := cd.FetchUserdata()
+		filename, err := cd.FetchUserdata()
 		if err != nil {
-			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %q: want %q, got %q", tt, nil, err)
 		}
-		if string(userdata) != tt.userdata {
+		if string(filename) != tt.filename {
-			t.Fatalf("bad userdata for %+v: want %q, got %q", tt, tt.userdata, userdata)
+			t.Fatalf("bad path for %q: want %q, got %q", tt, tt.filename, filename)
 		}
 	}
 }

datasource/datasource.go

@@ -1,38 +1,11 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package datasource
 
-import (
-	"net"
-)
-
 type Datasource interface {
 	IsAvailable() bool
 	AvailabilityChanges() bool
 	ConfigRoot() string
-	FetchMetadata() (Metadata, error)
+	FetchMetadata() ([]byte, error)
 	FetchUserdata() ([]byte, error)
+	FetchNetworkConfig(string) ([]byte, error)
 	Type() string
 }
-
-type Metadata struct {
-	PublicIPv4    net.IP
-	PublicIPv6    net.IP
-	PrivateIPv4   net.IP
-	PrivateIPv6   net.IP
-	Hostname      string
-	SSHPublicKeys map[string]string
-	NetworkConfig interface{}
-}

datasource/file/file.go

@@ -1,24 +1,8 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package file
 
 import (
 	"io/ioutil"
 	"os"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
 type localFile struct {
@@ -42,14 +26,18 @@ func (f *localFile) ConfigRoot() string {
 	return ""
 }
 
-func (f *localFile) FetchMetadata() (datasource.Metadata, error) {
+func (f *localFile) FetchMetadata() ([]byte, error) {
-	return datasource.Metadata{}, nil
+	return []byte{}, nil
 }
 
 func (f *localFile) FetchUserdata() ([]byte, error) {
 	return ioutil.ReadFile(f.path)
 }
 
+func (f *localFile) FetchNetworkConfig(filename string) ([]byte, error) {
+	return nil, nil
+}
+
 func (f *localFile) Type() string {
 	return "local-file"
 }

datasource/metadata/cloudsigma/server_context.go

@@ -0,0 +1,145 @@
+package cloudsigma
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"os"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/cloudsigma/cepgo"
+)
+
+const (
+	userDataFieldName = "cloudinit-user-data"
+)
+
+type serverContextService struct {
+	client interface {
+		All() (interface{}, error)
+		Key(string) (interface{}, error)
+		Meta() (map[string]string, error)
+		FetchRaw(string) ([]byte, error)
+	}
+}
+
+func NewServerContextService() *serverContextService {
+	return &serverContextService{
+		client: cepgo.NewCepgo(),
+	}
+}
+
+func (_ *serverContextService) IsAvailable() bool {
+	productNameFile, err := os.Open("/sys/class/dmi/id/product_name")
+	if err != nil {
+		return false
+	}
+	productName := make([]byte, 10)
+	_, err = productNameFile.Read(productName)
+	return err == nil && string(productName) == "CloudSigma"
+}
+
+func (_ *serverContextService) AvailabilityChanges() bool {
+	return true
+}
+
+func (_ *serverContextService) ConfigRoot() string {
+	return ""
+}
+
+func (_ *serverContextService) Type() string {
+	return "server-context"
+}
+
+func (scs *serverContextService) FetchMetadata() ([]byte, error) {
+	var (
+		inputMetadata struct {
+			Name string            `json:"name"`
+			UUID string            `json:"uuid"`
+			Meta map[string]string `json:"meta"`
+			Nics []struct {
+				Runtime struct {
+					InterfaceType string `json:"interface_type"`
+					IPv4          struct {
+						IP string `json:"uuid"`
+					} `json:"ip_v4"`
+				} `json:"runtime"`
+			} `json:"nics"`
+		}
+		outputMetadata struct {
+			Hostname   string            `json:"name"`
+			PublicKeys map[string]string `json:"public_keys"`
+			LocalIPv4  string            `json:"local-ipv4"`
+			PublicIPv4 string            `json:"public-ipv4"`
+		}
+	)
+
+	rawMetadata, err := scs.client.FetchRaw("")
+	if err != nil {
+		return []byte{}, err
+	}
+
+	err = json.Unmarshal(rawMetadata, &inputMetadata)
+	if err != nil {
+		return []byte{}, err
+	}
+
+	if inputMetadata.Name != "" {
+		outputMetadata.Hostname = inputMetadata.Name
+	} else {
+		outputMetadata.Hostname = inputMetadata.UUID
+	}
+
+	if key, ok := inputMetadata.Meta["ssh_public_key"]; ok {
+		splitted := strings.Split(key, " ")
+		outputMetadata.PublicKeys = make(map[string]string)
+		outputMetadata.PublicKeys[splitted[len(splitted)-1]] = key
+	}
+
+	for _, nic := range inputMetadata.Nics {
+		if nic.Runtime.IPv4.IP != "" {
+			if nic.Runtime.InterfaceType == "public" {
+				outputMetadata.PublicIPv4 = nic.Runtime.IPv4.IP
+			} else {
+				outputMetadata.LocalIPv4 = nic.Runtime.IPv4.IP
+			}
+		}
+	}
+
+	return json.Marshal(outputMetadata)
+}
+
+func (scs *serverContextService) FetchUserdata() ([]byte, error) {
+	metadata, err := scs.client.Meta()
+	if err != nil {
+		return []byte{}, err
+	}
+
+	userData, ok := metadata[userDataFieldName]
+	if ok && isBase64Encoded(userDataFieldName, metadata) {
+		if decodedUserData, err := base64.StdEncoding.DecodeString(userData); err == nil {
+			return decodedUserData, nil
+		} else {
+			return []byte{}, nil
+		}
+	}
+
+	return []byte(userData), nil
+}
+
+func (scs *serverContextService) FetchNetworkConfig(a string) ([]byte, error) {
+	return nil, nil
+}
+
+func isBase64Encoded(field string, userdata map[string]string) bool {
+	base64Fields, ok := userdata["base64_fields"]
+	if !ok {
+		return false
+	}
+
+	for _, base64Field := range strings.Split(base64Fields, ",") {
+		if field == base64Field {
+			return true
+		}
+	}
+	return false
+}

datasource/metadata/cloudsigma/server_context_test.go

@@ -0,0 +1,152 @@
+package cloudsigma
+
+import (
+	"encoding/json"
+	"reflect"
+	"testing"
+)
+
+type fakeCepgoClient struct {
+	raw  []byte
+	meta map[string]string
+	keys map[string]interface{}
+	err  error
+}
+
+func (f *fakeCepgoClient) All() (interface{}, error) {
+	return f.keys, f.err
+}
+
+func (f *fakeCepgoClient) Key(key string) (interface{}, error) {
+	return f.keys[key], f.err
+}
+
+func (f *fakeCepgoClient) Meta() (map[string]string, error) {
+	return f.meta, f.err
+}
+
+func (f *fakeCepgoClient) FetchRaw(key string) ([]byte, error) {
+	return f.raw, f.err
+}
+
+func TestServerContextFetchMetadata(t *testing.T) {
+	var metadata struct {
+		Hostname   string            `json:"name"`
+		PublicKeys map[string]string `json:"public_keys"`
+		LocalIPv4  string            `json:"local-ipv4"`
+		PublicIPv4 string            `json:"public-ipv4"`
+	}
+	client := new(fakeCepgoClient)
+	scs := NewServerContextService()
+	scs.client = client
+	client.raw = []byte(`{
+		"context": true,
+		"cpu": 4000,
+		"cpu_model": null,
+		"cpus_instead_of_cores": false,
+		"enable_numa": false,
+		"grantees": [],
+		"hv_relaxed": false,
+		"hv_tsc": false,
+		"jobs": [],
+		"mem": 4294967296,
+		"meta": {
+			"base64_fields": "cloudinit-user-data",
+			"cloudinit-user-data": "I2Nsb3VkLWNvbmZpZwoKaG9zdG5hbWU6IGNvcmVvczE=",
+			"ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2E.../hQ5D5 john@doe"
+		},
+		"name": "coreos",
+		"nics": [
+			{
+				"runtime": {
+					"interface_type": "public",
+					"ip_v4": {
+						"uuid": "31.171.251.74"
+					},
+					"ip_v6": null
+				},
+				"vlan": null
+			}
+		],
+		"smp": 2,
+		"status": "running",
+		"uuid": "20a0059b-041e-4d0c-bcc6-9b2852de48b3"
+	}`)
+
+	metadataBytes, err := scs.FetchMetadata()
+	if err != nil {
+		t.Error(err.Error())
+	}
+
+	if err := json.Unmarshal(metadataBytes, &metadata); err != nil {
+		t.Error(err.Error())
+	}
+
+	if metadata.Hostname != "coreos" {
+		t.Errorf("Hostname is not 'coreos' but %s instead", metadata.Hostname)
+	}
+
+	if metadata.PublicKeys["john@doe"] != "ssh-rsa AAAAB3NzaC1yc2E.../hQ5D5 john@doe" {
+		t.Error("Public SSH Keys are not being read properly")
+	}
+
+	if metadata.LocalIPv4 != "" {
+		t.Errorf("Local IP is not empty but %s instead", metadata.LocalIPv4)
+	}
+
+	if metadata.PublicIPv4 != "31.171.251.74" {
+		t.Errorf("Local IP is not 31.171.251.74 but %s instead", metadata.PublicIPv4)
+	}
+}
+
+func TestServerContextFetchUserdata(t *testing.T) {
+	client := new(fakeCepgoClient)
+	scs := NewServerContextService()
+	scs.client = client
+	userdataSets := []struct {
+		in  map[string]string
+		err bool
+		out []byte
+	}{
+		{map[string]string{
+			"base64_fields":       "cloudinit-user-data",
+			"cloudinit-user-data": "aG9zdG5hbWU6IGNvcmVvc190ZXN0",
+		}, false, []byte("hostname: coreos_test")},
+		{map[string]string{
+			"cloudinit-user-data": "#cloud-config\\nhostname: coreos1",
+		}, false, []byte("#cloud-config\\nhostname: coreos1")},
+		{map[string]string{}, false, []byte{}},
+	}
+
+	for i, set := range userdataSets {
+		client.meta = set.in
+		got, err := scs.FetchUserdata()
+		if (err != nil) != set.err {
+			t.Errorf("case %d: bad error state (got %t, want %t)", i, err != nil, set.err)
+		}
+
+		if !reflect.DeepEqual(got, set.out) {
+			t.Errorf("case %d: got %s, want %s", i, got, set.out)
+		}
+	}
+}
+
+func TestServerContextDecodingBase64UserData(t *testing.T) {
+	base64Sets := []struct {
+		in  string
+		out bool
+	}{
+		{"cloudinit-user-data,foo,bar", true},
+		{"bar,cloudinit-user-data,foo,bar", true},
+		{"cloudinit-user-data", true},
+		{"", false},
+		{"foo", false},
+	}
+
+	for _, set := range base64Sets {
+		userdata := map[string]string{"base64_fields": set.in}
+		if isBase64Encoded("cloudinit-user-data", userdata) != set.out {
+			t.Errorf("isBase64Encoded(cloudinit-user-data, %s) should be %t", userdata, set.out)
+		}
+	}
+}

datasource/metadata/digitalocean/metadata.go

@@ -1,25 +1,9 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package digitalocean
 
 import (
 	"encoding/json"
-	"net"
 	"strconv"
 
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 )
 
@@ -38,11 +22,10 @@ type Address struct {
 }
 
 type Interface struct {
-	IPv4       *Address `json:"ipv4"`
+	IPv4 *Address `json:"ipv4"`
-	IPv6       *Address `json:"ipv6"`
+	IPv6 *Address `json:"ipv6"`
-	AnchorIPv4 *Address `json:"anchor_ipv4"`
+	MAC  string   `json:"mac"`
-	MAC        string   `json:"mac"`
+	Type string   `json:"type"`
-	Type       string   `json:"type"`
 }
 
 type Interfaces struct {
@@ -62,6 +45,8 @@ type Metadata struct {
 }
 
 type metadataService struct {
+	interfaces Interfaces
+	dns        DNS
 	metadata.MetadataService
 }
 
@@ -69,41 +54,52 @@ func NewDatasource(root string) *metadataService {
 	return &metadataService{MetadataService: metadata.NewDatasource(root, apiVersion, userdataUrl, metadataPath)}
 }
 
-func (ms *metadataService) FetchMetadata() (metadata datasource.Metadata, err error) {
+func (ms *metadataService) FetchMetadata() ([]byte, error) {
-	var data []byte
+	data, err := ms.FetchData(ms.MetadataUrl())
-	var m Metadata
+	if err != nil || len(data) == 0 {
-
+		return []byte{}, err
-	if data, err = ms.FetchData(ms.MetadataUrl()); err != nil || len(data) == 0 {
-		return
-	}
-	if err = json.Unmarshal(data, &m); err != nil {
-		return
 	}
 
-	if len(m.Interfaces.Public) > 0 {
+	var metadata Metadata
-		if m.Interfaces.Public[0].IPv4 != nil {
+	if err := json.Unmarshal(data, &metadata); err != nil {
-			metadata.PublicIPv4 = net.ParseIP(m.Interfaces.Public[0].IPv4.IPAddress)
+		return []byte{}, err
-		}
-		if m.Interfaces.Public[0].IPv6 != nil {
-			metadata.PublicIPv6 = net.ParseIP(m.Interfaces.Public[0].IPv6.IPAddress)
-		}
 	}
-	if len(m.Interfaces.Private) > 0 {
-		if m.Interfaces.Private[0].IPv4 != nil {
-			metadata.PrivateIPv4 = net.ParseIP(m.Interfaces.Private[0].IPv4.IPAddress)
-		}
-		if m.Interfaces.Private[0].IPv6 != nil {
-			metadata.PrivateIPv6 = net.ParseIP(m.Interfaces.Private[0].IPv6.IPAddress)
-		}
-	}
-	metadata.Hostname = m.Hostname
-	metadata.SSHPublicKeys = map[string]string{}
-	for i, key := range m.PublicKeys {
-		metadata.SSHPublicKeys[strconv.Itoa(i)] = key
-	}
-	metadata.NetworkConfig = m
 
-	return
+	ms.interfaces = metadata.Interfaces
+	ms.dns = metadata.DNS
+
+	attrs := make(map[string]interface{})
+	if len(metadata.Interfaces.Public) > 0 {
+		if metadata.Interfaces.Public[0].IPv4 != nil {
+			attrs["public-ipv4"] = metadata.Interfaces.Public[0].IPv4.IPAddress
+		}
+		if metadata.Interfaces.Public[0].IPv6 != nil {
+			attrs["public-ipv6"] = metadata.Interfaces.Public[0].IPv6.IPAddress
+		}
+	}
+	if len(metadata.Interfaces.Private) > 0 {
+		if metadata.Interfaces.Private[0].IPv4 != nil {
+			attrs["local-ipv4"] = metadata.Interfaces.Private[0].IPv4.IPAddress
+		}
+		if metadata.Interfaces.Private[0].IPv6 != nil {
+			attrs["local-ipv6"] = metadata.Interfaces.Private[0].IPv6.IPAddress
+		}
+	}
+	attrs["hostname"] = metadata.Hostname
+	keys := make(map[string]string)
+	for i, key := range metadata.PublicKeys {
+		keys[strconv.Itoa(i)] = key
+	}
+	attrs["public_keys"] = keys
+
+	return json.Marshal(attrs)
+}
+
+func (ms metadataService) FetchNetworkConfig(filename string) ([]byte, error) {
+	return json.Marshal(Metadata{
+		Interfaces: ms.interfaces,
+		DNS:        ms.dns,
+	})
 }
 
 func (ms metadataService) Type() string {

datasource/metadata/digitalocean/metadata_test.go

@@ -1,26 +1,10 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package digitalocean
 
 import (
+	"bytes"
 	"fmt"
-	"net"
-	"reflect"
 	"testing"
 
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
 	"github.com/coreos/coreos-cloudinit/pkg"
@@ -38,7 +22,7 @@ func TestFetchMetadata(t *testing.T) {
 		root         string
 		metadataPath string
 		resources    map[string]string
-		expect       datasource.Metadata
+		expect       []byte
 		clientErr    error
 		expectErr    error
 	}{
@@ -83,45 +67,17 @@ func TestFetchMetadata(t *testing.T) {
   }
 }`,
 			},
-			expect: datasource.Metadata{
+			expect: []byte(`{"hostname":"","public-ipv4":"192.168.1.2","public-ipv6":"fe00::","public_keys":{"0":"publickey1","1":"publickey2"}}`),
-				PublicIPv4: net.ParseIP("192.168.1.2"),
-				PublicIPv6: net.ParseIP("fe00::"),
-				SSHPublicKeys: map[string]string{
-					"0": "publickey1",
-					"1": "publickey2",
-				},
-				NetworkConfig: Metadata{
-					Interfaces: Interfaces{
-						Public: []Interface{
-							Interface{
-								IPv4: &Address{
-									IPAddress: "192.168.1.2",
-									Netmask:   "255.255.255.0",
-									Gateway:   "192.168.1.1",
-								},
-								IPv6: &Address{
-									IPAddress: "fe00::",
-									Cidr:      126,
-									Gateway:   "fe00::",
-								},
-								MAC:  "ab:cd:ef:gh:ij",
-								Type: "public",
-							},
-						},
-					},
-					PublicKeys: []string{"publickey1", "publickey2"},
-				},
-			},
 		},
 		{
-			clientErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
+			clientErr: pkg.ErrTimeout{fmt.Errorf("test error")},
-			expectErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
+			expectErr: pkg.ErrTimeout{fmt.Errorf("test error")},
 		},
 	} {
 		service := &metadataService{
 			MetadataService: metadata.MetadataService{
 				Root:         tt.root,
-				Client:       &test.HttpClient{Resources: tt.resources, Err: tt.clientErr},
+				Client:       &test.HttpClient{tt.resources, tt.clientErr},
 				MetadataPath: tt.metadataPath,
 			},
 		}
@@ -129,8 +85,8 @@ func TestFetchMetadata(t *testing.T) {
 		if Error(err) != Error(tt.expectErr) {
 			t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
 		}
-		if !reflect.DeepEqual(tt.expect, metadata) {
+		if !bytes.Equal(metadata, tt.expect) {
-			t.Fatalf("bad fetch (%q): want %#q, got %#q", tt.resources, tt.expect, metadata)
+			t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
 		}
 	}
 }

datasource/metadata/ec2/metadata.go

@@ -1,28 +1,12 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package ec2
 
 import (
 	"bufio"
 	"bytes"
+	"encoding/json"
 	"fmt"
-	"log"
-	"net"
 	"strings"
 
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 	"github.com/coreos/coreos-cloudinit/pkg"
 )
@@ -42,51 +26,59 @@ func NewDatasource(root string) *metadataService {
 	return &metadataService{metadata.NewDatasource(root, apiVersion, userdataPath, metadataPath)}
 }
 
-func (ms metadataService) FetchMetadata() (datasource.Metadata, error) {
+func (ms metadataService) FetchMetadata() ([]byte, error) {
-	metadata := datasource.Metadata{}
+	attrs := make(map[string]interface{})
-
 	if keynames, err := ms.fetchAttributes(fmt.Sprintf("%s/public-keys", ms.MetadataUrl())); err == nil {
 		keyIDs := make(map[string]string)
 		for _, keyname := range keynames {
 			tokens := strings.SplitN(keyname, "=", 2)
 			if len(tokens) != 2 {
-				return metadata, fmt.Errorf("malformed public key: %q", keyname)
+				return nil, fmt.Errorf("malformed public key: %q", keyname)
 			}
 			keyIDs[tokens[1]] = tokens[0]
 		}
 
-		metadata.SSHPublicKeys = map[string]string{}
+		keys := make(map[string]string)
 		for name, id := range keyIDs {
 			sshkey, err := ms.fetchAttribute(fmt.Sprintf("%s/public-keys/%s/openssh-key", ms.MetadataUrl(), id))
 			if err != nil {
-				return metadata, err
+				return nil, err
 			}
-			metadata.SSHPublicKeys[name] = sshkey
+			keys[name] = sshkey
-			log.Printf("Found SSH key for %q\n", name)
+			fmt.Printf("Found SSH key for %q\n", name)
 		}
+		attrs["public_keys"] = keys
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return metadata, err
+		return nil, err
 	}
 
 	if hostname, err := ms.fetchAttribute(fmt.Sprintf("%s/hostname", ms.MetadataUrl())); err == nil {
-		metadata.Hostname = strings.Split(hostname, " ")[0]
+		attrs["hostname"] = hostname
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return metadata, err
+		return nil, err
 	}
 
 	if localAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/local-ipv4", ms.MetadataUrl())); err == nil {
-		metadata.PrivateIPv4 = net.ParseIP(localAddr)
+		attrs["local-ipv4"] = localAddr
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return metadata, err
+		return nil, err
 	}
 
 	if publicAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/public-ipv4", ms.MetadataUrl())); err == nil {
-		metadata.PublicIPv4 = net.ParseIP(publicAddr)
+		attrs["public-ipv4"] = publicAddr
 	} else if _, ok := err.(pkg.ErrNotFound); !ok {
-		return metadata, err
+		return nil, err
 	}
 
-	return metadata, nil
+	if content_path, err := ms.fetchAttribute(fmt.Sprintf("%s/network_config/content_path", ms.MetadataUrl())); err == nil {
+		attrs["network_config"] = map[string]string{
+			"content_path": content_path,
+		}
+	} else if _, ok := err.(pkg.ErrNotFound); !ok {
+		return nil, err
+	}
+
+	return json.Marshal(attrs)
 }
 
 func (ms metadataService) Type() string {

datasource/metadata/ec2/metadata_test.go

@@ -1,26 +1,11 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package ec2
 
 import (
+	"bytes"
 	"fmt"
-	"net"
 	"reflect"
 	"testing"
 
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata"
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
 	"github.com/coreos/coreos-cloudinit/pkg"
@@ -73,7 +58,7 @@ func TestFetchAttributes(t *testing.T) {
 		},
 	} {
 		service := metadataService{metadata.MetadataService{
-			Client: &test.HttpClient{Resources: s.resources, Err: s.err},
+			Client: &test.HttpClient{s.resources, s.err},
 		}}
 		for _, tt := range s.tests {
 			attrs, err := service.fetchAttributes(tt.path)
@@ -127,7 +112,7 @@ func TestFetchAttribute(t *testing.T) {
 		},
 	} {
 		service := metadataService{metadata.MetadataService{
-			Client: &test.HttpClient{Resources: s.resources, Err: s.err},
+			Client: &test.HttpClient{s.resources, s.err},
 		}}
 		for _, tt := range s.tests {
 			attr, err := service.fetchAttribute(tt.path)
@@ -146,7 +131,7 @@ func TestFetchMetadata(t *testing.T) {
 		root         string
 		metadataPath string
 		resources    map[string]string
-		expect       datasource.Metadata
+		expect       []byte
 		clientErr    error
 		expectErr    error
 	}{
@@ -162,54 +147,32 @@ func TestFetchMetadata(t *testing.T) {
 			root:         "/",
 			metadataPath: "2009-04-04/meta-data",
 			resources: map[string]string{
-				"/2009-04-04/meta-data/hostname":                  "host",
+				"/2009-04-04/meta-data/hostname":                    "host",
-				"/2009-04-04/meta-data/local-ipv4":                "1.2.3.4",
+				"/2009-04-04/meta-data/local-ipv4":                  "1.2.3.4",
-				"/2009-04-04/meta-data/public-ipv4":               "5.6.7.8",
+				"/2009-04-04/meta-data/public-ipv4":                 "5.6.7.8",
-				"/2009-04-04/meta-data/public-keys":               "0=test1\n",
+				"/2009-04-04/meta-data/public-keys":                 "0=test1\n",
-				"/2009-04-04/meta-data/public-keys/0":             "openssh-key",
+				"/2009-04-04/meta-data/public-keys/0":               "openssh-key",
-				"/2009-04-04/meta-data/public-keys/0/openssh-key": "key",
+				"/2009-04-04/meta-data/public-keys/0/openssh-key":   "key",
-			},
+				"/2009-04-04/meta-data/network_config/content_path": "path",
-			expect: datasource.Metadata{
-				Hostname:      "host",
-				PrivateIPv4:   net.ParseIP("1.2.3.4"),
-				PublicIPv4:    net.ParseIP("5.6.7.8"),
-				SSHPublicKeys: map[string]string{"test1": "key"},
 			},
+			expect: []byte(`{"hostname":"host","local-ipv4":"1.2.3.4","network_config":{"content_path":"path"},"public-ipv4":"5.6.7.8","public_keys":{"test1":"key"}}`),
 		},
 		{
-			root:         "/",
+			clientErr: pkg.ErrTimeout{fmt.Errorf("test error")},
-			metadataPath: "2009-04-04/meta-data",
+			expectErr: pkg.ErrTimeout{fmt.Errorf("test error")},
-			resources: map[string]string{
-				"/2009-04-04/meta-data/hostname":                  "host domain another_domain",
-				"/2009-04-04/meta-data/local-ipv4":                "1.2.3.4",
-				"/2009-04-04/meta-data/public-ipv4":               "5.6.7.8",
-				"/2009-04-04/meta-data/public-keys":               "0=test1\n",
-				"/2009-04-04/meta-data/public-keys/0":             "openssh-key",
-				"/2009-04-04/meta-data/public-keys/0/openssh-key": "key",
-			},
-			expect: datasource.Metadata{
-				Hostname:      "host",
-				PrivateIPv4:   net.ParseIP("1.2.3.4"),
-				PublicIPv4:    net.ParseIP("5.6.7.8"),
-				SSHPublicKeys: map[string]string{"test1": "key"},
-			},
-		},
-		{
-			clientErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
-			expectErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
 		},
 	} {
 		service := &metadataService{metadata.MetadataService{
 			Root:         tt.root,
-			Client:       &test.HttpClient{Resources: tt.resources, Err: tt.clientErr},
+			Client:       &test.HttpClient{tt.resources, tt.clientErr},
 			MetadataPath: tt.metadataPath,
 		}}
 		metadata, err := service.FetchMetadata()
 		if Error(err) != Error(tt.expectErr) {
 			t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
 		}
-		if !reflect.DeepEqual(tt.expect, metadata) {
+		if !bytes.Equal(metadata, tt.expect) {
-			t.Fatalf("bad fetch (%q): want %#v, got %#v", tt.resources, tt.expect, metadata)
+			t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
 		}
 	}
 }

datasource/metadata/metadata.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package metadata
 
 import (
@@ -52,6 +38,10 @@ func (ms MetadataService) FetchUserdata() ([]byte, error) {
 	return ms.FetchData(ms.UserdataUrl())
 }
 
+func (ms MetadataService) FetchNetworkConfig(filename string) ([]byte, error) {
+	return nil, nil
+}
+
 func (ms MetadataService) FetchData(url string) ([]byte, error) {
 	if data, err := ms.Client.GetRetry(url); err == nil {
 		return data, err

datasource/metadata/metadata_test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package metadata
 
 import (
@@ -26,7 +12,7 @@ import (
 func TestAvailabilityChanges(t *testing.T) {
 	want := true
 	if ac := (MetadataService{}).AvailabilityChanges(); ac != want {
-		t.Fatalf("bad AvailabilityChanges: want %t, got %t", want, ac)
+		t.Fatalf("bad AvailabilityChanges: want %q, got %q", want, ac)
 	}
 }
 
@@ -53,11 +39,11 @@ func TestIsAvailable(t *testing.T) {
 	} {
 		service := &MetadataService{
 			Root:       tt.root,
-			Client:     &test.HttpClient{Resources: tt.resources, Err: nil},
+			Client:     &test.HttpClient{tt.resources, nil},
 			ApiVersion: tt.apiVersion,
 		}
 		if a := service.IsAvailable(); a != tt.expect {
-			t.Fatalf("bad isAvailable (%q): want %t, got %t", tt.resources, tt.expect, a)
+			t.Fatalf("bad isAvailable (%q): want %q, got %q", tt.resources, tt.expect, a)
 		}
 	}
 }
@@ -81,18 +67,18 @@ func TestFetchUserdata(t *testing.T) {
 		},
 		{
 			root:      "/",
-			clientErr: pkg.ErrNotFound{Err: fmt.Errorf("test not found error")},
+			clientErr: pkg.ErrNotFound{fmt.Errorf("test not found error")},
 			userdata:  []byte{},
 		},
 		{
 			root:      "/",
-			clientErr: pkg.ErrTimeout{Err: fmt.Errorf("test timeout error")},
+			clientErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
-			expectErr: pkg.ErrTimeout{Err: fmt.Errorf("test timeout error")},
+			expectErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
 		},
 	} {
 		service := &MetadataService{
 			Root:         tt.root,
-			Client:       &test.HttpClient{Resources: tt.resources, Err: tt.clientErr},
+			Client:       &test.HttpClient{tt.resources, tt.clientErr},
 			UserdataPath: tt.userdataPath,
 		}
 		data, err := service.FetchUserdata()

datasource/metadata/openstack/metadata.go

@@ -1,112 +0,0 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
-
-package openstack
-
-import (
-	"encoding/json"
-	"net"
-	"strconv"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
-	"github.com/coreos/coreos-cloudinit/datasource/metadata"
-)
-
-const (
-	DefaultAddress = "http://169.254.169.254/"
-	apiVersion     = "openstack/latest"
-	userdataUrl    = apiVersion + "/user_data"
-	metadataPath   = apiVersion + "/meta_data.json"
-)
-
-type Address struct {
-	IPAddress string `json:"ip_address"`
-	Netmask   string `json:"netmask"`
-	Cidr      int    `json:"cidr"`
-	Gateway   string `json:"gateway"`
-}
-
-type Interface struct {
-	IPv4 *Address `json:"ipv4"`
-	IPv6 *Address `json:"ipv6"`
-	MAC  string   `json:"mac"`
-	Type string   `json:"type"`
-}
-
-type Interfaces struct {
-	Public  []Interface `json:"public"`
-	Private []Interface `json:"private"`
-}
-
-type DNS struct {
-	Nameservers []string `json:"nameservers"`
-}
-
-type Metadata struct {
-	Hostname   string            `json:"hostname"`
-	Interfaces Interfaces        `json:"interfaces"`
-	PublicKeys map[string]string `json:"public_keys"`
-	DNS        DNS               `json:"dns"`
-}
-
-type metadataService struct {
-	metadata.MetadataService
-}
-
-func NewDatasource(root string) *metadataService {
-	return &metadataService{MetadataService: metadata.NewDatasource(root, apiVersion, userdataUrl, metadataPath)}
-}
-
-func (ms *metadataService) FetchMetadata() (metadata datasource.Metadata, err error) {
-	var data []byte
-	var m Metadata
-
-	if data, err = ms.FetchData(ms.MetadataUrl()); err != nil || len(data) == 0 {
-		return
-	}
-
-	if err = json.Unmarshal(data, &m); err != nil {
-		return
-	}
-
-	if len(m.Interfaces.Public) > 0 {
-		if m.Interfaces.Public[0].IPv4 != nil {
-			metadata.PublicIPv4 = net.ParseIP(m.Interfaces.Public[0].IPv4.IPAddress)
-		}
-		if m.Interfaces.Public[0].IPv6 != nil {
-			metadata.PublicIPv6 = net.ParseIP(m.Interfaces.Public[0].IPv6.IPAddress)
-		}
-	}
-	if len(m.Interfaces.Private) > 0 {
-		if m.Interfaces.Private[0].IPv4 != nil {
-			metadata.PrivateIPv4 = net.ParseIP(m.Interfaces.Private[0].IPv4.IPAddress)
-		}
-		if m.Interfaces.Private[0].IPv6 != nil {
-			metadata.PrivateIPv6 = net.ParseIP(m.Interfaces.Private[0].IPv6.IPAddress)
-		}
-	}
-
-	metadata.Hostname = m.Hostname
-	metadata.SSHPublicKeys = map[string]string{}
-	metadata.SSHPublicKeys[strconv.Itoa(0)] = m.PublicKeys["root"]
-	metadata.NetworkConfig = data
-
-	return
-}
-
-func (ms metadataService) Type() string {
-	return "openstack-metadata-service"
-}

datasource/metadata/openstack/metadata_test.go

@@ -1,115 +0,0 @@
-/*
-   Copyright 2014 CoreOS, Inc.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-*/
-
-package openstack
-
-import (
-	"bytes"
-	"fmt"
-	"testing"
-
-	"github.com/coreos/coreos-cloudinit/datasource/metadata"
-	"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
-	"github.com/coreos/coreos-cloudinit/pkg"
-)
-
-func TestType(t *testing.T) {
-	want := "openstack-metadata-service"
-	if kind := (metadataService{}).Type(); kind != want {
-		t.Fatalf("bad type: want %q, got %q", want, kind)
-	}
-}
-
-func TestFetchMetadata(t *testing.T) {
-	for _, tt := range []struct {
-		root         string
-		metadataPath string
-		resources    map[string]string
-		expect       []byte
-		clientErr    error
-		expectErr    error
-	}{
-		{
-			root:         "/",
-			metadataPath: "v1.json",
-			resources: map[string]string{
-				"/v1.json": "bad",
-			},
-			expectErr: fmt.Errorf("invalid character 'b' looking for beginning of value"),
-		},
-		{
-			root:         "/",
-			metadataPath: "v1.json",
-			resources: map[string]string{
-				"/v1.json": `{
-  "droplet_id": 1,
-  "user_data": "hello",
-  "vendor_data": "hello",
-  "public_keys": [
-    "publickey1",
-    "publickey2"
-  ],
-  "region": "nyc2",
-  "interfaces": {
-    "public": [
-      {
-        "ipv4": {
-          "ip_address": "192.168.1.2",
-          "netmask": "255.255.255.0",
-          "gateway": "192.168.1.1"
-        },
-        "ipv6": {
-          "ip_address": "fe00::",
-          "cidr": 126,
-          "gateway": "fe00::"
-        },
-        "mac": "ab:cd:ef:gh:ij",
-        "type": "public"
-      }
-    ]
-  }
-}`,
-			},
-			expect: []byte(`{"hostname":"","public-ipv4":"192.168.1.2","public-ipv6":"fe00::","public_keys":{"0":"publickey1","1":"publickey2"}}`),
-		},
-		{
-			clientErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
-			expectErr: pkg.ErrTimeout{Err: fmt.Errorf("test error")},
-		},
-	} {
-		service := &metadataService{
-			MetadataService: metadata.MetadataService{
-				Root:         tt.root,
-				Client:       &test.HttpClient{Resources: tt.resources, Err: tt.clientErr},
-				MetadataPath: tt.metadataPath,
-			},
-		}
-		metadata, err := service.FetchMetadata()
-		if Error(err) != Error(tt.expectErr) {
-			t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
-		}
-		if !bytes.Equal(metadata, tt.expect) {
-			t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
-		}
-	}
-}
-
-func Error(err error) string {
-	if err != nil {
-		return err.Error()
-	}
-	return ""
-}

datasource/metadata/packet/metadata.go

@@ -1,106 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package packet
-
-import (
-	"encoding/json"
-	"net"
-	"strconv"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
-	"github.com/coreos/coreos-cloudinit/datasource/metadata"
-)
-
-const (
-	DefaultAddress = "https://metadata.packet.net/"
-	apiVersion     = ""
-	userdataUrl    = "userdata"
-	metadataPath   = "metadata"
-)
-
-type Netblock struct {
-	Address       net.IP `json:"address"`
-	Cidr          int    `json:"cidr"`
-	Netmask       net.IP `json:"netmask"`
-	Gateway       net.IP `json:"gateway"`
-	AddressFamily int    `json:"address_family"`
-	Public        bool   `json:"public"`
-}
-
-type Nic struct {
-	Name string `json:"name"`
-	Mac  string `json:"mac"`
-}
-
-type NetworkData struct {
-	Interfaces []Nic      `json:"interfaces"`
-	Netblocks  []Netblock `json:"addresses"`
-	DNS        []net.IP   `json:"dns"`
-}
-
-// Metadata that will be pulled from the https://metadata.packet.net/metadata only. We have the opportunity to add more later.
-type Metadata struct {
-	Hostname    string      `json:"hostname"`
-	SSHKeys     []string    `json:"ssh_keys"`
-	NetworkData NetworkData `json:"network"`
-}
-
-type metadataService struct {
-	metadata.MetadataService
-}
-
-func NewDatasource(root string) *metadataService {
-	return &metadataService{MetadataService: metadata.NewDatasource(root, apiVersion, userdataUrl, metadataPath)}
-}
-
-func (ms *metadataService) FetchMetadata() (metadata datasource.Metadata, err error) {
-	var data []byte
-	var m Metadata
-
-	if data, err = ms.FetchData(ms.MetadataUrl()); err != nil || len(data) == 0 {
-		return
-	}
-
-	if err = json.Unmarshal(data, &m); err != nil {
-		return
-	}
-
-	if len(m.NetworkData.Netblocks) > 0 {
-		for _, Netblock := range m.NetworkData.Netblocks {
-			if Netblock.AddressFamily == 4 {
-				if Netblock.Public == true {
-					metadata.PublicIPv4 = Netblock.Address
-				} else {
-					metadata.PrivateIPv4 = Netblock.Address
-				}
-			} else {
-				metadata.PublicIPv6 = Netblock.Address
-			}
-		}
-	}
-	metadata.Hostname = m.Hostname
-	metadata.SSHPublicKeys = map[string]string{}
-	for i, key := range m.SSHKeys {
-		metadata.SSHPublicKeys[strconv.Itoa(i)] = key
-	}
-
-	metadata.NetworkConfig = m.NetworkData
-
-	return
-}
-
-func (ms metadataService) Type() string {
-	return "packet-metadata-service"
-}

datasource/metadata/test/test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package test
 
 import (

datasource/proc_cmdline/proc_cmdline.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package proc_cmdline
 
 import (
@@ -20,7 +6,6 @@ import (
 	"log"
 	"strings"
 
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/pkg"
 )
 
@@ -56,8 +41,8 @@ func (c *procCmdline) ConfigRoot() string {
 	return ""
 }
 
-func (c *procCmdline) FetchMetadata() (datasource.Metadata, error) {
+func (c *procCmdline) FetchMetadata() ([]byte, error) {
-	return datasource.Metadata{}, nil
+	return []byte{}, nil
 }
 
 func (c *procCmdline) FetchUserdata() ([]byte, error) {
@@ -81,6 +66,10 @@ func (c *procCmdline) FetchUserdata() ([]byte, error) {
 	return cfg, nil
 }
 
+func (c *procCmdline) FetchNetworkConfig(filename string) ([]byte, error) {
+	return nil, nil
+}
+
 func (c *procCmdline) Type() string {
 	return "proc-cmdline"
 }

datasource/proc_cmdline/proc_cmdline_test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package proc_cmdline
 
 import (

datasource/test/filesystem.go

@@ -1,57 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package test
-
-import (
-	"fmt"
-	"os"
-	"path"
-)
-
-type MockFilesystem map[string]File
-
-type File struct {
-	Path      string
-	Contents  string
-	Directory bool
-}
-
-func (m MockFilesystem) ReadFile(filename string) ([]byte, error) {
-	if f, ok := m[path.Clean(filename)]; ok {
-		if f.Directory {
-			return nil, fmt.Errorf("read %s: is a directory", filename)
-		}
-		return []byte(f.Contents), nil
-	}
-	return nil, os.ErrNotExist
-}
-
-func NewMockFilesystem(files ...File) MockFilesystem {
-	fs := MockFilesystem{}
-	for _, file := range files {
-		fs[file.Path] = file
-
-		// Create the directories leading up to the file
-		p := path.Dir(file.Path)
-		for p != "/" && p != "." {
-			if f, ok := fs[p]; ok && !f.Directory {
-				panic(fmt.Sprintf("%q already exists and is not a directory (%#v)", p, f))
-			}
-			fs[p] = File{Path: p, Directory: true}
-			p = path.Dir(p)
-		}
-	}
-	return fs
-}

datasource/test/filesystem_test.go

@@ -1,115 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package test
-
-import (
-	"errors"
-	"os"
-	"reflect"
-	"testing"
-)
-
-func TestReadFile(t *testing.T) {
-	tests := []struct {
-		filesystem MockFilesystem
-
-		filename string
-		contents string
-		err      error
-	}{
-		{
-			filename: "dne",
-			err:      os.ErrNotExist,
-		},
-		{
-			filesystem: MockFilesystem{
-				"exists": File{Contents: "hi"},
-			},
-			filename: "exists",
-			contents: "hi",
-		},
-		{
-			filesystem: MockFilesystem{
-				"dir": File{Directory: true},
-			},
-			filename: "dir",
-			err:      errors.New("read dir: is a directory"),
-		},
-	}
-
-	for i, tt := range tests {
-		contents, err := tt.filesystem.ReadFile(tt.filename)
-		if tt.contents != string(contents) {
-			t.Errorf("bad contents (test %d): want %q, got %q", i, tt.contents, string(contents))
-		}
-		if !reflect.DeepEqual(tt.err, err) {
-			t.Errorf("bad error (test %d): want %v, got %v", i, tt.err, err)
-		}
-	}
-}
-
-func TestNewMockFilesystem(t *testing.T) {
-	tests := []struct {
-		files []File
-
-		filesystem MockFilesystem
-	}{
-		{
-			filesystem: MockFilesystem{},
-		},
-		{
-			files: []File{File{Path: "file"}},
-			filesystem: MockFilesystem{
-				"file": File{Path: "file"},
-			},
-		},
-		{
-			files: []File{File{Path: "/file"}},
-			filesystem: MockFilesystem{
-				"/file": File{Path: "/file"},
-			},
-		},
-		{
-			files: []File{File{Path: "/dir/file"}},
-			filesystem: MockFilesystem{
-				"/dir":      File{Path: "/dir", Directory: true},
-				"/dir/file": File{Path: "/dir/file"},
-			},
-		},
-		{
-			files: []File{File{Path: "/dir/dir/file"}},
-			filesystem: MockFilesystem{
-				"/dir":          File{Path: "/dir", Directory: true},
-				"/dir/dir":      File{Path: "/dir/dir", Directory: true},
-				"/dir/dir/file": File{Path: "/dir/dir/file"},
-			},
-		},
-		{
-			files: []File{File{Path: "/dir/dir/dir", Directory: true}},
-			filesystem: MockFilesystem{
-				"/dir":         File{Path: "/dir", Directory: true},
-				"/dir/dir":     File{Path: "/dir/dir", Directory: true},
-				"/dir/dir/dir": File{Path: "/dir/dir/dir", Directory: true},
-			},
-		},
-	}
-
-	for i, tt := range tests {
-		filesystem := NewMockFilesystem(tt.files...)
-		if !reflect.DeepEqual(tt.filesystem, filesystem) {
-			t.Errorf("bad filesystem (test %d): want %#v, got %#v", i, tt.filesystem, filesystem)
-		}
-	}
-}

datasource/url/url.go

@@ -1,21 +1,6 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package url
 
 import (
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/pkg"
 )
 
@@ -41,8 +26,8 @@ func (f *remoteFile) ConfigRoot() string {
 	return ""
 }
 
-func (f *remoteFile) FetchMetadata() (datasource.Metadata, error) {
+func (f *remoteFile) FetchMetadata() ([]byte, error) {
-	return datasource.Metadata{}, nil
+	return []byte{}, nil
 }
 
 func (f *remoteFile) FetchUserdata() ([]byte, error) {
@@ -50,6 +35,10 @@ func (f *remoteFile) FetchUserdata() ([]byte, error) {
 	return client.GetRetry(f.url)
 }
 
+func (f *remoteFile) FetchNetworkConfig(filename string) ([]byte, error) {
+	return nil, nil
+}
+
 func (f *remoteFile) Type() string {
 	return "url"
 }

datasource/vmware/vmware.go

@@ -1,183 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package vmware
-
-import (
-	"fmt"
-	"log"
-	"net"
-
-	"github.com/coreos/coreos-cloudinit/config"
-	"github.com/coreos/coreos-cloudinit/datasource"
-	"github.com/coreos/coreos-cloudinit/pkg"
-
-	"github.com/sigma/vmw-guestinfo/rpcvmx"
-	"github.com/sigma/vmw-guestinfo/vmcheck"
-)
-
-type readConfigFunction func(key string) (string, error)
-type urlDownloadFunction func(url string) ([]byte, error)
-
-type vmware struct {
-	readConfig  readConfigFunction
-	urlDownload urlDownloadFunction
-}
-
-func NewDatasource() *vmware {
-	return &vmware{
-		readConfig:  readConfig,
-		urlDownload: urlDownload,
-	}
-}
-
-func (v vmware) IsAvailable() bool {
-	return vmcheck.IsVirtualWorld()
-}
-
-func (v vmware) AvailabilityChanges() bool {
-	return false
-}
-
-func (v vmware) ConfigRoot() string {
-	return "/"
-}
-
-func (v vmware) FetchMetadata() (metadata datasource.Metadata, err error) {
-	metadata.Hostname, _ = v.readConfig("hostname")
-
-	netconf := map[string]string{}
-	saveConfig := func(key string, args ...interface{}) string {
-		key = fmt.Sprintf(key, args...)
-		val, _ := v.readConfig(key)
-		if val != "" {
-			netconf[key] = val
-		}
-		return val
-	}
-
-	for i := 0; ; i++ {
-		if nameserver := saveConfig("dns.server.%d", i); nameserver == "" {
-			break
-		}
-	}
-
-	found := true
-	for i := 0; found; i++ {
-		found = false
-
-		found = (saveConfig("interface.%d.name", i) != "") || found
-		found = (saveConfig("interface.%d.mac", i) != "") || found
-		found = (saveConfig("interface.%d.dhcp", i) != "") || found
-
-		role, _ := v.readConfig(fmt.Sprintf("interface.%d.role", i))
-		for a := 0; ; a++ {
-			address := saveConfig("interface.%d.ip.%d.address", i, a)
-			if address == "" {
-				break
-			} else {
-				found = true
-			}
-
-			ip, _, err := net.ParseCIDR(address)
-			if err != nil {
-				return metadata, err
-			}
-
-			switch role {
-			case "public":
-				if ip.To4() != nil {
-					metadata.PublicIPv4 = ip
-				} else {
-					metadata.PublicIPv6 = ip
-				}
-			case "private":
-				if ip.To4() != nil {
-					metadata.PrivateIPv4 = ip
-				} else {
-					metadata.PrivateIPv6 = ip
-				}
-			case "":
-			default:
-				return metadata, fmt.Errorf("unrecognized role: %q", role)
-			}
-		}
-
-		for r := 0; ; r++ {
-			gateway := saveConfig("interface.%d.route.%d.gateway", i, r)
-			destination := saveConfig("interface.%d.route.%d.destination", i, r)
-
-			if gateway == "" && destination == "" {
-				break
-			} else {
-				found = true
-			}
-		}
-	}
-	metadata.NetworkConfig = netconf
-
-	return
-}
-
-func (v vmware) FetchUserdata() ([]byte, error) {
-	encoding, err := v.readConfig("coreos.config.data.encoding")
-	if err != nil {
-		return nil, err
-	}
-
-	data, err := v.readConfig("coreos.config.data")
-	if err != nil {
-		return nil, err
-	}
-
-	// Try to fallback to url if no explicit data
-	if data == "" {
-		url, err := v.readConfig("coreos.config.url")
-		if err != nil {
-			return nil, err
-		}
-
-		if url != "" {
-			rawData, err := v.urlDownload(url)
-			if err != nil {
-				return nil, err
-			}
-			data = string(rawData)
-		}
-	}
-
-	if encoding != "" {
-		return config.DecodeContent(data, encoding)
-	}
-	return []byte(data), nil
-}
-
-func (v vmware) Type() string {
-	return "vmware"
-}
-
-func urlDownload(url string) ([]byte, error) {
-	client := pkg.NewHttpClient()
-	return client.GetRetry(url)
-}
-
-func readConfig(key string) (string, error) {
-	data, err := rpcvmx.NewConfig().GetString(key, "")
-	if err == nil {
-		log.Printf("Read from %q: %q\n", key, data)
-	} else {
-		log.Printf("Failed to read from %q: %v\n", key, err)
-	}
-	return data, err
-}

datasource/vmware/vmware_test.go

@@ -1,216 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package vmware
-
-import (
-	"errors"
-	"net"
-	"reflect"
-	"testing"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
-)
-
-type MockHypervisor map[string]string
-
-func (h MockHypervisor) ReadConfig(key string) (string, error) {
-	return h[key], nil
-}
-
-func TestFetchMetadata(t *testing.T) {
-	tests := []struct {
-		variables MockHypervisor
-
-		metadata datasource.Metadata
-		err      error
-	}{
-		{
-			variables: map[string]string{
-				"interface.0.mac":  "test mac",
-				"interface.0.dhcp": "yes",
-			},
-			metadata: datasource.Metadata{
-				NetworkConfig: map[string]string{
-					"interface.0.mac":  "test mac",
-					"interface.0.dhcp": "yes",
-				},
-			},
-		},
-		{
-			variables: map[string]string{
-				"interface.0.name": "test name",
-				"interface.0.dhcp": "yes",
-			},
-			metadata: datasource.Metadata{
-				NetworkConfig: map[string]string{
-					"interface.0.name": "test name",
-					"interface.0.dhcp": "yes",
-				},
-			},
-		},
-		{
-			variables: map[string]string{
-				"hostname":                        "test host",
-				"interface.0.mac":                 "test mac",
-				"interface.0.role":                "private",
-				"interface.0.ip.0.address":        "fe00::100/64",
-				"interface.0.route.0.gateway":     "fe00::1",
-				"interface.0.route.0.destination": "::",
-			},
-			metadata: datasource.Metadata{
-				Hostname:    "test host",
-				PrivateIPv6: net.ParseIP("fe00::100"),
-				NetworkConfig: map[string]string{
-					"interface.0.mac":                 "test mac",
-					"interface.0.ip.0.address":        "fe00::100/64",
-					"interface.0.route.0.gateway":     "fe00::1",
-					"interface.0.route.0.destination": "::",
-				},
-			},
-		},
-		{
-			variables: map[string]string{
-				"hostname":                        "test host",
-				"interface.0.name":                "test name",
-				"interface.0.role":                "public",
-				"interface.0.ip.0.address":        "10.0.0.100/24",
-				"interface.0.ip.1.address":        "10.0.0.101/24",
-				"interface.0.route.0.gateway":     "10.0.0.1",
-				"interface.0.route.0.destination": "0.0.0.0",
-				"interface.1.mac":                 "test mac",
-				"interface.1.role":                "private",
-				"interface.1.route.0.gateway":     "10.0.0.2",
-				"interface.1.route.0.destination": "0.0.0.0",
-				"interface.1.ip.0.address":        "10.0.0.102/24",
-			},
-			metadata: datasource.Metadata{
-				Hostname:    "test host",
-				PublicIPv4:  net.ParseIP("10.0.0.101"),
-				PrivateIPv4: net.ParseIP("10.0.0.102"),
-				NetworkConfig: map[string]string{
-					"interface.0.name":                "test name",
-					"interface.0.ip.0.address":        "10.0.0.100/24",
-					"interface.0.ip.1.address":        "10.0.0.101/24",
-					"interface.0.route.0.gateway":     "10.0.0.1",
-					"interface.0.route.0.destination": "0.0.0.0",
-					"interface.1.mac":                 "test mac",
-					"interface.1.route.0.gateway":     "10.0.0.2",
-					"interface.1.route.0.destination": "0.0.0.0",
-					"interface.1.ip.0.address":        "10.0.0.102/24",
-				},
-			},
-		},
-	}
-
-	for i, tt := range tests {
-		v := vmware{readConfig: tt.variables.ReadConfig}
-		metadata, err := v.FetchMetadata()
-		if !reflect.DeepEqual(tt.err, err) {
-			t.Errorf("bad error (#%d): want %v, got %v", i, tt.err, err)
-		}
-		if !reflect.DeepEqual(tt.metadata, metadata) {
-			t.Errorf("bad metadata (#%d): want %#v, got %#v", i, tt.metadata, metadata)
-		}
-	}
-}
-
-func TestFetchUserdata(t *testing.T) {
-	tests := []struct {
-		variables MockHypervisor
-
-		userdata string
-		err      error
-	}{
-		{},
-		{
-			variables: map[string]string{"coreos.config.data": "test config"},
-			userdata:  "test config",
-		},
-		{
-			variables: map[string]string{
-				"coreos.config.data.encoding": "",
-				"coreos.config.data":          "test config",
-			},
-			userdata: "test config",
-		},
-		{
-			variables: map[string]string{
-				"coreos.config.data.encoding": "base64",
-				"coreos.config.data":          "dGVzdCBjb25maWc=",
-			},
-			userdata: "test config",
-		},
-		{
-			variables: map[string]string{
-				"coreos.config.data.encoding": "gzip+base64",
-				"coreos.config.data":          "H4sIABaoWlUAAytJLS5RSM7PS8tMBwCQiHNZCwAAAA==",
-			},
-			userdata: "test config",
-		},
-		{
-			variables: map[string]string{
-				"coreos.config.data.encoding": "test encoding",
-			},
-			err: errors.New(`Unsupported encoding "test encoding"`),
-		},
-		{
-			variables: map[string]string{
-				"coreos.config.url": "http://good.example.com",
-			},
-			userdata: "test config",
-		},
-		{
-			variables: map[string]string{
-				"coreos.config.url": "http://bad.example.com",
-			},
-			err: errors.New("Not found"),
-		},
-	}
-
-	var downloader urlDownloadFunction = func(url string) ([]byte, error) {
-		mapping := map[string]struct {
-			data []byte
-			err  error
-		}{
-			"http://good.example.com": {[]byte("test config"), nil},
-			"http://bad.example.com":  {nil, errors.New("Not found")},
-		}
-		val := mapping[url]
-		return val.data, val.err
-	}
-
-	for i, tt := range tests {
-		v := vmware{
-			readConfig:  tt.variables.ReadConfig,
-			urlDownload: downloader,
-		}
-		userdata, err := v.FetchUserdata()
-		if !reflect.DeepEqual(tt.err, err) {
-			t.Errorf("bad error (#%d): want %v, got %v", i, tt.err, err)
-		}
-		if tt.userdata != string(userdata) {
-			t.Errorf("bad userdata (#%d): want %q, got %q", i, tt.userdata, userdata)
-		}
-	}
-}
-
-func TestFetchUserdataError(t *testing.T) {
-	testErr := errors.New("test error")
-	_, err := vmware{readConfig: func(_ string) (string, error) { return "", testErr }}.FetchUserdata()
-
-	if testErr != err {
-		t.Errorf("bad error: want %v, got %v", testErr, err)
-	}
-}

datasource/waagent/waagent.go

@@ -1,28 +1,13 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package waagent
 
 import (
+	"encoding/json"
 	"encoding/xml"
+	"fmt"
 	"io/ioutil"
-	"log"
 	"net"
 	"os"
 	"path"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
 )
 
 type waagent struct {
@@ -47,13 +32,13 @@ func (a *waagent) ConfigRoot() string {
 	return a.root
 }
 
-func (a *waagent) FetchMetadata() (metadata datasource.Metadata, err error) {
+func (a *waagent) FetchMetadata() ([]byte, error) {
-	var metadataBytes []byte
+	metadataBytes, err := a.tryReadFile(path.Join(a.root, "SharedConfig.xml"))
-	if metadataBytes, err = a.tryReadFile(path.Join(a.root, "SharedConfig.xml")); err != nil {
+	if err != nil {
-		return
+		return nil, err
 	}
 	if len(metadataBytes) == 0 {
-		return
+		return metadataBytes, nil
 	}
 
 	type Instance struct {
@@ -75,40 +60,46 @@ func (a *waagent) FetchMetadata() (metadata datasource.Metadata, err error) {
 		}
 	}
 
-	var m SharedConfig
+	var metadata SharedConfig
-	if err = xml.Unmarshal(metadataBytes, &m); err != nil {
+	if err := xml.Unmarshal(metadataBytes, &metadata); err != nil {
-		return
+		return nil, err
 	}
 
 	var instance Instance
-	for _, i := range m.Instances.Instances {
+	for _, i := range metadata.Instances.Instances {
-		if i.Id == m.Incarnation.Instance {
+		if i.Id == metadata.Incarnation.Instance {
 			instance = i
 			break
 		}
 	}
 
-	metadata.PrivateIPv4 = net.ParseIP(instance.Address)
+	attrs := map[string]string{
+		"local-ipv4": instance.Address,
+	}
 	for _, e := range instance.InputEndpoints.Endpoints {
 		host, _, err := net.SplitHostPort(e.LoadBalancedPublicAddress)
 		if err == nil {
-			metadata.PublicIPv4 = net.ParseIP(host)
+			attrs["public-ipv4"] = host
 			break
 		}
 	}
-	return
+	return json.Marshal(attrs)
 }
 
 func (a *waagent) FetchUserdata() ([]byte, error) {
 	return a.tryReadFile(path.Join(a.root, "CustomData"))
 }
 
+func (a *waagent) FetchNetworkConfig(filename string) ([]byte, error) {
+	return nil, nil
+}
+
 func (a *waagent) Type() string {
 	return "waagent"
 }
 
 func (a *waagent) tryReadFile(filename string) ([]byte, error) {
-	log.Printf("Attempting to read from %q\n", filename)
+	fmt.Printf("Attempting to read from %q\n", filename)
 	data, err := a.readFile(filename)
 	if os.IsNotExist(err) {
 		err = nil

datasource/waagent/waagent_test.go

@@ -1,49 +1,45 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package waagent
 
 import (
-	"net"
+	"encoding/json"
+	"os"
 	"reflect"
 	"testing"
-
-	"github.com/coreos/coreos-cloudinit/datasource"
-	"github.com/coreos/coreos-cloudinit/datasource/test"
 )
 
+type mockFilesystem map[string][]byte
+
+func (m mockFilesystem) readFile(filename string) ([]byte, error) {
+	if contents := m[filename]; contents != nil {
+		return contents, nil
+	}
+	return nil, os.ErrNotExist
+}
+
 func TestFetchMetadata(t *testing.T) {
 	for _, tt := range []struct {
 		root     string
-		files    test.MockFilesystem
+		files    mockFilesystem
-		metadata datasource.Metadata
+		metadata map[string]string
 	}{
 		{
-			root:  "/",
+			"/",
-			files: test.NewMockFilesystem(),
+			mockFilesystem{},
+			nil,
 		},
 		{
-			root:  "/",
+			"/",
-			files: test.NewMockFilesystem(test.File{Path: "/SharedConfig.xml", Contents: ""}),
+			mockFilesystem{"/SharedConfig.xml": []byte("")},
+			nil,
 		},
 		{
-			root:  "/var/lib/waagent",
+			"/var/lib/waagent",
-			files: test.NewMockFilesystem(test.File{Path: "/var/lib/waagent/SharedConfig.xml", Contents: ""}),
+			mockFilesystem{"/var/lib/waagent/SharedConfig.xml": []byte("")},
+			nil,
 		},
 		{
-			root: "/var/lib/waagent",
+			"/var/lib/waagent",
-			files: test.NewMockFilesystem(test.File{Path: "/var/lib/waagent/SharedConfig.xml", Contents: `<?xml version="1.0" encoding="utf-8"?>
+			mockFilesystem{"/var/lib/waagent/SharedConfig.xml": []byte(`<?xml version="1.0" encoding="utf-8"?>
 <SharedConfig version="1.0.0.0" goalStateIncarnation="1">
   <Deployment name="c8f9e4c9c18948e1bebf57c5685da756" guid="{1d10394f-c741-4a1a-a6bb-278f213c5a5e}" incarnation="0" isNonCancellableTopologyChangeEnabled="false">
     <Service name="core-test-1" guid="{00000000-0000-0000-0000-000000000000}" />
@@ -79,20 +75,26 @@ func TestFetchMetadata(t *testing.T) {
       </InputEndpoints>
     </Instance>
   </Instances>
-</SharedConfig>`}),
+</SharedConfig>`)},
-			metadata: datasource.Metadata{
+			map[string]string{
-				PrivateIPv4: net.ParseIP("100.73.202.64"),
+				"local-ipv4":  "100.73.202.64",
-				PublicIPv4:  net.ParseIP("191.239.39.77"),
+				"public-ipv4": "191.239.39.77",
 			},
 		},
 	} {
-		a := waagent{tt.root, tt.files.ReadFile}
+		a := waagent{tt.root, tt.files.readFile}
-		metadata, err := a.FetchMetadata()
+		metadataBytes, err := a.FetchMetadata()
 		if err != nil {
-			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %q: want %q, got %q", tt, nil, err)
+		}
+		var metadata map[string]string
+		if len(metadataBytes) > 0 {
+			if err := json.Unmarshal(metadataBytes, &metadata); err != nil {
+				panic(err)
+			}
 		}
 		if !reflect.DeepEqual(tt.metadata, metadata) {
-			t.Fatalf("bad metadata for %+v: want %#v, got %#v", tt, tt.metadata, metadata)
+			t.Fatalf("bad metadata for %q: want %q, got %q", tt, tt.metadata, metadata)
 		}
 	}
 }
@@ -100,25 +102,25 @@ func TestFetchMetadata(t *testing.T) {
 func TestFetchUserdata(t *testing.T) {
 	for _, tt := range []struct {
 		root  string
-		files test.MockFilesystem
+		files mockFilesystem
 	}{
 		{
 			"/",
-			test.NewMockFilesystem(),
+			mockFilesystem{},
 		},
 		{
 			"/",
-			test.NewMockFilesystem(test.File{Path: "/CustomData", Contents: ""}),
+			mockFilesystem{"/CustomData": []byte{}},
 		},
 		{
 			"/var/lib/waagent/",
-			test.NewMockFilesystem(test.File{Path: "/var/lib/waagent/CustomData", Contents: ""}),
+			mockFilesystem{"/var/lib/waagent/CustomData": []byte{}},
 		},
 	} {
-		a := waagent{tt.root, tt.files.ReadFile}
+		a := waagent{tt.root, tt.files.readFile}
 		_, err := a.FetchUserdata()
 		if err != nil {
-			t.Fatalf("bad error for %+v: want %v, got %q", tt, nil, err)
+			t.Fatalf("bad error for %q: want %q, got %q", tt, nil, err)
 		}
 	}
 }

initialize/config.go

@@ -1,29 +1,13 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
 	"errors"
 	"fmt"
 	"log"
-	"os"
-	"os/exec"
 	"path"
-	"strings"
 
-	"github.com/coreos/coreos-cloudinit/config"
+	"github.com/coreos/coreos-cloudinit/third_party/gopkg.in/yaml.v1"
+
 	"github.com/coreos/coreos-cloudinit/network"
 	"github.com/coreos/coreos-cloudinit/system"
 )
@@ -33,56 +17,151 @@ import (
 type CloudConfigFile interface {
 	// File should either return (*system.File, error), or (nil, nil) if nothing
 	// needs to be done for this configuration option.
-	File() (*system.File, error)
+	File(root string) (*system.File, error)
 }
 
 // CloudConfigUnit represents a CoreOS specific configuration option that can generate
 // associated system.Units to be created/enabled appropriately
 type CloudConfigUnit interface {
-	Units() []system.Unit
+	Units(root string) ([]system.Unit, error)
 }
 
-func isLock(env *Environment) bool {
+// CloudConfig encapsulates the entire cloud-config configuration file and maps directly to YAML
-	if _, err := os.Stat(path.Join(env.Workspace(), ".lock")); err != nil {
+type CloudConfig struct {
-		return false
+	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
+	Coreos            struct {
+		Etcd   EtcdEnvironment
+		Fleet  FleetEnvironment
+		OEM    OEMRelease
+		Update UpdateConfig
+		Units  []system.Unit
 	}
-	return true
+	WriteFiles        []system.File `yaml:"write_files"`
+	Hostname          string
+	Users             []system.User
+	ManageEtcHosts    EtcHosts `yaml:"manage_etc_hosts"`
+	NetworkConfigPath string
+	NetworkConfig     string
 }
 
-func Lock(env *Environment) error {
+type warner func(format string, v ...interface{})
-	if !isLock(env) {
+
-		fp, err := os.OpenFile(path.Join(env.Workspace(), ".lock"), os.O_WRONLY|os.O_CREATE|os.O_EXCL|os.O_TRUNC, os.FileMode(0644))
+// warnOnUnrecognizedKeys parses the contents of a cloud-config file and calls
-		if err != nil {
+// warn(msg, key) for every unrecognized key (i.e. those not present in CloudConfig)
-			return err
+func warnOnUnrecognizedKeys(contents string, warn warner) {
+	// Generate a map of all understood cloud config options
+	var cc map[string]interface{}
+	b, _ := yaml.Marshal(&CloudConfig{})
+	yaml.Unmarshal(b, &cc)
+
+	// Now unmarshal the entire provided contents
+	var c map[string]interface{}
+	yaml.Unmarshal([]byte(contents), &c)
+
+	// Check that every key in the contents exists in the cloud config
+	for k, _ := range c {
+		if _, ok := cc[k]; !ok {
+			warn("Warning: unrecognized key %q in provided cloud config - ignoring section", k)
 		}
-		return fp.Close()
 	}
-	return nil
+
+	// Check for unrecognized coreos options, if any are set
+	if coreos, ok := c["coreos"]; ok {
+		if set, ok := coreos.(map[interface{}]interface{}); ok {
+			known := cc["coreos"].(map[interface{}]interface{})
+			for k, _ := range set {
+				if key, ok := k.(string); ok {
+					if _, ok := known[key]; !ok {
+						warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", key)
+					}
+				} else {
+					warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", k)
+				}
+			}
+		}
+	}
+
+	// Check for any badly-specified users, if any are set
+	if users, ok := c["users"]; ok {
+		var known map[string]interface{}
+		b, _ := yaml.Marshal(&system.User{})
+		yaml.Unmarshal(b, &known)
+
+		if set, ok := users.([]interface{}); ok {
+			for _, u := range set {
+				if user, ok := u.(map[interface{}]interface{}); ok {
+					for k, _ := range user {
+						if key, ok := k.(string); ok {
+							if _, ok := known[key]; !ok {
+								warn("Warning: unrecognized key %q in user section of cloud config - ignoring", key)
+							}
+						} else {
+							warn("Warning: unrecognized key %q in user section of cloud config - ignoring", k)
+						}
+					}
+				}
+			}
+		}
+	}
+
+	// Check for any badly-specified files, if any are set
+	if files, ok := c["write_files"]; ok {
+		var known map[string]interface{}
+		b, _ := yaml.Marshal(&system.File{})
+		yaml.Unmarshal(b, &known)
+
+		if set, ok := files.([]interface{}); ok {
+			for _, f := range set {
+				if file, ok := f.(map[interface{}]interface{}); ok {
+					for k, _ := range file {
+						if key, ok := k.(string); ok {
+							if _, ok := known[key]; !ok {
+								warn("Warning: unrecognized key %q in file section of cloud config - ignoring", key)
+							}
+						} else {
+							warn("Warning: unrecognized key %q in file section of cloud config - ignoring", k)
+						}
+					}
+				}
+			}
+		}
+	}
+}
+
+// NewCloudConfig instantiates a new CloudConfig from the given contents (a
+// string of YAML), returning any error encountered. It will ignore unknown
+// fields but log encountering them.
+func NewCloudConfig(contents string) (*CloudConfig, error) {
+	var cfg CloudConfig
+	err := yaml.Unmarshal([]byte(contents), &cfg)
+	if err != nil {
+		return &cfg, err
+	}
+	warnOnUnrecognizedKeys(contents, log.Printf)
+	return &cfg, nil
+}
+
+func (cc CloudConfig) String() string {
+	bytes, err := yaml.Marshal(cc)
+	if err != nil {
+		return ""
+	}
+
+	stringified := string(bytes)
+	stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
+
+	return stringified
 }
 
 // Apply renders a CloudConfig to an Environment. This can involve things like
 // configuring the hostname, adding new users, writing various configuration
 // files to disk, and manipulating systemd services.
-func Apply(cfg config.CloudConfig, ifaces []network.InterfaceGenerator, env *Environment) error {
+func Apply(cfg CloudConfig, env *Environment) error {
-	var err error
+	if cfg.Hostname != "" {
-
+		if err := system.SetHostname(cfg.Hostname); err != nil {
-	for _, cmdline := range cfg.RunCMD {
+			return err
-		prog := strings.Fields(cmdline)[0]
-		args := strings.Fields(cmdline)[1:]
-		exec.Command(prog, args...).Run()
-	}
-
-	if err = os.MkdirAll(env.Workspace(), os.FileMode(0755)); err != nil {
-		return err
-	}
-
-	if !isLock(env) {
-		if cfg.Hostname != "" {
-			if err = system.SetHostname(cfg.Hostname); err != nil {
-				return err
-			}
-			log.Printf("Set hostname to %s", cfg.Hostname)
 		}
+		log.Printf("Set hostname to %s", cfg.Hostname)
 	}
 
 	for _, user := range cfg.Users {
@@ -91,164 +170,120 @@ func Apply(cfg config.CloudConfig, ifaces []network.InterfaceGenerator, env *Env
 			continue
 		}
 
-		if !isLock(env) {
+		if system.UserExists(&user) {
-			if system.UserExists(&user) {
+			log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
-				log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
+			if user.PasswordHash != "" {
-				if user.PasswordHash != "" {
+				log.Printf("Setting '%s' user's password", user.Name)
-					log.Printf("Setting '%s' user's password", user.Name)
+				if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
-					if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
+					log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
-						log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
-						return err
-					}
-				}
-			} else {
-				log.Printf("Creating user '%s'", user.Name)
-				if err = system.CreateUser(&user); err != nil {
-					log.Printf("Failed creating user '%s': %v", user.Name, err)
 					return err
 				}
 			}
-
+		} else {
-			if err = system.LockUnlockUser(&user); err != nil {
+			log.Printf("Creating user '%s'", user.Name)
-				log.Printf("Failed lock/unlock user '%s': %v", user.Name, err)
+			if err := system.CreateUser(&user); err != nil {
+				log.Printf("Failed creating user '%s': %v", user.Name, err)
 				return err
 			}
 		}
 
 		if len(user.SSHAuthorizedKeys) > 0 {
 			log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
-			if err = system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
+			if err := system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
 				return err
 			}
 		}
 		if user.SSHImportGithubUser != "" {
 			log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", user.SSHImportGithubUser, user.Name)
-			if err = SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
+			if err := SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
-				return err
-			}
-		}
-		for _, u := range user.SSHImportGithubUsers {
-			log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", u, user.Name)
-			if err = SSHImportGithubUser(user.Name, u); err != nil {
 				return err
 			}
 		}
 		if user.SSHImportURL != "" {
 			log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
-			if err = SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
+			if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
 				return err
 			}
 		}
 	}
 
 	if len(cfg.SSHAuthorizedKeys) > 0 {
-		err = system.AuthorizeSSHKeys(cfg.SystemInfo.DefaultUser.Name, env.SSHKeyName(), cfg.SSHAuthorizedKeys)
+		err := system.AuthorizeSSHKeys("core", env.SSHKeyName(), cfg.SSHAuthorizedKeys)
 		if err == nil {
-			log.Printf("Authorized SSH keys for %s user", cfg.SystemInfo.DefaultUser.Name)
+			log.Printf("Authorized SSH keys for core user")
 		} else {
 			return err
 		}
 	}
 
-	if !isLock(env) {
+	for _, ccf := range []CloudConfigFile{cfg.Coreos.OEM, cfg.Coreos.Update, cfg.ManageEtcHosts} {
-		var writeFiles []system.File
+		f, err := ccf.File(env.Root())
-		for _, file := range cfg.WriteFiles {
+		if err != nil {
-			writeFiles = append(writeFiles, system.File{File: file})
+			return err
 		}
+		if f != nil {
+			cfg.WriteFiles = append(cfg.WriteFiles, *f)
+		}
+	}
 
-		for _, ccf := range []CloudConfigFile{
+	for _, ccu := range []CloudConfigUnit{cfg.Coreos.Etcd, cfg.Coreos.Fleet, cfg.Coreos.Update} {
-			system.OEM{OEM: cfg.CoreOS.OEM},
+		u, err := ccu.Units(env.Root())
-			system.Update{Update: cfg.CoreOS.Update, ReadConfig: system.DefaultReadConfig},
+		if err != nil {
-			system.EtcHosts{EtcHosts: cfg.ManageEtcHosts},
+			return err
-			system.Flannel{Flannel: cfg.CoreOS.Flannel},
+		}
-		} {
+		cfg.Coreos.Units = append(cfg.Coreos.Units, u...)
-			f, err := ccf.File()
+	}
+
+	wroteEnvironment := false
+	for _, file := range cfg.WriteFiles {
+		fullPath, err := system.WriteFile(&file, env.Root())
+		if err != nil {
+			return err
+		}
+		if path.Clean(file.Path) == "/etc/environment" {
+			wroteEnvironment = true
+		}
+		log.Printf("Wrote file %s to filesystem", fullPath)
+	}
+
+	if !wroteEnvironment {
+		ef := env.DefaultEnvironmentFile()
+		if ef != nil {
+			err := system.WriteEnvFile(ef, env.Root())
 			if err != nil {
 				return err
 			}
-			if f != nil {
+			log.Printf("Updated /etc/environment")
-				writeFiles = append(writeFiles, *f)
+		}
-			}
+	}
+
+	if env.NetconfType() != "" {
+		var interfaces []network.InterfaceGenerator
+		var err error
+		switch env.NetconfType() {
+		case "debian":
+			interfaces, err = network.ProcessDebianNetconf(cfg.NetworkConfig)
+		case "digitalocean":
+			interfaces, err = network.ProcessDigitalOceanNetconf(cfg.NetworkConfig)
+		default:
+			return fmt.Errorf("Unsupported network config format %q", env.NetconfType())
 		}
 
-		var units []system.Unit
+		if err != nil {
-		for _, u := range cfg.CoreOS.Units {
+			return err
-			units = append(units, system.Unit{Unit: u})
 		}
 
-		for _, ccu := range []CloudConfigUnit{
+		if err := system.WriteNetworkdConfigs(interfaces); err != nil {
-			system.Etcd{Etcd: cfg.CoreOS.Etcd},
+			return err
-			system.Etcd2{Etcd2: cfg.CoreOS.Etcd2},
-			system.Fleet{Fleet: cfg.CoreOS.Fleet},
-			system.Locksmith{Locksmith: cfg.CoreOS.Locksmith},
-			system.Update{Update: cfg.CoreOS.Update, ReadConfig: system.DefaultReadConfig},
-		} {
-			units = append(units, ccu.Units()...)
 		}
-
+		if err := system.RestartNetwork(interfaces); err != nil {
-		wroteEnvironment := false
-		for _, file := range writeFiles {
-			fullPath, err := system.WriteFile(&file, env.Root())
-			if err != nil {
-				return err
-			}
-			if path.Clean(file.Path) == "/etc/environment" {
-				wroteEnvironment = true
-			}
-			log.Printf("Wrote file %s to filesystem", fullPath)
-		}
-
-		if !wroteEnvironment {
-			ef := env.DefaultEnvironmentFile()
-			if ef != nil {
-				err := system.WriteEnvFile(ef, env.Root())
-				if err != nil {
-					return err
-				}
-				log.Printf("Updated /etc/environment")
-			}
-		}
-
-		if len(ifaces) > 0 {
-			units = append(units, createNetworkingUnits(ifaces)...)
-			if err = system.RestartNetwork(ifaces); err != nil {
-				return err
-			}
-		}
-
-		um := system.NewUnitManager(env.Root())
-		if err = processUnits(units, env.Root(), um); err != nil {
 			return err
 		}
 	}
 
-	if cfg.ResizeRootfs {
+	um := system.NewUnitManager(env.Root())
-		log.Printf("resize root filesystem")
+	return processUnits(cfg.Coreos.Units, env.Root(), um)
-		if err = system.ResizeRootFS(); err != nil {
-			return err
-		}
-	}
 
-	return Lock(env)
-}
-
-func createNetworkingUnits(interfaces []network.InterfaceGenerator) (units []system.Unit) {
-	appendNewUnit := func(units []system.Unit, name, content string) []system.Unit {
-		if content == "" {
-			return units
-		}
-		return append(units, system.Unit{Unit: config.Unit{
-			Name:    name,
-			Runtime: true,
-			Content: content,
-		}})
-	}
-	for _, i := range interfaces {
-		units = appendNewUnit(units, fmt.Sprintf("%s.netdev", i.Filename()), i.Netdev())
-		units = appendNewUnit(units, fmt.Sprintf("%s.link", i.Filename()), i.Link())
-		units = appendNewUnit(units, fmt.Sprintf("%s.network", i.Filename()), i.Network())
-	}
-	return units
 }
 
 // processUnits takes a set of Units and applies them to the given root using
@@ -257,92 +292,66 @@ func createNetworkingUnits(interfaces []network.InterfaceGenerator) (units []sys
 // commands against units. It returns any error encountered.
 func processUnits(units []system.Unit, root string, um system.UnitManager) error {
 	type action struct {
-		unit    system.Unit
+		unit    string
 		command string
 	}
 	actions := make([]action, 0, len(units))
 	reload := false
-	restartNetworkd := false
 	for _, unit := range units {
-		if unit.Name == "" {
+		dst := unit.Destination(root)
-			log.Printf("Skipping unit without name")
-			continue
-		}
-
 		if unit.Content != "" {
-			log.Printf("Writing unit %q to filesystem", unit.Name)
+			log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
-			if err := um.PlaceUnit(unit); err != nil {
+			if err := um.PlaceUnit(&unit, dst); err != nil {
 				return err
 			}
-			log.Printf("Wrote unit %q", unit.Name)
+			log.Printf("Placed unit %s at %s", unit.Name, dst)
 			reload = true
 		}
 
-		for _, dropin := range unit.DropIns {
-			if dropin.Name != "" && dropin.Content != "" {
-				log.Printf("Writing drop-in unit %q to filesystem", dropin.Name)
-				if err := um.PlaceUnitDropIn(unit, dropin); err != nil {
-					return err
-				}
-				log.Printf("Wrote drop-in unit %q", dropin.Name)
-				reload = true
-			}
-		}
-
 		if unit.Mask {
-			log.Printf("Masking unit file %q", unit.Name)
+			log.Printf("Masking unit file %s", unit.Name)
-			if err := um.MaskUnit(unit); err != nil {
+			if err := um.MaskUnit(&unit); err != nil {
 				return err
 			}
 		} else if unit.Runtime {
-			log.Printf("Ensuring runtime unit file %q is unmasked", unit.Name)
+			log.Printf("Ensuring runtime unit file %s is unmasked", unit.Name)
-			if err := um.UnmaskUnit(unit); err != nil {
+			if err := um.UnmaskUnit(&unit); err != nil {
 				return err
 			}
 		}
 
 		if unit.Enable {
 			if unit.Group() != "network" {
-				log.Printf("Enabling unit file %q", unit.Name)
+				log.Printf("Enabling unit file %s", unit.Name)
-				if err := um.EnableUnitFile(unit); err != nil {
+				if err := um.EnableUnitFile(unit.Name, unit.Runtime); err != nil {
 					return err
 				}
-				log.Printf("Enabled unit %q", unit.Name)
+				log.Printf("Enabled unit %s", unit.Name)
 			} else {
-				log.Printf("Skipping enable for network-like unit %q", unit.Name)
+				log.Printf("Skipping enable for network-like unit %s", unit.Name)
 			}
 		}
 
 		if unit.Group() == "network" {
-			restartNetworkd = true
+			actions = append(actions, action{"systemd-networkd.service", "restart"})
 		} else if unit.Command != "" {
-			actions = append(actions, action{unit, unit.Command})
+			actions = append(actions, action{unit.Name, unit.Command})
 		}
 	}
 
 	if reload {
 		if err := um.DaemonReload(); err != nil {
-			return errors.New(fmt.Sprintf("failed systemd daemon-reload: %s", err))
+			return errors.New(fmt.Sprintf("failed systemd daemon-reload: %v", err))
 		}
 	}
 
-	if restartNetworkd {
-		log.Printf("Restarting systemd-networkd")
-		networkd := system.Unit{Unit: config.Unit{Name: "systemd-networkd.service"}}
-		res, err := um.RunUnitCommand(networkd, "restart")
-		if err != nil {
-			return err
-		}
-		log.Printf("Restarted systemd-networkd (%s)", res)
-	}
-
 	for _, action := range actions {
-		log.Printf("Calling unit command %q on %q'", action.command, action.unit.Name)
+		log.Printf("Calling unit command '%s %s'", action.command, action.unit)
-		res, err := um.RunUnitCommand(action.unit, action.command)
+		res, err := um.RunUnitCommand(action.command, action.unit)
 		if err != nil {
 			return err
 		}
-		log.Printf("Result of %q on %q: %s", action.command, action.unit.Name, res)
+		log.Printf("Result of '%s %s': %s", action.command, action.unit, res)
 	}
 
 	return nil

initialize/config_test.go

@@ -1,299 +1,470 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
-	"reflect"
+	"fmt"
+	"strings"
 	"testing"
 
-	"github.com/coreos/coreos-cloudinit/config"
-	"github.com/coreos/coreos-cloudinit/network"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
+func TestCloudConfigInvalidKeys(t *testing.T) {
+	defer func() {
+		if r := recover(); r != nil {
+			t.Fatalf("panic while instantiating CloudConfig with nil keys: %v", r)
+		}
+	}()
+
+	for _, tt := range []struct {
+		contents string
+	}{
+		{"coreos:"},
+		{"ssh_authorized_keys:"},
+		{"ssh_authorized_keys:\n  -"},
+		{"ssh_authorized_keys:\n  - 0:"},
+		{"write_files:"},
+		{"write_files:\n  -"},
+		{"write_files:\n  - 0:"},
+		{"users:"},
+		{"users:\n  -"},
+		{"users:\n  - 0:"},
+	} {
+		_, err := NewCloudConfig(tt.contents)
+		if err != nil {
+			t.Fatalf("error instantiating CloudConfig with invalid keys: %v", err)
+		}
+	}
+}
+
+func TestCloudConfigUnknownKeys(t *testing.T) {
+	contents := `
+coreos: 
+  etcd:
+    discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
+  coreos_unknown:
+    foo: "bar"
+section_unknown:
+  dunno:
+    something
+bare_unknown:
+  bar
+write_files:
+  - content: fun
+    path: /var/party
+    file_unknown: nofun
+users:
+  - name: fry
+    passwd: somehash
+    user_unknown: philip
+hostname:
+  foo
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
+	}
+	if cfg.Hostname != "foo" {
+		t.Fatalf("hostname not correctly set when invalid keys are present")
+	}
+	if len(cfg.Coreos.Etcd) < 1 {
+		t.Fatalf("etcd section not correctly set when invalid keys are present")
+	}
+	if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
+		t.Fatalf("write_files section not correctly set when invalid keys are present")
+	}
+	if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
+		t.Fatalf("users section not correctly set when invalid keys are present")
+	}
+
+	var warnings string
+	catchWarn := func(f string, v ...interface{}) {
+		warnings += fmt.Sprintf(f, v...)
+	}
+
+	warnOnUnrecognizedKeys(contents, catchWarn)
+
+	if !strings.Contains(warnings, "coreos_unknown") {
+		t.Errorf("warnings did not catch unrecognized coreos option coreos_unknown")
+	}
+	if !strings.Contains(warnings, "bare_unknown") {
+		t.Errorf("warnings did not catch unrecognized key bare_unknown")
+	}
+	if !strings.Contains(warnings, "section_unknown") {
+		t.Errorf("warnings did not catch unrecognized key section_unknown")
+	}
+	if !strings.Contains(warnings, "user_unknown") {
+		t.Errorf("warnings did not catch unrecognized user key user_unknown")
+	}
+	if !strings.Contains(warnings, "file_unknown") {
+		t.Errorf("warnings did not catch unrecognized file key file_unknown")
+	}
+}
+
+// Assert that the parsing of a cloud config file "generally works"
+func TestCloudConfigEmpty(t *testing.T) {
+	cfg, err := NewCloudConfig("")
+	if err != nil {
+		t.Fatalf("Encountered unexpected error :%v", err)
+	}
+
+	keys := cfg.SSHAuthorizedKeys
+	if len(keys) != 0 {
+		t.Error("Parsed incorrect number of SSH keys")
+	}
+
+	if len(cfg.WriteFiles) != 0 {
+		t.Error("Expected zero WriteFiles")
+	}
+
+	if cfg.Hostname != "" {
+		t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
+	}
+}
+
+// Assert that the parsing of a cloud config file "generally works"
+func TestCloudConfig(t *testing.T) {
+	contents := `
+coreos: 
+  etcd:
+    discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
+  update:
+    reboot-strategy: reboot
+  units:
+    - name: 50-eth0.network
+      runtime: yes
+      content: '[Match]
+ 
+    Name=eth47
+ 
+ 
+    [Network]
+ 
+    Address=10.209.171.177/19
+ 
+'
+  oem:
+    id: rackspace
+    name: Rackspace Cloud Servers
+    version-id: 168.0.0
+    home-url: https://www.rackspace.com/cloud/servers/
+    bug-report-url: https://github.com/coreos/coreos-overlay
+ssh_authorized_keys:
+  - foobar
+  - foobaz
+write_files:
+  - content: |
+      penny
+      elroy
+    path: /etc/dogepack.conf
+    permissions: '0644'
+    owner: root:dogepack
+hostname: trontastic
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error :%v", err)
+	}
+
+	keys := cfg.SSHAuthorizedKeys
+	if len(keys) != 2 {
+		t.Error("Parsed incorrect number of SSH keys")
+	} else if keys[0] != "foobar" {
+		t.Error("Expected first SSH key to be 'foobar'")
+	} else if keys[1] != "foobaz" {
+		t.Error("Expected first SSH key to be 'foobaz'")
+	}
+
+	if len(cfg.WriteFiles) != 1 {
+		t.Error("Failed to parse correct number of write_files")
+	} else {
+		wf := cfg.WriteFiles[0]
+		if wf.Content != "penny\nelroy\n" {
+			t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
+		}
+		if wf.Encoding != "" {
+			t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
+		}
+		if perm, _ := wf.Permissions(); perm != 0644 {
+			t.Errorf("WriteFile has incorrect permissions %s", perm)
+		}
+		if wf.Path != "/etc/dogepack.conf" {
+			t.Errorf("WriteFile has incorrect path %s", wf.Path)
+		}
+		if wf.Owner != "root:dogepack" {
+			t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
+		}
+	}
+
+	if len(cfg.Coreos.Units) != 1 {
+		t.Error("Failed to parse correct number of units")
+	} else {
+		u := cfg.Coreos.Units[0]
+		expect := `[Match]
+Name=eth47
+
+[Network]
+Address=10.209.171.177/19
+`
+		if u.Content != expect {
+			t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
+		}
+		if u.Runtime != true {
+			t.Errorf("Unit has incorrect runtime value")
+		}
+		if u.Name != "50-eth0.network" {
+			t.Errorf("Unit has incorrect name %s", u.Name)
+		}
+		if u.Type() != "network" {
+			t.Errorf("Unit has incorrect type '%s'", u.Type())
+		}
+	}
+
+	if cfg.Coreos.OEM.ID != "rackspace" {
+		t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
+	}
+
+	if cfg.Hostname != "trontastic" {
+		t.Errorf("Failed to parse hostname")
+	}
+	if cfg.Coreos.Update["reboot-strategy"] != "reboot" {
+		t.Errorf("Failed to parse locksmith strategy")
+	}
+}
+
+// Assert that our interface conversion doesn't panic
+func TestCloudConfigKeysNotList(t *testing.T) {
+	contents := `
+ssh_authorized_keys:
+  - foo: bar
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	keys := cfg.SSHAuthorizedKeys
+	if len(keys) != 0 {
+		t.Error("Parsed incorrect number of SSH keys")
+	}
+}
+
+func TestCloudConfigSerializationHeader(t *testing.T) {
+	cfg, _ := NewCloudConfig("")
+	contents := cfg.String()
+	header := strings.SplitN(contents, "\n", 2)[0]
+	if header != "#cloud-config" {
+		t.Fatalf("Serialized config did not have expected header")
+	}
+}
+
+// TestDropInIgnored asserts that users are unable to set DropIn=True on units
+func TestDropInIgnored(t *testing.T) {
+	contents := `
+coreos:
+  units:
+    - name: test
+      dropin: true
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil || len(cfg.Coreos.Units) != 1 {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+	if len(cfg.Coreos.Units) != 1 || cfg.Coreos.Units[0].Name != "test" {
+		t.Fatalf("Expected 1 unit, but got %d: %v", len(cfg.Coreos.Units), cfg.Coreos.Units)
+	}
+	if cfg.Coreos.Units[0].DropIn {
+		t.Errorf("dropin option on unit in cloud-config was not ignored!")
+	}
+}
+
+func TestCloudConfigUsers(t *testing.T) {
+	contents := `
+users:
+  - name: elroy
+    passwd: somehash
+    ssh-authorized-keys:
+      - somekey
+    gecos: arbitrary comment
+    homedir: /home/place
+    no-create-home: yes
+    primary-group: things
+    groups:
+      - ping
+      - pong
+    no-user-group: true
+    system: y
+    no-log-init: True
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.PasswordHash != "somehash" {
+		t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
+	}
+
+	if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
+		t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
+	} else {
+		key := user.SSHAuthorizedKeys[0]
+		if key != "somekey" {
+			t.Errorf("User SSH key is %q, expected 'somekey'", key)
+		}
+	}
+
+	if user.GECOS != "arbitrary comment" {
+		t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
+	}
+
+	if user.Homedir != "/home/place" {
+		t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
+	}
+
+	if !user.NoCreateHome {
+		t.Errorf("Failed to parse no-create-home field")
+	}
+
+	if user.PrimaryGroup != "things" {
+		t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
+	}
+
+	if len(user.Groups) != 2 {
+		t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
+	} else {
+		if user.Groups[0] != "ping" {
+			t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
+		}
+		if user.Groups[1] != "pong" {
+			t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
+		}
+	}
+
+	if !user.NoUserGroup {
+		t.Errorf("Failed to parse no-user-group field")
+	}
+
+	if !user.System {
+		t.Errorf("Failed to parse system field")
+	}
+
+	if !user.NoLogInit {
+		t.Errorf("Failed to parse no-log-init field")
+	}
+}
+
 type TestUnitManager struct {
 	placed   []string
 	enabled  []string
 	masked   []string
 	unmasked []string
-	commands []UnitAction
+	commands map[string]string
 	reload   bool
 }
 
-type UnitAction struct {
+func (tum *TestUnitManager) PlaceUnit(unit *system.Unit, dst string) error {
-	unit    string
+	tum.placed = append(tum.placed, unit.Name)
-	command string
-}
-
-func (tum *TestUnitManager) PlaceUnit(u system.Unit) error {
-	tum.placed = append(tum.placed, u.Name)
 	return nil
 }
-func (tum *TestUnitManager) PlaceUnitDropIn(u system.Unit, d config.UnitDropIn) error {
+func (tum *TestUnitManager) EnableUnitFile(unit string, runtime bool) error {
-	tum.placed = append(tum.placed, u.Name+".d/"+d.Name)
+	tum.enabled = append(tum.enabled, unit)
 	return nil
 }
-func (tum *TestUnitManager) EnableUnitFile(u system.Unit) error {
+func (tum *TestUnitManager) RunUnitCommand(command, unit string) (string, error) {
-	tum.enabled = append(tum.enabled, u.Name)
+	tum.commands = make(map[string]string)
-	return nil
+	tum.commands[unit] = command
-}
-func (tum *TestUnitManager) RunUnitCommand(u system.Unit, c string) (string, error) {
-	tum.commands = append(tum.commands, UnitAction{u.Name, c})
 	return "", nil
 }
 func (tum *TestUnitManager) DaemonReload() error {
 	tum.reload = true
 	return nil
 }
-func (tum *TestUnitManager) MaskUnit(u system.Unit) error {
+func (tum *TestUnitManager) MaskUnit(unit *system.Unit) error {
-	tum.masked = append(tum.masked, u.Name)
+	tum.masked = append(tum.masked, unit.Name)
 	return nil
 }
-func (tum *TestUnitManager) UnmaskUnit(u system.Unit) error {
+func (tum *TestUnitManager) UnmaskUnit(unit *system.Unit) error {
-	tum.unmasked = append(tum.unmasked, u.Name)
+	tum.unmasked = append(tum.unmasked, unit.Name)
 	return nil
 }
 
-type mockInterface struct {
-	name           string
-	filename       string
-	netdev         string
-	link           string
-	network        string
-	kind           string
-	modprobeParams string
-}
-
-func (i mockInterface) Name() string {
-	return i.name
-}
-
-func (i mockInterface) Filename() string {
-	return i.filename
-}
-
-func (i mockInterface) Netdev() string {
-	return i.netdev
-}
-
-func (i mockInterface) Link() string {
-	return i.link
-}
-
-func (i mockInterface) Network() string {
-	return i.network
-}
-
-func (i mockInterface) Type() string {
-	return i.kind
-}
-
-func (i mockInterface) ModprobeParams() string {
-	return i.modprobeParams
-}
-
-func TestCreateNetworkingUnits(t *testing.T) {
-	for _, tt := range []struct {
-		interfaces []network.InterfaceGenerator
-		expect     []system.Unit
-	}{
-		{nil, nil},
-		{
-			[]network.InterfaceGenerator{
-				network.InterfaceGenerator(mockInterface{filename: "test"}),
-			},
-			nil,
-		},
-		{
-			[]network.InterfaceGenerator{
-				network.InterfaceGenerator(mockInterface{filename: "test1", netdev: "test netdev"}),
-				network.InterfaceGenerator(mockInterface{filename: "test2", link: "test link"}),
-				network.InterfaceGenerator(mockInterface{filename: "test3", network: "test network"}),
-			},
-			[]system.Unit{
-				system.Unit{Unit: config.Unit{Name: "test1.netdev", Runtime: true, Content: "test netdev"}},
-				system.Unit{Unit: config.Unit{Name: "test2.link", Runtime: true, Content: "test link"}},
-				system.Unit{Unit: config.Unit{Name: "test3.network", Runtime: true, Content: "test network"}},
-			},
-		},
-		{
-			[]network.InterfaceGenerator{
-				network.InterfaceGenerator(mockInterface{filename: "test", netdev: "test netdev", link: "test link", network: "test network"}),
-			},
-			[]system.Unit{
-				system.Unit{Unit: config.Unit{Name: "test.netdev", Runtime: true, Content: "test netdev"}},
-				system.Unit{Unit: config.Unit{Name: "test.link", Runtime: true, Content: "test link"}},
-				system.Unit{Unit: config.Unit{Name: "test.network", Runtime: true, Content: "test network"}},
-			},
-		},
-	} {
-		units := createNetworkingUnits(tt.interfaces)
-		if !reflect.DeepEqual(tt.expect, units) {
-			t.Errorf("bad units (%+v): want %#v, got %#v", tt.interfaces, tt.expect, units)
-		}
-	}
-}
-
 func TestProcessUnits(t *testing.T) {
-	tests := []struct {
+	tum := &TestUnitManager{}
-		units []system.Unit
+	units := []system.Unit{
-
+		system.Unit{
-		result TestUnitManager
+			Name: "foo",
-	}{
+			Mask: true,
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name: "foo",
-					Mask: true,
-				}},
-			},
-			result: TestUnitManager{
-				masked: []string{"foo"},
-			},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name:    "baz.service",
-					Content: "[Service]\nExecStart=/bin/baz",
-					Command: "start",
-				}},
-				system.Unit{Unit: config.Unit{
-					Name:    "foo.network",
-					Content: "[Network]\nFoo=true",
-				}},
-				system.Unit{Unit: config.Unit{
-					Name:    "bar.network",
-					Content: "[Network]\nBar=true",
-				}},
-			},
-			result: TestUnitManager{
-				placed: []string{"baz.service", "foo.network", "bar.network"},
-				commands: []UnitAction{
-					UnitAction{"systemd-networkd.service", "restart"},
-					UnitAction{"baz.service", "start"},
-				},
-				reload: true,
-			},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name:    "baz.service",
-					Content: "[Service]\nExecStart=/bin/true",
-				}},
-			},
-			result: TestUnitManager{
-				placed: []string{"baz.service"},
-				reload: true,
-			},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name:    "locksmithd.service",
-					Runtime: true,
-				}},
-			},
-			result: TestUnitManager{
-				unmasked: []string{"locksmithd.service"},
-			},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name:   "woof",
-					Enable: true,
-				}},
-			},
-			result: TestUnitManager{
-				enabled: []string{"woof"},
-			},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name:    "hi.service",
-					Runtime: true,
-					Content: "[Service]\nExecStart=/bin/echo hi",
-					DropIns: []config.UnitDropIn{
-						{
-							Name:    "lo.conf",
-							Content: "[Service]\nExecStart=/bin/echo lo",
-						},
-						{
-							Name:    "bye.conf",
-							Content: "[Service]\nExecStart=/bin/echo bye",
-						},
-					},
-				}},
-			},
-			result: TestUnitManager{
-				placed:   []string{"hi.service", "hi.service.d/lo.conf", "hi.service.d/bye.conf"},
-				unmasked: []string{"hi.service"},
-				reload:   true,
-			},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					DropIns: []config.UnitDropIn{
-						{
-							Name:    "lo.conf",
-							Content: "[Service]\nExecStart=/bin/echo lo",
-						},
-					},
-				}},
-			},
-			result: TestUnitManager{},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name: "hi.service",
-					DropIns: []config.UnitDropIn{
-						{
-							Content: "[Service]\nExecStart=/bin/echo lo",
-						},
-					},
-				}},
-			},
-			result: TestUnitManager{},
-		},
-		{
-			units: []system.Unit{
-				system.Unit{Unit: config.Unit{
-					Name: "hi.service",
-					DropIns: []config.UnitDropIn{
-						{
-							Name: "lo.conf",
-						},
-					},
-				}},
-			},
-			result: TestUnitManager{},
 		},
 	}
+	if err := processUnits(units, "", tum); err != nil {
+		t.Fatalf("unexpected error calling processUnits: %v", err)
+	}
+	if len(tum.masked) != 1 || tum.masked[0] != "foo" {
+		t.Errorf("expected foo to be masked, but found %v", tum.masked)
+	}
 
-	for _, tt := range tests {
+	tum = &TestUnitManager{}
-		tum := &TestUnitManager{}
+	units = []system.Unit{
-		if err := processUnits(tt.units, "", tum); err != nil {
+		system.Unit{
-			t.Errorf("bad error (%+v): want nil, got %s", tt.units, err)
+			Name: "bar.network",
-		}
+		},
-		if !reflect.DeepEqual(tt.result, *tum) {
+	}
-			t.Errorf("bad result (%+v): want %+v, got %+v", tt.units, tt.result, tum)
+	if err := processUnits(units, "", tum); err != nil {
-		}
+		t.Fatalf("unexpected error calling processUnits: %v", err)
+	}
+	if _, ok := tum.commands["systemd-networkd.service"]; !ok {
+		t.Errorf("expected systemd-networkd.service to be reloaded!")
+	}
+
+	tum = &TestUnitManager{}
+	units = []system.Unit{
+		system.Unit{
+			Name:    "baz.service",
+			Content: "[Service]\nExecStart=/bin/true",
+		},
+	}
+	if err := processUnits(units, "", tum); err != nil {
+		t.Fatalf("unexpected error calling processUnits: %v", err)
+	}
+	if len(tum.placed) != 1 || tum.placed[0] != "baz.service" {
+		t.Fatalf("expected baz.service to be written, but got %v", tum.placed)
+	}
+
+	tum = &TestUnitManager{}
+	units = []system.Unit{
+		system.Unit{
+			Name:    "locksmithd.service",
+			Runtime: true,
+		},
+	}
+	if err := processUnits(units, "", tum); err != nil {
+		t.Fatalf("unexpected error calling processUnits: %v", err)
+	}
+	if len(tum.unmasked) != 1 || tum.unmasked[0] != "locksmithd.service" {
+		t.Fatalf("expected locksmithd.service to be unmasked, but got %v", tum.unmasked)
+	}
+
+	tum = &TestUnitManager{}
+	units = []system.Unit{
+		system.Unit{
+			Name:   "woof",
+			Enable: true,
+		},
+	}
+	if err := processUnits(units, "", tum); err != nil {
+		t.Fatalf("unexpected error calling processUnits: %v", err)
+	}
+	if len(tum.enabled) != 1 || tum.enabled[0] != "woof" {
+		t.Fatalf("expected woof to be enabled, but got %v", tum.enabled)
 	}
 }

initialize/env.go

@@ -1,28 +1,11 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
-	"net"
 	"os"
 	"path"
 	"regexp"
 	"strings"
 
-	"github.com/coreos/coreos-cloudinit/config"
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -32,25 +15,28 @@ type Environment struct {
 	root          string
 	configRoot    string
 	workspace     string
+	netconfType   string
 	sshKeyName    string
 	substitutions map[string]string
 }
 
 // TODO(jonboulle): this is getting unwieldy, should be able to simplify the interface somehow
-func NewEnvironment(root, configRoot, workspace, sshKeyName string, metadata datasource.Metadata) *Environment {
+func NewEnvironment(root, configRoot, workspace, netconfType, sshKeyName string, substitutions map[string]string) *Environment {
-	firstNonNull := func(ip net.IP, env string) string {
+	if substitutions == nil {
-		if ip == nil {
+		substitutions = make(map[string]string)
-			return env
+	}
+	// If certain values are not in the supplied substitution, fall back to retrieving them from the environment
+	for k, v := range map[string]string{
+		"$public_ipv4":  os.Getenv("COREOS_PUBLIC_IPV4"),
+		"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
+		"$public_ipv6":  os.Getenv("COREOS_PUBLIC_IPV6"),
+		"$private_ipv6": os.Getenv("COREOS_PRIVATE_IPV6"),
+	} {
+		if _, ok := substitutions[k]; !ok {
+			substitutions[k] = v
 		}
-		return ip.String()
 	}
-	substitutions := map[string]string{
+	return &Environment{root, configRoot, workspace, netconfType, sshKeyName, substitutions}
-		"$public_ipv4":  firstNonNull(metadata.PublicIPv4, os.Getenv("COREOS_PUBLIC_IPV4")),
-		"$private_ipv4": firstNonNull(metadata.PrivateIPv4, os.Getenv("COREOS_PRIVATE_IPV4")),
-		"$public_ipv6":  firstNonNull(metadata.PublicIPv6, os.Getenv("COREOS_PUBLIC_IPV6")),
-		"$private_ipv6": firstNonNull(metadata.PrivateIPv6, os.Getenv("COREOS_PRIVATE_IPV6")),
-	}
-	return &Environment{root, configRoot, workspace, sshKeyName, substitutions}
 }
 
 func (e *Environment) Workspace() string {
@@ -65,6 +51,10 @@ func (e *Environment) ConfigRoot() string {
 	return e.configRoot
 }
 
+func (e *Environment) NetconfType() string {
+	return e.netconfType
+}
+
 func (e *Environment) SSHKeyName() string {
 	return e.sshKeyName
 }
@@ -91,9 +81,9 @@ func (e *Environment) Apply(data string) string {
 
 func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
 	ef := system.EnvFile{
-		File: &system.File{File: config.File{
+		File: &system.File{
 			Path: "/etc/environment",
-		}},
+		},
 		Vars: map[string]string{},
 	}
 	if ip, ok := e.substitutions["$public_ipv4"]; ok && len(ip) > 0 {
@@ -114,3 +104,16 @@ func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
 		return &ef
 	}
 }
+
+// normalizeSvcEnv standardizes the keys of the map (environment variables for a service)
+// by replacing any dashes with underscores and ensuring they are entirely upper case.
+// For example, "some-env" --> "SOME_ENV"
+func normalizeSvcEnv(m map[string]string) map[string]string {
+	out := make(map[string]string, len(m))
+	for key, val := range m {
+		key = strings.ToUpper(key)
+		key = strings.Replace(key, "-", "_", -1)
+		out[key] = val
+	}
+	return out
+}

initialize/env_test.go

@@ -1,27 +1,11 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
 	"io/ioutil"
-	"net"
 	"os"
 	"path"
 	"testing"
 
-	"github.com/coreos/coreos-cloudinit/datasource"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -31,18 +15,18 @@ func TestEnvironmentApply(t *testing.T) {
 	os.Setenv("COREOS_PUBLIC_IPV6", "1234::")
 	os.Setenv("COREOS_PRIVATE_IPV6", "5678::")
 	for _, tt := range []struct {
-		metadata datasource.Metadata
+		subs  map[string]string
-		input    string
+		input string
-		out      string
+		out   string
 	}{
 		{
 			// Substituting both values directly should always take precedence
 			// over environment variables
-			datasource.Metadata{
+			map[string]string{
-				PublicIPv4:  net.ParseIP("192.0.2.3"),
+				"$public_ipv4":  "192.0.2.3",
-				PrivateIPv4: net.ParseIP("192.0.2.203"),
+				"$private_ipv4": "192.0.2.203",
-				PublicIPv6:  net.ParseIP("fe00:1234::"),
+				"$public_ipv6":  "fe00:1234::",
-				PrivateIPv6: net.ParseIP("fe00:5678::"),
+				"$private_ipv6": "fe00:5678::",
 			},
 			`[Service]
 ExecStart=/usr/bin/echo "$public_ipv4 $public_ipv6"
@@ -55,29 +39,25 @@ ExecStop=/usr/bin/echo $unknown`,
 		},
 		{
 			// Substituting one value directly while falling back with the other
-			datasource.Metadata{
+			map[string]string{"$private_ipv4": "127.0.0.1"},
-				PrivateIPv4: net.ParseIP("127.0.0.1"),
-			},
 			"$private_ipv4\n$public_ipv4",
 			"127.0.0.1\n1.2.3.4",
 		},
 		{
 			// Falling back to environment variables for both values
-			datasource.Metadata{},
+			map[string]string{"foo": "bar"},
 			"$private_ipv4\n$public_ipv4",
 			"5.6.7.8\n1.2.3.4",
 		},
 		{
 			// No substitutions
-			datasource.Metadata{},
+			nil,
 			"$private_ipv4\nfoobar",
 			"5.6.7.8\nfoobar",
 		},
 		{
 			// Escaping substitutions
-			datasource.Metadata{
+			map[string]string{"$private_ipv4": "127.0.0.1"},
-				PrivateIPv4: net.ParseIP("127.0.0.1"),
-			},
 			`\$private_ipv4
 $private_ipv4
 addr: \$private_ipv4
@@ -89,13 +69,13 @@ addr: $private_ipv4
 		},
 		{
 			// No substitutions with escaping
-			datasource.Metadata{},
+			nil,
 			"\\$test\n$test",
 			"\\$test\n$test",
 		},
 	} {
 
-		env := NewEnvironment("./", "./", "./", "", tt.metadata)
+		env := NewEnvironment("./", "./", "./", "", "", tt.subs)
 		got := env.Apply(tt.input)
 		if got != tt.out {
 			t.Fatalf("Environment incorrectly applied.\ngot:\n%s\nwant:\n%s", got, tt.out)
@@ -104,11 +84,11 @@ addr: $private_ipv4
 }
 
 func TestEnvironmentFile(t *testing.T) {
-	metadata := datasource.Metadata{
+	subs := map[string]string{
-		PublicIPv4:  net.ParseIP("1.2.3.4"),
+		"$public_ipv4":  "1.2.3.4",
-		PrivateIPv4: net.ParseIP("5.6.7.8"),
+		"$private_ipv4": "5.6.7.8",
-		PublicIPv6:  net.ParseIP("1234::"),
+		"$public_ipv6":  "1234::",
-		PrivateIPv6: net.ParseIP("5678::"),
+		"$private_ipv6": "5678::",
 	}
 	expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PRIVATE_IPV6=5678::\nCOREOS_PUBLIC_IPV4=1.2.3.4\nCOREOS_PUBLIC_IPV6=1234::\n"
 
@@ -118,7 +98,7 @@ func TestEnvironmentFile(t *testing.T) {
 	}
 	defer os.RemoveAll(dir)
 
-	env := NewEnvironment("./", "./", "./", "", metadata)
+	env := NewEnvironment("./", "./", "./", "", "", subs)
 	ef := env.DefaultEnvironmentFile()
 	err = system.WriteEnvFile(ef, dir)
 	if err != nil {
@@ -137,10 +117,14 @@ func TestEnvironmentFile(t *testing.T) {
 }
 
 func TestEnvironmentFileNil(t *testing.T) {
-	os.Clearenv()
+	subs := map[string]string{
-	metadata := datasource.Metadata{}
+		"$public_ipv4":  "",
+		"$private_ipv4": "",
+		"$public_ipv6":  "",
+		"$private_ipv6": "",
+	}
 
-	env := NewEnvironment("./", "./", "./", "", metadata)
+	env := NewEnvironment("./", "./", "./", "", "", subs)
 	ef := env.DefaultEnvironmentFile()
 	if ef != nil {
 		t.Fatalf("Environment file not nil: %v", ef)

initialize/etcd.go

@@ -0,0 +1,63 @@
+package initialize
+
+import (
+	"errors"
+	"fmt"
+	"sort"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type EtcdEnvironment map[string]string
+
+func (ee EtcdEnvironment) String() (out string) {
+	norm := normalizeSvcEnv(ee)
+
+	if val, ok := norm["DISCOVERY_URL"]; ok {
+		delete(norm, "DISCOVERY_URL")
+		if _, ok := norm["DISCOVERY"]; !ok {
+			norm["DISCOVERY"] = val
+		}
+	}
+
+	var sorted sort.StringSlice
+	for k, _ := range norm {
+		sorted = append(sorted, k)
+	}
+	sorted.Sort()
+
+	out += "[Service]\n"
+
+	for _, key := range sorted {
+		val := norm[key]
+		out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
+	}
+
+	return
+}
+
+// Units creates a Unit file drop-in for etcd, using any configured
+// options and adding a default MachineID if unset.
+func (ee EtcdEnvironment) Units(root string) ([]system.Unit, error) {
+	if len(ee) < 1 {
+		return nil, nil
+	}
+
+	if _, ok := ee["name"]; !ok {
+		if machineID := system.MachineID(root); machineID != "" {
+			ee["name"] = machineID
+		} else if hostname, err := system.Hostname(); err == nil {
+			ee["name"] = hostname
+		} else {
+			return nil, errors.New("Unable to determine default etcd name")
+		}
+	}
+
+	etcd := system.Unit{
+		Name:    "etcd.service",
+		Runtime: true,
+		DropIn:  true,
+		Content: ee.String(),
+	}
+	return []system.Unit{etcd}, nil
+}

initialize/etcd_test.go

@@ -0,0 +1,184 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+func TestEtcdEnvironment(t *testing.T) {
+	cfg := make(EtcdEnvironment, 0)
+	cfg["discovery"] = "http://disco.example.com/foobar"
+	cfg["peer-bind-addr"] = "127.0.0.1:7002"
+
+	env := cfg.String()
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestEtcdEnvironmentDiscoveryURLTranslated(t *testing.T) {
+	cfg := make(EtcdEnvironment, 0)
+	cfg["discovery_url"] = "http://disco.example.com/foobar"
+	cfg["peer-bind-addr"] = "127.0.0.1:7002"
+
+	env := cfg.String()
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestEtcdEnvironmentDiscoveryOverridesDiscoveryURL(t *testing.T) {
+	cfg := make(EtcdEnvironment, 0)
+	cfg["discovery_url"] = "ping"
+	cfg["discovery"] = "pong"
+	cfg["peer-bind-addr"] = "127.0.0.1:7002"
+
+	env := cfg.String()
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=pong"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
+	ee := EtcdEnvironment{
+		"name":           "node001",
+		"discovery":      "http://disco.example.com/foobar",
+		"peer-bind-addr": "127.0.0.1:7002",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	sd := system.NewUnitManager(dir)
+
+	uu, err := ee.Units(dir)
+	if err != nil {
+		t.Fatalf("Generating etcd unit failed: %v", err)
+	}
+	if len(uu) != 1 {
+		t.Fatalf("Expected 1 unit to be returned, got %d", len(uu))
+	}
+	u := uu[0]
+
+	dst := u.Destination(dir)
+	os.Stderr.WriteString("writing to " + dir + "\n")
+	if err := sd.PlaceUnit(&u, dst); err != nil {
+		t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `[Service]
+Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
+Environment="ETCD_NAME=node001"
+Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}
+
+func TestEtcdEnvironmentEmptyNoOp(t *testing.T) {
+	ee := EtcdEnvironment{}
+	uu, err := ee.Units("")
+	if err != nil {
+		t.Fatalf("Unexpected error: %v", err)
+	}
+	if len(uu) > 0 {
+		t.Fatalf("Generated etcd units unexpectedly: %v")
+	}
+}
+
+func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
+	ee := EtcdEnvironment{"foo": "bar"}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	sd := system.NewUnitManager(dir)
+
+	os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
+	err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
+	if err != nil {
+		t.Fatalf("Failed writing out /etc/machine-id: %v", err)
+	}
+
+	uu, err := ee.Units(dir)
+	if err != nil {
+		t.Fatalf("Generating etcd unit failed: %v", err)
+	}
+	if len(uu) == 0 {
+		t.Fatalf("Returned empty etcd units unexpectedly")
+	}
+	u := uu[0]
+
+	dst := u.Destination(dir)
+	os.Stderr.WriteString("writing to " + dir + "\n")
+	if err := sd.PlaceUnit(&u, dst); err != nil {
+		t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `[Service]
+Environment="ETCD_FOO=bar"
+Environment="ETCD_NAME=node007"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}
+
+func TestEtcdEnvironmentWhenNil(t *testing.T) {
+	// EtcdEnvironment will be a nil map if it wasn't in the yaml
+	var ee EtcdEnvironment
+	if ee != nil {
+		t.Fatalf("EtcdEnvironment is not nil")
+	}
+	uu, err := ee.Units("")
+	if len(uu) != 0 || err != nil {
+		t.Fatalf("Units returned value for nil input")
+	}
+}

initialize/fleet.go

@@ -0,0 +1,35 @@
+package initialize
+
+import (
+	"fmt"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type FleetEnvironment map[string]string
+
+func (fe FleetEnvironment) String() (out string) {
+	norm := normalizeSvcEnv(fe)
+	out += "[Service]\n"
+
+	for key, val := range norm {
+		out += fmt.Sprintf("Environment=\"FLEET_%s=%s\"\n", key, val)
+	}
+
+	return
+}
+
+// Units generates a Unit file drop-in for fleet, if any fleet options were
+// configured in cloud-config
+func (fe FleetEnvironment) Units(root string) ([]system.Unit, error) {
+	if len(fe) < 1 {
+		return nil, nil
+	}
+	fleet := system.Unit{
+		Name:    "fleet.service",
+		Runtime: true,
+		DropIn:  true,
+		Content: fe.String(),
+	}
+	return []system.Unit{fleet}, nil
+}

initialize/fleet_test.go

@@ -0,0 +1,43 @@
+package initialize
+
+import "testing"
+
+func TestFleetEnvironment(t *testing.T) {
+	cfg := make(FleetEnvironment, 0)
+	cfg["public-ip"] = "12.34.56.78"
+
+	env := cfg.String()
+
+	expect := `[Service]
+Environment="FLEET_PUBLIC_IP=12.34.56.78"
+`
+
+	if env != expect {
+		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
+	}
+}
+
+func TestFleetUnit(t *testing.T) {
+	cfg := make(FleetEnvironment, 0)
+	uu, err := cfg.Units("/")
+	if len(uu) != 0 {
+		t.Errorf("unexpectedly generated unit with empty FleetEnvironment")
+	}
+
+	cfg["public-ip"] = "12.34.56.78"
+
+	uu, err = cfg.Units("/")
+	if err != nil {
+		t.Errorf("error generating fleet unit: %v", err)
+	}
+	if len(uu) != 1 {
+		t.Fatalf("expected 1 unit generated, got %d", len(uu))
+	}
+	u := uu[0]
+	if !u.Runtime {
+		t.Errorf("bad Runtime for generated fleet unit!")
+	}
+	if !u.DropIn {
+		t.Errorf("bad DropIn for generated fleet unit!")
+	}
+}

initialize/github.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (

initialize/github_test.go

@@ -0,0 +1,32 @@
+package initialize
+
+import (
+	"testing"
+)
+
+func TestCloudConfigUsersGithubUser(t *testing.T) {
+
+	contents := `
+users:
+  - name: elroy
+    coreos-ssh-import-github: bcwaldon
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.SSHImportGithubUser != "bcwaldon" {
+		t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
+	}
+}

initialize/manage_etc_hosts.go

@@ -0,0 +1,46 @@
+package initialize
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const DefaultIpv4Address = "127.0.0.1"
+
+type EtcHosts string
+
+func (eh EtcHosts) generateEtcHosts() (out string, err error) {
+	if eh != "localhost" {
+		return "", errors.New("Invalid option to manage_etc_hosts")
+	}
+
+	// use the operating system hostname
+	hostname, err := os.Hostname()
+	if err != nil {
+		return "", err
+	}
+
+	return fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname), nil
+
+}
+
+func (eh EtcHosts) File(root string) (*system.File, error) {
+	if eh == "" {
+		return nil, nil
+	}
+
+	etcHosts, err := eh.generateEtcHosts()
+	if err != nil {
+		return nil, err
+	}
+
+	return &system.File{
+		Path:               path.Join("etc", "hosts"),
+		RawFilePermissions: "0644",
+		Content:            etcHosts,
+	}, nil
+}

initialize/manage_etc_hosts_test.go

@@ -0,0 +1,83 @@
+package initialize
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+func TestCloudConfigManageEtcHosts(t *testing.T) {
+	contents := `
+manage_etc_hosts: localhost
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	manageEtcHosts := cfg.ManageEtcHosts
+
+	if manageEtcHosts != "localhost" {
+		t.Errorf("ManageEtcHosts value is %q, expected 'localhost'", manageEtcHosts)
+	}
+}
+
+func TestManageEtcHostsInvalidValue(t *testing.T) {
+	eh := EtcHosts("invalid")
+	if f, err := eh.File(""); err == nil || f != nil {
+		t.Fatalf("EtcHosts File succeeded with invalid value!")
+	}
+}
+
+func TestEtcHostsWrittenToDisk(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	eh := EtcHosts("localhost")
+
+	f, err := eh.File(dir)
+	if err != nil {
+		t.Fatalf("Error calling File on EtcHosts: %v", err)
+	}
+	if f == nil {
+		t.Fatalf("manageEtcHosts returned nil file unexpectedly")
+	}
+
+	if _, err := system.WriteFile(f, dir); err != nil {
+		t.Fatalf("Error writing EtcHosts: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "hosts")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	hostname, err := os.Hostname()
+	if err != nil {
+		t.Fatalf("Unable to read OS hostname: %v", err)
+	}
+
+	expect := fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname)
+
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}

initialize/meta_data.go

@@ -0,0 +1,72 @@
+package initialize
+
+import (
+	"encoding/json"
+	"sort"
+)
+
+// ParseMetaData parses a JSON blob in the OpenStack metadata service format,
+// and converts it to a partially hydrated CloudConfig.
+func ParseMetaData(contents string) (*CloudConfig, error) {
+	if len(contents) == 0 {
+		return nil, nil
+	}
+	var metadata struct {
+		SSHAuthorizedKeyMap map[string]string `json:"public_keys"`
+		Hostname            string            `json:"hostname"`
+		NetworkConfig       struct {
+			ContentPath string `json:"content_path"`
+		} `json:"network_config"`
+	}
+	if err := json.Unmarshal([]byte(contents), &metadata); err != nil {
+		return nil, err
+	}
+
+	var cfg CloudConfig
+	if len(metadata.SSHAuthorizedKeyMap) > 0 {
+		cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap))
+		for _, name := range sortedKeys(metadata.SSHAuthorizedKeyMap) {
+			cfg.SSHAuthorizedKeys = append(cfg.SSHAuthorizedKeys, metadata.SSHAuthorizedKeyMap[name])
+		}
+	}
+	cfg.Hostname = metadata.Hostname
+	cfg.NetworkConfigPath = metadata.NetworkConfig.ContentPath
+	return &cfg, nil
+}
+
+// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service
+// format and returns a substitution map possibly containing private_ipv4,
+// public_ipv4, private_ipv6, and public_ipv6 addresses.
+func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) {
+	var ips struct {
+		PublicIPv4  string `json:"public-ipv4"`
+		PrivateIPv4 string `json:"local-ipv4"`
+		PublicIPv6  string `json:"public-ipv6"`
+		PrivateIPv6 string `json:"local-ipv6"`
+	}
+	if err := json.Unmarshal(contents, &ips); err != nil {
+		return nil, err
+	}
+	m := make(map[string]string)
+	if ips.PrivateIPv4 != "" {
+		m["$private_ipv4"] = ips.PrivateIPv4
+	}
+	if ips.PublicIPv4 != "" {
+		m["$public_ipv4"] = ips.PublicIPv4
+	}
+	if ips.PrivateIPv6 != "" {
+		m["$private_ipv6"] = ips.PrivateIPv6
+	}
+	if ips.PublicIPv6 != "" {
+		m["$public_ipv6"] = ips.PublicIPv6
+	}
+	return m, nil
+}
+
+func sortedKeys(m map[string]string) (keys []string) {
+	for key := range m {
+		keys = append(keys, key)
+	}
+	sort.Strings(keys)
+	return
+}

initialize/meta_data_test.go

@@ -0,0 +1,69 @@
+package initialize
+
+import "reflect"
+import "testing"
+
+func TestParseMetadata(t *testing.T) {
+	for i, tt := range []struct {
+		in   string
+		want *CloudConfig
+		err  bool
+	}{
+		{"", nil, false},
+		{`garbage, invalid json`, nil, true},
+		{`{"foo": "bar"}`, &CloudConfig{}, false},
+		{`{"network_config": {"content_path": "asdf"}}`, &CloudConfig{NetworkConfigPath: "asdf"}, false},
+		{`{"hostname": "turkleton"}`, &CloudConfig{Hostname: "turkleton"}, false},
+		{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"alice", "jill"}}, false},
+		{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
+	} {
+		got, err := ParseMetaData(tt.in)
+		if tt.err != (err != nil) {
+			t.Errorf("case #%d: bad error state: got %t, want %t (err=%v)", i, (err != nil), tt.err, err)
+		}
+		if got == nil {
+			if tt.want != nil {
+				t.Errorf("case #%d: unexpected nil output", i)
+			}
+		} else if tt.want == nil {
+			t.Errorf("case #%d: unexpected non-nil output", i)
+		} else {
+			if !reflect.DeepEqual(*got, *tt.want) {
+				t.Errorf("case #%d: bad output:\ngot\n%v\nwant\n%v", i, *got, *tt.want)
+			}
+		}
+	}
+
+}
+
+func TestExtractIPsFromMetadata(t *testing.T) {
+	for i, tt := range []struct {
+		in  []byte
+		err bool
+		out map[string]string
+	}{
+		{
+			[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4", "public-ipv6": "1234::", "local-ipv6": "5678::"}`),
+			false,
+			map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4", "$public_ipv6": "1234::", "$private_ipv6": "5678::"},
+		},
+		{
+			[]byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`),
+			false,
+			map[string]string{"$private_ipv4": "127.0.0.1"},
+		},
+		{
+			[]byte(`garbage`),
+			true,
+			nil,
+		},
+	} {
+		got, err := ExtractIPsFromMetadata(tt.in)
+		if (err != nil) != tt.err {
+			t.Errorf("bad error state (got %t, want %t)", err != nil, tt.err)
+		}
+		if !reflect.DeepEqual(got, tt.out) {
+			t.Errorf("case %d: got %s, want %s", i, got, tt.out)
+		}
+	}
+}

initialize/oem.go

@@ -0,0 +1,41 @@
+package initialize
+
+import (
+	"fmt"
+	"path"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type OEMRelease struct {
+	ID           string `yaml:"id"`
+	Name         string `yaml:"name"`
+	VersionID    string `yaml:"version-id"`
+	HomeURL      string `yaml:"home-url"`
+	BugReportURL string `yaml:"bug-report-url"`
+}
+
+func (oem OEMRelease) String() string {
+	fields := []string{
+		fmt.Sprintf("ID=%s", oem.ID),
+		fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
+		fmt.Sprintf("NAME=%q", oem.Name),
+		fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
+		fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
+	}
+
+	return strings.Join(fields, "\n") + "\n"
+}
+
+func (oem OEMRelease) File(root string) (*system.File, error) {
+	if oem.ID == "" {
+		return nil, nil
+	}
+
+	return &system.File{
+		Path:               path.Join("etc", "oem-release"),
+		RawFilePermissions: "0644",
+		Content:            oem.String(),
+	}, nil
+}

initialize/oem_test.go

@@ -0,0 +1,63 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+func TestOEMReleaseWrittenToDisk(t *testing.T) {
+	oem := OEMRelease{
+		ID:           "rackspace",
+		Name:         "Rackspace Cloud Servers",
+		VersionID:    "168.0.0",
+		HomeURL:      "https://www.rackspace.com/cloud/servers/",
+		BugReportURL: "https://github.com/coreos/coreos-overlay",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+
+	f, err := oem.File(dir)
+	if err != nil {
+		t.Fatalf("Processing of OEMRelease failed: %v", err)
+	}
+	if f == nil {
+		t.Fatalf("OEMRelease returned nil file unexpectedly")
+	}
+
+	if _, err := system.WriteFile(f, dir); err != nil {
+		t.Fatalf("Writing of OEMRelease failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "oem-release")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `ID=rackspace
+VERSION_ID=168.0.0
+NAME="Rackspace Cloud Servers"
+HOME_URL="https://www.rackspace.com/cloud/servers/"
+BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}

initialize/ssh_keys.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (

initialize/ssh_keys_test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
@@ -53,4 +39,31 @@ func TestCloudConfigUsersUrlMarshal(t *testing.T) {
 	if keys[2] != expected {
 		t.Fatalf("expected %s, got %s", expected, keys[2])
 	}
+
+}
+func TestCloudConfigUsersSSHImportURL(t *testing.T) {
+
+	contents := `
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
+		t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
+	}
 }

initialize/update.go

@@ -0,0 +1,165 @@
+package initialize
+
+import (
+	"bufio"
+	"errors"
+	"fmt"
+	"os"
+	"path"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const (
+	locksmithUnit    = "locksmithd.service"
+	updateEngineUnit = "update-engine.service"
+)
+
+// updateOption represents a configurable update option, which, if set, will be
+// written into update.conf, replacing any existing value for the option
+type updateOption struct {
+	key    string   // key used to configure this option in cloud-config
+	valid  []string // valid values for the option
+	prefix string   // prefix for the option in the update.conf file
+	value  string   // used to store the new value in update.conf (including prefix)
+	seen   bool     // whether the option has been seen in any existing update.conf
+}
+
+// updateOptions defines the update options understood by cloud-config.
+// The keys represent the string used in cloud-config to configure the option.
+var updateOptions = []*updateOption{
+	&updateOption{
+		key:    "reboot-strategy",
+		prefix: "REBOOT_STRATEGY=",
+		valid:  []string{"best-effort", "etcd-lock", "reboot", "off"},
+	},
+	&updateOption{
+		key:    "group",
+		prefix: "GROUP=",
+	},
+	&updateOption{
+		key:    "server",
+		prefix: "SERVER=",
+	},
+}
+
+// isValid checks whether a supplied value is valid for this option
+func (uo updateOption) isValid(val string) bool {
+	if len(uo.valid) == 0 {
+		return true
+	}
+	for _, v := range uo.valid {
+		if val == v {
+			return true
+		}
+	}
+	return false
+}
+
+type UpdateConfig map[string]string
+
+// File generates an `/etc/coreos/update.conf` file (if any update
+// configuration options are set in cloud-config) by either rewriting the
+// existing file on disk, or starting from `/usr/share/coreos/update.conf`
+func (uc UpdateConfig) File(root string) (*system.File, error) {
+	if len(uc) < 1 {
+		return nil, nil
+	}
+
+	var out string
+
+	// Generate the list of possible substitutions to be performed based on the options that are configured
+	subs := make([]*updateOption, 0)
+	for _, uo := range updateOptions {
+		val, ok := uc[uo.key]
+		if !ok {
+			continue
+		}
+		if !uo.isValid(val) {
+			return nil, errors.New(fmt.Sprintf("invalid value %v for option %v (valid options: %v)", val, uo.key, uo.valid))
+		}
+		uo.value = uo.prefix + val
+		subs = append(subs, uo)
+	}
+
+	etcUpdate := path.Join(root, "etc", "coreos", "update.conf")
+	usrUpdate := path.Join(root, "usr", "share", "coreos", "update.conf")
+
+	conf, err := os.Open(etcUpdate)
+	if os.IsNotExist(err) {
+		conf, err = os.Open(usrUpdate)
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	scanner := bufio.NewScanner(conf)
+
+	for scanner.Scan() {
+		line := scanner.Text()
+		for _, s := range subs {
+			if strings.HasPrefix(line, s.prefix) {
+				line = s.value
+				s.seen = true
+				break
+			}
+		}
+		out += line
+		out += "\n"
+		if err := scanner.Err(); err != nil {
+			return nil, err
+		}
+	}
+
+	for _, s := range subs {
+		if !s.seen {
+			out += s.value
+			out += "\n"
+		}
+	}
+
+	return &system.File{
+		Path:               path.Join("etc", "coreos", "update.conf"),
+		RawFilePermissions: "0644",
+		Content:            out,
+	}, nil
+}
+
+// Units generates units for the cloud-init initializer to act on:
+// - a locksmith system.Unit, if "reboot-strategy" was set in cloud-config
+// - an update_engine system.Unit, if "group" was set in cloud-config
+func (uc UpdateConfig) Units(root string) ([]system.Unit, error) {
+	var units []system.Unit
+	if strategy, ok := uc["reboot-strategy"]; ok {
+		ls := &system.Unit{
+			Name:    locksmithUnit,
+			Command: "restart",
+			Mask:    false,
+			Runtime: true,
+		}
+
+		if strategy == "off" {
+			ls.Command = "stop"
+			ls.Mask = true
+		}
+		units = append(units, *ls)
+	}
+
+	rue := false
+	if _, ok := uc["group"]; ok {
+		rue = true
+	}
+	if _, ok := uc["server"]; ok {
+		rue = true
+	}
+	if rue {
+		ue := system.Unit{
+			Name:    updateEngineUnit,
+			Command: "restart",
+		}
+		units = append(units, ue)
+	}
+
+	return units, nil
+}

initialize/update_test.go

@@ -0,0 +1,232 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"sort"
+	"strings"
+	"testing"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const (
+	base = `SERVER=https://example.com
+GROUP=thegroupc`
+	configured = base + `
+REBOOT_STRATEGY=awesome
+`
+	expected = base + `
+REBOOT_STRATEGY=etcd-lock
+`
+)
+
+func setupFixtures(dir string) {
+	os.MkdirAll(path.Join(dir, "usr", "share", "coreos"), 0755)
+	os.MkdirAll(path.Join(dir, "run", "systemd", "system"), 0755)
+
+	ioutil.WriteFile(path.Join(dir, "usr", "share", "coreos", "update.conf"), []byte(base), 0644)
+}
+
+func TestEmptyUpdateConfig(t *testing.T) {
+	uc := &UpdateConfig{}
+	f, err := uc.File("")
+	if err != nil {
+		t.Error("unexpected error getting file from empty UpdateConfig")
+	}
+	if f != nil {
+		t.Errorf("getting file from empty UpdateConfig should have returned nil, got %v", f)
+	}
+	uu, err := uc.Units("")
+	if err != nil {
+		t.Error("unexpected error getting unit from empty UpdateConfig")
+	}
+	if len(uu) != 0 {
+		t.Errorf("getting unit from empty UpdateConfig should have returned zero units, got %d", len(uu))
+	}
+}
+
+func TestInvalidUpdateOptions(t *testing.T) {
+	uon := &updateOption{
+		key:    "numbers",
+		prefix: "numero_",
+		valid:  []string{"one", "two"},
+	}
+	uoa := &updateOption{
+		key:    "any_will_do",
+		prefix: "any_",
+	}
+
+	if !uon.isValid("one") {
+		t.Error("update option did not accept valid option \"one\"")
+	}
+	if uon.isValid("three") {
+		t.Error("update option accepted invalid option \"three\"")
+	}
+	for _, s := range []string{"one", "asdf", "foobarbaz"} {
+		if !uoa.isValid(s) {
+			t.Errorf("update option with no \"valid\" field did not accept %q", s)
+		}
+	}
+
+	uc := &UpdateConfig{"reboot-strategy": "wizzlewazzle"}
+	f, err := uc.File("")
+	if err == nil {
+		t.Errorf("File did not give an error on invalid UpdateOption")
+	}
+	if f != nil {
+		t.Errorf("File did not return a nil file on invalid UpdateOption")
+	}
+}
+
+func TestServerGroupOptions(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	setupFixtures(dir)
+	u := &UpdateConfig{"group": "master", "server": "http://foo.com"}
+
+	want := `
+GROUP=master
+SERVER=http://foo.com`
+
+	f, err := u.File(dir)
+	if err != nil {
+		t.Errorf("unexpected error getting file from UpdateConfig: %v", err)
+	} else if f == nil {
+		t.Error("unexpectedly got empty file from UpdateConfig")
+	} else {
+		out := strings.Split(f.Content, "\n")
+		sort.Strings(out)
+		got := strings.Join(out, "\n")
+		if got != want {
+			t.Errorf("File has incorrect contents, got %v, want %v", got, want)
+		}
+	}
+
+	uu, err := u.Units(dir)
+	if err != nil {
+		t.Errorf("unexpected error getting units from UpdateConfig: %v", err)
+	} else if len(uu) != 1 {
+		t.Errorf("unexpected number of files returned from UpdateConfig: want 1, got %d", len(uu))
+	} else {
+		unit := uu[0]
+		if unit.Name != "update-engine.service" {
+			t.Errorf("bad name for generated unit: want update-engine.service, got %s", unit.Name)
+		}
+		if unit.Command != "restart" {
+			t.Errorf("bad command for generated unit: want restart, got %s", unit.Command)
+		}
+	}
+}
+
+func TestRebootStrategies(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	setupFixtures(dir)
+	strategies := []struct {
+		name     string
+		line     string
+		uMask    bool
+		uCommand string
+	}{
+		{"best-effort", "REBOOT_STRATEGY=best-effort", false, "restart"},
+		{"etcd-lock", "REBOOT_STRATEGY=etcd-lock", false, "restart"},
+		{"reboot", "REBOOT_STRATEGY=reboot", false, "restart"},
+		{"off", "REBOOT_STRATEGY=off", true, "stop"},
+	}
+	for _, s := range strategies {
+		uc := &UpdateConfig{"reboot-strategy": s.name}
+		f, err := uc.File(dir)
+		if err != nil {
+			t.Errorf("update failed to generate file for reboot-strategy=%v: %v", s.name, err)
+		} else if f == nil {
+			t.Errorf("generated empty file for reboot-strategy=%v", s.name)
+		} else {
+			seen := false
+			for _, line := range strings.Split(f.Content, "\n") {
+				if line == s.line {
+					seen = true
+					break
+				}
+			}
+			if !seen {
+				t.Errorf("couldn't find expected line %v for reboot-strategy=%v", s.line)
+			}
+		}
+		uu, err := uc.Units(dir)
+		if err != nil {
+			t.Errorf("failed to generate unit for reboot-strategy=%v!", s.name)
+		} else if len(uu) != 1 {
+			t.Errorf("unexpected number of units for reboot-strategy=%v: %d", s.name, len(uu))
+		} else {
+			u := uu[0]
+			if u.Name != locksmithUnit {
+				t.Errorf("unit generated for reboot strategy=%v had bad name: %v", s.name, u.Name)
+			}
+			if u.Mask != s.uMask {
+				t.Errorf("unit generated for reboot strategy=%v had bad mask: %t", s.name, u.Mask)
+			}
+			if u.Command != s.uCommand {
+				t.Errorf("unit generated for reboot strategy=%v had bad command: %v", s.name, u.Command)
+			}
+		}
+	}
+
+}
+
+func TestUpdateConfWrittenToDisk(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer os.RemoveAll(dir)
+	setupFixtures(dir)
+
+	for i := 0; i < 2; i++ {
+		if i == 1 {
+			err = ioutil.WriteFile(path.Join(dir, "etc", "coreos", "update.conf"), []byte(configured), 0644)
+			if err != nil {
+				t.Fatal(err)
+			}
+		}
+		uc := &UpdateConfig{"reboot-strategy": "etcd-lock"}
+
+		f, err := uc.File(dir)
+		if err != nil {
+			t.Fatalf("Processing UpdateConfig failed: %v", err)
+		} else if f == nil {
+			t.Fatal("Unexpectedly got nil updateconfig file")
+		}
+
+		if _, err := system.WriteFile(f, dir); err != nil {
+			t.Fatalf("Error writing update config: %v", err)
+		}
+
+		fullPath := path.Join(dir, "etc", "coreos", "update.conf")
+
+		fi, err := os.Stat(fullPath)
+		if err != nil {
+			t.Fatalf("Unable to stat file: %v", err)
+		}
+
+		if fi.Mode() != os.FileMode(0644) {
+			t.Errorf("File has incorrect mode: %v", fi.Mode())
+		}
+
+		contents, err := ioutil.ReadFile(fullPath)
+		if err != nil {
+			t.Fatalf("Unable to read expected file: %v", err)
+		}
+
+		if string(contents) != expected {
+			t.Fatalf("File has incorrect contents, got %v, wanted %v", string(contents), expected)
+		}
+	}
+}

initialize/user_data.go

@@ -1,45 +1,31 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
-	"errors"
+	"fmt"
 	"log"
+	"strings"
 
-	"github.com/coreos/coreos-cloudinit/config"
+	"github.com/coreos/coreos-cloudinit/system"
-)
-
-var (
-	ErrIgnitionConfig = errors.New("not a config (found Ignition)")
 )
 
 func ParseUserData(contents string) (interface{}, error) {
 	if len(contents) == 0 {
 		return nil, nil
 	}
+	header := strings.SplitN(contents, "\n", 2)[0]
 
-	switch {
+	// Explicitly trim the header so we can handle user-data from
-	case config.IsScript(contents):
+	// non-unix operating systems. The rest of the file is parsed
+	// by yaml, which correctly handles CRLF.
+	header = strings.TrimSpace(header)
+
+	if strings.HasPrefix(header, "#!") {
 		log.Printf("Parsing user-data as script")
-		return config.NewScript(contents)
+		return system.Script(contents), nil
-	case config.IsCloudConfig(contents):
+	} else if header == "#cloud-config" {
 		log.Printf("Parsing user-data as cloud-config")
-		return config.NewCloudConfig(contents)
+		return NewCloudConfig(contents)
-	case config.IsIgnitionConfig(contents):
+	} else {
-		return nil, ErrIgnitionConfig
+		return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
-	default:
-		return nil, errors.New("Unrecognized user-data format")
 	}
 }

initialize/user_data_test.go

@@ -1,23 +1,7 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
 	"testing"
-
-	"github.com/coreos/coreos-cloudinit/config"
 )
 
 func TestParseHeaderCRLF(t *testing.T) {
@@ -47,13 +31,13 @@ func TestParseHeaderCRLF(t *testing.T) {
 }
 
 func TestParseConfigCRLF(t *testing.T) {
-	contents := "#cloud-config \r\nhostname: foo\r\nssh_authorized_keys:\r\n  - foobar\r\n"
+	contents := "#cloud-config\r\nhostname: foo\r\nssh_authorized_keys:\r\n  - foobar\r\n"
 	ud, err := ParseUserData(contents)
 	if err != nil {
 		t.Fatalf("Failed parsing config: %v", err)
 	}
 
-	cfg := ud.(*config.CloudConfig)
+	cfg := ud.(*CloudConfig)
 
 	if cfg.Hostname != "foo" {
 		t.Error("Failed parsing hostname from config")

initialize/workspace.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package initialize
 
 import (
@@ -19,7 +5,6 @@ import (
 	"path"
 	"strings"
 
-	"github.com/coreos/coreos-cloudinit/config"
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
@@ -36,7 +21,7 @@ func PrepWorkspace(workspace string) error {
 	return nil
 }
 
-func PersistScriptInWorkspace(script config.Script, workspace string) (string, error) {
+func PersistScriptInWorkspace(script system.Script, workspace string) (string, error) {
 	scriptsPath := path.Join(workspace, "scripts")
 	tmp, err := ioutil.TempFile(scriptsPath, "")
 	if err != nil {
@@ -46,21 +31,21 @@ func PersistScriptInWorkspace(script config.Script, workspace string) (string, e
 
 	relpath := strings.TrimPrefix(tmp.Name(), workspace)
 
-	file := system.File{File: config.File{
+	file := system.File{
 		Path:               relpath,
 		RawFilePermissions: "0744",
 		Content:            string(script),
-	}}
+	}
 
 	return system.WriteFile(&file, workspace)
 }
 
 func PersistUnitNameInWorkspace(name string, workspace string) error {
-	file := system.File{File: config.File{
+	file := system.File{
 		Path:               path.Join("scripts", "unit-name"),
 		RawFilePermissions: "0644",
 		Content:            name,
-	}}
+	}
 	_, err := system.WriteFile(&file, workspace)
 	return err
 }

network/debian.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (
@@ -19,9 +5,9 @@ import (
 	"strings"
 )
 
-func ProcessDebianNetconf(config []byte) ([]InterfaceGenerator, error) {
+func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) {
 	log.Println("Processing Debian network config")
-	lines := formatConfig(string(config))
+	lines := formatConfig(config)
 	stanzas, err := parseStanzas(lines)
 	if err != nil {
 		return nil, err

network/debian_test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (
@@ -44,10 +30,10 @@ func TestProcessDebianNetconf(t *testing.T) {
 		{"auto eth1\nauto eth2", false, 0},
 		{"iface eth1 inet manual", false, 1},
 	} {
-		interfaces, err := ProcessDebianNetconf([]byte(tt.in))
+		interfaces, err := ProcessDebianNetconf(tt.in)
 		failed := err != nil
 		if tt.fail != failed {
-			t.Fatalf("bad failure state for %q: got %t, want %t", tt.in, failed, tt.fail)
+			t.Fatalf("bad failure state for %q: got %b, want %b", failed, tt.fail)
 		}
 		if tt.n != -1 && tt.n != len(interfaces) {
 			t.Fatalf("bad number of interfaces for %q: got %d, want %q", tt.in, len(interfaces), tt.n)

network/digitalocean.go

@@ -1,20 +1,7 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (
+	"encoding/json"
 	"fmt"
 	"log"
 	"net"
@@ -22,18 +9,26 @@ import (
 	"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
 )
 
-func ProcessDigitalOceanNetconf(config digitalocean.Metadata) ([]InterfaceGenerator, error) {
+func ProcessDigitalOceanNetconf(config string) ([]InterfaceGenerator, error) {
 	log.Println("Processing DigitalOcean network config")
+	if config == "" {
+		return nil, nil
+	}
+
+	var cfg digitalocean.Metadata
+	if err := json.Unmarshal([]byte(config), &cfg); err != nil {
+		return nil, err
+	}
 
 	log.Println("Parsing nameservers")
-	nameservers, err := parseNameservers(config.DNS)
+	nameservers, err := parseNameservers(cfg.DNS)
 	if err != nil {
 		return nil, err
 	}
 	log.Printf("Parsed %d nameservers\n", len(nameservers))
 
 	log.Println("Parsing interfaces")
-	generators, err := parseInterfaces(config.Interfaces, nameservers)
+	generators, err := parseInterfaces(cfg.Interfaces, nameservers)
 	if err != nil {
 		return nil, err
 	}
@@ -43,9 +38,9 @@ func ProcessDigitalOceanNetconf(config digitalocean.Metadata) ([]InterfaceGenera
 	return generators, nil
 }
 
-func parseNameservers(config digitalocean.DNS) ([]net.IP, error) {
+func parseNameservers(cfg digitalocean.DNS) ([]net.IP, error) {
-	nameservers := make([]net.IP, 0, len(config.Nameservers))
+	nameservers := make([]net.IP, 0, len(cfg.Nameservers))
-	for _, ns := range config.Nameservers {
+	for _, ns := range cfg.Nameservers {
 		if ip := net.ParseIP(ns); ip == nil {
 			return nil, fmt.Errorf("could not parse %q as nameserver IP address", ns)
 		} else {
@@ -55,16 +50,16 @@ func parseNameservers(config digitalocean.DNS) ([]net.IP, error) {
 	return nameservers, nil
 }
 
-func parseInterfaces(config digitalocean.Interfaces, nameservers []net.IP) ([]InterfaceGenerator, error) {
+func parseInterfaces(cfg digitalocean.Interfaces, nameservers []net.IP) ([]InterfaceGenerator, error) {
-	generators := make([]InterfaceGenerator, 0, len(config.Public)+len(config.Private))
+	generators := make([]InterfaceGenerator, 0, len(cfg.Public)+len(cfg.Private))
-	for _, iface := range config.Public {
+	for _, iface := range cfg.Public {
 		if generator, err := parseInterface(iface, nameservers, true); err == nil {
 			generators = append(generators, &physicalInterface{*generator})
 		} else {
 			return nil, err
 		}
 	}
-	for _, iface := range config.Private {
+	for _, iface := range cfg.Private {
 		if generator, err := parseInterface(iface, []net.IP{}, false); err == nil {
 			generators = append(generators, &physicalInterface{*generator})
 		} else {
@@ -126,28 +121,6 @@ func parseInterface(iface digitalocean.Interface, nameservers []net.IP, useRoute
 			})
 		}
 	}
-	if iface.AnchorIPv4 != nil {
-		var ip, mask net.IP
-		if ip = net.ParseIP(iface.AnchorIPv4.IPAddress); ip == nil {
-			return nil, fmt.Errorf("could not parse %q as anchor IPv4 address", iface.AnchorIPv4.IPAddress)
-		}
-		if mask = net.ParseIP(iface.AnchorIPv4.Netmask); mask == nil {
-			return nil, fmt.Errorf("could not parse %q as anchor IPv4 mask", iface.AnchorIPv4.Netmask)
-		}
-		addresses = append(addresses, net.IPNet{
-			IP:   ip,
-			Mask: net.IPMask(mask),
-		})
-
-		if useRoute {
-			routes = append(routes, route{
-				destination: net.IPNet{
-					IP:   net.IPv4zero,
-					Mask: net.IPMask(net.IPv4zero),
-				},
-			})
-		}
-	}
 
 	hwaddr, err := net.ParseMAC(iface.MAC)
 	if err != nil {

network/digitalocean_test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (
@@ -34,11 +20,11 @@ func TestParseNameservers(t *testing.T) {
 			nss: []net.IP{},
 		},
 		{
-			dns: digitalocean.DNS{Nameservers: []string{"1.2.3.4"}},
+			dns: digitalocean.DNS{[]string{"1.2.3.4"}},
 			nss: []net.IP{net.ParseIP("1.2.3.4")},
 		},
 		{
-			dns: digitalocean.DNS{Nameservers: []string{"bad"}},
+			dns: digitalocean.DNS{[]string{"bad"}},
 			err: errors.New("could not parse \"bad\" as nameserver IP address"),
 		},
 	} {
@@ -52,14 +38,6 @@ func TestParseNameservers(t *testing.T) {
 	}
 }
 
-func mkInvalidMAC() error {
-	if isGo15 {
-		return &net.AddrError{Err: "invalid MAC address", Addr: "bad"}
-	} else {
-		return errors.New("invalid MAC address: bad")
-	}
-}
-
 func TestParseInterface(t *testing.T) {
 	for _, tt := range []struct {
 		cfg      digitalocean.Interface
@@ -72,7 +50,7 @@ func TestParseInterface(t *testing.T) {
 			cfg: digitalocean.Interface{
 				MAC: "bad",
 			},
-			err: mkInvalidMAC(),
+			err: errors.New("invalid MAC address: bad"),
 		},
 		{
 			cfg: digitalocean.Interface{
@@ -138,10 +116,7 @@ func TestParseInterface(t *testing.T) {
 			iface: &logicalInterface{
 				hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
 				config: configMethodStatic{
-					addresses: []net.IPNet{net.IPNet{
+					addresses:   []net.IPNet{net.IPNet{net.ParseIP("1.2.3.4"), net.IPMask(net.ParseIP("255.255.0.0"))}},
-						IP:   net.ParseIP("1.2.3.4"),
-						Mask: net.IPMask(net.ParseIP("255.255.0.0")),
-					}},
 					nameservers: []net.IP{},
 					routes:      []route{},
 				},
@@ -174,15 +149,9 @@ func TestParseInterface(t *testing.T) {
 			iface: &logicalInterface{
 				hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
 				config: configMethodStatic{
-					addresses: []net.IPNet{net.IPNet{
+					addresses:   []net.IPNet{net.IPNet{net.ParseIP("1.2.3.4"), net.IPMask(net.ParseIP("255.255.0.0"))}},
-						IP:   net.ParseIP("1.2.3.4"),
-						Mask: net.IPMask(net.ParseIP("255.255.0.0")),
-					}},
 					nameservers: []net.IP{},
-					routes: []route{route{
+					routes:      []route{route{net.IPNet{net.IPv4zero, net.IPMask(net.IPv4zero)}, net.ParseIP("5.6.7.8")}},
-						net.IPNet{IP: net.IPv4zero, Mask: net.IPMask(net.IPv4zero)},
-						net.ParseIP("5.6.7.8"),
-					}},
 				},
 			},
 		},
@@ -210,10 +179,7 @@ func TestParseInterface(t *testing.T) {
 			iface: &logicalInterface{
 				hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
 				config: configMethodStatic{
-					addresses: []net.IPNet{net.IPNet{
+					addresses:   []net.IPNet{net.IPNet{net.ParseIP("fe00::"), net.IPMask(net.ParseIP("ffff::"))}},
-						IP:   net.ParseIP("fe00::"),
-						Mask: net.IPMask(net.ParseIP("ffff::")),
-					}},
 					nameservers: []net.IP{},
 					routes:      []route{},
 				},
@@ -246,79 +212,9 @@ func TestParseInterface(t *testing.T) {
 			iface: &logicalInterface{
 				hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
 				config: configMethodStatic{
-					addresses: []net.IPNet{net.IPNet{
+					addresses:   []net.IPNet{net.IPNet{net.ParseIP("fe00::"), net.IPMask(net.ParseIP("ffff::"))}},
-						IP:   net.ParseIP("fe00::"),
-						Mask: net.IPMask(net.ParseIP("ffff::")),
-					}},
 					nameservers: []net.IP{},
-					routes: []route{route{
+					routes:      []route{route{net.IPNet{net.IPv6zero, net.IPMask(net.IPv6zero)}, net.ParseIP("fe00:1234::")}},
-						net.IPNet{IP: net.IPv6zero, Mask: net.IPMask(net.IPv6zero)},
-						net.ParseIP("fe00:1234::"),
-					}},
-				},
-			},
-		},
-
-		{
-			cfg: digitalocean.Interface{
-				MAC: "01:23:45:67:89:AB",
-				AnchorIPv4: &digitalocean.Address{
-					IPAddress: "bad",
-					Netmask:   "255.255.0.0",
-				},
-			},
-			nss: []net.IP{},
-			err: errors.New("could not parse \"bad\" as anchor IPv4 address"),
-		},
-		{
-			cfg: digitalocean.Interface{
-				MAC: "01:23:45:67:89:AB",
-				AnchorIPv4: &digitalocean.Address{
-					IPAddress: "1.2.3.4",
-					Netmask:   "bad",
-				},
-			},
-			nss: []net.IP{},
-			err: errors.New("could not parse \"bad\" as anchor IPv4 mask"),
-		},
-		{
-			cfg: digitalocean.Interface{
-				MAC: "01:23:45:67:89:AB",
-				IPv4: &digitalocean.Address{
-					IPAddress: "1.2.3.4",
-					Netmask:   "255.255.0.0",
-					Gateway:   "5.6.7.8",
-				},
-				AnchorIPv4: &digitalocean.Address{
-					IPAddress: "7.8.9.10",
-					Netmask:   "255.255.0.0",
-				},
-			},
-			useRoute: true,
-			nss:      []net.IP{},
-			iface: &logicalInterface{
-				hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
-				config: configMethodStatic{
-					addresses: []net.IPNet{
-						{
-							IP:   net.ParseIP("1.2.3.4"),
-							Mask: net.IPMask(net.ParseIP("255.255.0.0")),
-						},
-						{
-							IP:   net.ParseIP("7.8.9.10"),
-							Mask: net.IPMask(net.ParseIP("255.255.0.0")),
-						},
-					},
-					nameservers: []net.IP{},
-					routes: []route{
-						{
-							destination: net.IPNet{IP: net.IPv4zero, Mask: net.IPMask(net.IPv4zero)},
-							gateway:     net.ParseIP("5.6.7.8"),
-						},
-						{
-							destination: net.IPNet{IP: net.IPv4zero, Mask: net.IPMask(net.IPv4zero)},
-						},
-					},
 				},
 			},
 		},
@@ -409,13 +305,13 @@ func TestParseInterfaces(t *testing.T) {
 			cfg: digitalocean.Interfaces{
 				Public: []digitalocean.Interface{{MAC: "bad"}},
 			},
-			err: mkInvalidMAC(),
+			err: errors.New("invalid MAC address: bad"),
 		},
 		{
 			cfg: digitalocean.Interfaces{
 				Private: []digitalocean.Interface{{MAC: "bad"}},
 			},
-			err: mkInvalidMAC(),
+			err: errors.New("invalid MAC address: bad"),
 		},
 	} {
 		ifaces, err := parseInterfaces(tt.cfg, tt.nss)
@@ -430,33 +326,23 @@ func TestParseInterfaces(t *testing.T) {
 
 func TestProcessDigitalOceanNetconf(t *testing.T) {
 	for _, tt := range []struct {
-		cfg    digitalocean.Metadata
+		cfg    string
 		ifaces []InterfaceGenerator
 		err    error
 	}{
 		{
-			cfg: digitalocean.Metadata{
+			cfg: ``,
-				DNS: digitalocean.DNS{
+		},
-					Nameservers: []string{"bad"},
+		{
-				},
+			cfg: `{"dns":{"nameservers":["bad"]}}`,
-			},
 			err: errors.New("could not parse \"bad\" as nameserver IP address"),
 		},
 		{
-			cfg: digitalocean.Metadata{
+			cfg: `{"interfaces":{"public":[{"ipv4":{"ip_address":"bad"}}]}}`,
-				Interfaces: digitalocean.Interfaces{
-					Public: []digitalocean.Interface{
-						digitalocean.Interface{
-							IPv4: &digitalocean.Address{
-								IPAddress: "bad",
-							},
-						},
-					},
-				},
-			},
 			err: errors.New("could not parse \"bad\" as IPv4 address"),
 		},
 		{
+			cfg:    `{}`,
 			ifaces: []InterfaceGenerator{},
 		},
 	} {

network/interface.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (
@@ -130,17 +116,7 @@ type bondInterface struct {
 }
 
 func (b *bondInterface) Netdev() string {
-	config := fmt.Sprintf("[NetDev]\nKind=bond\nName=%s\n", b.name)
+	return fmt.Sprintf("[NetDev]\nKind=bond\nName=%s\n", b.name)
-	if b.hwaddr != nil {
-		config += fmt.Sprintf("MACAddress=%s\n", b.hwaddr.String())
-	}
-
-	config += fmt.Sprintf("\n[Bond]\n")
-	for _, name := range sortedKeys(b.options) {
-		config += fmt.Sprintf("%s=%s\n", name, b.options[name])
-	}
-
-	return config
 }
 
 func (b *bondInterface) Type() string {

network/interface_test.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (
@@ -52,7 +38,7 @@ func TestInterfaceGenerators(t *testing.T) {
 		},
 		{
 			name:    "testname",
-			netdev:  "[NetDev]\nKind=bond\nName=testname\n\n[Bond]\n",
+			netdev:  "[NetDev]\nKind=bond\nName=testname\n",
 			network: "[Match]\nName=testname\n\n[Network]\nBond=testbond1\nVLAN=testvlan1\nVLAN=testvlan2\nDHCP=true\n",
 			kind:    "bond",
 			iface: &bondInterface{logicalInterface: logicalInterface{

network/is_go15_false_test.go

@@ -1,5 +0,0 @@
-// +build !go1.5
-
-package network
-
-const isGo15 = false

network/is_go15_true_test.go

@@ -1,5 +0,0 @@
-// +build go1.5
-
-package network
-
-const isGo15 = true

network/packet.go

@@ -1,127 +0,0 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package network
-
-import (
-	"net"
-
-	"github.com/coreos/coreos-cloudinit/datasource/metadata/packet"
-)
-
-func ProcessPacketNetconf(netdata packet.NetworkData) ([]InterfaceGenerator, error) {
-	var nameservers []net.IP
-	if netdata.DNS != nil {
-		nameservers = netdata.DNS
-	} else {
-		nameservers = append(nameservers, net.ParseIP("8.8.8.8"), net.ParseIP("8.8.4.4"))
-	}
-
-	generators, err := parseNetwork(netdata, nameservers)
-	if err != nil {
-		return nil, err
-	}
-
-	return generators, nil
-}
-
-func parseNetwork(netdata packet.NetworkData, nameservers []net.IP) ([]InterfaceGenerator, error) {
-	var interfaces []InterfaceGenerator
-	var addresses []net.IPNet
-	var routes []route
-	for _, netblock := range netdata.Netblocks {
-		addresses = append(addresses, net.IPNet{
-			IP:   netblock.Address,
-			Mask: net.IPMask(netblock.Netmask),
-		})
-		if netblock.Public == false {
-			routes = append(routes, route{
-				destination: net.IPNet{
-					IP:   net.IPv4(10, 0, 0, 0),
-					Mask: net.IPv4Mask(255, 0, 0, 0),
-				},
-				gateway: netblock.Gateway,
-			})
-		} else {
-			if netblock.AddressFamily == 4 {
-				routes = append(routes, route{
-					destination: net.IPNet{
-						IP:   net.IPv4zero,
-						Mask: net.IPMask(net.IPv4zero),
-					},
-					gateway: netblock.Gateway,
-				})
-			} else {
-				routes = append(routes, route{
-					destination: net.IPNet{
-						IP:   net.IPv6zero,
-						Mask: net.IPMask(net.IPv6zero),
-					},
-					gateway: netblock.Gateway,
-				})
-			}
-		}
-	}
-
-	bond := bondInterface{
-		logicalInterface: logicalInterface{
-			name: "bond0",
-			config: configMethodStatic{
-				addresses:   addresses,
-				nameservers: nameservers,
-				routes:      routes,
-			},
-		},
-		options: map[string]string{
-			"Mode":             "802.3ad",
-			"LACPTransmitRate": "fast",
-			"MIIMonitorSec":    ".2",
-			"UpDelaySec":       ".2",
-			"DownDelaySec":     ".2",
-		},
-	}
-
-	for _, iface := range netdata.Interfaces {
-		if iface.Name != "chassis0" && iface.Name != "ipmi0" {
-			bond.slaves = append(bond.slaves, iface.Name)
-			if iface.Name == "enp1s0f0" {
-				bond.hwaddr, _ = net.ParseMAC(iface.Mac)
-			}
-		}
-	}
-
-	for _, iface := range netdata.Interfaces {
-		if iface.Name != "chassis0" && iface.Name != "ipmi0" {
-			p := physicalInterface{
-				logicalInterface: logicalInterface{
-					name: iface.Name,
-					config: configMethodStatic{
-						nameservers: nameservers,
-					},
-					children: []networkInterface{&bond},
-				},
-			}
-
-			if iface.Name == "enp1s0f0" {
-				p.configDepth = 20
-			}
-
-			interfaces = append(interfaces, &p)
-		}
-	}
-
-	interfaces = append(interfaces, &bond)
-
-	return interfaces, nil
-}

network/stanza.go

@@ -1,17 +1,3 @@
-// Copyright 2015 CoreOS, Inc.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
 package network
 
 import (