Merge pull request #76 from marineam/units2

fix(units): Relax ordering requirements for now.

Documentation/cloud-config.md

@@ -8,29 +8,93 @@ Only a subset of [cloud-config functionality][cloud-config] is implemented. A se
 
 ## CoreOS Parameters
 
-### coreos.etcd.discovery_url
+### coreos.etcd
 
-The value of `coreos.etcd.discovery_url` will be used to discover the instance's etcd peers using the [etcd discovery protocol][disco-proto]. Usage of the [public discovery service][disco-service] is encouraged. **Note:** this is currently Amazon-only.
+The `coreos.etcd.*` options are translated to a partial systemd unit acting as an etcd configuration file.
+We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
 
 ```
 #cloud-config
 
 coreos:
     etcd:
-    discovery_url: https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877
+        name: node001
+	# generate a new token for each unique cluster from https://discovery.etcd.io/new
+        discovery: https://discovery.etcd.io/<token>
+	# multi-region and multi-cloud deployments need to use $public_ipv4
+        addr: $public_ipv4:4001
+        peer-addr: $private_ipv4:7001
 ```
 
-[disco-proto]: https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md
+...will generate a systemd unit drop-in like this:
-[disco-service]: http://discovery.etcd.io
+
+```
+[Service]
+Environment="ETCD_NAME=node001"
+Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
+Environment="ETCD_ADDR=203.0.113.29:4001"
+Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
+```
+
+For more information about the available configuration options, see the [etcd documentation][etcd-config].
+Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
+
+[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
+
+### coreos.oem
+
+These fields are borrowed from the [os-release spec][os-release] and repurposed
+as a way for coreos-cloudinit to know about the OEM partition on this machine:
+
+- **id**: Lowercase string identifying the OEM
+- **name**: Human-friendly string representing the OEM
+- **version-id**: Lowercase string identifying the version of the OEM
+- **home-url**: Link to the homepage of the provider or OEM
+- **bug-report-url**: Link to a place to file bug reports about this OEM
+
+coreos-cloudinit renders these fields to `/etc/oem-release`.
+If no **id** field is provided, coreos-cloudinit will ignore this section.
+
+For example, the following cloud-config document...
+
+```
+#cloud-config
+coreos:
+  oem:
+    id: rackspace
+    name: Rackspace Cloud Servers
+    version-id: 168.0.0
+    home-url: https://www.rackspace.com/cloud/servers/
+    bug-report-url: https://github.com/coreos/coreos-overlay
+```
+
+...would be rendered to the following `/etc/oem-release`:
+
+```
+ID=rackspace
+NAME="Rackspace Cloud Servers"
+VERSION_ID=168.0.0
+HOME_URL="https://www.rackspace.com/cloud/servers/"
+BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
+```
+
+[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html
 
 ### coreos.units
 
 Arbitrary systemd units may be provided in the `coreos.units` attribute.
 `coreos.units` is a list of objects with the following fields:
 
-- **name**: string representing unit's name
+- **name**: String representing unit's name. Required.
-- **runtime**: boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` flag to `systemd enable`.
+- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` argument to `systemd enable`. Default value is false.
-- **content**: plaintext string representing entire unit file
+- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
+- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
+
+**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
+
+##### Examples
+
+Write a unit to disk, automatically starting it.
 
 ```
 #cloud-config
@@ -53,6 +117,19 @@ coreos:
           WantedBy=local.target
 ```
 
+Start the builtin `etcd` and `fleet` services:
+
+```
+# cloud-config
+
+coreos:
+    units:
+      - name: etcd.service
+        command: start
+      - name: fleet.service
+        command: start
+```
+
 ## Cloud-Config Parameters
 
 ### ssh_authorized_keys
@@ -96,6 +173,7 @@ All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the use
 - **no-user-group**: Boolean. Skip default group creation.
 - **ssh-authorized-keys**: List of public SSH keys to authorize for this user
 - **coreos-ssh-import-github**: Authorize SSH keys from Github user
+- **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint.
 - **system**: Create the user as a system user. No home directory will be created.
 - **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
 
@@ -140,6 +218,41 @@ perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
 
 Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed.  On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms. 
 
+#### Retrieving ssh authorized keys from a GitHub user
+
+Using the field `coreos-ssh-import-github` you can make coreos-cloudinit to add the public ssh keys from a GitHub user as authorized keys to a server.
+
+```
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-github: elroy
+```
+
+#### Retrieving ssh authorized keys from an http endpoint
+
+coreos-cloudinit can also pull public SSH keys from any http endpoint that matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
+For example, if you have an installation of GitHub Enterprise, you can provide a complete url with an authentication token:
+
+```
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://token:<OAUTH-TOKEN>@github-enterprise.example.com/users/elroy/keys
+```
+
+You can also provide any url which response matches that json format for public keys:
+
+```
+#cloud-config
+
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://example.com/public-keys
+```
+
 ### write_files
 
 Inject an arbitrary set of files to the local filesystem.
@@ -150,12 +263,18 @@ Provide a list of objects with the following attributes:
 - **permissions**: String representing file permissions in octal notation (i.e. '0644')
 - **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
 
-## user-data Script
+Explicitly not implemented is the **encoding** attribute.
+The **content** field must represent exactly what should be written to disk.
 
-Simply set your user-data to a script where the first line is a shebang:
+### manage_etc_hosts
+
+Have coreos-cloudinit manage your /etc/hosts file for local name resolution.
+The only supported value is "localhost" which will cause your system's hostname
+to resolve to "127.0.0.1".  This is helpful when the host does not have DNS
+infrastructure in place to resolve its own hostname, for example, when using Vagrant.
 
 ```
-#!/bin/bash
+#cloud-config
 
-echo 'Hello, world!'
+manage_etc_hosts: localhost
 ```

Documentation/config-drive.md

@@ -0,0 +1,30 @@
+# Distribution via Config Drive
+
+CoreOS supports providing configuration data via [config drive][config-drive]
+disk images. Currently only providing a single script or cloud config file is
+supported.
+
+[config-drive]: http://docs.openstack.org/user-guide/content/enable_config_drive.html#config_drive_contents
+
+## Contents and Format
+
+The image should be a single FAT or ISO9660 file system with the label
+`config-2` and the configuration data should be located at
+`openstack/latest/user_data`.
+
+For example, to wrap up a config named `user_data` in a config drive image:
+
+    mkdir -p /tmp/new-drive/openstack/latest
+    cp user_data /tmp/new-drive/openstack/latest/user_data
+    mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
+    rm -r /tmp/new-drive
+
+## QEMU virtfs
+
+One exception to the above, when using QEMU it is possible to skip creating an
+image and use a plain directory containing the same contents:
+
+    qemu-system-x86_64 \
+        -fsdev local,id=conf,security_model=none,readonly,path=/tmp/new-drive \
+        -device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \
+        [usual qemu options here...]

README.md

@@ -1,9 +1,79 @@
 # coreos-cloudinit
 
-coreos-cloudinit enables a user to customize CoreOS machines by providing either an executable script or a cloud-config document as instance user-data.
+coreos-cloudinit enables a user to customize CoreOS machines by providing either a cloud-config document or an executable script through user-data.
 
-## Supported Cloud-Config Features
+## Configuration with cloud-config
 
-A subset of [cloud-config][cloud-config] is implemented in coreos-cloudinit and is [documented here](https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md). In addition specific CoreOS paramaters were added for unit files, etcd discovery urls, and others.
+A subset of the [official cloud-config spec][official-cloud-config] is implemented by coreos-cloudinit.
+Additionally, several [CoreOS-specific options][custom-cloud-config] have been implemented to support interacting with unit files, bootstrapping etcd clusters, and more.
+All supported cloud-config parameters are [documented here][all-cloud-config]. 
 
-[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
+[official-cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
+[custom-cloud-config]: https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/cloud-config.md#coreos-parameters
+[all-cloud-config]: https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md
+
+The following is an example cloud-config document:
+
+```
+#cloud-config
+
+coreos:
+    units:
+      - name: etcd.service
+        command: start
+
+users:
+  - name: core
+    passwd: $1$allJZawX$00S5T756I5PGdQga5qhqv1
+
+write_files:
+  - path: /etc/resolv.conf
+    content: |
+        nameserver 192.0.2.2
+        nameserver 192.0.2.3
+```
+
+## Executing a Script
+
+coreos-cloudinit supports executing user-data as a script instead of parsing it as a cloud-config document.
+Make sure the first line of your user-data is a shebang and coreos-cloudinit will attempt to execute it:
+
+```
+#!/bin/bash
+
+echo 'Hello, world!'
+```
+
+## user-data Field Substitution
+
+coreos-cloudinit will replace the following set of tokens in your user-data with system-generated values.
+
+| Token         | Description |
+| ------------- | ----------- |
+| $public_ipv4  | Public IPv4 address of machine |
+| $private_ipv4 | Private IPv4 address of machine |
+
+These values are determined by CoreOS based on the given provider on which your machine is running.
+Read more about provider-specific functionality in the [CoreOS OEM documentation][oem-doc].
+
+[oem-doc]: https://coreos.com/docs/sdk-distributors/distributors/notes-for-distributors/
+
+For example, submitting the following user-data...
+
+```
+#cloud-config
+coreos:
+    etcd:
+        addr: $public_ipv4:4001
+        peer-addr: $private_ipv4:7001
+```
+
+...will result in this cloud-config document being executed:
+
+```
+#cloud-config
+coreos:
+    etcd:
+        addr: 203.0.113.29:4001
+        peer-addr: 192.0.2.13:7001
+```

coreos-cloudinit.go

@@ -1,8 +1,6 @@
 package main
 
 import (
-	"bufio"
-	"bytes"
 	"flag"
 	"fmt"
 	"log"
@@ -14,7 +12,7 @@ import (
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
-const version = "0.2.0"
+const version = "0.4.0+git"
 
 func main() {
 	var printVersion bool
@@ -58,7 +56,7 @@ func main() {
 	}
 
 	log.Printf("Fetching user-data from datasource of type %q", ds.Type())
-	userdata, err := ds.Fetch()
+	userdataBytes, err := ds.Fetch()
 	if err != nil {
 		log.Printf("Failed fetching user-data from datasource: %v", err)
 		if ignoreFailure {
@@ -68,11 +66,16 @@ func main() {
 		}
 	}
 
-	if len(userdata) == 0 {
+	if len(userdataBytes) == 0 {
 		log.Printf("No user data to handle, exiting.")
 		os.Exit(0)
 	}
 
+	env := initialize.NewEnvironment("/", workspace)
+
+	userdata := string(userdataBytes)
+	userdata = env.Apply(userdata)
+
 	parsed, err := ParseUserData(userdata)
 	if err != nil {
 		log.Printf("Failed parsing user-data: %v", err)
@@ -83,7 +86,6 @@ func main() {
 		}
 	}
 
-	env := initialize.NewEnvironment("/", workspace)
 	err = initialize.PrepWorkspace(env.Workspace())
 	if err != nil {
 		log.Fatalf("Failed preparing workspace: %v", err)
@@ -107,16 +109,14 @@ func main() {
 	}
 }
 
-func ParseUserData(contents []byte) (interface{}, error) {
+func ParseUserData(contents string) (interface{}, error) {
-	bytereader := bytes.NewReader(contents)
+	header := strings.SplitN(contents, "\n", 2)[0]
-	bufreader := bufio.NewReader(bytereader)
-	header, _ := bufreader.ReadString('\n')
 
 	if strings.HasPrefix(header, "#!") {
 		log.Printf("Parsing user-data as script")
 		return system.Script(contents), nil
 
-	} else if header == "#cloud-config\n" {
+	} else if header == "#cloud-config" {
 		log.Printf("Parsing user-data as cloud-config")
 		cfg, err := initialize.NewCloudConfig(contents)
 		if err != nil {

initialize/config.go

@@ -14,17 +14,18 @@ type CloudConfig struct {
 	SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
 	Coreos            struct {
 		Etcd  EtcdEnvironment
-		Fleet struct{ Autostart bool }
 		Units []system.Unit
+		OEM   OEMRelease
 	}
 	WriteFiles     []system.File `yaml:"write_files"`
 	Hostname       string
 	Users          []system.User
+	ManageEtcHosts string `yaml:"manage_etc_hosts"`
 }
 
-func NewCloudConfig(contents []byte) (*CloudConfig, error) {
+func NewCloudConfig(contents string) (*CloudConfig, error) {
 	var cfg CloudConfig
-	err := goyaml.Unmarshal(contents, &cfg)
+	err := goyaml.Unmarshal([]byte(contents), &cfg)
 	return &cfg, err
 }
 
@@ -48,6 +49,13 @@ func Apply(cfg CloudConfig, env *Environment) error {
 		log.Printf("Set hostname to %s", cfg.Hostname)
 	}
 
+	if cfg.Coreos.OEM.ID != "" {
+		if err := WriteOEMRelease(&cfg.Coreos.OEM, env.Root()); err != nil {
+			return err
+		}
+		log.Printf("Wrote /etc/oem-release to filesystem")
+	}
+
 	if len(cfg.Users) > 0 {
 		for _, user := range cfg.Users {
 			if user.Name == "" {
@@ -84,6 +92,12 @@ func Apply(cfg CloudConfig, env *Environment) error {
 					return err
 				}
 			}
+			if user.SSHImportURL != "" {
+				log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
+				if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
+					return err
+				}
+			}
 		}
 	}
 
@@ -115,8 +129,11 @@ func Apply(cfg CloudConfig, env *Environment) error {
 	}
 
 	if len(cfg.Coreos.Units) > 0 {
+		commands := make(map[string]string, 0)
+
 		for _, unit := range cfg.Coreos.Units {
-			log.Printf("Placing unit %s on filesystem", unit.Name)
+			if unit.Content != "" {
+				log.Printf("Writing unit %s to filesystem", unit.Name)
 				dst, err := system.PlaceUnit(&unit, env.Root())
 				if err != nil {
 					return err
@@ -133,17 +150,34 @@ func Apply(cfg CloudConfig, env *Environment) error {
 					log.Printf("Skipping enable for network-like unit %s", unit.Name)
 				}
 			}
-		system.DaemonReload()
+
-		system.StartUnits(cfg.Coreos.Units)
+			if unit.Group() == "network" {
+				commands["systemd-networkd.service"] = "restart"
+			} else {
+				if unit.Command != "" {
+					commands[unit.Name] = unit.Command
+				}
+			}
 		}
 
-	if cfg.Coreos.Fleet.Autostart {
+		for unit, command := range commands {
-		err := system.StartUnitByName("fleet.service")
+			log.Printf("Calling unit command '%s %s'", command, unit)
-		if err == nil {
+			res, err := system.RunUnitCommand(command, unit)
-			log.Printf("Started fleet service.")
+			if err != nil {
-		} else {
 				return err
 			}
+			log.Printf("Result of '%s %s': %s", command, unit, res)
+		}
+	}
+
+	if cfg.ManageEtcHosts != "" {
+
+		if err := WriteEtcHosts(cfg.ManageEtcHosts, env.Root()); err != nil {
+			log.Fatalf("Failed to write /etc/hosts to filesystem: %v", err)
+		}
+
+		log.Printf("Wrote /etc/hosts file to filesystem")
+
 	}
 
 	return nil

initialize/config_test.go

@@ -7,7 +7,7 @@ import (
 
 // Assert that the parsing of a cloud config file "generally works"
 func TestCloudConfigEmpty(t *testing.T) {
-	cfg, err := NewCloudConfig([]byte{})
+	cfg, err := NewCloudConfig("")
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
 	}
@@ -17,10 +17,6 @@ func TestCloudConfigEmpty(t *testing.T) {
 		t.Error("Parsed incorrect number of SSH keys")
 	}
 
-	if cfg.Coreos.Fleet.Autostart {
-		t.Error("Expected AutostartFleet not to be defined")
-	}
-
 	if len(cfg.WriteFiles) != 0 {
 		t.Error("Expected zero WriteFiles")
 	}
@@ -32,12 +28,10 @@ func TestCloudConfigEmpty(t *testing.T) {
 
 // Assert that the parsing of a cloud config file "generally works"
 func TestCloudConfig(t *testing.T) {
-	contents := []byte(`
+	contents := `
 coreos: 
   etcd:
     discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
-  fleet:
-    autostart: Yes
   units:
     - name: 50-eth0.network
       runtime: yes
@@ -51,6 +45,12 @@ coreos:
     Address=10.209.171.177/19
  
 '
+  oem:
+    id: rackspace
+    name: Rackspace Cloud Servers
+    version-id: 168.0.0
+    home-url: https://www.rackspace.com/cloud/servers/
+    bug-report-url: https://github.com/coreos/coreos-overlay
 ssh_authorized_keys:
   - foobar
   - foobaz
@@ -62,7 +62,7 @@ write_files:
     permissions: '0644'
     owner: root:dogepack
 hostname: trontastic
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
@@ -77,10 +77,6 @@ hostname: trontastic
 		t.Error("Expected first SSH key to be 'foobaz'")
 	}
 
-	if !cfg.Coreos.Fleet.Autostart {
-		t.Error("Expected AutostartFleet to be true")
-	}
-
 	if len(cfg.WriteFiles) != 1 {
 		t.Error("Failed to parse correct number of write_files")
 	} else {
@@ -126,6 +122,10 @@ Address=10.209.171.177/19
 		}
 	}
 
+	if cfg.Coreos.OEM.ID != "rackspace" {
+		t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
+	}
+
 	if cfg.Hostname != "trontastic" {
 		t.Errorf("Failed to parse hostname")
 	}
@@ -133,10 +133,10 @@ Address=10.209.171.177/19
 
 // Assert that our interface conversion doesn't panic
 func TestCloudConfigKeysNotList(t *testing.T) {
-	contents := []byte(`
+	contents := `
 ssh_authorized_keys:
   - foo: bar
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error :%v", err)
@@ -149,7 +149,7 @@ ssh_authorized_keys:
 }
 
 func TestCloudConfigSerializationHeader(t *testing.T) {
-	cfg, _ := NewCloudConfig([]byte{})
+	cfg, _ := NewCloudConfig("")
 	contents := cfg.String()
 	header := strings.SplitN(contents, "\n", 2)[0]
 	if header != "#cloud-config" {
@@ -158,7 +158,7 @@ func TestCloudConfigSerializationHeader(t *testing.T) {
 }
 
 func TestCloudConfigUsers(t *testing.T) {
-	contents := []byte(`
+	contents := `
 users:
   - name: elroy
     passwd: somehash
@@ -174,7 +174,7 @@ users:
     no-user-group: true
     system: y
     no-log-init: True
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error: %v", err)

initialize/env.go

@@ -1,7 +1,9 @@
 package initialize
 
 import (
+	"os"
 	"path"
+	"strings"
 )
 
 const DefaultSSHKeyName = "coreos-cloudinit"
@@ -10,10 +12,15 @@ type Environment struct {
 	root          string
 	workspace     string
 	sshKeyName    string
+	substitutions map[string]string
 }
 
 func NewEnvironment(root, workspace string) *Environment {
-	return &Environment{root, workspace, DefaultSSHKeyName}
+	substitutions := map[string]string{
+		"$public_ipv4":  os.Getenv("COREOS_PUBLIC_IPV4"),
+		"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
+	}
+	return &Environment{root, workspace, DefaultSSHKeyName, substitutions}
 }
 
 func (self *Environment) Workspace() string {
@@ -31,3 +38,10 @@ func (self *Environment) SSHKeyName() string {
 func (self *Environment) SetSSHKeyName(name string) {
 	self.sshKeyName = name
 }
+
+func (self *Environment) Apply(data string) string {
+	for key, val := range self.substitutions {
+		data = strings.Replace(data, key, val, -1)
+	}
+	return data
+}

initialize/env_test.go

@@ -0,0 +1,27 @@
+package initialize
+
+import (
+	"os"
+	"testing"
+)
+
+func TestEnvironmentApply(t *testing.T) {
+	os.Setenv("COREOS_PUBLIC_IPV4", "192.0.2.3")
+	os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.203")
+	env := NewEnvironment("./", "./")
+	input := `[Service]
+ExecStart=/usr/bin/echo "$public_ipv4"
+ExecStop=/usr/bin/echo $private_ipv4
+ExecStop=/usr/bin/echo $unknown
+`
+	expected := `[Service]
+ExecStart=/usr/bin/echo "192.0.2.3"
+ExecStop=/usr/bin/echo 192.0.2.203
+ExecStop=/usr/bin/echo $unknown
+`
+
+	output := env.Apply(input)
+	if output != expected {
+		t.Fatalf("Environment incorrectly applied.\nOutput:\n%s\nExpected:\n%s", output, expected)
+	}
+}

initialize/etcd.go

@@ -3,7 +3,6 @@ package initialize
 import (
 	"errors"
 	"fmt"
-	"os"
 	"path"
 	"strings"
 
@@ -32,20 +31,9 @@ func (ec EtcdEnvironment) String() (out string) {
 		}
 	}
 
-	public := os.Getenv("COREOS_PUBLIC_IPV4")
-	private := os.Getenv("COREOS_PRIVATE_IPV4")
-
 	out += "[Service]\n"
 
 	for key, val := range norm {
-		if public != "" {
-			val = strings.Replace(val, "$public_ipv4", public, -1)
-		}
-
-		if private != "" {
-			val = strings.Replace(val, "$private_ipv4", private, -1)
-		}
-
 		out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
 	}
 

initialize/etcd_test.go

@@ -58,25 +58,6 @@ Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
 	}
 }
 
-func TestEtcdEnvironmentReplacement(t *testing.T) {
-	os.Clearenv()
-	os.Setenv("COREOS_PUBLIC_IPV4", "203.0.113.29")
-	os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.13")
-
-	cfg := make(EtcdEnvironment, 0)
-	cfg["bind-addr"] = "$public_ipv4:4001"
-	cfg["peer-bind-addr"] = "$private_ipv4:7001"
-
-	env := cfg.String()
-	expect := `[Service]
-Environment="ETCD_BIND_ADDR=203.0.113.29:4001"
-Environment="ETCD_PEER_BIND_ADDR=192.0.2.13:7001"
-`
-	if env != expect {
-		t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
-	}
-}
-
 func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
 	ec := EtcdEnvironment{
 		"name": "node001",

initialize/github.go

@@ -1,52 +1,18 @@
 package initialize
 
 import (
-	"encoding/json"
 	"fmt"
-	"io/ioutil"
-	"net/http"
 
 	"github.com/coreos/coreos-cloudinit/system"
 )
 
-type GithubUserKey struct {
-	Id  int    `json:"id"`
-	Key string `json:"key"`
-}
-
-func fetchGithubKeys(github_url string) ([]string, error) {
-	res, err := http.Get(github_url)
-	defer res.Body.Close()
-	if err != nil {
-		return nil, err
-	}
-	body, err := ioutil.ReadAll(res.Body)
-	if err != nil {
-		return nil, err
-	}
-	var data []GithubUserKey
-	err = json.Unmarshal(body, &data)
-	if err != nil {
-		return nil, err
-	}
-	keys := make([]string, 0)
-	for _, key := range data {
-		keys = append(keys, key.Key)
-	}
-	return keys, err
-
-}
-
 func SSHImportGithubUser(system_user string, github_user string) error {
 	url := fmt.Sprintf("https://api.github.com/users/%s/keys", github_user)
-	keys, err := fetchGithubKeys(url)
+	keys, err := fetchUserKeys(url)
 	if err != nil {
 		return err
 	}
+
 	key_name := fmt.Sprintf("github-%s", github_user)
-	err = system.AuthorizeSSHKeys(system_user, key_name, keys)
+	return system.AuthorizeSSHKeys(system_user, key_name, keys)
-	if err != nil {
-		return err
-	}
-	return nil
 }

initialize/github_test.go

@@ -1,55 +1,16 @@
 package initialize
 
 import (
-	"fmt"
-	"net/http"
-	"net/http/httptest"
 	"testing"
 )
 
-func TestCloudConfigUsersGithubMarshal(t *testing.T) {
-	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		gh_res := `
-[
-  {
-    "id": 67057,
-    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
-  },
-  {
-    "id": 3340477,
-    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANxpzIbTzKTeBRaOIdUxwwGwvDasTfU/PonhbNIuhYjc+xFGvBRTumox2F+luVAKKs4WdvA4nJXaY1OFi6DZftk5Bp4E2JaSzp8ulAzHsMexDdv6LGHGEJj/qdHAL1vHk2K89PpwRFSRZI8XRBLjvkr4ZgBKLG5ZILXPJEPP2j3lAAAAFQCtxoTnV8wy0c4grcGrQ+1sCsD7WQAAAIAqZsW2GviMe1RQrbZT0xAZmI64XRPrnLsoLxycHWlS7r6uUln2c6Ae2MB/YF0d4Kd1XZii9GHj7rrypqEo7MW8uSabhu70nmu1J8m2O3Dsr+4oJLeat9vwPsJV92IKO0jQwjKnAOHOiB9JKGeCw+NfXfogbti9/q38Q6XcS+SI5wAAAIEA1803Y2h+tOOpZXAsNIwl9mRfExWzLQ3L7knwJdznQu/6SW1H/1oyoYLebuk187Qj2UFI5qQ6AZNc49DvohWx0Cg6ABcyubNyoaCjZKWIdxVnItHWNbLe//+tyTu0I2eQwJOORsEPK5gMpf599C7wXQ//DzZOWbTWiHEX52gCTmk="
-  },
-  {
-    "id": 5224438,
-    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
-  }
-]
-`
-		fmt.Fprintln(w, gh_res)
-	}))
-	defer ts.Close()
-
-	keys, err := fetchGithubKeys(ts.URL)
-	if err != nil {
-		t.Fatalf("Encountered unexpected error: %v", err)
-	}
-	expected := "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
-	if keys[0] != expected {
-		t.Fatalf("expected %s, got %s", expected, keys[0])
-	}
-	expected = "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
-	if keys[2] != expected {
-		t.Fatalf("expected %s, got %s", expected, keys[2])
-	}
-
-}
 func TestCloudConfigUsersGithubUser(t *testing.T) {
 
-	contents := []byte(`
+	contents := `
 users:
   - name: elroy
     coreos-ssh-import-github: bcwaldon
-`)
+`
 	cfg, err := NewCloudConfig(contents)
 	if err != nil {
 		t.Fatalf("Encountered unexpected error: %v", err)

initialize/manage_etc_hosts.go

@@ -0,0 +1,44 @@
+package initialize
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+const DefaultIpv4Address = "127.0.0.1"
+
+func generateEtcHosts(option string) (out string, err error) {
+	if option != "localhost" {
+		return "", errors.New("Invalid option to manage_etc_hosts")
+	}
+
+	// use the operating system hostname
+	hostname, err := os.Hostname()
+	if err != nil {
+		return "", err
+	}
+
+	return fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname), nil
+
+}
+
+// Write an /etc/hosts file
+func WriteEtcHosts(option string, root string) error {
+
+	etcHosts, err := generateEtcHosts(option)
+	if err != nil {
+		return err
+	}
+
+	file := system.File{
+		Path:               path.Join(root, "etc", "hosts"),
+		RawFilePermissions: "0644",
+		Content:            etcHosts,
+	}
+
+	return system.WriteFile(&file)
+}

initialize/manage_etc_hosts_test.go

@@ -0,0 +1,76 @@
+package initialize
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
+	"testing"
+)
+
+func TestCloudConfigManageEtcHosts(t *testing.T) {
+	contents := `
+manage_etc_hosts: localhost
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	manageEtcHosts := cfg.ManageEtcHosts
+
+	if manageEtcHosts != "localhost" {
+		t.Errorf("ManageEtcHosts value is %q, expected 'localhost'", manageEtcHosts)
+	}
+}
+
+func TestManageEtcHostsInvalidValue(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer rmdir(dir)
+
+	if err := WriteEtcHosts("invalid", dir); err == nil {
+		t.Fatalf("WriteEtcHosts succeeded with invalid value: %v", err)
+	}
+}
+
+func TestEtcHostsWrittenToDisk(t *testing.T) {
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer rmdir(dir)
+
+	if err := WriteEtcHosts("localhost", dir); err != nil {
+		t.Fatalf("WriteEtcHosts failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "hosts")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	hostname, err := os.Hostname()
+	if err != nil {
+		t.Fatalf("Unable to read OS hostname: %v", err)
+	}
+
+	expect := fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname)
+
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}

initialize/oem.go

@@ -0,0 +1,39 @@
+package initialize
+
+import (
+	"fmt"
+	"path"
+	"strings"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type OEMRelease struct {
+	ID           string `yaml:"id"`
+	Name         string `yaml:"name"`
+	VersionID    string `yaml:"version-id"`
+	HomeURL      string `yaml:"home-url"`
+	BugReportURL string `yaml:"bug-report-url"`
+}
+
+func (oem *OEMRelease) String() string {
+	fields := []string{
+		fmt.Sprintf("ID=%s", oem.ID),
+		fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
+		fmt.Sprintf("NAME=%q", oem.Name),
+		fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
+		fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
+	}
+
+	return strings.Join(fields, "\n") + "\n"
+}
+
+func WriteOEMRelease(oem *OEMRelease, root string) error {
+	file := system.File{
+		Path:               path.Join(root, "etc", "oem-release"),
+		RawFilePermissions: "0644",
+		Content:            oem.String(),
+	}
+
+	return system.WriteFile(&file)
+}

initialize/oem_test.go

@@ -0,0 +1,54 @@
+package initialize
+
+import (
+	"io/ioutil"
+	"os"
+	"path"
+	"syscall"
+	"testing"
+)
+
+func TestOEMReleaseWrittenToDisk(t *testing.T) {
+	oem := OEMRelease{
+		ID:           "rackspace",
+		Name:         "Rackspace Cloud Servers",
+		VersionID:    "168.0.0",
+		HomeURL:      "https://www.rackspace.com/cloud/servers/",
+		BugReportURL: "https://github.com/coreos/coreos-overlay",
+	}
+	dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
+	if err != nil {
+		t.Fatalf("Unable to create tempdir: %v", err)
+	}
+	defer syscall.Rmdir(dir)
+
+	if err := WriteOEMRelease(&oem, dir); err != nil {
+		t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
+	}
+
+	fullPath := path.Join(dir, "etc", "oem-release")
+
+	fi, err := os.Stat(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to stat file: %v", err)
+	}
+
+	if fi.Mode() != os.FileMode(0644) {
+		t.Errorf("File has incorrect mode: %v", fi.Mode())
+	}
+
+	contents, err := ioutil.ReadFile(fullPath)
+	if err != nil {
+		t.Fatalf("Unable to read expected file: %v", err)
+	}
+
+	expect := `ID=rackspace
+VERSION_ID=168.0.0
+NAME="Rackspace Cloud Servers"
+HOME_URL="https://www.rackspace.com/cloud/servers/"
+BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
+`
+	if string(contents) != expect {
+		t.Fatalf("File has incorrect contents")
+	}
+}

initialize/ssh_keys.go

@@ -0,0 +1,47 @@
+package initialize
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+
+	"github.com/coreos/coreos-cloudinit/system"
+)
+
+type UserKey struct {
+	ID  int    `json:"id,omitempty"`
+	Key string `json:"key"`
+}
+
+func SSHImportKeysFromURL(system_user string, url string) error {
+	keys, err := fetchUserKeys(url)
+	if err != nil {
+		return err
+	}
+
+	key_name := fmt.Sprintf("coreos-cloudinit-%s", system_user)
+	return system.AuthorizeSSHKeys(system_user, key_name, keys)
+}
+
+func fetchUserKeys(url string) ([]string, error) {
+	res, err := http.Get(url)
+	defer res.Body.Close()
+	if err != nil {
+		return nil, err
+	}
+	body, err := ioutil.ReadAll(res.Body)
+	if err != nil {
+		return nil, err
+	}
+	var data []UserKey
+	err = json.Unmarshal(body, &data)
+	if err != nil {
+		return nil, err
+	}
+	keys := make([]string, 0)
+	for _, key := range data {
+		keys = append(keys, key.Key)
+	}
+	return keys, err
+}

initialize/ssh_keys_test.go

@@ -0,0 +1,69 @@
+package initialize
+
+import (
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+func TestCloudConfigUsersUrlMarshal(t *testing.T) {
+	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		gh_res := `
+[
+  {
+    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
+  },
+  {
+    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANxpzIbTzKTeBRaOIdUxwwGwvDasTfU/PonhbNIuhYjc+xFGvBRTumox2F+luVAKKs4WdvA4nJXaY1OFi6DZftk5Bp4E2JaSzp8ulAzHsMexDdv6LGHGEJj/qdHAL1vHk2K89PpwRFSRZI8XRBLjvkr4ZgBKLG5ZILXPJEPP2j3lAAAAFQCtxoTnV8wy0c4grcGrQ+1sCsD7WQAAAIAqZsW2GviMe1RQrbZT0xAZmI64XRPrnLsoLxycHWlS7r6uUln2c6Ae2MB/YF0d4Kd1XZii9GHj7rrypqEo7MW8uSabhu70nmu1J8m2O3Dsr+4oJLeat9vwPsJV92IKO0jQwjKnAOHOiB9JKGeCw+NfXfogbti9/q38Q6XcS+SI5wAAAIEA1803Y2h+tOOpZXAsNIwl9mRfExWzLQ3L7knwJdznQu/6SW1H/1oyoYLebuk187Qj2UFI5qQ6AZNc49DvohWx0Cg6ABcyubNyoaCjZKWIdxVnItHWNbLe//+tyTu0I2eQwJOORsEPK5gMpf599C7wXQ//DzZOWbTWiHEX52gCTmk="
+  },
+  {
+    "id": 5224438,
+    "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
+  }
+]
+`
+		fmt.Fprintln(w, gh_res)
+	}))
+	defer ts.Close()
+
+	keys, err := fetchUserKeys(ts.URL)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+	expected := "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
+	if keys[0] != expected {
+		t.Fatalf("expected %s, got %s", expected, keys[0])
+	}
+	expected = "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
+	if keys[2] != expected {
+		t.Fatalf("expected %s, got %s", expected, keys[2])
+	}
+
+}
+func TestCloudConfigUsersSSHImportURL(t *testing.T) {
+
+	contents := `
+users:
+  - name: elroy
+    coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
+`
+	cfg, err := NewCloudConfig(contents)
+	if err != nil {
+		t.Fatalf("Encountered unexpected error: %v", err)
+	}
+
+	if len(cfg.Users) != 1 {
+		t.Fatalf("Parsed %d users, expected 1", cfg.Users)
+	}
+
+	user := cfg.Users[0]
+
+	if user.Name != "elroy" {
+		t.Errorf("User name is %q, expected 'elroy'", user.Name)
+	}
+
+	if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
+		t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
+	}
+}

system/systemd.go

@@ -21,6 +21,7 @@ type Unit struct {
 	Name    string
 	Runtime bool
 	Content string
+	Command string
 }
 
 func (u *Unit) Type() string {
@@ -80,34 +81,33 @@ func EnableUnitFile(file string, runtime bool) error {
 	return err
 }
 
-func separateNetworkUnits(units []Unit) ([]Unit, []Unit) {
+func RunUnitCommand(command, unit string) (string, error) {
-	networkUnits := make([]Unit, 0)
+	conn, err := dbus.New()
-	nonNetworkUnits := make([]Unit, 0)
+	if err != nil {
-	for _, unit := range units {
+		return "", err
-		if unit.Group() == "network" {
-			networkUnits = append(networkUnits, unit)
-		} else {
-			nonNetworkUnits = append(nonNetworkUnits, unit)
-		}
-	}
-	return networkUnits, nonNetworkUnits
-}
-
-func StartUnits(units []Unit) error {
-	networkUnits, nonNetworkUnits := separateNetworkUnits(units)
-	if len(networkUnits) > 0 {
-		if err := RestartUnitByName("systemd-networkd.service"); err != nil {
-			return err
-		}
 	}
 
-	for _, unit := range nonNetworkUnits {
+	var fn func(string, string) (string, error)
-		if err := RestartUnitByName(unit.Name); err != nil {
+	switch command {
-			return err
+	case "start":
-		}
+		fn = conn.StartUnit
+	case "stop":
+		fn = conn.StopUnit
+	case "restart":
+		fn = conn.RestartUnit
+	case "reload":
+		fn = conn.ReloadUnit
+	case "try-restart":
+		fn = conn.TryRestartUnit
+	case "reload-or-restart":
+		fn = conn.ReloadOrRestartUnit
+	case "reload-or-try-restart":
+		fn = conn.ReloadOrTryRestartUnit
+	default:
+		return "", fmt.Errorf("Unsupported systemd command %q", command)
 	}
 
-	return nil
+	return fn(unit, "replace")
 }
 
 func DaemonReload() error {
@@ -116,31 +116,7 @@ func DaemonReload() error {
 		return err
 	}
 
-	_, err = conn.Reload()
+	return conn.Reload()
-	return err
-}
-
-func RestartUnitByName(name string) error {
-	log.Printf("Restarting unit %s", name)
-	conn, err := dbus.New()
-	if err != nil {
-		return err
-	}
-
-	output, err := conn.RestartUnit(name, "replace")
-	log.Printf("Restart completed with '%s'", output)
-
-	return err
-}
-
-func StartUnitByName(name string) error {
-	conn, err := dbus.New()
-	if err != nil {
-		return err
-	}
-
-	_, err = conn.StartUnit(name, "replace")
-	return err
 }
 
 func ExecuteScript(scriptPath string) (string, error) {

system/user.go

@@ -13,6 +13,7 @@ type User struct {
 	PasswordHash        string   `yaml:"passwd"`
 	SSHAuthorizedKeys   []string `yaml:"ssh-authorized-keys"`
 	SSHImportGithubUser string   `yaml:"coreos-ssh-import-github"`
+	SSHImportURL        string   `yaml:"coreos-ssh-import-url"`
 	GECOS               string   `yaml:"gecos"`
 	Homedir             string   `yaml:"homedir"`
 	NoCreateHome        bool     `yaml:"no-create-home"`
@@ -33,6 +34,8 @@ func CreateUser(u *User) error {
 
 	if u.PasswordHash != "" {
 		args = append(args, "--password", u.PasswordHash)
+	} else {
+		args = append(args, "--password", "*")
 	}
 
 	if u.GECOS != "" {

third_party/github.com/coreos/go-systemd/dbus/dbus.go

@@ -18,6 +18,8 @@ limitations under the License.
 package dbus
 
 import (
+	"os"
+	"strconv"
 	"strings"
 	"sync"
 
@@ -73,7 +75,12 @@ func (c *Conn) initConnection() error {
 		return err
 	}
 
-	err = c.sysconn.Auth(nil)
+	// Only use EXTERNAL method, and hardcode the uid (not username)
+	// to avoid a username lookup (which requires a dynamically linked
+	// libc)
+	methods := []dbus.Auth{dbus.AuthExternal(strconv.Itoa(os.Getuid()))}
+
+	err = c.sysconn.Auth(methods)
 	if err != nil {
 		c.sysconn.Close()
 		return err

third_party/github.com/coreos/go-systemd/dbus/methods.go

@@ -35,6 +35,7 @@ func (c *Conn) jobComplete(signal *dbus.Signal) {
 	out, ok := c.jobListener.jobs[job]
 	if ok {
 		out <- result
+		delete(c.jobListener.jobs, job)
 	}
 	c.jobListener.Unlock()
 }
@@ -137,8 +138,8 @@ func (c *Conn) KillUnit(name string, signal int32) {
 	c.sysobj.Call("org.freedesktop.systemd1.Manager.KillUnit", 0, name, "all", signal).Store()
 }
 
-// GetUnitProperties takes the unit name and returns all of its dbus object properties.
+// getProperties takes the unit name and returns all of its dbus object properties, for the given dbus interface
-func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
+func (c *Conn) getProperties(unit string, dbusInterface string) (map[string]interface{}, error) {
 	var err error
 	var props map[string]dbus.Variant
 
@@ -148,7 +149,7 @@ func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
 	}
 
 	obj := c.sysconn.Object("org.freedesktop.systemd1", path)
-	err = obj.Call("org.freedesktop.DBus.Properties.GetAll", 0, "org.freedesktop.systemd1.Unit").Store(&props)
+	err = obj.Call("org.freedesktop.DBus.Properties.GetAll", 0, dbusInterface).Store(&props)
 	if err != nil {
 		return nil, err
 	}
@@ -161,6 +162,55 @@ func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
 	return out, nil
 }
 
+// GetUnitProperties takes the unit name and returns all of its dbus object properties.
+func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
+	return c.getProperties(unit, "org.freedesktop.systemd1.Unit")
+}
+
+func (c *Conn) getProperty(unit string, dbusInterface string, propertyName string) (*Property, error) {
+	var err error
+	var prop dbus.Variant
+
+	path := ObjectPath("/org/freedesktop/systemd1/unit/" + unit)
+	if !path.IsValid() {
+		return nil, errors.New("invalid unit name: " + unit)
+	}
+
+	obj := c.sysconn.Object("org.freedesktop.systemd1", path)
+	err = obj.Call("org.freedesktop.DBus.Properties.Get", 0, dbusInterface, propertyName).Store(&prop)
+	if err != nil {
+		return nil, err
+	}
+
+	return &Property{Name: propertyName, Value: prop}, nil
+}
+
+func (c *Conn) GetUnitProperty(unit string, propertyName string) (*Property, error) {
+	return c.getProperty(unit, "org.freedesktop.systemd1.Unit", propertyName)
+}
+
+// GetUnitTypeProperties returns the extra properties for a unit, specific to the unit type.
+// Valid values for unitType: Service, Socket, Target, Device, Mount, Automount, Snapshot, Timer, Swap, Path, Slice, Scope
+// return "dbus.Error: Unknown interface" if the unitType is not the correct type of the unit
+func (c *Conn) GetUnitTypeProperties(unit string, unitType string) (map[string]interface{}, error) {
+	return c.getProperties(unit, "org.freedesktop.systemd1."+unitType)
+}
+
+// SetUnitProperties() may be used to modify certain unit properties at runtime.
+// Not all properties may be changed at runtime, but many resource management
+// settings (primarily those in systemd.cgroup(5)) may. The changes are applied
+// instantly, and stored on disk for future boots, unless runtime is true, in which
+// case the settings only apply until the next reboot. name is the name of the unit
+// to modify. properties are the settings to set, encoded as an array of property
+// name and value pairs.
+func (c *Conn) SetUnitProperties(name string, runtime bool, properties ...Property) error {
+	return c.sysobj.Call("SetUnitProperties", 0, name, runtime, properties).Store()
+}
+
+func (c *Conn) GetUnitTypeProperty(unit string, unitType string, propertyName string) (*Property, error) {
+	return c.getProperty(unit, "org.freedesktop.systemd1." + unitType, propertyName)
+}
+
 // ListUnits returns an array with all currently loaded units. Note that
 // units may be known by multiple names at the same time, and hence there might
 // be more unit names loaded than actual units behind them.
@@ -253,8 +303,52 @@ type EnableUnitFileChange struct {
 	Destination string // Destination of the symlink
 }
 
+// DisableUnitFiles() may be used to disable one or more units in the system (by
+// removing symlinks to them from /etc or /run).
+//
+// It takes a list of unit files to disable (either just file names or full
+// absolute paths if the unit files are residing outside the usual unit
+// search paths), and one boolean: whether the unit was enabled for runtime
+// only (true, /run), or persistently (false, /etc).
+//
+// This call returns an array with the changes made. The changes list
+// consists of structures with three strings: the type of the change (one of
+// symlink or unlink), the file name of the symlink and the destination of the
+// symlink.
+func (c *Conn) DisableUnitFiles(files []string, runtime bool) ([]DisableUnitFileChange, error) {
+	result := make([][]interface{}, 0)
+	err := c.sysobj.Call("DisableUnitFiles", 0, files, runtime).Store(&result)
+	if err != nil {
+		return nil, err
+	}
+
+	resultInterface := make([]interface{}, len(result))
+	for i := range result {
+		resultInterface[i] = result[i]
+	}
+
+	changes := make([]DisableUnitFileChange, len(result))
+	changesInterface := make([]interface{}, len(changes))
+	for i := range changes {
+		changesInterface[i] = &changes[i]
+	}
+
+	err = dbus.Store(resultInterface, changesInterface...)
+	if err != nil {
+		return nil, err
+	}
+
+	return changes, nil
+}
+
+type DisableUnitFileChange struct {
+	Type        string // Type of the change (one of symlink or unlink)
+	Filename    string // File name of the symlink
+	Destination string // Destination of the symlink
+}
+
 // Reload instructs systemd to scan for and reload unit files. This is
 // equivalent to a 'systemctl daemon-reload'.
-func (c *Conn) Reload() (string, error) {
+func (c *Conn) Reload() error {
-	return c.runJob("org.freedesktop.systemd1.Manager.Reload")
+	return c.sysobj.Call("org.freedesktop.systemd1.Manager.Reload", 0).Store()
 }

third_party/github.com/coreos/go-systemd/dbus/methods_test.go

@@ -18,9 +18,11 @@ package dbus
 
 import (
 	"fmt"
+	"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
 	"math/rand"
 	"os"
 	"path/filepath"
+	"reflect"
 	"testing"
 )
 
@@ -50,13 +52,16 @@ func setupUnit(target string, conn *Conn, t *testing.T) {
 	fixture := []string{abs}
 
 	install, changes, err := conn.EnableUnitFiles(fixture, true, true)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	if install != false {
 		t.Fatal("Install was true")
 	}
 
 	if len(changes) < 1 {
-		t.Fatal("Expected one change, got %v", changes)
+		t.Fatalf("Expected one change, got %v", changes)
 	}
 
 	if changes[0].Filename != targetRun {
@@ -118,6 +123,37 @@ func TestStartStopUnit(t *testing.T) {
 	}
 }
 
+// Enables a unit and then immediately tears it down
+func TestEnableDisableUnit(t *testing.T) {
+	target := "enable-disable.service"
+	conn := setupConn(t)
+
+	setupUnit(target, conn, t)
+
+	abs, err := filepath.Abs("../fixtures/" + target)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	path := filepath.Join("/run/systemd/system/", target)
+
+	// 2. Disable the unit
+	changes, err := conn.DisableUnitFiles([]string{abs}, true)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if len(changes) != 1 {
+		t.Fatalf("Changes should include the path, %v", changes)
+	}
+	if changes[0].Filename != path {
+		t.Fatalf("Change should include correct filename, %+v", changes[0])
+	}
+	if changes[0].Destination != "" {
+		t.Fatalf("Change destination should be empty, %+v", changes[0])
+	}
+}
+
 // TestGetUnitProperties reads the `-.mount` which should exist on all systemd
 // systems and ensures that one of its properties is valid.
 func TestGetUnitProperties(t *testing.T) {
@@ -139,6 +175,20 @@ func TestGetUnitProperties(t *testing.T) {
 	if names[0] != "system.slice" {
 		t.Fatal("unexpected wants for /")
 	}
+
+	prop, err := conn.GetUnitProperty(unit, "Wants")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if prop.Name != "Wants" {
+		t.Fatal("unexpected property name")
+	}
+
+	val := prop.Value.Value().([]string)
+	if !reflect.DeepEqual(val, names) {
+		t.Fatal("unexpected property value")
+	}
 }
 
 // TestGetUnitPropertiesRejectsInvalidName attempts to get the properties for a
@@ -150,10 +200,37 @@ func TestGetUnitPropertiesRejectsInvalidName(t *testing.T) {
 	unit := "//invalid#$^/"
 
 	_, err := conn.GetUnitProperties(unit)
-
 	if err == nil {
 		t.Fatal("Expected an error, got nil")
 	}
+
+	_, err = conn.GetUnitProperty(unit, "Wants")
+	if err == nil {
+		t.Fatal("Expected an error, got nil")
+	}
+}
+
+// TestSetUnitProperties changes a cgroup setting on the `tmp.mount`
+// which should exist on all systemd systems and ensures that the
+// property was set.
+func TestSetUnitProperties(t *testing.T) {
+	conn := setupConn(t)
+
+	unit := "tmp.mount"
+
+	if err := conn.SetUnitProperties(unit, true, Property{"CPUShares", dbus.MakeVariant(uint64(1023))}); err != nil {
+		t.Fatal(err)
+	}
+
+	info, err := conn.GetUnitTypeProperties(unit, "Mount")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	value := info["CPUShares"].(uint64)
+	if value != 1023 {
+		t.Fatal("CPUShares of unit is not 1023, %s", value)
+	}
 }
 
 // Ensure that basic transient unit starting and stopping works.
@@ -211,3 +288,27 @@ func TestStartStopTransientUnit(t *testing.T) {
 		t.Fatalf("Test unit found in list, should be stopped")
 	}
 }
+
+func TestConnJobListener(t *testing.T) {
+	target := "start-stop.service"
+	conn := setupConn(t)
+
+	setupUnit(target, conn, t)
+
+	jobSize := len(conn.jobListener.jobs)
+
+	_, err := conn.StartUnit(target, "replace")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = conn.StopUnit(target, "replace")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	currentJobSize := len(conn.jobListener.jobs)
+	if jobSize != currentJobSize {
+		t.Fatal("JobListener jobs leaked")
+	}
+}

third_party/github.com/coreos/go-systemd/dbus/properties.go

@@ -209,3 +209,12 @@ func PropPropagatesReloadTo(units ...string) Property {
 func PropRequiresMountsFor(units ...string) Property {
 	return propDependency("RequiresMountsFor", units)
 }
+
+// PropSlice sets the Slice unit property.  See
+// http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#Slice=
+func PropSlice(slice string) Property {
+	return Property{
+		Name:  "Slice",
+		Value: dbus.MakeVariant(slice),
+	}
+}

units/90-configdrive.rules

@@ -0,0 +1,11 @@
+# Automatically trigger configdrive mounting.
+
+ACTION!="add|change", GOTO="coreos_configdrive_end"
+
+# A normal config drive. Block device formatted with iso9660 or fat
+SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="iso9660|vfat", ENV{ID_FS_LABEL}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-block.service"
+
+# Addtionally support virtfs from QEMU
+SUBSYSTEM=="virtio", DRIVER=="9pnet_virtio", ATTR{mount_tag}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-virtfs.service"
+
+LABEL="coreos_configdrive_end"

units/configdrive-block.service

@@ -0,0 +1,15 @@
+[Unit]
+Description=Mount config drive
+Conflicts=configdrive-virtfs.service umount.target
+ConditionPathIsMountPoint=!/media/configdrive
+# Only mount config drive block devices automatically in virtual machines
+ConditionVirtualization=vm
+
+# OpenStack defined config drive so they get to stick their name in it
+Wants=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+Before=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=no
+ExecStart=/bin/mount -t auto -o ro,x-mount.mkdir LABEL=config-2 /media/configdrive

units/configdrive-virtfs.service

@@ -0,0 +1,18 @@
+[Unit]
+Description=Mount config drive from virtfs
+Conflicts=configdrive-block.service umount.target
+ConditionPathIsMountPoint=!/media/configdrive
+ConditionVirtualization=vm
+
+# OpenStack defined config drive so they get to stick their name in it
+Wants=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+Before=user-cloudinit@media-configdrive-openstack-latest-user_data.service
+
+# Support old style setup for now
+Wants=addon-run@media-configdrive.service addon-config@media-configdrive.service
+Before=addon-run@media-configdrive.service addon-config@media-configdrive.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=no
+ExecStart=/bin/mount -t 9p -o trans=virtio,version=9p2000.L,x-mount.mkdir config-2 /media/configdrive

units/system-cloudinit@.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=Load cloud-config from %f
+Requires=dbus.service
+After=dbus.service
+Before=system-config.target
+ConditionFileNotEmpty=%f
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

units/system-config.target

@@ -0,0 +1,10 @@
+[Unit]
+Description=Load system-provided cloud configs
+
+# Generate /etc/environment
+Requires=coreos-setup-environment.service
+After=coreos-setup-environment.service
+
+# Load OEM cloud-config.yml
+Requires=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service
+After=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service

units/user-cloudinit@.service

@@ -0,0 +1,12 @@
+[Unit]
+Description=Load cloud-config from %f
+Requires=coreos-setup-environment.service
+After=coreos-setup-environment.service
+Before=user-config.target
+ConditionFileNotEmpty=%f
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+EnvironmentFile=-/etc/environment
+ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

units/user-config.target

@@ -0,0 +1,8 @@
+[Unit]
+Description=Load user-provided cloud configs
+Requires=system-config.target
+After=system-config.target
+
+# Load user_data placed by coreos-install
+Requires=user-cloudinit@var-lib-coreos\x2dinstall-user_data.service
+After=user-cloudinit@var-lib-coreos\x2dinstall-user_data.service