Compare commits

..

54 Commits

Author SHA1 Message Date
Alex Crawford
0785840fe3 coreos-cloudinit: bump to 0.10.4 2014-09-24 19:25:34 -07:00
Alex Crawford
c10bfc2f56 Merge pull request #240 from epankala/euca4_compat_fix
AWS: Eucalyptus 4.x compatibility fix
2014-09-24 10:55:39 -07:00
Janne Paenkaelae
2f954dcdc2 AWS: Eucalyptus 4.x compatibility fix
For Eucalyptus 4.0.1 requesting metadata seem to work differently as with EC2.

In Euca:
> curl http://169.254.169.254/2009-04-04
<?xml version="1.0"?><Response><Errors><Error><Code>404 Not Found</Code><Message>unknown</Message></Error></Errors><RequestID>unknown</RequestID></Response>core@localhost ~ $

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

In AWS EC2
> curl http://169.254.169.254/2009-04-04
"" (zero bytes)

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

As the isAvailable() function in metadata.go tests only for errorcode
it fails in Euca.
2014-09-24 20:33:29 +03:00
Alex Crawford
cdfc94f4e9 Merge pull request #234 from crawford/validate
config: explicitly specify fields and seperate config and application
2014-09-24 07:42:09 -07:00
Alex Crawford
18e2f98414 cloudconfig: refactor config
- Move CloudConfig into config package
- Add YAML tags to CloudConfig
2014-09-23 17:59:32 -07:00
Alex Crawford
4b472795c4 user: move User into config package
- Add YAML tags for the fields
2014-09-23 17:59:19 -07:00
Alex Crawford
85b8d804c8 file: refactor config
- Seperate the config from Permissions()
- Add YAML tags for the fields
2014-09-23 17:59:16 -07:00
Alex Crawford
1fbbaaec19 unit: refactor config
- Seperate the config from Destination()
- Add YAML tags for the fields
2014-09-23 17:58:32 -07:00
Alex Crawford
667dbd8fb7 update: refactor config
- Explicitly specify all of the valid options for Update
- Seperate the config from File() and Units()
- Add YAML tags for the fields
2014-09-23 17:57:43 -07:00
Alex Crawford
6730cb7227 oem: refactor the config
- Seperate the config from File()
- Add YAML tags for the fields
2014-09-23 16:08:23 -07:00
Alex Crawford
9454522033 fleet: refactor config
- Explicitly specify all of the valid options for fleet
- Seperate the config from Units()
- Add YAML tags for the fields
2014-09-23 16:07:53 -07:00
Alex Crawford
c255739a93 etcd: refactor config
- Explicitly specify all of the valid options for etcd
- Remove the default name generation (ETCD_NAME is set by its unit file now)
- Seperate the etcd config from Units()
- Remove support for DISCOVERY_URL
- Add YAML tags for the fields
2014-09-23 16:07:13 -07:00
Alex Crawford
2051cd3e1c Merge pull request #238 from crawford/docs
docs: fix documentation of coreos.units.command
2014-09-23 11:33:44 -07:00
Alex Crawford
b52cb3fea3 docs: fix documentation of coreos.units.command 2014-09-23 11:32:15 -07:00
Alex Crawford
da5f85b3fb coreos-cloudinit: bump to 0.10.3+git 2014-09-17 12:19:27 -07:00
Alex Crawford
9999178538 coreos-cloudinit: bump to 0.10.3 2014-09-17 12:19:13 -07:00
Alex Crawford
8f766e4666 Merge pull request #235 from crawford/routes
network: add support for CIDR addresses Debian routes
2014-09-17 12:18:16 -07:00
Alex Crawford
2d28d16c92 network: add support for CIDR addresses Debian routes
OnMetal is changing their template from:
`route add -net 1.2.3.0 netmask 255.255.255.0 gw 10.1.2.1 || true`
to:
`route add -net 1.2.3.0/24 gw 10.1.2.1 || true`
2014-09-16 17:36:34 -07:00
Alex Crawford
e9cd09dd7b coreos-cloudinit: bump to 0.10.2+git 2014-09-14 08:19:57 -07:00
Alex Crawford
8370b30aa2 coreos-cloudinit: bump to 0.10.2 2014-09-14 08:19:33 -07:00
Alex Crawford
3e015cc3a1 Merge pull request #233 from crawford/configdrive
configdrive: don't fail if no network config was provided
2014-09-14 08:18:14 -07:00
Alex Crawford
a0fe6d0884 configdrive: return an empty network config when filename is empty
Additionally, don't bother checking for a network config if it isn't going to
be processed.
2014-09-13 21:51:51 -07:00
Alex Crawford
585ce5fcd9 Revert "metadata: don't fail if no network config was provided"
This reverts commit c1f373e648.
2014-09-13 21:01:42 -07:00
Alex Crawford
72445796ca coreos-cloudinit: bump to 0.10.1+git 2014-09-12 16:48:15 -07:00
Alex Crawford
7342d91a85 coreos-cloudinit: bump to 0.10.1 2014-09-12 16:47:58 -07:00
Alex Crawford
db1bc51c98 Merge pull request #231 from crawford/netconf
metadata: don't fail if no network config was provided
2014-09-12 16:35:24 -07:00
Alex Crawford
c1f373e648 metadata: don't fail if no network config was provided 2014-09-12 16:29:27 -07:00
Alex Crawford
db49a16002 coreos-cloudinit: bump to 0.10.0+git 2014-09-11 17:37:05 -07:00
Alex Crawford
a4a6c281d9 coreos-cloudinit: bump to 0.10.0 2014-09-11 17:36:38 -07:00
Alex Crawford
17f8733121 Merge pull request #228 from crawford/sub
env: add support for escaping environment substitutions
2014-09-11 15:34:03 -07:00
Alex Crawford
7dec922618 env: add support for escaping environment substitutions 2014-09-11 15:30:33 -07:00
Alex Crawford
54d3ae27af Merge pull request #226 from crawford/oem
flags: add oem flag
2014-09-11 13:25:21 -07:00
Alex Crawford
ee2416af64 flags: move the flags into their own namespace 2014-09-11 12:00:17 -07:00
Alex Crawford
cda037f9a5 flags: add oem flag
The oem flag will allow each of the OEMs to specify one flag only, acting as a
shortcut to their specific configuration. This will allow us to update which
options each OEM uses when running cloudinit.
2014-09-11 12:00:17 -07:00
Alex Crawford
549806cf64 Merge pull request #227 from crawford/ipv6
metadata: add support for IPv6 variable substitution
2014-09-11 10:45:33 -07:00
Alex Crawford
56815a6756 metadata: add support for IPv6 variable substitution 2014-09-11 10:43:02 -07:00
Alex Crawford
24a6f7c49c Merge pull request #225 from crawford/exit
userdata: change handling of bad userdata
2014-09-10 19:16:12 -07:00
Alex Crawford
98484be434 userdata: change handling of bad userdata
Don't fail after encountering bad userdata. Continue processing the metadata
and then exit. This will allow people with bad userdata to actually log in and
see the error.
2014-09-10 17:50:23 -07:00
Jonathan Boulle
9024659296 Merge pull request #217 from ecnahc515/patch-1
Fix broken link to fleet config
2014-09-09 15:19:18 -07:00
Chance Zibolski
fc6940f7ba Documentation: More specific link to fleet config.
Add an anchor tag to the url to take person directly to config section.
2014-09-09 15:15:55 -07:00
Brian Waldon
f2fd95699b Merge pull request #224 from bcwaldon/typo
docs: fix a typo
2014-09-09 12:36:42 -07:00
bdevloed
65db96cc7c docs: fix a typo 2014-09-09 12:31:54 -07:00
Alex Crawford
c17b93b5c0 Merge pull request #223 from crawford/yaml
third_party: sync third_party/gopkg.in/yaml.v1
2014-09-08 19:28:59 -07:00
Alex Crawford
d352f8ce6a Merge pull request #222 from crawford/contribute
docs: Update maintainers and contribution guide
2014-09-08 15:54:30 -07:00
Alex Crawford
78aa2c56ec yaml: replace goyaml with yaml 2014-09-08 13:25:27 -07:00
Alex Crawford
c5b3788282 third_party: sync third_party/gopkg.in/yaml.v1
Update launchpad.net/goyaml to gopkg.in/yaml.v1
2014-09-08 13:23:50 -07:00
Alex Crawford
5e98970bb5 docs: Update maintainers and contribution guide 2014-09-08 12:55:17 -07:00
Alex Crawford
cbdd446c55 Merge pull request #220 from crawford/docs
docs: Update list of platforms supporting variable substitutions
2014-09-05 11:51:45 -07:00
Alex Crawford
316cadcf44 docs: Update list of platforms supporting variable substitutions 2014-09-04 12:57:19 -07:00
Alex Crawford
5a939be21b coreos-cloudinit: bump to 0.9.6+git 2014-09-02 17:49:09 -07:00
Alex Crawford
8d76c64386 coreos-cloudinit: bump to 0.9.6 2014-09-02 17:48:45 -07:00
Alex Crawford
1b854eb51e Merge pull request #218 from crawford/units
units: Ensure that the units are executed in order
2014-09-02 17:40:37 -07:00
Alex Crawford
9fcf338bf3 units: Ensure that the units are executed in order 2014-09-02 17:15:32 -07:00
Alex Crawford
fda72bdb5c coreos-cloudinit: bump to 0.9.5+git 2014-09-02 10:10:59 -07:00
82 changed files with 2289 additions and 1970 deletions

View File

@@ -39,22 +39,25 @@ Thanks for your contributions!
### Format of the Commit Message
We follow a rough convention for commit messages borrowed from AngularJS. This
is an example of a commit:
We follow a rough convention for commit messages that is designed to answer two
questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
```
feat(scripts/test-cluster): add a cluster test command
environment: write new keys in consistent order
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
Go 1.3 randomizes the ordering of keys when iterating over a map.
Sort the keys to make this ordering consistent.
Fixes #38
```
The format can be described more formally as follows:
```
<type>(<scope>): <subject>
<subsystem>: <what changed>
<BLANK LINE>
<body>
<why this change was made>
<BLANK LINE>
<footer>
```
@@ -63,25 +66,3 @@ The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
#### Subject Line
The subject line contains a succinct description of the change.
#### Allowed `<type>`s
- *feat* (feature)
- *fix* (bug fix)
- *docs* (documentation)
- *style* (formatting, missing semi colons, …)
- *refactor*
- *test* (when adding missing tests)
- *chore* (maintain)
#### Allowed `<scope>`s
Scopes can anything specifying the place of the commit change in the code base -
for example, "api", "store", etc.
For more details on the commit format, see the [AngularJS commit style
guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).

View File

@@ -4,7 +4,7 @@ CoreOS allows you to declaratively customize various OS-level items, such as net
## Configuration File
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file, which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
@@ -68,7 +68,7 @@ Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, and Vagrant._
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
@@ -95,7 +95,7 @@ Environment="FLEET_METADATA=region=us-west"
For more information on fleet configuration, see the [fleet documentation][fleet-config].
[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md
[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md#configuration
#### update
@@ -130,11 +130,11 @@ The `coreos.units.*` parameters define a list of arbitrary systemd units to star
Each item is an object with the following fields:
- **name**: String representing unit's name. Required.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. Default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. The default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. The default value is false.
- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. Default value is false.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. The default behavior is to not execute any commands.
- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. The default value is false.
**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.

3
MAINTAINERS Normal file
View File

@@ -0,0 +1,3 @@
Alex Crawford <alex.crawford@coreos.com> (@crawford)
Jonathan Boulle <jonathan.boulle@coreos.com> (@jonboulle)
Brian Waldon <brian.waldon@coreos.com> (@bcwaldon)

198
config/config.go Normal file
View File

@@ -0,0 +1,198 @@
package config
import (
"fmt"
"log"
"reflect"
"strings"
"github.com/coreos/coreos-cloudinit/third_party/gopkg.in/yaml.v1"
)
// CloudConfig encapsulates the entire cloud-config configuration file and maps
// directly to YAML. Fields that cannot be set in the cloud-config (fields
// used for internal use) have the YAML tag '-' so that they aren't marshalled.
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd Etcd `yaml:"etcd"`
Fleet Fleet `yaml:"fleet"`
OEM OEM `yaml:"oem"`
Update Update `yaml:"update"`
Units []Unit `yaml:"units"`
} `yaml:"coreos"`
WriteFiles []File `yaml:"write_files"`
Hostname string `yaml:"hostname"`
Users []User `yaml:"users"`
ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
NetworkConfigPath string `yaml:"-"`
NetworkConfig string `yaml:"-"`
}
// NewCloudConfig instantiates a new CloudConfig from the given contents (a
// string of YAML), returning any error encountered. It will ignore unknown
// fields but log encountering them.
func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig
err := yaml.Unmarshal([]byte(contents), &cfg)
if err != nil {
return &cfg, err
}
warnOnUnrecognizedKeys(contents, log.Printf)
return &cfg, nil
}
func (cc CloudConfig) String() string {
bytes, err := yaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
// IsZero returns whether or not the parameter is the zero value for its type.
// If the parameter is a struct, only the exported fields are considered.
func IsZero(c interface{}) bool {
return isZero(reflect.ValueOf(c))
}
// AssertValid checks the fields in the structure and makes sure that they
// contain valid values as specified by the 'valid' flag. Empty fields are
// implicitly valid.
func AssertValid(c interface{}) error {
ct := reflect.TypeOf(c)
cv := reflect.ValueOf(c)
for i := 0; i < ct.NumField(); i++ {
ft := ct.Field(i)
if !isFieldExported(ft) {
continue
}
valid := ft.Tag.Get("valid")
val := cv.Field(i)
if !isValid(val, valid) {
return fmt.Errorf("invalid value \"%v\" for option %q (valid options: %q)", val.Interface(), ft.Name, valid)
}
}
return nil
}
func isZero(v reflect.Value) bool {
switch v.Kind() {
case reflect.Struct:
vt := v.Type()
for i := 0; i < v.NumField(); i++ {
if isFieldExported(vt.Field(i)) && !isZero(v.Field(i)) {
return false
}
}
return true
default:
return v.Interface() == reflect.Zero(v.Type()).Interface()
}
}
func isFieldExported(f reflect.StructField) bool {
return f.PkgPath == ""
}
func isValid(v reflect.Value, valid string) bool {
if valid == "" || isZero(v) {
return true
}
vs := fmt.Sprintf("%v", v.Interface())
for _, valid := range strings.Split(valid, ",") {
if vs == valid {
return true
}
}
return false
}
type warner func(format string, v ...interface{})
// warnOnUnrecognizedKeys parses the contents of a cloud-config file and calls
// warn(msg, key) for every unrecognized key (i.e. those not present in CloudConfig)
func warnOnUnrecognizedKeys(contents string, warn warner) {
// Generate a map of all understood cloud config options
var cc map[string]interface{}
b, _ := yaml.Marshal(&CloudConfig{})
yaml.Unmarshal(b, &cc)
// Now unmarshal the entire provided contents
var c map[string]interface{}
yaml.Unmarshal([]byte(contents), &c)
// Check that every key in the contents exists in the cloud config
for k, _ := range c {
if _, ok := cc[k]; !ok {
warn("Warning: unrecognized key %q in provided cloud config - ignoring section", k)
}
}
// Check for unrecognized coreos options, if any are set
if coreos, ok := c["coreos"]; ok {
if set, ok := coreos.(map[interface{}]interface{}); ok {
known := cc["coreos"].(map[interface{}]interface{})
for k, _ := range set {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", k)
}
}
}
}
// Check for any badly-specified users, if any are set
if users, ok := c["users"]; ok {
var known map[string]interface{}
b, _ := yaml.Marshal(&User{})
yaml.Unmarshal(b, &known)
if set, ok := users.([]interface{}); ok {
for _, u := range set {
if user, ok := u.(map[interface{}]interface{}); ok {
for k, _ := range user {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", k)
}
}
}
}
}
}
// Check for any badly-specified files, if any are set
if files, ok := c["write_files"]; ok {
var known map[string]interface{}
b, _ := yaml.Marshal(&File{})
yaml.Unmarshal(b, &known)
if set, ok := files.([]interface{}); ok {
for _, f := range set {
if file, ok := f.(map[interface{}]interface{}); ok {
for k, _ := range file {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", k)
}
}
}
}
}
}
}

473
config/config_test.go Normal file
View File

@@ -0,0 +1,473 @@
package config
import (
"errors"
"fmt"
"reflect"
"strings"
"testing"
)
func TestIsZero(t *testing.T) {
for _, tt := range []struct {
c interface{}
empty bool
}{
{struct{}{}, true},
{struct{ a, b string }{}, true},
{struct{ A, b string }{}, true},
{struct{ A, B string }{}, true},
{struct{ A string }{A: "hello"}, false},
{struct{ A int }{}, true},
{struct{ A int }{A: 1}, false},
} {
if empty := IsZero(tt.c); tt.empty != empty {
t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.empty, empty)
}
}
}
func TestAssertValid(t *testing.T) {
for _, tt := range []struct {
c interface{}
err error
}{
{struct{}{}, nil},
{struct {
A, b string `valid:"1,2"`
}{}, nil},
{struct {
A, b string `valid:"1,2"`
}{A: "1", b: "2"}, nil},
{struct {
A, b string `valid:"1,2"`
}{A: "1", b: "hello"}, nil},
{struct {
A, b string `valid:"1,2"`
}{A: "hello", b: "2"}, errors.New("invalid value \"hello\" for option \"A\" (valid options: \"1,2\")")},
{struct {
A, b int `valid:"1,2"`
}{}, nil},
{struct {
A, b int `valid:"1,2"`
}{A: 1, b: 2}, nil},
{struct {
A, b int `valid:"1,2"`
}{A: 1, b: 9}, nil},
{struct {
A, b int `valid:"1,2"`
}{A: 9, b: 2}, errors.New("invalid value \"9\" for option \"A\" (valid options: \"1,2\")")},
} {
if err := AssertValid(tt.c); !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.err, err)
}
}
}
func TestCloudConfigInvalidKeys(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("panic while instantiating CloudConfig with nil keys: %v", r)
}
}()
for _, tt := range []struct {
contents string
}{
{"coreos:"},
{"ssh_authorized_keys:"},
{"ssh_authorized_keys:\n -"},
{"ssh_authorized_keys:\n - 0:"},
{"write_files:"},
{"write_files:\n -"},
{"write_files:\n - 0:"},
{"users:"},
{"users:\n -"},
{"users:\n - 0:"},
} {
_, err := NewCloudConfig(tt.contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with invalid keys: %v", err)
}
}
}
func TestCloudConfigUnknownKeys(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
coreos_unknown:
foo: "bar"
section_unknown:
dunno:
something
bare_unknown:
bar
write_files:
- content: fun
path: /var/party
file_unknown: nofun
users:
- name: fry
passwd: somehash
user_unknown: philip
hostname:
foo
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
}
if cfg.Hostname != "foo" {
t.Fatalf("hostname not correctly set when invalid keys are present")
}
if cfg.Coreos.Etcd.Discovery != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
t.Fatalf("etcd section not correctly set when invalid keys are present")
}
if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
t.Fatalf("write_files section not correctly set when invalid keys are present")
}
if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
t.Fatalf("users section not correctly set when invalid keys are present")
}
var warnings string
catchWarn := func(f string, v ...interface{}) {
warnings += fmt.Sprintf(f, v...)
}
warnOnUnrecognizedKeys(contents, catchWarn)
if !strings.Contains(warnings, "coreos_unknown") {
t.Errorf("warnings did not catch unrecognized coreos option coreos_unknown")
}
if !strings.Contains(warnings, "bare_unknown") {
t.Errorf("warnings did not catch unrecognized key bare_unknown")
}
if !strings.Contains(warnings, "section_unknown") {
t.Errorf("warnings did not catch unrecognized key section_unknown")
}
if !strings.Contains(warnings, "user_unknown") {
t.Errorf("warnings did not catch unrecognized user key user_unknown")
}
if !strings.Contains(warnings, "file_unknown") {
t.Errorf("warnings did not catch unrecognized file key file_unknown")
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig("")
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
if cfg.Hostname != "" {
t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
update:
reboot-strategy: reboot
units:
- name: 50-eth0.network
runtime: yes
content: '[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
write_files:
- content: |
penny
elroy
path: /etc/dogepack.conf
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 2 {
t.Error("Parsed incorrect number of SSH keys")
} else if keys[0] != "foobar" {
t.Error("Expected first SSH key to be 'foobar'")
} else if keys[1] != "foobaz" {
t.Error("Expected first SSH key to be 'foobaz'")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
wf := cfg.WriteFiles[0]
if wf.Content != "penny\nelroy\n" {
t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
}
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if wf.RawFilePermissions != "0644" {
t.Errorf("WriteFile has incorrect permissions %s", wf.RawFilePermissions)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
}
if wf.Owner != "root:dogepack" {
t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
}
}
if len(cfg.Coreos.Units) != 1 {
t.Error("Failed to parse correct number of units")
} else {
u := cfg.Coreos.Units[0]
expect := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if u.Content != expect {
t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
}
if u.Runtime != true {
t.Errorf("Unit has incorrect runtime value")
}
if u.Name != "50-eth0.network" {
t.Errorf("Unit has incorrect name %s", u.Name)
}
if u.Type() != "network" {
t.Errorf("Unit has incorrect type '%s'", u.Type())
}
}
if cfg.Coreos.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
if cfg.Coreos.Update.RebootStrategy != "reboot" {
t.Errorf("Failed to parse locksmith strategy")
}
}
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := `
ssh_authorized_keys:
- foo: bar
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig("")
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
t.Fatalf("Serialized config did not have expected header")
}
}
// TestDropInIgnored asserts that users are unable to set DropIn=True on units
func TestDropInIgnored(t *testing.T) {
contents := `
coreos:
units:
- name: test
dropin: true
`
cfg, err := NewCloudConfig(contents)
if err != nil || len(cfg.Coreos.Units) != 1 {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Coreos.Units) != 1 || cfg.Coreos.Units[0].Name != "test" {
t.Fatalf("Expected 1 unit, but got %d: %v", len(cfg.Coreos.Units), cfg.Coreos.Units)
}
if cfg.Coreos.Units[0].DropIn {
t.Errorf("dropin option on unit in cloud-config was not ignored!")
}
}
func TestCloudConfigUsers(t *testing.T) {
contents := `
users:
- name: elroy
passwd: somehash
ssh-authorized-keys:
- somekey
gecos: arbitrary comment
homedir: /home/place
no-create-home: yes
primary-group: things
groups:
- ping
- pong
no-user-group: true
system: y
no-log-init: True
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.PasswordHash != "somehash" {
t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
}
if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
} else {
key := user.SSHAuthorizedKeys[0]
if key != "somekey" {
t.Errorf("User SSH key is %q, expected 'somekey'", key)
}
}
if user.GECOS != "arbitrary comment" {
t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
}
if user.Homedir != "/home/place" {
t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
}
if !user.NoCreateHome {
t.Errorf("Failed to parse no-create-home field")
}
if user.PrimaryGroup != "things" {
t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
}
if len(user.Groups) != 2 {
t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
} else {
if user.Groups[0] != "ping" {
t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
}
if user.Groups[1] != "pong" {
t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
}
}
if !user.NoUserGroup {
t.Errorf("Failed to parse no-user-group field")
}
if !user.System {
t.Errorf("Failed to parse system field")
}
if !user.NoLogInit {
t.Errorf("Failed to parse no-log-init field")
}
}
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}
func TestCloudConfigUsersSSHImportURL(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
}
}

3
config/etc_hosts.go Normal file
View File

@@ -0,0 +1,3 @@
package config
type EtcHosts string

32
config/etcd.go Normal file
View File

@@ -0,0 +1,32 @@
package config
type Etcd struct {
Addr string `yaml:"addr" env:"ETCD_ADDR"`
BindAddr string `yaml:"bind-addr" env:"ETCD_BIND_ADDR"`
CAFile string `yaml:"ca-file" env:"ETCD_CA_FILE"`
CertFile string `yaml:"cert-file" env:"ETCD_CERT_FILE"`
ClusterActiveSize string `yaml:"cluster-active-size" env:"ETCD_CLUSTER_ACTIVE_SIZE"`
ClusterRemoveDelay string `yaml:"cluster-remove-delay" env:"ETCD_CLUSTER_REMOVE_DELAY"`
ClusterSyncInterval string `yaml:"cluster-sync-interval" env:"ETCD_CLUSTER_SYNC_INTERVAL"`
Cors string `yaml:"cors" env:"ETCD_CORS"`
CPUProfileFile string `yaml:"cpu-profile-file" env:"ETCD_CPU_PROFILE_FILE"`
DataDir string `yaml:"data-dir" env:"ETCD_DATA_DIR"`
Discovery string `yaml:"discovery" env:"ETCD_DISCOVERY"`
HTTPReadTimeout string `yaml:"http-read-timeout" env:"ETCD_HTTP_READ_TIMEOUT"`
HTTPWriteTimeout string `yaml:"http-write-timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"`
KeyFile string `yaml:"key-file" env:"ETCD_KEY_FILE"`
MaxClusterSize string `yaml:"max-cluster-size" env:"ETCD_MAX_CLUSTER_SIZE"`
MaxResultBuffer string `yaml:"max-result-buffer" env:"ETCD_MAX_RESULT_BUFFER"`
MaxRetryAttempts string `yaml:"max-retry-attempts" env:"ETCD_MAX_RETRY_ATTEMPTS"`
Name string `yaml:"name" env:"ETCD_NAME"`
PeerAddr string `yaml:"peer-addr" env:"ETCD_PEER_ADDR"`
PeerBindAddr string `yaml:"peer-bind-addr" env:"ETCD_PEER_BIND_ADDR"`
PeerCAFile string `yaml:"peer-ca-file" env:"ETCD_PEER_CA_FILE"`
PeerCertFile string `yaml:"peer-cert-file" env:"ETCD_PEER_CERT_FILE"`
PeerKeyFile string `yaml:"peer-key-file" env:"ETCD_PEER_KEY_FILE"`
Peers string `yaml:"peers" env:"ETCD_PEERS"`
PeersFile string `yaml:"peers-file" env:"ETCD_PEERS_FILE"`
Snapshot string `yaml:"snapshot" env:"ETCD_SNAPSHOT"`
Verbose string `yaml:"verbose" env:"ETCD_VERBOSE"`
VeryVerbose string `yaml:"very-verbose" env:"ETCD_VERY_VERBOSE"`
}

9
config/file.go Normal file
View File

@@ -0,0 +1,9 @@
package config
type File struct {
Encoding string `yaml:"-"`
Content string `yaml:"content"`
Owner string `yaml:"owner"`
Path string `yaml:"path"`
RawFilePermissions string `yaml:"permissions"`
}

14
config/fleet.go Normal file
View File

@@ -0,0 +1,14 @@
package config
type Fleet struct {
AgentTTL string `yaml:"agent-ttl" env:"FLEET_AGENT_TTL"`
EngineReconcileInterval string `yaml:"engine-reconcile-interval" env:"FLEET_ENGINE_RECONCILE_INTERVAL"`
EtcdCAFile string `yaml:"etcd-cafile" env:"FLEET_ETCD_CAFILE"`
EtcdCertFile string `yaml:"etcd-certfile" env:"FLEET_ETCD_CERTFILE"`
EtcdKeyFile string `yaml:"etcd-keyfile" env:"FLEET_ETCD_KEYFILE"`
EtcdRequestTimeout string `yaml:"etcd-request-timeout" env:"FLEET_ETCD_REQUEST_TIMEOUT"`
EtcdServers string `yaml:"etcd-servers" env:"FLEET_ETCD_SERVERS"`
Metadata string `yaml:"metadata" env:"FLEET_METADATA"`
PublicIP string `yaml:"public-ip" env:"FLEET_PUBLIC_IP"`
Verbosity string `yaml:"verbosity" env:"FLEET_VERBOSITY"`
}

9
config/oem.go Normal file
View File

@@ -0,0 +1,9 @@
package config
type OEM struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version-id"`
HomeURL string `yaml:"home-url"`
BugReportURL string `yaml:"bug-report-url"`
}

34
config/unit.go Normal file
View File

@@ -0,0 +1,34 @@
package config
import (
"path/filepath"
"strings"
)
type Unit struct {
Name string `yaml:"name"`
Mask bool `yaml:"mask"`
Enable bool `yaml:"enable"`
Runtime bool `yaml:"runtime"`
Content string `yaml:"content"`
Command string `yaml:"command"`
// For drop-in units, a cloudinit.conf is generated.
// This is currently unbound in YAML (and hence unsettable in cloud-config files)
// until the correct behaviour for multiple drop-in units is determined.
DropIn bool `yaml:"-"`
}
func (u *Unit) Type() string {
ext := filepath.Ext(u.Name)
return strings.TrimLeft(ext, ".")
}
func (u *Unit) Group() string {
switch u.Type() {
case "network", "netdev", "link":
return "network"
default:
return "system"
}
}

7
config/update.go Normal file
View File

@@ -0,0 +1,7 @@
package config
type Update struct {
RebootStrategy string `yaml:"reboot-strategy" env:"REBOOT_STRATEGY" valid:"best-effort,etcd-lock,reboot,off"`
Group string `yaml:"group" env:"GROUP"`
Server string `yaml:"server" env:"SERVER"`
}

17
config/user.go Normal file
View File

@@ -0,0 +1,17 @@
package config
type User struct {
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
SSHImportGithubUser string `yaml:"coreos-ssh-import-github"`
SSHImportURL string `yaml:"coreos-ssh-import-url"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
}

View File

@@ -7,6 +7,7 @@ import (
"sync"
"time"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/datasource"
"github.com/coreos/coreos-cloudinit/datasource/configdrive"
"github.com/coreos/coreos-cloudinit/datasource/file"
@@ -21,94 +22,125 @@ import (
)
const (
version = "0.9.5"
version = "0.10.4"
datasourceInterval = 100 * time.Millisecond
datasourceMaxInterval = 30 * time.Second
datasourceTimeout = 5 * time.Minute
)
var (
printVersion bool
ignoreFailure bool
sources struct {
file string
configDrive string
metadataService bool
ec2MetadataService string
cloudSigmaMetadataService bool
digitalOceanMetadataService string
url string
procCmdLine bool
}
convertNetconf string
workspace string
sshKeyName string
flags = struct {
printVersion bool
ignoreFailure bool
sources struct {
file string
configDrive string
metadataService bool
ec2MetadataService string
cloudSigmaMetadataService bool
digitalOceanMetadataService string
url string
procCmdLine bool
}
convertNetconf string
workspace string
sshKeyName string
oem string
}{}
)
func init() {
flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
flag.StringVar(&sources.file, "from-file", "", "Read user-data from provided file")
flag.StringVar(&sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory")
flag.BoolVar(&sources.metadataService, "from-metadata-service", false, "[DEPRECATED - Use -from-ec2-metadata] Download data from metadata service")
flag.StringVar(&sources.ec2MetadataService, "from-ec2-metadata", "", "Download EC2 data from the provided url")
flag.BoolVar(&sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context")
flag.StringVar(&sources.digitalOceanMetadataService, "from-digitalocean-metadata", "", "Download DigitalOcean data from the provided url")
flag.StringVar(&sources.url, "from-url", "", "Download user-data from provided url")
flag.BoolVar(&sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", proc_cmdline.ProcCmdlineLocation, proc_cmdline.ProcCmdlineCloudConfigFlag))
flag.StringVar(&convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files")
flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
flag.BoolVar(&flags.printVersion, "version", false, "Print the version and exit")
flag.BoolVar(&flags.ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
flag.StringVar(&flags.sources.file, "from-file", "", "Read user-data from provided file")
flag.StringVar(&flags.sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory")
flag.BoolVar(&flags.sources.metadataService, "from-metadata-service", false, "[DEPRECATED - Use -from-ec2-metadata] Download data from metadata service")
flag.StringVar(&flags.sources.ec2MetadataService, "from-ec2-metadata", "", "Download EC2 data from the provided url")
flag.BoolVar(&flags.sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context")
flag.StringVar(&flags.sources.digitalOceanMetadataService, "from-digitalocean-metadata", "", "Download DigitalOcean data from the provided url")
flag.StringVar(&flags.sources.url, "from-url", "", "Download user-data from provided url")
flag.BoolVar(&flags.sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", proc_cmdline.ProcCmdlineLocation, proc_cmdline.ProcCmdlineCloudConfigFlag))
flag.StringVar(&flags.oem, "oem", "", "Use the settings specific to the provided OEM")
flag.StringVar(&flags.convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files")
flag.StringVar(&flags.workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
flag.StringVar(&flags.sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
}
type oemConfig map[string]string
var (
oemConfigs = map[string]oemConfig{
"digitalocean": oemConfig{
"from-digitalocean-metadata": "http://169.254.169.254/",
"convert-netconf": "digitalocean",
},
"ec2-compat": oemConfig{
"from-ec2-metadata": "http://169.254.169.254/",
"from-configdrive": "/media/configdrive",
},
"rackspace-onmetal": oemConfig{
"from-configdrive": "/media/configdrive",
"convert-netconf": "debian",
},
}
)
func main() {
failure := false
flag.Parse()
die := func() {
if ignoreFailure {
os.Exit(0)
if c, ok := oemConfigs[flags.oem]; ok {
for k, v := range c {
flag.Set(k, v)
}
os.Exit(1)
} else if flags.oem != "" {
oems := make([]string, 0, len(oemConfigs))
for k := range oemConfigs {
oems = append(oems, k)
}
fmt.Printf("Invalid option to --oem: %q. Supported options: %q\n", flags.oem, oems)
os.Exit(2)
}
if printVersion == true {
if flags.printVersion == true {
fmt.Printf("coreos-cloudinit version %s\n", version)
os.Exit(0)
}
switch convertNetconf {
switch flags.convertNetconf {
case "":
case "debian":
case "digitalocean":
default:
fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian, digitalocean'\n", convertNetconf)
os.Exit(1)
fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian, digitalocean'\n", flags.convertNetconf)
os.Exit(2)
}
dss := getDatasources()
if len(dss) == 0 {
fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-ec2-metadata, --from-cloudsigma-metadata, --from-url or --from-proc-cmdline")
os.Exit(1)
os.Exit(2)
}
ds := selectDatasource(dss)
if ds == nil {
fmt.Println("No datasources available in time")
die()
os.Exit(1)
}
fmt.Printf("Fetching user-data from datasource of type %q\n", ds.Type())
userdataBytes, err := ds.FetchUserdata()
if err != nil {
fmt.Printf("Failed fetching user-data from datasource: %v\n", err)
die()
fmt.Printf("Failed fetching user-data from datasource: %v\nContinuing...\n", err)
failure = true
}
fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
metadataBytes, err := ds.FetchMetadata()
if err != nil {
fmt.Printf("Failed fetching meta-data from datasource: %v\n", err)
die()
os.Exit(1)
}
// Extract IPv4 addresses from metadata if possible
@@ -117,44 +149,44 @@ func main() {
subs, err = initialize.ExtractIPsFromMetadata(metadataBytes)
if err != nil {
fmt.Printf("Failed extracting IPs from meta-data: %v\n", err)
die()
os.Exit(1)
}
}
// Apply environment to user-data
env := initialize.NewEnvironment("/", ds.ConfigRoot(), workspace, convertNetconf, sshKeyName, subs)
env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.convertNetconf, flags.sshKeyName, subs)
userdata := env.Apply(string(userdataBytes))
var ccm, ccu *initialize.CloudConfig
var ccm, ccu *config.CloudConfig
var script *system.Script
if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil {
fmt.Printf("Failed to parse meta-data: %v\n", err)
die()
os.Exit(1)
}
if ccm != nil {
if ccm != nil && flags.convertNetconf != "" {
fmt.Printf("Fetching network config from datasource of type %q\n", ds.Type())
netconfBytes, err := ds.FetchNetworkConfig(ccm.NetworkConfigPath)
if err != nil {
fmt.Printf("Failed fetching network config from datasource: %v\n", err)
die()
os.Exit(1)
}
ccm.NetworkConfig = string(netconfBytes)
}
if ud, err := initialize.ParseUserData(userdata); err != nil {
fmt.Printf("Failed to parse user-data: %v\n", err)
die()
fmt.Printf("Failed to parse user-data: %v\nContinuing...\n", err)
failure = true
} else {
switch t := ud.(type) {
case *initialize.CloudConfig:
case *config.CloudConfig:
ccu = t
case system.Script:
script = &t
}
}
var cc *initialize.CloudConfig
var cc *config.CloudConfig
if ccm != nil && ccu != nil {
fmt.Println("Merging cloud-config from meta-data and user-data")
merged := mergeCloudConfig(*ccm, *ccu)
@@ -172,16 +204,20 @@ func main() {
if cc != nil {
if err = initialize.Apply(*cc, env); err != nil {
fmt.Printf("Failed to apply cloud-config: %v\n", err)
die()
os.Exit(1)
}
}
if script != nil {
if err = runScript(*script, env); err != nil {
fmt.Printf("Failed to run script: %v\n", err)
die()
os.Exit(1)
}
}
if failure && !flags.ignoreFailure {
os.Exit(1)
}
}
// mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from
@@ -189,7 +225,7 @@ func main() {
// not already set on udcc (i.e. user-data always takes precedence)
// NB: This needs to be kept in sync with ParseMetadata so that it tracks all
// elements of a CloudConfig which that function can populate.
func mergeCloudConfig(mdcc, udcc initialize.CloudConfig) (cc initialize.CloudConfig) {
func mergeCloudConfig(mdcc, udcc config.CloudConfig) (cc config.CloudConfig) {
if mdcc.Hostname != "" {
if udcc.Hostname != "" {
fmt.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)\n", udcc.Hostname, mdcc.Hostname)
@@ -222,28 +258,28 @@ func mergeCloudConfig(mdcc, udcc initialize.CloudConfig) (cc initialize.CloudCon
// on the different source command-line flags.
func getDatasources() []datasource.Datasource {
dss := make([]datasource.Datasource, 0, 5)
if sources.file != "" {
dss = append(dss, file.NewDatasource(sources.file))
if flags.sources.file != "" {
dss = append(dss, file.NewDatasource(flags.sources.file))
}
if sources.url != "" {
dss = append(dss, url.NewDatasource(sources.url))
if flags.sources.url != "" {
dss = append(dss, url.NewDatasource(flags.sources.url))
}
if sources.configDrive != "" {
dss = append(dss, configdrive.NewDatasource(sources.configDrive))
if flags.sources.configDrive != "" {
dss = append(dss, configdrive.NewDatasource(flags.sources.configDrive))
}
if sources.metadataService {
if flags.sources.metadataService {
dss = append(dss, ec2.NewDatasource(ec2.DefaultAddress))
}
if sources.ec2MetadataService != "" {
dss = append(dss, ec2.NewDatasource(sources.ec2MetadataService))
if flags.sources.ec2MetadataService != "" {
dss = append(dss, ec2.NewDatasource(flags.sources.ec2MetadataService))
}
if sources.cloudSigmaMetadataService {
if flags.sources.cloudSigmaMetadataService {
dss = append(dss, cloudsigma.NewServerContextService())
}
if sources.digitalOceanMetadataService != "" {
dss = append(dss, digitalocean.NewDatasource(sources.digitalOceanMetadataService))
if flags.sources.digitalOceanMetadataService != "" {
dss = append(dss, digitalocean.NewDatasource(flags.sources.digitalOceanMetadataService))
}
if sources.procCmdLine {
if flags.sources.procCmdLine {
dss = append(dss, proc_cmdline.NewDatasource())
}
return dss

View File

@@ -4,37 +4,37 @@ import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/initialize"
"github.com/coreos/coreos-cloudinit/config"
)
func TestMergeCloudConfig(t *testing.T) {
simplecc := initialize.CloudConfig{
simplecc := config.CloudConfig{
SSHAuthorizedKeys: []string{"abc", "def"},
Hostname: "foobar",
NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{}`,
}
for i, tt := range []struct {
udcc initialize.CloudConfig
mdcc initialize.CloudConfig
want initialize.CloudConfig
udcc config.CloudConfig
mdcc config.CloudConfig
want config.CloudConfig
}{
{
// If mdcc is empty, udcc should be returned unchanged
simplecc,
initialize.CloudConfig{},
config.CloudConfig{},
simplecc,
},
{
// If udcc is empty, mdcc should be returned unchanged(overridden)
initialize.CloudConfig{},
config.CloudConfig{},
simplecc,
simplecc,
},
{
// user-data should override completely in the case of conflicts
simplecc,
initialize.CloudConfig{
config.CloudConfig{
Hostname: "meta-hostname",
NetworkConfigPath: "/path/meta",
NetworkConfig: `{"hostname":"test"}`,
@@ -43,17 +43,17 @@ func TestMergeCloudConfig(t *testing.T) {
},
{
// Mixed merge should succeed
initialize.CloudConfig{
config.CloudConfig{
SSHAuthorizedKeys: []string{"abc", "def"},
Hostname: "user-hostname",
NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{"hostname":"test"}`,
},
initialize.CloudConfig{
config.CloudConfig{
SSHAuthorizedKeys: []string{"woof", "qux"},
Hostname: "meta-hostname",
},
initialize.CloudConfig{
config.CloudConfig{
SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"},
Hostname: "user-hostname",
NetworkConfigPath: "/path/somewhere",
@@ -62,15 +62,15 @@ func TestMergeCloudConfig(t *testing.T) {
},
{
// Completely non-conflicting merge should be fine
initialize.CloudConfig{
config.CloudConfig{
Hostname: "supercool",
},
initialize.CloudConfig{
config.CloudConfig{
SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
NetworkConfigPath: "/dev/fun",
NetworkConfig: `{"hostname":"test"}`,
},
initialize.CloudConfig{
config.CloudConfig{
Hostname: "supercool",
SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
NetworkConfigPath: "/dev/fun",
@@ -79,33 +79,33 @@ func TestMergeCloudConfig(t *testing.T) {
},
{
// Non-mergeable settings in user-data should not be affected
initialize.CloudConfig{
config.CloudConfig{
Hostname: "mememe",
ManageEtcHosts: initialize.EtcHosts("lolz"),
ManageEtcHosts: config.EtcHosts("lolz"),
},
initialize.CloudConfig{
config.CloudConfig{
Hostname: "youyouyou",
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
initialize.CloudConfig{
config.CloudConfig{
Hostname: "mememe",
ManageEtcHosts: initialize.EtcHosts("lolz"),
ManageEtcHosts: config.EtcHosts("lolz"),
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
},
{
// Non-mergeable (unexpected) settings in meta-data are ignored
initialize.CloudConfig{
config.CloudConfig{
Hostname: "mememe",
},
initialize.CloudConfig{
ManageEtcHosts: initialize.EtcHosts("lolz"),
config.CloudConfig{
ManageEtcHosts: config.EtcHosts("lolz"),
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
initialize.CloudConfig{
config.CloudConfig{
Hostname: "mememe",
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,

View File

@@ -42,6 +42,9 @@ func (cd *configDrive) FetchUserdata() ([]byte, error) {
}
func (cd *configDrive) FetchNetworkConfig(filename string) ([]byte, error) {
if filename == "" {
return []byte{}, nil
}
return cd.tryReadFile(path.Join(cd.openstackRoot(), filename))
}

View File

@@ -13,9 +13,9 @@ import (
const (
DefaultAddress = "http://169.254.169.254/"
apiVersion = "2009-04-04"
userdataPath = apiVersion + "/user-data"
metadataPath = apiVersion + "/meta-data"
apiVersion = "2009-04-04/"
userdataPath = apiVersion + "user-data"
metadataPath = apiVersion + "meta-data"
)
type metadataService struct {

View File

@@ -6,8 +6,7 @@ import (
"log"
"path"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/network"
"github.com/coreos/coreos-cloudinit/system"
)
@@ -17,146 +16,19 @@ import (
type CloudConfigFile interface {
// File should either return (*system.File, error), or (nil, nil) if nothing
// needs to be done for this configuration option.
File(root string) (*system.File, error)
File() (*system.File, error)
}
// CloudConfigUnit represents a CoreOS specific configuration option that can generate
// associated system.Units to be created/enabled appropriately
type CloudConfigUnit interface {
Units(root string) ([]system.Unit, error)
}
// CloudConfig encapsulates the entire cloud-config configuration file and maps directly to YAML
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd EtcdEnvironment
Fleet FleetEnvironment
OEM OEMRelease
Update UpdateConfig
Units []system.Unit
}
WriteFiles []system.File `yaml:"write_files"`
Hostname string
Users []system.User
ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
NetworkConfigPath string
NetworkConfig string
}
type warner func(format string, v ...interface{})
// warnOnUnrecognizedKeys parses the contents of a cloud-config file and calls
// warn(msg, key) for every unrecognized key (i.e. those not present in CloudConfig)
func warnOnUnrecognizedKeys(contents string, warn warner) {
// Generate a map of all understood cloud config options
var cc map[string]interface{}
b, _ := goyaml.Marshal(&CloudConfig{})
goyaml.Unmarshal(b, &cc)
// Now unmarshal the entire provided contents
var c map[string]interface{}
goyaml.Unmarshal([]byte(contents), &c)
// Check that every key in the contents exists in the cloud config
for k, _ := range c {
if _, ok := cc[k]; !ok {
warn("Warning: unrecognized key %q in provided cloud config - ignoring section", k)
}
}
// Check for unrecognized coreos options, if any are set
if coreos, ok := c["coreos"]; ok {
if set, ok := coreos.(map[interface{}]interface{}); ok {
known := cc["coreos"].(map[interface{}]interface{})
for k, _ := range set {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", k)
}
}
}
}
// Check for any badly-specified users, if any are set
if users, ok := c["users"]; ok {
var known map[string]interface{}
b, _ := goyaml.Marshal(&system.User{})
goyaml.Unmarshal(b, &known)
if set, ok := users.([]interface{}); ok {
for _, u := range set {
if user, ok := u.(map[interface{}]interface{}); ok {
for k, _ := range user {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", k)
}
}
}
}
}
}
// Check for any badly-specified files, if any are set
if files, ok := c["write_files"]; ok {
var known map[string]interface{}
b, _ := goyaml.Marshal(&system.File{})
goyaml.Unmarshal(b, &known)
if set, ok := files.([]interface{}); ok {
for _, f := range set {
if file, ok := f.(map[interface{}]interface{}); ok {
for k, _ := range file {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", k)
}
}
}
}
}
}
}
// NewCloudConfig instantiates a new CloudConfig from the given contents (a
// string of YAML), returning any error encountered. It will ignore unknown
// fields but log encountering them.
func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig
err := goyaml.Unmarshal([]byte(contents), &cfg)
if err != nil {
return &cfg, err
}
warnOnUnrecognizedKeys(contents, log.Printf)
return &cfg, nil
}
func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
Units() ([]system.Unit, error)
}
// Apply renders a CloudConfig to an Environment. This can involve things like
// configuring the hostname, adding new users, writing various configuration
// files to disk, and manipulating systemd services.
func Apply(cfg CloudConfig, env *Environment) error {
func Apply(cfg config.CloudConfig, env *Environment) error {
if cfg.Hostname != "" {
if err := system.SetHostname(cfg.Hostname); err != nil {
return err
@@ -216,26 +88,44 @@ func Apply(cfg CloudConfig, env *Environment) error {
}
}
for _, ccf := range []CloudConfigFile{cfg.Coreos.OEM, cfg.Coreos.Update, cfg.ManageEtcHosts} {
f, err := ccf.File(env.Root())
var writeFiles []system.File
for _, file := range cfg.WriteFiles {
writeFiles = append(writeFiles, system.File{file})
}
for _, ccf := range []CloudConfigFile{
system.OEM{cfg.Coreos.OEM},
system.Update{cfg.Coreos.Update, system.DefaultReadConfig},
system.EtcHosts{cfg.ManageEtcHosts},
} {
f, err := ccf.File()
if err != nil {
return err
}
if f != nil {
cfg.WriteFiles = append(cfg.WriteFiles, *f)
writeFiles = append(writeFiles, *f)
}
}
for _, ccu := range []CloudConfigUnit{cfg.Coreos.Etcd, cfg.Coreos.Fleet, cfg.Coreos.Update} {
u, err := ccu.Units(env.Root())
var units []system.Unit
for _, u := range cfg.Coreos.Units {
units = append(units, system.Unit{u})
}
for _, ccu := range []CloudConfigUnit{
system.Etcd{cfg.Coreos.Etcd},
system.Fleet{cfg.Coreos.Fleet},
system.Update{cfg.Coreos.Update, system.DefaultReadConfig},
} {
u, err := ccu.Units()
if err != nil {
return err
}
cfg.Coreos.Units = append(cfg.Coreos.Units, u...)
units = append(units, u...)
}
wroteEnvironment := false
for _, file := range cfg.WriteFiles {
for _, file := range writeFiles {
fullPath, err := system.WriteFile(&file, env.Root())
if err != nil {
return err
@@ -282,7 +172,7 @@ func Apply(cfg CloudConfig, env *Environment) error {
}
um := system.NewUnitManager(env.Root())
return processUnits(cfg.Coreos.Units, env.Root(), um)
return processUnits(units, env.Root(), um)
}
@@ -291,7 +181,11 @@ func Apply(cfg CloudConfig, env *Environment) error {
// disk, masking/unmasking units, or invoking systemd
// commands against units. It returns any error encountered.
func processUnits(units []system.Unit, root string, um system.UnitManager) error {
commands := make(map[string]string, 0)
type action struct {
unit string
command string
}
actions := make([]action, 0, len(units))
reload := false
for _, unit := range units {
dst := unit.Destination(root)
@@ -329,9 +223,9 @@ func processUnits(units []system.Unit, root string, um system.UnitManager) error
}
if unit.Group() == "network" {
commands["systemd-networkd.service"] = "restart"
actions = append(actions, action{"systemd-networkd.service", "restart"})
} else if unit.Command != "" {
commands[unit.Name] = unit.Command
actions = append(actions, action{unit.Name, unit.Command})
}
}
@@ -341,13 +235,13 @@ func processUnits(units []system.Unit, root string, um system.UnitManager) error
}
}
for unit, command := range commands {
log.Printf("Calling unit command '%s %s'", command, unit)
res, err := um.RunUnitCommand(command, unit)
for _, action := range actions {
log.Printf("Calling unit command '%s %s'", action.command, action.unit)
res, err := um.RunUnitCommand(action.command, action.unit)
if err != nil {
return err
}
log.Printf("Result of '%s %s': %s", command, unit, res)
log.Printf("Result of '%s %s': %s", action.command, action.unit, res)
}
return nil

View File

@@ -1,368 +1,12 @@
package initialize
import (
"fmt"
"strings"
"testing"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
func TestCloudConfigInvalidKeys(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("panic while instantiating CloudConfig with nil keys: %v", r)
}
}()
for _, tt := range []struct {
contents string
}{
{"coreos:"},
{"ssh_authorized_keys:"},
{"ssh_authorized_keys:\n -"},
{"ssh_authorized_keys:\n - 0:"},
{"write_files:"},
{"write_files:\n -"},
{"write_files:\n - 0:"},
{"users:"},
{"users:\n -"},
{"users:\n - 0:"},
} {
_, err := NewCloudConfig(tt.contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with invalid keys: %v", err)
}
}
}
func TestCloudConfigUnknownKeys(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
coreos_unknown:
foo: "bar"
section_unknown:
dunno:
something
bare_unknown:
bar
write_files:
- content: fun
path: /var/party
file_unknown: nofun
users:
- name: fry
passwd: somehash
user_unknown: philip
hostname:
foo
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
}
if cfg.Hostname != "foo" {
t.Fatalf("hostname not correctly set when invalid keys are present")
}
if len(cfg.Coreos.Etcd) < 1 {
t.Fatalf("etcd section not correctly set when invalid keys are present")
}
if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
t.Fatalf("write_files section not correctly set when invalid keys are present")
}
if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
t.Fatalf("users section not correctly set when invalid keys are present")
}
var warnings string
catchWarn := func(f string, v ...interface{}) {
warnings += fmt.Sprintf(f, v...)
}
warnOnUnrecognizedKeys(contents, catchWarn)
if !strings.Contains(warnings, "coreos_unknown") {
t.Errorf("warnings did not catch unrecognized coreos option coreos_unknown")
}
if !strings.Contains(warnings, "bare_unknown") {
t.Errorf("warnings did not catch unrecognized key bare_unknown")
}
if !strings.Contains(warnings, "section_unknown") {
t.Errorf("warnings did not catch unrecognized key section_unknown")
}
if !strings.Contains(warnings, "user_unknown") {
t.Errorf("warnings did not catch unrecognized user key user_unknown")
}
if !strings.Contains(warnings, "file_unknown") {
t.Errorf("warnings did not catch unrecognized file key file_unknown")
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig("")
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
if cfg.Hostname != "" {
t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
update:
reboot-strategy: reboot
units:
- name: 50-eth0.network
runtime: yes
content: '[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
write_files:
- content: |
penny
elroy
path: /etc/dogepack.conf
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 2 {
t.Error("Parsed incorrect number of SSH keys")
} else if keys[0] != "foobar" {
t.Error("Expected first SSH key to be 'foobar'")
} else if keys[1] != "foobaz" {
t.Error("Expected first SSH key to be 'foobaz'")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
wf := cfg.WriteFiles[0]
if wf.Content != "penny\nelroy\n" {
t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
}
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if perm, _ := wf.Permissions(); perm != 0644 {
t.Errorf("WriteFile has incorrect permissions %s", perm)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
}
if wf.Owner != "root:dogepack" {
t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
}
}
if len(cfg.Coreos.Units) != 1 {
t.Error("Failed to parse correct number of units")
} else {
u := cfg.Coreos.Units[0]
expect := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if u.Content != expect {
t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
}
if u.Runtime != true {
t.Errorf("Unit has incorrect runtime value")
}
if u.Name != "50-eth0.network" {
t.Errorf("Unit has incorrect name %s", u.Name)
}
if u.Type() != "network" {
t.Errorf("Unit has incorrect type '%s'", u.Type())
}
}
if cfg.Coreos.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
if cfg.Coreos.Update["reboot-strategy"] != "reboot" {
t.Errorf("Failed to parse locksmith strategy")
}
}
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := `
ssh_authorized_keys:
- foo: bar
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig("")
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
t.Fatalf("Serialized config did not have expected header")
}
}
// TestDropInIgnored asserts that users are unable to set DropIn=True on units
func TestDropInIgnored(t *testing.T) {
contents := `
coreos:
units:
- name: test
dropin: true
`
cfg, err := NewCloudConfig(contents)
if err != nil || len(cfg.Coreos.Units) != 1 {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Coreos.Units) != 1 || cfg.Coreos.Units[0].Name != "test" {
t.Fatalf("Expected 1 unit, but got %d: %v", len(cfg.Coreos.Units), cfg.Coreos.Units)
}
if cfg.Coreos.Units[0].DropIn {
t.Errorf("dropin option on unit in cloud-config was not ignored!")
}
}
func TestCloudConfigUsers(t *testing.T) {
contents := `
users:
- name: elroy
passwd: somehash
ssh-authorized-keys:
- somekey
gecos: arbitrary comment
homedir: /home/place
no-create-home: yes
primary-group: things
groups:
- ping
- pong
no-user-group: true
system: y
no-log-init: True
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.PasswordHash != "somehash" {
t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
}
if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
} else {
key := user.SSHAuthorizedKeys[0]
if key != "somekey" {
t.Errorf("User SSH key is %q, expected 'somekey'", key)
}
}
if user.GECOS != "arbitrary comment" {
t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
}
if user.Homedir != "/home/place" {
t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
}
if !user.NoCreateHome {
t.Errorf("Failed to parse no-create-home field")
}
if user.PrimaryGroup != "things" {
t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
}
if len(user.Groups) != 2 {
t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
} else {
if user.Groups[0] != "ping" {
t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
}
if user.Groups[1] != "pong" {
t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
}
}
if !user.NoUserGroup {
t.Errorf("Failed to parse no-user-group field")
}
if !user.System {
t.Errorf("Failed to parse system field")
}
if !user.NoLogInit {
t.Errorf("Failed to parse no-log-init field")
}
}
type TestUnitManager struct {
placed []string
enabled []string
@@ -401,10 +45,10 @@ func (tum *TestUnitManager) UnmaskUnit(unit *system.Unit) error {
func TestProcessUnits(t *testing.T) {
tum := &TestUnitManager{}
units := []system.Unit{
system.Unit{
system.Unit{config.Unit{
Name: "foo",
Mask: true,
},
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
@@ -415,9 +59,9 @@ func TestProcessUnits(t *testing.T) {
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{
system.Unit{config.Unit{
Name: "bar.network",
},
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
@@ -428,10 +72,10 @@ func TestProcessUnits(t *testing.T) {
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{
system.Unit{config.Unit{
Name: "baz.service",
Content: "[Service]\nExecStart=/bin/true",
},
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
@@ -442,10 +86,10 @@ func TestProcessUnits(t *testing.T) {
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{
system.Unit{config.Unit{
Name: "locksmithd.service",
Runtime: true,
},
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
@@ -456,10 +100,10 @@ func TestProcessUnits(t *testing.T) {
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{
system.Unit{config.Unit{
Name: "woof",
Enable: true,
},
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)

View File

@@ -3,8 +3,10 @@ package initialize
import (
"os"
"path"
"regexp"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
@@ -28,6 +30,8 @@ func NewEnvironment(root, configRoot, workspace, netconfType, sshKeyName string,
for k, v := range map[string]string{
"$public_ipv4": os.Getenv("COREOS_PUBLIC_IPV4"),
"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
"$public_ipv6": os.Getenv("COREOS_PUBLIC_IPV6"),
"$private_ipv6": os.Getenv("COREOS_PRIVATE_IPV6"),
} {
if _, ok := substitutions[k]; !ok {
substitutions[k] = v
@@ -60,18 +64,27 @@ func (e *Environment) SetSSHKeyName(name string) {
e.sshKeyName = name
}
// Apply goes through the map of substitutions and replaces all instances of
// the keys with their respective values. It supports escaping substitutions
// with a leading '\'.
func (e *Environment) Apply(data string) string {
for key, val := range e.substitutions {
data = strings.Replace(data, key, val, -1)
matchKey := strings.Replace(key, `$`, `\$`, -1)
replKey := strings.Replace(key, `$`, `$$`, -1)
// "key" -> "val"
data = regexp.MustCompile(`([^\\]|^)`+matchKey).ReplaceAllString(data, `${1}`+val)
// "\key" -> "key"
data = regexp.MustCompile(`\\`+matchKey).ReplaceAllString(data, replKey)
}
return data
}
func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
ef := system.EnvFile{
File: &system.File{
File: &system.File{config.File{
Path: "/etc/environment",
},
}},
Vars: map[string]string{},
}
if ip, ok := e.substitutions["$public_ipv4"]; ok && len(ip) > 0 {
@@ -80,22 +93,15 @@ func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
if ip, ok := e.substitutions["$private_ipv4"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PRIVATE_IPV4"] = ip
}
if ip, ok := e.substitutions["$public_ipv6"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PUBLIC_IPV6"] = ip
}
if ip, ok := e.substitutions["$private_ipv6"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PRIVATE_IPV6"] = ip
}
if len(ef.Vars) == 0 {
return nil
} else {
return &ef
}
}
// normalizeSvcEnv standardizes the keys of the map (environment variables for a service)
// by replacing any dashes with underscores and ensuring they are entirely upper case.
// For example, "some-env" --> "SOME_ENV"
func normalizeSvcEnv(m map[string]string) map[string]string {
out := make(map[string]string, len(m))
for key, val := range m {
key = strings.ToUpper(key)
key = strings.Replace(key, "-", "_", -1)
out[key] = val
}
return out
}

View File

@@ -12,6 +12,8 @@ import (
func TestEnvironmentApply(t *testing.T) {
os.Setenv("COREOS_PUBLIC_IPV4", "1.2.3.4")
os.Setenv("COREOS_PRIVATE_IPV4", "5.6.7.8")
os.Setenv("COREOS_PUBLIC_IPV6", "1234::")
os.Setenv("COREOS_PRIVATE_IPV6", "5678::")
for _, tt := range []struct {
subs map[string]string
input string
@@ -23,14 +25,16 @@ func TestEnvironmentApply(t *testing.T) {
map[string]string{
"$public_ipv4": "192.0.2.3",
"$private_ipv4": "192.0.2.203",
"$public_ipv6": "fe00:1234::",
"$private_ipv6": "fe00:5678::",
},
`[Service]
ExecStart=/usr/bin/echo "$public_ipv4"
ExecStop=/usr/bin/echo $private_ipv4
ExecStart=/usr/bin/echo "$public_ipv4 $public_ipv6"
ExecStop=/usr/bin/echo $private_ipv4 $private_ipv6
ExecStop=/usr/bin/echo $unknown`,
`[Service]
ExecStart=/usr/bin/echo "192.0.2.3"
ExecStop=/usr/bin/echo 192.0.2.203
ExecStart=/usr/bin/echo "192.0.2.3 fe00:1234::"
ExecStop=/usr/bin/echo 192.0.2.203 fe00:5678::
ExecStop=/usr/bin/echo $unknown`,
},
{
@@ -51,6 +55,24 @@ ExecStop=/usr/bin/echo $unknown`,
"$private_ipv4\nfoobar",
"5.6.7.8\nfoobar",
},
{
// Escaping substitutions
map[string]string{"$private_ipv4": "127.0.0.1"},
`\$private_ipv4
$private_ipv4
addr: \$private_ipv4
\\$private_ipv4`,
`$private_ipv4
127.0.0.1
addr: $private_ipv4
\$private_ipv4`,
},
{
// No substitutions with escaping
nil,
"\\$test\n$test",
"\\$test\n$test",
},
} {
env := NewEnvironment("./", "./", "./", "", "", tt.subs)
@@ -65,8 +87,10 @@ func TestEnvironmentFile(t *testing.T) {
subs := map[string]string{
"$public_ipv4": "1.2.3.4",
"$private_ipv4": "5.6.7.8",
"$public_ipv6": "1234::",
"$private_ipv6": "5678::",
}
expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PUBLIC_IPV4=1.2.3.4\n"
expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PRIVATE_IPV6=5678::\nCOREOS_PUBLIC_IPV4=1.2.3.4\nCOREOS_PUBLIC_IPV6=1234::\n"
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
@@ -96,6 +120,8 @@ func TestEnvironmentFileNil(t *testing.T) {
subs := map[string]string{
"$public_ipv4": "",
"$private_ipv4": "",
"$public_ipv6": "",
"$private_ipv6": "",
}
env := NewEnvironment("./", "./", "./", "", "", subs)

View File

@@ -1,63 +0,0 @@
package initialize
import (
"errors"
"fmt"
"sort"
"github.com/coreos/coreos-cloudinit/system"
)
type EtcdEnvironment map[string]string
func (ee EtcdEnvironment) String() (out string) {
norm := normalizeSvcEnv(ee)
if val, ok := norm["DISCOVERY_URL"]; ok {
delete(norm, "DISCOVERY_URL")
if _, ok := norm["DISCOVERY"]; !ok {
norm["DISCOVERY"] = val
}
}
var sorted sort.StringSlice
for k, _ := range norm {
sorted = append(sorted, k)
}
sorted.Sort()
out += "[Service]\n"
for _, key := range sorted {
val := norm[key]
out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
}
return
}
// Units creates a Unit file drop-in for etcd, using any configured
// options and adding a default MachineID if unset.
func (ee EtcdEnvironment) Units(root string) ([]system.Unit, error) {
if len(ee) < 1 {
return nil, nil
}
if _, ok := ee["name"]; !ok {
if machineID := system.MachineID(root); machineID != "" {
ee["name"] = machineID
} else if hostname, err := system.Hostname(); err == nil {
ee["name"] = hostname
} else {
return nil, errors.New("Unable to determine default etcd name")
}
}
etcd := system.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: ee.String(),
}
return []system.Unit{etcd}, nil
}

View File

@@ -1,184 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestEtcdEnvironment(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryURLTranslated(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryOverridesDiscoveryURL(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "ping"
cfg["discovery"] = "pong"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=pong"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
ee := EtcdEnvironment{
"name": "node001",
"discovery": "http://disco.example.com/foobar",
"peer-bind-addr": "127.0.0.1:7002",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
sd := system.NewUnitManager(dir)
uu, err := ee.Units(dir)
if err != nil {
t.Fatalf("Generating etcd unit failed: %v", err)
}
if len(uu) != 1 {
t.Fatalf("Expected 1 unit to be returned, got %d", len(uu))
}
u := uu[0]
dst := u.Destination(dir)
os.Stderr.WriteString("writing to " + dir + "\n")
if err := sd.PlaceUnit(&u, dst); err != nil {
t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_NAME=node001"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func TestEtcdEnvironmentEmptyNoOp(t *testing.T) {
ee := EtcdEnvironment{}
uu, err := ee.Units("")
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
if len(uu) > 0 {
t.Fatalf("Generated etcd units unexpectedly: %v")
}
}
func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
ee := EtcdEnvironment{"foo": "bar"}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
sd := system.NewUnitManager(dir)
os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
if err != nil {
t.Fatalf("Failed writing out /etc/machine-id: %v", err)
}
uu, err := ee.Units(dir)
if err != nil {
t.Fatalf("Generating etcd unit failed: %v", err)
}
if len(uu) == 0 {
t.Fatalf("Returned empty etcd units unexpectedly")
}
u := uu[0]
dst := u.Destination(dir)
os.Stderr.WriteString("writing to " + dir + "\n")
if err := sd.PlaceUnit(&u, dst); err != nil {
t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_FOO=bar"
Environment="ETCD_NAME=node007"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func TestEtcdEnvironmentWhenNil(t *testing.T) {
// EtcdEnvironment will be a nil map if it wasn't in the yaml
var ee EtcdEnvironment
if ee != nil {
t.Fatalf("EtcdEnvironment is not nil")
}
uu, err := ee.Units("")
if len(uu) != 0 || err != nil {
t.Fatalf("Units returned value for nil input")
}
}

View File

@@ -1,35 +0,0 @@
package initialize
import (
"fmt"
"github.com/coreos/coreos-cloudinit/system"
)
type FleetEnvironment map[string]string
func (fe FleetEnvironment) String() (out string) {
norm := normalizeSvcEnv(fe)
out += "[Service]\n"
for key, val := range norm {
out += fmt.Sprintf("Environment=\"FLEET_%s=%s\"\n", key, val)
}
return
}
// Units generates a Unit file drop-in for fleet, if any fleet options were
// configured in cloud-config
func (fe FleetEnvironment) Units(root string) ([]system.Unit, error) {
if len(fe) < 1 {
return nil, nil
}
fleet := system.Unit{
Name: "fleet.service",
Runtime: true,
DropIn: true,
Content: fe.String(),
}
return []system.Unit{fleet}, nil
}

View File

@@ -1,43 +0,0 @@
package initialize
import "testing"
func TestFleetEnvironment(t *testing.T) {
cfg := make(FleetEnvironment, 0)
cfg["public-ip"] = "12.34.56.78"
env := cfg.String()
expect := `[Service]
Environment="FLEET_PUBLIC_IP=12.34.56.78"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestFleetUnit(t *testing.T) {
cfg := make(FleetEnvironment, 0)
uu, err := cfg.Units("/")
if len(uu) != 0 {
t.Errorf("unexpectedly generated unit with empty FleetEnvironment")
}
cfg["public-ip"] = "12.34.56.78"
uu, err = cfg.Units("/")
if err != nil {
t.Errorf("error generating fleet unit: %v", err)
}
if len(uu) != 1 {
t.Fatalf("expected 1 unit generated, got %d", len(uu))
}
u := uu[0]
if !u.Runtime {
t.Errorf("bad Runtime for generated fleet unit!")
}
if !u.DropIn {
t.Errorf("bad DropIn for generated fleet unit!")
}
}

View File

@@ -1,32 +0,0 @@
package initialize
import (
"testing"
)
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}

View File

@@ -1,83 +0,0 @@
package initialize
import (
"fmt"
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestCloudConfigManageEtcHosts(t *testing.T) {
contents := `
manage_etc_hosts: localhost
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
manageEtcHosts := cfg.ManageEtcHosts
if manageEtcHosts != "localhost" {
t.Errorf("ManageEtcHosts value is %q, expected 'localhost'", manageEtcHosts)
}
}
func TestManageEtcHostsInvalidValue(t *testing.T) {
eh := EtcHosts("invalid")
if f, err := eh.File(""); err == nil || f != nil {
t.Fatalf("EtcHosts File succeeded with invalid value!")
}
}
func TestEtcHostsWrittenToDisk(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
eh := EtcHosts("localhost")
f, err := eh.File(dir)
if err != nil {
t.Fatalf("Error calling File on EtcHosts: %v", err)
}
if f == nil {
t.Fatalf("manageEtcHosts returned nil file unexpectedly")
}
if _, err := system.WriteFile(f, dir); err != nil {
t.Fatalf("Error writing EtcHosts: %v", err)
}
fullPath := path.Join(dir, "etc", "hosts")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
hostname, err := os.Hostname()
if err != nil {
t.Fatalf("Unable to read OS hostname: %v", err)
}
expect := fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname)
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

View File

@@ -3,11 +3,13 @@ package initialize
import (
"encoding/json"
"sort"
"github.com/coreos/coreos-cloudinit/config"
)
// ParseMetaData parses a JSON blob in the OpenStack metadata service format, and
// converts it to a partially hydrated CloudConfig
func ParseMetaData(contents string) (*CloudConfig, error) {
// ParseMetaData parses a JSON blob in the OpenStack metadata service format,
// and converts it to a partially hydrated CloudConfig.
func ParseMetaData(contents string) (*config.CloudConfig, error) {
if len(contents) == 0 {
return nil, nil
}
@@ -22,7 +24,7 @@ func ParseMetaData(contents string) (*CloudConfig, error) {
return nil, err
}
var cfg CloudConfig
var cfg config.CloudConfig
if len(metadata.SSHAuthorizedKeyMap) > 0 {
cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap))
for _, name := range sortedKeys(metadata.SSHAuthorizedKeyMap) {
@@ -34,22 +36,31 @@ func ParseMetaData(contents string) (*CloudConfig, error) {
return &cfg, nil
}
// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service format,
// and returns a substitution map possibly containing private_ipv4 and public_ipv4 addresses
// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service
// format and returns a substitution map possibly containing private_ipv4,
// public_ipv4, private_ipv6, and public_ipv6 addresses.
func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) {
var ips struct {
Public string `json:"public-ipv4"`
Private string `json:"local-ipv4"`
PublicIPv4 string `json:"public-ipv4"`
PrivateIPv4 string `json:"local-ipv4"`
PublicIPv6 string `json:"public-ipv6"`
PrivateIPv6 string `json:"local-ipv6"`
}
if err := json.Unmarshal(contents, &ips); err != nil {
return nil, err
}
m := make(map[string]string)
if ips.Private != "" {
m["$private_ipv4"] = ips.Private
if ips.PrivateIPv4 != "" {
m["$private_ipv4"] = ips.PrivateIPv4
}
if ips.Public != "" {
m["$public_ipv4"] = ips.Public
if ips.PublicIPv4 != "" {
m["$public_ipv4"] = ips.PublicIPv4
}
if ips.PrivateIPv6 != "" {
m["$private_ipv6"] = ips.PrivateIPv6
}
if ips.PublicIPv6 != "" {
m["$public_ipv6"] = ips.PublicIPv6
}
return m, nil
}

View File

@@ -1,21 +1,25 @@
package initialize
import "reflect"
import "testing"
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestParseMetadata(t *testing.T) {
for i, tt := range []struct {
in string
want *CloudConfig
want *config.CloudConfig
err bool
}{
{"", nil, false},
{`garbage, invalid json`, nil, true},
{`{"foo": "bar"}`, &CloudConfig{}, false},
{`{"network_config": {"content_path": "asdf"}}`, &CloudConfig{NetworkConfigPath: "asdf"}, false},
{`{"hostname": "turkleton"}`, &CloudConfig{Hostname: "turkleton"}, false},
{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"alice", "jill"}}, false},
{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
{`{"foo": "bar"}`, &config.CloudConfig{}, false},
{`{"network_config": {"content_path": "asdf"}}`, &config.CloudConfig{NetworkConfigPath: "asdf"}, false},
{`{"hostname": "turkleton"}`, &config.CloudConfig{Hostname: "turkleton"}, false},
{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &config.CloudConfig{SSHAuthorizedKeys: []string{"alice", "jill"}}, false},
{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &config.CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
} {
got, err := ParseMetaData(tt.in)
if tt.err != (err != nil) {
@@ -43,9 +47,9 @@ func TestExtractIPsFromMetadata(t *testing.T) {
out map[string]string
}{
{
[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4"}`),
[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4", "public-ipv6": "1234::", "local-ipv6": "5678::"}`),
false,
map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4"},
map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4", "$public_ipv6": "1234::", "$private_ipv6": "5678::"},
},
{
[]byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`),

View File

@@ -1,41 +0,0 @@
package initialize
import (
"fmt"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
type OEMRelease struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version-id"`
HomeURL string `yaml:"home-url"`
BugReportURL string `yaml:"bug-report-url"`
}
func (oem OEMRelease) String() string {
fields := []string{
fmt.Sprintf("ID=%s", oem.ID),
fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
fmt.Sprintf("NAME=%q", oem.Name),
fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
}
return strings.Join(fields, "\n") + "\n"
}
func (oem OEMRelease) File(root string) (*system.File, error) {
if oem.ID == "" {
return nil, nil
}
return &system.File{
Path: path.Join("etc", "oem-release"),
RawFilePermissions: "0644",
Content: oem.String(),
}, nil
}

View File

@@ -1,63 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestOEMReleaseWrittenToDisk(t *testing.T) {
oem := OEMRelease{
ID: "rackspace",
Name: "Rackspace Cloud Servers",
VersionID: "168.0.0",
HomeURL: "https://www.rackspace.com/cloud/servers/",
BugReportURL: "https://github.com/coreos/coreos-overlay",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
f, err := oem.File(dir)
if err != nil {
t.Fatalf("Processing of OEMRelease failed: %v", err)
}
if f == nil {
t.Fatalf("OEMRelease returned nil file unexpectedly")
}
if _, err := system.WriteFile(f, dir); err != nil {
t.Fatalf("Writing of OEMRelease failed: %v", err)
}
fullPath := path.Join(dir, "etc", "oem-release")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `ID=rackspace
VERSION_ID=168.0.0
NAME="Rackspace Cloud Servers"
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

View File

@@ -39,31 +39,4 @@ func TestCloudConfigUsersUrlMarshal(t *testing.T) {
if keys[2] != expected {
t.Fatalf("expected %s, got %s", expected, keys[2])
}
}
func TestCloudConfigUsersSSHImportURL(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
}
}

View File

@@ -1,165 +0,0 @@
package initialize
import (
"bufio"
"errors"
"fmt"
"os"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
const (
locksmithUnit = "locksmithd.service"
updateEngineUnit = "update-engine.service"
)
// updateOption represents a configurable update option, which, if set, will be
// written into update.conf, replacing any existing value for the option
type updateOption struct {
key string // key used to configure this option in cloud-config
valid []string // valid values for the option
prefix string // prefix for the option in the update.conf file
value string // used to store the new value in update.conf (including prefix)
seen bool // whether the option has been seen in any existing update.conf
}
// updateOptions defines the update options understood by cloud-config.
// The keys represent the string used in cloud-config to configure the option.
var updateOptions = []*updateOption{
&updateOption{
key: "reboot-strategy",
prefix: "REBOOT_STRATEGY=",
valid: []string{"best-effort", "etcd-lock", "reboot", "off"},
},
&updateOption{
key: "group",
prefix: "GROUP=",
},
&updateOption{
key: "server",
prefix: "SERVER=",
},
}
// isValid checks whether a supplied value is valid for this option
func (uo updateOption) isValid(val string) bool {
if len(uo.valid) == 0 {
return true
}
for _, v := range uo.valid {
if val == v {
return true
}
}
return false
}
type UpdateConfig map[string]string
// File generates an `/etc/coreos/update.conf` file (if any update
// configuration options are set in cloud-config) by either rewriting the
// existing file on disk, or starting from `/usr/share/coreos/update.conf`
func (uc UpdateConfig) File(root string) (*system.File, error) {
if len(uc) < 1 {
return nil, nil
}
var out string
// Generate the list of possible substitutions to be performed based on the options that are configured
subs := make([]*updateOption, 0)
for _, uo := range updateOptions {
val, ok := uc[uo.key]
if !ok {
continue
}
if !uo.isValid(val) {
return nil, errors.New(fmt.Sprintf("invalid value %v for option %v (valid options: %v)", val, uo.key, uo.valid))
}
uo.value = uo.prefix + val
subs = append(subs, uo)
}
etcUpdate := path.Join(root, "etc", "coreos", "update.conf")
usrUpdate := path.Join(root, "usr", "share", "coreos", "update.conf")
conf, err := os.Open(etcUpdate)
if os.IsNotExist(err) {
conf, err = os.Open(usrUpdate)
}
if err != nil {
return nil, err
}
scanner := bufio.NewScanner(conf)
for scanner.Scan() {
line := scanner.Text()
for _, s := range subs {
if strings.HasPrefix(line, s.prefix) {
line = s.value
s.seen = true
break
}
}
out += line
out += "\n"
if err := scanner.Err(); err != nil {
return nil, err
}
}
for _, s := range subs {
if !s.seen {
out += s.value
out += "\n"
}
}
return &system.File{
Path: path.Join("etc", "coreos", "update.conf"),
RawFilePermissions: "0644",
Content: out,
}, nil
}
// Units generates units for the cloud-init initializer to act on:
// - a locksmith system.Unit, if "reboot-strategy" was set in cloud-config
// - an update_engine system.Unit, if "group" was set in cloud-config
func (uc UpdateConfig) Units(root string) ([]system.Unit, error) {
var units []system.Unit
if strategy, ok := uc["reboot-strategy"]; ok {
ls := &system.Unit{
Name: locksmithUnit,
Command: "restart",
Mask: false,
Runtime: true,
}
if strategy == "off" {
ls.Command = "stop"
ls.Mask = true
}
units = append(units, *ls)
}
rue := false
if _, ok := uc["group"]; ok {
rue = true
}
if _, ok := uc["server"]; ok {
rue = true
}
if rue {
ue := system.Unit{
Name: updateEngineUnit,
Command: "restart",
}
units = append(units, ue)
}
return units, nil
}

View File

@@ -1,232 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"sort"
"strings"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
const (
base = `SERVER=https://example.com
GROUP=thegroupc`
configured = base + `
REBOOT_STRATEGY=awesome
`
expected = base + `
REBOOT_STRATEGY=etcd-lock
`
)
func setupFixtures(dir string) {
os.MkdirAll(path.Join(dir, "usr", "share", "coreos"), 0755)
os.MkdirAll(path.Join(dir, "run", "systemd", "system"), 0755)
ioutil.WriteFile(path.Join(dir, "usr", "share", "coreos", "update.conf"), []byte(base), 0644)
}
func TestEmptyUpdateConfig(t *testing.T) {
uc := &UpdateConfig{}
f, err := uc.File("")
if err != nil {
t.Error("unexpected error getting file from empty UpdateConfig")
}
if f != nil {
t.Errorf("getting file from empty UpdateConfig should have returned nil, got %v", f)
}
uu, err := uc.Units("")
if err != nil {
t.Error("unexpected error getting unit from empty UpdateConfig")
}
if len(uu) != 0 {
t.Errorf("getting unit from empty UpdateConfig should have returned zero units, got %d", len(uu))
}
}
func TestInvalidUpdateOptions(t *testing.T) {
uon := &updateOption{
key: "numbers",
prefix: "numero_",
valid: []string{"one", "two"},
}
uoa := &updateOption{
key: "any_will_do",
prefix: "any_",
}
if !uon.isValid("one") {
t.Error("update option did not accept valid option \"one\"")
}
if uon.isValid("three") {
t.Error("update option accepted invalid option \"three\"")
}
for _, s := range []string{"one", "asdf", "foobarbaz"} {
if !uoa.isValid(s) {
t.Errorf("update option with no \"valid\" field did not accept %q", s)
}
}
uc := &UpdateConfig{"reboot-strategy": "wizzlewazzle"}
f, err := uc.File("")
if err == nil {
t.Errorf("File did not give an error on invalid UpdateOption")
}
if f != nil {
t.Errorf("File did not return a nil file on invalid UpdateOption")
}
}
func TestServerGroupOptions(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
u := &UpdateConfig{"group": "master", "server": "http://foo.com"}
want := `
GROUP=master
SERVER=http://foo.com`
f, err := u.File(dir)
if err != nil {
t.Errorf("unexpected error getting file from UpdateConfig: %v", err)
} else if f == nil {
t.Error("unexpectedly got empty file from UpdateConfig")
} else {
out := strings.Split(f.Content, "\n")
sort.Strings(out)
got := strings.Join(out, "\n")
if got != want {
t.Errorf("File has incorrect contents, got %v, want %v", got, want)
}
}
uu, err := u.Units(dir)
if err != nil {
t.Errorf("unexpected error getting units from UpdateConfig: %v", err)
} else if len(uu) != 1 {
t.Errorf("unexpected number of files returned from UpdateConfig: want 1, got %d", len(uu))
} else {
unit := uu[0]
if unit.Name != "update-engine.service" {
t.Errorf("bad name for generated unit: want update-engine.service, got %s", unit.Name)
}
if unit.Command != "restart" {
t.Errorf("bad command for generated unit: want restart, got %s", unit.Command)
}
}
}
func TestRebootStrategies(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
strategies := []struct {
name string
line string
uMask bool
uCommand string
}{
{"best-effort", "REBOOT_STRATEGY=best-effort", false, "restart"},
{"etcd-lock", "REBOOT_STRATEGY=etcd-lock", false, "restart"},
{"reboot", "REBOOT_STRATEGY=reboot", false, "restart"},
{"off", "REBOOT_STRATEGY=off", true, "stop"},
}
for _, s := range strategies {
uc := &UpdateConfig{"reboot-strategy": s.name}
f, err := uc.File(dir)
if err != nil {
t.Errorf("update failed to generate file for reboot-strategy=%v: %v", s.name, err)
} else if f == nil {
t.Errorf("generated empty file for reboot-strategy=%v", s.name)
} else {
seen := false
for _, line := range strings.Split(f.Content, "\n") {
if line == s.line {
seen = true
break
}
}
if !seen {
t.Errorf("couldn't find expected line %v for reboot-strategy=%v", s.line)
}
}
uu, err := uc.Units(dir)
if err != nil {
t.Errorf("failed to generate unit for reboot-strategy=%v!", s.name)
} else if len(uu) != 1 {
t.Errorf("unexpected number of units for reboot-strategy=%v: %d", s.name, len(uu))
} else {
u := uu[0]
if u.Name != locksmithUnit {
t.Errorf("unit generated for reboot strategy=%v had bad name: %v", s.name, u.Name)
}
if u.Mask != s.uMask {
t.Errorf("unit generated for reboot strategy=%v had bad mask: %t", s.name, u.Mask)
}
if u.Command != s.uCommand {
t.Errorf("unit generated for reboot strategy=%v had bad command: %v", s.name, u.Command)
}
}
}
}
func TestUpdateConfWrittenToDisk(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
for i := 0; i < 2; i++ {
if i == 1 {
err = ioutil.WriteFile(path.Join(dir, "etc", "coreos", "update.conf"), []byte(configured), 0644)
if err != nil {
t.Fatal(err)
}
}
uc := &UpdateConfig{"reboot-strategy": "etcd-lock"}
f, err := uc.File(dir)
if err != nil {
t.Fatalf("Processing UpdateConfig failed: %v", err)
} else if f == nil {
t.Fatal("Unexpectedly got nil updateconfig file")
}
if _, err := system.WriteFile(f, dir); err != nil {
t.Fatalf("Error writing update config: %v", err)
}
fullPath := path.Join(dir, "etc", "coreos", "update.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expected {
t.Fatalf("File has incorrect contents, got %v, wanted %v", string(contents), expected)
}
}
}

View File

@@ -5,6 +5,7 @@ import (
"log"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
@@ -16,7 +17,7 @@ func ParseUserData(contents string) (interface{}, error) {
// Explicitly trim the header so we can handle user-data from
// non-unix operating systems. The rest of the file is parsed
// by goyaml, which correctly handles CRLF.
// by yaml, which correctly handles CRLF.
header = strings.TrimSpace(header)
if strings.HasPrefix(header, "#!") {
@@ -24,7 +25,7 @@ func ParseUserData(contents string) (interface{}, error) {
return system.Script(contents), nil
} else if header == "#cloud-config" {
log.Printf("Parsing user-data as cloud-config")
return NewCloudConfig(contents)
return config.NewCloudConfig(contents)
} else {
return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
}

View File

@@ -2,6 +2,8 @@ package initialize
import (
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestParseHeaderCRLF(t *testing.T) {
@@ -37,7 +39,7 @@ func TestParseConfigCRLF(t *testing.T) {
t.Fatalf("Failed parsing config: %v", err)
}
cfg := ud.(*CloudConfig)
cfg := ud.(*config.CloudConfig)
if cfg.Hostname != "foo" {
t.Error("Failed parsing hostname from config")

View File

@@ -5,6 +5,7 @@ import (
"path"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
@@ -31,21 +32,21 @@ func PersistScriptInWorkspace(script system.Script, workspace string) (string, e
relpath := strings.TrimPrefix(tmp.Name(), workspace)
file := system.File{
file := system.File{config.File{
Path: relpath,
RawFilePermissions: "0744",
Content: string(script),
}
}}
return system.WriteFile(&file, workspace)
}
func PersistUnitNameInWorkspace(name string, workspace string) error {
file := system.File{
file := system.File{config.File{
Path: path.Join("scripts", "unit-name"),
RawFilePermissions: "0644",
Content: name,
}
}}
_, err := system.WriteFile(&file, workspace)
return err
}

View File

@@ -236,7 +236,11 @@ func parseInterfaceStanza(attributes []string, options []string) (*stanzaInterfa
for i, field := range fields[:len(fields)-1] {
switch field {
case "-net":
route.destination.IP = net.ParseIP(fields[i+1])
if _, dst, err := net.ParseCIDR(fields[i+1]); err == nil {
route.destination = *dst
} else {
route.destination.IP = net.ParseIP(fields[i+1])
}
case "netmask":
route.destination.Mask = net.IPMask(net.ParseIP(fields[i+1]).To4())
case "gw":

View File

@@ -285,31 +285,57 @@ func TestBadParseInterfaceStanzasStaticPostUp(t *testing.T) {
}
func TestParseInterfaceStanzaStaticPostUp(t *testing.T) {
options := []string{
"address 192.168.1.100",
"netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0 netmask 255.255.255.0",
}
expect := []route{
for _, tt := range []struct {
options []string
expect []route
}{
{
destination: net.IPNet{
IP: net.IPv4(192, 168, 1, 0),
Mask: net.IPv4Mask(255, 255, 255, 0),
options: []string{
"address 192.168.1.100",
"netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0 netmask 255.255.255.0",
},
expect: []route{
{
destination: net.IPNet{
IP: net.IPv4(192, 168, 1, 0),
Mask: net.IPv4Mask(255, 255, 255, 0),
},
gateway: net.IPv4(192, 168, 1, 1),
},
},
gateway: net.IPv4(192, 168, 1, 1),
},
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
if err != nil {
t.FailNow()
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.FailNow()
}
if !reflect.DeepEqual(static.routes, expect) {
t.FailNow()
{
options: []string{
"address 192.168.1.100",
"netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0/24 || true",
},
expect: []route{
{
destination: func() net.IPNet {
if _, net, err := net.ParseCIDR("192.168.1.0/24"); err == nil {
return *net
} else {
panic(err)
}
}(),
gateway: net.IPv4(192, 168, 1, 1),
},
},
},
} {
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, tt.options)
if err != nil {
t.Fatalf("bad error (%+v): want nil, got %s\n", tt, err)
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.Fatalf("bad config method (%+v): want configMethodStatic, got %T\n", tt, iface.configMethod)
}
if !reflect.DeepEqual(static.routes, tt.expect) {
t.Fatalf("bad routes (%+v): want %#v, got %#v\n", tt, tt.expect, static.routes)
}
}
}

26
system/env.go Normal file
View File

@@ -0,0 +1,26 @@
package system
import (
"fmt"
"reflect"
)
// dropinContents generates the contents for a drop-in unit given the config.
// The argument must be a struct from the 'config' package.
func dropinContents(e interface{}) string {
et := reflect.TypeOf(e)
ev := reflect.ValueOf(e)
var out string
for i := 0; i < et.NumField(); i++ {
if val := ev.Field(i).String(); val != "" {
key := et.Field(i).Tag.Get("env")
out += fmt.Sprintf("Environment=\"%s=%s\"\n", key, val)
}
}
if out == "" {
return ""
}
return "[Service]\n" + out
}

View File

@@ -7,6 +7,8 @@ import (
"strings"
"syscall"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
const (
@@ -48,9 +50,9 @@ func TestWriteEnvFileUpdate(t *testing.T) {
}
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueUpdate,
}
@@ -95,9 +97,9 @@ func TestWriteEnvFileUpdateNoNewline(t *testing.T) {
}
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueUpdate,
}
@@ -136,9 +138,9 @@ func TestWriteEnvFileCreate(t *testing.T) {
fullPath := path.Join(dir, name)
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueUpdate,
}
@@ -174,9 +176,9 @@ func TestWriteEnvFileNoop(t *testing.T) {
}
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueNoop,
}
@@ -221,9 +223,9 @@ func TestWriteEnvFileUpdateDos(t *testing.T) {
}
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueUpdate,
}
@@ -270,9 +272,9 @@ func TestWriteEnvFileDos2Unix(t *testing.T) {
}
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueNoop,
}
@@ -318,9 +320,9 @@ func TestWriteEnvFileEmpty(t *testing.T) {
}
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueEmpty,
}
@@ -360,9 +362,9 @@ func TestWriteEnvFileEmptyNoCreate(t *testing.T) {
fullPath := path.Join(dir, name)
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueEmpty,
}
@@ -391,9 +393,9 @@ func TestWriteEnvFilePermFailure(t *testing.T) {
ioutil.WriteFile(fullPath, []byte(base), 0000)
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueUpdate,
}
@@ -413,9 +415,9 @@ func TestWriteEnvFileNameFailure(t *testing.T) {
name := "foo.conf"
ef := EnvFile{
File: &File{
File: &File{config.File{
Path: name,
},
}},
Vars: valueInvalid,
}

View File

@@ -1,4 +1,4 @@
package initialize
package system
import (
"errors"
@@ -6,15 +6,17 @@ import (
"os"
"path"
"github.com/coreos/coreos-cloudinit/system"
"github.com/coreos/coreos-cloudinit/config"
)
const DefaultIpv4Address = "127.0.0.1"
type EtcHosts string
type EtcHosts struct {
Config config.EtcHosts
}
func (eh EtcHosts) generateEtcHosts() (out string, err error) {
if eh != "localhost" {
if eh.Config != "localhost" {
return "", errors.New("Invalid option to manage_etc_hosts")
}
@@ -28,8 +30,8 @@ func (eh EtcHosts) generateEtcHosts() (out string, err error) {
}
func (eh EtcHosts) File(root string) (*system.File, error) {
if eh == "" {
func (eh EtcHosts) File() (*File, error) {
if eh.Config == "" {
return nil, nil
}
@@ -38,9 +40,9 @@ func (eh EtcHosts) File(root string) (*system.File, error) {
return nil, err
}
return &system.File{
return &File{config.File{
Path: path.Join("etc", "hosts"),
RawFilePermissions: "0644",
Content: etcHosts,
}, nil
}}, nil
}

46
system/etc_hosts_test.go Normal file
View File

@@ -0,0 +1,46 @@
package system
import (
"fmt"
"os"
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestEtcdHostsFile(t *testing.T) {
hostname, err := os.Hostname()
if err != nil {
panic(err)
}
for _, tt := range []struct {
config config.EtcHosts
file *File
err error
}{
{
"invalid",
nil,
fmt.Errorf("Invalid option to manage_etc_hosts"),
},
{
"localhost",
&File{config.File{
Content: fmt.Sprintf("127.0.0.1 %s\n", hostname),
Path: "etc/hosts",
RawFilePermissions: "0644",
}},
nil,
},
} {
file, err := EtcHosts{tt.config}.File()
if !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad error (%q): want %q, got %q", tt.config, tt.err, err)
}
if !reflect.DeepEqual(tt.file, file) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.file, file)
}
}
}

25
system/etcd.go Normal file
View File

@@ -0,0 +1,25 @@
package system
import (
"github.com/coreos/coreos-cloudinit/config"
)
// Etcd is a top-level structure which embeds its underlying configuration,
// config.Etcd, and provides the system-specific Unit().
type Etcd struct {
config.Etcd
}
// Units creates a Unit file drop-in for etcd, using any configured options.
func (ee Etcd) Units() ([]Unit, error) {
content := dropinContents(ee.Etcd)
if content == "" {
return nil, nil
}
return []Unit{{config.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: content,
}}}, nil
}

60
system/etcd_test.go Normal file
View File

@@ -0,0 +1,60 @@
package system
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestEtcdUnits(t *testing.T) {
for _, tt := range []struct {
config config.Etcd
units []Unit
}{
{
config.Etcd{},
nil,
},
{
config.Etcd{
Discovery: "http://disco.example.com/foobar",
PeerBindAddr: "127.0.0.1:7002",
},
[]Unit{{config.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`,
}}},
},
{
config.Etcd{
Name: "node001",
Discovery: "http://disco.example.com/foobar",
PeerBindAddr: "127.0.0.1:7002",
},
[]Unit{{config.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_NAME=node001"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`,
}}},
},
} {
units, err := Etcd{tt.config}.Units()
if err != nil {
t.Errorf("bad error (%q): want %q, got %q", tt.config, nil, err)
}
if !reflect.DeepEqual(tt.units, units) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.units, units)
}
}
}

View File

@@ -8,14 +8,14 @@ import (
"os/exec"
"path"
"strconv"
"github.com/coreos/coreos-cloudinit/config"
)
// File is a top-level structure which embeds its underlying configuration,
// config.File, and provides the system-specific Permissions().
type File struct {
Encoding string
Content string
Owner string
Path string
RawFilePermissions string `yaml:"permissions"`
config.File
}
func (f *File) Permissions() (os.FileMode, error) {

View File

@@ -5,6 +5,8 @@ import (
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestWriteFileUnencodedContent(t *testing.T) {
@@ -17,11 +19,11 @@ func TestWriteFileUnencodedContent(t *testing.T) {
fn := "foo"
fullPath := path.Join(dir, fn)
wf := File{
wf := File{config.File{
Path: fn,
Content: "bar",
RawFilePermissions: "0644",
}
}}
path, err := WriteFile(&wf, dir)
if err != nil {
@@ -56,11 +58,11 @@ func TestWriteFileInvalidPermission(t *testing.T) {
}
defer os.RemoveAll(dir)
wf := File{
wf := File{config.File{
Path: path.Join(dir, "tmp", "foo"),
Content: "bar",
RawFilePermissions: "pants",
}
}}
if _, err := WriteFile(&wf, dir); err == nil {
t.Fatalf("Expected error to be raised when writing file with invalid permission")
@@ -77,10 +79,10 @@ func TestWriteFilePermissions(t *testing.T) {
fn := "foo"
fullPath := path.Join(dir, fn)
wf := File{
wf := File{config.File{
Path: fn,
RawFilePermissions: "0755",
}
}}
path, err := WriteFile(&wf, dir)
if err != nil {
@@ -106,11 +108,11 @@ func TestWriteFileEncodedContent(t *testing.T) {
}
defer os.RemoveAll(dir)
wf := File{
wf := File{config.File{
Path: path.Join(dir, "tmp", "foo"),
Content: "",
Encoding: "base64",
}
}}
if _, err := WriteFile(&wf, dir); err == nil {
t.Fatalf("Expected error to be raised when writing file with encoding")

26
system/fleet.go Normal file
View File

@@ -0,0 +1,26 @@
package system
import (
"github.com/coreos/coreos-cloudinit/config"
)
// Fleet is a top-level structure which embeds its underlying configuration,
// config.Fleet, and provides the system-specific Unit().
type Fleet struct {
config.Fleet
}
// Units generates a Unit file drop-in for fleet, if any fleet options were
// configured in cloud-config
func (fe Fleet) Units() ([]Unit, error) {
content := dropinContents(fe.Fleet)
if content == "" {
return nil, nil
}
return []Unit{{config.Unit{
Name: "fleet.service",
Runtime: true,
DropIn: true,
Content: content,
}}}, nil
}

41
system/fleet_test.go Normal file
View File

@@ -0,0 +1,41 @@
package system
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestFleetUnits(t *testing.T) {
for _, tt := range []struct {
config config.Fleet
units []Unit
}{
{
config.Fleet{},
nil,
},
{
config.Fleet{
PublicIP: "12.34.56.78",
},
[]Unit{{config.Unit{
Name: "fleet.service",
Content: `[Service]
Environment="FLEET_PUBLIC_IP=12.34.56.78"
`,
Runtime: true,
DropIn: true,
}}},
},
} {
units, err := Fleet{tt.config}.Units()
if err != nil {
t.Errorf("bad error (%q): want %q, got %q", tt.config, nil, err)
}
if !reflect.DeepEqual(units, tt.units) {
t.Errorf("bad units (%q): want %q, got %q", tt.config, tt.units, units)
}
}
}

View File

@@ -8,6 +8,7 @@ import (
"strings"
"time"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/network"
"github.com/coreos/coreos-cloudinit/third_party/github.com/dotcloud/docker/pkg/netlink"
)
@@ -108,11 +109,11 @@ func WriteNetworkdConfigs(interfaces []network.InterfaceGenerator) error {
return nil
}
func writeConfig(filename string, config string) error {
if config == "" {
func writeConfig(filename string, content string) error {
if content == "" {
return nil
}
log.Printf("Writing networkd unit %q\n", filename)
_, err := WriteFile(&File{Content: config, Path: filename}, runtimeNetworkPath)
_, err := WriteFile(&File{config.File{Content: content, Path: filename}}, runtimeNetworkPath)
return err
}

32
system/oem.go Normal file
View File

@@ -0,0 +1,32 @@
package system
import (
"fmt"
"path"
"github.com/coreos/coreos-cloudinit/config"
)
// OEM is a top-level structure which embeds its underlying configuration,
// config.OEM, and provides the system-specific File().
type OEM struct {
config.OEM
}
func (oem OEM) File() (*File, error) {
if oem.ID == "" {
return nil, nil
}
content := fmt.Sprintf("ID=%s\n", oem.ID)
content += fmt.Sprintf("VERSION_ID=%s\n", oem.VersionID)
content += fmt.Sprintf("NAME=%q\n", oem.Name)
content += fmt.Sprintf("HOME_URL=%q\n", oem.HomeURL)
content += fmt.Sprintf("BUG_REPORT_URL=%q\n", oem.BugReportURL)
return &File{config.File{
Path: path.Join("etc", "oem-release"),
RawFilePermissions: "0644",
Content: content,
}}, nil
}

47
system/oem_test.go Normal file
View File

@@ -0,0 +1,47 @@
package system
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestOEMFile(t *testing.T) {
for _, tt := range []struct {
config config.OEM
file *File
}{
{
config.OEM{},
nil,
},
{
config.OEM{
ID: "rackspace",
Name: "Rackspace Cloud Servers",
VersionID: "168.0.0",
HomeURL: "https://www.rackspace.com/cloud/servers/",
BugReportURL: "https://github.com/coreos/coreos-overlay",
},
&File{config.File{
Path: "etc/oem-release",
RawFilePermissions: "0644",
Content: `ID=rackspace
VERSION_ID=168.0.0
NAME="Rackspace Cloud Servers"
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
`,
}},
},
} {
file, err := OEM{tt.config}.File()
if err != nil {
t.Errorf("bad error (%q): want %q, got %q", tt.config, nil, err)
}
if !reflect.DeepEqual(tt.file, file) {
t.Errorf("bad file (%q): want %#v, got %#v", tt.config, tt.file, file)
}
}
}

View File

@@ -10,6 +10,7 @@ import (
"path/filepath"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
)
@@ -35,11 +36,11 @@ func (s *systemd) PlaceUnit(u *Unit, dst string) error {
}
}
file := File{
file := File{config.File{
Path: filepath.Base(dst),
Content: u.Content,
RawFilePermissions: "0644",
}
}}
_, err := WriteFile(&file, dir)
if err != nil {

View File

@@ -5,10 +5,12 @@ import (
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestPlaceNetworkUnit(t *testing.T) {
u := Unit{
u := Unit{config.Unit{
Name: "50-eth0.network",
Runtime: true,
Content: `[Match]
@@ -17,7 +19,7 @@ Name=eth47
[Network]
Address=10.209.171.177/19
`,
}
}}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
@@ -66,10 +68,10 @@ func TestUnitDestination(t *testing.T) {
dir := "/some/dir"
name := "foobar.service"
u := Unit{
u := Unit{config.Unit{
Name: name,
DropIn: false,
}
}}
dst := u.Destination(dir)
expectDst := path.Join(dir, "etc", "systemd", "system", "foobar.service")
@@ -87,14 +89,14 @@ func TestUnitDestination(t *testing.T) {
}
func TestPlaceMountUnit(t *testing.T) {
u := Unit{
u := Unit{config.Unit{
Name: "media-state.mount",
Runtime: false,
Content: `[Mount]
What=/dev/sdb1
Where=/media/state
`,
}
}}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
@@ -162,7 +164,7 @@ func TestMaskUnit(t *testing.T) {
sd := &systemd{dir}
// Ensure mask works with units that do not currently exist
uf := &Unit{Name: "foo.service"}
uf := &Unit{config.Unit{Name: "foo.service"}}
if err := sd.MaskUnit(uf); err != nil {
t.Fatalf("Unable to mask new unit: %v", err)
}
@@ -176,7 +178,7 @@ func TestMaskUnit(t *testing.T) {
}
// Ensure mask works with unit files that already exist
ub := &Unit{Name: "bar.service"}
ub := &Unit{config.Unit{Name: "bar.service"}}
barPath := path.Join(dir, "etc", "systemd", "system", "bar.service")
if _, err := os.Create(barPath); err != nil {
t.Fatalf("Error creating new unit file: %v", err)
@@ -202,12 +204,12 @@ func TestUnmaskUnit(t *testing.T) {
sd := &systemd{dir}
nilUnit := &Unit{Name: "null.service"}
nilUnit := &Unit{config.Unit{Name: "null.service"}}
if err := sd.UnmaskUnit(nilUnit); err != nil {
t.Errorf("unexpected error from unmasking nonexistent unit: %v", err)
}
uf := &Unit{Name: "foo.service", Content: "[Service]\nExecStart=/bin/true"}
uf := &Unit{config.Unit{Name: "foo.service", Content: "[Service]\nExecStart=/bin/true"}}
dst := uf.Destination(dir)
if err := os.MkdirAll(path.Dir(dst), os.FileMode(0755)); err != nil {
t.Fatalf("Unable to create unit directory: %v", err)
@@ -227,7 +229,7 @@ func TestUnmaskUnit(t *testing.T) {
t.Errorf("unmask of non-empty unit mutated unit contents unexpectedly")
}
ub := &Unit{Name: "bar.service"}
ub := &Unit{config.Unit{Name: "bar.service"}}
dst = ub.Destination(dir)
if err := os.Symlink("/dev/null", dst); err != nil {
t.Fatalf("Unable to create masked unit: %v", err)

View File

@@ -3,8 +3,8 @@ package system
import (
"fmt"
"path"
"path/filepath"
"strings"
"github.com/coreos/coreos-cloudinit/config"
)
// Name for drop-in service configuration files created by cloudconfig
@@ -19,33 +19,10 @@ type UnitManager interface {
UnmaskUnit(unit *Unit) error
}
// Unit is a top-level structure which embeds its underlying configuration,
// config.Unit, and provides the system-specific Destination().
type Unit struct {
Name string
Mask bool
Enable bool
Runtime bool
Content string
Command string
// For drop-in units, a cloudinit.conf is generated.
// This is currently unbound in YAML (and hence unsettable in cloud-config files)
// until the correct behaviour for multiple drop-in units is determined.
DropIn bool `yaml:"-"`
}
func (u *Unit) Type() string {
ext := filepath.Ext(u.Name)
return strings.TrimLeft(ext, ".")
}
func (u *Unit) Group() (group string) {
t := u.Type()
if t == "network" || t == "netdev" || t == "link" {
group = "network"
} else {
group = "system"
}
return
config.Unit
}
type Script []byte

137
system/update.go Normal file
View File

@@ -0,0 +1,137 @@
package system
import (
"bufio"
"fmt"
"io"
"os"
"path"
"reflect"
"sort"
"strings"
"github.com/coreos/coreos-cloudinit/config"
)
const (
locksmithUnit = "locksmithd.service"
updateEngineUnit = "update-engine.service"
)
// Update is a top-level structure which contains its underlying configuration,
// config.Update, a function for reading the configuration (the default
// implementation reading from the filesystem), and provides the system-specific
// File() and Unit().
type Update struct {
Config config.Update
ReadConfig func() (io.Reader, error)
}
func DefaultReadConfig() (io.Reader, error) {
etcUpdate := path.Join("/etc", "coreos", "update.conf")
usrUpdate := path.Join("/usr", "share", "coreos", "update.conf")
f, err := os.Open(etcUpdate)
if os.IsNotExist(err) {
f, err = os.Open(usrUpdate)
}
return f, err
}
// File generates an `/etc/coreos/update.conf` file (if any update
// configuration options are set in cloud-config) by either rewriting the
// existing file on disk, or starting from `/usr/share/coreos/update.conf`
func (uc Update) File() (*File, error) {
if config.IsZero(uc.Config) {
return nil, nil
}
if err := config.AssertValid(uc.Config); err != nil {
return nil, err
}
// Generate the list of possible substitutions to be performed based on the options that are configured
subs := map[string]string{}
uct := reflect.TypeOf(uc.Config)
ucv := reflect.ValueOf(uc.Config)
for i := 0; i < uct.NumField(); i++ {
val := ucv.Field(i).String()
if val == "" {
continue
}
env := uct.Field(i).Tag.Get("env")
subs[env] = fmt.Sprintf("%s=%s", env, val)
}
conf, err := uc.ReadConfig()
if err != nil {
return nil, err
}
scanner := bufio.NewScanner(conf)
var out string
for scanner.Scan() {
line := scanner.Text()
for env, value := range subs {
if strings.HasPrefix(line, env) {
line = value
delete(subs, env)
break
}
}
out += line
out += "\n"
if err := scanner.Err(); err != nil {
return nil, err
}
}
for _, key := range sortedKeys(subs) {
out += subs[key]
out += "\n"
}
return &File{config.File{
Path: path.Join("etc", "coreos", "update.conf"),
RawFilePermissions: "0644",
Content: out,
}}, nil
}
// Units generates units for the cloud-init initializer to act on:
// - a locksmith Unit, if "reboot-strategy" was set in cloud-config
// - an update_engine Unit, if "group" or "server" was set in cloud-config
func (uc Update) Units() ([]Unit, error) {
var units []Unit
if uc.Config.RebootStrategy != "" {
ls := &Unit{config.Unit{
Name: locksmithUnit,
Command: "restart",
Mask: false,
Runtime: true,
}}
if uc.Config.RebootStrategy == "off" {
ls.Command = "stop"
ls.Mask = true
}
units = append(units, *ls)
}
if uc.Config.Group != "" || uc.Config.Server != "" {
ue := Unit{config.Unit{
Name: updateEngineUnit,
Command: "restart",
}}
units = append(units, ue)
}
return units, nil
}
func sortedKeys(m map[string]string) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}

151
system/update_test.go Normal file
View File

@@ -0,0 +1,151 @@
package system
import (
"errors"
"io"
"reflect"
"strings"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func testReadConfig(config string) func() (io.Reader, error) {
return func() (io.Reader, error) {
return strings.NewReader(config), nil
}
}
func TestUpdateUnits(t *testing.T) {
for _, tt := range []struct {
config config.Update
units []Unit
err error
}{
{
config: config.Update{},
},
{
config: config.Update{Group: "master", Server: "http://foo.com"},
units: []Unit{{config.Unit{
Name: "update-engine.service",
Command: "restart",
}}},
},
{
config: config.Update{RebootStrategy: "best-effort"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "restart",
Runtime: true,
}}},
},
{
config: config.Update{RebootStrategy: "etcd-lock"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "restart",
Runtime: true,
}}},
},
{
config: config.Update{RebootStrategy: "reboot"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "restart",
Runtime: true,
}}},
},
{
config: config.Update{RebootStrategy: "off"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "stop",
Runtime: true,
Mask: true,
}}},
},
} {
units, err := Update{tt.config, testReadConfig("")}.Units()
if !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad error (%q): want %q, got %q", tt.config, tt.err, err)
}
if !reflect.DeepEqual(tt.units, units) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.units, units)
}
}
}
func TestUpdateFile(t *testing.T) {
for _, tt := range []struct {
config config.Update
orig string
file *File
err error
}{
{
config: config.Update{},
},
{
config: config.Update{RebootStrategy: "wizzlewazzle"},
err: errors.New("invalid value \"wizzlewazzle\" for option \"RebootStrategy\" (valid options: \"best-effort,etcd-lock,reboot,off\")"),
},
{
config: config.Update{Group: "master", Server: "http://foo.com"},
file: &File{config.File{
Content: "GROUP=master\nSERVER=http://foo.com\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "best-effort"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=best-effort\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "etcd-lock"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=etcd-lock\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "reboot"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=reboot\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "off"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=off\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "etcd-lock"},
orig: "SERVER=https://example.com\nGROUP=thegroupc\nREBOOT_STRATEGY=awesome",
file: &File{config.File{
Content: "SERVER=https://example.com\nGROUP=thegroupc\nREBOOT_STRATEGY=etcd-lock\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
} {
file, err := Update{tt.config, testReadConfig(tt.orig)}.File()
if !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad error (%q): want %q, got %q", tt.config, tt.err, err)
}
if !reflect.DeepEqual(tt.file, file) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.file, file)
}
}
}

View File

@@ -6,30 +6,16 @@ import (
"os/exec"
"os/user"
"strings"
"github.com/coreos/coreos-cloudinit/config"
)
type User struct {
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
SSHImportGithubUser string `yaml:"coreos-ssh-import-github"`
SSHImportURL string `yaml:"coreos-ssh-import-url"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
}
func UserExists(u *User) bool {
func UserExists(u *config.User) bool {
_, err := user.Lookup(u.Name)
return err == nil
}
func CreateUser(u *User) error {
func CreateUser(u *config.User) error {
args := []string{}
if u.PasswordHash != "" {

29
test
View File

@@ -13,19 +13,22 @@ COVER=${COVER:-"-cover"}
source ./build
declare -a TESTPKGS=(initialize
system
datasource
datasource/configdrive
datasource/file
datasource/metadata
datasource/metadata/cloudsigma
datasource/metadata/digitalocean
datasource/metadata/ec2
datasource/proc_cmdline
datasource/url
pkg
network)
declare -a TESTPKGS=(
config
datasource
datasource/configdrive
datasource/file
datasource/metadata
datasource/metadata/cloudsigma
datasource/metadata/digitalocean
datasource/metadata/ec2
datasource/proc_cmdline
datasource/url
initialize
network
pkg
system
)
if [ -z "$PKG" ]; then
GOFMTPATH="${TESTPKGS[*]} coreos-cloudinit.go"

View File

@@ -1,3 +1,15 @@
The following files were ported to Go from C files of libyaml, and thus
are still covered by their original copyright and license:
apic.go
emitterc.go
parserc.go
readerc.go
scannerc.go
writerc.go
yamlh.go
yamlprivateh.go
Copyright (c) 2006 Kirill Simonov
Permission is hereby granted, free of charge, to any person obtaining a copy of

128
third_party/gopkg.in/yaml.v1/README.md vendored Normal file
View File

@@ -0,0 +1,128 @@
# YAML support for the Go language
Introduction
------------
The yaml package enables Go programs to comfortably encode and decode YAML
values. It was developed within [Canonical](https://www.canonical.com) as
part of the [juju](https://juju.ubuntu.com) project, and is based on a
pure Go port of the well-known [libyaml](http://pyyaml.org/wiki/LibYAML)
C library to parse and generate YAML data quickly and reliably.
Compatibility
-------------
The yaml package is almost compatible with YAML 1.1, including support for
anchors, tags, etc. There are still a few missing bits, such as document
merging, base-60 floats (huh?), and multi-document unmarshalling. These
features are not hard to add, and will be introduced as necessary.
Installation and usage
----------------------
The import path for the package is *gopkg.in/yaml.v1*.
To install it, run:
go get gopkg.in/yaml.v1
API documentation
-----------------
If opened in a browser, the import path itself leads to the API documentation:
* [https://gopkg.in/yaml.v1](https://gopkg.in/yaml.v1)
API stability
-------------
The package API for yaml v1 will remain stable as described in [gopkg.in](https://gopkg.in).
License
-------
The yaml package is licensed under the LGPL with an exception that allows it to be linked statically. Please see the LICENSE file for details.
Example
-------
```Go
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v1"
)
var data = `
a: Easy!
b:
c: 2
d: [3, 4]
`
type T struct {
A string
B struct{C int; D []int ",flow"}
}
func main() {
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
m := make(map[interface{}]interface{})
err = yaml.Unmarshal([]byte(data), &m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m:\n%v\n\n", m)
d, err = yaml.Marshal(&m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m dump:\n%s\n\n", string(d))
}
```
This example will generate the following output:
```
--- t:
{Easy! {2 [3 4]}}
--- t dump:
a: Easy!
b:
c: 2
d: [3, 4]
--- m:
map[a:Easy! b:map[c:2 d:[3 4]]]
--- m dump:
a: Easy!
b:
c: 2
d:
- 3
- 4
```

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"io"

View File

@@ -1,8 +1,9 @@
package goyaml
package yaml
import (
"reflect"
"strconv"
"time"
)
const (
@@ -211,6 +212,16 @@ func newDecoder() *decoder {
// returned to call SetYAML() with the value of *out once it's defined.
//
func (d *decoder) setter(tag string, out *reflect.Value, good *bool) (set func()) {
if (*out).Kind() != reflect.Ptr && (*out).CanAddr() {
setter, _ := (*out).Addr().Interface().(Setter)
if setter != nil {
var arg interface{}
*out = reflect.ValueOf(&arg).Elem()
return func() {
*good = setter.SetYAML(tag, arg)
}
}
}
again := true
for again {
again = false
@@ -279,16 +290,19 @@ func (d *decoder) alias(n *node, out reflect.Value) (good bool) {
return good
}
var durationType = reflect.TypeOf(time.Duration(0))
func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
var tag string
var resolved interface{}
if n.tag == "" && !n.implicit {
tag = "!!str"
resolved = n.value
} else {
tag, resolved = resolve(n.tag, n.value)
if set := d.setter(tag, &out, &good); set != nil {
defer set()
}
}
if set := d.setter(tag, &out, &good); set != nil {
defer set()
}
switch out.Kind() {
case reflect.String:
@@ -320,6 +334,14 @@ func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
out.SetInt(int64(resolved))
good = true
}
case string:
if out.Type() == durationType {
d, err := time.ParseDuration(resolved)
if err == nil {
out.SetInt(int64(d))
good = true
}
}
}
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
switch resolved := resolved.(type) {
@@ -437,6 +459,10 @@ func (d *decoder) mapping(n *node, out reflect.Value) (good bool) {
}
l := len(n.children)
for i := 0; i < l; i += 2 {
if isMerge(n.children[i]) {
d.merge(n.children[i+1], out)
continue
}
k := reflect.New(kt).Elem()
if d.unmarshal(n.children[i], k) {
e := reflect.New(et).Elem()
@@ -456,7 +482,12 @@ func (d *decoder) mappingStruct(n *node, out reflect.Value) (good bool) {
name := settableValueOf("")
l := len(n.children)
for i := 0; i < l; i += 2 {
if !d.unmarshal(n.children[i], name) {
ni := n.children[i]
if isMerge(ni) {
d.merge(n.children[i+1], out)
continue
}
if !d.unmarshal(ni, name) {
continue
}
if info, ok := sinfo.FieldsMap[name.String()]; ok {
@@ -471,3 +502,37 @@ func (d *decoder) mappingStruct(n *node, out reflect.Value) (good bool) {
}
return true
}
func (d *decoder) merge(n *node, out reflect.Value) {
const wantMap = "map merge requires map or sequence of maps as the value"
switch n.kind {
case mappingNode:
d.unmarshal(n, out)
case aliasNode:
an, ok := d.doc.anchors[n.value]
if ok && an.kind != mappingNode {
panic(wantMap)
}
d.unmarshal(n, out)
case sequenceNode:
// Step backwards as earlier nodes take precedence.
for i := len(n.children)-1; i >= 0; i-- {
ni := n.children[i]
if ni.kind == aliasNode {
an, ok := d.doc.anchors[ni.value]
if ok && an.kind != mappingNode {
panic(wantMap)
}
} else if ni.kind != mappingNode {
panic(wantMap)
}
d.unmarshal(ni, out)
}
default:
panic(wantMap)
}
}
func isMerge(n *node) bool {
return n.kind == scalarNode && n.value == "<<" && (n.implicit == true || n.tag == "!!merge" || n.tag == "tag:yaml.org,2002:merge")
}

View File

@@ -1,10 +1,11 @@
package goyaml_test
package yaml_test
import (
. "launchpad.net/gocheck"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
. "gopkg.in/check.v1"
"gopkg.in/yaml.v1"
"math"
"reflect"
"time"
)
var unmarshalIntTest = 123
@@ -350,6 +351,32 @@ var unmarshalTests = []struct {
C inlineB `yaml:",inline"`
}{1, inlineB{2, inlineC{3}}},
},
// bug 1243827
{
"a: -b_c",
map[string]interface{}{"a": "-b_c"},
},
{
"a: +b_c",
map[string]interface{}{"a": "+b_c"},
},
{
"a: 50cent_of_dollar",
map[string]interface{}{"a": "50cent_of_dollar"},
},
// Duration
{
"a: 3s",
map[string]time.Duration{"a": 3 * time.Second},
},
// Issue #24.
{
"a: <foo>",
map[string]string{"a": "<foo>"},
},
}
type inlineB struct {
@@ -377,7 +404,7 @@ func (s *S) TestUnmarshal(c *C) {
pv := reflect.New(pt.Elem())
value = pv.Interface()
}
err := goyaml.Unmarshal([]byte(item.data), value)
err := yaml.Unmarshal([]byte(item.data), value)
c.Assert(err, IsNil, Commentf("Item #%d", i))
if t.Kind() == reflect.String {
c.Assert(*value.(*string), Equals, item.value, Commentf("Item #%d", i))
@@ -389,7 +416,7 @@ func (s *S) TestUnmarshal(c *C) {
func (s *S) TestUnmarshalNaN(c *C) {
value := map[string]interface{}{}
err := goyaml.Unmarshal([]byte("notanum: .NaN"), &value)
err := yaml.Unmarshal([]byte("notanum: .NaN"), &value)
c.Assert(err, IsNil)
c.Assert(math.IsNaN(value["notanum"].(float64)), Equals, true)
}
@@ -408,7 +435,7 @@ var unmarshalErrorTests = []struct {
func (s *S) TestUnmarshalErrors(c *C) {
for _, item := range unmarshalErrorTests {
var value interface{}
err := goyaml.Unmarshal([]byte(item.data), &value)
err := yaml.Unmarshal([]byte(item.data), &value)
c.Assert(err, ErrorMatches, item.error, Commentf("Partial unmarshal: %#v", value))
}
}
@@ -421,6 +448,8 @@ var setterTests = []struct {
{"_: [1,A]", "!!seq", []interface{}{1, "A"}},
{"_: 10", "!!int", 10},
{"_: null", "!!null", nil},
{`_: BAR!`, "!!str", "BAR!"},
{`_: "BAR!"`, "!!str", "BAR!"},
{"_: !!foo 'BAR!'", "!!foo", "BAR!"},
}
@@ -442,17 +471,31 @@ func (o *typeWithSetter) SetYAML(tag string, value interface{}) (ok bool) {
return true
}
type typeWithSetterField struct {
type setterPointerType struct {
Field *typeWithSetter "_"
}
func (s *S) TestUnmarshalWithSetter(c *C) {
type setterValueType struct {
Field typeWithSetter "_"
}
func (s *S) TestUnmarshalWithPointerSetter(c *C) {
for _, item := range setterTests {
obj := &typeWithSetterField{}
err := goyaml.Unmarshal([]byte(item.data), obj)
obj := &setterPointerType{}
err := yaml.Unmarshal([]byte(item.data), obj)
c.Assert(err, IsNil)
c.Assert(obj.Field, NotNil,
Commentf("Pointer not initialized (%#v)", item.value))
c.Assert(obj.Field, NotNil, Commentf("Pointer not initialized (%#v)", item.value))
c.Assert(obj.Field.tag, Equals, item.tag)
c.Assert(obj.Field.value, DeepEquals, item.value)
}
}
func (s *S) TestUnmarshalWithValueSetter(c *C) {
for _, item := range setterTests {
obj := &setterValueType{}
err := yaml.Unmarshal([]byte(item.data), obj)
c.Assert(err, IsNil)
c.Assert(obj.Field, NotNil, Commentf("Pointer not initialized (%#v)", item.value))
c.Assert(obj.Field.tag, Equals, item.tag)
c.Assert(obj.Field.value, DeepEquals, item.value)
}
@@ -460,7 +503,7 @@ func (s *S) TestUnmarshalWithSetter(c *C) {
func (s *S) TestUnmarshalWholeDocumentWithSetter(c *C) {
obj := &typeWithSetter{}
err := goyaml.Unmarshal([]byte(setterTests[0].data), obj)
err := yaml.Unmarshal([]byte(setterTests[0].data), obj)
c.Assert(err, IsNil)
c.Assert(obj.tag, Equals, setterTests[0].tag)
value, ok := obj.value.(map[interface{}]interface{})
@@ -477,8 +520,8 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
}()
m := map[string]*typeWithSetter{}
data := "{abc: 1, def: 2, ghi: 3, jkl: 4}"
err := goyaml.Unmarshal([]byte(data), m)
data := `{abc: 1, def: 2, ghi: 3, jkl: 4}`
err := yaml.Unmarshal([]byte(data), m)
c.Assert(err, IsNil)
c.Assert(m["abc"], NotNil)
c.Assert(m["def"], IsNil)
@@ -489,6 +532,98 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
c.Assert(m["ghi"].value, Equals, 3)
}
// From http://yaml.org/type/merge.html
var mergeTests = `
anchors:
- &CENTER { "x": 1, "y": 2 }
- &LEFT { "x": 0, "y": 2 }
- &BIG { "r": 10 }
- &SMALL { "r": 1 }
# All the following maps are equal:
plain:
# Explicit keys
"x": 1
"y": 2
"r": 10
label: center/big
mergeOne:
# Merge one map
<< : *CENTER
"r": 10
label: center/big
mergeMultiple:
# Merge multiple maps
<< : [ *CENTER, *BIG ]
label: center/big
override:
# Override
<< : [ *BIG, *LEFT, *SMALL ]
"x": 1
label: center/big
shortTag:
# Explicit short merge tag
!!merge "<<" : [ *CENTER, *BIG ]
label: center/big
longTag:
# Explicit merge long tag
!<tag:yaml.org,2002:merge> "<<" : [ *CENTER, *BIG ]
label: center/big
inlineMap:
# Inlined map
<< : {"x": 1, "y": 2, "r": 10}
label: center/big
inlineSequenceMap:
# Inlined map in sequence
<< : [ *CENTER, {"r": 10} ]
label: center/big
`
func (s *S) TestMerge(c *C) {
var want = map[interface{}]interface{}{
"x": 1,
"y": 2,
"r": 10,
"label": "center/big",
}
var m map[string]interface{}
err := yaml.Unmarshal([]byte(mergeTests), &m)
c.Assert(err, IsNil)
for name, test := range m {
if name == "anchors" {
continue
}
c.Assert(test, DeepEquals, want, Commentf("test %q failed", name))
}
}
func (s *S) TestMergeStruct(c *C) {
type Data struct {
X, Y, R int
Label string
}
want := Data{1, 2, 10, "center/big"}
var m map[string]Data
err := yaml.Unmarshal([]byte(mergeTests), &m)
c.Assert(err, IsNil)
for name, test := range m {
if name == "anchors" {
continue
}
c.Assert(test, Equals, want, Commentf("test %q failed", name))
}
}
//var data []byte
//func init() {
// var err error
@@ -502,7 +637,7 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
// var err error
// for i := 0; i < c.N; i++ {
// var v map[string]interface{}
// err = goyaml.Unmarshal(data, &v)
// err = yaml.Unmarshal(data, &v)
// }
// if err != nil {
// panic(err)
@@ -511,9 +646,9 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
//
//func (s *S) BenchmarkMarshal(c *C) {
// var v map[string]interface{}
// goyaml.Unmarshal(data, &v)
// yaml.Unmarshal(data, &v)
// c.ResetTimer()
// for i := 0; i < c.N; i++ {
// goyaml.Marshal(&v)
// yaml.Marshal(&v)
// }
//}

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"bytes"

View File

@@ -1,9 +1,10 @@
package goyaml
package yaml
import (
"reflect"
"sort"
"strconv"
"time"
)
type encoder struct {
@@ -85,7 +86,11 @@ func (e *encoder) marshal(tag string, in reflect.Value) {
case reflect.String:
e.stringv(tag, in)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
e.intv(tag, in)
if in.Type() == durationType {
e.stringv(tag, reflect.ValueOf(in.Interface().(time.Duration).String()))
} else {
e.intv(tag, in)
}
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
e.uintv(tag, in)
case reflect.Float32, reflect.Float64:

View File

@@ -1,12 +1,13 @@
package goyaml_test
package yaml_test
import (
"fmt"
. "launchpad.net/gocheck"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
"gopkg.in/yaml.v1"
. "gopkg.in/check.v1"
"math"
"strconv"
"strings"
"time"
)
var marshalIntTest = 123
@@ -212,11 +213,23 @@ var marshalTests = []struct {
}{1, inlineB{2, inlineC{3}}},
"a: 1\nb: 2\nc: 3\n",
},
// Duration
{
map[string]time.Duration{"a": 3 * time.Second},
"a: 3s\n",
},
// Issue #24.
{
map[string]string{"a": "<foo>"},
"a: <foo>\n",
},
}
func (s *S) TestMarshal(c *C) {
for _, item := range marshalTests {
data, err := goyaml.Marshal(item.value)
data, err := yaml.Marshal(item.value)
c.Assert(err, IsNil)
c.Assert(string(data), Equals, item.data)
}
@@ -237,7 +250,7 @@ var marshalErrorTests = []struct {
func (s *S) TestMarshalErrors(c *C) {
for _, item := range marshalErrorTests {
_, err := goyaml.Marshal(item.value)
_, err := yaml.Marshal(item.value)
c.Assert(err, ErrorMatches, item.error)
}
}
@@ -269,12 +282,12 @@ func (s *S) TestMarshalTypeCache(c *C) {
var err error
func() {
type T struct{ A int }
data, err = goyaml.Marshal(&T{})
data, err = yaml.Marshal(&T{})
c.Assert(err, IsNil)
}()
func() {
type T struct{ B int }
data, err = goyaml.Marshal(&T{})
data, err = yaml.Marshal(&T{})
c.Assert(err, IsNil)
}()
c.Assert(string(data), Equals, "b: 0\n")
@@ -298,7 +311,7 @@ func (s *S) TestMashalWithGetter(c *C) {
obj := &typeWithGetterField{}
obj.Field.tag = item.tag
obj.Field.value = item.value
data, err := goyaml.Marshal(obj)
data, err := yaml.Marshal(obj)
c.Assert(err, IsNil)
c.Assert(string(data), Equals, string(item.data))
}
@@ -308,7 +321,7 @@ func (s *S) TestUnmarshalWholeDocumentWithGetter(c *C) {
obj := &typeWithGetter{}
obj.tag = ""
obj.value = map[string]string{"hello": "world!"}
data, err := goyaml.Marshal(obj)
data, err := yaml.Marshal(obj)
c.Assert(err, IsNil)
c.Assert(string(data), Equals, "hello: world!\n")
}
@@ -356,7 +369,7 @@ func (s *S) TestSortedOutput(c *C) {
for _, k := range order {
m[k] = 1
}
data, err := goyaml.Marshal(m)
data, err := yaml.Marshal(m)
c.Assert(err, IsNil)
out := "\n" + string(data)
last := 0

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"bytes"

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"io"

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"math"
@@ -27,7 +27,6 @@ func init() {
t[int(c)] = 'M' // In map
}
t[int('.')] = '.' // Float (potentially in map)
t[int('<')] = '<' // Merge
var resolveMapList = []struct {
v interface{}
@@ -45,6 +44,7 @@ func init() {
{math.Inf(+1), "!!float", []string{".inf", ".Inf", ".INF"}},
{math.Inf(+1), "!!float", []string{"+.inf", "+.Inf", "+.INF"}},
{math.Inf(-1), "!!float", []string{"-.inf", "-.Inf", "-.INF"}},
{"<<", "!!merge", []string{"<<"}},
}
m := resolveMap
@@ -113,13 +113,8 @@ func resolve(tag string, in string) (rtag string, out interface{}) {
case 'D', 'S':
// Int, float, or timestamp.
for i := 0; i != len(in); i++ {
if in[i] == '_' {
in = strings.Replace(in, "_", "", -1)
break
}
}
intv, err := strconv.ParseInt(in, 0, 64)
plain := strings.Replace(in, "_", "", -1)
intv, err := strconv.ParseInt(plain, 0, 64)
if err == nil {
if intv == int64(int(intv)) {
return "!!int", int(intv)
@@ -127,26 +122,23 @@ func resolve(tag string, in string) (rtag string, out interface{}) {
return "!!int", intv
}
}
floatv, err := strconv.ParseFloat(in, 64)
floatv, err := strconv.ParseFloat(plain, 64)
if err == nil {
return "!!float", floatv
}
if strings.HasPrefix(in, "0b") {
intv, err := strconv.ParseInt(in[2:], 2, 64)
if strings.HasPrefix(plain, "0b") {
intv, err := strconv.ParseInt(plain[2:], 2, 64)
if err == nil {
return "!!int", int(intv)
}
} else if strings.HasPrefix(in, "-0b") {
intv, err := strconv.ParseInt(in[3:], 2, 64)
} else if strings.HasPrefix(plain, "-0b") {
intv, err := strconv.ParseInt(plain[3:], 2, 64)
if err == nil {
return "!!int", -int(intv)
}
}
// XXX Handle timestamps here.
case '<':
// XXX Handle merge (<<) here.
default:
panic("resolveTable item not yet handled: " +
string([]byte{c}) + " (with " + in + ")")

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"bytes"

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"reflect"

View File

@@ -1,7 +1,7 @@
package goyaml_test
package yaml_test
import (
. "launchpad.net/gocheck"
. "gopkg.in/check.v1"
"testing"
)

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
// Set the writer error and return false.
func yaml_emitter_set_writer_error(emitter *yaml_emitter_t, problem string) bool {

View File

@@ -1,5 +1,10 @@
// Package goyaml implements YAML support for the Go language.
package goyaml
// Package yaml implements YAML support for the Go language.
//
// Source code and other details for the project are available at GitHub:
//
// https://github.com/go-yaml/yaml
//
package yaml
import (
"errors"
@@ -28,32 +33,31 @@ func handleErr(err *error) {
}
}
// Objects implementing the goyaml.Setter interface will receive the YAML
// tag and value via the SetYAML method during unmarshaling, rather than
// being implicitly assigned by the goyaml machinery. If setting the value
// works, the method should return true. If it returns false, the given
// value will be omitted from maps and slices.
// The Setter interface may be implemented by types to do their own custom
// unmarshalling of YAML values, rather than being implicitly assigned by
// the yaml package machinery. If setting the value works, the method should
// return true. If it returns false, the value is considered unsupported
// and is omitted from maps and slices.
type Setter interface {
SetYAML(tag string, value interface{}) bool
}
// Objects implementing the goyaml.Getter interface will get the GetYAML()
// method called when goyaml is requested to marshal the given value, and
// the result of this method will be marshaled in place of the actual object.
// The Getter interface is implemented by types to do their own custom
// marshalling into a YAML tag and value.
type Getter interface {
GetYAML() (tag string, value interface{})
}
// Unmarshal decodes the first document found within the in byte slice
// and assigns decoded values into the object pointed by out.
// and assigns decoded values into the out value.
//
// Maps, pointers to structs and ints, etc, may all be used as out values.
// If an internal pointer within a struct is not initialized, goyaml
// will initialize it if necessary for unmarshalling the provided data,
// but the struct provided as out must not be a nil pointer.
// Maps and pointers (to a struct, string, int, etc) are accepted as out
// values. If an internal pointer within a struct is not initialized,
// the yaml package will initialize it if necessary for unmarshalling
// the provided data. The out parameter must not be nil.
//
// The type of the decoded values and the type of out will be considered,
// and Unmarshal() will do the best possible job to unmarshal values
// and Unmarshal will do the best possible job to unmarshal values
// appropriately. It is NOT considered an error, though, to skip values
// because they are not available in the decoded YAML, or if they are not
// compatible with the out value. To ensure something was properly
@@ -61,11 +65,11 @@ type Getter interface {
// field (usually the zero value).
//
// Struct fields are only unmarshalled if they are exported (have an
// upper case first letter), and will be unmarshalled using the field
// name lowercased by default. When custom field names are desired, the
// tag value may be used to tweak the name. Everything before the first
// comma in the field tag will be used as the name. The values following
// the comma are used to tweak the marshalling process (see Marshal).
// upper case first letter), and are unmarshalled using the field name
// lowercased as the default key. Custom keys may be defined via the
// "yaml" name in the field tag: the content preceding the first comma
// is used as the key, and the following comma-separated options are
// used to tweak the marshalling process (see Marshal).
// Conflicting names result in a runtime error.
//
// For example:
@@ -75,7 +79,7 @@ type Getter interface {
// B int
// }
// var T t
// goyaml.Unmarshal([]byte("a: 1\nb: 2"), &t)
// yaml.Unmarshal([]byte("a: 1\nb: 2"), &t)
//
// See the documentation of Marshal for the format of tags and a list of
// supported tag options.
@@ -94,14 +98,16 @@ func Unmarshal(in []byte, out interface{}) (err error) {
// Marshal serializes the value provided into a YAML document. The structure
// of the generated document will reflect the structure of the value itself.
// Maps, pointers to structs and ints, etc, may all be used as the in value.
// Maps and pointers (to struct, string, int, etc) are accepted as the in value.
//
// In the case of struct values, only exported fields will be serialized.
// The lowercased field name is used as the key for each exported field,
// but this behavior may be changed using the respective field tag.
// The tag may also contain flags to tweak the marshalling behavior for
// the field. Conflicting names result in a runtime error. The tag format
// accepted is:
// Struct fields are only unmarshalled if they are exported (have an upper case
// first letter), and are unmarshalled using the field name lowercased as the
// default key. Custom keys may be defined via the "yaml" name in the field
// tag: the content preceding the first comma is used as the key, and the
// following comma-separated options are used to tweak the marshalling process.
// Conflicting names result in a runtime error.
//
// The field tag format accepted is:
//
// `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)`
//
@@ -126,8 +132,8 @@ func Unmarshal(in []byte, out interface{}) (err error) {
// F int "a,omitempty"
// B int
// }
// goyaml.Marshal(&T{B: 2}) // Returns "b: 2\n"
// goyaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n"
// yaml.Marshal(&T{B: 2}) // Returns "b: 2\n"
// yaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n"
//
func Marshal(in interface{}) (out []byte, err error) {
defer handleErr(&err)
@@ -142,7 +148,7 @@ func Marshal(in interface{}) (out []byte, err error) {
// --------------------------------------------------------------------------
// Maintain a mapping of keys to structure field indexes
// The code in this section was copied from gobson.
// The code in this section was copied from mgo/bson.
// structInfo holds details for the serialization of fields of
// a given struct.

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
import (
"io"

View File

@@ -1,4 +1,4 @@
package goyaml
package yaml
const (
// The size of the input raw buffer.

View File

@@ -1,14 +0,0 @@
[568].out
_*
*.cgo*.*
yaml-*/stamp-h1
yaml-*/Makefile
yaml-*/*/Makefile
yaml-*/libtool
yaml-*/config*
yaml-*/*/*.lo
yaml-*/*/*.la
yaml-*/*/.libs
yaml-*/*/.deps
yaml-*/tests/*

View File

@@ -1 +0,0 @@
propose -cr -for=lp:goyaml

View File

@@ -1,20 +0,0 @@
#!/bin/sh
set -e
BADFMT=`find * -name '*.go' | xargs gofmt -l`
if [ -n "$BADFMT" ]; then
BADFMT=`echo "$BADFMT" | sed "s/^/ /"`
echo -e "gofmt is sad:\n\n$BADFMT"
exit 1
fi
VERSION=`go version | awk '{print $3}'`
if [ $VERSION == 'devel' ]; then
go tool vet \
-methods \
-printf \
-rangeloops \
-printfuncs 'ErrorContextf:1,notFoundf:0,badReqErrorf:0,Commitf:0,Snapshotf:0,Debugf:0' \
.
fi

View File

@@ -1,39 +0,0 @@
include $(GOROOT)/src/Make.inc
YAML=yaml-0.1.3
LIBYAML=$(PWD)/$(YAML)/src/.libs/libyaml.a
TARG=launchpad.net/goyaml
GOFILES=\
goyaml.go\
resolve.go\
CGOFILES=\
decode.go\
encode.go\
CGO_OFILES+=\
helpers.o\
api.o\
scanner.o\
reader.o\
parser.o\
writer.o\
emitter.o\
GOFMT=gofmt
BADFMT:=$(shell $(GOFMT) -l $(GOFILES) $(CGOFILES) $(wildcard *_test.go))
all: package
gofmt: $(BADFMT)
@for F in $(BADFMT); do $(GOFMT) -w $$F && echo $$F; done
include $(GOROOT)/src/Make.pkg
ifneq ($(BADFMT),)
ifneq ($(MAKECMDGOALS),gofmt)
$(warning WARNING: make gofmt: $(BADFMT))
endif
endif