Compare commits

...

82 Commits

Author SHA1 Message Date
Alex Crawford
bbe86a216e coreos-cloudinit: bump to 0.10.5 2014-09-28 09:16:49 -07:00
Alex Crawford
effc4cec21 Revert "Merge pull request #234 from crawford/validate"
This reverts commit cdfc94f4e9, reversing
changes made to 2051cd3e1c.
2014-09-28 09:15:25 -07:00
Alex Crawford
75e288c553 coreos-cloudinit: bump to 0.10.4+git 2014-09-24 19:25:55 -07:00
Alex Crawford
0785840fe3 coreos-cloudinit: bump to 0.10.4 2014-09-24 19:25:34 -07:00
Alex Crawford
c10bfc2f56 Merge pull request #240 from epankala/euca4_compat_fix
AWS: Eucalyptus 4.x compatibility fix
2014-09-24 10:55:39 -07:00
Janne Paenkaelae
2f954dcdc2 AWS: Eucalyptus 4.x compatibility fix
For Eucalyptus 4.0.1 requesting metadata seem to work differently as with EC2.

In Euca:
> curl http://169.254.169.254/2009-04-04
<?xml version="1.0"?><Response><Errors><Error><Code>404 Not Found</Code><Message>unknown</Message></Error></Errors><RequestID>unknown</RequestID></Response>core@localhost ~ $

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

In AWS EC2
> curl http://169.254.169.254/2009-04-04
"" (zero bytes)

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

As the isAvailable() function in metadata.go tests only for errorcode
it fails in Euca.
2014-09-24 20:33:29 +03:00
Alex Crawford
cdfc94f4e9 Merge pull request #234 from crawford/validate
config: explicitly specify fields and seperate config and application
2014-09-24 07:42:09 -07:00
Alex Crawford
18e2f98414 cloudconfig: refactor config
- Move CloudConfig into config package
- Add YAML tags to CloudConfig
2014-09-23 17:59:32 -07:00
Alex Crawford
4b472795c4 user: move User into config package
- Add YAML tags for the fields
2014-09-23 17:59:19 -07:00
Alex Crawford
85b8d804c8 file: refactor config
- Seperate the config from Permissions()
- Add YAML tags for the fields
2014-09-23 17:59:16 -07:00
Alex Crawford
1fbbaaec19 unit: refactor config
- Seperate the config from Destination()
- Add YAML tags for the fields
2014-09-23 17:58:32 -07:00
Alex Crawford
667dbd8fb7 update: refactor config
- Explicitly specify all of the valid options for Update
- Seperate the config from File() and Units()
- Add YAML tags for the fields
2014-09-23 17:57:43 -07:00
Alex Crawford
6730cb7227 oem: refactor the config
- Seperate the config from File()
- Add YAML tags for the fields
2014-09-23 16:08:23 -07:00
Alex Crawford
9454522033 fleet: refactor config
- Explicitly specify all of the valid options for fleet
- Seperate the config from Units()
- Add YAML tags for the fields
2014-09-23 16:07:53 -07:00
Alex Crawford
c255739a93 etcd: refactor config
- Explicitly specify all of the valid options for etcd
- Remove the default name generation (ETCD_NAME is set by its unit file now)
- Seperate the etcd config from Units()
- Remove support for DISCOVERY_URL
- Add YAML tags for the fields
2014-09-23 16:07:13 -07:00
Alex Crawford
2051cd3e1c Merge pull request #238 from crawford/docs
docs: fix documentation of coreos.units.command
2014-09-23 11:33:44 -07:00
Alex Crawford
b52cb3fea3 docs: fix documentation of coreos.units.command 2014-09-23 11:32:15 -07:00
Alex Crawford
da5f85b3fb coreos-cloudinit: bump to 0.10.3+git 2014-09-17 12:19:27 -07:00
Alex Crawford
9999178538 coreos-cloudinit: bump to 0.10.3 2014-09-17 12:19:13 -07:00
Alex Crawford
8f766e4666 Merge pull request #235 from crawford/routes
network: add support for CIDR addresses Debian routes
2014-09-17 12:18:16 -07:00
Alex Crawford
2d28d16c92 network: add support for CIDR addresses Debian routes
OnMetal is changing their template from:
`route add -net 1.2.3.0 netmask 255.255.255.0 gw 10.1.2.1 || true`
to:
`route add -net 1.2.3.0/24 gw 10.1.2.1 || true`
2014-09-16 17:36:34 -07:00
Alex Crawford
e9cd09dd7b coreos-cloudinit: bump to 0.10.2+git 2014-09-14 08:19:57 -07:00
Alex Crawford
8370b30aa2 coreos-cloudinit: bump to 0.10.2 2014-09-14 08:19:33 -07:00
Alex Crawford
3e015cc3a1 Merge pull request #233 from crawford/configdrive
configdrive: don't fail if no network config was provided
2014-09-14 08:18:14 -07:00
Alex Crawford
a0fe6d0884 configdrive: return an empty network config when filename is empty
Additionally, don't bother checking for a network config if it isn't going to
be processed.
2014-09-13 21:51:51 -07:00
Alex Crawford
585ce5fcd9 Revert "metadata: don't fail if no network config was provided"
This reverts commit c1f373e648.
2014-09-13 21:01:42 -07:00
Alex Crawford
72445796ca coreos-cloudinit: bump to 0.10.1+git 2014-09-12 16:48:15 -07:00
Alex Crawford
7342d91a85 coreos-cloudinit: bump to 0.10.1 2014-09-12 16:47:58 -07:00
Alex Crawford
db1bc51c98 Merge pull request #231 from crawford/netconf
metadata: don't fail if no network config was provided
2014-09-12 16:35:24 -07:00
Alex Crawford
c1f373e648 metadata: don't fail if no network config was provided 2014-09-12 16:29:27 -07:00
Alex Crawford
db49a16002 coreos-cloudinit: bump to 0.10.0+git 2014-09-11 17:37:05 -07:00
Alex Crawford
a4a6c281d9 coreos-cloudinit: bump to 0.10.0 2014-09-11 17:36:38 -07:00
Alex Crawford
17f8733121 Merge pull request #228 from crawford/sub
env: add support for escaping environment substitutions
2014-09-11 15:34:03 -07:00
Alex Crawford
7dec922618 env: add support for escaping environment substitutions 2014-09-11 15:30:33 -07:00
Alex Crawford
54d3ae27af Merge pull request #226 from crawford/oem
flags: add oem flag
2014-09-11 13:25:21 -07:00
Alex Crawford
ee2416af64 flags: move the flags into their own namespace 2014-09-11 12:00:17 -07:00
Alex Crawford
cda037f9a5 flags: add oem flag
The oem flag will allow each of the OEMs to specify one flag only, acting as a
shortcut to their specific configuration. This will allow us to update which
options each OEM uses when running cloudinit.
2014-09-11 12:00:17 -07:00
Alex Crawford
549806cf64 Merge pull request #227 from crawford/ipv6
metadata: add support for IPv6 variable substitution
2014-09-11 10:45:33 -07:00
Alex Crawford
56815a6756 metadata: add support for IPv6 variable substitution 2014-09-11 10:43:02 -07:00
Alex Crawford
24a6f7c49c Merge pull request #225 from crawford/exit
userdata: change handling of bad userdata
2014-09-10 19:16:12 -07:00
Alex Crawford
98484be434 userdata: change handling of bad userdata
Don't fail after encountering bad userdata. Continue processing the metadata
and then exit. This will allow people with bad userdata to actually log in and
see the error.
2014-09-10 17:50:23 -07:00
Jonathan Boulle
9024659296 Merge pull request #217 from ecnahc515/patch-1
Fix broken link to fleet config
2014-09-09 15:19:18 -07:00
Chance Zibolski
fc6940f7ba Documentation: More specific link to fleet config.
Add an anchor tag to the url to take person directly to config section.
2014-09-09 15:15:55 -07:00
Brian Waldon
f2fd95699b Merge pull request #224 from bcwaldon/typo
docs: fix a typo
2014-09-09 12:36:42 -07:00
bdevloed
65db96cc7c docs: fix a typo 2014-09-09 12:31:54 -07:00
Alex Crawford
c17b93b5c0 Merge pull request #223 from crawford/yaml
third_party: sync third_party/gopkg.in/yaml.v1
2014-09-08 19:28:59 -07:00
Alex Crawford
d352f8ce6a Merge pull request #222 from crawford/contribute
docs: Update maintainers and contribution guide
2014-09-08 15:54:30 -07:00
Alex Crawford
78aa2c56ec yaml: replace goyaml with yaml 2014-09-08 13:25:27 -07:00
Alex Crawford
c5b3788282 third_party: sync third_party/gopkg.in/yaml.v1
Update launchpad.net/goyaml to gopkg.in/yaml.v1
2014-09-08 13:23:50 -07:00
Alex Crawford
5e98970bb5 docs: Update maintainers and contribution guide 2014-09-08 12:55:17 -07:00
Alex Crawford
cbdd446c55 Merge pull request #220 from crawford/docs
docs: Update list of platforms supporting variable substitutions
2014-09-05 11:51:45 -07:00
Alex Crawford
316cadcf44 docs: Update list of platforms supporting variable substitutions 2014-09-04 12:57:19 -07:00
Alex Crawford
5a939be21b coreos-cloudinit: bump to 0.9.6+git 2014-09-02 17:49:09 -07:00
Alex Crawford
8d76c64386 coreos-cloudinit: bump to 0.9.6 2014-09-02 17:48:45 -07:00
Alex Crawford
1b854eb51e Merge pull request #218 from crawford/units
units: Ensure that the units are executed in order
2014-09-02 17:40:37 -07:00
Alex Crawford
9fcf338bf3 units: Ensure that the units are executed in order 2014-09-02 17:15:32 -07:00
Alex Crawford
fda72bdb5c coreos-cloudinit: bump to 0.9.5+git 2014-09-02 10:10:59 -07:00
Alex Crawford
685a38c6c8 coreos-cloudinit: bump to 0.9.5 2014-09-02 10:10:41 -07:00
Alex Crawford
9d15f2cfaf Merge pull request #213 from crawford/digitalocean
digitalocean: Add support for DigitalOcean
2014-09-01 16:55:12 -07:00
Alex Crawford
2134fce791 digitalocean: Add tests for network unit generation 2014-09-01 16:53:15 -07:00
Alex Crawford
3abd6b2225 digitalocean: Add DigitalOcean metadata service
Move debian-related processing into its own file.
2014-09-01 16:53:15 -07:00
Alex Crawford
2a8e6c9566 network: Fall back to MAC address if there is no name 2014-09-01 09:29:45 -07:00
Alex Crawford
abe43537da metadata: Merge the network config 2014-09-01 09:29:45 -07:00
Jonathan Boulle
3a550af651 Merge pull request #216 from robszumski/patch-2
docs: fix broken link to fleet docs
2014-08-29 11:22:13 -07:00
Rob Szumski
61c3a0eb2d docs: fix broken link to fleet docs 2014-08-29 11:17:05 -07:00
Brian Waldon
480176bc11 Merge pull request #214 from bcwaldon/clarify-write-files
doc: clarify docs around write_files
2014-08-28 20:24:11 -07:00
Brian Waldon
01b18eb551 squash: fix spacing 2014-08-28 13:48:58 -07:00
Brian Waldon
970ef435b6 doc: clarify docs around write_files 2014-08-28 13:33:59 -07:00
Alex Crawford
e8d0021140 Merge pull request #212 from crawford/metadata
refactor: Refactor metadata and datasources to be more testable
2014-08-26 18:45:10 -07:00
Alex Crawford
e9ec78ac6f test: Refactor interface tests a little 2014-08-26 13:18:46 -07:00
Alex Crawford
4a2e417781 network: Add support for multiple addresses and HW addresses
Logical Interfaces can be assigned a hardware address allowing them
to match on MAC address. The static config method also needs to
support specifying multiple addresses.
2014-08-26 13:07:38 -07:00
Alex Crawford
604ef7ecb4 datasource: Add FetchNetworkConfig
FetchNetworkConfig is currently only used by ConfigDrive to read the
network config file from the disk.
2014-08-26 13:04:43 -07:00
Alex Crawford
c39dd5cc67 networkd: Fix bug causing bonding to always be loaded 2014-08-26 13:04:21 -07:00
Alex Crawford
a923161f4a metadata: Refactor common parts out of ec2 2014-08-26 12:02:56 -07:00
Alex Crawford
e59e2f6cd5 Merge pull request #210 from crawford/test
test: Add gofmt to test
2014-08-25 17:04:04 -07:00
Alex Crawford
e90fe3eba8 test: Add gofmt to test 2014-08-25 12:48:52 -07:00
Alex Crawford
fb0187b197 gofmt: sort 2014-08-25 12:35:40 -07:00
Michael Marineau
6babe74716 Merge pull request #209 from marineam/go13
travis: enable testing under go 1.3
2014-08-25 12:26:23 -07:00
Michael Marineau
b1e88284ca travis: enable testing under go 1.3 2014-08-25 12:21:07 -07:00
Alex Crawford
18a65f7dac Merge pull request #208 from crawford/go
test: Fix tests for Go 1.3
2014-08-25 12:19:52 -07:00
Alex Crawford
0c212c72c9 test: Fix tests for Go 1.3 2014-08-25 12:01:27 -07:00
Alex Crawford
6a800d8cc0 coreos-cloudinit: bump to 0.9.4+git 2014-08-24 18:41:20 -07:00
58 changed files with 1986 additions and 834 deletions

View File

@@ -1,5 +1,7 @@
language: go language: go
go: 1.2 go:
- 1.3
- 1.2
install: install:
- go get code.google.com/p/go.tools/cmd/cover - go get code.google.com/p/go.tools/cmd/cover

View File

@@ -39,22 +39,25 @@ Thanks for your contributions!
### Format of the Commit Message ### Format of the Commit Message
We follow a rough convention for commit messages borrowed from AngularJS. This We follow a rough convention for commit messages that is designed to answer two
is an example of a commit: questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
``` ```
feat(scripts/test-cluster): add a cluster test command environment: write new keys in consistent order
this uses tmux to setup a test cluster that you can easily kill and Go 1.3 randomizes the ordering of keys when iterating over a map.
start for debugging. Sort the keys to make this ordering consistent.
Fixes #38
``` ```
The format can be described more formally as follows: The format can be described more formally as follows:
``` ```
<type>(<scope>): <subject> <subsystem>: <what changed>
<BLANK LINE> <BLANK LINE>
<body> <why this change was made>
<BLANK LINE> <BLANK LINE>
<footer> <footer>
``` ```
@@ -63,25 +66,3 @@ The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters. second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various This allows the message to be easier to read on GitHub as well as in various
git tools. git tools.
#### Subject Line
The subject line contains a succinct description of the change.
#### Allowed `<type>`s
- *feat* (feature)
- *fix* (bug fix)
- *docs* (documentation)
- *style* (formatting, missing semi colons, …)
- *refactor*
- *test* (when adding missing tests)
- *chore* (maintain)
#### Allowed `<scope>`s
Scopes can anything specifying the place of the commit change in the code base -
for example, "api", "store", etc.
For more details on the commit format, see the [AngularJS commit style
guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).

View File

@@ -4,7 +4,7 @@ CoreOS allows you to declaratively customize various OS-level items, such as net
## Configuration File ## Configuration File
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units. The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file, which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms. We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
@@ -68,7 +68,7 @@ Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
For more information about the available configuration parameters, see the [etcd documentation][etcd-config]. For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
Note that hyphens in the coreos.etcd.* keys are mapped to underscores. Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, and Vagrant._ _Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md [etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
@@ -95,7 +95,7 @@ Environment="FLEET_METADATA=region=us-west"
For more information on fleet configuration, see the [fleet documentation][fleet-config]. For more information on fleet configuration, see the [fleet documentation][fleet-config].
[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/configuration.md [fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md#configuration
#### update #### update
@@ -130,11 +130,11 @@ The `coreos.units.*` parameters define a list of arbitrary systemd units to star
Each item is an object with the following fields: Each item is an object with the following fields:
- **name**: String representing unit's name. Required. - **name**: String representing unit's name. Required.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. Default value is false. - **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. The default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false. - **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. The default value is false.
- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already. - **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart. - **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. The default behavior is to not execute any commands.
- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. Default value is false. - **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. The default value is false.
**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place. **NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
@@ -298,7 +298,8 @@ users:
### write_files ### write_files
The `write-file` parameter defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys: The `write_files` directive defines a set of files to create on the local filesystem.
Each item in the list may have the following keys:
- **path**: Absolute location on disk where contents should be written - **path**: Absolute location on disk where contents should be written
- **content**: Data to write at the provided `path` - **content**: Data to write at the provided `path`
@@ -311,11 +312,16 @@ The **content** field must represent exactly what should be written to disk.
```yaml ```yaml
#cloud-config #cloud-config
write_files: write_files:
- path: /etc/fleet/fleet.conf - path: /etc/resolv.conf
permissions: 0644 permissions: 0644
owner: root
content: | content: |
verbosity=1 nameserver 8.8.8.8
metadata="region=us-west,type=ssd" - path: /etc/motd
permissions: 0644
owner: root
content: |
Good news, everyone!
``` ```
### manage_etc_hosts ### manage_etc_hosts

3
MAINTAINERS Normal file
View File

@@ -0,0 +1,3 @@
Alex Crawford <alex.crawford@coreos.com> (@crawford)
Jonathan Boulle <jonathan.boulle@coreos.com> (@jonboulle)
Brian Waldon <brian.waldon@coreos.com> (@bcwaldon)

View File

@@ -11,6 +11,7 @@ import (
"github.com/coreos/coreos-cloudinit/datasource/configdrive" "github.com/coreos/coreos-cloudinit/datasource/configdrive"
"github.com/coreos/coreos-cloudinit/datasource/file" "github.com/coreos/coreos-cloudinit/datasource/file"
"github.com/coreos/coreos-cloudinit/datasource/metadata/cloudsigma" "github.com/coreos/coreos-cloudinit/datasource/metadata/cloudsigma"
"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
"github.com/coreos/coreos-cloudinit/datasource/metadata/ec2" "github.com/coreos/coreos-cloudinit/datasource/metadata/ec2"
"github.com/coreos/coreos-cloudinit/datasource/proc_cmdline" "github.com/coreos/coreos-cloudinit/datasource/proc_cmdline"
"github.com/coreos/coreos-cloudinit/datasource/url" "github.com/coreos/coreos-cloudinit/datasource/url"
@@ -20,13 +21,14 @@ import (
) )
const ( const (
version = "0.9.4" version = "0.10.5"
datasourceInterval = 100 * time.Millisecond datasourceInterval = 100 * time.Millisecond
datasourceMaxInterval = 30 * time.Second datasourceMaxInterval = 30 * time.Second
datasourceTimeout = 5 * time.Minute datasourceTimeout = 5 * time.Minute
) )
var ( var (
flags = struct {
printVersion bool printVersion bool
ignoreFailure bool ignoreFailure bool
sources struct { sources struct {
@@ -35,81 +37,109 @@ var (
metadataService bool metadataService bool
ec2MetadataService string ec2MetadataService string
cloudSigmaMetadataService bool cloudSigmaMetadataService bool
digitalOceanMetadataService string
url string url string
procCmdLine bool procCmdLine bool
} }
convertNetconf string convertNetconf string
workspace string workspace string
sshKeyName string sshKeyName string
oem string
}{}
) )
func init() { func init() {
flag.BoolVar(&printVersion, "version", false, "Print the version and exit") flag.BoolVar(&flags.printVersion, "version", false, "Print the version and exit")
flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data") flag.BoolVar(&flags.ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
flag.StringVar(&sources.file, "from-file", "", "Read user-data from provided file") flag.StringVar(&flags.sources.file, "from-file", "", "Read user-data from provided file")
flag.StringVar(&sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory") flag.StringVar(&flags.sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory")
flag.BoolVar(&sources.metadataService, "from-metadata-service", false, "[DEPRECATED - Use -from-ec2-metadata] Download data from metadata service") flag.BoolVar(&flags.sources.metadataService, "from-metadata-service", false, "[DEPRECATED - Use -from-ec2-metadata] Download data from metadata service")
flag.StringVar(&sources.ec2MetadataService, "from-ec2-metadata", "", "Download data from the provided metadata service") flag.StringVar(&flags.sources.ec2MetadataService, "from-ec2-metadata", "", "Download EC2 data from the provided url")
flag.BoolVar(&sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context") flag.BoolVar(&flags.sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context")
flag.StringVar(&sources.url, "from-url", "", "Download user-data from provided url") flag.StringVar(&flags.sources.digitalOceanMetadataService, "from-digitalocean-metadata", "", "Download DigitalOcean data from the provided url")
flag.BoolVar(&sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", proc_cmdline.ProcCmdlineLocation, proc_cmdline.ProcCmdlineCloudConfigFlag)) flag.StringVar(&flags.sources.url, "from-url", "", "Download user-data from provided url")
flag.StringVar(&convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files (requires the -from-configdrive flag)") flag.BoolVar(&flags.sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", proc_cmdline.ProcCmdlineLocation, proc_cmdline.ProcCmdlineCloudConfigFlag))
flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data") flag.StringVar(&flags.oem, "oem", "", "Use the settings specific to the provided OEM")
flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name") flag.StringVar(&flags.convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files")
flag.StringVar(&flags.workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
flag.StringVar(&flags.sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
} }
type oemConfig map[string]string
var (
oemConfigs = map[string]oemConfig{
"digitalocean": oemConfig{
"from-digitalocean-metadata": "http://169.254.169.254/",
"convert-netconf": "digitalocean",
},
"ec2-compat": oemConfig{
"from-ec2-metadata": "http://169.254.169.254/",
"from-configdrive": "/media/configdrive",
},
"rackspace-onmetal": oemConfig{
"from-configdrive": "/media/configdrive",
"convert-netconf": "debian",
},
}
)
func main() { func main() {
failure := false
flag.Parse() flag.Parse()
die := func() { if c, ok := oemConfigs[flags.oem]; ok {
if ignoreFailure { for k, v := range c {
os.Exit(0) flag.Set(k, v)
} }
os.Exit(1) } else if flags.oem != "" {
oems := make([]string, 0, len(oemConfigs))
for k := range oemConfigs {
oems = append(oems, k)
}
fmt.Printf("Invalid option to --oem: %q. Supported options: %q\n", flags.oem, oems)
os.Exit(2)
} }
if printVersion == true { if flags.printVersion == true {
fmt.Printf("coreos-cloudinit version %s\n", version) fmt.Printf("coreos-cloudinit version %s\n", version)
os.Exit(0) os.Exit(0)
} }
if convertNetconf != "" && sources.configDrive == "" { switch flags.convertNetconf {
fmt.Println("-convert-netconf flag requires -from-configdrive")
os.Exit(1)
}
switch convertNetconf {
case "": case "":
case "debian": case "debian":
case "digitalocean":
default: default:
fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian'\n", convertNetconf) fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian, digitalocean'\n", flags.convertNetconf)
os.Exit(1) os.Exit(2)
} }
dss := getDatasources() dss := getDatasources()
if len(dss) == 0 { if len(dss) == 0 {
fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-ec2-metadata, --from-cloudsigma-metadata, --from-url or --from-proc-cmdline") fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-ec2-metadata, --from-cloudsigma-metadata, --from-url or --from-proc-cmdline")
os.Exit(1) os.Exit(2)
} }
ds := selectDatasource(dss) ds := selectDatasource(dss)
if ds == nil { if ds == nil {
fmt.Println("No datasources available in time") fmt.Println("No datasources available in time")
die() os.Exit(1)
} }
fmt.Printf("Fetching user-data from datasource of type %q\n", ds.Type()) fmt.Printf("Fetching user-data from datasource of type %q\n", ds.Type())
userdataBytes, err := ds.FetchUserdata() userdataBytes, err := ds.FetchUserdata()
if err != nil { if err != nil {
fmt.Printf("Failed fetching user-data from datasource: %v\n", err) fmt.Printf("Failed fetching user-data from datasource: %v\nContinuing...\n", err)
die() failure = true
} }
fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type()) fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
metadataBytes, err := ds.FetchMetadata() metadataBytes, err := ds.FetchMetadata()
if err != nil { if err != nil {
fmt.Printf("Failed fetching meta-data from datasource: %v\n", err) fmt.Printf("Failed fetching meta-data from datasource: %v\n", err)
die() os.Exit(1)
} }
// Extract IPv4 addresses from metadata if possible // Extract IPv4 addresses from metadata if possible
@@ -118,23 +148,34 @@ func main() {
subs, err = initialize.ExtractIPsFromMetadata(metadataBytes) subs, err = initialize.ExtractIPsFromMetadata(metadataBytes)
if err != nil { if err != nil {
fmt.Printf("Failed extracting IPs from meta-data: %v\n", err) fmt.Printf("Failed extracting IPs from meta-data: %v\n", err)
die() os.Exit(1)
} }
} }
// Apply environment to user-data // Apply environment to user-data
env := initialize.NewEnvironment("/", ds.ConfigRoot(), workspace, convertNetconf, sshKeyName, subs) env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.convertNetconf, flags.sshKeyName, subs)
userdata := env.Apply(string(userdataBytes)) userdata := env.Apply(string(userdataBytes))
var ccm, ccu *initialize.CloudConfig var ccm, ccu *initialize.CloudConfig
var script *system.Script var script *system.Script
if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil { if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil {
fmt.Printf("Failed to parse meta-data: %v\n", err) fmt.Printf("Failed to parse meta-data: %v\n", err)
die() os.Exit(1)
} }
if ccm != nil && flags.convertNetconf != "" {
fmt.Printf("Fetching network config from datasource of type %q\n", ds.Type())
netconfBytes, err := ds.FetchNetworkConfig(ccm.NetworkConfigPath)
if err != nil {
fmt.Printf("Failed fetching network config from datasource: %v\n", err)
os.Exit(1)
}
ccm.NetworkConfig = string(netconfBytes)
}
if ud, err := initialize.ParseUserData(userdata); err != nil { if ud, err := initialize.ParseUserData(userdata); err != nil {
fmt.Printf("Failed to parse user-data: %v\n", err) fmt.Printf("Failed to parse user-data: %v\nContinuing...\n", err)
die() failure = true
} else { } else {
switch t := ud.(type) { switch t := ud.(type) {
case *initialize.CloudConfig: case *initialize.CloudConfig:
@@ -162,16 +203,20 @@ func main() {
if cc != nil { if cc != nil {
if err = initialize.Apply(*cc, env); err != nil { if err = initialize.Apply(*cc, env); err != nil {
fmt.Printf("Failed to apply cloud-config: %v\n", err) fmt.Printf("Failed to apply cloud-config: %v\n", err)
die() os.Exit(1)
} }
} }
if script != nil { if script != nil {
if err = runScript(*script, env); err != nil { if err = runScript(*script, env); err != nil {
fmt.Printf("Failed to run script: %v\n", err) fmt.Printf("Failed to run script: %v\n", err)
die() os.Exit(1)
} }
} }
if failure && !flags.ignoreFailure {
os.Exit(1)
}
} }
// mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from // mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from
@@ -198,6 +243,13 @@ func mergeCloudConfig(mdcc, udcc initialize.CloudConfig) (cc initialize.CloudCon
udcc.NetworkConfigPath = mdcc.NetworkConfigPath udcc.NetworkConfigPath = mdcc.NetworkConfigPath
} }
} }
if mdcc.NetworkConfig != "" {
if udcc.NetworkConfig != "" {
fmt.Printf("Warning: user-data NetworkConfig %s overrides metadata NetworkConfig %s\n", udcc.NetworkConfig, mdcc.NetworkConfig)
} else {
udcc.NetworkConfig = mdcc.NetworkConfig
}
}
return udcc return udcc
} }
@@ -205,25 +257,28 @@ func mergeCloudConfig(mdcc, udcc initialize.CloudConfig) (cc initialize.CloudCon
// on the different source command-line flags. // on the different source command-line flags.
func getDatasources() []datasource.Datasource { func getDatasources() []datasource.Datasource {
dss := make([]datasource.Datasource, 0, 5) dss := make([]datasource.Datasource, 0, 5)
if sources.file != "" { if flags.sources.file != "" {
dss = append(dss, file.NewDatasource(sources.file)) dss = append(dss, file.NewDatasource(flags.sources.file))
} }
if sources.url != "" { if flags.sources.url != "" {
dss = append(dss, url.NewDatasource(sources.url)) dss = append(dss, url.NewDatasource(flags.sources.url))
} }
if sources.configDrive != "" { if flags.sources.configDrive != "" {
dss = append(dss, configdrive.NewDatasource(sources.configDrive)) dss = append(dss, configdrive.NewDatasource(flags.sources.configDrive))
} }
if sources.metadataService { if flags.sources.metadataService {
dss = append(dss, ec2.NewDatasource(ec2.DefaultAddress)) dss = append(dss, ec2.NewDatasource(ec2.DefaultAddress))
} }
if sources.ec2MetadataService != "" { if flags.sources.ec2MetadataService != "" {
dss = append(dss, ec2.NewDatasource(sources.ec2MetadataService)) dss = append(dss, ec2.NewDatasource(flags.sources.ec2MetadataService))
} }
if sources.cloudSigmaMetadataService { if flags.sources.cloudSigmaMetadataService {
dss = append(dss, cloudsigma.NewServerContextService()) dss = append(dss, cloudsigma.NewServerContextService())
} }
if sources.procCmdLine { if flags.sources.digitalOceanMetadataService != "" {
dss = append(dss, digitalocean.NewDatasource(flags.sources.digitalOceanMetadataService))
}
if flags.sources.procCmdLine {
dss = append(dss, proc_cmdline.NewDatasource()) dss = append(dss, proc_cmdline.NewDatasource())
} }
return dss return dss

View File

@@ -12,6 +12,7 @@ func TestMergeCloudConfig(t *testing.T) {
SSHAuthorizedKeys: []string{"abc", "def"}, SSHAuthorizedKeys: []string{"abc", "def"},
Hostname: "foobar", Hostname: "foobar",
NetworkConfigPath: "/path/somewhere", NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{}`,
} }
for i, tt := range []struct { for i, tt := range []struct {
udcc initialize.CloudConfig udcc initialize.CloudConfig
@@ -36,6 +37,7 @@ func TestMergeCloudConfig(t *testing.T) {
initialize.CloudConfig{ initialize.CloudConfig{
Hostname: "meta-hostname", Hostname: "meta-hostname",
NetworkConfigPath: "/path/meta", NetworkConfigPath: "/path/meta",
NetworkConfig: `{"hostname":"test"}`,
}, },
simplecc, simplecc,
}, },
@@ -45,6 +47,7 @@ func TestMergeCloudConfig(t *testing.T) {
SSHAuthorizedKeys: []string{"abc", "def"}, SSHAuthorizedKeys: []string{"abc", "def"},
Hostname: "user-hostname", Hostname: "user-hostname",
NetworkConfigPath: "/path/somewhere", NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{"hostname":"test"}`,
}, },
initialize.CloudConfig{ initialize.CloudConfig{
SSHAuthorizedKeys: []string{"woof", "qux"}, SSHAuthorizedKeys: []string{"woof", "qux"},
@@ -54,6 +57,7 @@ func TestMergeCloudConfig(t *testing.T) {
SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"}, SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"},
Hostname: "user-hostname", Hostname: "user-hostname",
NetworkConfigPath: "/path/somewhere", NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{"hostname":"test"}`,
}, },
}, },
{ {
@@ -64,11 +68,13 @@ func TestMergeCloudConfig(t *testing.T) {
initialize.CloudConfig{ initialize.CloudConfig{
SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"}, SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
NetworkConfigPath: "/dev/fun", NetworkConfigPath: "/dev/fun",
NetworkConfig: `{"hostname":"test"}`,
}, },
initialize.CloudConfig{ initialize.CloudConfig{
Hostname: "supercool", Hostname: "supercool",
SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"}, SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
NetworkConfigPath: "/dev/fun", NetworkConfigPath: "/dev/fun",
NetworkConfig: `{"hostname":"test"}`,
}, },
}, },
{ {
@@ -80,11 +86,13 @@ func TestMergeCloudConfig(t *testing.T) {
initialize.CloudConfig{ initialize.CloudConfig{
Hostname: "youyouyou", Hostname: "youyouyou",
NetworkConfigPath: "meta-meta-yo", NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
}, },
initialize.CloudConfig{ initialize.CloudConfig{
Hostname: "mememe", Hostname: "mememe",
ManageEtcHosts: initialize.EtcHosts("lolz"), ManageEtcHosts: initialize.EtcHosts("lolz"),
NetworkConfigPath: "meta-meta-yo", NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
}, },
}, },
{ {
@@ -95,10 +103,12 @@ func TestMergeCloudConfig(t *testing.T) {
initialize.CloudConfig{ initialize.CloudConfig{
ManageEtcHosts: initialize.EtcHosts("lolz"), ManageEtcHosts: initialize.EtcHosts("lolz"),
NetworkConfigPath: "meta-meta-yo", NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
}, },
initialize.CloudConfig{ initialize.CloudConfig{
Hostname: "mememe", Hostname: "mememe",
NetworkConfigPath: "meta-meta-yo", NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
}, },
}, },
} { } {

View File

@@ -41,6 +41,13 @@ func (cd *configDrive) FetchUserdata() ([]byte, error) {
return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "user_data")) return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "user_data"))
} }
func (cd *configDrive) FetchNetworkConfig(filename string) ([]byte, error) {
if filename == "" {
return []byte{}, nil
}
return cd.tryReadFile(path.Join(cd.openstackRoot(), filename))
}
func (cd *configDrive) Type() string { func (cd *configDrive) Type() string {
return "cloud-drive" return "cloud-drive"
} }

View File

@@ -1,15 +1,11 @@
package datasource package datasource
const (
Ec2ApiVersion = "2009-04-04"
OpenstackApiVersion = "2012-08-10"
)
type Datasource interface { type Datasource interface {
IsAvailable() bool IsAvailable() bool
AvailabilityChanges() bool AvailabilityChanges() bool
ConfigRoot() string ConfigRoot() string
FetchMetadata() ([]byte, error) FetchMetadata() ([]byte, error)
FetchUserdata() ([]byte, error) FetchUserdata() ([]byte, error)
FetchNetworkConfig(string) ([]byte, error)
Type() string Type() string
} }

View File

@@ -34,6 +34,10 @@ func (f *localFile) FetchUserdata() ([]byte, error) {
return ioutil.ReadFile(f.path) return ioutil.ReadFile(f.path)
} }
func (f *localFile) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (f *localFile) Type() string { func (f *localFile) Type() string {
return "local-file" return "local-file"
} }

View File

@@ -126,6 +126,10 @@ func (scs *serverContextService) FetchUserdata() ([]byte, error) {
return []byte(userData), nil return []byte(userData), nil
} }
func (scs *serverContextService) FetchNetworkConfig(a string) ([]byte, error) {
return nil, nil
}
func isBase64Encoded(field string, userdata map[string]string) bool { func isBase64Encoded(field string, userdata map[string]string) bool {
base64Fields, ok := userdata["base64_fields"] base64Fields, ok := userdata["base64_fields"]
if !ok { if !ok {

View File

@@ -0,0 +1,107 @@
package digitalocean
import (
"encoding/json"
"strconv"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
)
const (
DefaultAddress = "http://169.254.169.254/"
apiVersion = "metadata/v1"
userdataUrl = apiVersion + "/user-data"
metadataPath = apiVersion + ".json"
)
type Address struct {
IPAddress string `json:"ip_address"`
Netmask string `json:"netmask"`
Cidr int `json:"cidr"`
Gateway string `json:"gateway"`
}
type Interface struct {
IPv4 *Address `json:"ipv4"`
IPv6 *Address `json:"ipv6"`
MAC string `json:"mac"`
Type string `json:"type"`
}
type Interfaces struct {
Public []Interface `json:"public"`
Private []Interface `json:"private"`
}
type DNS struct {
Nameservers []string `json:"nameservers"`
}
type Metadata struct {
Hostname string `json:"hostname"`
Interfaces Interfaces `json:"interfaces"`
PublicKeys []string `json:"public_keys"`
DNS DNS `json:"dns"`
}
type metadataService struct {
interfaces Interfaces
dns DNS
metadata.MetadataService
}
func NewDatasource(root string) *metadataService {
return &metadataService{MetadataService: metadata.NewDatasource(root, apiVersion, userdataUrl, metadataPath)}
}
func (ms *metadataService) FetchMetadata() ([]byte, error) {
data, err := ms.FetchData(ms.MetadataUrl())
if err != nil || len(data) == 0 {
return []byte{}, err
}
var metadata Metadata
if err := json.Unmarshal(data, &metadata); err != nil {
return []byte{}, err
}
ms.interfaces = metadata.Interfaces
ms.dns = metadata.DNS
attrs := make(map[string]interface{})
if len(metadata.Interfaces.Public) > 0 {
if metadata.Interfaces.Public[0].IPv4 != nil {
attrs["public-ipv4"] = metadata.Interfaces.Public[0].IPv4.IPAddress
}
if metadata.Interfaces.Public[0].IPv6 != nil {
attrs["public-ipv6"] = metadata.Interfaces.Public[0].IPv6.IPAddress
}
}
if len(metadata.Interfaces.Private) > 0 {
if metadata.Interfaces.Private[0].IPv4 != nil {
attrs["local-ipv4"] = metadata.Interfaces.Private[0].IPv4.IPAddress
}
if metadata.Interfaces.Private[0].IPv6 != nil {
attrs["local-ipv6"] = metadata.Interfaces.Private[0].IPv6.IPAddress
}
}
attrs["hostname"] = metadata.Hostname
keys := make(map[string]string)
for i, key := range metadata.PublicKeys {
keys[strconv.Itoa(i)] = key
}
attrs["public_keys"] = keys
return json.Marshal(attrs)
}
func (ms metadataService) FetchNetworkConfig(filename string) ([]byte, error) {
return json.Marshal(Metadata{
Interfaces: ms.interfaces,
DNS: ms.dns,
})
}
func (ms metadataService) Type() string {
return "digitalocean-metadata-service"
}

View File

@@ -0,0 +1,99 @@
package digitalocean
import (
"bytes"
"fmt"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
"github.com/coreos/coreos-cloudinit/pkg"
)
func TestType(t *testing.T) {
want := "digitalocean-metadata-service"
if kind := (metadataService{}).Type(); kind != want {
t.Fatalf("bad type: want %q, got %q", want, kind)
}
}
func TestFetchMetadata(t *testing.T) {
for _, tt := range []struct {
root string
metadataPath string
resources map[string]string
expect []byte
clientErr error
expectErr error
}{
{
root: "/",
metadataPath: "v1.json",
resources: map[string]string{
"/v1.json": "bad",
},
expectErr: fmt.Errorf("invalid character 'b' looking for beginning of value"),
},
{
root: "/",
metadataPath: "v1.json",
resources: map[string]string{
"/v1.json": `{
"droplet_id": 1,
"user_data": "hello",
"vendor_data": "hello",
"public_keys": [
"publickey1",
"publickey2"
],
"region": "nyc2",
"interfaces": {
"public": [
{
"ipv4": {
"ip_address": "192.168.1.2",
"netmask": "255.255.255.0",
"gateway": "192.168.1.1"
},
"ipv6": {
"ip_address": "fe00::",
"cidr": 126,
"gateway": "fe00::"
},
"mac": "ab:cd:ef:gh:ij",
"type": "public"
}
]
}
}`,
},
expect: []byte(`{"hostname":"","public-ipv4":"192.168.1.2","public-ipv6":"fe00::","public_keys":{"0":"publickey1","1":"publickey2"}}`),
},
{
clientErr: pkg.ErrTimeout{fmt.Errorf("test error")},
expectErr: pkg.ErrTimeout{fmt.Errorf("test error")},
},
} {
service := &metadataService{
MetadataService: metadata.MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, tt.clientErr},
MetadataPath: tt.metadataPath,
},
}
metadata, err := service.FetchMetadata()
if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
}
if !bytes.Equal(metadata, tt.expect) {
t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
}
}
}
func Error(err error) string {
if err != nil {
return err.Error()
}
return ""
}

View File

@@ -7,44 +7,28 @@ import (
"fmt" "fmt"
"strings" "strings"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
"github.com/coreos/coreos-cloudinit/pkg" "github.com/coreos/coreos-cloudinit/pkg"
) )
const ( const (
DefaultAddress = "http://169.254.169.254/" DefaultAddress = "http://169.254.169.254/"
apiVersion = "2009-04-04" apiVersion = "2009-04-04/"
userdataUrl = apiVersion + "/user-data" userdataPath = apiVersion + "user-data"
metadataUrl = apiVersion + "/meta-data" metadataPath = apiVersion + "meta-data"
) )
type metadataService struct { type metadataService struct {
root string metadata.MetadataService
client pkg.Getter
} }
func NewDatasource(root string) *metadataService { func NewDatasource(root string) *metadataService {
if !strings.HasSuffix(root, "/") { return &metadataService{metadata.NewDatasource(root, apiVersion, userdataPath, metadataPath)}
root += "/"
}
return &metadataService{root, pkg.NewHttpClient()}
}
func (ms metadataService) IsAvailable() bool {
_, err := ms.client.Get(ms.root + apiVersion)
return (err == nil)
}
func (ms metadataService) AvailabilityChanges() bool {
return true
}
func (ms metadataService) ConfigRoot() string {
return ms.root
} }
func (ms metadataService) FetchMetadata() ([]byte, error) { func (ms metadataService) FetchMetadata() ([]byte, error) {
attrs := make(map[string]interface{}) attrs := make(map[string]interface{})
if keynames, err := fetchAttributes(ms.client, fmt.Sprintf("%s/public-keys", ms.metadataUrl())); err == nil { if keynames, err := ms.fetchAttributes(fmt.Sprintf("%s/public-keys", ms.MetadataUrl())); err == nil {
keyIDs := make(map[string]string) keyIDs := make(map[string]string)
for _, keyname := range keynames { for _, keyname := range keynames {
tokens := strings.SplitN(keyname, "=", 2) tokens := strings.SplitN(keyname, "=", 2)
@@ -56,7 +40,7 @@ func (ms metadataService) FetchMetadata() ([]byte, error) {
keys := make(map[string]string) keys := make(map[string]string)
for name, id := range keyIDs { for name, id := range keyIDs {
sshkey, err := fetchAttribute(ms.client, fmt.Sprintf("%s/public-keys/%s/openssh-key", ms.metadataUrl(), id)) sshkey, err := ms.fetchAttribute(fmt.Sprintf("%s/public-keys/%s/openssh-key", ms.MetadataUrl(), id))
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -68,25 +52,25 @@ func (ms metadataService) FetchMetadata() ([]byte, error) {
return nil, err return nil, err
} }
if hostname, err := fetchAttribute(ms.client, fmt.Sprintf("%s/hostname", ms.metadataUrl())); err == nil { if hostname, err := ms.fetchAttribute(fmt.Sprintf("%s/hostname", ms.MetadataUrl())); err == nil {
attrs["hostname"] = hostname attrs["hostname"] = hostname
} else if _, ok := err.(pkg.ErrNotFound); !ok { } else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err return nil, err
} }
if localAddr, err := fetchAttribute(ms.client, fmt.Sprintf("%s/local-ipv4", ms.metadataUrl())); err == nil { if localAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/local-ipv4", ms.MetadataUrl())); err == nil {
attrs["local-ipv4"] = localAddr attrs["local-ipv4"] = localAddr
} else if _, ok := err.(pkg.ErrNotFound); !ok { } else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err return nil, err
} }
if publicAddr, err := fetchAttribute(ms.client, fmt.Sprintf("%s/public-ipv4", ms.metadataUrl())); err == nil { if publicAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/public-ipv4", ms.MetadataUrl())); err == nil {
attrs["public-ipv4"] = publicAddr attrs["public-ipv4"] = publicAddr
} else if _, ok := err.(pkg.ErrNotFound); !ok { } else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err return nil, err
} }
if content_path, err := fetchAttribute(ms.client, fmt.Sprintf("%s/network_config/content_path", ms.metadataUrl())); err == nil { if content_path, err := ms.fetchAttribute(fmt.Sprintf("%s/network_config/content_path", ms.MetadataUrl())); err == nil {
attrs["network_config"] = map[string]string{ attrs["network_config"] = map[string]string{
"content_path": content_path, "content_path": content_path,
} }
@@ -97,30 +81,12 @@ func (ms metadataService) FetchMetadata() ([]byte, error) {
return json.Marshal(attrs) return json.Marshal(attrs)
} }
func (ms metadataService) FetchUserdata() ([]byte, error) {
if data, err := ms.client.GetRetry(ms.userdataUrl()); err == nil {
return data, err
} else if _, ok := err.(pkg.ErrNotFound); ok {
return []byte{}, nil
} else {
return data, err
}
}
func (ms metadataService) Type() string { func (ms metadataService) Type() string {
return "ec2-metadata-service" return "ec2-metadata-service"
} }
func (ms metadataService) metadataUrl() string { func (ms metadataService) fetchAttributes(url string) ([]string, error) {
return (ms.root + metadataUrl) resp, err := ms.FetchData(url)
}
func (ms metadataService) userdataUrl() string {
return (ms.root + userdataUrl)
}
func fetchAttributes(client pkg.Getter, url string) ([]string, error) {
resp, err := client.GetRetry(url)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -132,8 +98,8 @@ func fetchAttributes(client pkg.Getter, url string) ([]string, error) {
return data, scanner.Err() return data, scanner.Err()
} }
func fetchAttribute(client pkg.Getter, url string) (string, error) { func (ms metadataService) fetchAttribute(url string) (string, error) {
if attrs, err := fetchAttributes(client, url); err == nil && len(attrs) > 0 { if attrs, err := ms.fetchAttributes(url); err == nil && len(attrs) > 0 {
return attrs[0], nil return attrs[0], nil
} else { } else {
return "", err return "", err

View File

@@ -6,36 +6,11 @@ import (
"reflect" "reflect"
"testing" "testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
"github.com/coreos/coreos-cloudinit/pkg" "github.com/coreos/coreos-cloudinit/pkg"
) )
type testHttpClient struct {
resources map[string]string
err error
}
func (t *testHttpClient) GetRetry(url string) ([]byte, error) {
if t.err != nil {
return nil, t.err
}
if val, ok := t.resources[url]; ok {
return []byte(val), nil
} else {
return nil, pkg.ErrNotFound{fmt.Errorf("not found: %q", url)}
}
}
func (t *testHttpClient) Get(url string) ([]byte, error) {
return t.GetRetry(url)
}
func TestAvailabilityChanges(t *testing.T) {
want := true
if ac := (metadataService{}).AvailabilityChanges(); ac != want {
t.Fatalf("bad AvailabilityChanges: want %q, got %q", want, ac)
}
}
func TestType(t *testing.T) { func TestType(t *testing.T) {
want := "ec2-metadata-service" want := "ec2-metadata-service"
if kind := (metadataService{}).Type(); kind != want { if kind := (metadataService{}).Type(); kind != want {
@@ -43,102 +18,6 @@ func TestType(t *testing.T) {
} }
} }
func TestIsAvailable(t *testing.T) {
for _, tt := range []struct {
root string
resources map[string]string
expect bool
}{
{
root: "/",
resources: map[string]string{
"/2009-04-04": "",
},
expect: true,
},
{
root: "/",
resources: map[string]string{},
expect: false,
},
} {
service := &metadataService{tt.root, &testHttpClient{tt.resources, nil}}
if a := service.IsAvailable(); a != tt.expect {
t.Fatalf("bad isAvailable (%q): want %q, got %q", tt.resources, tt.expect, a)
}
}
}
func TestFetchUserdata(t *testing.T) {
for _, tt := range []struct {
root string
resources map[string]string
userdata []byte
clientErr error
expectErr error
}{
{
root: "/",
resources: map[string]string{
"/2009-04-04/user-data": "hello",
},
userdata: []byte("hello"),
},
{
root: "/",
clientErr: pkg.ErrNotFound{fmt.Errorf("test not found error")},
userdata: []byte{},
},
{
root: "/",
clientErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
expectErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
},
} {
service := &metadataService{tt.root, &testHttpClient{tt.resources, tt.clientErr}}
data, err := service.FetchUserdata()
if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
}
if !bytes.Equal(data, tt.userdata) {
t.Fatalf("bad userdata (%q): want %q, got %q", tt.resources, tt.userdata, data)
}
}
}
func TestUrls(t *testing.T) {
for _, tt := range []struct {
root string
expectRoot string
userdata string
metadata string
}{
{
root: "/",
expectRoot: "/",
userdata: "/2009-04-04/user-data",
metadata: "/2009-04-04/meta-data",
},
{
root: "http://169.254.169.254/",
expectRoot: "http://169.254.169.254/",
userdata: "http://169.254.169.254/2009-04-04/user-data",
metadata: "http://169.254.169.254/2009-04-04/meta-data",
},
} {
service := &metadataService{tt.root, nil}
if url := service.userdataUrl(); url != tt.userdata {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.userdata, url)
}
if url := service.metadataUrl(); url != tt.metadata {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.metadata, url)
}
if url := service.ConfigRoot(); url != tt.expectRoot {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.expectRoot, url)
}
}
}
func TestFetchAttributes(t *testing.T) { func TestFetchAttributes(t *testing.T) {
for _, s := range []struct { for _, s := range []struct {
resources map[string]string resources map[string]string
@@ -169,7 +48,7 @@ func TestFetchAttributes(t *testing.T) {
}, },
}, },
{ {
err: pkg.ErrNotFound{fmt.Errorf("test error")}, err: fmt.Errorf("test error"),
tests: []struct { tests: []struct {
path string path string
val []string val []string
@@ -178,9 +57,11 @@ func TestFetchAttributes(t *testing.T) {
}, },
}, },
} { } {
client := &testHttpClient{s.resources, s.err} service := metadataService{metadata.MetadataService{
Client: &test.HttpClient{s.resources, s.err},
}}
for _, tt := range s.tests { for _, tt := range s.tests {
attrs, err := fetchAttributes(client, tt.path) attrs, err := service.fetchAttributes(tt.path)
if err != s.err { if err != s.err {
t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.resources, s.err, err) t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.resources, s.err, err)
} }
@@ -221,7 +102,7 @@ func TestFetchAttribute(t *testing.T) {
}, },
}, },
{ {
err: pkg.ErrNotFound{fmt.Errorf("test error")}, err: fmt.Errorf("test error"),
tests: []struct { tests: []struct {
path string path string
val string val string
@@ -230,9 +111,11 @@ func TestFetchAttribute(t *testing.T) {
}, },
}, },
} { } {
client := &testHttpClient{s.resources, s.err} service := metadataService{metadata.MetadataService{
Client: &test.HttpClient{s.resources, s.err},
}}
for _, tt := range s.tests { for _, tt := range s.tests {
attr, err := fetchAttribute(client, tt.path) attr, err := service.fetchAttribute(tt.path)
if err != s.err { if err != s.err {
t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.resources, s.err, err) t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.resources, s.err, err)
} }
@@ -246,6 +129,7 @@ func TestFetchAttribute(t *testing.T) {
func TestFetchMetadata(t *testing.T) { func TestFetchMetadata(t *testing.T) {
for _, tt := range []struct { for _, tt := range []struct {
root string root string
metadataPath string
resources map[string]string resources map[string]string
expect []byte expect []byte
clientErr error clientErr error
@@ -253,6 +137,7 @@ func TestFetchMetadata(t *testing.T) {
}{ }{
{ {
root: "/", root: "/",
metadataPath: "2009-04-04/meta-data",
resources: map[string]string{ resources: map[string]string{
"/2009-04-04/meta-data/public-keys": "bad\n", "/2009-04-04/meta-data/public-keys": "bad\n",
}, },
@@ -260,6 +145,7 @@ func TestFetchMetadata(t *testing.T) {
}, },
{ {
root: "/", root: "/",
metadataPath: "2009-04-04/meta-data",
resources: map[string]string{ resources: map[string]string{
"/2009-04-04/meta-data/hostname": "host", "/2009-04-04/meta-data/hostname": "host",
"/2009-04-04/meta-data/local-ipv4": "1.2.3.4", "/2009-04-04/meta-data/local-ipv4": "1.2.3.4",
@@ -276,7 +162,11 @@ func TestFetchMetadata(t *testing.T) {
expectErr: pkg.ErrTimeout{fmt.Errorf("test error")}, expectErr: pkg.ErrTimeout{fmt.Errorf("test error")},
}, },
} { } {
service := &metadataService{tt.root, &testHttpClient{tt.resources, tt.clientErr}} service := &metadataService{metadata.MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, tt.clientErr},
MetadataPath: tt.metadataPath,
}}
metadata, err := service.FetchMetadata() metadata, err := service.FetchMetadata()
if Error(err) != Error(tt.expectErr) { if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err) t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
@@ -287,35 +177,6 @@ func TestFetchMetadata(t *testing.T) {
} }
} }
func TestNewDatasource(t *testing.T) {
for _, tt := range []struct {
root string
expectRoot string
}{
{
root: "",
expectRoot: "/",
},
{
root: "/",
expectRoot: "/",
},
{
root: "http://169.254.169.254",
expectRoot: "http://169.254.169.254/",
},
{
root: "http://169.254.169.254/",
expectRoot: "http://169.254.169.254/",
},
} {
service := NewDatasource(tt.root)
if service.root != tt.expectRoot {
t.Fatalf("bad root (%q): want %q, got %q", tt.root, tt.expectRoot, service.root)
}
}
}
func Error(err error) string { func Error(err error) string {
if err != nil { if err != nil {
return err.Error() return err.Error()

View File

@@ -0,0 +1,61 @@
package metadata
import (
"strings"
"github.com/coreos/coreos-cloudinit/pkg"
)
type MetadataService struct {
Root string
Client pkg.Getter
ApiVersion string
UserdataPath string
MetadataPath string
}
func NewDatasource(root, apiVersion, userdataPath, metadataPath string) MetadataService {
if !strings.HasSuffix(root, "/") {
root += "/"
}
return MetadataService{root, pkg.NewHttpClient(), apiVersion, userdataPath, metadataPath}
}
func (ms MetadataService) IsAvailable() bool {
_, err := ms.Client.Get(ms.Root + ms.ApiVersion)
return (err == nil)
}
func (ms MetadataService) AvailabilityChanges() bool {
return true
}
func (ms MetadataService) ConfigRoot() string {
return ms.Root
}
func (ms MetadataService) FetchUserdata() ([]byte, error) {
return ms.FetchData(ms.UserdataUrl())
}
func (ms MetadataService) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (ms MetadataService) FetchData(url string) ([]byte, error) {
if data, err := ms.Client.GetRetry(url); err == nil {
return data, err
} else if _, ok := err.(pkg.ErrNotFound); ok {
return []byte{}, nil
} else {
return data, err
}
}
func (ms MetadataService) MetadataUrl() string {
return (ms.Root + ms.MetadataPath)
}
func (ms MetadataService) UserdataUrl() string {
return (ms.Root + ms.UserdataPath)
}

View File

@@ -0,0 +1,171 @@
package metadata
import (
"bytes"
"fmt"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
"github.com/coreos/coreos-cloudinit/pkg"
)
func TestAvailabilityChanges(t *testing.T) {
want := true
if ac := (MetadataService{}).AvailabilityChanges(); ac != want {
t.Fatalf("bad AvailabilityChanges: want %q, got %q", want, ac)
}
}
func TestIsAvailable(t *testing.T) {
for _, tt := range []struct {
root string
apiVersion string
resources map[string]string
expect bool
}{
{
root: "/",
apiVersion: "2009-04-04",
resources: map[string]string{
"/2009-04-04": "",
},
expect: true,
},
{
root: "/",
resources: map[string]string{},
expect: false,
},
} {
service := &MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, nil},
ApiVersion: tt.apiVersion,
}
if a := service.IsAvailable(); a != tt.expect {
t.Fatalf("bad isAvailable (%q): want %q, got %q", tt.resources, tt.expect, a)
}
}
}
func TestFetchUserdata(t *testing.T) {
for _, tt := range []struct {
root string
userdataPath string
resources map[string]string
userdata []byte
clientErr error
expectErr error
}{
{
root: "/",
userdataPath: "2009-04-04/user-data",
resources: map[string]string{
"/2009-04-04/user-data": "hello",
},
userdata: []byte("hello"),
},
{
root: "/",
clientErr: pkg.ErrNotFound{fmt.Errorf("test not found error")},
userdata: []byte{},
},
{
root: "/",
clientErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
expectErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
},
} {
service := &MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, tt.clientErr},
UserdataPath: tt.userdataPath,
}
data, err := service.FetchUserdata()
if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
}
if !bytes.Equal(data, tt.userdata) {
t.Fatalf("bad userdata (%q): want %q, got %q", tt.resources, tt.userdata, data)
}
}
}
func TestUrls(t *testing.T) {
for _, tt := range []struct {
root string
userdataPath string
metadataPath string
expectRoot string
userdata string
metadata string
}{
{
root: "/",
userdataPath: "2009-04-04/user-data",
metadataPath: "2009-04-04/meta-data",
expectRoot: "/",
userdata: "/2009-04-04/user-data",
metadata: "/2009-04-04/meta-data",
},
{
root: "http://169.254.169.254/",
userdataPath: "2009-04-04/user-data",
metadataPath: "2009-04-04/meta-data",
expectRoot: "http://169.254.169.254/",
userdata: "http://169.254.169.254/2009-04-04/user-data",
metadata: "http://169.254.169.254/2009-04-04/meta-data",
},
} {
service := &MetadataService{
Root: tt.root,
UserdataPath: tt.userdataPath,
MetadataPath: tt.metadataPath,
}
if url := service.UserdataUrl(); url != tt.userdata {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.userdata, url)
}
if url := service.MetadataUrl(); url != tt.metadata {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.metadata, url)
}
if url := service.ConfigRoot(); url != tt.expectRoot {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.expectRoot, url)
}
}
}
func TestNewDatasource(t *testing.T) {
for _, tt := range []struct {
root string
expectRoot string
}{
{
root: "",
expectRoot: "/",
},
{
root: "/",
expectRoot: "/",
},
{
root: "http://169.254.169.254",
expectRoot: "http://169.254.169.254/",
},
{
root: "http://169.254.169.254/",
expectRoot: "http://169.254.169.254/",
},
} {
service := NewDatasource(tt.root, "", "", "")
if service.Root != tt.expectRoot {
t.Fatalf("bad root (%q): want %q, got %q", tt.root, tt.expectRoot, service.Root)
}
}
}
func Error(err error) string {
if err != nil {
return err.Error()
}
return ""
}

View File

@@ -0,0 +1,27 @@
package test
import (
"fmt"
"github.com/coreos/coreos-cloudinit/pkg"
)
type HttpClient struct {
Resources map[string]string
Err error
}
func (t *HttpClient) GetRetry(url string) ([]byte, error) {
if t.Err != nil {
return nil, t.Err
}
if val, ok := t.Resources[url]; ok {
return []byte(val), nil
} else {
return nil, pkg.ErrNotFound{fmt.Errorf("not found: %q", url)}
}
}
func (t *HttpClient) Get(url string) ([]byte, error) {
return t.GetRetry(url)
}

View File

@@ -66,6 +66,10 @@ func (c *procCmdline) FetchUserdata() ([]byte, error) {
return cfg, nil return cfg, nil
} }
func (c *procCmdline) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (c *procCmdline) Type() string { func (c *procCmdline) Type() string {
return "proc-cmdline" return "proc-cmdline"
} }

View File

@@ -1,6 +1,8 @@
package url package url
import "github.com/coreos/coreos-cloudinit/pkg" import (
"github.com/coreos/coreos-cloudinit/pkg"
)
type remoteFile struct { type remoteFile struct {
url string url string
@@ -33,6 +35,10 @@ func (f *remoteFile) FetchUserdata() ([]byte, error) {
return client.GetRetry(f.url) return client.GetRetry(f.url)
} }
func (f *remoteFile) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (f *remoteFile) Type() string { func (f *remoteFile) Type() string {
return "url" return "url"
} }

View File

@@ -3,11 +3,10 @@ package initialize
import ( import (
"errors" "errors"
"fmt" "fmt"
"io/ioutil"
"log" "log"
"path" "path"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml" "github.com/coreos/coreos-cloudinit/third_party/gopkg.in/yaml.v1"
"github.com/coreos/coreos-cloudinit/network" "github.com/coreos/coreos-cloudinit/network"
"github.com/coreos/coreos-cloudinit/system" "github.com/coreos/coreos-cloudinit/system"
@@ -42,6 +41,7 @@ type CloudConfig struct {
Users []system.User Users []system.User
ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"` ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
NetworkConfigPath string NetworkConfigPath string
NetworkConfig string
} }
type warner func(format string, v ...interface{}) type warner func(format string, v ...interface{})
@@ -51,12 +51,12 @@ type warner func(format string, v ...interface{})
func warnOnUnrecognizedKeys(contents string, warn warner) { func warnOnUnrecognizedKeys(contents string, warn warner) {
// Generate a map of all understood cloud config options // Generate a map of all understood cloud config options
var cc map[string]interface{} var cc map[string]interface{}
b, _ := goyaml.Marshal(&CloudConfig{}) b, _ := yaml.Marshal(&CloudConfig{})
goyaml.Unmarshal(b, &cc) yaml.Unmarshal(b, &cc)
// Now unmarshal the entire provided contents // Now unmarshal the entire provided contents
var c map[string]interface{} var c map[string]interface{}
goyaml.Unmarshal([]byte(contents), &c) yaml.Unmarshal([]byte(contents), &c)
// Check that every key in the contents exists in the cloud config // Check that every key in the contents exists in the cloud config
for k, _ := range c { for k, _ := range c {
@@ -84,8 +84,8 @@ func warnOnUnrecognizedKeys(contents string, warn warner) {
// Check for any badly-specified users, if any are set // Check for any badly-specified users, if any are set
if users, ok := c["users"]; ok { if users, ok := c["users"]; ok {
var known map[string]interface{} var known map[string]interface{}
b, _ := goyaml.Marshal(&system.User{}) b, _ := yaml.Marshal(&system.User{})
goyaml.Unmarshal(b, &known) yaml.Unmarshal(b, &known)
if set, ok := users.([]interface{}); ok { if set, ok := users.([]interface{}); ok {
for _, u := range set { for _, u := range set {
@@ -107,8 +107,8 @@ func warnOnUnrecognizedKeys(contents string, warn warner) {
// Check for any badly-specified files, if any are set // Check for any badly-specified files, if any are set
if files, ok := c["write_files"]; ok { if files, ok := c["write_files"]; ok {
var known map[string]interface{} var known map[string]interface{}
b, _ := goyaml.Marshal(&system.File{}) b, _ := yaml.Marshal(&system.File{})
goyaml.Unmarshal(b, &known) yaml.Unmarshal(b, &known)
if set, ok := files.([]interface{}); ok { if set, ok := files.([]interface{}); ok {
for _, f := range set { for _, f := range set {
@@ -133,7 +133,7 @@ func warnOnUnrecognizedKeys(contents string, warn warner) {
// fields but log encountering them. // fields but log encountering them.
func NewCloudConfig(contents string) (*CloudConfig, error) { func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig var cfg CloudConfig
err := goyaml.Unmarshal([]byte(contents), &cfg) err := yaml.Unmarshal([]byte(contents), &cfg)
if err != nil { if err != nil {
return &cfg, err return &cfg, err
} }
@@ -142,7 +142,7 @@ func NewCloudConfig(contents string) (*CloudConfig, error) {
} }
func (cc CloudConfig) String() string { func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc) bytes, err := yaml.Marshal(cc)
if err != nil { if err != nil {
return "" return ""
} }
@@ -258,17 +258,13 @@ func Apply(cfg CloudConfig, env *Environment) error {
} }
if env.NetconfType() != "" { if env.NetconfType() != "" {
filename := path.Join(env.ConfigRoot(), cfg.NetworkConfigPath)
log.Printf("Attempting to read config from %q\n", filename)
netconfBytes, err := ioutil.ReadFile(filename)
if err != nil {
return err
}
var interfaces []network.InterfaceGenerator var interfaces []network.InterfaceGenerator
var err error
switch env.NetconfType() { switch env.NetconfType() {
case "debian": case "debian":
interfaces, err = network.ProcessDebianNetconf(string(netconfBytes)) interfaces, err = network.ProcessDebianNetconf(cfg.NetworkConfig)
case "digitalocean":
interfaces, err = network.ProcessDigitalOceanNetconf(cfg.NetworkConfig)
default: default:
return fmt.Errorf("Unsupported network config format %q", env.NetconfType()) return fmt.Errorf("Unsupported network config format %q", env.NetconfType())
} }
@@ -295,7 +291,11 @@ func Apply(cfg CloudConfig, env *Environment) error {
// disk, masking/unmasking units, or invoking systemd // disk, masking/unmasking units, or invoking systemd
// commands against units. It returns any error encountered. // commands against units. It returns any error encountered.
func processUnits(units []system.Unit, root string, um system.UnitManager) error { func processUnits(units []system.Unit, root string, um system.UnitManager) error {
commands := make(map[string]string, 0) type action struct {
unit string
command string
}
actions := make([]action, 0, len(units))
reload := false reload := false
for _, unit := range units { for _, unit := range units {
dst := unit.Destination(root) dst := unit.Destination(root)
@@ -333,9 +333,9 @@ func processUnits(units []system.Unit, root string, um system.UnitManager) error
} }
if unit.Group() == "network" { if unit.Group() == "network" {
commands["systemd-networkd.service"] = "restart" actions = append(actions, action{"systemd-networkd.service", "restart"})
} else if unit.Command != "" { } else if unit.Command != "" {
commands[unit.Name] = unit.Command actions = append(actions, action{unit.Name, unit.Command})
} }
} }
@@ -345,13 +345,13 @@ func processUnits(units []system.Unit, root string, um system.UnitManager) error
} }
} }
for unit, command := range commands { for _, action := range actions {
log.Printf("Calling unit command '%s %s'", command, unit) log.Printf("Calling unit command '%s %s'", action.command, action.unit)
res, err := um.RunUnitCommand(command, unit) res, err := um.RunUnitCommand(action.command, action.unit)
if err != nil { if err != nil {
return err return err
} }
log.Printf("Result of '%s %s': %s", command, unit, res) log.Printf("Result of '%s %s': %s", action.command, action.unit, res)
} }
return nil return nil

View File

@@ -3,6 +3,7 @@ package initialize
import ( import (
"os" "os"
"path" "path"
"regexp"
"strings" "strings"
"github.com/coreos/coreos-cloudinit/system" "github.com/coreos/coreos-cloudinit/system"
@@ -28,6 +29,8 @@ func NewEnvironment(root, configRoot, workspace, netconfType, sshKeyName string,
for k, v := range map[string]string{ for k, v := range map[string]string{
"$public_ipv4": os.Getenv("COREOS_PUBLIC_IPV4"), "$public_ipv4": os.Getenv("COREOS_PUBLIC_IPV4"),
"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"), "$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
"$public_ipv6": os.Getenv("COREOS_PUBLIC_IPV6"),
"$private_ipv6": os.Getenv("COREOS_PRIVATE_IPV6"),
} { } {
if _, ok := substitutions[k]; !ok { if _, ok := substitutions[k]; !ok {
substitutions[k] = v substitutions[k] = v
@@ -60,9 +63,18 @@ func (e *Environment) SetSSHKeyName(name string) {
e.sshKeyName = name e.sshKeyName = name
} }
// Apply goes through the map of substitutions and replaces all instances of
// the keys with their respective values. It supports escaping substitutions
// with a leading '\'.
func (e *Environment) Apply(data string) string { func (e *Environment) Apply(data string) string {
for key, val := range e.substitutions { for key, val := range e.substitutions {
data = strings.Replace(data, key, val, -1) matchKey := strings.Replace(key, `$`, `\$`, -1)
replKey := strings.Replace(key, `$`, `$$`, -1)
// "key" -> "val"
data = regexp.MustCompile(`([^\\]|^)`+matchKey).ReplaceAllString(data, `${1}`+val)
// "\key" -> "key"
data = regexp.MustCompile(`\\`+matchKey).ReplaceAllString(data, replKey)
} }
return data return data
} }
@@ -80,6 +92,12 @@ func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
if ip, ok := e.substitutions["$private_ipv4"]; ok && len(ip) > 0 { if ip, ok := e.substitutions["$private_ipv4"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PRIVATE_IPV4"] = ip ef.Vars["COREOS_PRIVATE_IPV4"] = ip
} }
if ip, ok := e.substitutions["$public_ipv6"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PUBLIC_IPV6"] = ip
}
if ip, ok := e.substitutions["$private_ipv6"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PRIVATE_IPV6"] = ip
}
if len(ef.Vars) == 0 { if len(ef.Vars) == 0 {
return nil return nil
} else { } else {

View File

@@ -12,6 +12,8 @@ import (
func TestEnvironmentApply(t *testing.T) { func TestEnvironmentApply(t *testing.T) {
os.Setenv("COREOS_PUBLIC_IPV4", "1.2.3.4") os.Setenv("COREOS_PUBLIC_IPV4", "1.2.3.4")
os.Setenv("COREOS_PRIVATE_IPV4", "5.6.7.8") os.Setenv("COREOS_PRIVATE_IPV4", "5.6.7.8")
os.Setenv("COREOS_PUBLIC_IPV6", "1234::")
os.Setenv("COREOS_PRIVATE_IPV6", "5678::")
for _, tt := range []struct { for _, tt := range []struct {
subs map[string]string subs map[string]string
input string input string
@@ -23,14 +25,16 @@ func TestEnvironmentApply(t *testing.T) {
map[string]string{ map[string]string{
"$public_ipv4": "192.0.2.3", "$public_ipv4": "192.0.2.3",
"$private_ipv4": "192.0.2.203", "$private_ipv4": "192.0.2.203",
"$public_ipv6": "fe00:1234::",
"$private_ipv6": "fe00:5678::",
}, },
`[Service] `[Service]
ExecStart=/usr/bin/echo "$public_ipv4" ExecStart=/usr/bin/echo "$public_ipv4 $public_ipv6"
ExecStop=/usr/bin/echo $private_ipv4 ExecStop=/usr/bin/echo $private_ipv4 $private_ipv6
ExecStop=/usr/bin/echo $unknown`, ExecStop=/usr/bin/echo $unknown`,
`[Service] `[Service]
ExecStart=/usr/bin/echo "192.0.2.3" ExecStart=/usr/bin/echo "192.0.2.3 fe00:1234::"
ExecStop=/usr/bin/echo 192.0.2.203 ExecStop=/usr/bin/echo 192.0.2.203 fe00:5678::
ExecStop=/usr/bin/echo $unknown`, ExecStop=/usr/bin/echo $unknown`,
}, },
{ {
@@ -51,6 +55,24 @@ ExecStop=/usr/bin/echo $unknown`,
"$private_ipv4\nfoobar", "$private_ipv4\nfoobar",
"5.6.7.8\nfoobar", "5.6.7.8\nfoobar",
}, },
{
// Escaping substitutions
map[string]string{"$private_ipv4": "127.0.0.1"},
`\$private_ipv4
$private_ipv4
addr: \$private_ipv4
\\$private_ipv4`,
`$private_ipv4
127.0.0.1
addr: $private_ipv4
\$private_ipv4`,
},
{
// No substitutions with escaping
nil,
"\\$test\n$test",
"\\$test\n$test",
},
} { } {
env := NewEnvironment("./", "./", "./", "", "", tt.subs) env := NewEnvironment("./", "./", "./", "", "", tt.subs)
@@ -65,8 +87,10 @@ func TestEnvironmentFile(t *testing.T) {
subs := map[string]string{ subs := map[string]string{
"$public_ipv4": "1.2.3.4", "$public_ipv4": "1.2.3.4",
"$private_ipv4": "5.6.7.8", "$private_ipv4": "5.6.7.8",
"$public_ipv6": "1234::",
"$private_ipv6": "5678::",
} }
expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PUBLIC_IPV4=1.2.3.4\n" expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PRIVATE_IPV6=5678::\nCOREOS_PUBLIC_IPV4=1.2.3.4\nCOREOS_PUBLIC_IPV6=1234::\n"
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-") dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil { if err != nil {
@@ -96,6 +120,8 @@ func TestEnvironmentFileNil(t *testing.T) {
subs := map[string]string{ subs := map[string]string{
"$public_ipv4": "", "$public_ipv4": "",
"$private_ipv4": "", "$private_ipv4": "",
"$public_ipv6": "",
"$private_ipv6": "",
} }
env := NewEnvironment("./", "./", "./", "", "", subs) env := NewEnvironment("./", "./", "./", "", "", subs)

View File

@@ -1,9 +1,12 @@
package initialize package initialize
import "encoding/json" import (
"encoding/json"
"sort"
)
// ParseMetaData parses a JSON blob in the OpenStack metadata service format, and // ParseMetaData parses a JSON blob in the OpenStack metadata service format,
// converts it to a partially hydrated CloudConfig // and converts it to a partially hydrated CloudConfig.
func ParseMetaData(contents string) (*CloudConfig, error) { func ParseMetaData(contents string) (*CloudConfig, error) {
if len(contents) == 0 { if len(contents) == 0 {
return nil, nil return nil, nil
@@ -22,8 +25,8 @@ func ParseMetaData(contents string) (*CloudConfig, error) {
var cfg CloudConfig var cfg CloudConfig
if len(metadata.SSHAuthorizedKeyMap) > 0 { if len(metadata.SSHAuthorizedKeyMap) > 0 {
cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap)) cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap))
for _, key := range metadata.SSHAuthorizedKeyMap { for _, name := range sortedKeys(metadata.SSHAuthorizedKeyMap) {
cfg.SSHAuthorizedKeys = append(cfg.SSHAuthorizedKeys, key) cfg.SSHAuthorizedKeys = append(cfg.SSHAuthorizedKeys, metadata.SSHAuthorizedKeyMap[name])
} }
} }
cfg.Hostname = metadata.Hostname cfg.Hostname = metadata.Hostname
@@ -31,22 +34,39 @@ func ParseMetaData(contents string) (*CloudConfig, error) {
return &cfg, nil return &cfg, nil
} }
// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service format, // ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service
// and returns a substitution map possibly containing private_ipv4 and public_ipv4 addresses // format and returns a substitution map possibly containing private_ipv4,
// public_ipv4, private_ipv6, and public_ipv6 addresses.
func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) { func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) {
var ips struct { var ips struct {
Public string `json:"public-ipv4"` PublicIPv4 string `json:"public-ipv4"`
Private string `json:"local-ipv4"` PrivateIPv4 string `json:"local-ipv4"`
PublicIPv6 string `json:"public-ipv6"`
PrivateIPv6 string `json:"local-ipv6"`
} }
if err := json.Unmarshal(contents, &ips); err != nil { if err := json.Unmarshal(contents, &ips); err != nil {
return nil, err return nil, err
} }
m := make(map[string]string) m := make(map[string]string)
if ips.Private != "" { if ips.PrivateIPv4 != "" {
m["$private_ipv4"] = ips.Private m["$private_ipv4"] = ips.PrivateIPv4
} }
if ips.Public != "" { if ips.PublicIPv4 != "" {
m["$public_ipv4"] = ips.Public m["$public_ipv4"] = ips.PublicIPv4
}
if ips.PrivateIPv6 != "" {
m["$private_ipv6"] = ips.PrivateIPv6
}
if ips.PublicIPv6 != "" {
m["$public_ipv6"] = ips.PublicIPv6
} }
return m, nil return m, nil
} }
func sortedKeys(m map[string]string) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}

View File

@@ -14,7 +14,7 @@ func TestParseMetadata(t *testing.T) {
{`{"foo": "bar"}`, &CloudConfig{}, false}, {`{"foo": "bar"}`, &CloudConfig{}, false},
{`{"network_config": {"content_path": "asdf"}}`, &CloudConfig{NetworkConfigPath: "asdf"}, false}, {`{"network_config": {"content_path": "asdf"}}`, &CloudConfig{NetworkConfigPath: "asdf"}, false},
{`{"hostname": "turkleton"}`, &CloudConfig{Hostname: "turkleton"}, false}, {`{"hostname": "turkleton"}`, &CloudConfig{Hostname: "turkleton"}, false},
{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"jill", "alice"}}, false}, {`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"alice", "jill"}}, false},
{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false}, {`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
} { } {
got, err := ParseMetaData(tt.in) got, err := ParseMetaData(tt.in)
@@ -43,9 +43,9 @@ func TestExtractIPsFromMetadata(t *testing.T) {
out map[string]string out map[string]string
}{ }{
{ {
[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4"}`), []byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4", "public-ipv6": "1234::", "local-ipv6": "5678::"}`),
false, false,
map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4"}, map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4", "$public_ipv6": "1234::", "$private_ipv6": "5678::"},
}, },
{ {
[]byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`), []byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`),

View File

@@ -16,7 +16,7 @@ func ParseUserData(contents string) (interface{}, error) {
// Explicitly trim the header so we can handle user-data from // Explicitly trim the header so we can handle user-data from
// non-unix operating systems. The rest of the file is parsed // non-unix operating systems. The rest of the file is parsed
// by goyaml, which correctly handles CRLF. // by yaml, which correctly handles CRLF.
header = strings.TrimSpace(header) header = strings.TrimSpace(header)
if strings.HasPrefix(header, "#!") { if strings.HasPrefix(header, "#!") {

View File

@@ -1,10 +1,12 @@
package network package network
import ( import (
"log"
"strings" "strings"
) )
func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) { func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) {
log.Println("Processing Debian network config")
lines := formatConfig(config) lines := formatConfig(config)
stanzas, err := parseStanzas(lines) stanzas, err := parseStanzas(lines)
if err != nil { if err != nil {
@@ -18,7 +20,9 @@ func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) {
interfaces = append(interfaces, s) interfaces = append(interfaces, s)
} }
} }
log.Printf("Parsed %d network interfaces\n", len(interfaces))
log.Println("Processed Debian network config")
return buildInterfaces(interfaces), nil return buildInterfaces(interfaces), nil
} }

142
network/digitalocean.go Normal file
View File

@@ -0,0 +1,142 @@
package network
import (
"encoding/json"
"fmt"
"log"
"net"
"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
)
func ProcessDigitalOceanNetconf(config string) ([]InterfaceGenerator, error) {
log.Println("Processing DigitalOcean network config")
if config == "" {
return nil, nil
}
var cfg digitalocean.Metadata
if err := json.Unmarshal([]byte(config), &cfg); err != nil {
return nil, err
}
log.Println("Parsing nameservers")
nameservers, err := parseNameservers(cfg.DNS)
if err != nil {
return nil, err
}
log.Printf("Parsed %d nameservers\n", len(nameservers))
log.Println("Parsing interfaces")
generators, err := parseInterfaces(cfg.Interfaces, nameservers)
if err != nil {
return nil, err
}
log.Printf("Parsed %d network interfaces\n", len(generators))
log.Println("Processed DigitalOcean network config")
return generators, nil
}
func parseNameservers(cfg digitalocean.DNS) ([]net.IP, error) {
nameservers := make([]net.IP, 0, len(cfg.Nameservers))
for _, ns := range cfg.Nameservers {
if ip := net.ParseIP(ns); ip == nil {
return nil, fmt.Errorf("could not parse %q as nameserver IP address", ns)
} else {
nameservers = append(nameservers, ip)
}
}
return nameservers, nil
}
func parseInterfaces(cfg digitalocean.Interfaces, nameservers []net.IP) ([]InterfaceGenerator, error) {
generators := make([]InterfaceGenerator, 0, len(cfg.Public)+len(cfg.Private))
for _, iface := range cfg.Public {
if generator, err := parseInterface(iface, nameservers, true); err == nil {
generators = append(generators, &physicalInterface{*generator})
} else {
return nil, err
}
}
for _, iface := range cfg.Private {
if generator, err := parseInterface(iface, []net.IP{}, false); err == nil {
generators = append(generators, &physicalInterface{*generator})
} else {
return nil, err
}
}
return generators, nil
}
func parseInterface(iface digitalocean.Interface, nameservers []net.IP, useRoute bool) (*logicalInterface, error) {
routes := make([]route, 0)
addresses := make([]net.IPNet, 0)
if iface.IPv4 != nil {
var ip, mask, gateway net.IP
if ip = net.ParseIP(iface.IPv4.IPAddress); ip == nil {
return nil, fmt.Errorf("could not parse %q as IPv4 address", iface.IPv4.IPAddress)
}
if mask = net.ParseIP(iface.IPv4.Netmask); mask == nil {
return nil, fmt.Errorf("could not parse %q as IPv4 mask", iface.IPv4.Netmask)
}
addresses = append(addresses, net.IPNet{
IP: ip,
Mask: net.IPMask(mask),
})
if useRoute {
if gateway = net.ParseIP(iface.IPv4.Gateway); gateway == nil {
return nil, fmt.Errorf("could not parse %q as IPv4 gateway", iface.IPv4.Gateway)
}
routes = append(routes, route{
destination: net.IPNet{
IP: net.IPv4zero,
Mask: net.IPMask(net.IPv4zero),
},
gateway: gateway,
})
}
}
if iface.IPv6 != nil {
var ip, gateway net.IP
if ip = net.ParseIP(iface.IPv6.IPAddress); ip == nil {
return nil, fmt.Errorf("could not parse %q as IPv6 address", iface.IPv6.IPAddress)
}
addresses = append(addresses, net.IPNet{
IP: ip,
Mask: net.CIDRMask(iface.IPv6.Cidr, net.IPv6len*8),
})
if useRoute {
if gateway = net.ParseIP(iface.IPv6.Gateway); gateway == nil {
return nil, fmt.Errorf("could not parse %q as IPv6 gateway", iface.IPv6.Gateway)
}
routes = append(routes, route{
destination: net.IPNet{
IP: net.IPv6zero,
Mask: net.IPMask(net.IPv6zero),
},
gateway: gateway,
})
}
}
hwaddr, err := net.ParseMAC(iface.MAC)
if err != nil {
return nil, err
}
if nameservers == nil {
nameservers = []net.IP{}
}
return &logicalInterface{
hwaddr: hwaddr,
config: configMethodStatic{
addresses: addresses,
nameservers: nameservers,
routes: routes,
},
}, nil
}

View File

@@ -0,0 +1,367 @@
package network
import (
"errors"
"net"
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
)
func TestParseNameservers(t *testing.T) {
for _, tt := range []struct {
dns digitalocean.DNS
nss []net.IP
err error
}{
{
dns: digitalocean.DNS{},
nss: []net.IP{},
},
{
dns: digitalocean.DNS{[]string{"1.2.3.4"}},
nss: []net.IP{net.ParseIP("1.2.3.4")},
},
{
dns: digitalocean.DNS{[]string{"bad"}},
err: errors.New("could not parse \"bad\" as nameserver IP address"),
},
} {
nss, err := parseNameservers(tt.dns)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%+v): want %q, got %q", tt.dns, tt.err, err)
}
if !reflect.DeepEqual(tt.nss, nss) {
t.Fatalf("bad nameservers (%+v): want %#v, got %#v", tt.dns, tt.nss, nss)
}
}
}
func TestParseInterface(t *testing.T) {
for _, tt := range []struct {
cfg digitalocean.Interface
nss []net.IP
useRoute bool
iface *logicalInterface
err error
}{
{
cfg: digitalocean.Interface{
MAC: "bad",
},
err: errors.New("invalid MAC address: bad"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
},
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
},
useRoute: true,
nss: []net.IP{net.ParseIP("1.2.3.4")},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{net.ParseIP("1.2.3.4")},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "bad",
Netmask: "255.255.0.0",
},
},
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv4 address"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "bad",
},
},
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv4 mask"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "255.255.0.0",
Gateway: "ignoreme",
},
},
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("1.2.3.4"), net.IPMask(net.ParseIP("255.255.0.0"))}},
nameservers: []net.IP{},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "255.255.0.0",
Gateway: "bad",
},
},
useRoute: true,
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv4 gateway"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "255.255.0.0",
Gateway: "5.6.7.8",
},
},
useRoute: true,
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("1.2.3.4"), net.IPMask(net.ParseIP("255.255.0.0"))}},
nameservers: []net.IP{},
routes: []route{route{net.IPNet{net.IPv4zero, net.IPMask(net.IPv4zero)}, net.ParseIP("5.6.7.8")}},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "bad",
Cidr: 16,
},
},
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv6 address"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "fe00::",
Cidr: 16,
Gateway: "ignoreme",
},
},
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("fe00::"), net.IPMask(net.ParseIP("ffff::"))}},
nameservers: []net.IP{},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "fe00::",
Cidr: 16,
Gateway: "bad",
},
},
useRoute: true,
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv6 gateway"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "fe00::",
Cidr: 16,
Gateway: "fe00:1234::",
},
},
useRoute: true,
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("fe00::"), net.IPMask(net.ParseIP("ffff::"))}},
nameservers: []net.IP{},
routes: []route{route{net.IPNet{net.IPv6zero, net.IPMask(net.IPv6zero)}, net.ParseIP("fe00:1234::")}},
},
},
},
} {
iface, err := parseInterface(tt.cfg, tt.nss, tt.useRoute)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%+v): want %q, got %q", tt.cfg, tt.err, err)
}
if !reflect.DeepEqual(tt.iface, iface) {
t.Fatalf("bad interface (%+v): want %#v, got %#v", tt.cfg, tt.iface, iface)
}
}
}
func TestParseInterfaces(t *testing.T) {
for _, tt := range []struct {
cfg digitalocean.Interfaces
nss []net.IP
ifaces []InterfaceGenerator
err error
}{
{
ifaces: []InterfaceGenerator{},
},
{
cfg: digitalocean.Interfaces{
Public: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Private: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Public: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
nss: []net.IP{net.ParseIP("1.2.3.4")},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{net.ParseIP("1.2.3.4")},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Private: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
nss: []net.IP{net.ParseIP("1.2.3.4")},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Public: []digitalocean.Interface{{MAC: "bad"}},
},
err: errors.New("invalid MAC address: bad"),
},
{
cfg: digitalocean.Interfaces{
Private: []digitalocean.Interface{{MAC: "bad"}},
},
err: errors.New("invalid MAC address: bad"),
},
} {
ifaces, err := parseInterfaces(tt.cfg, tt.nss)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%+v): want %q, got %q", tt.cfg, tt.err, err)
}
if !reflect.DeepEqual(tt.ifaces, ifaces) {
t.Fatalf("bad interfaces (%+v): want %#v, got %#v", tt.cfg, tt.ifaces, ifaces)
}
}
}
func TestProcessDigitalOceanNetconf(t *testing.T) {
for _, tt := range []struct {
cfg string
ifaces []InterfaceGenerator
err error
}{
{
cfg: ``,
},
{
cfg: `{"dns":{"nameservers":["bad"]}}`,
err: errors.New("could not parse \"bad\" as nameserver IP address"),
},
{
cfg: `{"interfaces":{"public":[{"ipv4":{"ip_address":"bad"}}]}}`,
err: errors.New("could not parse \"bad\" as IPv4 address"),
},
{
cfg: `{}`,
ifaces: []InterfaceGenerator{},
},
} {
ifaces, err := ProcessDigitalOceanNetconf(tt.cfg)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%q): want %q, got %q", tt.cfg, tt.err, err)
}
if !reflect.DeepEqual(tt.ifaces, ifaces) {
t.Fatalf("bad interfaces (%q): want %#v, got %#v", tt.cfg, tt.ifaces, ifaces)
}
}
}
func errorsEqual(a, b error) bool {
if a == nil && b == nil {
return true
}
if (a != nil && b == nil) || (a == nil && b != nil) {
return false
}
return (a.Error() == b.Error())
}

View File

@@ -2,6 +2,8 @@ package network
import ( import (
"fmt" "fmt"
"net"
"sort"
"strconv" "strconv"
"strings" "strings"
) )
@@ -24,13 +26,25 @@ type networkInterface interface {
type logicalInterface struct { type logicalInterface struct {
name string name string
hwaddr net.HardwareAddr
config configMethod config configMethod
children []networkInterface children []networkInterface
configDepth int configDepth int
} }
func (i *logicalInterface) Name() string {
return i.name
}
func (i *logicalInterface) Network() string { func (i *logicalInterface) Network() string {
config := fmt.Sprintf("[Match]\nName=%s\n\n[Network]\n", i.name) config := fmt.Sprintln("[Match]")
if i.name != "" {
config += fmt.Sprintf("Name=%s\n", i.name)
}
if i.hwaddr != nil {
config += fmt.Sprintf("MACAddress=%s\n", i.hwaddr)
}
config += "\n[Network]\n"
for _, child := range i.children { for _, child := range i.children {
switch iface := child.(type) { switch iface := child.(type) {
@@ -46,8 +60,8 @@ func (i *logicalInterface) Network() string {
for _, nameserver := range conf.nameservers { for _, nameserver := range conf.nameservers {
config += fmt.Sprintf("DNS=%s\n", nameserver) config += fmt.Sprintf("DNS=%s\n", nameserver)
} }
if conf.address.IP != nil { for _, addr := range conf.addresses {
config += fmt.Sprintf("\n[Address]\nAddress=%s\n", conf.address.String()) config += fmt.Sprintf("\n[Address]\nAddress=%s\n", addr.String())
} }
for _, route := range conf.routes { for _, route := range conf.routes {
config += fmt.Sprintf("\n[Route]\nDestination=%s\nGateway=%s\n", route.destination.String(), route.gateway) config += fmt.Sprintf("\n[Route]\nDestination=%s\nGateway=%s\n", route.destination.String(), route.gateway)
@@ -63,8 +77,16 @@ func (i *logicalInterface) Link() string {
return "" return ""
} }
func (i *logicalInterface) Netdev() string {
return ""
}
func (i *logicalInterface) Filename() string { func (i *logicalInterface) Filename() string {
return fmt.Sprintf("%02x-%s", i.configDepth, i.name) name := i.name
if name == "" {
name = i.hwaddr.String()
}
return fmt.Sprintf("%02x-%s", i.configDepth, name)
} }
func (i *logicalInterface) Children() []networkInterface { func (i *logicalInterface) Children() []networkInterface {
@@ -83,14 +105,6 @@ type physicalInterface struct {
logicalInterface logicalInterface
} }
func (p *physicalInterface) Name() string {
return p.name
}
func (p *physicalInterface) Netdev() string {
return ""
}
func (p *physicalInterface) Type() string { func (p *physicalInterface) Type() string {
return "physical" return "physical"
} }
@@ -101,10 +115,6 @@ type bondInterface struct {
options map[string]string options map[string]string
} }
func (b *bondInterface) Name() string {
return b.name
}
func (b *bondInterface) Netdev() string { func (b *bondInterface) Netdev() string {
return fmt.Sprintf("[NetDev]\nKind=bond\nName=%s\n", b.name) return fmt.Sprintf("[NetDev]\nKind=bond\nName=%s\n", b.name)
} }
@@ -115,8 +125,8 @@ func (b *bondInterface) Type() string {
func (b *bondInterface) ModprobeParams() string { func (b *bondInterface) ModprobeParams() string {
params := "" params := ""
for name, val := range b.options { for _, name := range sortedKeys(b.options) {
params += fmt.Sprintf("%s=%s ", name, val) params += fmt.Sprintf("%s=%s ", name, b.options[name])
} }
params = strings.TrimSuffix(params, " ") params = strings.TrimSuffix(params, " ")
return params return params
@@ -128,10 +138,6 @@ type vlanInterface struct {
rawDevice string rawDevice string
} }
func (v *vlanInterface) Name() string {
return v.name
}
func (v *vlanInterface) Netdev() string { func (v *vlanInterface) Netdev() string {
config := fmt.Sprintf("[NetDev]\nKind=vlan\nName=%s\n", v.name) config := fmt.Sprintf("[NetDev]\nKind=vlan\nName=%s\n", v.name)
switch c := v.config.(type) { switch c := v.config.(type) {
@@ -158,8 +164,8 @@ func buildInterfaces(stanzas []*stanzaInterface) []InterfaceGenerator {
markConfigDepths(interfaceMap) markConfigDepths(interfaceMap)
interfaces := make([]InterfaceGenerator, 0, len(interfaceMap)) interfaces := make([]InterfaceGenerator, 0, len(interfaceMap))
for _, iface := range interfaceMap { for _, name := range sortedInterfaces(interfaceMap) {
interfaces = append(interfaces, iface) interfaces = append(interfaces, interfaceMap[name])
} }
return interfaces return interfaces
@@ -239,7 +245,8 @@ func createInterfaces(stanzas []*stanzaInterface) map[string]networkInterface {
} }
func linkAncestors(interfaceMap map[string]networkInterface) { func linkAncestors(interfaceMap map[string]networkInterface) {
for _, iface := range interfaceMap { for _, name := range sortedInterfaces(interfaceMap) {
iface := interfaceMap[name]
switch i := iface.(type) { switch i := iface.(type) {
case *vlanInterface: case *vlanInterface:
if parent, ok := interfaceMap[i.rawDevice]; ok { if parent, ok := interfaceMap[i.rawDevice]; ok {
@@ -291,3 +298,19 @@ func setDepth(iface networkInterface) int {
iface.setConfigDepth(maxDepth) iface.setConfigDepth(maxDepth)
return (maxDepth + 1) return (maxDepth + 1)
} }
func sortedKeys(m map[string]string) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}
func sortedInterfaces(m map[string]networkInterface) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}

View File

@@ -6,241 +6,101 @@ import (
"testing" "testing"
) )
func TestPhysicalInterfaceName(t *testing.T) { func TestInterfaceGenerators(t *testing.T) {
p := physicalInterface{logicalInterface{name: "testname"}} for _, tt := range []struct {
if p.Name() != "testname" { name string
t.FailNow() netdev string
} link string
} network string
kind string
func TestPhysicalInterfaceNetdev(t *testing.T) { iface InterfaceGenerator
p := physicalInterface{} }{
if p.Netdev() != "" { {
t.FailNow() name: "",
} network: "[Match]\nMACAddress=00:01:02:03:04:05\n\n[Network]\n",
} kind: "physical",
iface: &physicalInterface{logicalInterface{
func TestPhysicalInterfaceLink(t *testing.T) { hwaddr: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5}),
p := physicalInterface{} }},
if p.Link() != "" { },
t.FailNow() {
} name: "testname",
} network: "[Match]\nName=testname\n\n[Network]\nBond=testbond1\nVLAN=testvlan1\nVLAN=testvlan2\n",
kind: "physical",
func TestPhysicalInterfaceNetwork(t *testing.T) { iface: &physicalInterface{logicalInterface{
p := physicalInterface{logicalInterface{
name: "testname", name: "testname",
children: []networkInterface{ children: []networkInterface{
&bondInterface{ &bondInterface{logicalInterface: logicalInterface{name: "testbond1"}},
logicalInterface{ &vlanInterface{logicalInterface: logicalInterface{name: "testvlan1"}, id: 1},
name: "testbond1", &vlanInterface{logicalInterface: logicalInterface{name: "testvlan2"}, id: 1},
}, },
nil, }},
nil,
}, },
&vlanInterface{ {
logicalInterface{ name: "testname",
name: "testvlan1", netdev: "[NetDev]\nKind=bond\nName=testname\n",
}, network: "[Match]\nName=testname\n\n[Network]\nBond=testbond1\nVLAN=testvlan1\nVLAN=testvlan2\nDHCP=true\n",
1, kind: "bond",
"", iface: &bondInterface{logicalInterface: logicalInterface{
},
&vlanInterface{
logicalInterface{
name: "testvlan2",
},
1,
"",
},
},
}}
network := `[Match]
Name=testname
[Network]
Bond=testbond1
VLAN=testvlan1
VLAN=testvlan2
`
if p.Network() != network {
t.FailNow()
}
}
func TestBondInterfaceName(t *testing.T) {
b := bondInterface{logicalInterface{name: "testname"}, nil, nil}
if b.Name() != "testname" {
t.FailNow()
}
}
func TestBondInterfaceNetdev(t *testing.T) {
b := bondInterface{logicalInterface{name: "testname"}, nil, nil}
netdev := `[NetDev]
Kind=bond
Name=testname
`
if b.Netdev() != netdev {
t.FailNow()
}
}
func TestBondInterfaceLink(t *testing.T) {
b := bondInterface{}
if b.Link() != "" {
t.FailNow()
}
}
func TestBondInterfaceNetwork(t *testing.T) {
b := bondInterface{
logicalInterface{
name: "testname", name: "testname",
config: configMethodDHCP{}, config: configMethodDHCP{},
children: []networkInterface{ children: []networkInterface{
&bondInterface{ &bondInterface{logicalInterface: logicalInterface{name: "testbond1"}},
logicalInterface{ &vlanInterface{logicalInterface: logicalInterface{name: "testvlan1"}, id: 1},
name: "testbond1", &vlanInterface{logicalInterface: logicalInterface{name: "testvlan2"}, id: 1},
}, },
nil, }},
nil,
},
&vlanInterface{
logicalInterface{
name: "testvlan1",
},
1,
"",
},
&vlanInterface{
logicalInterface{
name: "testvlan2",
},
1,
"",
},
},
},
nil,
nil,
}
network := `[Match]
Name=testname
[Network]
Bond=testbond1
VLAN=testvlan1
VLAN=testvlan2
DHCP=true
`
if b.Network() != network {
t.FailNow()
}
}
func TestVLANInterfaceName(t *testing.T) {
v := vlanInterface{logicalInterface{name: "testname"}, 1, ""}
if v.Name() != "testname" {
t.FailNow()
}
}
func TestVLANInterfaceNetdev(t *testing.T) {
for _, tt := range []struct {
i vlanInterface
l string
}{
{
vlanInterface{logicalInterface{name: "testname"}, 1, ""},
"[NetDev]\nKind=vlan\nName=testname\n\n[VLAN]\nId=1\n",
}, },
{ {
vlanInterface{logicalInterface{name: "testname", config: configMethodStatic{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""}, name: "testname",
"[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n", netdev: "[NetDev]\nKind=vlan\nName=testname\n\n[VLAN]\nId=1\n",
network: "[Match]\nName=testname\n\n[Network]\n",
kind: "vlan",
iface: &vlanInterface{logicalInterface{name: "testname"}, 1, ""},
}, },
{ {
vlanInterface{logicalInterface{name: "testname", config: configMethodDHCP{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""}, name: "testname",
"[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n", netdev: "[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n",
network: "[Match]\nName=testname\n\n[Network]\n",
kind: "vlan",
iface: &vlanInterface{logicalInterface{name: "testname", config: configMethodStatic{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""},
}, },
} { {
if tt.i.Netdev() != tt.l { name: "testname",
t.Fatalf("bad netdev config (%q): got %q, want %q", tt.i, tt.i.Netdev(), tt.l) netdev: "[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n",
} network: "[Match]\nName=testname\n\n[Network]\nDHCP=true\n",
} kind: "vlan",
} iface: &vlanInterface{logicalInterface{name: "testname", config: configMethodDHCP{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""},
},
func TestVLANInterfaceLink(t *testing.T) { {
v := vlanInterface{} name: "testname",
if v.Link() != "" { netdev: "[NetDev]\nKind=vlan\nName=testname\n\n[VLAN]\nId=0\n",
t.FailNow() network: "[Match]\nName=testname\n\n[Network]\nDNS=8.8.8.8\n\n[Address]\nAddress=192.168.1.100/24\n\n[Route]\nDestination=0.0.0.0/0\nGateway=1.2.3.4\n",
} kind: "vlan",
} iface: &vlanInterface{logicalInterface: logicalInterface{
func TestVLANInterfaceNetwork(t *testing.T) {
v := vlanInterface{
logicalInterface{
name: "testname", name: "testname",
config: configMethodStatic{ config: configMethodStatic{
address: net.IPNet{ addresses: []net.IPNet{{IP: []byte{192, 168, 1, 100}, Mask: []byte{255, 255, 255, 0}}},
IP: []byte{192, 168, 1, 100}, nameservers: []net.IP{[]byte{8, 8, 8, 8}},
Mask: []byte{255, 255, 255, 0}, routes: []route{route{destination: net.IPNet{IP: []byte{0, 0, 0, 0}, Mask: []byte{0, 0, 0, 0}}, gateway: []byte{1, 2, 3, 4}}},
}, },
nameservers: []net.IP{ }},
[]byte{8, 8, 8, 8},
},
routes: []route{
route{
destination: net.IPNet{
IP: []byte{0, 0, 0, 0},
Mask: []byte{0, 0, 0, 0},
},
gateway: []byte{1, 2, 3, 4},
},
},
},
},
0,
"",
}
network := `[Match]
Name=testname
[Network]
DNS=8.8.8.8
[Address]
Address=192.168.1.100/24
[Route]
Destination=0.0.0.0/0
Gateway=1.2.3.4
`
if v.Network() != network {
t.Log(v.Network())
t.FailNow()
}
}
func TestType(t *testing.T) {
for _, tt := range []struct {
i InterfaceGenerator
t string
}{
{
i: &physicalInterface{},
t: "physical",
},
{
i: &vlanInterface{},
t: "vlan",
},
{
i: &bondInterface{},
t: "bond",
}, },
} { } {
if tp := tt.i.Type(); tp != tt.t { if name := tt.iface.Name(); name != tt.name {
t.Fatalf("bad type (%q): got %s, want %s", tt.i, tp, tt.t) t.Fatalf("bad name (%q): want %q, got %q", tt.iface, tt.name, name)
}
if netdev := tt.iface.Netdev(); netdev != tt.netdev {
t.Fatalf("bad netdev (%q): want %q, got %q", tt.iface, tt.netdev, netdev)
}
if link := tt.iface.Link(); link != tt.link {
t.Fatalf("bad link (%q): want %q, got %q", tt.iface, tt.link, link)
}
if network := tt.iface.Network(); network != tt.network {
t.Fatalf("bad network (%q): want %q, got %q", tt.iface, tt.network, network)
}
if kind := tt.iface.Type(); kind != tt.kind {
t.Fatalf("bad type (%q): want %q, got %q", tt.iface, tt.kind, kind)
} }
} }
} }
@@ -469,7 +329,7 @@ func TestBuildInterfaces(t *testing.T) {
configDepth: 2, configDepth: 2,
}, },
} }
expect := []InterfaceGenerator{eth0, bond0, bond1, vlan0, vlan1} expect := []InterfaceGenerator{bond0, bond1, eth0, vlan0, vlan1}
if !reflect.DeepEqual(interfaces, expect) { if !reflect.DeepEqual(interfaces, expect) {
t.FailNow() t.FailNow()
} }
@@ -484,6 +344,8 @@ func TestFilename(t *testing.T) {
{logicalInterface{name: "iface", configDepth: 9}, "09-iface"}, {logicalInterface{name: "iface", configDepth: 9}, "09-iface"},
{logicalInterface{name: "iface", configDepth: 10}, "0a-iface"}, {logicalInterface{name: "iface", configDepth: 10}, "0a-iface"},
{logicalInterface{name: "iface", configDepth: 53}, "35-iface"}, {logicalInterface{name: "iface", configDepth: 53}, "35-iface"},
{logicalInterface{hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}), configDepth: 1}, "01-01:23:45:67:89:ab"},
{logicalInterface{name: "iface", hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}), configDepth: 1}, "01-iface"},
} { } {
if tt.i.Filename() != tt.f { if tt.i.Filename() != tt.f {
t.Fatalf("bad filename (%q): got %q, want %q", tt.i, tt.i.Filename(), tt.f) t.Fatalf("bad filename (%q): got %q, want %q", tt.i, tt.i.Filename(), tt.f)

View File

@@ -37,7 +37,7 @@ type route struct {
type configMethod interface{} type configMethod interface{}
type configMethodStatic struct { type configMethodStatic struct {
address net.IPNet addresses []net.IPNet
nameservers []net.IP nameservers []net.IP
routes []route routes []route
hwaddress net.HardwareAddr hwaddress net.HardwareAddr
@@ -193,20 +193,21 @@ func parseInterfaceStanza(attributes []string, options []string) (*stanzaInterfa
switch confMethod { switch confMethod {
case "static": case "static":
config := configMethodStatic{ config := configMethodStatic{
addresses: make([]net.IPNet, 1),
routes: make([]route, 0), routes: make([]route, 0),
nameservers: make([]net.IP, 0), nameservers: make([]net.IP, 0),
} }
if addresses, ok := optionMap["address"]; ok { if addresses, ok := optionMap["address"]; ok {
if len(addresses) == 1 { if len(addresses) == 1 {
config.address.IP = net.ParseIP(addresses[0]) config.addresses[0].IP = net.ParseIP(addresses[0])
} }
} }
if netmasks, ok := optionMap["netmask"]; ok { if netmasks, ok := optionMap["netmask"]; ok {
if len(netmasks) == 1 { if len(netmasks) == 1 {
config.address.Mask = net.IPMask(net.ParseIP(netmasks[0]).To4()) config.addresses[0].Mask = net.IPMask(net.ParseIP(netmasks[0]).To4())
} }
} }
if config.address.IP == nil || config.address.Mask == nil { if config.addresses[0].IP == nil || config.addresses[0].Mask == nil {
return nil, fmt.Errorf("malformed static network config for %q", iface) return nil, fmt.Errorf("malformed static network config for %q", iface)
} }
if gateways, ok := optionMap["gateway"]; ok { if gateways, ok := optionMap["gateway"]; ok {
@@ -235,7 +236,11 @@ func parseInterfaceStanza(attributes []string, options []string) (*stanzaInterfa
for i, field := range fields[:len(fields)-1] { for i, field := range fields[:len(fields)-1] {
switch field { switch field {
case "-net": case "-net":
if _, dst, err := net.ParseCIDR(fields[i+1]); err == nil {
route.destination = *dst
} else {
route.destination.IP = net.ParseIP(fields[i+1]) route.destination.IP = net.ParseIP(fields[i+1])
}
case "netmask": case "netmask":
route.destination.Mask = net.IPMask(net.ParseIP(fields[i+1]).To4()) route.destination.Mask = net.IPMask(net.ParseIP(fields[i+1]).To4())
case "gw": case "gw":

View File

@@ -194,9 +194,11 @@ func TestParseVLANStanzas(t *testing.T) {
func TestParseInterfaceStanzaStaticAddress(t *testing.T) { func TestParseInterfaceStanzaStaticAddress(t *testing.T) {
options := []string{"address 192.168.1.100", "netmask 255.255.255.0"} options := []string{"address 192.168.1.100", "netmask 255.255.255.0"}
expect := net.IPNet{ expect := []net.IPNet{
{
IP: net.IPv4(192, 168, 1, 100), IP: net.IPv4(192, 168, 1, 100),
Mask: net.IPv4Mask(255, 255, 255, 0), Mask: net.IPv4Mask(255, 255, 255, 0),
},
} }
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options) iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
@@ -207,7 +209,7 @@ func TestParseInterfaceStanzaStaticAddress(t *testing.T) {
if !ok { if !ok {
t.FailNow() t.FailNow()
} }
if !reflect.DeepEqual(static.address, expect) { if !reflect.DeepEqual(static.addresses, expect) {
t.FailNow() t.FailNow()
} }
} }
@@ -283,12 +285,17 @@ func TestBadParseInterfaceStanzasStaticPostUp(t *testing.T) {
} }
func TestParseInterfaceStanzaStaticPostUp(t *testing.T) { func TestParseInterfaceStanzaStaticPostUp(t *testing.T) {
options := []string{ for _, tt := range []struct {
options []string
expect []route
}{
{
options: []string{
"address 192.168.1.100", "address 192.168.1.100",
"netmask 255.255.255.0", "netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0 netmask 255.255.255.0", "post-up route add gw 192.168.1.1 -net 192.168.1.0 netmask 255.255.255.0",
} },
expect := []route{ expect: []route{
{ {
destination: net.IPNet{ destination: net.IPNet{
IP: net.IPv4(192, 168, 1, 0), IP: net.IPv4(192, 168, 1, 0),
@@ -296,18 +303,39 @@ func TestParseInterfaceStanzaStaticPostUp(t *testing.T) {
}, },
gateway: net.IPv4(192, 168, 1, 1), gateway: net.IPv4(192, 168, 1, 1),
}, },
},
},
{
options: []string{
"address 192.168.1.100",
"netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0/24 || true",
},
expect: []route{
{
destination: func() net.IPNet {
if _, net, err := net.ParseCIDR("192.168.1.0/24"); err == nil {
return *net
} else {
panic(err)
} }
}(),
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options) gateway: net.IPv4(192, 168, 1, 1),
},
},
},
} {
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, tt.options)
if err != nil { if err != nil {
t.FailNow() t.Fatalf("bad error (%+v): want nil, got %s\n", tt, err)
} }
static, ok := iface.configMethod.(configMethodStatic) static, ok := iface.configMethod.(configMethodStatic)
if !ok { if !ok {
t.FailNow() t.Fatalf("bad config method (%+v): want configMethodStatic, got %T\n", tt, iface.configMethod)
}
if !reflect.DeepEqual(static.routes, tt.expect) {
t.Fatalf("bad routes (%+v): want %#v, got %#v\n", tt, tt.expect, static.routes)
} }
if !reflect.DeepEqual(static.routes, expect) {
t.FailNow()
} }
} }

View File

@@ -74,15 +74,14 @@ func maybeProbe8012q(interfaces []network.InterfaceGenerator) error {
} }
func maybeProbeBonding(interfaces []network.InterfaceGenerator) error { func maybeProbeBonding(interfaces []network.InterfaceGenerator) error {
args := []string{"bonding"}
for _, iface := range interfaces { for _, iface := range interfaces {
if iface.Type() == "bond" { if iface.Type() == "bond" {
args = append(args, strings.Split(iface.ModprobeParams(), " ")...) args := append([]string{"bonding"}, strings.Split(iface.ModprobeParams(), " ")...)
break
}
}
log.Printf("Probing LKM %q (%q)\n", "bonding", args) log.Printf("Probing LKM %q (%q)\n", "bonding", args)
return exec.Command("modprobe", args...).Run() return exec.Command("modprobe", args...).Run()
}
}
return nil
} }
func restartNetworkd() error { func restartNetworkd() error {

10
test
View File

@@ -18,7 +18,9 @@ declare -a TESTPKGS=(initialize
datasource datasource
datasource/configdrive datasource/configdrive
datasource/file datasource/file
datasource/metadata
datasource/metadata/cloudsigma datasource/metadata/cloudsigma
datasource/metadata/digitalocean
datasource/metadata/ec2 datasource/metadata/ec2
datasource/proc_cmdline datasource/proc_cmdline
datasource/url datasource/url
@@ -26,9 +28,9 @@ declare -a TESTPKGS=(initialize
network) network)
if [ -z "$PKG" ]; then if [ -z "$PKG" ]; then
GOFMTPATH="$TESTPKGS coreos-cloudinit.go" GOFMTPATH="${TESTPKGS[*]} coreos-cloudinit.go"
# prepend repo path to each package # prepend repo path to each package
TESTPKGS="${TESTPKGS[@]/#/${REPO_PATH}/} ./" TESTPKGS="${TESTPKGS[*]/#/${REPO_PATH}/} ./"
else else
GOFMTPATH="$TESTPKGS" GOFMTPATH="$TESTPKGS"
# strip out slashes and dots from PKG=./foo/ # strip out slashes and dots from PKG=./foo/
@@ -43,5 +45,9 @@ go test ${COVER} $@ ${TESTPKGS}
echo "Checking gofmt..." echo "Checking gofmt..."
fmtRes=$(gofmt -l $GOFMTPATH) fmtRes=$(gofmt -l $GOFMTPATH)
if [ -n "$fmtRes" ]; then
echo "$fmtRes"
exit 1
fi
echo "Success" echo "Success"

View File

@@ -1,3 +1,15 @@
The following files were ported to Go from C files of libyaml, and thus
are still covered by their original copyright and license:
apic.go
emitterc.go
parserc.go
readerc.go
scannerc.go
writerc.go
yamlh.go
yamlprivateh.go
Copyright (c) 2006 Kirill Simonov Copyright (c) 2006 Kirill Simonov
Permission is hereby granted, free of charge, to any person obtaining a copy of Permission is hereby granted, free of charge, to any person obtaining a copy of

128
third_party/gopkg.in/yaml.v1/README.md vendored Normal file
View File

@@ -0,0 +1,128 @@
# YAML support for the Go language
Introduction
------------
The yaml package enables Go programs to comfortably encode and decode YAML
values. It was developed within [Canonical](https://www.canonical.com) as
part of the [juju](https://juju.ubuntu.com) project, and is based on a
pure Go port of the well-known [libyaml](http://pyyaml.org/wiki/LibYAML)
C library to parse and generate YAML data quickly and reliably.
Compatibility
-------------
The yaml package is almost compatible with YAML 1.1, including support for
anchors, tags, etc. There are still a few missing bits, such as document
merging, base-60 floats (huh?), and multi-document unmarshalling. These
features are not hard to add, and will be introduced as necessary.
Installation and usage
----------------------
The import path for the package is *gopkg.in/yaml.v1*.
To install it, run:
go get gopkg.in/yaml.v1
API documentation
-----------------
If opened in a browser, the import path itself leads to the API documentation:
* [https://gopkg.in/yaml.v1](https://gopkg.in/yaml.v1)
API stability
-------------
The package API for yaml v1 will remain stable as described in [gopkg.in](https://gopkg.in).
License
-------
The yaml package is licensed under the LGPL with an exception that allows it to be linked statically. Please see the LICENSE file for details.
Example
-------
```Go
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v1"
)
var data = `
a: Easy!
b:
c: 2
d: [3, 4]
`
type T struct {
A string
B struct{C int; D []int ",flow"}
}
func main() {
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
m := make(map[interface{}]interface{})
err = yaml.Unmarshal([]byte(data), &m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m:\n%v\n\n", m)
d, err = yaml.Marshal(&m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m dump:\n%s\n\n", string(d))
}
```
This example will generate the following output:
```
--- t:
{Easy! {2 [3 4]}}
--- t dump:
a: Easy!
b:
c: 2
d: [3, 4]
--- m:
map[a:Easy! b:map[c:2 d:[3 4]]]
--- m dump:
a: Easy!
b:
c: 2
d:
- 3
- 4
```

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"io" "io"

View File

@@ -1,8 +1,9 @@
package goyaml package yaml
import ( import (
"reflect" "reflect"
"strconv" "strconv"
"time"
) )
const ( const (
@@ -211,6 +212,16 @@ func newDecoder() *decoder {
// returned to call SetYAML() with the value of *out once it's defined. // returned to call SetYAML() with the value of *out once it's defined.
// //
func (d *decoder) setter(tag string, out *reflect.Value, good *bool) (set func()) { func (d *decoder) setter(tag string, out *reflect.Value, good *bool) (set func()) {
if (*out).Kind() != reflect.Ptr && (*out).CanAddr() {
setter, _ := (*out).Addr().Interface().(Setter)
if setter != nil {
var arg interface{}
*out = reflect.ValueOf(&arg).Elem()
return func() {
*good = setter.SetYAML(tag, arg)
}
}
}
again := true again := true
for again { for again {
again = false again = false
@@ -279,17 +290,20 @@ func (d *decoder) alias(n *node, out reflect.Value) (good bool) {
return good return good
} }
var durationType = reflect.TypeOf(time.Duration(0))
func (d *decoder) scalar(n *node, out reflect.Value) (good bool) { func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
var tag string var tag string
var resolved interface{} var resolved interface{}
if n.tag == "" && !n.implicit { if n.tag == "" && !n.implicit {
tag = "!!str"
resolved = n.value resolved = n.value
} else { } else {
tag, resolved = resolve(n.tag, n.value) tag, resolved = resolve(n.tag, n.value)
}
if set := d.setter(tag, &out, &good); set != nil { if set := d.setter(tag, &out, &good); set != nil {
defer set() defer set()
} }
}
switch out.Kind() { switch out.Kind() {
case reflect.String: case reflect.String:
if resolved != nil { if resolved != nil {
@@ -320,6 +334,14 @@ func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
out.SetInt(int64(resolved)) out.SetInt(int64(resolved))
good = true good = true
} }
case string:
if out.Type() == durationType {
d, err := time.ParseDuration(resolved)
if err == nil {
out.SetInt(int64(d))
good = true
}
}
} }
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr: case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
switch resolved := resolved.(type) { switch resolved := resolved.(type) {
@@ -437,6 +459,10 @@ func (d *decoder) mapping(n *node, out reflect.Value) (good bool) {
} }
l := len(n.children) l := len(n.children)
for i := 0; i < l; i += 2 { for i := 0; i < l; i += 2 {
if isMerge(n.children[i]) {
d.merge(n.children[i+1], out)
continue
}
k := reflect.New(kt).Elem() k := reflect.New(kt).Elem()
if d.unmarshal(n.children[i], k) { if d.unmarshal(n.children[i], k) {
e := reflect.New(et).Elem() e := reflect.New(et).Elem()
@@ -456,7 +482,12 @@ func (d *decoder) mappingStruct(n *node, out reflect.Value) (good bool) {
name := settableValueOf("") name := settableValueOf("")
l := len(n.children) l := len(n.children)
for i := 0; i < l; i += 2 { for i := 0; i < l; i += 2 {
if !d.unmarshal(n.children[i], name) { ni := n.children[i]
if isMerge(ni) {
d.merge(n.children[i+1], out)
continue
}
if !d.unmarshal(ni, name) {
continue continue
} }
if info, ok := sinfo.FieldsMap[name.String()]; ok { if info, ok := sinfo.FieldsMap[name.String()]; ok {
@@ -471,3 +502,37 @@ func (d *decoder) mappingStruct(n *node, out reflect.Value) (good bool) {
} }
return true return true
} }
func (d *decoder) merge(n *node, out reflect.Value) {
const wantMap = "map merge requires map or sequence of maps as the value"
switch n.kind {
case mappingNode:
d.unmarshal(n, out)
case aliasNode:
an, ok := d.doc.anchors[n.value]
if ok && an.kind != mappingNode {
panic(wantMap)
}
d.unmarshal(n, out)
case sequenceNode:
// Step backwards as earlier nodes take precedence.
for i := len(n.children)-1; i >= 0; i-- {
ni := n.children[i]
if ni.kind == aliasNode {
an, ok := d.doc.anchors[ni.value]
if ok && an.kind != mappingNode {
panic(wantMap)
}
} else if ni.kind != mappingNode {
panic(wantMap)
}
d.unmarshal(ni, out)
}
default:
panic(wantMap)
}
}
func isMerge(n *node) bool {
return n.kind == scalarNode && n.value == "<<" && (n.implicit == true || n.tag == "!!merge" || n.tag == "tag:yaml.org,2002:merge")
}

View File

@@ -1,10 +1,11 @@
package goyaml_test package yaml_test
import ( import (
. "launchpad.net/gocheck" . "gopkg.in/check.v1"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml" "gopkg.in/yaml.v1"
"math" "math"
"reflect" "reflect"
"time"
) )
var unmarshalIntTest = 123 var unmarshalIntTest = 123
@@ -350,6 +351,32 @@ var unmarshalTests = []struct {
C inlineB `yaml:",inline"` C inlineB `yaml:",inline"`
}{1, inlineB{2, inlineC{3}}}, }{1, inlineB{2, inlineC{3}}},
}, },
// bug 1243827
{
"a: -b_c",
map[string]interface{}{"a": "-b_c"},
},
{
"a: +b_c",
map[string]interface{}{"a": "+b_c"},
},
{
"a: 50cent_of_dollar",
map[string]interface{}{"a": "50cent_of_dollar"},
},
// Duration
{
"a: 3s",
map[string]time.Duration{"a": 3 * time.Second},
},
// Issue #24.
{
"a: <foo>",
map[string]string{"a": "<foo>"},
},
} }
type inlineB struct { type inlineB struct {
@@ -377,7 +404,7 @@ func (s *S) TestUnmarshal(c *C) {
pv := reflect.New(pt.Elem()) pv := reflect.New(pt.Elem())
value = pv.Interface() value = pv.Interface()
} }
err := goyaml.Unmarshal([]byte(item.data), value) err := yaml.Unmarshal([]byte(item.data), value)
c.Assert(err, IsNil, Commentf("Item #%d", i)) c.Assert(err, IsNil, Commentf("Item #%d", i))
if t.Kind() == reflect.String { if t.Kind() == reflect.String {
c.Assert(*value.(*string), Equals, item.value, Commentf("Item #%d", i)) c.Assert(*value.(*string), Equals, item.value, Commentf("Item #%d", i))
@@ -389,7 +416,7 @@ func (s *S) TestUnmarshal(c *C) {
func (s *S) TestUnmarshalNaN(c *C) { func (s *S) TestUnmarshalNaN(c *C) {
value := map[string]interface{}{} value := map[string]interface{}{}
err := goyaml.Unmarshal([]byte("notanum: .NaN"), &value) err := yaml.Unmarshal([]byte("notanum: .NaN"), &value)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(math.IsNaN(value["notanum"].(float64)), Equals, true) c.Assert(math.IsNaN(value["notanum"].(float64)), Equals, true)
} }
@@ -408,7 +435,7 @@ var unmarshalErrorTests = []struct {
func (s *S) TestUnmarshalErrors(c *C) { func (s *S) TestUnmarshalErrors(c *C) {
for _, item := range unmarshalErrorTests { for _, item := range unmarshalErrorTests {
var value interface{} var value interface{}
err := goyaml.Unmarshal([]byte(item.data), &value) err := yaml.Unmarshal([]byte(item.data), &value)
c.Assert(err, ErrorMatches, item.error, Commentf("Partial unmarshal: %#v", value)) c.Assert(err, ErrorMatches, item.error, Commentf("Partial unmarshal: %#v", value))
} }
} }
@@ -421,6 +448,8 @@ var setterTests = []struct {
{"_: [1,A]", "!!seq", []interface{}{1, "A"}}, {"_: [1,A]", "!!seq", []interface{}{1, "A"}},
{"_: 10", "!!int", 10}, {"_: 10", "!!int", 10},
{"_: null", "!!null", nil}, {"_: null", "!!null", nil},
{`_: BAR!`, "!!str", "BAR!"},
{`_: "BAR!"`, "!!str", "BAR!"},
{"_: !!foo 'BAR!'", "!!foo", "BAR!"}, {"_: !!foo 'BAR!'", "!!foo", "BAR!"},
} }
@@ -442,17 +471,31 @@ func (o *typeWithSetter) SetYAML(tag string, value interface{}) (ok bool) {
return true return true
} }
type typeWithSetterField struct { type setterPointerType struct {
Field *typeWithSetter "_" Field *typeWithSetter "_"
} }
func (s *S) TestUnmarshalWithSetter(c *C) { type setterValueType struct {
Field typeWithSetter "_"
}
func (s *S) TestUnmarshalWithPointerSetter(c *C) {
for _, item := range setterTests { for _, item := range setterTests {
obj := &typeWithSetterField{} obj := &setterPointerType{}
err := goyaml.Unmarshal([]byte(item.data), obj) err := yaml.Unmarshal([]byte(item.data), obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(obj.Field, NotNil, c.Assert(obj.Field, NotNil, Commentf("Pointer not initialized (%#v)", item.value))
Commentf("Pointer not initialized (%#v)", item.value)) c.Assert(obj.Field.tag, Equals, item.tag)
c.Assert(obj.Field.value, DeepEquals, item.value)
}
}
func (s *S) TestUnmarshalWithValueSetter(c *C) {
for _, item := range setterTests {
obj := &setterValueType{}
err := yaml.Unmarshal([]byte(item.data), obj)
c.Assert(err, IsNil)
c.Assert(obj.Field, NotNil, Commentf("Pointer not initialized (%#v)", item.value))
c.Assert(obj.Field.tag, Equals, item.tag) c.Assert(obj.Field.tag, Equals, item.tag)
c.Assert(obj.Field.value, DeepEquals, item.value) c.Assert(obj.Field.value, DeepEquals, item.value)
} }
@@ -460,7 +503,7 @@ func (s *S) TestUnmarshalWithSetter(c *C) {
func (s *S) TestUnmarshalWholeDocumentWithSetter(c *C) { func (s *S) TestUnmarshalWholeDocumentWithSetter(c *C) {
obj := &typeWithSetter{} obj := &typeWithSetter{}
err := goyaml.Unmarshal([]byte(setterTests[0].data), obj) err := yaml.Unmarshal([]byte(setterTests[0].data), obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(obj.tag, Equals, setterTests[0].tag) c.Assert(obj.tag, Equals, setterTests[0].tag)
value, ok := obj.value.(map[interface{}]interface{}) value, ok := obj.value.(map[interface{}]interface{})
@@ -477,8 +520,8 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
}() }()
m := map[string]*typeWithSetter{} m := map[string]*typeWithSetter{}
data := "{abc: 1, def: 2, ghi: 3, jkl: 4}" data := `{abc: 1, def: 2, ghi: 3, jkl: 4}`
err := goyaml.Unmarshal([]byte(data), m) err := yaml.Unmarshal([]byte(data), m)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(m["abc"], NotNil) c.Assert(m["abc"], NotNil)
c.Assert(m["def"], IsNil) c.Assert(m["def"], IsNil)
@@ -489,6 +532,98 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
c.Assert(m["ghi"].value, Equals, 3) c.Assert(m["ghi"].value, Equals, 3)
} }
// From http://yaml.org/type/merge.html
var mergeTests = `
anchors:
- &CENTER { "x": 1, "y": 2 }
- &LEFT { "x": 0, "y": 2 }
- &BIG { "r": 10 }
- &SMALL { "r": 1 }
# All the following maps are equal:
plain:
# Explicit keys
"x": 1
"y": 2
"r": 10
label: center/big
mergeOne:
# Merge one map
<< : *CENTER
"r": 10
label: center/big
mergeMultiple:
# Merge multiple maps
<< : [ *CENTER, *BIG ]
label: center/big
override:
# Override
<< : [ *BIG, *LEFT, *SMALL ]
"x": 1
label: center/big
shortTag:
# Explicit short merge tag
!!merge "<<" : [ *CENTER, *BIG ]
label: center/big
longTag:
# Explicit merge long tag
!<tag:yaml.org,2002:merge> "<<" : [ *CENTER, *BIG ]
label: center/big
inlineMap:
# Inlined map
<< : {"x": 1, "y": 2, "r": 10}
label: center/big
inlineSequenceMap:
# Inlined map in sequence
<< : [ *CENTER, {"r": 10} ]
label: center/big
`
func (s *S) TestMerge(c *C) {
var want = map[interface{}]interface{}{
"x": 1,
"y": 2,
"r": 10,
"label": "center/big",
}
var m map[string]interface{}
err := yaml.Unmarshal([]byte(mergeTests), &m)
c.Assert(err, IsNil)
for name, test := range m {
if name == "anchors" {
continue
}
c.Assert(test, DeepEquals, want, Commentf("test %q failed", name))
}
}
func (s *S) TestMergeStruct(c *C) {
type Data struct {
X, Y, R int
Label string
}
want := Data{1, 2, 10, "center/big"}
var m map[string]Data
err := yaml.Unmarshal([]byte(mergeTests), &m)
c.Assert(err, IsNil)
for name, test := range m {
if name == "anchors" {
continue
}
c.Assert(test, Equals, want, Commentf("test %q failed", name))
}
}
//var data []byte //var data []byte
//func init() { //func init() {
// var err error // var err error
@@ -502,7 +637,7 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
// var err error // var err error
// for i := 0; i < c.N; i++ { // for i := 0; i < c.N; i++ {
// var v map[string]interface{} // var v map[string]interface{}
// err = goyaml.Unmarshal(data, &v) // err = yaml.Unmarshal(data, &v)
// } // }
// if err != nil { // if err != nil {
// panic(err) // panic(err)
@@ -511,9 +646,9 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
// //
//func (s *S) BenchmarkMarshal(c *C) { //func (s *S) BenchmarkMarshal(c *C) {
// var v map[string]interface{} // var v map[string]interface{}
// goyaml.Unmarshal(data, &v) // yaml.Unmarshal(data, &v)
// c.ResetTimer() // c.ResetTimer()
// for i := 0; i < c.N; i++ { // for i := 0; i < c.N; i++ {
// goyaml.Marshal(&v) // yaml.Marshal(&v)
// } // }
//} //}

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"bytes" "bytes"

View File

@@ -1,9 +1,10 @@
package goyaml package yaml
import ( import (
"reflect" "reflect"
"sort" "sort"
"strconv" "strconv"
"time"
) )
type encoder struct { type encoder struct {
@@ -85,7 +86,11 @@ func (e *encoder) marshal(tag string, in reflect.Value) {
case reflect.String: case reflect.String:
e.stringv(tag, in) e.stringv(tag, in)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
if in.Type() == durationType {
e.stringv(tag, reflect.ValueOf(in.Interface().(time.Duration).String()))
} else {
e.intv(tag, in) e.intv(tag, in)
}
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr: case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
e.uintv(tag, in) e.uintv(tag, in)
case reflect.Float32, reflect.Float64: case reflect.Float32, reflect.Float64:

View File

@@ -1,12 +1,13 @@
package goyaml_test package yaml_test
import ( import (
"fmt" "fmt"
. "launchpad.net/gocheck" "gopkg.in/yaml.v1"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml" . "gopkg.in/check.v1"
"math" "math"
"strconv" "strconv"
"strings" "strings"
"time"
) )
var marshalIntTest = 123 var marshalIntTest = 123
@@ -212,11 +213,23 @@ var marshalTests = []struct {
}{1, inlineB{2, inlineC{3}}}, }{1, inlineB{2, inlineC{3}}},
"a: 1\nb: 2\nc: 3\n", "a: 1\nb: 2\nc: 3\n",
}, },
// Duration
{
map[string]time.Duration{"a": 3 * time.Second},
"a: 3s\n",
},
// Issue #24.
{
map[string]string{"a": "<foo>"},
"a: <foo>\n",
},
} }
func (s *S) TestMarshal(c *C) { func (s *S) TestMarshal(c *C) {
for _, item := range marshalTests { for _, item := range marshalTests {
data, err := goyaml.Marshal(item.value) data, err := yaml.Marshal(item.value)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(string(data), Equals, item.data) c.Assert(string(data), Equals, item.data)
} }
@@ -237,7 +250,7 @@ var marshalErrorTests = []struct {
func (s *S) TestMarshalErrors(c *C) { func (s *S) TestMarshalErrors(c *C) {
for _, item := range marshalErrorTests { for _, item := range marshalErrorTests {
_, err := goyaml.Marshal(item.value) _, err := yaml.Marshal(item.value)
c.Assert(err, ErrorMatches, item.error) c.Assert(err, ErrorMatches, item.error)
} }
} }
@@ -269,12 +282,12 @@ func (s *S) TestMarshalTypeCache(c *C) {
var err error var err error
func() { func() {
type T struct{ A int } type T struct{ A int }
data, err = goyaml.Marshal(&T{}) data, err = yaml.Marshal(&T{})
c.Assert(err, IsNil) c.Assert(err, IsNil)
}() }()
func() { func() {
type T struct{ B int } type T struct{ B int }
data, err = goyaml.Marshal(&T{}) data, err = yaml.Marshal(&T{})
c.Assert(err, IsNil) c.Assert(err, IsNil)
}() }()
c.Assert(string(data), Equals, "b: 0\n") c.Assert(string(data), Equals, "b: 0\n")
@@ -298,7 +311,7 @@ func (s *S) TestMashalWithGetter(c *C) {
obj := &typeWithGetterField{} obj := &typeWithGetterField{}
obj.Field.tag = item.tag obj.Field.tag = item.tag
obj.Field.value = item.value obj.Field.value = item.value
data, err := goyaml.Marshal(obj) data, err := yaml.Marshal(obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(string(data), Equals, string(item.data)) c.Assert(string(data), Equals, string(item.data))
} }
@@ -308,7 +321,7 @@ func (s *S) TestUnmarshalWholeDocumentWithGetter(c *C) {
obj := &typeWithGetter{} obj := &typeWithGetter{}
obj.tag = "" obj.tag = ""
obj.value = map[string]string{"hello": "world!"} obj.value = map[string]string{"hello": "world!"}
data, err := goyaml.Marshal(obj) data, err := yaml.Marshal(obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(string(data), Equals, "hello: world!\n") c.Assert(string(data), Equals, "hello: world!\n")
} }
@@ -356,7 +369,7 @@ func (s *S) TestSortedOutput(c *C) {
for _, k := range order { for _, k := range order {
m[k] = 1 m[k] = 1
} }
data, err := goyaml.Marshal(m) data, err := yaml.Marshal(m)
c.Assert(err, IsNil) c.Assert(err, IsNil)
out := "\n" + string(data) out := "\n" + string(data)
last := 0 last := 0

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"bytes" "bytes"

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"io" "io"

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"math" "math"
@@ -27,7 +27,6 @@ func init() {
t[int(c)] = 'M' // In map t[int(c)] = 'M' // In map
} }
t[int('.')] = '.' // Float (potentially in map) t[int('.')] = '.' // Float (potentially in map)
t[int('<')] = '<' // Merge
var resolveMapList = []struct { var resolveMapList = []struct {
v interface{} v interface{}
@@ -45,6 +44,7 @@ func init() {
{math.Inf(+1), "!!float", []string{".inf", ".Inf", ".INF"}}, {math.Inf(+1), "!!float", []string{".inf", ".Inf", ".INF"}},
{math.Inf(+1), "!!float", []string{"+.inf", "+.Inf", "+.INF"}}, {math.Inf(+1), "!!float", []string{"+.inf", "+.Inf", "+.INF"}},
{math.Inf(-1), "!!float", []string{"-.inf", "-.Inf", "-.INF"}}, {math.Inf(-1), "!!float", []string{"-.inf", "-.Inf", "-.INF"}},
{"<<", "!!merge", []string{"<<"}},
} }
m := resolveMap m := resolveMap
@@ -113,13 +113,8 @@ func resolve(tag string, in string) (rtag string, out interface{}) {
case 'D', 'S': case 'D', 'S':
// Int, float, or timestamp. // Int, float, or timestamp.
for i := 0; i != len(in); i++ { plain := strings.Replace(in, "_", "", -1)
if in[i] == '_' { intv, err := strconv.ParseInt(plain, 0, 64)
in = strings.Replace(in, "_", "", -1)
break
}
}
intv, err := strconv.ParseInt(in, 0, 64)
if err == nil { if err == nil {
if intv == int64(int(intv)) { if intv == int64(int(intv)) {
return "!!int", int(intv) return "!!int", int(intv)
@@ -127,26 +122,23 @@ func resolve(tag string, in string) (rtag string, out interface{}) {
return "!!int", intv return "!!int", intv
} }
} }
floatv, err := strconv.ParseFloat(in, 64) floatv, err := strconv.ParseFloat(plain, 64)
if err == nil { if err == nil {
return "!!float", floatv return "!!float", floatv
} }
if strings.HasPrefix(in, "0b") { if strings.HasPrefix(plain, "0b") {
intv, err := strconv.ParseInt(in[2:], 2, 64) intv, err := strconv.ParseInt(plain[2:], 2, 64)
if err == nil { if err == nil {
return "!!int", int(intv) return "!!int", int(intv)
} }
} else if strings.HasPrefix(in, "-0b") { } else if strings.HasPrefix(plain, "-0b") {
intv, err := strconv.ParseInt(in[3:], 2, 64) intv, err := strconv.ParseInt(plain[3:], 2, 64)
if err == nil { if err == nil {
return "!!int", -int(intv) return "!!int", -int(intv)
} }
} }
// XXX Handle timestamps here. // XXX Handle timestamps here.
case '<':
// XXX Handle merge (<<) here.
default: default:
panic("resolveTable item not yet handled: " + panic("resolveTable item not yet handled: " +
string([]byte{c}) + " (with " + in + ")") string([]byte{c}) + " (with " + in + ")")

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"bytes" "bytes"

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"reflect" "reflect"

View File

@@ -1,7 +1,7 @@
package goyaml_test package yaml_test
import ( import (
. "launchpad.net/gocheck" . "gopkg.in/check.v1"
"testing" "testing"
) )

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
// Set the writer error and return false. // Set the writer error and return false.
func yaml_emitter_set_writer_error(emitter *yaml_emitter_t, problem string) bool { func yaml_emitter_set_writer_error(emitter *yaml_emitter_t, problem string) bool {

View File

@@ -1,5 +1,10 @@
// Package goyaml implements YAML support for the Go language. // Package yaml implements YAML support for the Go language.
package goyaml //
// Source code and other details for the project are available at GitHub:
//
// https://github.com/go-yaml/yaml
//
package yaml
import ( import (
"errors" "errors"
@@ -28,32 +33,31 @@ func handleErr(err *error) {
} }
} }
// Objects implementing the goyaml.Setter interface will receive the YAML // The Setter interface may be implemented by types to do their own custom
// tag and value via the SetYAML method during unmarshaling, rather than // unmarshalling of YAML values, rather than being implicitly assigned by
// being implicitly assigned by the goyaml machinery. If setting the value // the yaml package machinery. If setting the value works, the method should
// works, the method should return true. If it returns false, the given // return true. If it returns false, the value is considered unsupported
// value will be omitted from maps and slices. // and is omitted from maps and slices.
type Setter interface { type Setter interface {
SetYAML(tag string, value interface{}) bool SetYAML(tag string, value interface{}) bool
} }
// Objects implementing the goyaml.Getter interface will get the GetYAML() // The Getter interface is implemented by types to do their own custom
// method called when goyaml is requested to marshal the given value, and // marshalling into a YAML tag and value.
// the result of this method will be marshaled in place of the actual object.
type Getter interface { type Getter interface {
GetYAML() (tag string, value interface{}) GetYAML() (tag string, value interface{})
} }
// Unmarshal decodes the first document found within the in byte slice // Unmarshal decodes the first document found within the in byte slice
// and assigns decoded values into the object pointed by out. // and assigns decoded values into the out value.
// //
// Maps, pointers to structs and ints, etc, may all be used as out values. // Maps and pointers (to a struct, string, int, etc) are accepted as out
// If an internal pointer within a struct is not initialized, goyaml // values. If an internal pointer within a struct is not initialized,
// will initialize it if necessary for unmarshalling the provided data, // the yaml package will initialize it if necessary for unmarshalling
// but the struct provided as out must not be a nil pointer. // the provided data. The out parameter must not be nil.
// //
// The type of the decoded values and the type of out will be considered, // The type of the decoded values and the type of out will be considered,
// and Unmarshal() will do the best possible job to unmarshal values // and Unmarshal will do the best possible job to unmarshal values
// appropriately. It is NOT considered an error, though, to skip values // appropriately. It is NOT considered an error, though, to skip values
// because they are not available in the decoded YAML, or if they are not // because they are not available in the decoded YAML, or if they are not
// compatible with the out value. To ensure something was properly // compatible with the out value. To ensure something was properly
@@ -61,11 +65,11 @@ type Getter interface {
// field (usually the zero value). // field (usually the zero value).
// //
// Struct fields are only unmarshalled if they are exported (have an // Struct fields are only unmarshalled if they are exported (have an
// upper case first letter), and will be unmarshalled using the field // upper case first letter), and are unmarshalled using the field name
// name lowercased by default. When custom field names are desired, the // lowercased as the default key. Custom keys may be defined via the
// tag value may be used to tweak the name. Everything before the first // "yaml" name in the field tag: the content preceding the first comma
// comma in the field tag will be used as the name. The values following // is used as the key, and the following comma-separated options are
// the comma are used to tweak the marshalling process (see Marshal). // used to tweak the marshalling process (see Marshal).
// Conflicting names result in a runtime error. // Conflicting names result in a runtime error.
// //
// For example: // For example:
@@ -75,7 +79,7 @@ type Getter interface {
// B int // B int
// } // }
// var T t // var T t
// goyaml.Unmarshal([]byte("a: 1\nb: 2"), &t) // yaml.Unmarshal([]byte("a: 1\nb: 2"), &t)
// //
// See the documentation of Marshal for the format of tags and a list of // See the documentation of Marshal for the format of tags and a list of
// supported tag options. // supported tag options.
@@ -94,14 +98,16 @@ func Unmarshal(in []byte, out interface{}) (err error) {
// Marshal serializes the value provided into a YAML document. The structure // Marshal serializes the value provided into a YAML document. The structure
// of the generated document will reflect the structure of the value itself. // of the generated document will reflect the structure of the value itself.
// Maps, pointers to structs and ints, etc, may all be used as the in value. // Maps and pointers (to struct, string, int, etc) are accepted as the in value.
// //
// In the case of struct values, only exported fields will be serialized. // Struct fields are only unmarshalled if they are exported (have an upper case
// The lowercased field name is used as the key for each exported field, // first letter), and are unmarshalled using the field name lowercased as the
// but this behavior may be changed using the respective field tag. // default key. Custom keys may be defined via the "yaml" name in the field
// The tag may also contain flags to tweak the marshalling behavior for // tag: the content preceding the first comma is used as the key, and the
// the field. Conflicting names result in a runtime error. The tag format // following comma-separated options are used to tweak the marshalling process.
// accepted is: // Conflicting names result in a runtime error.
//
// The field tag format accepted is:
// //
// `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)` // `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)`
// //
@@ -126,8 +132,8 @@ func Unmarshal(in []byte, out interface{}) (err error) {
// F int "a,omitempty" // F int "a,omitempty"
// B int // B int
// } // }
// goyaml.Marshal(&T{B: 2}) // Returns "b: 2\n" // yaml.Marshal(&T{B: 2}) // Returns "b: 2\n"
// goyaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n" // yaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n"
// //
func Marshal(in interface{}) (out []byte, err error) { func Marshal(in interface{}) (out []byte, err error) {
defer handleErr(&err) defer handleErr(&err)
@@ -142,7 +148,7 @@ func Marshal(in interface{}) (out []byte, err error) {
// -------------------------------------------------------------------------- // --------------------------------------------------------------------------
// Maintain a mapping of keys to structure field indexes // Maintain a mapping of keys to structure field indexes
// The code in this section was copied from gobson. // The code in this section was copied from mgo/bson.
// structInfo holds details for the serialization of fields of // structInfo holds details for the serialization of fields of
// a given struct. // a given struct.

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"io" "io"

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
const ( const (
// The size of the input raw buffer. // The size of the input raw buffer.

View File

@@ -1,14 +0,0 @@
[568].out
_*
*.cgo*.*
yaml-*/stamp-h1
yaml-*/Makefile
yaml-*/*/Makefile
yaml-*/libtool
yaml-*/config*
yaml-*/*/*.lo
yaml-*/*/*.la
yaml-*/*/.libs
yaml-*/*/.deps
yaml-*/tests/*

View File

@@ -1 +0,0 @@
propose -cr -for=lp:goyaml

View File

@@ -1,20 +0,0 @@
#!/bin/sh
set -e
BADFMT=`find * -name '*.go' | xargs gofmt -l`
if [ -n "$BADFMT" ]; then
BADFMT=`echo "$BADFMT" | sed "s/^/ /"`
echo -e "gofmt is sad:\n\n$BADFMT"
exit 1
fi
VERSION=`go version | awk '{print $3}'`
if [ $VERSION == 'devel' ]; then
go tool vet \
-methods \
-printf \
-rangeloops \
-printfuncs 'ErrorContextf:1,notFoundf:0,badReqErrorf:0,Commitf:0,Snapshotf:0,Debugf:0' \
.
fi

View File

@@ -1,39 +0,0 @@
include $(GOROOT)/src/Make.inc
YAML=yaml-0.1.3
LIBYAML=$(PWD)/$(YAML)/src/.libs/libyaml.a
TARG=launchpad.net/goyaml
GOFILES=\
goyaml.go\
resolve.go\
CGOFILES=\
decode.go\
encode.go\
CGO_OFILES+=\
helpers.o\
api.o\
scanner.o\
reader.o\
parser.o\
writer.o\
emitter.o\
GOFMT=gofmt
BADFMT:=$(shell $(GOFMT) -l $(GOFILES) $(CGOFILES) $(wildcard *_test.go))
all: package
gofmt: $(BADFMT)
@for F in $(BADFMT); do $(GOFMT) -w $$F && echo $$F; done
include $(GOROOT)/src/Make.pkg
ifneq ($(BADFMT),)
ifneq ($(MAKECMDGOALS),gofmt)
$(warning WARNING: make gofmt: $(BADFMT))
endif
endif