Compare commits

...

350 Commits

Author SHA1 Message Date
Alex Crawford
ae3676096c coreos-cloudinit: bump to v1.2.1 2015-01-21 14:29:25 -08:00
Alex Crawford
a548b557ed doc: add coreos-ssh-import-github-users 2015-01-21 14:28:43 -08:00
Alex Crawford
a9c132a706 coreos-cloudinit: bump to v1.2.0+git 2015-01-20 14:42:48 -08:00
Alex Crawford
c3c4b86a3b coreos-cloudinit: bump to v1.2.0 2015-01-20 14:42:18 -08:00
Alex Crawford
44142ff8af Merge pull request #301 from crawford/github
config: add support for multiple github users
2015-01-20 14:26:38 -08:00
Alex Crawford
e9529ede44 config: add support for multiple github users 2015-01-20 13:45:52 -08:00
Alex Crawford
4b5b801171 Merge pull request #295 from crawford/rules
config/validate: add some sanity checks
2015-01-15 16:25:37 -08:00
Alex Crawford
551cbb1e5d config/validate: add rule for file encoding 2015-01-15 15:01:44 -08:00
Alex Crawford
3c93938f8a decode: refactor file decoding into config package 2015-01-15 15:01:44 -08:00
Alex Crawford
f61c08c246 config/validate: add rule for coreos.write_files 2015-01-15 15:01:44 -08:00
Alex Crawford
571903cec6 config/validate: add rule for etcd discovery token 2015-01-15 15:01:44 -08:00
Alex Crawford
bdbd1930ed config/validate: add rule for file paths 2015-01-15 15:01:44 -08:00
Alex Crawford
cc75a943ba Merge pull request #300 from crawford/float
validate: allow promotion of int to float64
2015-01-15 13:17:44 -08:00
Alex Crawford
fc77ba6355 validate: allow promotion of int to float64 2015-01-14 17:54:01 -08:00
Eugene Yakubovich
7cfa0df7c4 Merge pull request #299 from eyakubovich/master
config: document and update flannel config values
2015-01-13 15:55:42 -08:00
Eugene Yakubovich
58f0dadaf9 config: document and update flannel config values
Fixes #297
2015-01-13 15:51:47 -08:00
Michael Marineau
1ab530f157 Merge pull request #293 from realestate-com-au/master
select first available hostname returned by EC2 metadata.
2015-01-05 12:47:27 -08:00
Kevin Yung
13e4b77130 ec2: allow spaces seperated hostname in metadata
AWS hostname metadata will return space seperated hostname and domain
names when DHCPOptionSet is using multiple domain names. This patch will
cater for this scenario.
2015-01-05 16:01:57 +11:00
Alex Crawford
54c62cbb70 coreos-cloudinit: bump to v1.1.0+git 2014-12-30 16:52:10 +01:00
Alex Crawford
c8e864fef5 coreos-cloudinit: bump to v1.1.0 2014-12-30 16:51:24 +01:00
Alex Crawford
60a3377e7c Merge pull request #290 from crawford/yaml
Improved YAML parsing
2014-12-30 16:24:12 +01:00
Alex Crawford
5527f09778 config: fix parsing of file permissions
These reintroduces the braindead '744' syntax for file permissions. Even
though this number isn't octal, it is assumed by convention to be. In
order to pull this off, coerceNodes() was introduced to try to
counteract the type inferrencing that occurs during the yaml
unmarshalling. The config is unmarshalled twice: once into an empty
interface and once into the CloudConfig structure. The two resulting
node structures are combined together. The nodes from the CloudConfig
process replace those from the interface{} when the types of the two
nodes are compatible. For example, with the input `0744`, yaml
interprets that as the integer 484 giving us the nodes '0744'(string)
and 484(int). Because the types string and int are compatible, we opt to
take the string node instead of the integer.
2014-12-30 16:20:21 +01:00
Alex Crawford
54a64454b9 validate: fix printing for non-string values 2014-12-30 16:20:21 +01:00
Alex Crawford
0e70d4f01f config: add validity check for file permissions 2014-12-30 16:20:21 +01:00
Alex Crawford
af8e590575 config: change valid tag to use regexp
A regular expression is much more useful than a list of strings.
2014-12-30 16:20:21 +01:00
Alex Crawford
40d943fb7a reboot-strategy: remove the 'false' value
Since we no longer do a two-stage unmarshal, the 'false' value will no
longer be necessary.
2014-12-30 16:20:21 +01:00
Alex Crawford
248536a5cd config: use a YAML transform to normalize keys
This removes the problematic two-pass unmarshalling.
2014-12-30 16:20:21 +01:00
Alex Crawford
4ed1d03c97 godeps: bump github.com/coreos/yaml 2014-12-30 16:20:20 +01:00
Alex Crawford
057ab37364 config: seperate the CoreOS type from CloudConfig
Renamed Coreos to CoreOS while I was at it.
2014-12-30 16:20:20 +01:00
Alex Crawford
182241c8d3 config: clean up and remove some tests
Small modification to make these align with our test-table-style tests.
Also removed TestCloudConfigInvalidKeys since it hasn't been a useful
test since d3294bcb86.
2014-12-30 16:19:00 +01:00
Michael Marineau
edced59fa6 Merge pull request #281 from thommay/flannel_env_file
Create an environment file for flannel
2014-12-29 15:07:08 -08:00
Thom May
9be836df31 Create an environment file for flannel
Rather than using a systemd overlay, allow docker to load the
environment file. This is due to coreos/coreos-overlay#1002
2014-12-29 10:27:22 +00:00
Jonathan Boulle
4e54447b8e Merge pull request #286 from jonboulle/master
Godeps: switch to coreos/yaml
2014-12-20 15:43:55 -08:00
Jonathan Boulle
999c38b09b Godeps: switch to coreos/yaml 2014-12-20 15:31:02 -08:00
Alex Crawford
06d13de5c3 coreos-cloudinit: bump to v1.0.2+git 2014-12-12 17:38:28 -08:00
Alex Crawford
5b0903d162 coreos-cloudinit: bump to v1.0.2 2014-12-12 17:37:39 -08:00
Alex Crawford
10669be7c0 Merge pull request #284 from crawford/travis
test: disable Travis sudo capability
2014-12-12 17:28:47 -08:00
Alex Crawford
2edae741e1 test: disable Travis sudo capability 2014-12-12 16:46:18 -08:00
Alex Crawford
ea90e553d1 Merge pull request #282 from crawford/network
network: write network units with user units
2014-12-12 15:12:50 -08:00
Alex Crawford
b0cfd86902 network: write network units with user units
This allows us to test the network unit generation as well as removing
some special-cased code.
2014-12-12 15:08:03 -08:00
Alex Crawford
565a9540c9 Merge pull request #283 from crawford/validate
validate: empty user_data is also valid
2014-12-12 15:05:51 -08:00
Alex Crawford
fd10e27b99 validate: empty user_data is also valid 2014-12-12 14:49:42 -08:00
Michael Marineau
39763d772c Merge pull request #280 from marineam/go1.4
travis: Add Go 1.4 as a test target
2014-12-11 16:34:00 -08:00
Michael Marineau
ee69b77bfb travis: Add Go 1.4 as a test target 2014-12-11 15:29:36 -08:00
Jonathan Boulle
353444e56d Merge pull request #279 from cnelson/write_files_encoding_support
Add support for the encoding key to write_files
2014-12-09 17:37:34 -08:00
cnelson
112ba1e31f Added support for the encoding key in write_files
Supported encodings are base64, gzip, and base64 encoded gzip
2014-12-09 17:35:33 -08:00
cnelson
9c3cd9e69c bumped version of yaml.v1 2014-12-09 07:49:59 -08:00
Alex Crawford
685d8317bc Merge pull request #275 from mwhooker/master
Enable configuration of locksmithd
2014-12-05 14:09:40 -08:00
Matthew Hooker
f42d102b26 Also fix wording of Flannel section 2014-12-04 23:55:26 -08:00
Matthew Hooker
c944e9ef94 Enable configuration of locksmithd 2014-12-04 23:53:31 -08:00
Alex Crawford
f10d6e8bef coreos-cloudinit: bump to v1.0.1+git 2014-12-04 16:35:20 -08:00
Alex Crawford
f3f3af79fd coreos-cloudinit: bump to v1.0.1 2014-12-04 16:34:57 -08:00
Alex Crawford
0e63aa0f6b Merge pull request #276 from crawford/networkd
initialize: restart networkd before other units
2014-12-04 16:33:02 -08:00
Alex Crawford
b254e17e89 Merge pull request #263 from robszumski/docs-validator
docs: link to validator
2014-12-04 16:28:21 -08:00
Alex Crawford
5c059b66f0 initialize: restart networkd before other units 2014-12-04 15:25:44 -08:00
Alex Crawford
c628bef666 Merge pull request #273 from crawford/networkd
initialize: only restart networkd once per config
2014-12-02 12:54:27 -08:00
Alex Crawford
2270db3f7a initialize: only restart networkd once per config
Regression introduced by 9fcf338bf3.

Networkd was erroneously being restarted once per network unit. It
should only restart once for the entire config.
2014-12-02 12:46:35 -08:00
Alex Crawford
d0d467813d Merge pull request #251 from Vladimiroff/master
metadata: Populate CloudSigma's IPs properly
2014-12-01 14:52:11 -08:00
Alex Crawford
123f111efe coreos-cloudinit: bump to 1.0.0+git 2014-11-26 14:19:29 -08:00
Alex Crawford
521ecfdab5 coreos-cloudinit: bump to 1.0.0 2014-11-26 14:19:13 -08:00
Alex Crawford
6d0fdf1a47 Merge pull request #268 from crawford/dropins
drop-in: add support for drop-ins
2014-11-26 14:14:49 -08:00
Alex Crawford
ffc54b028c drop-in: add support for drop-ins
This allows a list of drop-ins for a unit to be declared inline within a
cloud-config. For example:

  #cloud-config
  coreos:
    units:
      - name: docker.service
        drop-ins:
          - name: 50-insecure-registry.conf
            content: |
              [Service]
              Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
2014-11-26 14:09:35 -08:00
Alex Crawford
420f7cf202 system: clean up TestPlaceUnit() 2014-11-26 10:32:43 -08:00
Alex Crawford
624df676d0 config/unit: move Type() and Group() out of config 2014-11-26 10:32:43 -08:00
Alex Crawford
75ed8dacf9 initialize: clean up TestProcessUnits() 2014-11-26 10:32:43 -08:00
Alex Crawford
dcaabe4d4a system: clean up UnitManager interface 2014-11-26 10:32:43 -08:00
Alex Crawford
92c57423ba Merge pull request #269 from crawford/valid
validate: promote invalid values to an error
2014-11-26 10:32:27 -08:00
Alex Crawford
7447e133c9 validate: promote invalid values to an error 2014-11-26 10:29:09 -08:00
Eugene Yakubovich
4e466c12da Merge pull request #267 from thommay/flannel_unit
the flannel service is called flanneld
2014-11-25 12:30:58 -08:00
Thom May
333468dba3 the flannel service is called flanneld 2014-11-25 14:00:53 +00:00
Alex Crawford
55c3a793ad coreos-cloudinit: bump to 0.11.4+git 2014-11-21 20:11:54 -08:00
Alex Crawford
eca51031c8 coreos-cloudinit: bump to 0.11.4 2014-11-21 20:11:37 -08:00
Alex Crawford
19522bcb82 Merge pull request #266 from crawford/config
config: update configs to match etcd, fleet, and flannel
2014-11-21 20:10:34 -08:00
Alex Crawford
62248ea33d config/fleet: fix configs
Added EtcdKeyPrefix and fixed the types of EngineReconcileInterval and EtcdRequestTimeout.
2014-11-21 16:57:00 -08:00
Alex Crawford
d2a19cc86d config/flannel: correct - vs _ 2014-11-21 16:57:00 -08:00
Alex Crawford
08131ffab1 config/etcd: fix configs
This new table is pulled from the etcd codebase rather than the docs...

Added:
 GraphiteHost
 PeerElectionTimeout
 PeerHeartbeatInterval
 PeerKeyFile
 RetryInterval
 SnapshotCount
 StrTrace
 VeryVeryVerbose

Fixed types:
 ClusterActiveSize
 ClusterRemoveDelay
 ClusterSyncInterval
 HTTPReadTimeout
 HTTPWriteTimeout
 MaxResultBuffer
 MaxRetryAttempts
 Snapshot
 Verbose
 VeryVerbose

Renamed:
 Cors

Removed:
 MaxClusterSize
 CPUProfileFile
2014-11-21 16:57:00 -08:00
Alex Crawford
4a0019c669 config: add support for float64 2014-11-21 16:13:49 -08:00
Alex Crawford
3275ead1ec coreos-cloudinit: bump to 0.11.3+git 2014-11-21 12:25:26 -08:00
Alex Crawford
32b6a55724 coreos-cloudinit: bump to 0.11.3 2014-11-21 12:25:04 -08:00
Alex Crawford
6c43644369 Merge pull request #265 from crawford/update
config/update: add "off" as a valid strategy
2014-11-21 12:22:45 -08:00
Alex Crawford
e6593d49e6 config/update: add "off" as a valid strategy
It was assumed that the user would specify the reboot strategy as an
unquoted value. In the case that they turn off updates, `off` is
interpreted as a boolean and the normalization pass converts that to
`false`. In the event that the user uses `"off"`, it's interpreted as a
string and not modified.
2014-11-21 10:41:03 -08:00
Alex Crawford
ab752b239f coreos-cloudinit: bump to 0.11.2+git 2014-11-20 11:29:25 -08:00
Alex Crawford
0742e4d357 coreos-cloudinit: bump to 0.11.2 2014-11-20 11:29:12 -08:00
Alex Crawford
78f586ec9e Merge pull request #262 from crawford/permissions
config: fix parsing of file permissions
2014-11-20 11:28:11 -08:00
Alex Crawford
6f91b76d79 docs: correct type of permissions 2014-11-20 11:14:44 -08:00
Alex Crawford
5c80ccacc4 config: fix parsing of file permissions
The file permissions can be specified (unfortunately) as a string or an
octal integer. During the normalization step, every field is
unmarshalled into an interface{}. String types are kept in tact but
integers are converted to decimal integers. If the raw config
represented the permissions as an octal, it would be converted to
decimal _before_ it was saved to RawFilePermissions. Permissions() would
then try to convert it again, assuming it was an octal. The new behavior
doesn't assume the radix of the number, allowing decimal and octal
input.
2014-11-20 11:14:44 -08:00
Rob Szumski
44fdf95d99 docs: mention validate flag 2014-11-20 11:12:31 -08:00
Rob Szumski
0a62614eec docs: link to validator 2014-11-20 10:58:57 -08:00
Alex Crawford
97758b343b coreos-cloudinit: bump to 0.11.1+git 2014-11-18 12:14:34 -08:00
Alex Crawford
fb6f52b360 coreos-cloudinit: bump to 0.11.1 2014-11-18 12:14:29 -08:00
Alex Crawford
786cd2a539 Merge pull request #259 from crawford/hyphen
config/validate: disable - vs _ message for now
2014-11-18 12:12:26 -08:00
Alex Crawford
45793f1254 config/validate: disable - vs _ message for now 2014-11-18 12:11:50 -08:00
Alex Crawford
b621756d92 Merge pull request #258 from crawford/header
config/validate: fix line number for header check
2014-11-18 12:11:35 -08:00
Alex Crawford
a5b5c700a6 config/validate: fix line number for header check 2014-11-18 12:02:23 -08:00
Kiril Vladimirov
ea95920f31 fix(datasource/CloudSigma): Make sure DHCP has run 2014-11-17 15:35:10 +02:00
Alex Crawford
d7602f3c08 Merge pull request #244 from eyakubovich/master
flannel: added flannel support and helper to make dropins
2014-11-14 10:46:19 -08:00
Eugene Yakubovich
a20addd05e flannel: added flannel support and helper to make dropins
fleet, flannel, and etcd all generate dropins from config.
To reduce code duplication, factor out a helper to do that.
2014-11-14 10:45:23 -08:00
Alex Crawford
d9d89a6fa0 coreos-cloudinit: bump to 0.11.0+git 2014-11-14 10:42:00 -08:00
Alex Crawford
3c26376326 coreos-cloudinit: bump to 0.11.0 2014-11-14 10:41:47 -08:00
Alex Crawford
d3294bcb86 Merge pull request #254 from crawford/validator
config: add new validator
2014-11-12 17:40:16 -08:00
Alex Crawford
dda314b518 flags: add validate flag
This will allow the user to run a standalone validation.
2014-11-12 16:48:57 -08:00
Alex Crawford
055a3c339a config/validate: add new config validator
This validator is still experimental and is going to need new rules in the
future. This lays out the general framework.
2014-11-12 16:48:57 -08:00
Alex Crawford
51f37100a1 config: remove config validator 2014-11-07 10:18:08 -08:00
Alex Crawford
88e8265cd6 config: seperate AssertValid and AssertStructValid
Added an error structure to make it possible to get the specifics of the failure.
2014-11-07 10:14:34 -08:00
Alex Crawford
6e2db882e6 script: move Script into config package 2014-11-07 10:13:52 -08:00
Alex Crawford
3e2823df1b Merge pull request #256 from crawford/hyphen
config: deprecate - in favor of _ for key names
2014-11-03 14:54:23 -08:00
Alex Crawford
46cb51cf91 Merge pull request #257 from crawford/networkd
networkd: remove double-restart workaround
2014-11-03 14:25:38 -08:00
Alex Crawford
1a6cee5305 networkd: remove double-restart workaround
The kernel fixed the underlying issue in 763e0ec and e721f87.
2014-11-03 14:11:15 -08:00
Alex Crawford
d02aa18839 config: deprecate - in favor of _ for key names
In all of the YAML tags, - has been replaced with _. normalizeConfig() and
normalizeKeys() have also been added to perform the normalization of the input
cloud-config.

As part of the normalization process, falsey values are converted to "false".
The "off" update strategy is no exception and as a result the "off" update
strategy has been changed to "false".
2014-11-03 12:09:52 -08:00
Alex Crawford
e9bda98b54 Merge pull request #252 from crawford/vet
go vet
2014-10-23 12:03:01 -07:00
Alex Crawford
badc874b74 travis: install go vet 2014-10-23 11:47:24 -07:00
Alex Crawford
c9e8c887b8 test: run go vet 2014-10-23 11:46:40 -07:00
Alex Crawford
8be307de49 *: fix warnings from go vet 2014-10-23 11:46:08 -07:00
Alex Crawford
562c474275 system: embed config within EtcHosts and Update 2014-10-23 11:44:15 -07:00
Kiril Vladimirov
b6062f0644 fix(datasource/CloudSigma): Populate local IPv4 address properly 2014-10-23 15:03:23 +03:00
Kiril Vladimirov
c5fada6e69 fix(datasource/CloudSigma): Populate public IPv4 address properly 2014-10-23 13:21:49 +03:00
Jonathan Boulle
5c5834863b Merge pull request #250 from jonboulle/master
*: switch to Godeps
2014-10-20 12:09:04 -07:00
Jonathan Boulle
44f0a949c5 *: switch to Godeps 2014-10-20 12:04:03 -07:00
Jonathan Boulle
106c4e7a2c Merge pull request #249 from jonboulle/license_header
*: add license header to all source files
2014-10-17 15:42:20 -07:00
Jonathan Boulle
6c1ba590aa *: add license header to all source files 2014-10-17 15:36:22 -07:00
Alex Crawford
45da664c59 Merge pull request #246 from crawford/master
Add support for Azure
2014-10-12 21:37:34 -07:00
Alex Crawford
2a71551ef2 azure: add support for azure (via azure agent) 2014-10-11 09:19:47 -07:00
Alex Crawford
84e1cb3242 datasource/waagent: add support for WAAgent metadata 2014-10-11 09:19:47 -07:00
Jonathan Boulle
5214ead926 Merge pull request #245 from jonboulle/units
init: simplify CloudConfigUnit interface
2014-10-06 15:26:36 -07:00
Jonathan Boulle
e2c24c4cef init: simplify CloudConfigUnit interface 2014-10-06 15:14:29 -07:00
Alex Crawford
75e288c553 coreos-cloudinit: bump to 0.10.4+git 2014-09-24 19:25:55 -07:00
Alex Crawford
0785840fe3 coreos-cloudinit: bump to 0.10.4 2014-09-24 19:25:34 -07:00
Alex Crawford
c10bfc2f56 Merge pull request #240 from epankala/euca4_compat_fix
AWS: Eucalyptus 4.x compatibility fix
2014-09-24 10:55:39 -07:00
Janne Paenkaelae
2f954dcdc2 AWS: Eucalyptus 4.x compatibility fix
For Eucalyptus 4.0.1 requesting metadata seem to work differently as with EC2.

In Euca:
> curl http://169.254.169.254/2009-04-04
<?xml version="1.0"?><Response><Errors><Error><Code>404 Not Found</Code><Message>unknown</Message></Error></Errors><RequestID>unknown</RequestID></Response>core@localhost ~ $

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

In AWS EC2
> curl http://169.254.169.254/2009-04-04
"" (zero bytes)

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

As the isAvailable() function in metadata.go tests only for errorcode
it fails in Euca.
2014-09-24 20:33:29 +03:00
Alex Crawford
cdfc94f4e9 Merge pull request #234 from crawford/validate
config: explicitly specify fields and seperate config and application
2014-09-24 07:42:09 -07:00
Alex Crawford
18e2f98414 cloudconfig: refactor config
- Move CloudConfig into config package
- Add YAML tags to CloudConfig
2014-09-23 17:59:32 -07:00
Alex Crawford
4b472795c4 user: move User into config package
- Add YAML tags for the fields
2014-09-23 17:59:19 -07:00
Alex Crawford
85b8d804c8 file: refactor config
- Seperate the config from Permissions()
- Add YAML tags for the fields
2014-09-23 17:59:16 -07:00
Alex Crawford
1fbbaaec19 unit: refactor config
- Seperate the config from Destination()
- Add YAML tags for the fields
2014-09-23 17:58:32 -07:00
Alex Crawford
667dbd8fb7 update: refactor config
- Explicitly specify all of the valid options for Update
- Seperate the config from File() and Units()
- Add YAML tags for the fields
2014-09-23 17:57:43 -07:00
Alex Crawford
6730cb7227 oem: refactor the config
- Seperate the config from File()
- Add YAML tags for the fields
2014-09-23 16:08:23 -07:00
Alex Crawford
9454522033 fleet: refactor config
- Explicitly specify all of the valid options for fleet
- Seperate the config from Units()
- Add YAML tags for the fields
2014-09-23 16:07:53 -07:00
Alex Crawford
c255739a93 etcd: refactor config
- Explicitly specify all of the valid options for etcd
- Remove the default name generation (ETCD_NAME is set by its unit file now)
- Seperate the etcd config from Units()
- Remove support for DISCOVERY_URL
- Add YAML tags for the fields
2014-09-23 16:07:13 -07:00
Alex Crawford
2051cd3e1c Merge pull request #238 from crawford/docs
docs: fix documentation of coreos.units.command
2014-09-23 11:33:44 -07:00
Alex Crawford
b52cb3fea3 docs: fix documentation of coreos.units.command 2014-09-23 11:32:15 -07:00
Alex Crawford
da5f85b3fb coreos-cloudinit: bump to 0.10.3+git 2014-09-17 12:19:27 -07:00
Alex Crawford
9999178538 coreos-cloudinit: bump to 0.10.3 2014-09-17 12:19:13 -07:00
Alex Crawford
8f766e4666 Merge pull request #235 from crawford/routes
network: add support for CIDR addresses Debian routes
2014-09-17 12:18:16 -07:00
Alex Crawford
2d28d16c92 network: add support for CIDR addresses Debian routes
OnMetal is changing their template from:
`route add -net 1.2.3.0 netmask 255.255.255.0 gw 10.1.2.1 || true`
to:
`route add -net 1.2.3.0/24 gw 10.1.2.1 || true`
2014-09-16 17:36:34 -07:00
Alex Crawford
e9cd09dd7b coreos-cloudinit: bump to 0.10.2+git 2014-09-14 08:19:57 -07:00
Alex Crawford
8370b30aa2 coreos-cloudinit: bump to 0.10.2 2014-09-14 08:19:33 -07:00
Alex Crawford
3e015cc3a1 Merge pull request #233 from crawford/configdrive
configdrive: don't fail if no network config was provided
2014-09-14 08:18:14 -07:00
Alex Crawford
a0fe6d0884 configdrive: return an empty network config when filename is empty
Additionally, don't bother checking for a network config if it isn't going to
be processed.
2014-09-13 21:51:51 -07:00
Alex Crawford
585ce5fcd9 Revert "metadata: don't fail if no network config was provided"
This reverts commit c1f373e648.
2014-09-13 21:01:42 -07:00
Alex Crawford
72445796ca coreos-cloudinit: bump to 0.10.1+git 2014-09-12 16:48:15 -07:00
Alex Crawford
7342d91a85 coreos-cloudinit: bump to 0.10.1 2014-09-12 16:47:58 -07:00
Alex Crawford
db1bc51c98 Merge pull request #231 from crawford/netconf
metadata: don't fail if no network config was provided
2014-09-12 16:35:24 -07:00
Alex Crawford
c1f373e648 metadata: don't fail if no network config was provided 2014-09-12 16:29:27 -07:00
Alex Crawford
db49a16002 coreos-cloudinit: bump to 0.10.0+git 2014-09-11 17:37:05 -07:00
Alex Crawford
a4a6c281d9 coreos-cloudinit: bump to 0.10.0 2014-09-11 17:36:38 -07:00
Alex Crawford
17f8733121 Merge pull request #228 from crawford/sub
env: add support for escaping environment substitutions
2014-09-11 15:34:03 -07:00
Alex Crawford
7dec922618 env: add support for escaping environment substitutions 2014-09-11 15:30:33 -07:00
Alex Crawford
54d3ae27af Merge pull request #226 from crawford/oem
flags: add oem flag
2014-09-11 13:25:21 -07:00
Alex Crawford
ee2416af64 flags: move the flags into their own namespace 2014-09-11 12:00:17 -07:00
Alex Crawford
cda037f9a5 flags: add oem flag
The oem flag will allow each of the OEMs to specify one flag only, acting as a
shortcut to their specific configuration. This will allow us to update which
options each OEM uses when running cloudinit.
2014-09-11 12:00:17 -07:00
Alex Crawford
549806cf64 Merge pull request #227 from crawford/ipv6
metadata: add support for IPv6 variable substitution
2014-09-11 10:45:33 -07:00
Alex Crawford
56815a6756 metadata: add support for IPv6 variable substitution 2014-09-11 10:43:02 -07:00
Alex Crawford
24a6f7c49c Merge pull request #225 from crawford/exit
userdata: change handling of bad userdata
2014-09-10 19:16:12 -07:00
Alex Crawford
98484be434 userdata: change handling of bad userdata
Don't fail after encountering bad userdata. Continue processing the metadata
and then exit. This will allow people with bad userdata to actually log in and
see the error.
2014-09-10 17:50:23 -07:00
Jonathan Boulle
9024659296 Merge pull request #217 from ecnahc515/patch-1
Fix broken link to fleet config
2014-09-09 15:19:18 -07:00
Chance Zibolski
fc6940f7ba Documentation: More specific link to fleet config.
Add an anchor tag to the url to take person directly to config section.
2014-09-09 15:15:55 -07:00
Brian Waldon
f2fd95699b Merge pull request #224 from bcwaldon/typo
docs: fix a typo
2014-09-09 12:36:42 -07:00
bdevloed
65db96cc7c docs: fix a typo 2014-09-09 12:31:54 -07:00
Alex Crawford
c17b93b5c0 Merge pull request #223 from crawford/yaml
third_party: sync third_party/gopkg.in/yaml.v1
2014-09-08 19:28:59 -07:00
Alex Crawford
d352f8ce6a Merge pull request #222 from crawford/contribute
docs: Update maintainers and contribution guide
2014-09-08 15:54:30 -07:00
Alex Crawford
78aa2c56ec yaml: replace goyaml with yaml 2014-09-08 13:25:27 -07:00
Alex Crawford
c5b3788282 third_party: sync third_party/gopkg.in/yaml.v1
Update launchpad.net/goyaml to gopkg.in/yaml.v1
2014-09-08 13:23:50 -07:00
Alex Crawford
5e98970bb5 docs: Update maintainers and contribution guide 2014-09-08 12:55:17 -07:00
Alex Crawford
cbdd446c55 Merge pull request #220 from crawford/docs
docs: Update list of platforms supporting variable substitutions
2014-09-05 11:51:45 -07:00
Alex Crawford
316cadcf44 docs: Update list of platforms supporting variable substitutions 2014-09-04 12:57:19 -07:00
Alex Crawford
5a939be21b coreos-cloudinit: bump to 0.9.6+git 2014-09-02 17:49:09 -07:00
Alex Crawford
8d76c64386 coreos-cloudinit: bump to 0.9.6 2014-09-02 17:48:45 -07:00
Alex Crawford
1b854eb51e Merge pull request #218 from crawford/units
units: Ensure that the units are executed in order
2014-09-02 17:40:37 -07:00
Alex Crawford
9fcf338bf3 units: Ensure that the units are executed in order 2014-09-02 17:15:32 -07:00
Alex Crawford
fda72bdb5c coreos-cloudinit: bump to 0.9.5+git 2014-09-02 10:10:59 -07:00
Alex Crawford
685a38c6c8 coreos-cloudinit: bump to 0.9.5 2014-09-02 10:10:41 -07:00
Alex Crawford
9d15f2cfaf Merge pull request #213 from crawford/digitalocean
digitalocean: Add support for DigitalOcean
2014-09-01 16:55:12 -07:00
Alex Crawford
2134fce791 digitalocean: Add tests for network unit generation 2014-09-01 16:53:15 -07:00
Alex Crawford
3abd6b2225 digitalocean: Add DigitalOcean metadata service
Move debian-related processing into its own file.
2014-09-01 16:53:15 -07:00
Alex Crawford
2a8e6c9566 network: Fall back to MAC address if there is no name 2014-09-01 09:29:45 -07:00
Alex Crawford
abe43537da metadata: Merge the network config 2014-09-01 09:29:45 -07:00
Jonathan Boulle
3a550af651 Merge pull request #216 from robszumski/patch-2
docs: fix broken link to fleet docs
2014-08-29 11:22:13 -07:00
Rob Szumski
61c3a0eb2d docs: fix broken link to fleet docs 2014-08-29 11:17:05 -07:00
Brian Waldon
480176bc11 Merge pull request #214 from bcwaldon/clarify-write-files
doc: clarify docs around write_files
2014-08-28 20:24:11 -07:00
Brian Waldon
01b18eb551 squash: fix spacing 2014-08-28 13:48:58 -07:00
Brian Waldon
970ef435b6 doc: clarify docs around write_files 2014-08-28 13:33:59 -07:00
Alex Crawford
e8d0021140 Merge pull request #212 from crawford/metadata
refactor: Refactor metadata and datasources to be more testable
2014-08-26 18:45:10 -07:00
Alex Crawford
e9ec78ac6f test: Refactor interface tests a little 2014-08-26 13:18:46 -07:00
Alex Crawford
4a2e417781 network: Add support for multiple addresses and HW addresses
Logical Interfaces can be assigned a hardware address allowing them
to match on MAC address. The static config method also needs to
support specifying multiple addresses.
2014-08-26 13:07:38 -07:00
Alex Crawford
604ef7ecb4 datasource: Add FetchNetworkConfig
FetchNetworkConfig is currently only used by ConfigDrive to read the
network config file from the disk.
2014-08-26 13:04:43 -07:00
Alex Crawford
c39dd5cc67 networkd: Fix bug causing bonding to always be loaded 2014-08-26 13:04:21 -07:00
Alex Crawford
a923161f4a metadata: Refactor common parts out of ec2 2014-08-26 12:02:56 -07:00
Alex Crawford
e59e2f6cd5 Merge pull request #210 from crawford/test
test: Add gofmt to test
2014-08-25 17:04:04 -07:00
Alex Crawford
e90fe3eba8 test: Add gofmt to test 2014-08-25 12:48:52 -07:00
Alex Crawford
fb0187b197 gofmt: sort 2014-08-25 12:35:40 -07:00
Michael Marineau
6babe74716 Merge pull request #209 from marineam/go13
travis: enable testing under go 1.3
2014-08-25 12:26:23 -07:00
Michael Marineau
b1e88284ca travis: enable testing under go 1.3 2014-08-25 12:21:07 -07:00
Alex Crawford
18a65f7dac Merge pull request #208 from crawford/go
test: Fix tests for Go 1.3
2014-08-25 12:19:52 -07:00
Alex Crawford
0c212c72c9 test: Fix tests for Go 1.3 2014-08-25 12:01:27 -07:00
Alex Crawford
6a800d8cc0 coreos-cloudinit: bump to 0.9.4+git 2014-08-24 18:41:20 -07:00
Alex Crawford
5e112147bb coreos-cloudinit: bump to 0.9.4 2014-08-24 18:40:53 -07:00
Alex Crawford
7e78b1563f Merge pull request #206 from crawford/tests
test: Enable tests for CloudSigma datasource
2014-08-24 18:36:38 -07:00
Alex Crawford
ecbe81f103 test: Enable tests for CloudSigma datasource 2014-08-24 17:08:49 -07:00
Alex Crawford
45c20c1dd3 Merge pull request #196 from Vladimiroff/cloudsigma
cloudsigma: Add support for CloudSigma datasource
2014-08-15 15:21:33 -07:00
Alex Crawford
8ce925a060 coreos-cloudinit: bump to 0.9.3+git 2014-08-15 10:47:28 -07:00
Alex Crawford
eadb6ef42c coreos-cloudinit: bump to 0.9.3 2014-08-15 10:46:46 -07:00
Alex Crawford
7518f0ec93 Merge pull request #204 from crawford/configdrive
configdrive: Remove broken support for ec2 metadata
2014-08-15 10:43:26 -07:00
Alex Crawford
f0b9eaf2fe configdrive: Remove broken support for ec2 metadata
As it turns out, certain metadata is only present in the ec2 flavor
of metadata (e.g. public_ipv4) and other data is only present in
the openstack flavor (e.g. network_config). For now, just read the
openstack metadata.
2014-08-15 10:35:21 -07:00
Kiril Vladimirov
7320a2cbf2 feat(datasource/metadata): Add datasource for CloudSigma 2014-08-15 12:08:55 +03:00
Kiril Vladimirov
57950b3ed9 add(goserial): import github.com/tarm/goserial 2014-08-15 12:08:34 +03:00
Kiril Vladimirov
85c6a2a16a add(cepgo): import github.com/cloudsigma/cepgo 2014-08-15 12:07:58 +03:00
Jonathan Boulle
24b44e86a6 coreos-cloudinit: bump to 0.9.2+git 2014-08-12 11:38:51 -07:00
Jonathan Boulle
2f52ad4ef8 coreos-cloudinit: bump to 0.9.2 2014-08-12 11:38:12 -07:00
Jonathan Boulle
735d6c6161 Merge pull request #202 from jonboulle/env
environment: write new keys in consistent order
2014-08-11 22:40:42 -07:00
Alex Crawford
1cf275bad6 Merge pull request #201 from crawford/configdrive
configdrive: fix root path
2014-08-11 20:11:17 -07:00
Jonathan Boulle
f1c97cb4d5 environment: write new keys in consistent order 2014-08-11 18:24:58 -07:00
Alex Crawford
d143904aa9 configdrive: fix root path 2014-08-11 17:57:10 -07:00
Jonathan Boulle
c428ce2cc5 Merge pull request #200 from jonboulle/fu
initialize: use correct heuristic to check if etcdenvironment is set
2014-08-11 17:44:44 -07:00
Jonathan Boulle
dfb5b4fc3a initialize: use correct heuristic to check if etcdenvironment is set
In some circumstances (e.g. nova-agent-watcher) cloudconfig files will
be created where the EtcdEnvironment is an empty map, and hence != nil.
If this is the case we should not do anything at all (because the user
hasn't explicitly asked us to configure etcd). This change standardises
behaviour with the check that we already do for FleetEnvironment.
2014-08-11 16:01:08 -07:00
Alex Crawford
97d5538533 Merge pull request #197 from crawford/ec2
datasource: Fix ec2 URLs
2014-08-06 22:45:03 -07:00
Alex Crawford
6b8f82b5d3 datasource: Fix ec2 URLs
_ vs -
2014-08-06 21:31:43 -07:00
Alex Crawford
facde6609f Merge pull request #194 from crawford/metadata
datasource: Refactoring datasources
2014-08-06 15:55:13 -07:00
Alex Crawford
d68ae84b37 metadata: Refactor metadata service into ec2 metadata
Added more testing.
2014-08-05 17:19:43 -07:00
Alex Crawford
54aa39543b timeouts: Use After() instead of Tick() 2014-08-04 15:10:14 -07:00
Alex Crawford
8566a2c118 datasource: Move datasources into their own packages. 2014-08-04 15:10:07 -07:00
Alex Crawford
49ac083af5 coreos-cloudinit: bump to 0.9.1+git 2014-08-04 14:14:24 -07:00
Alex Crawford
5d65ca230a coreos-cloudinit: bump to 0.9.1 2014-08-04 14:13:51 -07:00
Alex Crawford
38b3e1213a Merge pull request #188 from crawford/configdrive
configdrive: Use the EC2 metadata over OpenStack
2014-08-04 11:12:06 -07:00
Alex Crawford
4eedca26e9 configdrive: Use the EC2 metadata over OpenStack
Standardize on specific EC2 and OpenStack versions and add tests.
2014-08-04 10:18:29 -07:00
Brian Waldon
f2b342c8be doc: escape user.home example 2014-08-01 13:20:44 -07:00
Michael Marineau
c19d8f6b61 Merge pull request #193 from benjic/cloudconfig_variables
docs(quick-start): Clarified use of fields in cloud config
2014-07-24 11:02:03 -07:00
Benjamin Campbell
7913f74351 docs(quick-start) Enumerated supported platforms
Following suggestion a list of platforms that *do* support cloud config variables. In addition minor mark up formatting is added.
2014-07-24 11:54:31 -06:00
Benjamin Campbell
5593408be8 docs(quick-start): Clarified use of fields in cloud config
Updated the language to illustrate that fields in a cloud config is not
supported in all environments. This is expressed explicitly in PXE and
install to disk pages. The quick start lacked this information and is
inconsistent.
2014-07-24 11:27:35 -06:00
Alex Crawford
7fc67c2acf Merge pull request #191 from crawford/panic
config: Verify that type assertions are valid
2014-07-22 11:51:39 -07:00
Alex Crawford
b093094292 config: Verify that type assertions are valid 2014-07-22 11:39:20 -07:00
Michael Marineau
9a80fd714a Merge pull request #181 from robszumski/docs-startup
fix(docs): clarity around boot behavior and unit usage
2014-07-21 22:12:19 -07:00
Rob Szumski
fef5473881 fix(docs): clarity around boot behavior and unit usage 2014-07-21 21:41:00 -07:00
Alex Crawford
bf5a2b208f coreos-cloudinit: bump to 0.9.0+git 2014-07-21 19:17:14 -07:00
Alex Crawford
364507fb75 coreos-cloudinit: bump to 0.9.0 2014-07-21 19:16:11 -07:00
Alex Crawford
08d4842502 Merge pull request #190 from crawford/logs
Logs
2014-07-21 12:22:41 -07:00
Alex Crawford
21e32e44f8 system: Add more logging for networkd 2014-07-21 11:25:22 -07:00
Alex Crawford
7a06dee16f system: Cleanup redundant code 2014-07-21 11:24:42 -07:00
Alex Crawford
ff9cf5743d Merge pull request #187 from crawford/order
networkd: Reverse lexicographic order of generated unit files
2014-07-18 13:23:58 -07:00
Alex Crawford
1b10a3a187 networkd: Reverse lexicographic order of generated unit files 2014-07-17 20:47:37 -07:00
Michael Marineau
10838e001d Merge pull request #186 from robszumski/add-highlighting
feat(docs): add syntax highlighting
2014-07-15 15:26:33 -07:00
Rob Szumski
96370ac5b9 feat(docs): add syntax highlighting 2014-07-14 16:16:14 -07:00
Michael Marineau
0b82cd074d Merge pull request #180 from marineam/systemd_testing
chore(*): split out unit processing from config.Apply
2014-07-11 20:09:08 -07:00
Alex Crawford
a974e85103 Merge pull request #174 from crawford/teeth
networkd: Fix issues with bonding and VLANs
2014-07-11 15:48:02 -07:00
Michael Marineau
f0450662b0 Merge pull request #183 from marineam/fix
tests: fix error messages, use Fatalf
2014-07-11 15:40:54 -07:00
Michael Marineau
03e29d1291 tests: fix error messages, use Fatalf 2014-07-11 15:38:04 -07:00
Michael Marineau
98ae5d88aa coreos-cloudinit: bump to 0.8.9+git 2014-07-11 14:40:57 -07:00
Michael Marineau
bf5d3539c9 coreos-cloudinit: bump to 0.8.9 2014-07-11 14:40:34 -07:00
Michael Marineau
5e4cbcd909 Merge pull request #182 from marineam/fix
env_file: fix broken test cases
2014-07-11 14:38:56 -07:00
Michael Marineau
a270c4c737 env_file: fix broken test cases
TestWriteEnvFileDos2Unix had a copy/paste bug, it shouldn't have
asserted that mtime doesn't change because the file is actually being
modified in this test. This didn't come up earlier because the actual
comparison wasn't using Time.Equal as it should have.

Instead switch to comparing inode numbers which is the actual thing I
wanted to test for in the first place, just accessing them is much more
awkard. Now all tests where it is relevant check the inode in addition
to the contents.
2014-07-11 13:35:10 -07:00
Michael Marineau
f356a8a690 coreos-cloudinit: bump to 0.8.8+git 2014-07-11 11:13:01 -07:00
Michael Marineau
b1a897d75c coreos-cloudinit: bump to 0.8.8 2014-07-11 11:12:15 -07:00
Jonathan Boulle
be51f4eba0 chore(*): split out unit processing from config.Apply 2014-07-11 10:44:19 -07:00
Michael Marineau
a55e2cd49b Merge pull request #178 from marineam/env
Write /etc/environment
2014-07-11 10:39:33 -07:00
Michael Marineau
983501e43b environment: add support for updating /etc/environment with IP values
To maintain the behavior of the coreos-setup-environment that has
started to move into cloudinit we need to write out /etc/environment
with the public and private addresses, if known. The file is updated so
that other contents are not replaced. This behavior is disabled entirely
if /etc/environment was written by a write_files entry.
2014-07-11 10:34:44 -07:00
Alex Crawford
e3037f18a6 networkd: Restart networkd twice to work around race
https://bugs.freedesktop.org/show_bug.cgi?id=76077
2014-07-10 23:40:42 -07:00
Alex Crawford
fe388a3ab6 networkd: Create config directory before writing config 2014-07-10 23:40:42 -07:00
Alex Crawford
c820f2b1cf bonding: Add support for probing the bonding module with parameters
Until support for bonding params is added to networkd, this will be
neccessary in order to use bonding parameters (i.e. miimon, mode).
This also makes it such that the 8012q module will only be loaded if
the network config makes use of VLANs.
2014-07-10 23:40:42 -07:00
Michael Marineau
81824be3bf system: new file writer for updating env-style files
This can be used to safely update config files cloudinit does not have
exclusive control over. For example update.conf or /etc/environment.
2014-07-10 15:53:32 -07:00
Michael Marineau
98c26440be Merge pull request #176 from jayofdoom/master
Document need for #cloud-config in cloud-config.yml
2014-07-09 16:41:00 -07:00
Jay Faulkner
3b5fcc393b Document need for #cloud-config in cloud-config.yml
- cloud-config.yml does not work if it's missing the #cloud-config
  directive at the top. This is undocumented, except in the examples.
2014-07-09 16:36:11 -07:00
Alex Crawford
9528077340 coreos-cloudinit: bump to 0.8.7+git 2014-07-02 15:20:45 -07:00
Alex Crawford
4355a05d55 coreos-cloudinit: bump to 0.8.7 2014-07-02 15:20:26 -07:00
Alex Crawford
52c44923dd Merge pull request #173 from crawford/metadata
metadata-service: remove check for OpenStack meta_data.json
2014-07-02 15:19:37 -07:00
Alex Crawford
47748ef4b6 metadata-service: remove check for OpenStack meta_data.json
The meta_data.json blob under OpenStack doesn't actually contain all
of the metadata... Fall back to explicitly requesting each attribute.
2014-07-02 14:38:23 -07:00
Alex Crawford
8eca10200e coreos-cloudinit: bump to 0.8.6+git 2014-07-01 16:17:00 -07:00
Alex Crawford
43be8c8996 coreos-cloudinit: bump to 0.8.6 2014-07-01 16:16:41 -07:00
Alex Crawford
19b4b1160e Merge pull request #171 from crawford/err
metadata-service: Handle no user-data
2014-07-01 16:15:32 -07:00
Alex Crawford
ce6fccfb3c metadata-service: Handle no user-data 2014-07-01 16:10:18 -07:00
Alex Crawford
7d89aefb82 coreos-cloudinit: bump to 0.8.5+git 2014-07-01 15:45:49 -07:00
Alex Crawford
2369e2a920 coreos-cloudinit: bump to 0.8.5 2014-07-01 15:45:23 -07:00
Alex Crawford
6d808048d3 Merge pull request #170 from crawford/metadata
metadata: Fetch the public and private IP addresses
2014-07-01 15:44:14 -07:00
Alex Crawford
276f0b5d99 metadata: Fetch the public and private IP addresses 2014-07-01 14:43:19 -07:00
Jonathan Boulle
92bd5ca5d4 coreos-cloudinit: bump to 0.8.4+git 2014-07-01 12:16:09 -07:00
Jonathan Boulle
5b5ffea126 coreos-cloudinit: bump to 0.8.4 2014-07-01 12:15:48 -07:00
Jonathan Boulle
18068e9375 Merge pull request #169 from jonboulle/pebkac
coreos-cloudinit: apply environment to userdata string
2014-07-01 12:15:06 -07:00
Jonathan Boulle
1b3cabb035 coreos-cloudinit: apply environment to userdata string 2014-07-01 12:08:42 -07:00
Jonathan Boulle
1be2bec1c2 coreos-cloudinit: bump to 0.8.3+git 2014-06-30 22:12:13 -07:00
Jonathan Boulle
f3bd5f543e coreos-cloudinit: bump to 0.8.3 2014-06-30 22:11:15 -07:00
Jonathan Boulle
660feb59b9 Merge pull request #168 from jonboulle/foo
fix ordering error in mergeCloudConfig
2014-06-30 22:08:47 -07:00
Jonathan Boulle
9673dbe12b coreos-cloudinit: fix ordering error in merge invocation 2014-06-30 22:07:05 -07:00
Alex Crawford
2be435dd83 coreos-cloudinit: bump to 0.8.2+git 2014-06-30 18:11:14 -07:00
Alex Crawford
2d91369596 coreos-cloudinit: bump to 0.8.2 2014-06-30 18:10:20 -07:00
Alex Crawford
d8d3928978 Merge pull request #167 from crawford/sshkeys
metadata-service: fix ssh key retrieval and application
2014-06-30 18:08:04 -07:00
Alex Crawford
7fcc540154 metadata-service: fix ssh key retrieval and application
The metadata service wasn't properly fetching the ssh keys from metadata.
Drop the key traversal in favor of explict key urls.
2014-06-30 17:45:08 -07:00
Jonathan Boulle
cb7fbd4668 Merge pull request #166 from jonboulle/merge
cloudinit: merge cloudconfig info from user-data and meta-data
2014-06-30 11:27:47 -07:00
Jonathan Boulle
d4e048a1f4 ParseUserData: return nil on empty input string 2014-06-30 11:27:33 -07:00
Jonathan Boulle
231c0fa20b initialize: add tests for ParseMetadata 2014-06-27 23:53:06 -07:00
Jonathan Boulle
1aabacc769 cloudinit: merge cloudconfig info from user-data and meta-data
This attempts to retrieve cloudconfigs from two sources: the meta-data
service, and the user-data service. If only one cloudconfig is found,
that is applied to the system. If both services return a cloudconfig,
the two are merged into a single cloudconfig which is then applied to
the system.

Only a subset of parameters are merged (because the meta-data service
currently only partially populates a cloudconfig). In the event of any
conflicts, parameters in the user-data cloudconfig take precedence over
those in the meta-data cloudconfig.
2014-06-27 23:48:48 -07:00
Alex Crawford
6a2927d701 coreos-cloudinit: bump to 0.8.1+git 2014-06-27 15:00:05 -07:00
Alex Crawford
126188510b coreos-cloudinit: bump to 0.8.1 2014-06-27 14:59:56 -07:00
Alex Crawford
4627ccb444 Merge pull request #165 from crawford/units
units: update dependencies
2014-06-27 14:55:48 -07:00
Alex Crawford
aff372111a units: update dependencies 2014-06-27 14:29:59 -07:00
Alex Crawford
c7081b9918 coreos-cloudinit: bump to 0.8.0+git 2014-06-27 11:33:56 -07:00
Alex Crawford
9ba3b18b59 coreos-cloudinit: bump to 0.8.0 2014-06-27 11:32:52 -07:00
Alex Crawford
099de62e9a Merge pull request #164 from crawford/datasources
datasources: Add support for specifying multiple datasources
2014-06-27 00:25:09 -07:00
Alex Crawford
c089216cb5 datasources: Add support for specifying multiple datasources
If multiple sources are specified, the first available source is used.
2014-06-26 22:32:39 -07:00
Alex Crawford
68dc902ed1 HttpClient: Refactor timeout into two seperate functions 2014-06-26 15:16:22 -07:00
Jonathan Boulle
ad66b1c92f Merge pull request #163 from jonboulle/net
coreos-cloudinit: restrict convert-netconf to configdrive
2014-06-25 14:35:27 -07:00
Jonathan Boulle
fbdece2762 coreos-cloudinit: restrict convert-netconf to configdrive 2014-06-25 14:28:11 -07:00
Jonathan Boulle
f85eafb7ca Merge pull request #162 from jonboulle/fffffffffffffffffff
initialize/env: handle nil substitution maps properly
2014-06-25 12:10:00 -07:00
Jonathan Boulle
f0dba2294e initialize/env: handle nil substitution maps properly 2014-06-25 12:07:48 -07:00
Alex Crawford
bda3948382 Merge pull request #159 from crawford/metadata
metadataService: Check both ec2 and openstack urls more explicitly
2014-06-25 11:26:19 -07:00
Alex Crawford
fae81c78f3 metadataService: Check both ec2 and openstack urls more explicitly
Remove the root url parameter for -from-metadata-service since this
is a guarenteed value. Additionally, check for both ec2 and openstack
urls for the metadata and userdata. Fix a bug with the -from-url
option and a panic on an empty response.
2014-06-25 11:19:11 -07:00
Alex Crawford
a5dec7d7bd cloudconfig: Process metadata before userdata
This gives the options in userdata a higher precedence over metadata.
2014-06-25 10:35:26 -07:00
Jonathan Boulle
e1222c9885 Merge pull request #161 from jonboulle/doc
metadata: add links to metadata source information
2014-06-25 08:55:26 -07:00
Jonathan Boulle
ded3bcf122 metadata: add links to metadata source information 2014-06-24 19:26:07 -07:00
Jonathan Boulle
80d00cde94 Merge pull request #158 from jonboulle/nettttt
cloudinit: retrieve IPv4 addresses from metadata
2014-06-24 19:11:15 -07:00
Jonathan Boulle
2805d70ece initialize/env: add notes about tests 2014-06-24 18:52:08 -07:00
Jonathan Boulle
439b7e8b98 initialize/env: fall back to COREOS_*_IPV4 env variables 2014-06-24 18:49:49 -07:00
Jonathan Boulle
ba1c1e97d0 cloudinit: retrieve IPv4 addresses from metadata
This uses the new MetadataService implementation to retrieve values for
$private_ipv4 and $public_ipv4 substitutions, instead of using
environment variables.
2014-06-24 17:46:06 -07:00
Alex Crawford
8a50fd8595 Merge pull request #154 from crawford/metadata
metadata-service: Add new datasource to download from metadata service
2014-06-24 15:18:27 -07:00
Alex Crawford
465bcce72c metadata_service: Add tests for constructing metadata 2014-06-24 15:08:03 -07:00
Alex Crawford
361edeebc6 metadata-service: Add metadata-service datasource
Move the old metadata-service datasource to url datasource. This new datasource
checks for the existance of meta-data.json and if it doesn't exist, walks the
meta-data directory to build a metadata blob.
2014-06-24 15:08:03 -07:00
Jonathan Boulle
29a7b0e34f Merge pull request #155 from jonboulle/etcd
etcdenvironment: order map keys consistently
2014-06-24 15:02:04 -07:00
Alex Crawford
8496ffb53a HttpClient: Wrap errors with error classes 2014-06-24 15:01:31 -07:00
Jonathan Boulle
2c717a6cd1 Merge pull request #157 from jonboulle/core
coreos-cloudinit: clean up flag handling
2014-06-24 14:57:18 -07:00
Jonathan Boulle
13a91c9181 coreos-cloudinit: clean up flag handling 2014-06-24 14:19:55 -07:00
Jonathan Boulle
338e1b64ab etcdenvironment: order map keys consistently 2014-06-23 15:13:11 -07:00
Alex Crawford
8eb0636034 Merge pull request #149 from crawford/network
feat(network): Add support for blind interfaces, support for hwaddress, and bug fixes
2014-06-20 17:53:37 -07:00
Alex Crawford
f7c25a1b83 doc(debian-interfaces): Add basic docs for convert-netconf 2014-06-20 17:51:57 -07:00
Alex Crawford
d6a0d0908c fix(network): Generate prefixes to ensure proper lexicographical ordering
In order for networkd to properly configure the network interfaces, the configs must be
prefixed to ensure that they load in the correct order (parent interfaces have a lower
prefix than their children).
2014-06-20 17:51:57 -07:00
Alex Crawford
5c89afc18a Merge pull request #152 from crawford/metadata
feat(meta_data): Add partial support for meta_data.json
2014-06-20 17:47:08 -07:00
Michael Marineau
376cc4bcac chore(coreos-cloudinit): bump to 0.7.7+git 2014-06-18 15:01:13 -07:00
Michael Marineau
d0a6d6f92f chore(coreos-cloudinit): bump to 0.7.7 2014-06-18 14:55:38 -07:00
Michael Marineau
2be1e52f32 Merge pull request #151 from marineam/mount
fix(configdrive): Use mount units, give virtfs a new mount point.
2014-06-18 13:51:11 -07:00
Michael Marineau
784a71e2bf fix(configdrive): Use mount units, give virtfs a new mount point.
Currently systemd cannot track dependencies on configdrive very well
because it is mounted via a service instead of a mount unit. Also since
the interaction between path and mount units can lead to unexpected
behavior if something goes wrong the cloudinit service is now triggered
explicitly by the mount again. The configdrive path unit remains only as
a fall back for containers where the mount unit doesn't kick in. Better
to have two mechanisms that trigger the cloudinit service than none. :)

Since mounting a virtfs based configdrive requires different mount
options and two different mount units cannot refer to the same path the
virtfs version now mounts to /media/configvirtfs.

There are also two new kernel options:
- `coreos.configdrive=1`: enable config drive on physical hardware.
- `coreos.configdrive=0`: disable config drive on virtual machines.
2014-06-18 13:01:19 -07:00
Alex Crawford
e6cf83a2e5 refactor(netconf): Move netconf processing and handle metadata 2014-06-18 12:43:41 -07:00
Alex Crawford
840c208b60 feat(metadata): Distinguish between userdata and metadata for datasources 2014-06-18 12:34:31 -07:00
Alex Crawford
29ed6b38bd refactor(env): Add the config root and netconf type to datasource and env 2014-06-18 12:27:15 -07:00
Alex Crawford
259c7e1fe2 fix(sshKeyName): Use the SSH key name provided 2014-06-18 11:47:17 -07:00
Alex Crawford
033c8d352f feat(network): Add support for hwaddress
Currently only supports the ether mode of hwaddress. No immediate plans
to support ax25, ARCnet, or netrom.
2014-06-14 21:30:14 -07:00
Alex Crawford
16d7e8af48 fix(network): Take down all interfaces properly
The map of interfaces wasn't being populated correctly. Also, clean up some prints.
2014-06-13 20:53:59 -07:00
Alex Crawford
159f4a2c7c feat(network): Add support for blind interfaces
It is valid for an interface to reference another, otherwise undeclared,
interface (i.e. a bond enslaves eth0 without eth0 having its own iface stanza).
In order to associate the two interfaces, the undeclared interface needs to be
implicitly created so that it can be referenced by the other. This adds the
capability to forward-declare interfaces in addition to cleaning up the
process a little bit.
2014-06-11 22:07:33 -07:00
Michael Marineau
160668284c chore(coreos-cloudinit): bump to 0.7.6+git 2014-06-07 16:04:33 -04:00
Michael Marineau
41b9dfcb1c chore(coreos-cloudinit): bump to 0.7.6 2014-06-07 16:01:31 -04:00
Michael Marineau
ef4c3483b6 Merge pull request #146 from marineam/fix
fix(update): Fix restart of update-engine
2014-06-07 13:04:49 -07:00
Michael Marineau
4bdf633075 fix(update): Fix restart of update-engine
The name was missing .service.
2014-06-07 12:08:22 -07:00
Brian Waldon
c9fc718e18 Merge pull request #145 from bcwaldon/drop-group-req
Relax requirements of update group value
2014-06-06 11:43:22 -07:00
Brian Waldon
4461b3d33d fix(update): Relax requirements of update group value 2014-06-06 11:29:09 -07:00
Jonathan Boulle
c6a1412f6b chore(coreos-cloudinit): bump to 0.7.5+git 2014-06-06 11:14:39 -07:00
239 changed files with 12772 additions and 4173 deletions

View File

@@ -1,8 +1,17 @@
language: go language: go
go: 1.2 sudo: false
matrix:
include:
- go: 1.4
env: TOOLS_CMD=golang.org/x/tools/cmd
- go: 1.3
env: TOOLS_CMD=code.google.com/p/go.tools/cmd
- go: 1.2
env: TOOLS_CMD=code.google.com/p/go.tools/cmd
install: install:
- go get code.google.com/p/go.tools/cmd/cover - go get ${TOOLS_CMD}/cover
- go get ${TOOLS_CMD}/vet
script: script:
- ./test - ./test

View File

@@ -39,22 +39,25 @@ Thanks for your contributions!
### Format of the Commit Message ### Format of the Commit Message
We follow a rough convention for commit messages borrowed from AngularJS. This We follow a rough convention for commit messages that is designed to answer two
is an example of a commit: questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
``` ```
feat(scripts/test-cluster): add a cluster test command environment: write new keys in consistent order
this uses tmux to setup a test cluster that you can easily kill and Go 1.3 randomizes the ordering of keys when iterating over a map.
start for debugging. Sort the keys to make this ordering consistent.
Fixes #38
``` ```
The format can be described more formally as follows: The format can be described more formally as follows:
``` ```
<type>(<scope>): <subject> <subsystem>: <what changed>
<BLANK LINE> <BLANK LINE>
<body> <why this change was made>
<BLANK LINE> <BLANK LINE>
<footer> <footer>
``` ```
@@ -63,25 +66,3 @@ The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters. second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various This allows the message to be easier to read on GitHub as well as in various
git tools. git tools.
#### Subject Line
The subject line contains a succinct description of the change.
#### Allowed `<type>`s
- *feat* (feature)
- *fix* (bug fix)
- *docs* (documentation)
- *style* (formatting, missing semi colons, …)
- *refactor*
- *test* (when adding missing tests)
- *chore* (maintain)
#### Allowed `<scope>`s
Scopes can anything specifying the place of the commit change in the code base -
for example, "api", "store", etc.
For more details on the commit format, see the [AngularJS commit style
guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).

View File

@@ -13,7 +13,7 @@ If no **id** field is provided, coreos-cloudinit will ignore this section.
For example, the following cloud-config document... For example, the following cloud-config document...
``` ```yaml
#cloud-config #cloud-config
coreos: coreos:
oem: oem:
@@ -26,7 +26,7 @@ coreos:
...would be rendered to the following `/etc/oem-release`: ...would be rendered to the following `/etc/oem-release`:
``` ```yaml
ID=rackspace ID=rackspace
NAME="Rackspace Cloud Servers" NAME="Rackspace Cloud Servers"
VERSION_ID=168.0.0 VERSION_ID=168.0.0

View File

@@ -2,9 +2,11 @@
CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime.
Your cloud-config is processed during each boot. Invalid cloud-config won't be processed but will be logged in the journal. You can validate your cloud-config with the [CoreOS validator]({{site.url}}/validate) or by running `coreos-cloudinit -validate`.
## Configuration File ## Configuration File
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units. The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file, which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms. We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
@@ -16,7 +18,7 @@ We've designed our implementation to allow the same cloud-config file to work ac
The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values. The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values.
A cloud-config file should contain an associative array which has zero or more of the following keys: A cloud-config file must contain `#cloud-config`, followed by an associative array which has zero or more of the following keys:
- `coreos` - `coreos`
- `ssh_authorized_keys` - `ssh_authorized_keys`
@@ -40,9 +42,9 @@ CoreOS tries to conform to each platform's native method to provide user data. E
#### etcd #### etcd
The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file. The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document... If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
``` ```yaml
#cloud-config #cloud-config
coreos: coreos:
@@ -57,7 +59,7 @@ coreos:
...will generate a systemd unit drop-in like this: ...will generate a systemd unit drop-in like this:
``` ```yaml
[Service] [Service]
Environment="ETCD_NAME=node001" Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>" Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
@@ -66,7 +68,8 @@ Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
``` ```
For more information about the available configuration parameters, see the [etcd documentation][etcd-config]. For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md [etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
@@ -74,7 +77,7 @@ Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document... The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
``` ```yaml
#cloud-config #cloud-config
coreos: coreos:
@@ -85,7 +88,7 @@ coreos:
...will generate a systemd unit drop-in like this: ...will generate a systemd unit drop-in like this:
``` ```yaml
[Service] [Service]
Environment="FLEET_PUBLIC_IP=203.0.113.29" Environment="FLEET_PUBLIC_IP=203.0.113.29"
Environment="FLEET_METADATA=region=us-west" Environment="FLEET_METADATA=region=us-west"
@@ -93,7 +96,64 @@ Environment="FLEET_METADATA=region=us-west"
For more information on fleet configuration, see the [fleet documentation][fleet-config]. For more information on fleet configuration, see the [fleet documentation][fleet-config].
[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/configuration.md [fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md#configuration
#### flannel
The `coreos.flannel.*` parameters also work very similarly to `coreos.etcd.*`
and `coreos.fleet.*`. They can be used to set environment variables for
flanneld. For example, the following cloud-config...
```yaml
#cloud-config
coreos:
flannel:
etcd_prefix: /coreos.com/network2
```
...will generate a systemd unit drop-in like so:
```
[Service]
Environment="FLANNELD_ETCD_PREFIX=/coreos.com/network2"
```
List of flannel configuration parameters:
- **etcd_endpoints**: Comma separated list of etcd endpoints
- **etcd_cafile**: Path to CA file used for TLS communication with etcd
- **etcd_certfile**: Path to certificate file used for TLS communication with etcd
- **etcd_keyfile**: Path to private key file used for TLS communication with etcd
- **etcd_prefix**: Etcd prefix path to be used for flannel keys
- **ip_masq**: Install IP masquerade rules for traffic outside of flannel subnet
- **subnet_file**: Path to flannel subnet file to write out
- **interface**: Interface (name or IP) that should be used for inter-host communication
[flannel-readme]: https://github.com/coreos/flannel/blob/master/README.md
#### locksmith
The `coreos.locksmith.*` parameters can be used to set environment variables
for locksmith. For example, the following cloud-config...
```yaml
#cloud-config
coreos:
locksmith:
endpoint: example.com:4001
```
...will generate a systemd unit drop-in like so:
```
[Service]
Environment="LOCKSMITHD_ENDPOINT=example.com:4001"
```
For the complete list of locksmith configuraion parameters, see the [locksmith documentation][locksmith-readme].
[locksmith-readme]: https://github.com/coreos/locksmith/blob/master/README.md
#### update #### update
@@ -114,7 +174,7 @@ The `reboot-strategy` parameter also affects the behaviour of [locksmith](https:
##### Example ##### Example
``` ```yaml
#cloud-config #cloud-config
coreos: coreos:
update: update:
@@ -123,14 +183,20 @@ coreos:
#### units #### units
The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields: The `coreos.units.*` parameters define a list of arbitrary systemd units to start after booting. This feature is intended to help you start essential services required to mount storage and configure networking in order to join the CoreOS cluster. It is not intended to be a Chef/Puppet replacement.
Each item is an object with the following fields:
- **name**: String representing unit's name. Required. - **name**: String representing unit's name. Required.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. Default value is false. - **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. The default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false. - **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. The default value is false.
- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already. - **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart. - **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. The default behavior is to not execute any commands.
- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. Default value is false. - **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. The default value is false.
- **drop-ins**: A list of unit drop-ins with the following fields:
- **name**: String representing unit's name. Required.
- **content**: Plaintext string representing entire file. Required.
**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place. **NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
@@ -138,7 +204,7 @@ The `coreos.units.*` parameters define a list of arbitrary systemd units to star
Write a unit to disk, automatically starting it. Write a unit to disk, automatically starting it.
``` ```yaml
#cloud-config #cloud-config
coreos: coreos:
@@ -157,9 +223,24 @@ coreos:
ExecStop=/usr/bin/docker stop -t 2 redis_server ExecStop=/usr/bin/docker stop -t 2 redis_server
``` ```
Add the DOCKER_OPTS environment variable to docker.service.
```yaml
#cloud-config
coreos:
units:
- name: docker.service
drop-ins:
- name: 50-insecure-registry.conf
content: |
[Service]
Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
```
Start the built-in `etcd` and `fleet` services: Start the built-in `etcd` and `fleet` services:
``` ```yaml
#cloud-config #cloud-config
coreos: coreos:
@@ -177,7 +258,7 @@ The `ssh_authorized_keys` parameter adds public SSH keys which will be authorize
The keys will be named "coreos-cloudinit" by default. The keys will be named "coreos-cloudinit" by default.
Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`. Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`.
``` ```yaml
#cloud-config #cloud-config
ssh_authorized_keys: ssh_authorized_keys:
@@ -189,7 +270,7 @@ ssh_authorized_keys:
The `hostname` parameter defines the system's hostname. The `hostname` parameter defines the system's hostname.
This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`). This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
``` ```yaml
#cloud-config #cloud-config
hostname: coreos1 hostname: coreos1
@@ -203,13 +284,14 @@ All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the use
- **name**: Required. Login name of user - **name**: Required. Login name of user
- **gecos**: GECOS comment of user - **gecos**: GECOS comment of user
- **passwd**: Hash of the password to use for this user - **passwd**: Hash of the password to use for this user
- **homedir**: User's home directory. Defaults to /home/<name> - **homedir**: User's home directory. Defaults to /home/\<name\>
- **no-create-home**: Boolean. Skip home directory creation. - **no-create-home**: Boolean. Skip home directory creation.
- **primary-group**: Default group for the user. Defaults to a new group created named after the user. - **primary-group**: Default group for the user. Defaults to a new group created named after the user.
- **groups**: Add user to these additional groups - **groups**: Add user to these additional groups
- **no-user-group**: Boolean. Skip default group creation. - **no-user-group**: Boolean. Skip default group creation.
- **ssh-authorized-keys**: List of public SSH keys to authorize for this user - **ssh-authorized-keys**: List of public SSH keys to authorize for this user
- **coreos-ssh-import-github**: Authorize SSH keys from Github user - **coreos-ssh-import-github**: Authorize SSH keys from Github user
- **coreos-ssh-import-github-users**: Authorize SSH keys from a list of Github users
- **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint. - **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint.
- **system**: Create the user as a system user. No home directory will be created. - **system**: Create the user as a system user. No home directory will be created.
- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases. - **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
@@ -222,7 +304,7 @@ The following fields are not yet implemented:
- **selinux-user**: Corresponding SELinux user - **selinux-user**: Corresponding SELinux user
- **ssh-import-id**: Import SSH keys by ID from Launchpad. - **ssh-import-id**: Import SSH keys by ID from Launchpad.
``` ```yaml
#cloud-config #cloud-config
users: users:
@@ -261,7 +343,7 @@ Using a higher number of rounds will help create more secure passwords, but give
Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server. Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server.
``` ```yaml
#cloud-config #cloud-config
users: users:
@@ -274,7 +356,7 @@ users:
We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user). We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token: For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token:
``` ```yaml
#cloud-config #cloud-config
users: users:
@@ -284,7 +366,7 @@ users:
You can also specify any URL whose response matches the JSON format for public keys: You can also specify any URL whose response matches the JSON format for public keys:
``` ```yaml
#cloud-config #cloud-config
users: users:
@@ -294,24 +376,50 @@ users:
### write_files ### write_files
The `write-file` parameter defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys: The `write_files` directive defines a set of files to create on the local filesystem.
Each item in the list may have the following keys:
- **path**: Absolute location on disk where contents should be written - **path**: Absolute location on disk where contents should be written
- **content**: Data to write at the provided `path` - **content**: Data to write at the provided `path`
- **permissions**: String representing file permissions in octal notation (i.e. '0644') - **permissions**: Integer representing file permissions, typically in octal notation (i.e. 0644)
- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`. - **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
- **encoding**: Optional. The encoding of the data in content. If not specified this defaults to the yaml document encoding (usually utf-8). Supported encoding types are:
- **b64, base64**: Base64 encoded content
- **gz, gzip**: gzip encoded content, for use with the !!binary tag
- **gz+b64, gz+base64, gzip+b64, gzip+base64**: Base64 encoded gzip content
Explicitly not implemented is the **encoding** attribute.
The **content** field must represent exactly what should be written to disk.
``` ```yaml
#cloud-config #cloud-config
write_files: write_files:
- path: /etc/fleet/fleet.conf - path: /etc/resolv.conf
permissions: 0644 permissions: 0644
owner: root
content: | content: |
verbosity=1 nameserver 8.8.8.8
metadata="region=us-west,type=ssd" - path: /etc/motd
permissions: 0644
owner: root
content: |
Good news, everyone!
- path: /tmp/like_this
permissions: 0644
owner: root
encoding: gzip
content: !!binary |
H4sIAKgdh1QAAwtITM5WyK1USMqvUCjPLMlQSMssS1VIya9KzVPIySwszS9SyCpNLwYARQFQ5CcAAAA=
- path: /tmp/or_like_this
permissions: 0644
owner: root
encoding: gzip+base64
content: |
H4sIAKgdh1QAAwtITM5WyK1USMqvUCjPLMlQSMssS1VIya9KzVPIySwszS9SyCpNLwYARQFQ5CcAAAA=
- path: /tmp/todolist
permissions: 0644
owner: root
encoding: base64
content: |
UGFjayBteSBib3ggd2l0aCBmaXZlIGRvemVuIGxpcXVvciBqdWdz
``` ```
### manage_etc_hosts ### manage_etc_hosts
@@ -321,7 +429,7 @@ Currently, the only supported value is "localhost" which will cause your system'
to resolve to "127.0.0.1". This is helpful when the host does not have DNS to resolve to "127.0.0.1". This is helpful when the host does not have DNS
infrastructure in place to resolve its own hostname, for example, when using Vagrant. infrastructure in place to resolve its own hostname, for example, when using Vagrant.
``` ```yaml
#cloud-config #cloud-config
manage_etc_hosts: localhost manage_etc_hosts: localhost

View File

@@ -14,17 +14,21 @@ The image should be a single FAT or ISO9660 file system with the label
For example, to wrap up a config named `user_data` in a config drive image: For example, to wrap up a config named `user_data` in a config drive image:
mkdir -p /tmp/new-drive/openstack/latest ```sh
cp user_data /tmp/new-drive/openstack/latest/user_data mkdir -p /tmp/new-drive/openstack/latest
mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive cp user_data /tmp/new-drive/openstack/latest/user_data
rm -r /tmp/new-drive mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
rm -r /tmp/new-drive
```
## QEMU virtfs ## QEMU virtfs
One exception to the above, when using QEMU it is possible to skip creating an One exception to the above, when using QEMU it is possible to skip creating an
image and use a plain directory containing the same contents: image and use a plain directory containing the same contents:
qemu-system-x86_64 \ ```sh
qemu-system-x86_64 \
-fsdev local,id=conf,security_model=none,readonly,path=/tmp/new-drive \ -fsdev local,id=conf,security_model=none,readonly,path=/tmp/new-drive \
-device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \ -device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \
[usual qemu options here...] [usual qemu options here...]
```

View File

@@ -0,0 +1,27 @@
#Debian Interfaces#
**WARNING**: This option is EXPERIMENTAL and may change or be removed at any
point.
There is basic support for converting from a Debian network configuration to
networkd unit files. The -convert-netconf=debian option is used to activate
this feature.
#convert-netconf#
Default: ""
Read the network config provided in cloud-drive and translate it from the
specified format into networkd unit files (requires the -from-configdrive
flag). Currently only supports "debian" which provides support for a small
subset of the [Debian network configuration]
(https://wiki.debian.org/NetworkConfiguration). These options include:
- interface config methods
- static
- address/netmask
- gateway
- hwaddress
- dns-nameservers
- dhcp
- hwaddress
- manual
- loopback
- vlan_raw_device
- bond-slaves

34
Godeps/Godeps.json generated Normal file
View File

@@ -0,0 +1,34 @@
{
"ImportPath": "github.com/coreos/coreos-cloudinit",
"GoVersion": "go1.3.3",
"Packages": [
"./..."
],
"Deps": [
{
"ImportPath": "github.com/cloudsigma/cepgo",
"Rev": "1bfc4895bf5c4d3b599f3f6ee142299488c8739b"
},
{
"ImportPath": "github.com/coreos/go-systemd/dbus",
"Rev": "4fbc5060a317b142e6c7bfbedb65596d5f0ab99b"
},
{
"ImportPath": "github.com/coreos/yaml",
"Rev": "6b16a5714269b2f70720a45406b1babd947a17ef"
},
{
"ImportPath": "github.com/dotcloud/docker/pkg/netlink",
"Comment": "v0.11.1-359-g55d41c3e21e1",
"Rev": "55d41c3e21e1593b944c06196ffb2ac57ab7f653"
},
{
"ImportPath": "github.com/guelfey/go.dbus",
"Rev": "f6a3a2366cc39b8479cadc499d3c735fb10fbdda"
},
{
"ImportPath": "github.com/tarm/goserial",
"Rev": "cdabc8d44e8e84f58f18074ae44337e1f2f375b9"
}
]
}

5
Godeps/Readme generated Normal file
View File

@@ -0,0 +1,5 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.

2
Godeps/_workspace/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,2 @@
/pkg
/bin

View File

@@ -0,0 +1,23 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,43 @@
cepgo
=====
Cepko implements easy-to-use communication with CloudSigma's VMs through a
virtual serial port without bothering with formatting the messages properly nor
parsing the output with the specific and sometimes confusing shell tools for
that purpose.
Having the server definition accessible by the VM can be useful in various
ways. For example it is possible to easily determine from within the VM, which
network interfaces are connected to public and which to private network.
Another use is to pass some data to initial VM setup scripts, like setting the
hostname to the VM name or passing ssh public keys through server meta.
Example usage:
package main
import (
"fmt"
"github.com/cloudsigma/cepgo"
)
func main() {
c := cepgo.NewCepgo()
result, err := c.Meta()
if err != nil {
panic(err)
}
fmt.Printf("%#v", result)
}
Output:
map[string]interface {}{
"optimize_for":"custom",
"ssh_public_key":"ssh-rsa AAA...",
"description":"[...]",
}
For more information take a look at the Server Context section of CloudSigma
API Docs: http://cloudsigma-docs.readthedocs.org/en/latest/server_context.html

View File

@@ -0,0 +1,186 @@
// Cepko implements easy-to-use communication with CloudSigma's VMs through a
// virtual serial port without bothering with formatting the messages properly
// nor parsing the output with the specific and sometimes confusing shell tools
// for that purpose.
//
// Having the server definition accessible by the VM can be useful in various
// ways. For example it is possible to easily determine from within the VM,
// which network interfaces are connected to public and which to private
// network. Another use is to pass some data to initial VM setup scripts, like
// setting the hostname to the VM name or passing ssh public keys through
// server meta.
//
// Example usage:
//
// package main
//
// import (
// "fmt"
//
// "github.com/cloudsigma/cepgo"
// )
//
// func main() {
// c := cepgo.NewCepgo()
// result, err := c.Meta()
// if err != nil {
// panic(err)
// }
// fmt.Printf("%#v", result)
// }
//
// Output:
//
// map[string]string{
// "optimize_for":"custom",
// "ssh_public_key":"ssh-rsa AAA...",
// "description":"[...]",
// }
//
// For more information take a look at the Server Context section API Docs:
// http://cloudsigma-docs.readthedocs.org/en/latest/server_context.html
package cepgo
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"runtime"
"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/tarm/goserial"
)
const (
requestPattern = "<\n%s\n>"
EOT = '\x04' // End Of Transmission
)
var (
SerialPort string = "/dev/ttyS1"
Baud int = 115200
)
// Sets the serial port. If the operating system is windows CloudSigma's server
// context is at COM2 port, otherwise (linux, freebsd, darwin) the port is
// being left to the default /dev/ttyS1.
func init() {
if runtime.GOOS == "windows" {
SerialPort = "COM2"
}
}
// The default fetcher makes the connection to the serial port,
// writes given query and reads until the EOT symbol.
func fetchViaSerialPort(key string) ([]byte, error) {
config := &serial.Config{Name: SerialPort, Baud: Baud}
connection, err := serial.OpenPort(config)
if err != nil {
return nil, err
}
query := fmt.Sprintf(requestPattern, key)
if _, err := connection.Write([]byte(query)); err != nil {
return nil, err
}
reader := bufio.NewReader(connection)
answer, err := reader.ReadBytes(EOT)
if err != nil {
return nil, err
}
return answer[0 : len(answer)-1], nil
}
// Queries to the serial port can be executed only from instance of this type.
// The result from each of them can be either interface{}, map[string]string or
// a single in case of single value is returned. There is also a public metod
// who directly calls the fetcher and returns raw []byte from the serial port.
type Cepgo struct {
fetcher func(string) ([]byte, error)
}
// Creates a Cepgo instance with the default serial port fetcher.
func NewCepgo() *Cepgo {
cepgo := new(Cepgo)
cepgo.fetcher = fetchViaSerialPort
return cepgo
}
// Creates a Cepgo instance with custom fetcher.
func NewCepgoFetcher(fetcher func(string) ([]byte, error)) *Cepgo {
cepgo := new(Cepgo)
cepgo.fetcher = fetcher
return cepgo
}
// Fetches raw []byte from the serial port using directly the fetcher member.
func (c *Cepgo) FetchRaw(key string) ([]byte, error) {
return c.fetcher(key)
}
// Fetches a single key and tries to unmarshal the result to json and returns
// it. If the unmarshalling fails it's safe to assume the result it's just a
// string and returns it.
func (c *Cepgo) Key(key string) (interface{}, error) {
var result interface{}
fetched, err := c.FetchRaw(key)
if err != nil {
return nil, err
}
err = json.Unmarshal(fetched, &result)
if err != nil {
return string(fetched), nil
}
return result, nil
}
// Fetches all the server context. Equivalent of c.Key("")
func (c *Cepgo) All() (interface{}, error) {
return c.Key("")
}
// Fetches only the object meta field and makes sure to return a proper
// map[string]string
func (c *Cepgo) Meta() (map[string]string, error) {
rawMeta, err := c.Key("/meta/")
if err != nil {
return nil, err
}
return typeAssertToMapOfStrings(rawMeta)
}
// Fetches only the global context and makes sure to return a proper
// map[string]string
func (c *Cepgo) GlobalContext() (map[string]string, error) {
rawContext, err := c.Key("/global_context/")
if err != nil {
return nil, err
}
return typeAssertToMapOfStrings(rawContext)
}
// Just a little helper function that uses type assertions in order to convert
// a interface{} to map[string]string if this is possible.
func typeAssertToMapOfStrings(raw interface{}) (map[string]string, error) {
result := make(map[string]string)
dictionary, ok := raw.(map[string]interface{})
if !ok {
return nil, errors.New("Received bytes are formatted badly")
}
for key, rawValue := range dictionary {
if value, ok := rawValue.(string); ok {
result[key] = value
} else {
return nil, errors.New("Server context metadata is formatted badly")
}
}
return result, nil
}

View File

@@ -0,0 +1,122 @@
package cepgo
import (
"encoding/json"
"testing"
)
func fetchMock(key string) ([]byte, error) {
context := []byte(`{
"context": true,
"cpu": 4000,
"cpu_model": null,
"cpus_instead_of_cores": false,
"enable_numa": false,
"global_context": {
"some_global_key": "some_global_val"
},
"grantees": [],
"hv_relaxed": false,
"hv_tsc": false,
"jobs": [],
"mem": 4294967296,
"meta": {
"base64_fields": "cloudinit-user-data",
"cloudinit-user-data": "I2Nsb3VkLWNvbmZpZwoKaG9zdG5hbWU6IGNvcmVvczE=",
"ssh_public_key": "ssh-rsa AAAAB2NzaC1yc2E.../hQ5D5 john@doe"
},
"name": "coreos",
"nics": [
{
"runtime": {
"interface_type": "public",
"ip_v4": {
"uuid": "31.171.251.74"
},
"ip_v6": null
},
"vlan": null
}
],
"smp": 2,
"status": "running",
"uuid": "20a0059b-041e-4d0c-bcc6-9b2852de48b3"
}`)
if key == "" {
return context, nil
}
var marshalledContext map[string]interface{}
err := json.Unmarshal(context, &marshalledContext)
if err != nil {
return nil, err
}
if key[0] == '/' {
key = key[1:]
}
if key[len(key)-1] == '/' {
key = key[:len(key)-1]
}
return json.Marshal(marshalledContext[key])
}
func TestAll(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
result, err := cepgo.All()
if err != nil {
t.Error(err)
}
for _, key := range []string{"meta", "name", "uuid", "global_context"} {
if _, ok := result.(map[string]interface{})[key]; !ok {
t.Errorf("%s not in all keys", key)
}
}
}
func TestKey(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
result, err := cepgo.Key("uuid")
if err != nil {
t.Error(err)
}
if _, ok := result.(string); !ok {
t.Errorf("%#v\n", result)
t.Error("Fetching the uuid did not return a string")
}
}
func TestMeta(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
meta, err := cepgo.Meta()
if err != nil {
t.Errorf("%#v\n", meta)
t.Error(err)
}
if _, ok := meta["ssh_public_key"]; !ok {
t.Error("ssh_public_key is not in the meta")
}
}
func TestGlobalContext(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
result, err := cepgo.GlobalContext()
if err != nil {
t.Error(err)
}
if _, ok := result["some_global_key"]; !ok {
t.Error("some_global_key is not in the global context")
}
}

View File

@@ -23,7 +23,7 @@ import (
"strings" "strings"
"sync" "sync"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
) )
const signalBuffer = 100 const signalBuffer = 100

View File

@@ -18,7 +18,7 @@ package dbus
import ( import (
"errors" "errors"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
) )
func (c *Conn) initJobs() { func (c *Conn) initJobs() {
@@ -208,7 +208,7 @@ func (c *Conn) SetUnitProperties(name string, runtime bool, properties ...Proper
} }
func (c *Conn) GetUnitTypeProperty(unit string, unitType string, propertyName string) (*Property, error) { func (c *Conn) GetUnitTypeProperty(unit string, unitType string, propertyName string) (*Property, error) {
return c.getProperty(unit, "org.freedesktop.systemd1." + unitType, propertyName) return c.getProperty(unit, "org.freedesktop.systemd1."+unitType, propertyName)
} }
// ListUnits returns an array with all currently loaded units. Note that // ListUnits returns an array with all currently loaded units. Note that

View File

@@ -18,7 +18,7 @@ package dbus
import ( import (
"fmt" "fmt"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
"math/rand" "math/rand"
"os" "os"
"path/filepath" "path/filepath"

View File

@@ -17,7 +17,7 @@ limitations under the License.
package dbus package dbus
import ( import (
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
) )
// From the systemd docs: // From the systemd docs:

View File

@@ -20,7 +20,7 @@ import (
"errors" "errors"
"time" "time"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
) )
const ( const (
@@ -101,7 +101,7 @@ func (c *Conn) SubscribeUnits(interval time.Duration) (<-chan map[string]*UnitSt
// SubscribeUnitsCustom is like SubscribeUnits but lets you specify the buffer // SubscribeUnitsCustom is like SubscribeUnits but lets you specify the buffer
// size of the channels, the comparison function for detecting changes and a filter // size of the channels, the comparison function for detecting changes and a filter
// function for cutting down on the noise that your channel receives. // function for cutting down on the noise that your channel receives.
func (c *Conn) SubscribeUnitsCustom(interval time.Duration, buffer int, isChanged func(*UnitStatus, *UnitStatus) bool, filterUnit func (string) bool) (<-chan map[string]*UnitStatus, <-chan error) { func (c *Conn) SubscribeUnitsCustom(interval time.Duration, buffer int, isChanged func(*UnitStatus, *UnitStatus) bool, filterUnit func(string) bool) (<-chan map[string]*UnitStatus, <-chan error) {
old := make(map[string]*UnitStatus) old := make(map[string]*UnitStatus)
statusChan := make(chan map[string]*UnitStatus, buffer) statusChan := make(chan map[string]*UnitStatus, buffer)
errChan := make(chan error, buffer) errChan := make(chan error, buffer)

View File

@@ -1,3 +1,6 @@
Copyright (c) 2011-2014 - Canonical Inc.
This software is licensed under the LGPLv3, included below. This software is licensed under the LGPLv3, included below.
As a special exception to the GNU Lesser General Public License version 3 As a special exception to the GNU Lesser General Public License version 3

View File

@@ -1,3 +1,15 @@
The following files were ported to Go from C files of libyaml, and thus
are still covered by their original copyright and license:
apic.go
emitterc.go
parserc.go
readerc.go
scannerc.go
writerc.go
yamlh.go
yamlprivateh.go
Copyright (c) 2006 Kirill Simonov Copyright (c) 2006 Kirill Simonov
Permission is hereby granted, free of charge, to any person obtaining a copy of Permission is hereby granted, free of charge, to any person obtaining a copy of

131
Godeps/_workspace/src/github.com/coreos/yaml/README.md generated vendored Normal file
View File

@@ -0,0 +1,131 @@
Note: This is a fork of https://github.com/go-yaml/yaml. The following README
doesn't necessarily apply to this fork.
# YAML support for the Go language
Introduction
------------
The yaml package enables Go programs to comfortably encode and decode YAML
values. It was developed within [Canonical](https://www.canonical.com) as
part of the [juju](https://juju.ubuntu.com) project, and is based on a
pure Go port of the well-known [libyaml](http://pyyaml.org/wiki/LibYAML)
C library to parse and generate YAML data quickly and reliably.
Compatibility
-------------
The yaml package supports most of YAML 1.1 and 1.2, including support for
anchors, tags, map merging, etc. Multi-document unmarshalling is not yet
implemented, and base-60 floats from YAML 1.1 are purposefully not
supported since they're a poor design and are gone in YAML 1.2.
Installation and usage
----------------------
The import path for the package is *gopkg.in/yaml.v1*.
To install it, run:
go get gopkg.in/yaml.v1
API documentation
-----------------
If opened in a browser, the import path itself leads to the API documentation:
* [https://gopkg.in/yaml.v1](https://gopkg.in/yaml.v1)
API stability
-------------
The package API for yaml v1 will remain stable as described in [gopkg.in](https://gopkg.in).
License
-------
The yaml package is licensed under the LGPL with an exception that allows it to be linked statically. Please see the LICENSE file for details.
Example
-------
```Go
package main
import (
"fmt"
"log"
"gopkg.in/yaml.v1"
)
var data = `
a: Easy!
b:
c: 2
d: [3, 4]
`
type T struct {
A string
B struct{C int; D []int ",flow"}
}
func main() {
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
m := make(map[interface{}]interface{})
err = yaml.Unmarshal([]byte(data), &m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m:\n%v\n\n", m)
d, err = yaml.Marshal(&m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m dump:\n%s\n\n", string(d))
}
```
This example will generate the following output:
```
--- t:
{Easy! {2 [3 4]}}
--- t dump:
a: Easy!
b:
c: 2
d: [3, 4]
--- m:
map[a:Easy! b:map[c:2 d:[3 4]]]
--- m dump:
a: Easy!
b:
c: 2
d:
- 3
- 4
```

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"io" "io"

View File

@@ -1,8 +1,11 @@
package goyaml package yaml
import ( import (
"encoding/base64"
"fmt"
"reflect" "reflect"
"strconv" "strconv"
"time"
) )
const ( const (
@@ -30,10 +33,12 @@ type parser struct {
parser yaml_parser_t parser yaml_parser_t
event yaml_event_t event yaml_event_t
doc *node doc *node
transform transformString
} }
func newParser(b []byte) *parser { func newParser(b []byte, t transformString) *parser {
p := parser{} p := parser{transform: t}
if !yaml_parser_initialize(&p.parser) { if !yaml_parser_initialize(&p.parser) {
panic("Failed to initialize YAML emitter") panic("Failed to initialize YAML emitter")
} }
@@ -62,7 +67,7 @@ func (p *parser) destroy() {
func (p *parser) skip() { func (p *parser) skip() {
if p.event.typ != yaml_NO_EVENT { if p.event.typ != yaml_NO_EVENT {
if p.event.typ == yaml_STREAM_END_EVENT { if p.event.typ == yaml_STREAM_END_EVENT {
panic("Attempted to go past the end of stream. Corrupted value?") fail("Attempted to go past the end of stream. Corrupted value?")
} }
yaml_event_delete(&p.event) yaml_event_delete(&p.event)
} }
@@ -88,7 +93,7 @@ func (p *parser) fail() {
} else { } else {
msg = "Unknown problem parsing YAML content" msg = "Unknown problem parsing YAML content"
} }
panic(where + msg) fail(where + msg)
} }
func (p *parser) anchor(n *node, anchor []byte) { func (p *parser) anchor(n *node, anchor []byte) {
@@ -113,10 +118,9 @@ func (p *parser) parse() *node {
// Happens when attempting to decode an empty buffer. // Happens when attempting to decode an empty buffer.
return nil return nil
default: default:
panic("Attempted to parse unknown event: " + panic("Attempted to parse unknown event: " + strconv.Itoa(int(p.event.typ)))
strconv.Itoa(int(p.event.typ)))
} }
panic("Unreachable") panic("unreachable")
} }
func (p *parser) node(kind int) *node { func (p *parser) node(kind int) *node {
@@ -134,8 +138,7 @@ func (p *parser) document() *node {
p.skip() p.skip()
n.children = append(n.children, p.parse()) n.children = append(n.children, p.parse())
if p.event.typ != yaml_DOCUMENT_END_EVENT { if p.event.typ != yaml_DOCUMENT_END_EVENT {
panic("Expected end of document event but got " + panic("Expected end of document event but got " + strconv.Itoa(int(p.event.typ)))
strconv.Itoa(int(p.event.typ)))
} }
p.skip() p.skip()
return n return n
@@ -174,7 +177,10 @@ func (p *parser) mapping() *node {
p.anchor(n, p.event.anchor) p.anchor(n, p.event.anchor)
p.skip() p.skip()
for p.event.typ != yaml_MAPPING_END_EVENT { for p.event.typ != yaml_MAPPING_END_EVENT {
n.children = append(n.children, p.parse(), p.parse()) key := p.parse()
key.value = p.transform(key.value)
value := p.parse()
n.children = append(n.children, key, value)
} }
p.skip() p.skip()
return n return n
@@ -211,11 +217,21 @@ func newDecoder() *decoder {
// returned to call SetYAML() with the value of *out once it's defined. // returned to call SetYAML() with the value of *out once it's defined.
// //
func (d *decoder) setter(tag string, out *reflect.Value, good *bool) (set func()) { func (d *decoder) setter(tag string, out *reflect.Value, good *bool) (set func()) {
if (*out).Kind() != reflect.Ptr && (*out).CanAddr() {
setter, _ := (*out).Addr().Interface().(Setter)
if setter != nil {
var arg interface{}
*out = reflect.ValueOf(&arg).Elem()
return func() {
*good = setter.SetYAML(shortTag(tag), arg)
}
}
}
again := true again := true
for again { for again {
again = false again = false
setter, _ := (*out).Interface().(Setter) setter, _ := (*out).Interface().(Setter)
if tag != "!!null" || setter != nil { if tag != yaml_NULL_TAG || setter != nil {
if pv := (*out); pv.Kind() == reflect.Ptr { if pv := (*out); pv.Kind() == reflect.Ptr {
if pv.IsNil() { if pv.IsNil() {
*out = reflect.New(pv.Type().Elem()).Elem() *out = reflect.New(pv.Type().Elem()).Elem()
@@ -231,7 +247,7 @@ func (d *decoder) setter(tag string, out *reflect.Value, good *bool) (set func()
var arg interface{} var arg interface{}
*out = reflect.ValueOf(&arg).Elem() *out = reflect.ValueOf(&arg).Elem()
return func() { return func() {
*good = setter.SetYAML(tag, arg) *good = setter.SetYAML(shortTag(tag), arg)
} }
} }
} }
@@ -268,10 +284,10 @@ func (d *decoder) document(n *node, out reflect.Value) (good bool) {
func (d *decoder) alias(n *node, out reflect.Value) (good bool) { func (d *decoder) alias(n *node, out reflect.Value) (good bool) {
an, ok := d.doc.anchors[n.value] an, ok := d.doc.anchors[n.value]
if !ok { if !ok {
panic("Unknown anchor '" + n.value + "' referenced") fail("Unknown anchor '" + n.value + "' referenced")
} }
if d.aliases[n.value] { if d.aliases[n.value] {
panic("Anchor '" + n.value + "' value contains itself") fail("Anchor '" + n.value + "' value contains itself")
} }
d.aliases[n.value] = true d.aliases[n.value] = true
good = d.unmarshal(an, out) good = d.unmarshal(an, out)
@@ -279,20 +295,50 @@ func (d *decoder) alias(n *node, out reflect.Value) (good bool) {
return good return good
} }
var zeroValue reflect.Value
func resetMap(out reflect.Value) {
for _, k := range out.MapKeys() {
out.SetMapIndex(k, zeroValue)
}
}
var durationType = reflect.TypeOf(time.Duration(0))
func (d *decoder) scalar(n *node, out reflect.Value) (good bool) { func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
var tag string var tag string
var resolved interface{} var resolved interface{}
if n.tag == "" && !n.implicit { if n.tag == "" && !n.implicit {
tag = yaml_STR_TAG
resolved = n.value resolved = n.value
} else { } else {
tag, resolved = resolve(n.tag, n.value) tag, resolved = resolve(n.tag, n.value)
if tag == yaml_BINARY_TAG {
data, err := base64.StdEncoding.DecodeString(resolved.(string))
if err != nil {
fail("!!binary value contains invalid base64 data")
}
resolved = string(data)
}
}
if set := d.setter(tag, &out, &good); set != nil { if set := d.setter(tag, &out, &good); set != nil {
defer set() defer set()
} }
if resolved == nil {
if out.Kind() == reflect.Map && !out.CanAddr() {
resetMap(out)
} else {
out.Set(reflect.Zero(out.Type()))
}
good = true
return
} }
switch out.Kind() { switch out.Kind() {
case reflect.String: case reflect.String:
if resolved != nil { if tag == yaml_BINARY_TAG {
out.SetString(resolved.(string))
good = true
} else if resolved != nil {
out.SetString(n.value) out.SetString(n.value)
good = true good = true
} }
@@ -320,6 +366,14 @@ func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
out.SetInt(int64(resolved)) out.SetInt(int64(resolved))
good = true good = true
} }
case string:
if out.Type() == durationType {
d, err := time.ParseDuration(resolved)
if err == nil {
out.SetInt(int64(d))
good = true
}
}
} }
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr: case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
switch resolved := resolved.(type) { switch resolved := resolved.(type) {
@@ -358,11 +412,6 @@ func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
good = true good = true
} }
case reflect.Ptr: case reflect.Ptr:
switch resolved.(type) {
case nil:
out.Set(reflect.Zero(out.Type()))
good = true
default:
if out.Type().Elem() == reflect.TypeOf(resolved) { if out.Type().Elem() == reflect.TypeOf(resolved) {
elem := reflect.New(out.Type().Elem()) elem := reflect.New(out.Type().Elem())
elem.Elem().Set(reflect.ValueOf(resolved)) elem.Elem().Set(reflect.ValueOf(resolved))
@@ -370,7 +419,6 @@ func (d *decoder) scalar(n *node, out reflect.Value) (good bool) {
good = true good = true
} }
} }
}
return good return good
} }
@@ -382,7 +430,7 @@ func settableValueOf(i interface{}) reflect.Value {
} }
func (d *decoder) sequence(n *node, out reflect.Value) (good bool) { func (d *decoder) sequence(n *node, out reflect.Value) (good bool) {
if set := d.setter("!!seq", &out, &good); set != nil { if set := d.setter(yaml_SEQ_TAG, &out, &good); set != nil {
defer set() defer set()
} }
var iface reflect.Value var iface reflect.Value
@@ -411,7 +459,7 @@ func (d *decoder) sequence(n *node, out reflect.Value) (good bool) {
} }
func (d *decoder) mapping(n *node, out reflect.Value) (good bool) { func (d *decoder) mapping(n *node, out reflect.Value) (good bool) {
if set := d.setter("!!map", &out, &good); set != nil { if set := d.setter(yaml_MAP_TAG, &out, &good); set != nil {
defer set() defer set()
} }
if out.Kind() == reflect.Struct { if out.Kind() == reflect.Struct {
@@ -437,8 +485,19 @@ func (d *decoder) mapping(n *node, out reflect.Value) (good bool) {
} }
l := len(n.children) l := len(n.children)
for i := 0; i < l; i += 2 { for i := 0; i < l; i += 2 {
if isMerge(n.children[i]) {
d.merge(n.children[i+1], out)
continue
}
k := reflect.New(kt).Elem() k := reflect.New(kt).Elem()
if d.unmarshal(n.children[i], k) { if d.unmarshal(n.children[i], k) {
kkind := k.Kind()
if kkind == reflect.Interface {
kkind = k.Elem().Kind()
}
if kkind == reflect.Map || kkind == reflect.Slice {
fail(fmt.Sprintf("invalid map key: %#v", k.Interface()))
}
e := reflect.New(et).Elem() e := reflect.New(et).Elem()
if d.unmarshal(n.children[i+1], e) { if d.unmarshal(n.children[i+1], e) {
out.SetMapIndex(k, e) out.SetMapIndex(k, e)
@@ -456,7 +515,12 @@ func (d *decoder) mappingStruct(n *node, out reflect.Value) (good bool) {
name := settableValueOf("") name := settableValueOf("")
l := len(n.children) l := len(n.children)
for i := 0; i < l; i += 2 { for i := 0; i < l; i += 2 {
if !d.unmarshal(n.children[i], name) { ni := n.children[i]
if isMerge(ni) {
d.merge(n.children[i+1], out)
continue
}
if !d.unmarshal(ni, name) {
continue continue
} }
if info, ok := sinfo.FieldsMap[name.String()]; ok { if info, ok := sinfo.FieldsMap[name.String()]; ok {
@@ -471,3 +535,37 @@ func (d *decoder) mappingStruct(n *node, out reflect.Value) (good bool) {
} }
return true return true
} }
func (d *decoder) merge(n *node, out reflect.Value) {
const wantMap = "map merge requires map or sequence of maps as the value"
switch n.kind {
case mappingNode:
d.unmarshal(n, out)
case aliasNode:
an, ok := d.doc.anchors[n.value]
if ok && an.kind != mappingNode {
fail(wantMap)
}
d.unmarshal(n, out)
case sequenceNode:
// Step backwards as earlier nodes take precedence.
for i := len(n.children) - 1; i >= 0; i-- {
ni := n.children[i]
if ni.kind == aliasNode {
an, ok := d.doc.anchors[ni.value]
if ok && an.kind != mappingNode {
fail(wantMap)
}
} else if ni.kind != mappingNode {
fail(wantMap)
}
d.unmarshal(ni, out)
}
default:
fail(wantMap)
}
}
func isMerge(n *node) bool {
return n.kind == scalarNode && n.value == "<<" && (n.implicit == true || n.tag == yaml_MERGE_TAG)
}

View File

@@ -1,10 +1,12 @@
package goyaml_test package yaml_test
import ( import (
. "launchpad.net/gocheck" "github.com/coreos/yaml"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml" . "gopkg.in/check.v1"
"math" "math"
"reflect" "reflect"
"strings"
"time"
) )
var unmarshalIntTest = 123 var unmarshalIntTest = 123
@@ -315,7 +317,10 @@ var unmarshalTests = []struct {
map[string]*string{"foo": new(string)}, map[string]*string{"foo": new(string)},
}, { }, {
"foo: null", "foo: null",
map[string]string{}, map[string]string{"foo": ""},
}, {
"foo: null",
map[string]interface{}{"foo": nil},
}, },
// Ignored field // Ignored field
@@ -350,6 +355,50 @@ var unmarshalTests = []struct {
C inlineB `yaml:",inline"` C inlineB `yaml:",inline"`
}{1, inlineB{2, inlineC{3}}}, }{1, inlineB{2, inlineC{3}}},
}, },
// bug 1243827
{
"a: -b_c",
map[string]interface{}{"a": "-b_c"},
},
{
"a: +b_c",
map[string]interface{}{"a": "+b_c"},
},
{
"a: 50cent_of_dollar",
map[string]interface{}{"a": "50cent_of_dollar"},
},
// Duration
{
"a: 3s",
map[string]time.Duration{"a": 3 * time.Second},
},
// Issue #24.
{
"a: <foo>",
map[string]string{"a": "<foo>"},
},
// Base 60 floats are obsolete and unsupported.
{
"a: 1:1\n",
map[string]string{"a": "1:1"},
},
// Binary data.
{
"a: !!binary gIGC\n",
map[string]string{"a": "\x80\x81\x82"},
}, {
"a: !!binary |\n " + strings.Repeat("kJCQ", 17) + "kJ\n CQ\n",
map[string]string{"a": strings.Repeat("\x90", 54)},
}, {
"a: !!binary |\n " + strings.Repeat("A", 70) + "\n ==\n",
map[string]string{"a": strings.Repeat("\x00", 52)},
},
} }
type inlineB struct { type inlineB struct {
@@ -377,7 +426,7 @@ func (s *S) TestUnmarshal(c *C) {
pv := reflect.New(pt.Elem()) pv := reflect.New(pt.Elem())
value = pv.Interface() value = pv.Interface()
} }
err := goyaml.Unmarshal([]byte(item.data), value) err := yaml.Unmarshal([]byte(item.data), value)
c.Assert(err, IsNil, Commentf("Item #%d", i)) c.Assert(err, IsNil, Commentf("Item #%d", i))
if t.Kind() == reflect.String { if t.Kind() == reflect.String {
c.Assert(*value.(*string), Equals, item.value, Commentf("Item #%d", i)) c.Assert(*value.(*string), Equals, item.value, Commentf("Item #%d", i))
@@ -389,7 +438,7 @@ func (s *S) TestUnmarshal(c *C) {
func (s *S) TestUnmarshalNaN(c *C) { func (s *S) TestUnmarshalNaN(c *C) {
value := map[string]interface{}{} value := map[string]interface{}{}
err := goyaml.Unmarshal([]byte("notanum: .NaN"), &value) err := yaml.Unmarshal([]byte("notanum: .NaN"), &value)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(math.IsNaN(value["notanum"].(float64)), Equals, true) c.Assert(math.IsNaN(value["notanum"].(float64)), Equals, true)
} }
@@ -397,18 +446,21 @@ func (s *S) TestUnmarshalNaN(c *C) {
var unmarshalErrorTests = []struct { var unmarshalErrorTests = []struct {
data, error string data, error string
}{ }{
{"v: !!float 'error'", "YAML error: Can't decode !!str 'error' as a !!float"}, {"v: !!float 'error'", "YAML error: cannot decode !!str `error` as a !!float"},
{"v: [A,", "YAML error: line 1: did not find expected node content"}, {"v: [A,", "YAML error: line 1: did not find expected node content"},
{"v:\n- [A,", "YAML error: line 2: did not find expected node content"}, {"v:\n- [A,", "YAML error: line 2: did not find expected node content"},
{"a: *b\n", "YAML error: Unknown anchor 'b' referenced"}, {"a: *b\n", "YAML error: Unknown anchor 'b' referenced"},
{"a: &a\n b: *a\n", "YAML error: Anchor 'a' value contains itself"}, {"a: &a\n b: *a\n", "YAML error: Anchor 'a' value contains itself"},
{"value: -", "YAML error: block sequence entries are not allowed in this context"}, {"value: -", "YAML error: block sequence entries are not allowed in this context"},
{"a: !!binary ==", "YAML error: !!binary value contains invalid base64 data"},
{"{[.]}", `YAML error: invalid map key: \[\]interface \{\}\{"\."\}`},
{"{{.}}", `YAML error: invalid map key: map\[interface\ \{\}\]interface \{\}\{".":interface \{\}\(nil\)\}`},
} }
func (s *S) TestUnmarshalErrors(c *C) { func (s *S) TestUnmarshalErrors(c *C) {
for _, item := range unmarshalErrorTests { for _, item := range unmarshalErrorTests {
var value interface{} var value interface{}
err := goyaml.Unmarshal([]byte(item.data), &value) err := yaml.Unmarshal([]byte(item.data), &value)
c.Assert(err, ErrorMatches, item.error, Commentf("Partial unmarshal: %#v", value)) c.Assert(err, ErrorMatches, item.error, Commentf("Partial unmarshal: %#v", value))
} }
} }
@@ -421,6 +473,8 @@ var setterTests = []struct {
{"_: [1,A]", "!!seq", []interface{}{1, "A"}}, {"_: [1,A]", "!!seq", []interface{}{1, "A"}},
{"_: 10", "!!int", 10}, {"_: 10", "!!int", 10},
{"_: null", "!!null", nil}, {"_: null", "!!null", nil},
{`_: BAR!`, "!!str", "BAR!"},
{`_: "BAR!"`, "!!str", "BAR!"},
{"_: !!foo 'BAR!'", "!!foo", "BAR!"}, {"_: !!foo 'BAR!'", "!!foo", "BAR!"},
} }
@@ -442,17 +496,31 @@ func (o *typeWithSetter) SetYAML(tag string, value interface{}) (ok bool) {
return true return true
} }
type typeWithSetterField struct { type setterPointerType struct {
Field *typeWithSetter "_" Field *typeWithSetter "_"
} }
func (s *S) TestUnmarshalWithSetter(c *C) { type setterValueType struct {
Field typeWithSetter "_"
}
func (s *S) TestUnmarshalWithPointerSetter(c *C) {
for _, item := range setterTests { for _, item := range setterTests {
obj := &typeWithSetterField{} obj := &setterPointerType{}
err := goyaml.Unmarshal([]byte(item.data), obj) err := yaml.Unmarshal([]byte(item.data), obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(obj.Field, NotNil, c.Assert(obj.Field, NotNil, Commentf("Pointer not initialized (%#v)", item.value))
Commentf("Pointer not initialized (%#v)", item.value)) c.Assert(obj.Field.tag, Equals, item.tag)
c.Assert(obj.Field.value, DeepEquals, item.value)
}
}
func (s *S) TestUnmarshalWithValueSetter(c *C) {
for _, item := range setterTests {
obj := &setterValueType{}
err := yaml.Unmarshal([]byte(item.data), obj)
c.Assert(err, IsNil)
c.Assert(obj.Field, NotNil, Commentf("Pointer not initialized (%#v)", item.value))
c.Assert(obj.Field.tag, Equals, item.tag) c.Assert(obj.Field.tag, Equals, item.tag)
c.Assert(obj.Field.value, DeepEquals, item.value) c.Assert(obj.Field.value, DeepEquals, item.value)
} }
@@ -460,7 +528,7 @@ func (s *S) TestUnmarshalWithSetter(c *C) {
func (s *S) TestUnmarshalWholeDocumentWithSetter(c *C) { func (s *S) TestUnmarshalWholeDocumentWithSetter(c *C) {
obj := &typeWithSetter{} obj := &typeWithSetter{}
err := goyaml.Unmarshal([]byte(setterTests[0].data), obj) err := yaml.Unmarshal([]byte(setterTests[0].data), obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(obj.tag, Equals, setterTests[0].tag) c.Assert(obj.tag, Equals, setterTests[0].tag)
value, ok := obj.value.(map[interface{}]interface{}) value, ok := obj.value.(map[interface{}]interface{})
@@ -477,8 +545,8 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
}() }()
m := map[string]*typeWithSetter{} m := map[string]*typeWithSetter{}
data := "{abc: 1, def: 2, ghi: 3, jkl: 4}" data := `{abc: 1, def: 2, ghi: 3, jkl: 4}`
err := goyaml.Unmarshal([]byte(data), m) err := yaml.Unmarshal([]byte(data), m)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(m["abc"], NotNil) c.Assert(m["abc"], NotNil)
c.Assert(m["def"], IsNil) c.Assert(m["def"], IsNil)
@@ -489,6 +557,139 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
c.Assert(m["ghi"].value, Equals, 3) c.Assert(m["ghi"].value, Equals, 3)
} }
func (s *S) TestUnmarshalWithTransform(c *C) {
data := `{a_b: 1, c-d: 2, e-f_g: 3, h_i-j: 4}`
expect := map[string]int{
"a_b": 1,
"c_d": 2,
"e_f_g": 3,
"h_i_j": 4,
}
m := map[string]int{}
yaml.UnmarshalMappingKeyTransform = func(i string) string {
return strings.Replace(i, "-", "_", -1)
}
err := yaml.Unmarshal([]byte(data), m)
c.Assert(err, IsNil)
c.Assert(m, DeepEquals, expect)
}
// From http://yaml.org/type/merge.html
var mergeTests = `
anchors:
- &CENTER { "x": 1, "y": 2 }
- &LEFT { "x": 0, "y": 2 }
- &BIG { "r": 10 }
- &SMALL { "r": 1 }
# All the following maps are equal:
plain:
# Explicit keys
"x": 1
"y": 2
"r": 10
label: center/big
mergeOne:
# Merge one map
<< : *CENTER
"r": 10
label: center/big
mergeMultiple:
# Merge multiple maps
<< : [ *CENTER, *BIG ]
label: center/big
override:
# Override
<< : [ *BIG, *LEFT, *SMALL ]
"x": 1
label: center/big
shortTag:
# Explicit short merge tag
!!merge "<<" : [ *CENTER, *BIG ]
label: center/big
longTag:
# Explicit merge long tag
!<tag:yaml.org,2002:merge> "<<" : [ *CENTER, *BIG ]
label: center/big
inlineMap:
# Inlined map
<< : {"x": 1, "y": 2, "r": 10}
label: center/big
inlineSequenceMap:
# Inlined map in sequence
<< : [ *CENTER, {"r": 10} ]
label: center/big
`
func (s *S) TestMerge(c *C) {
var want = map[interface{}]interface{}{
"x": 1,
"y": 2,
"r": 10,
"label": "center/big",
}
var m map[string]interface{}
err := yaml.Unmarshal([]byte(mergeTests), &m)
c.Assert(err, IsNil)
for name, test := range m {
if name == "anchors" {
continue
}
c.Assert(test, DeepEquals, want, Commentf("test %q failed", name))
}
}
func (s *S) TestMergeStruct(c *C) {
type Data struct {
X, Y, R int
Label string
}
want := Data{1, 2, 10, "center/big"}
var m map[string]Data
err := yaml.Unmarshal([]byte(mergeTests), &m)
c.Assert(err, IsNil)
for name, test := range m {
if name == "anchors" {
continue
}
c.Assert(test, Equals, want, Commentf("test %q failed", name))
}
}
var unmarshalNullTests = []func() interface{}{
func() interface{} { var v interface{}; v = "v"; return &v },
func() interface{} { var s = "s"; return &s },
func() interface{} { var s = "s"; sptr := &s; return &sptr },
func() interface{} { var i = 1; return &i },
func() interface{} { var i = 1; iptr := &i; return &iptr },
func() interface{} { m := map[string]int{"s": 1}; return &m },
func() interface{} { m := map[string]int{"s": 1}; return m },
}
func (s *S) TestUnmarshalNull(c *C) {
for _, test := range unmarshalNullTests {
item := test()
zero := reflect.Zero(reflect.TypeOf(item).Elem()).Interface()
err := yaml.Unmarshal([]byte("null"), item)
c.Assert(err, IsNil)
if reflect.TypeOf(item).Kind() == reflect.Map {
c.Assert(reflect.ValueOf(item).Interface(), DeepEquals, reflect.MakeMap(reflect.TypeOf(item)).Interface())
} else {
c.Assert(reflect.ValueOf(item).Elem().Interface(), DeepEquals, zero)
}
}
}
//var data []byte //var data []byte
//func init() { //func init() {
// var err error // var err error
@@ -502,7 +703,7 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
// var err error // var err error
// for i := 0; i < c.N; i++ { // for i := 0; i < c.N; i++ {
// var v map[string]interface{} // var v map[string]interface{}
// err = goyaml.Unmarshal(data, &v) // err = yaml.Unmarshal(data, &v)
// } // }
// if err != nil { // if err != nil {
// panic(err) // panic(err)
@@ -511,9 +712,9 @@ func (s *S) TestUnmarshalWithFalseSetterIgnoresValue(c *C) {
// //
//func (s *S) BenchmarkMarshal(c *C) { //func (s *S) BenchmarkMarshal(c *C) {
// var v map[string]interface{} // var v map[string]interface{}
// goyaml.Unmarshal(data, &v) // yaml.Unmarshal(data, &v)
// c.ResetTimer() // c.ResetTimer()
// for i := 0; i < c.N; i++ { // for i := 0; i < c.N; i++ {
// goyaml.Marshal(&v) // yaml.Marshal(&v)
// } // }
//} //}

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"bytes" "bytes"
@@ -973,9 +973,9 @@ func yaml_emitter_analyze_tag(emitter *yaml_emitter_t, tag []byte) bool {
if bytes.HasPrefix(tag, tag_directive.prefix) { if bytes.HasPrefix(tag, tag_directive.prefix) {
emitter.tag_data.handle = tag_directive.handle emitter.tag_data.handle = tag_directive.handle
emitter.tag_data.suffix = tag[len(tag_directive.prefix):] emitter.tag_data.suffix = tag[len(tag_directive.prefix):]
}
return true return true
} }
}
emitter.tag_data.suffix = tag emitter.tag_data.suffix = tag
return true return true
} }
@@ -1279,6 +1279,9 @@ func yaml_emitter_write_tag_content(emitter *yaml_emitter_t, value []byte, need_
for k := 0; k < w; k++ { for k := 0; k < w; k++ {
octet := value[i] octet := value[i]
i++ i++
if !put(emitter, '%') {
return false
}
c := octet >> 4 c := octet >> 4
if c < 10 { if c < 10 {

View File

@@ -1,9 +1,12 @@
package goyaml package yaml
import ( import (
"reflect" "reflect"
"regexp"
"sort" "sort"
"strconv" "strconv"
"strings"
"time"
) )
type encoder struct { type encoder struct {
@@ -49,14 +52,19 @@ func (e *encoder) must(ok bool) {
if msg == "" { if msg == "" {
msg = "Unknown problem generating YAML content" msg = "Unknown problem generating YAML content"
} }
panic(msg) fail(msg)
} }
} }
func (e *encoder) marshal(tag string, in reflect.Value) { func (e *encoder) marshal(tag string, in reflect.Value) {
if !in.IsValid() {
e.nilv()
return
}
var value interface{} var value interface{}
if getter, ok := in.Interface().(Getter); ok { if getter, ok := in.Interface().(Getter); ok {
tag, value = getter.GetYAML() tag, value = getter.GetYAML()
tag = longTag(tag)
if value == nil { if value == nil {
e.nilv() e.nilv()
return return
@@ -85,7 +93,11 @@ func (e *encoder) marshal(tag string, in reflect.Value) {
case reflect.String: case reflect.String:
e.stringv(tag, in) e.stringv(tag, in)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
if in.Type() == durationType {
e.stringv(tag, reflect.ValueOf(in.Interface().(time.Duration).String()))
} else {
e.intv(tag, in) e.intv(tag, in)
}
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr: case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
e.uintv(tag, in) e.uintv(tag, in)
case reflect.Float32, reflect.Float64: case reflect.Float32, reflect.Float64:
@@ -93,7 +105,7 @@ func (e *encoder) marshal(tag string, in reflect.Value) {
case reflect.Bool: case reflect.Bool:
e.boolv(tag, in) e.boolv(tag, in)
default: default:
panic("Can't marshal type yet: " + in.Type().String()) panic("Can't marshal type: " + in.Type().String())
} }
} }
@@ -162,11 +174,46 @@ func (e *encoder) slicev(tag string, in reflect.Value) {
e.emit() e.emit()
} }
// isBase60 returns whether s is in base 60 notation as defined in YAML 1.1.
//
// The base 60 float notation in YAML 1.1 is a terrible idea and is unsupported
// in YAML 1.2 and by this package, but these should be marshalled quoted for
// the time being for compatibility with other parsers.
func isBase60Float(s string) (result bool) {
// Fast path.
if s == "" {
return false
}
c := s[0]
if !(c == '+' || c == '-' || c >= '0' && c <= '9') || strings.IndexByte(s, ':') < 0 {
return false
}
// Do the full match.
return base60float.MatchString(s)
}
// From http://yaml.org/type/float.html, except the regular expression there
// is bogus. In practice parsers do not enforce the "\.[0-9_]*" suffix.
var base60float = regexp.MustCompile(`^[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+(?:\.[0-9_]*)?$`)
func (e *encoder) stringv(tag string, in reflect.Value) { func (e *encoder) stringv(tag string, in reflect.Value) {
var style yaml_scalar_style_t var style yaml_scalar_style_t
s := in.String() s := in.String()
if rtag, _ := resolve("", s); rtag != "!!str" { rtag, rs := resolve("", s)
if rtag == yaml_BINARY_TAG {
if tag == "" || tag == yaml_STR_TAG {
tag = rtag
s = rs.(string)
} else if tag == yaml_BINARY_TAG {
fail("explicitly tagged !!binary data must be base64-encoded")
} else {
fail("cannot marshal invalid UTF-8 data as " + shortTag(tag))
}
}
if tag == "" && (rtag != yaml_STR_TAG || isBase60Float(s)) {
style = yaml_DOUBLE_QUOTED_SCALAR_STYLE style = yaml_DOUBLE_QUOTED_SCALAR_STYLE
} else if strings.Contains(s, "\n") {
style = yaml_LITERAL_SCALAR_STYLE
} else { } else {
style = yaml_PLAIN_SCALAR_STYLE style = yaml_PLAIN_SCALAR_STYLE
} }
@@ -213,9 +260,6 @@ func (e *encoder) nilv() {
func (e *encoder) emitScalar(value, anchor, tag string, style yaml_scalar_style_t) { func (e *encoder) emitScalar(value, anchor, tag string, style yaml_scalar_style_t) {
implicit := tag == "" implicit := tag == ""
if !implicit {
style = yaml_PLAIN_SCALAR_STYLE
}
e.must(yaml_scalar_event_initialize(&e.event, []byte(anchor), []byte(tag), []byte(value), implicit, implicit, style)) e.must(yaml_scalar_event_initialize(&e.event, []byte(anchor), []byte(tag), []byte(value), implicit, implicit, style))
e.emit() e.emit()
} }

View File

@@ -1,12 +1,14 @@
package goyaml_test package yaml_test
import ( import (
"fmt" "fmt"
. "launchpad.net/gocheck"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
"math" "math"
"strconv" "strconv"
"strings" "strings"
"time"
"github.com/coreos/yaml"
. "gopkg.in/check.v1"
) )
var marshalIntTest = 123 var marshalIntTest = 123
@@ -16,6 +18,9 @@ var marshalTests = []struct {
data string data string
}{ }{
{ {
nil,
"null\n",
}, {
&struct{}{}, &struct{}{},
"{}\n", "{}\n",
}, { }, {
@@ -86,7 +91,7 @@ var marshalTests = []struct {
"v:\n- A\n- B\n", "v:\n- A\n- B\n",
}, { }, {
map[string][]string{"v": []string{"A", "B\nC"}}, map[string][]string{"v": []string{"A", "B\nC"}},
"v:\n- A\n- 'B\n\n C'\n", "v:\n- A\n- |-\n B\n C\n",
}, { }, {
map[string][]interface{}{"v": []interface{}{"A", 1, map[string][]int{"B": []int{2, 3}}}}, map[string][]interface{}{"v": []interface{}{"A", 1, map[string][]int{"B": []int{2, 3}}}},
"v:\n- A\n- 1\n- B:\n - 2\n - 3\n", "v:\n- A\n- 1\n- B:\n - 2\n - 3\n",
@@ -212,11 +217,51 @@ var marshalTests = []struct {
}{1, inlineB{2, inlineC{3}}}, }{1, inlineB{2, inlineC{3}}},
"a: 1\nb: 2\nc: 3\n", "a: 1\nb: 2\nc: 3\n",
}, },
// Duration
{
map[string]time.Duration{"a": 3 * time.Second},
"a: 3s\n",
},
// Issue #24: bug in map merging logic.
{
map[string]string{"a": "<foo>"},
"a: <foo>\n",
},
// Issue #34: marshal unsupported base 60 floats quoted for compatibility
// with old YAML 1.1 parsers.
{
map[string]string{"a": "1:1"},
"a: \"1:1\"\n",
},
// Binary data.
{
map[string]string{"a": "\x00"},
"a: \"\\0\"\n",
}, {
map[string]string{"a": "\x80\x81\x82"},
"a: !!binary gIGC\n",
}, {
map[string]string{"a": strings.Repeat("\x90", 54)},
"a: !!binary |\n " + strings.Repeat("kJCQ", 17) + "kJ\n CQ\n",
}, {
map[string]interface{}{"a": typeWithGetter{"!!str", "\x80\x81\x82"}},
"a: !!binary gIGC\n",
},
// Escaping of tags.
{
map[string]interface{}{"a": typeWithGetter{"foo!bar", 1}},
"a: !<foo%21bar> 1\n",
},
} }
func (s *S) TestMarshal(c *C) { func (s *S) TestMarshal(c *C) {
for _, item := range marshalTests { for _, item := range marshalTests {
data, err := goyaml.Marshal(item.value) data, err := yaml.Marshal(item.value)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(string(data), Equals, item.data) c.Assert(string(data), Equals, item.data)
} }
@@ -225,21 +270,30 @@ func (s *S) TestMarshal(c *C) {
var marshalErrorTests = []struct { var marshalErrorTests = []struct {
value interface{} value interface{}
error string error string
}{ panic string
{ }{{
&struct { value: &struct {
B int B int
inlineB ",inline" inlineB ",inline"
}{1, inlineB{2, inlineC{3}}}, }{1, inlineB{2, inlineC{3}}},
`Duplicated key 'b' in struct struct \{ B int; .*`, panic: `Duplicated key 'b' in struct struct \{ B int; .*`,
}, }, {
} value: typeWithGetter{"!!binary", "\x80"},
error: "YAML error: explicitly tagged !!binary data must be base64-encoded",
}, {
value: typeWithGetter{"!!float", "\x80"},
error: `YAML error: cannot marshal invalid UTF-8 data as !!float`,
}}
func (s *S) TestMarshalErrors(c *C) { func (s *S) TestMarshalErrors(c *C) {
for _, item := range marshalErrorTests { for _, item := range marshalErrorTests {
_, err := goyaml.Marshal(item.value) if item.panic != "" {
c.Assert(func() { yaml.Marshal(item.value) }, PanicMatches, item.panic)
} else {
_, err := yaml.Marshal(item.value)
c.Assert(err, ErrorMatches, item.error) c.Assert(err, ErrorMatches, item.error)
} }
}
} }
var marshalTaggedIfaceTest interface{} = &struct{ A string }{"B"} var marshalTaggedIfaceTest interface{} = &struct{ A string }{"B"}
@@ -269,12 +323,12 @@ func (s *S) TestMarshalTypeCache(c *C) {
var err error var err error
func() { func() {
type T struct{ A int } type T struct{ A int }
data, err = goyaml.Marshal(&T{}) data, err = yaml.Marshal(&T{})
c.Assert(err, IsNil) c.Assert(err, IsNil)
}() }()
func() { func() {
type T struct{ B int } type T struct{ B int }
data, err = goyaml.Marshal(&T{}) data, err = yaml.Marshal(&T{})
c.Assert(err, IsNil) c.Assert(err, IsNil)
}() }()
c.Assert(string(data), Equals, "b: 0\n") c.Assert(string(data), Equals, "b: 0\n")
@@ -298,7 +352,7 @@ func (s *S) TestMashalWithGetter(c *C) {
obj := &typeWithGetterField{} obj := &typeWithGetterField{}
obj.Field.tag = item.tag obj.Field.tag = item.tag
obj.Field.value = item.value obj.Field.value = item.value
data, err := goyaml.Marshal(obj) data, err := yaml.Marshal(obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(string(data), Equals, string(item.data)) c.Assert(string(data), Equals, string(item.data))
} }
@@ -308,7 +362,7 @@ func (s *S) TestUnmarshalWholeDocumentWithGetter(c *C) {
obj := &typeWithGetter{} obj := &typeWithGetter{}
obj.tag = "" obj.tag = ""
obj.value = map[string]string{"hello": "world!"} obj.value = map[string]string{"hello": "world!"}
data, err := goyaml.Marshal(obj) data, err := yaml.Marshal(obj)
c.Assert(err, IsNil) c.Assert(err, IsNil)
c.Assert(string(data), Equals, "hello: world!\n") c.Assert(string(data), Equals, "hello: world!\n")
} }
@@ -356,7 +410,7 @@ func (s *S) TestSortedOutput(c *C) {
for _, k := range order { for _, k := range order {
m[k] = 1 m[k] = 1
} }
data, err := goyaml.Marshal(m) data, err := yaml.Marshal(m)
c.Assert(err, IsNil) c.Assert(err, IsNil)
out := "\n" + string(data) out := "\n" + string(data)
last := 0 last := 0

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"bytes" "bytes"

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"io" "io"

190
Godeps/_workspace/src/github.com/coreos/yaml/resolve.go generated vendored Normal file
View File

@@ -0,0 +1,190 @@
package yaml
import (
"encoding/base64"
"fmt"
"math"
"strconv"
"strings"
"unicode/utf8"
)
// TODO: merge, timestamps, base 60 floats, omap.
type resolveMapItem struct {
value interface{}
tag string
}
var resolveTable = make([]byte, 256)
var resolveMap = make(map[string]resolveMapItem)
func init() {
t := resolveTable
t[int('+')] = 'S' // Sign
t[int('-')] = 'S'
for _, c := range "0123456789" {
t[int(c)] = 'D' // Digit
}
for _, c := range "yYnNtTfFoO~" {
t[int(c)] = 'M' // In map
}
t[int('.')] = '.' // Float (potentially in map)
var resolveMapList = []struct {
v interface{}
tag string
l []string
}{
{true, yaml_BOOL_TAG, []string{"y", "Y", "yes", "Yes", "YES"}},
{true, yaml_BOOL_TAG, []string{"true", "True", "TRUE"}},
{true, yaml_BOOL_TAG, []string{"on", "On", "ON"}},
{false, yaml_BOOL_TAG, []string{"n", "N", "no", "No", "NO"}},
{false, yaml_BOOL_TAG, []string{"false", "False", "FALSE"}},
{false, yaml_BOOL_TAG, []string{"off", "Off", "OFF"}},
{nil, yaml_NULL_TAG, []string{"", "~", "null", "Null", "NULL"}},
{math.NaN(), yaml_FLOAT_TAG, []string{".nan", ".NaN", ".NAN"}},
{math.Inf(+1), yaml_FLOAT_TAG, []string{".inf", ".Inf", ".INF"}},
{math.Inf(+1), yaml_FLOAT_TAG, []string{"+.inf", "+.Inf", "+.INF"}},
{math.Inf(-1), yaml_FLOAT_TAG, []string{"-.inf", "-.Inf", "-.INF"}},
{"<<", yaml_MERGE_TAG, []string{"<<"}},
}
m := resolveMap
for _, item := range resolveMapList {
for _, s := range item.l {
m[s] = resolveMapItem{item.v, item.tag}
}
}
}
const longTagPrefix = "tag:yaml.org,2002:"
func shortTag(tag string) string {
// TODO This can easily be made faster and produce less garbage.
if strings.HasPrefix(tag, longTagPrefix) {
return "!!" + tag[len(longTagPrefix):]
}
return tag
}
func longTag(tag string) string {
if strings.HasPrefix(tag, "!!") {
return longTagPrefix + tag[2:]
}
return tag
}
func resolvableTag(tag string) bool {
switch tag {
case "", yaml_STR_TAG, yaml_BOOL_TAG, yaml_INT_TAG, yaml_FLOAT_TAG, yaml_NULL_TAG:
return true
}
return false
}
func resolve(tag string, in string) (rtag string, out interface{}) {
if !resolvableTag(tag) {
return tag, in
}
defer func() {
switch tag {
case "", rtag, yaml_STR_TAG, yaml_BINARY_TAG:
return
}
fail(fmt.Sprintf("cannot decode %s `%s` as a %s", shortTag(rtag), in, shortTag(tag)))
}()
// Any data is accepted as a !!str or !!binary.
// Otherwise, the prefix is enough of a hint about what it might be.
hint := byte('N')
if in != "" {
hint = resolveTable[in[0]]
}
if hint != 0 && tag != yaml_STR_TAG && tag != yaml_BINARY_TAG {
// Handle things we can lookup in a map.
if item, ok := resolveMap[in]; ok {
return item.tag, item.value
}
// Base 60 floats are a bad idea, were dropped in YAML 1.2, and
// are purposefully unsupported here. They're still quoted on
// the way out for compatibility with other parser, though.
switch hint {
case 'M':
// We've already checked the map above.
case '.':
// Not in the map, so maybe a normal float.
floatv, err := strconv.ParseFloat(in, 64)
if err == nil {
return yaml_FLOAT_TAG, floatv
}
case 'D', 'S':
// Int, float, or timestamp.
plain := strings.Replace(in, "_", "", -1)
intv, err := strconv.ParseInt(plain, 0, 64)
if err == nil {
if intv == int64(int(intv)) {
return yaml_INT_TAG, int(intv)
} else {
return yaml_INT_TAG, intv
}
}
floatv, err := strconv.ParseFloat(plain, 64)
if err == nil {
return yaml_FLOAT_TAG, floatv
}
if strings.HasPrefix(plain, "0b") {
intv, err := strconv.ParseInt(plain[2:], 2, 64)
if err == nil {
return yaml_INT_TAG, int(intv)
}
} else if strings.HasPrefix(plain, "-0b") {
intv, err := strconv.ParseInt(plain[3:], 2, 64)
if err == nil {
return yaml_INT_TAG, -int(intv)
}
}
// XXX Handle timestamps here.
default:
panic("resolveTable item not yet handled: " + string(rune(hint)) + " (with " + in + ")")
}
}
if tag == yaml_BINARY_TAG {
return yaml_BINARY_TAG, in
}
if utf8.ValidString(in) {
return yaml_STR_TAG, in
}
return yaml_BINARY_TAG, encodeBase64(in)
}
// encodeBase64 encodes s as base64 that is broken up into multiple lines
// as appropriate for the resulting length.
func encodeBase64(s string) string {
const lineLen = 70
encLen := base64.StdEncoding.EncodedLen(len(s))
lines := encLen/lineLen + 1
buf := make([]byte, encLen*2+lines)
in := buf[0:encLen]
out := buf[encLen:]
base64.StdEncoding.Encode(in, []byte(s))
k := 0
for i := 0; i < len(in); i += lineLen {
j := i + lineLen
if j > len(in) {
j = len(in)
}
k += copy(out[k:], in[i:j])
if lines > 1 {
out[k] = '\n'
k++
}
}
return string(out[:k])
}

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"bytes" "bytes"

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"reflect" "reflect"

View File

@@ -1,7 +1,7 @@
package goyaml_test package yaml_test
import ( import (
. "launchpad.net/gocheck" . "gopkg.in/check.v1"
"testing" "testing"
) )

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
// Set the writer error and return false. // Set the writer error and return false.
func yaml_emitter_set_writer_error(emitter *yaml_emitter_t, problem string) bool { func yaml_emitter_set_writer_error(emitter *yaml_emitter_t, problem string) bool {

View File

@@ -1,59 +1,60 @@
// Package goyaml implements YAML support for the Go language. // Package yaml implements YAML support for the Go language.
package goyaml //
// Source code and other details for the project are available at GitHub:
//
// https://github.com/go-yaml/yaml
//
package yaml
import ( import (
"errors" "errors"
"fmt" "fmt"
"reflect" "reflect"
"runtime"
"strings" "strings"
"sync" "sync"
) )
type yamlError string
func fail(msg string) {
panic(yamlError(msg))
}
func handleErr(err *error) { func handleErr(err *error) {
if r := recover(); r != nil { if r := recover(); r != nil {
if _, ok := r.(runtime.Error); ok { if e, ok := r.(yamlError); ok {
panic(r) *err = errors.New("YAML error: " + string(e))
} else if _, ok := r.(*reflect.ValueError); ok {
panic(r)
} else if _, ok := r.(externalPanic); ok {
panic(r)
} else if s, ok := r.(string); ok {
*err = errors.New("YAML error: " + s)
} else if e, ok := r.(error); ok {
*err = e
} else { } else {
panic(r) panic(r)
} }
} }
} }
// Objects implementing the goyaml.Setter interface will receive the YAML // The Setter interface may be implemented by types to do their own custom
// tag and value via the SetYAML method during unmarshaling, rather than // unmarshalling of YAML values, rather than being implicitly assigned by
// being implicitly assigned by the goyaml machinery. If setting the value // the yaml package machinery. If setting the value works, the method should
// works, the method should return true. If it returns false, the given // return true. If it returns false, the value is considered unsupported
// value will be omitted from maps and slices. // and is omitted from maps and slices.
type Setter interface { type Setter interface {
SetYAML(tag string, value interface{}) bool SetYAML(tag string, value interface{}) bool
} }
// Objects implementing the goyaml.Getter interface will get the GetYAML() // The Getter interface is implemented by types to do their own custom
// method called when goyaml is requested to marshal the given value, and // marshalling into a YAML tag and value.
// the result of this method will be marshaled in place of the actual object.
type Getter interface { type Getter interface {
GetYAML() (tag string, value interface{}) GetYAML() (tag string, value interface{})
} }
// Unmarshal decodes the first document found within the in byte slice // Unmarshal decodes the first document found within the in byte slice
// and assigns decoded values into the object pointed by out. // and assigns decoded values into the out value.
// //
// Maps, pointers to structs and ints, etc, may all be used as out values. // Maps and pointers (to a struct, string, int, etc) are accepted as out
// If an internal pointer within a struct is not initialized, goyaml // values. If an internal pointer within a struct is not initialized,
// will initialize it if necessary for unmarshalling the provided data, // the yaml package will initialize it if necessary for unmarshalling
// but the struct provided as out must not be a nil pointer. // the provided data. The out parameter must not be nil.
// //
// The type of the decoded values and the type of out will be considered, // The type of the decoded values and the type of out will be considered,
// and Unmarshal() will do the best possible job to unmarshal values // and Unmarshal will do the best possible job to unmarshal values
// appropriately. It is NOT considered an error, though, to skip values // appropriately. It is NOT considered an error, though, to skip values
// because they are not available in the decoded YAML, or if they are not // because they are not available in the decoded YAML, or if they are not
// compatible with the out value. To ensure something was properly // compatible with the out value. To ensure something was properly
@@ -61,11 +62,11 @@ type Getter interface {
// field (usually the zero value). // field (usually the zero value).
// //
// Struct fields are only unmarshalled if they are exported (have an // Struct fields are only unmarshalled if they are exported (have an
// upper case first letter), and will be unmarshalled using the field // upper case first letter), and are unmarshalled using the field name
// name lowercased by default. When custom field names are desired, the // lowercased as the default key. Custom keys may be defined via the
// tag value may be used to tweak the name. Everything before the first // "yaml" name in the field tag: the content preceding the first comma
// comma in the field tag will be used as the name. The values following // is used as the key, and the following comma-separated options are
// the comma are used to tweak the marshalling process (see Marshal). // used to tweak the marshalling process (see Marshal).
// Conflicting names result in a runtime error. // Conflicting names result in a runtime error.
// //
// For example: // For example:
@@ -74,8 +75,8 @@ type Getter interface {
// F int `yaml:"a,omitempty"` // F int `yaml:"a,omitempty"`
// B int // B int
// } // }
// var T t // var t T
// goyaml.Unmarshal([]byte("a: 1\nb: 2"), &t) // yaml.Unmarshal([]byte("a: 1\nb: 2"), &t)
// //
// See the documentation of Marshal for the format of tags and a list of // See the documentation of Marshal for the format of tags and a list of
// supported tag options. // supported tag options.
@@ -83,25 +84,31 @@ type Getter interface {
func Unmarshal(in []byte, out interface{}) (err error) { func Unmarshal(in []byte, out interface{}) (err error) {
defer handleErr(&err) defer handleErr(&err)
d := newDecoder() d := newDecoder()
p := newParser(in) p := newParser(in, UnmarshalMappingKeyTransform)
defer p.destroy() defer p.destroy()
node := p.parse() node := p.parse()
if node != nil { if node != nil {
d.unmarshal(node, reflect.ValueOf(out)) v := reflect.ValueOf(out)
if v.Kind() == reflect.Ptr && !v.IsNil() {
v = v.Elem()
}
d.unmarshal(node, v)
} }
return nil return nil
} }
// Marshal serializes the value provided into a YAML document. The structure // Marshal serializes the value provided into a YAML document. The structure
// of the generated document will reflect the structure of the value itself. // of the generated document will reflect the structure of the value itself.
// Maps, pointers to structs and ints, etc, may all be used as the in value. // Maps and pointers (to struct, string, int, etc) are accepted as the in value.
// //
// In the case of struct values, only exported fields will be serialized. // Struct fields are only unmarshalled if they are exported (have an upper case
// The lowercased field name is used as the key for each exported field, // first letter), and are unmarshalled using the field name lowercased as the
// but this behavior may be changed using the respective field tag. // default key. Custom keys may be defined via the "yaml" name in the field
// The tag may also contain flags to tweak the marshalling behavior for // tag: the content preceding the first comma is used as the key, and the
// the field. Conflicting names result in a runtime error. The tag format // following comma-separated options are used to tweak the marshalling process.
// accepted is: // Conflicting names result in a runtime error.
//
// The field tag format accepted is:
// //
// `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)` // `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)`
// //
@@ -126,8 +133,8 @@ func Unmarshal(in []byte, out interface{}) (err error) {
// F int "a,omitempty" // F int "a,omitempty"
// B int // B int
// } // }
// goyaml.Marshal(&T{B: 2}) // Returns "b: 2\n" // yaml.Marshal(&T{B: 2}) // Returns "b: 2\n"
// goyaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n" // yaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n"
// //
func Marshal(in interface{}) (out []byte, err error) { func Marshal(in interface{}) (out []byte, err error) {
defer handleErr(&err) defer handleErr(&err)
@@ -139,10 +146,21 @@ func Marshal(in interface{}) (out []byte, err error) {
return return
} }
// UnmarshalMappingKeyTransform is a string transformation that is applied to
// each mapping key in a YAML document before it is unmarshalled. By default,
// UnmarshalMappingKeyTransform is an identity transform (no modification).
var UnmarshalMappingKeyTransform transformString = identityTransform
type transformString func(in string) (out string)
func identityTransform(in string) (out string) {
return in
}
// -------------------------------------------------------------------------- // --------------------------------------------------------------------------
// Maintain a mapping of keys to structure field indexes // Maintain a mapping of keys to structure field indexes
// The code in this section was copied from gobson. // The code in this section was copied from mgo/bson.
// structInfo holds details for the serialization of fields of // structInfo holds details for the serialization of fields of
// a given struct. // a given struct.
@@ -168,12 +186,6 @@ type fieldInfo struct {
var structMap = make(map[reflect.Type]*structInfo) var structMap = make(map[reflect.Type]*structInfo)
var fieldMapMutex sync.RWMutex var fieldMapMutex sync.RWMutex
type externalPanic string
func (e externalPanic) String() string {
return string(e)
}
func getStructInfo(st reflect.Type) (*structInfo, error) { func getStructInfo(st reflect.Type) (*structInfo, error) {
fieldMapMutex.RLock() fieldMapMutex.RLock()
sinfo, found := structMap[st] sinfo, found := structMap[st]
@@ -214,8 +226,7 @@ func getStructInfo(st reflect.Type) (*structInfo, error) {
case "inline": case "inline":
inline = true inline = true
default: default:
msg := fmt.Sprintf("Unsupported flag %q in tag %q of type %s", flag, tag, st) return nil, errors.New(fmt.Sprintf("Unsupported flag %q in tag %q of type %s", flag, tag, st))
panic(externalPanic(msg))
} }
} }
tag = fields[0] tag = fields[0]
@@ -223,6 +234,7 @@ func getStructInfo(st reflect.Type) (*structInfo, error) {
if inline { if inline {
switch field.Type.Kind() { switch field.Type.Kind() {
// TODO: Implement support for inline maps.
//case reflect.Map: //case reflect.Map:
// if inlineMap >= 0 { // if inlineMap >= 0 {
// return nil, errors.New("Multiple ,inline maps in struct " + st.String()) // return nil, errors.New("Multiple ,inline maps in struct " + st.String())
@@ -250,8 +262,8 @@ func getStructInfo(st reflect.Type) (*structInfo, error) {
fieldsList = append(fieldsList, finfo) fieldsList = append(fieldsList, finfo)
} }
default: default:
//panic("Option ,inline needs a struct value or map field") //return nil, errors.New("Option ,inline needs a struct value or map field")
panic("Option ,inline needs a struct value field") return nil, errors.New("Option ,inline needs a struct value field")
} }
continue continue
} }

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
import ( import (
"io" "io"
@@ -294,6 +294,10 @@ const (
yaml_SEQ_TAG = "tag:yaml.org,2002:seq" // The tag !!seq is used to denote sequences. yaml_SEQ_TAG = "tag:yaml.org,2002:seq" // The tag !!seq is used to denote sequences.
yaml_MAP_TAG = "tag:yaml.org,2002:map" // The tag !!map is used to denote mapping. yaml_MAP_TAG = "tag:yaml.org,2002:map" // The tag !!map is used to denote mapping.
// Not in original libyaml.
yaml_BINARY_TAG = "tag:yaml.org,2002:binary"
yaml_MERGE_TAG = "tag:yaml.org,2002:merge"
yaml_DEFAULT_SCALAR_TAG = yaml_STR_TAG // The default scalar tag is !!str. yaml_DEFAULT_SCALAR_TAG = yaml_STR_TAG // The default scalar tag is !!str.
yaml_DEFAULT_SEQUENCE_TAG = yaml_SEQ_TAG // The default sequence tag is !!seq. yaml_DEFAULT_SEQUENCE_TAG = yaml_SEQ_TAG // The default sequence tag is !!seq.
yaml_DEFAULT_MAPPING_TAG = yaml_MAP_TAG // The default mapping tag is !!map. yaml_DEFAULT_MAPPING_TAG = yaml_MAP_TAG // The default mapping tag is !!map.

View File

@@ -1,4 +1,4 @@
package goyaml package yaml
const ( const (
// The size of the input raw buffer. // The size of the input raw buffer.

View File

@@ -2,7 +2,7 @@ package introspect
import ( import (
"encoding/xml" "encoding/xml"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
"strings" "strings"
) )

View File

@@ -2,7 +2,7 @@ package introspect
import ( import (
"encoding/xml" "encoding/xml"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
"reflect" "reflect"
) )

View File

@@ -3,8 +3,8 @@
package prop package prop
import ( import (
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus/introspect" "github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/guelfey/go.dbus/introspect"
"sync" "sync"
) )

27
Godeps/_workspace/src/github.com/tarm/goserial/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,27 @@
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -0,0 +1,63 @@
GoSerial
========
A simple go package to allow you to read and write from the
serial port as a stream of bytes.
Details
-------
It aims to have the same API on all platforms, including windows. As
an added bonus, the windows package does not use cgo, so you can cross
compile for windows from another platform. Unfortunately goinstall
does not currently let you cross compile so you will have to do it
manually:
GOOS=windows make clean install
Currently there is very little in the way of configurability. You can
set the baud rate. Then you can Read(), Write(), or Close() the
connection. Read() will block until at least one byte is returned.
Write is the same. There is currently no exposed way to set the
timeouts, though patches are welcome.
Currently all ports are opened with 8 data bits, 1 stop bit, no
parity, no hardware flow control, and no software flow control. This
works fine for many real devices and many faux serial devices
including usb-to-serial converters and bluetooth serial ports.
You may Read() and Write() simulantiously on the same connection (from
different goroutines).
Usage
-----
```go
package main
import (
"github.com/tarm/goserial"
"log"
)
func main() {
c := &serial.Config{Name: "COM45", Baud: 115200}
s, err := serial.OpenPort(c)
if err != nil {
log.Fatal(err)
}
n, err := s.Write([]byte("test"))
if err != nil {
log.Fatal(err)
}
buf := make([]byte, 128)
n, err = s.Read(buf)
if err != nil {
log.Fatal(err)
}
log.Print("%q", buf[:n])
}
```
Possible Future Work
--------------------
- better tests (loopback etc)

View File

@@ -0,0 +1,61 @@
package serial
import (
"testing"
"time"
)
func TestConnection(t *testing.T) {
c0 := &Config{Name: "/dev/ttyUSB0", Baud: 115200}
c1 := &Config{Name: "/dev/ttyUSB1", Baud: 115200}
s1, err := OpenPort(c0)
if err != nil {
t.Fatal(err)
}
s2, err := OpenPort(c1)
if err != nil {
t.Fatal(err)
}
ch := make(chan int, 1)
go func() {
buf := make([]byte, 128)
var readCount int
for {
n, err := s2.Read(buf)
if err != nil {
t.Fatal(err)
}
readCount++
t.Logf("Read %v %v bytes: % 02x %s", readCount, n, buf[:n], buf[:n])
select {
case <-ch:
ch <- readCount
close(ch)
default:
}
}
}()
if _, err = s1.Write([]byte("hello")); err != nil {
t.Fatal(err)
}
if _, err = s1.Write([]byte(" ")); err != nil {
t.Fatal(err)
}
time.Sleep(time.Second)
if _, err = s1.Write([]byte("world")); err != nil {
t.Fatal(err)
}
time.Sleep(time.Second / 10)
ch <- 0
s1.Write([]byte(" ")) // We could be blocked in the read without this
c := <-ch
exp := 5
if c >= exp {
t.Fatalf("Expected less than %v read, got %v", exp, c)
}
}

View File

@@ -0,0 +1,99 @@
/*
Goserial is a simple go package to allow you to read and write from
the serial port as a stream of bytes.
It aims to have the same API on all platforms, including windows. As
an added bonus, the windows package does not use cgo, so you can cross
compile for windows from another platform. Unfortunately goinstall
does not currently let you cross compile so you will have to do it
manually:
GOOS=windows make clean install
Currently there is very little in the way of configurability. You can
set the baud rate. Then you can Read(), Write(), or Close() the
connection. Read() will block until at least one byte is returned.
Write is the same. There is currently no exposed way to set the
timeouts, though patches are welcome.
Currently all ports are opened with 8 data bits, 1 stop bit, no
parity, no hardware flow control, and no software flow control. This
works fine for many real devices and many faux serial devices
including usb-to-serial converters and bluetooth serial ports.
You may Read() and Write() simulantiously on the same connection (from
different goroutines).
Example usage:
package main
import (
"github.com/tarm/goserial"
"log"
)
func main() {
c := &serial.Config{Name: "COM5", Baud: 115200}
s, err := serial.OpenPort(c)
if err != nil {
log.Fatal(err)
}
n, err := s.Write([]byte("test"))
if err != nil {
log.Fatal(err)
}
buf := make([]byte, 128)
n, err = s.Read(buf)
if err != nil {
log.Fatal(err)
}
log.Print("%q", buf[:n])
}
*/
package serial
import "io"
// Config contains the information needed to open a serial port.
//
// Currently few options are implemented, but more may be added in the
// future (patches welcome), so it is recommended that you create a
// new config addressing the fields by name rather than by order.
//
// For example:
//
// c0 := &serial.Config{Name: "COM45", Baud: 115200}
// or
// c1 := new(serial.Config)
// c1.Name = "/dev/tty.usbserial"
// c1.Baud = 115200
//
type Config struct {
Name string
Baud int
// Size int // 0 get translated to 8
// Parity SomeNewTypeToGetCorrectDefaultOf_None
// StopBits SomeNewTypeToGetCorrectDefaultOf_1
// RTSFlowControl bool
// DTRFlowControl bool
// XONFlowControl bool
// CRLFTranslate bool
// TimeoutStuff int
}
// OpenPort opens a serial port with the specified configuration
func OpenPort(c *Config) (io.ReadWriteCloser, error) {
return openPort(c.Name, c.Baud)
}
// func Flush()
// func SendBreak()
// func RegisterBreakHandler(func())

View File

@@ -0,0 +1,90 @@
// +build linux,!cgo
package serial
import (
"io"
"os"
"syscall"
"unsafe"
)
func openPort(name string, baud int) (rwc io.ReadWriteCloser, err error) {
var bauds = map[int]uint32{
50: syscall.B50,
75: syscall.B75,
110: syscall.B110,
134: syscall.B134,
150: syscall.B150,
200: syscall.B200,
300: syscall.B300,
600: syscall.B600,
1200: syscall.B1200,
1800: syscall.B1800,
2400: syscall.B2400,
4800: syscall.B4800,
9600: syscall.B9600,
19200: syscall.B19200,
38400: syscall.B38400,
57600: syscall.B57600,
115200: syscall.B115200,
230400: syscall.B230400,
460800: syscall.B460800,
500000: syscall.B500000,
576000: syscall.B576000,
921600: syscall.B921600,
1000000: syscall.B1000000,
1152000: syscall.B1152000,
1500000: syscall.B1500000,
2000000: syscall.B2000000,
2500000: syscall.B2500000,
3000000: syscall.B3000000,
3500000: syscall.B3500000,
4000000: syscall.B4000000,
}
rate := bauds[baud]
if rate == 0 {
return
}
f, err := os.OpenFile(name, syscall.O_RDWR|syscall.O_NOCTTY|syscall.O_NONBLOCK, 0666)
if err != nil {
return nil, err
}
defer func() {
if err != nil && f != nil {
f.Close()
}
}()
fd := f.Fd()
t := syscall.Termios{
Iflag: syscall.IGNPAR,
Cflag: syscall.CS8 | syscall.CREAD | syscall.CLOCAL | rate,
Cc: [32]uint8{syscall.VMIN: 1},
Ispeed: rate,
Ospeed: rate,
}
if _, _, errno := syscall.Syscall6(
syscall.SYS_IOCTL,
uintptr(fd),
uintptr(syscall.TCSETS),
uintptr(unsafe.Pointer(&t)),
0,
0,
0,
); errno != 0 {
return nil, errno
}
if err = syscall.SetNonblock(int(fd), false); err != nil {
return
}
return f, nil
}

View File

@@ -0,0 +1,107 @@
// +build !windows,cgo
package serial
// #include <termios.h>
// #include <unistd.h>
import "C"
// TODO: Maybe change to using syscall package + ioctl instead of cgo
import (
"errors"
"fmt"
"io"
"os"
"syscall"
//"unsafe"
)
func openPort(name string, baud int) (rwc io.ReadWriteCloser, err error) {
f, err := os.OpenFile(name, syscall.O_RDWR|syscall.O_NOCTTY|syscall.O_NONBLOCK, 0666)
if err != nil {
return
}
fd := C.int(f.Fd())
if C.isatty(fd) != 1 {
f.Close()
return nil, errors.New("File is not a tty")
}
var st C.struct_termios
_, err = C.tcgetattr(fd, &st)
if err != nil {
f.Close()
return nil, err
}
var speed C.speed_t
switch baud {
case 115200:
speed = C.B115200
case 57600:
speed = C.B57600
case 38400:
speed = C.B38400
case 19200:
speed = C.B19200
case 9600:
speed = C.B9600
case 4800:
speed = C.B4800
case 2400:
speed = C.B2400
default:
f.Close()
return nil, fmt.Errorf("Unknown baud rate %v", baud)
}
_, err = C.cfsetispeed(&st, speed)
if err != nil {
f.Close()
return nil, err
}
_, err = C.cfsetospeed(&st, speed)
if err != nil {
f.Close()
return nil, err
}
// Select local mode
st.c_cflag |= (C.CLOCAL | C.CREAD)
// Select raw mode
st.c_lflag &= ^C.tcflag_t(C.ICANON | C.ECHO | C.ECHOE | C.ISIG)
st.c_oflag &= ^C.tcflag_t(C.OPOST)
_, err = C.tcsetattr(fd, C.TCSANOW, &st)
if err != nil {
f.Close()
return nil, err
}
//fmt.Println("Tweaking", name)
r1, _, e := syscall.Syscall(syscall.SYS_FCNTL,
uintptr(f.Fd()),
uintptr(syscall.F_SETFL),
uintptr(0))
if e != 0 || r1 != 0 {
s := fmt.Sprint("Clearing NONBLOCK syscall error:", e, r1)
f.Close()
return nil, errors.New(s)
}
/*
r1, _, e = syscall.Syscall(syscall.SYS_IOCTL,
uintptr(f.Fd()),
uintptr(0x80045402), // IOSSIOSPEED
uintptr(unsafe.Pointer(&baud)));
if e != 0 || r1 != 0 {
s := fmt.Sprint("Baudrate syscall error:", e, r1)
f.Close()
return nil, os.NewError(s)
}
*/
return f, nil
}

View File

@@ -0,0 +1,263 @@
// +build windows
package serial
import (
"fmt"
"io"
"os"
"sync"
"syscall"
"unsafe"
)
type serialPort struct {
f *os.File
fd syscall.Handle
rl sync.Mutex
wl sync.Mutex
ro *syscall.Overlapped
wo *syscall.Overlapped
}
type structDCB struct {
DCBlength, BaudRate uint32
flags [4]byte
wReserved, XonLim, XoffLim uint16
ByteSize, Parity, StopBits byte
XonChar, XoffChar, ErrorChar, EofChar, EvtChar byte
wReserved1 uint16
}
type structTimeouts struct {
ReadIntervalTimeout uint32
ReadTotalTimeoutMultiplier uint32
ReadTotalTimeoutConstant uint32
WriteTotalTimeoutMultiplier uint32
WriteTotalTimeoutConstant uint32
}
func openPort(name string, baud int) (rwc io.ReadWriteCloser, err error) {
if len(name) > 0 && name[0] != '\\' {
name = "\\\\.\\" + name
}
h, err := syscall.CreateFile(syscall.StringToUTF16Ptr(name),
syscall.GENERIC_READ|syscall.GENERIC_WRITE,
0,
nil,
syscall.OPEN_EXISTING,
syscall.FILE_ATTRIBUTE_NORMAL|syscall.FILE_FLAG_OVERLAPPED,
0)
if err != nil {
return nil, err
}
f := os.NewFile(uintptr(h), name)
defer func() {
if err != nil {
f.Close()
}
}()
if err = setCommState(h, baud); err != nil {
return
}
if err = setupComm(h, 64, 64); err != nil {
return
}
if err = setCommTimeouts(h); err != nil {
return
}
if err = setCommMask(h); err != nil {
return
}
ro, err := newOverlapped()
if err != nil {
return
}
wo, err := newOverlapped()
if err != nil {
return
}
port := new(serialPort)
port.f = f
port.fd = h
port.ro = ro
port.wo = wo
return port, nil
}
func (p *serialPort) Close() error {
return p.f.Close()
}
func (p *serialPort) Write(buf []byte) (int, error) {
p.wl.Lock()
defer p.wl.Unlock()
if err := resetEvent(p.wo.HEvent); err != nil {
return 0, err
}
var n uint32
err := syscall.WriteFile(p.fd, buf, &n, p.wo)
if err != nil && err != syscall.ERROR_IO_PENDING {
return int(n), err
}
return getOverlappedResult(p.fd, p.wo)
}
func (p *serialPort) Read(buf []byte) (int, error) {
if p == nil || p.f == nil {
return 0, fmt.Errorf("Invalid port on read %v %v", p, p.f)
}
p.rl.Lock()
defer p.rl.Unlock()
if err := resetEvent(p.ro.HEvent); err != nil {
return 0, err
}
var done uint32
err := syscall.ReadFile(p.fd, buf, &done, p.ro)
if err != nil && err != syscall.ERROR_IO_PENDING {
return int(done), err
}
return getOverlappedResult(p.fd, p.ro)
}
var (
nSetCommState,
nSetCommTimeouts,
nSetCommMask,
nSetupComm,
nGetOverlappedResult,
nCreateEvent,
nResetEvent uintptr
)
func init() {
k32, err := syscall.LoadLibrary("kernel32.dll")
if err != nil {
panic("LoadLibrary " + err.Error())
}
defer syscall.FreeLibrary(k32)
nSetCommState = getProcAddr(k32, "SetCommState")
nSetCommTimeouts = getProcAddr(k32, "SetCommTimeouts")
nSetCommMask = getProcAddr(k32, "SetCommMask")
nSetupComm = getProcAddr(k32, "SetupComm")
nGetOverlappedResult = getProcAddr(k32, "GetOverlappedResult")
nCreateEvent = getProcAddr(k32, "CreateEventW")
nResetEvent = getProcAddr(k32, "ResetEvent")
}
func getProcAddr(lib syscall.Handle, name string) uintptr {
addr, err := syscall.GetProcAddress(lib, name)
if err != nil {
panic(name + " " + err.Error())
}
return addr
}
func setCommState(h syscall.Handle, baud int) error {
var params structDCB
params.DCBlength = uint32(unsafe.Sizeof(params))
params.flags[0] = 0x01 // fBinary
params.flags[0] |= 0x10 // Assert DSR
params.BaudRate = uint32(baud)
params.ByteSize = 8
r, _, err := syscall.Syscall(nSetCommState, 2, uintptr(h), uintptr(unsafe.Pointer(&params)), 0)
if r == 0 {
return err
}
return nil
}
func setCommTimeouts(h syscall.Handle) error {
var timeouts structTimeouts
const MAXDWORD = 1<<32 - 1
timeouts.ReadIntervalTimeout = MAXDWORD
timeouts.ReadTotalTimeoutMultiplier = MAXDWORD
timeouts.ReadTotalTimeoutConstant = MAXDWORD - 1
/* From http://msdn.microsoft.com/en-us/library/aa363190(v=VS.85).aspx
For blocking I/O see below:
Remarks:
If an application sets ReadIntervalTimeout and
ReadTotalTimeoutMultiplier to MAXDWORD and sets
ReadTotalTimeoutConstant to a value greater than zero and
less than MAXDWORD, one of the following occurs when the
ReadFile function is called:
If there are any bytes in the input buffer, ReadFile returns
immediately with the bytes in the buffer.
If there are no bytes in the input buffer, ReadFile waits
until a byte arrives and then returns immediately.
If no bytes arrive within the time specified by
ReadTotalTimeoutConstant, ReadFile times out.
*/
r, _, err := syscall.Syscall(nSetCommTimeouts, 2, uintptr(h), uintptr(unsafe.Pointer(&timeouts)), 0)
if r == 0 {
return err
}
return nil
}
func setupComm(h syscall.Handle, in, out int) error {
r, _, err := syscall.Syscall(nSetupComm, 3, uintptr(h), uintptr(in), uintptr(out))
if r == 0 {
return err
}
return nil
}
func setCommMask(h syscall.Handle) error {
const EV_RXCHAR = 0x0001
r, _, err := syscall.Syscall(nSetCommMask, 2, uintptr(h), EV_RXCHAR, 0)
if r == 0 {
return err
}
return nil
}
func resetEvent(h syscall.Handle) error {
r, _, err := syscall.Syscall(nResetEvent, 1, uintptr(h), 0, 0)
if r == 0 {
return err
}
return nil
}
func newOverlapped() (*syscall.Overlapped, error) {
var overlapped syscall.Overlapped
r, _, err := syscall.Syscall6(nCreateEvent, 4, 0, 1, 0, 0, 0, 0)
if r == 0 {
return nil, err
}
overlapped.HEvent = syscall.Handle(r)
return &overlapped, nil
}
func getOverlappedResult(h syscall.Handle, overlapped *syscall.Overlapped) (int, error) {
var n int
r, _, err := syscall.Syscall6(nGetOverlappedResult, 4,
uintptr(h),
uintptr(unsafe.Pointer(overlapped)),
uintptr(unsafe.Pointer(&n)), 1, 0, 0)
if r == 0 {
return n, err
}
return n, nil
}

3
MAINTAINERS Normal file
View File

@@ -0,0 +1,3 @@
Alex Crawford <alex.crawford@coreos.com> (@crawford)
Jonathan Boulle <jonathan.boulle@coreos.com> (@jonboulle)
Brian Waldon <brian.waldon@coreos.com> (@bcwaldon)

158
config/config.go Normal file
View File

@@ -0,0 +1,158 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
import (
"fmt"
"reflect"
"regexp"
"strings"
"github.com/coreos/coreos-cloudinit/Godeps/_workspace/src/github.com/coreos/yaml"
)
// CloudConfig encapsulates the entire cloud-config configuration file and maps
// directly to YAML. Fields that cannot be set in the cloud-config (fields
// used for internal use) have the YAML tag '-' so that they aren't marshalled.
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
CoreOS CoreOS `yaml:"coreos"`
WriteFiles []File `yaml:"write_files"`
Hostname string `yaml:"hostname"`
Users []User `yaml:"users"`
ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
NetworkConfigPath string `yaml:"-"`
NetworkConfig string `yaml:"-"`
}
type CoreOS struct {
Etcd Etcd `yaml:"etcd"`
Flannel Flannel `yaml:"flannel"`
Fleet Fleet `yaml:"fleet"`
Locksmith Locksmith `yaml:"locksmith"`
OEM OEM `yaml:"oem"`
Update Update `yaml:"update"`
Units []Unit `yaml:"units"`
}
func IsCloudConfig(userdata string) bool {
header := strings.SplitN(userdata, "\n", 2)[0]
// Explicitly trim the header so we can handle user-data from
// non-unix operating systems. The rest of the file is parsed
// by yaml, which correctly handles CRLF.
header = strings.TrimSuffix(header, "\r")
return (header == "#cloud-config")
}
// NewCloudConfig instantiates a new CloudConfig from the given contents (a
// string of YAML), returning any error encountered. It will ignore unknown
// fields but log encountering them.
func NewCloudConfig(contents string) (*CloudConfig, error) {
yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
return strings.Replace(nameIn, "-", "_", -1)
}
var cfg CloudConfig
err := yaml.Unmarshal([]byte(contents), &cfg)
return &cfg, err
}
func (cc CloudConfig) String() string {
bytes, err := yaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
// IsZero returns whether or not the parameter is the zero value for its type.
// If the parameter is a struct, only the exported fields are considered.
func IsZero(c interface{}) bool {
return isZero(reflect.ValueOf(c))
}
type ErrorValid struct {
Value string
Valid string
Field string
}
func (e ErrorValid) Error() string {
return fmt.Sprintf("invalid value %q for option %q (valid options: %q)", e.Value, e.Field, e.Valid)
}
// AssertStructValid checks the fields in the structure and makes sure that
// they contain valid values as specified by the 'valid' flag. Empty fields are
// implicitly valid.
func AssertStructValid(c interface{}) error {
ct := reflect.TypeOf(c)
cv := reflect.ValueOf(c)
for i := 0; i < ct.NumField(); i++ {
ft := ct.Field(i)
if !isFieldExported(ft) {
continue
}
if err := AssertValid(cv.Field(i), ft.Tag.Get("valid")); err != nil {
err.Field = ft.Name
return err
}
}
return nil
}
// AssertValid checks to make sure that the given value is in the list of
// valid values. Zero values are implicitly valid.
func AssertValid(value reflect.Value, valid string) *ErrorValid {
if valid == "" || isZero(value) {
return nil
}
vs := fmt.Sprintf("%v", value.Interface())
if m, _ := regexp.MatchString(valid, vs); m {
return nil
}
return &ErrorValid{
Value: vs,
Valid: valid,
}
}
func isZero(v reflect.Value) bool {
switch v.Kind() {
case reflect.Struct:
vt := v.Type()
for i := 0; i < v.NumField(); i++ {
if isFieldExported(vt.Field(i)) && !isZero(v.Field(i)) {
return false
}
}
return true
default:
return v.Interface() == reflect.Zero(v.Type()).Interface()
}
}
func isFieldExported(f reflect.StructField) bool {
return f.PkgPath == ""
}

499
config/config_test.go Normal file
View File

@@ -0,0 +1,499 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
import (
"reflect"
"regexp"
"strings"
"testing"
)
func TestNewCloudConfig(t *testing.T) {
tests := []struct {
contents string
config CloudConfig
}{
{},
{
contents: "#cloud-config\nwrite_files:\n - path: underscore",
config: CloudConfig{WriteFiles: []File{File{Path: "underscore"}}},
},
{
contents: "#cloud-config\nwrite-files:\n - path: hyphen",
config: CloudConfig{WriteFiles: []File{File{Path: "hyphen"}}},
},
{
contents: "#cloud-config\ncoreos:\n update:\n reboot-strategy: off",
config: CloudConfig{CoreOS: CoreOS{Update: Update{RebootStrategy: "off"}}},
},
{
contents: "#cloud-config\ncoreos:\n update:\n reboot-strategy: false",
config: CloudConfig{CoreOS: CoreOS{Update: Update{RebootStrategy: "false"}}},
},
{
contents: "#cloud-config\nwrite_files:\n - permissions: 0744",
config: CloudConfig{WriteFiles: []File{File{RawFilePermissions: "0744"}}},
},
{
contents: "#cloud-config\nwrite_files:\n - permissions: 744",
config: CloudConfig{WriteFiles: []File{File{RawFilePermissions: "744"}}},
},
{
contents: "#cloud-config\nwrite_files:\n - permissions: '0744'",
config: CloudConfig{WriteFiles: []File{File{RawFilePermissions: "0744"}}},
},
{
contents: "#cloud-config\nwrite_files:\n - permissions: '744'",
config: CloudConfig{WriteFiles: []File{File{RawFilePermissions: "744"}}},
},
}
for i, tt := range tests {
config, err := NewCloudConfig(tt.contents)
if err != nil {
t.Errorf("bad error (test case #%d): want %v, got %s", i, nil, err)
}
if !reflect.DeepEqual(&tt.config, config) {
t.Errorf("bad config (test case #%d): want %#v, got %#v", i, tt.config, config)
}
}
}
func TestIsZero(t *testing.T) {
tests := []struct {
c interface{}
empty bool
}{
{struct{}{}, true},
{struct{ a, b string }{}, true},
{struct{ A, b string }{}, true},
{struct{ A, B string }{}, true},
{struct{ A string }{A: "hello"}, false},
{struct{ A int }{}, true},
{struct{ A int }{A: 1}, false},
}
for _, tt := range tests {
if empty := IsZero(tt.c); tt.empty != empty {
t.Errorf("bad result (%q): want %t, got %t", tt.c, tt.empty, empty)
}
}
}
func TestAssertStructValid(t *testing.T) {
tests := []struct {
c interface{}
err error
}{
{struct{}{}, nil},
{struct {
A, b string `valid:"^1|2$"`
}{}, nil},
{struct {
A, b string `valid:"^1|2$"`
}{A: "1", b: "2"}, nil},
{struct {
A, b string `valid:"^1|2$"`
}{A: "1", b: "hello"}, nil},
{struct {
A, b string `valid:"^1|2$"`
}{A: "hello", b: "2"}, &ErrorValid{Value: "hello", Field: "A", Valid: "^1|2$"}},
{struct {
A, b int `valid:"^1|2$"`
}{}, nil},
{struct {
A, b int `valid:"^1|2$"`
}{A: 1, b: 2}, nil},
{struct {
A, b int `valid:"^1|2$"`
}{A: 1, b: 9}, nil},
{struct {
A, b int `valid:"^1|2$"`
}{A: 9, b: 2}, &ErrorValid{Value: "9", Field: "A", Valid: "^1|2$"}},
}
for _, tt := range tests {
if err := AssertStructValid(tt.c); !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.err, err)
}
}
}
func TestConfigCompile(t *testing.T) {
tests := []interface{}{
Etcd{},
File{},
Flannel{},
Fleet{},
Locksmith{},
OEM{},
Unit{},
Update{},
}
for _, tt := range tests {
ttt := reflect.TypeOf(tt)
for i := 0; i < ttt.NumField(); i++ {
ft := ttt.Field(i)
if !isFieldExported(ft) {
continue
}
if _, err := regexp.Compile(ft.Tag.Get("valid")); err != nil {
t.Errorf("bad regexp(%s.%s): want %v, got %s", ttt.Name(), ft.Name, nil, err)
}
}
}
}
func TestCloudConfigUnknownKeys(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
coreos_unknown:
foo: "bar"
section_unknown:
dunno:
something
bare_unknown:
bar
write_files:
- content: fun
path: /var/party
file_unknown: nofun
users:
- name: fry
passwd: somehash
user_unknown: philip
hostname:
foo
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
}
if cfg.Hostname != "foo" {
t.Fatalf("hostname not correctly set when invalid keys are present")
}
if cfg.CoreOS.Etcd.Discovery != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
t.Fatalf("etcd section not correctly set when invalid keys are present")
}
if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
t.Fatalf("write_files section not correctly set when invalid keys are present")
}
if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
t.Fatalf("users section not correctly set when invalid keys are present")
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig("")
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
if cfg.Hostname != "" {
t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
update:
reboot_strategy: reboot
units:
- name: 50-eth0.network
runtime: yes
content: '[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version_id: 168.0.0
home_url: https://www.rackspace.com/cloud/servers/
bug_report_url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
write_files:
- content: |
penny
elroy
path: /etc/dogepack.conf
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 2 {
t.Error("Parsed incorrect number of SSH keys")
} else if keys[0] != "foobar" {
t.Error("Expected first SSH key to be 'foobar'")
} else if keys[1] != "foobaz" {
t.Error("Expected first SSH key to be 'foobaz'")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
wf := cfg.WriteFiles[0]
if wf.Content != "penny\nelroy\n" {
t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
}
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if wf.RawFilePermissions != "0644" {
t.Errorf("WriteFile has incorrect permissions %s", wf.RawFilePermissions)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
}
if wf.Owner != "root:dogepack" {
t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
}
}
if len(cfg.CoreOS.Units) != 1 {
t.Error("Failed to parse correct number of units")
} else {
u := cfg.CoreOS.Units[0]
expect := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if u.Content != expect {
t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
}
if u.Runtime != true {
t.Errorf("Unit has incorrect runtime value")
}
if u.Name != "50-eth0.network" {
t.Errorf("Unit has incorrect name %s", u.Name)
}
}
if cfg.CoreOS.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.CoreOS.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
if cfg.CoreOS.Update.RebootStrategy != "reboot" {
t.Errorf("Failed to parse locksmith strategy")
}
}
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := `
ssh_authorized_keys:
- foo: bar
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig("")
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
t.Fatalf("Serialized config did not have expected header")
}
}
func TestCloudConfigUsers(t *testing.T) {
contents := `
users:
- name: elroy
passwd: somehash
ssh_authorized_keys:
- somekey
gecos: arbitrary comment
homedir: /home/place
no_create_home: yes
primary_group: things
groups:
- ping
- pong
no_user_group: true
system: y
no_log_init: True
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", len(cfg.Users))
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.PasswordHash != "somehash" {
t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
}
if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
} else {
key := user.SSHAuthorizedKeys[0]
if key != "somekey" {
t.Errorf("User SSH key is %q, expected 'somekey'", key)
}
}
if user.GECOS != "arbitrary comment" {
t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
}
if user.Homedir != "/home/place" {
t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
}
if !user.NoCreateHome {
t.Errorf("Failed to parse no_create_home field")
}
if user.PrimaryGroup != "things" {
t.Errorf("Failed to parse primary_group field, got %q", user.PrimaryGroup)
}
if len(user.Groups) != 2 {
t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
} else {
if user.Groups[0] != "ping" {
t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
}
if user.Groups[1] != "pong" {
t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
}
}
if !user.NoUserGroup {
t.Errorf("Failed to parse no_user_group field")
}
if !user.System {
t.Errorf("Failed to parse system field")
}
if !user.NoLogInit {
t.Errorf("Failed to parse no_log_init field")
}
}
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos_ssh_import_github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", len(cfg.Users))
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}
func TestCloudConfigUsersSSHImportURL(t *testing.T) {
contents := `
users:
- name: elroy
coreos_ssh_import_url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", len(cfg.Users))
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
}
}

56
config/decode.go Normal file
View File

@@ -0,0 +1,56 @@
package config
import (
"bytes"
"compress/gzip"
"encoding/base64"
"fmt"
)
func DecodeBase64Content(content string) ([]byte, error) {
output, err := base64.StdEncoding.DecodeString(content)
if err != nil {
return nil, fmt.Errorf("Unable to decode base64: %q", err)
}
return output, nil
}
func DecodeGzipContent(content string) ([]byte, error) {
gzr, err := gzip.NewReader(bytes.NewReader([]byte(content)))
if err != nil {
return nil, fmt.Errorf("Unable to decode gzip: %q", err)
}
defer gzr.Close()
buf := new(bytes.Buffer)
buf.ReadFrom(gzr)
return buf.Bytes(), nil
}
func DecodeContent(content string, encoding string) ([]byte, error) {
switch encoding {
case "":
return []byte(content), nil
case "b64", "base64":
return DecodeBase64Content(content)
case "gz", "gzip":
return DecodeGzipContent(content)
case "gz+base64", "gzip+base64", "gz+b64", "gzip+b64":
gz, err := DecodeBase64Content(content)
if err != nil {
return nil, err
}
return DecodeGzipContent(string(gz))
}
return nil, fmt.Errorf("Unsupported encoding %q", encoding)
}

19
config/etc_hosts.go Normal file
View File

@@ -0,0 +1,19 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
type EtcHosts string

53
config/etcd.go Normal file
View File

@@ -0,0 +1,53 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
type Etcd struct {
Addr string `yaml:"addr" env:"ETCD_ADDR"`
BindAddr string `yaml:"bind_addr" env:"ETCD_BIND_ADDR"`
CAFile string `yaml:"ca_file" env:"ETCD_CA_FILE"`
CertFile string `yaml:"cert_file" env:"ETCD_CERT_FILE"`
ClusterActiveSize int `yaml:"cluster_active_size" env:"ETCD_CLUSTER_ACTIVE_SIZE"`
ClusterRemoveDelay float64 `yaml:"cluster_remove_delay" env:"ETCD_CLUSTER_REMOVE_DELAY"`
ClusterSyncInterval float64 `yaml:"cluster_sync_interval" env:"ETCD_CLUSTER_SYNC_INTERVAL"`
CorsOrigins string `yaml:"cors" env:"ETCD_CORS"`
DataDir string `yaml:"data_dir" env:"ETCD_DATA_DIR"`
Discovery string `yaml:"discovery" env:"ETCD_DISCOVERY"`
GraphiteHost string `yaml:"graphite_host" env:"ETCD_GRAPHITE_HOST"`
HTTPReadTimeout float64 `yaml:"http_read_timeout" env:"ETCD_HTTP_READ_TIMEOUT"`
HTTPWriteTimeout float64 `yaml:"http_write_timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"`
KeyFile string `yaml:"key_file" env:"ETCD_KEY_FILE"`
MaxResultBuffer int `yaml:"max_result_buffer" env:"ETCD_MAX_RESULT_BUFFER"`
MaxRetryAttempts int `yaml:"max_retry_attempts" env:"ETCD_MAX_RETRY_ATTEMPTS"`
Name string `yaml:"name" env:"ETCD_NAME"`
PeerAddr string `yaml:"peer_addr" env:"ETCD_PEER_ADDR"`
PeerBindAddr string `yaml:"peer_bind_addr" env:"ETCD_PEER_BIND_ADDR"`
PeerCAFile string `yaml:"peer_ca_file" env:"ETCD_PEER_CA_FILE"`
PeerCertFile string `yaml:"peer_cert_file" env:"ETCD_PEER_CERT_FILE"`
PeerElectionTimeout int `yaml:"peer_election_timeout" env:"ETCD_PEER_ELECTION_TIMEOUT"`
PeerHeartbeatInterval int `yaml:"peer_heartbeat_interval" env:"ETCD_PEER_HEARTBEAT_INTERVAL"`
PeerKeyFile string `yaml:"peer_key_file" env:"ETCD_PEER_KEY_FILE"`
Peers string `yaml:"peers" env:"ETCD_PEERS"`
PeersFile string `yaml:"peers_file" env:"ETCD_PEERS_FILE"`
RetryInterval float64 `yaml:"retry_interval" env:"ETCD_RETRY_INTERVAL"`
Snapshot bool `yaml:"snapshot" env:"ETCD_SNAPSHOT"`
SnapshotCount int `yaml:"snapshot_count" env:"ETCD_SNAPSHOTCOUNT"`
StrTrace string `yaml:"trace" env:"ETCD_TRACE"`
Verbose bool `yaml:"verbose" env:"ETCD_VERBOSE"`
VeryVerbose bool `yaml:"very_verbose" env:"ETCD_VERY_VERBOSE"`
VeryVeryVerbose bool `yaml:"very_very_verbose" env:"ETCD_VERY_VERY_VERBOSE"`
}

25
config/file.go Normal file
View File

@@ -0,0 +1,25 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
type File struct {
Encoding string `yaml:"encoding" valid:"^(base64|b64|gz|gzip|gz\\+base64|gzip\\+base64|gz\\+b64|gzip\\+b64)$"`
Content string `yaml:"content"`
Owner string `yaml:"owner"`
Path string `yaml:"path"`
RawFilePermissions string `yaml:"permissions" valid:"^0?[0-7]{3,4}$"`
}

71
config/file_test.go Normal file
View File

@@ -0,0 +1,71 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
import (
"testing"
)
func TestEncodingValid(t *testing.T) {
tests := []struct {
value string
isValid bool
}{
{value: "base64", isValid: true},
{value: "b64", isValid: true},
{value: "gz", isValid: true},
{value: "gzip", isValid: true},
{value: "gz+base64", isValid: true},
{value: "gzip+base64", isValid: true},
{value: "gz+b64", isValid: true},
{value: "gzip+b64", isValid: true},
{value: "gzzzzbase64", isValid: false},
{value: "gzipppbase64", isValid: false},
{value: "unknown", isValid: false},
}
for _, tt := range tests {
isValid := (nil == AssertStructValid(File{Encoding: tt.value}))
if tt.isValid != isValid {
t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
}
}
}
func TestRawFilePermissionsValid(t *testing.T) {
tests := []struct {
value string
isValid bool
}{
{value: "744", isValid: true},
{value: "0744", isValid: true},
{value: "1744", isValid: true},
{value: "01744", isValid: true},
{value: "11744", isValid: false},
{value: "rwxr--r--", isValid: false},
{value: "800", isValid: false},
}
for _, tt := range tests {
isValid := (nil == AssertStructValid(File{RawFilePermissions: tt.value}))
if tt.isValid != isValid {
t.Errorf("bad assert (%s): want %t, got %t", tt.value, tt.isValid, isValid)
}
}
}

12
config/flannel.go Normal file
View File

@@ -0,0 +1,12 @@
package config
type Flannel struct {
EtcdEndpoints string `yaml:"etcd_endpoints" env:"FLANNELD_ETCD_ENDPOINTS"`
EtcdCAFile string `yaml:"etcd_cafile" env:"FLANNELD_ETCD_CAFILE"`
EtcdCertFile string `yaml:"etcd_certfile" env:"FLANNELD_ETCD_CERTFILE"`
EtcdKeyFile string `yaml:"etcd_keyfile" env:"FLANNELD_ETCD_KEYFILE"`
EtcdPrefix string `yaml:"etcd_prefix" env:"FLANNELD_ETCD_PREFIX"`
IPMasq string `yaml:"ip_masq" env:"FLANNELD_IP_MASQ"`
SubnetFile string `yaml:"subnet_file" env:"FLANNELD_SUBNET_FILE"`
Iface string `yaml:"interface" env:"FLANNELD_IFACE"`
}

31
config/fleet.go Normal file
View File

@@ -0,0 +1,31 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
type Fleet struct {
AgentTTL string `yaml:"agent_ttl" env:"FLEET_AGENT_TTL"`
EngineReconcileInterval float64 `yaml:"engine_reconcile_interval" env:"FLEET_ENGINE_RECONCILE_INTERVAL"`
EtcdCAFile string `yaml:"etcd_cafile" env:"FLEET_ETCD_CAFILE"`
EtcdCertFile string `yaml:"etcd_certfile" env:"FLEET_ETCD_CERTFILE"`
EtcdKeyFile string `yaml:"etcd_keyfile" env:"FLEET_ETCD_KEYFILE"`
EtcdKeyPrefix string `yaml:"etcd_key_prefix" env:"FLEET_ETCD_KEY_PREFIX"`
EtcdRequestTimeout float64 `yaml:"etcd_request_timeout" env:"FLEET_ETCD_REQUEST_TIMEOUT"`
EtcdServers string `yaml:"etcd_servers" env:"FLEET_ETCD_SERVERS"`
Metadata string `yaml:"metadata" env:"FLEET_METADATA"`
PublicIP string `yaml:"public_ip" env:"FLEET_PUBLIC_IP"`
Verbosity int `yaml:"verbosity" env:"FLEET_VERBOSITY"`
}

8
config/locksmith.go Normal file
View File

@@ -0,0 +1,8 @@
package config
type Locksmith struct {
Endpoint string `yaml:"endpoint" env:"LOCKSMITHD_ENDPOINT"`
EtcdCAFile string `yaml:"etcd_cafile" env:"LOCKSMITHD_ETCD_CAFILE"`
EtcdCertFile string `yaml:"etcd_certfile" env:"LOCKSMITHD_ETCD_CERTFILE"`
EtcdKeyFile string `yaml:"etcd_keyfile" env:"LOCKSMITHD_ETCD_KEYFILE"`
}

25
config/oem.go Normal file
View File

@@ -0,0 +1,25 @@
/*
Copyright 2014 CoreOS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
type OEM struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version_id"`
HomeURL string `yaml:"home_url"`
BugReportURL string `yaml:"bug_report_url"`
}

Some files were not shown because too many files have changed in this diff Show More