Compare commits

..

96 Commits

Author SHA1 Message Date
bb0c415a77 Merge pull request 'config: add conditions' (#287) from cond-config into master
Some checks failed
/ autoupdate (push) Failing after 1m4s
Reviewed-on: #287
2024-01-15 00:51:30 +03:00
8182cb008a config: add conditions
Some checks failed
lint / lint (pull_request) Successful in 1m27s
pr / test (pull_request) Failing after 1m13s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2024-01-15 00:49:27 +03:00
0771fa0647 fixup multiple client handling (#281)
Some checks failed
/ autoupdate (push) Failing after 1m28s
Reviewed-on: #281
2023-11-13 08:23:22 +03:00
b1fd82adf8 Merge pull request 'database: add FormatDSN' (#277) from database-new into master
Some checks failed
/ autoupdate (push) Failing after 1m26s
Reviewed-on: #277
2023-11-02 01:35:12 +03:00
4bc0e25017 database: add FormatDSN
Some checks failed
lint / lint (pull_request) Failing after 1m28s
pr / test (pull_request) Failing after 2m37s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-11-02 01:31:26 +03:00
aec552aa0b Merge pull request 'database: initial import for dsn parsing' (#275) from database into master
Some checks failed
/ autoupdate (push) Failing after 1m28s
Reviewed-on: #275
2023-11-01 23:44:01 +03:00
cc81bed81d cleanup
Some checks failed
lint / lint (pull_request) Failing after 1m29s
pr / test (pull_request) Failing after 2m37s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-11-01 23:42:07 +03:00
7fde39fba5 database: initial import for dsn parsing
Some checks failed
lint / lint (pull_request) Failing after 1m29s
pr / test (pull_request) Failing after 2m40s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-11-01 23:40:39 +03:00
79438f11e0 logger: slog break import cycle
All checks were successful
/ autoupdate (push) Successful in 1m13s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-17 01:48:50 +03:00
8d19abfebd small fixes
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-17 01:11:35 +03:00
77f3731329 change dep
Some checks are pending
/ autoupdate (push) Waiting to run
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-17 01:10:52 +03:00
dbcf6bb74a logger: use key names from opts (#271)
All checks were successful
/ autoupdate (push) Successful in 1m14s
Reviewed-on: #271
2023-10-17 01:05:55 +03:00
17698440ed Merge pull request 'logger: improvements' (#270) from logtrace into master
All checks were successful
/ autoupdate (push) Successful in 1m13s
Reviewed-on: #270
2023-10-17 01:00:20 +03:00
eb8851ab58 logger: improvements
Some checks failed
lint / lint (pull_request) Successful in 1m8s
pr / test (pull_request) Failing after 1m5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-17 01:00:00 +03:00
69c6d78920 Merge pull request 'logger: add ability to fill fileds from context' (#269) from logger-slog into master
All checks were successful
/ autoupdate (push) Successful in 1m17s
Reviewed-on: #269
2023-10-16 23:44:23 +03:00
7568779db4 logger: add ability to fill fileds from context
Some checks failed
lint / lint (pull_request) Successful in 1m1s
pr / test (pull_request) Failing after 1m7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-16 23:43:39 +03:00
5c3fa31ad2 logger: slog improve
All checks were successful
/ autoupdate (push) Successful in 1m16s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-16 19:37:38 +03:00
7b9a4924ad Merge pull request 'logger: slog logger' (#266) from slog into master
All checks were successful
/ autoupdate (push) Successful in 1m12s
Reviewed-on: #266
2023-10-14 18:05:46 +03:00
40939e56a3 fixup
All checks were successful
lint / lint (pull_request) Successful in 1m28s
pr / test (pull_request) Successful in 1m11s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-14 17:23:01 +03:00
0df97183f8 tracer: update wrapper
Some checks failed
lint / lint (pull_request) Successful in 1m13s
pr / test (pull_request) Failing after 59s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-14 16:58:40 +03:00
08c40ff7d5 update workflow
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 13:58:39 +03:00
afc6f88852 update workflow
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 13:49:04 +03:00
dbcc2bc262 update workflow
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 12:07:51 +03:00
ccffa5cd72 Merge pull request 'Bump github.com/google/uuid from v1.3.0 to v1.3.1' (#265) from pkgdash/go_modules/github.com/google/uuid-v1.3.1 into master
Some checks are pending
/ autoupdate (push) Has started running
Reviewed-on: #265
2023-10-03 00:39:35 +03:00
gitea-actions
3612f639d6 2023-10-02 21:38:45 +00:00
32d0b13f91 update workflow
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 00:32:44 +03:00
4ad61003b9 update workflow
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 00:32:33 +03:00
7647a0d3b3 update workflow
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 00:32:07 +03:00
c16961e852 update workflow
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 00:24:17 +03:00
968feb931c update workflow
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-03 00:15:55 +03:00
911a863313 fixup deps
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-02 02:14:34 +03:00
fcaf323b59 fixup deps
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-02 02:13:17 +03:00
52fecd1772 fixup deps
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-02 01:19:37 +03:00
d5f35fe88a fixup deps
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-02 01:16:31 +03:00
1952053c1b fixup deps
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-02 00:46:29 +03:00
75b45d7313 fixup deps
All checks were successful
/ autoupdate (push) Successful in 1m10s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-02 00:35:21 +03:00
pkgdash
eec350c897 2023-10-01 18:38:57 +00:00
pkgdash
360feb0f41 2023-10-01 18:38:48 +00:00
e1975ab66a fixup deps
Some checks failed
/ autoupdate (push) Failing after 1m12s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-01 21:32:00 +03:00
2504345815 fixup deps
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-01 21:28:47 +03:00
9a95fda93f fixup
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-01 16:55:47 +03:00
7ee309a4ae fixup
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-01 16:46:09 +03:00
a47b6b92c4 fixup
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-10-01 16:28:24 +03:00
8d46a6498d update
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 01:32:47 +03:00
65a1612b51 update
Some checks are pending
/ autoupdate (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 00:55:27 +03:00
26783b341b update
Some checks failed
/ autoupdate (push) Failing after 37s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 00:53:53 +03:00
ec21d4f307 update
Some checks failed
/ autoupdate (push) Failing after 53s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 00:51:52 +03:00
a3ca5ebb97 update
Some checks failed
/ autoupdate (push) Failing after 54s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 00:49:32 +03:00
f67ee26042 update
Some checks failed
/ autoupdate (push) Failing after 38s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 00:47:29 +03:00
6b7bba709e fixup autoupdate
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-30 00:27:56 +03:00
8b082b90df disable workflow
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-27 14:42:42 +03:00
b499fe99ca rename cli
All checks were successful
/ autoupdate (push) Successful in 40s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-27 14:27:21 +03:00
178e6e956d updates
Some checks failed
/ autoupdate (push) Failing after 38s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-23 23:43:16 +03:00
a52db1969b add autoupdate
Some checks failed
/ autoupdate (push) Failing after 36s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-23 13:35:04 +03:00
27bf1c3d84 add autoupdate
All checks were successful
/ autoupdate (push) Successful in 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-23 12:54:15 +03:00
58b687f134 add autoupdate
All checks were successful
/ test_schedule (push) Successful in 4s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-23 12:52:02 +03:00
8a354bd468 add autoupdate
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-22 23:31:14 +03:00
f600195ee2 go mod tidy
Some checks failed
lint / lint (pull_request) Has been cancelled
pr / test (pull_request) Has been cancelled
2023-09-18 10:59:46 +03:00
fd471a89e2 deleted unnecessary operations
Some checks failed
pr / test (pull_request) Has been cancelled
lint / lint (pull_request) Has been cancelled
2023-09-18 10:59:21 +03:00
c346ac43dd add google/slog logger
Some checks failed
lint / lint (pull_request) Has been cancelled
pr / test (pull_request) Has been cancelled
2023-09-16 14:20:02 +03:00
666176138b Merge pull request 'util/reflect: add Equal func with ability to skip some fields' (#243) from util-reflect into master
Reviewed-on: #243
2023-09-12 10:30:48 +03:00
884a96ad32 util/reflect: add Equal func with ability to skip some fields
Some checks failed
lint / lint (pull_request) Successful in 1m18s
pr / test (pull_request) Failing after 1m5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-12 10:30:19 +03:00
65d715c9da Merge pull request 'tracer: improve' (#242) from tracer into master
Reviewed-on: #242
2023-09-08 14:01:01 +03:00
247c7f6fa2 tracer: improve
Some checks failed
pr / test (pull_request) Failing after 2m43s
lint / lint (pull_request) Failing after 1m28s
2023-09-08 13:58:11 +03:00
f890f1d8b4 tracer: add missing UniqLabels function
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-01 15:48:27 +03:00
54b13bbded Merge pull request 'tracer: improve tracing wrapper' (#240) from trracer into master
Reviewed-on: #240
2023-09-01 15:25:45 +03:00
d3f9078bec tracer: improve tracing wrapper
Some checks failed
lint / lint (pull_request) Failing after 1m28s
pr / test (pull_request) Failing after 2m40s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-01 15:25:25 +03:00
fcc29f9eac tracer: update wrapper
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-09-01 08:49:45 +03:00
e724425ce2 Merge pull request 'fix some options' (#237) from fixes into master
Reviewed-on: #237
2023-08-15 07:55:44 +03:00
bc55c2fa6f fix some options
Some checks failed
lint / lint (pull_request) Failing after 1m29s
pr / test (pull_request) Failing after 2m39s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-08-15 07:55:26 +03:00
fdd1589122 Merge pull request 'move options to dedicated package' (#234) from options_rewrite into master
Reviewed-on: #234
2023-07-29 00:47:28 +03:00
6f6f850af6 move options to dedicated package
Some checks failed
lint / lint (pull_request) Failing after 1m31s
pr / test (pull_request) Failing after 2m37s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-07-29 00:40:58 +03:00
b1dbd99ce2 Merge pull request 'config/default: add micro:generate uuid/id' (#231) from config-default-gen into master
Reviewed-on: #231
2023-07-13 20:11:12 +03:00
6e24807edf config/default: add micro:generate uuid/id
Some checks failed
lint / lint (pull_request) Failing after 1m28s
pr / test (pull_request) Failing after 2m43s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-07-13 20:10:47 +03:00
654d8fa7e4 Merge pull request 'tracer: add Flush method' (#224) from tracerimp into master
Reviewed-on: #224
2023-07-04 00:24:09 +03:00
dd1a9cd25a tracer: add Flush method
Some checks failed
lint / lint (pull_request) Failing after 1m29s
pr / test (pull_request) Failing after 2m37s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-07-04 00:23:50 +03:00
d463eb20cb Merge pull request 'util/time: ParseDuration fix' (#221) from timefix into master
Reviewed-on: #221
2023-05-29 13:59:59 +03:00
8d5e25f8cf util/time: ParseDuration fix
Some checks failed
lint / lint (pull_request) Successful in 1m8s
pr / test (pull_request) Failing after 1m6s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-05-29 13:59:30 +03:00
27e8043fed Merge pull request 'util/time: fix duration parsing' (#220) from mastertime into master
Reviewed-on: #220
2023-05-27 23:57:27 +03:00
4e86df1721 util/time: fix duration parsing
Some checks failed
pr / test (pull_request) Failing after 1m6s
lint / lint (pull_request) Successful in 1m10s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-05-27 23:56:51 +03:00
03de1ec38f Merge pull request 'cleanup interfaces for v4' (#217) from v4 into master
Reviewed-on: #217
2023-05-09 20:04:40 +03:00
819ad1117a cleanup interfaces for v4
Some checks failed
lint / lint (pull_request) Successful in 1m9s
pr / test (pull_request) Failing after 1m4s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-05-09 20:04:15 +03:00
a2a383606d Merge pull request 'util/test: update test cases code' (#216) from testcase into master
Reviewed-on: #216
2023-04-28 07:10:40 +03:00
55ce58617b util/test: update test cases code
Some checks failed
lint / lint (pull_request) Successful in 51s
pr / test (pull_request) Failing after 52s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-28 07:10:19 +03:00
0e587d923e Merge pull request 'meter: move metrics handling in broker implementations' (#215) from metrics into master
Reviewed-on: #215
2023-04-27 15:32:56 +03:00
fa0248c80c cleanup
All checks were successful
pr / test (pull_request) Successful in 50s
lint / lint (pull_request) Successful in 49s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-27 15:31:59 +03:00
054bd02b59 meter: move metrics handling in broker implementations
All checks were successful
lint / lint (pull_request) Successful in 1m4s
pr / test (pull_request) Successful in 50s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-27 15:30:55 +03:00
0cf246d2d6 Merge pull request 'util/io: add RedirectStderr' (#214) from io-redirect into master
Reviewed-on: #214
2023-04-24 12:59:31 +03:00
af278bd7d3 util/io: add RedirectStderr
All checks were successful
lint / lint (pull_request) Successful in 46s
pr / test (pull_request) Successful in 50s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-24 12:58:05 +03:00
814b90efe5 Merge pull request 'util/test: export GetCases func' (#213) from GetCases into master
Reviewed-on: #213
2023-04-19 01:23:53 +03:00
e403ae3d8e util/test: export GetCases func
All checks were successful
lint
test
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-19 01:23:34 +03:00
c9816a3957 Merge pull request 'util/test: add helper funcs' (#212) from test into master
Reviewed-on: #212
2023-04-19 00:33:28 +03:00
5691238a6a util/test: add helper funcs
All checks were successful
lint
test
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-18 23:47:12 +03:00
963a0fa7b7 Merge pull request 'gofmt -s code' (#209) from gofmt into master
Reviewed-on: #209
2023-04-11 23:34:41 +03:00
485257035c gofmt -s code
Some checks failed
lint
test
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-11 23:32:58 +03:00
ebd8ddf05b move to v4
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2023-04-11 22:21:25 +03:00
225 changed files with 4928 additions and 32748 deletions

3
.gitea/pkgdashcli.yaml Normal file
View File

@@ -0,0 +1,3 @@
branches:
- master
- v3

View File

@@ -0,0 +1,24 @@
on:
push:
branches:
- 'main'
- 'master'
- 'v3'
schedule:
- cron: '* * * * *'
#- cron: '@hourly'
jobs:
autoupdate:
runs-on: ubuntu-latest
steps:
- name: setup-go
uses: https://gitea.com/actions/setup-go@v3
with:
go-version: 1.21
- name: checkout
uses: https://gitea.com/actions/checkout@v3
- name: get pkgdashcli
run: GOPROXY=direct GONOSUMDB="git.unistack.org/*" GONOPROXY="git.unistack.org/*" GOBIN=/bin go install git.unistack.org/unistack-org/pkgdash/cmd/pkgdashcli@latest
- name: pkgdashcli check
run: /bin/pkgdashcli check

24
.gitea/workflows/lint.yml Normal file
View File

@@ -0,0 +1,24 @@
name: lint
on:
pull_request:
branches:
- master
- v3
jobs:
lint:
name: lint
runs-on: ubuntu-latest
steps:
- name: setup-go
uses: https://gitea.com/actions/setup-go@v3
with:
go-version: 1.21
- name: checkout
uses: https://gitea.com/actions/checkout@v3
- name: deps
run: go get -v -d ./...
- name: lint
uses: https://github.com/golangci/golangci-lint-action@v3.4.0
continue-on-error: true
with:
version: v1.52

23
.gitea/workflows/pr.yml Normal file
View File

@@ -0,0 +1,23 @@
name: pr
on:
pull_request:
branches:
- master
- v3
jobs:
test:
name: test
runs-on: ubuntu-latest
steps:
- name: checkout
uses: https://gitea.com/actions/checkout@v3
- name: setup-go
uses: https://gitea.com/actions/setup-go@v3
with:
go-version: 1.21
- name: deps
run: go get -v -t -d ./...
- name: test
env:
INTEGRATION_TESTS: yes
run: go test -mod readonly -v ./...

View File

@@ -1,53 +0,0 @@
name: coverage
on:
push:
branches: [ main, v3, v4 ]
paths-ignore:
- '.github/**'
- '.gitea/**'
pull_request:
branches: [ main, v3, v4 ]
jobs:
build:
if: github.server_url != 'https://github.com'
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v4
with:
filter: 'blob:none'
- name: setup go
uses: actions/setup-go@v5
with:
cache-dependency-path: "**/*.sum"
go-version: 'stable'
- name: test coverage
run: |
go test -v -cover ./... -covermode=count -coverprofile coverage.out -coverpkg ./...
go tool cover -func coverage.out -o coverage.out
- name: coverage badge
uses: tj-actions/coverage-badge-go@v2
with:
green: 80
filename: coverage.out
- uses: stefanzweifel/git-auto-commit-action@v4
name: autocommit
with:
commit_message: Apply Code Coverage Badge
skip_fetch: false
skip_checkout: false
file_pattern: ./README.md
- name: push
if: steps.auto-commit-action.outputs.changes_detected == 'true'
uses: ad-m/github-push-action@master
with:
github_token: ${{ github.token }}
branch: ${{ github.ref }}

View File

@@ -1,29 +0,0 @@
name: lint
on:
pull_request:
types: [opened, reopened, synchronize]
branches: [ master, v3, v4 ]
paths-ignore:
- '.github/**'
- '.gitea/**'
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v4
with:
filter: 'blob:none'
- name: setup go
uses: actions/setup-go@v5
with:
cache-dependency-path: "**/*.sum"
go-version: 'stable'
- name: setup deps
run: go get -v ./...
- name: run lint
uses: golangci/golangci-lint-action@v6
with:
version: 'latest'

View File

@@ -1,94 +0,0 @@
name: sync
on:
schedule:
- cron: '*/5 * * * *'
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
sync:
if: github.server_url != 'https://github.com'
runs-on: ubuntu-latest
steps:
- name: init
run: |
git config --global user.email "vtolstov <vtolstov@users.noreply.github.com>"
git config --global user.name "github-actions[bot]"
echo "machine git.unistack.org login vtolstov password ${{ secrets.TOKEN_GITEA }}" >> /root/.netrc
echo "machine github.com login vtolstov password ${{ secrets.TOKEN_GITHUB }}" >> /root/.netrc
- name: check master
id: check_master
run: |
src_hash=$(git ls-remote https://github.com/${GITHUB_REPOSITORY} refs/heads/master | cut -f1)
dst_hash=$(git ls-remote ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} refs/heads/master | cut -f1)
echo "src_hash=$src_hash"
echo "dst_hash=$dst_hash"
if [ "$src_hash" != "$dst_hash" ]; then
echo "sync_needed=true" >> $GITHUB_OUTPUT
else
echo "sync_needed=false" >> $GITHUB_OUTPUT
fi
- name: sync master
if: steps.check_master.outputs.sync_needed == 'true'
run: |
git clone --filter=blob:none --filter=tree:0 --branch master --single-branch ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} repo
cd repo
git remote add --no-tags --fetch --track master upstream https://github.com/${GITHUB_REPOSITORY}
git pull --rebase upstream master
git push upstream master --progress
git push origin master --progress
cd ../
rm -rf repo
- name: check v3
id: check_v3
run: |
src_hash=$(git ls-remote https://github.com/${GITHUB_REPOSITORY} refs/heads/v3 | cut -f1)
dst_hash=$(git ls-remote ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} refs/heads/v3 | cut -f1)
echo "src_hash=$src_hash"
echo "dst_hash=$dst_hash"
if [ "$src_hash" != "$dst_hash" ]; then
echo "sync_needed=true" >> $GITHUB_OUTPUT
else
echo "sync_needed=false" >> $GITHUB_OUTPUT
fi
- name: sync v3
if: steps.check_v3.outputs.sync_needed == 'true'
run: |
git clone --filter=blob:none --filter=tree:0 --branch v3 --single-branch ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} repo
cd repo
git remote add --no-tags --fetch --track v3 upstream https://github.com/${GITHUB_REPOSITORY}
git pull --rebase upstream v3
git push upstream v3 --progress
git push origin v3 --progress
cd ../
rm -rf repo
- name: check v4
id: check_v4
run: |
src_hash=$(git ls-remote https://github.com/${GITHUB_REPOSITORY} refs/heads/v4 | cut -f1)
dst_hash=$(git ls-remote ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} refs/heads/v4 | cut -f1)
echo "src_hash=$src_hash"
echo "dst_hash=$dst_hash"
if [ "$src_hash" != "$dst_hash" ]; then
echo "sync_needed=true" >> $GITHUB_OUTPUT
else
echo "sync_needed=false" >> $GITHUB_OUTPUT
fi
- name: sync v4
if: steps.check_v4.outputs.sync_needed == 'true'
run: |
git clone --filter=blob:none --filter=tree:0 --branch v4 --single-branch ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} repo
cd repo
git remote add --no-tags --fetch --track v4 upstream https://github.com/${GITHUB_REPOSITORY}
git pull --rebase upstream v4
git push upstream v4 --progress
git push origin v4 --progress
cd ../
rm -rf repo

View File

@@ -1,31 +0,0 @@
name: test
on:
pull_request:
types: [opened, reopened, synchronize]
branches: [ master, v3, v4 ]
push:
branches: [ master, v3, v4 ]
paths-ignore:
- '.github/**'
- '.gitea/**'
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v4
with:
filter: 'blob:none'
- name: setup go
uses: actions/setup-go@v5
with:
cache-dependency-path: "**/*.sum"
go-version: 'stable'
- name: setup deps
run: go get -v ./...
- name: run test
env:
INTEGRATION_TESTS: yes
run: go test -mod readonly -v ./...

View File

@@ -1,50 +0,0 @@
name: test
on:
pull_request:
types: [opened, reopened, synchronize]
branches: [ master, v3, v4 ]
push:
branches: [ master, v3, v4 ]
paths-ignore:
- '.github/**'
- '.gitea/**'
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v4
with:
filter: 'blob:none'
- name: checkout tests
uses: actions/checkout@v4
with:
ref: master
filter: 'blob:none'
repository: unistack-org/micro-tests
path: micro-tests
- name: setup go
uses: actions/setup-go@v5
with:
cache-dependency-path: "**/*.sum"
go-version: 'stable'
- name: setup go work
env:
GOWORK: ${{ github.workspace }}/go.work
run: |
go work init
go work use .
go work use micro-tests
- name: setup deps
env:
GOWORK: ${{ github.workspace }}/go.work
run: go get -v ./...
- name: run tests
env:
INTEGRATION_TESTS: yes
GOWORK: ${{ github.workspace }}/go.work
run: |
cd micro-tests
go test -mod readonly -v ./... || true

3
.gitignore vendored
View File

@@ -1,8 +1,6 @@
# Develop tools # Develop tools
/.vscode/ /.vscode/
/.idea/ /.idea/
.idea
.vscode
# Binaries for programs and plugins # Binaries for programs and plugins
*.exe *.exe
@@ -15,7 +13,6 @@
_obj _obj
_test _test
_build _build
.DS_Store
# Architecture specific extensions/prefixes # Architecture specific extensions/prefixes
*.[568vq] *.[568vq]

View File

@@ -1,5 +1,44 @@
run: run:
concurrency: 8 concurrency: 4
timeout: 5m deadline: 5m
issues-exit-code: 1 issues-exit-code: 1
tests: true tests: true
linters-settings:
govet:
check-shadowing: true
enable:
- fieldalignment
linters:
enable:
- govet
- deadcode
- errcheck
- govet
- ineffassign
- staticcheck
- structcheck
- typecheck
- unused
- varcheck
- bodyclose
- gci
- goconst
- gocritic
- gosimple
- gofmt
- gofumpt
- goimports
- revive
- gosec
- makezero
- misspell
- nakedret
- nestif
- nilerr
- noctx
- prealloc
- unconvert
- unparam
disable-all: false

View File

@@ -1,9 +1,4 @@
# Micro # Micro [![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Doc](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go.unistack.org/micro/v4?tab=overview) [![Status](https://github.com/unistack-org/micro/workflows/build/badge.svg?branch=master)](https://github.com/unistack-org/micro/actions?query=workflow%3Abuild+branch%3Amaster+event%3Apush) [![Lint](https://goreportcard.com/badge/go.unistack.org/micro/v4)](https://goreportcard.com/report/go.unistack.org/micro/v4) [![Coverage](https://codecov.io/gh/unistack-org/micro/branch/v4/graph/badge.svg?token=OZPO2LP7VS)](https://codecov.io/gh/unistack-org/micro)
![Coverage](https://img.shields.io/badge/Coverage-33.7%25-yellow)
[![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Doc](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go.unistack.org/micro/v4?tab=overview)
[![Status](https://git.unistack.org/unistack-org/micro/actions/workflows/job_tests.yml/badge.svg?branch=v4)](https://git.unistack.org/unistack-org/micro/actions?query=workflow%3Abuild+branch%3Av4+event%3Apush)
[![Lint](https://goreportcard.com/badge/go.unistack.org/micro/v4)](https://goreportcard.com/report/go.unistack.org/micro/v4)
Micro is a standard library for microservices. Micro is a standard library for microservices.
@@ -15,20 +10,30 @@ Micro provides the core requirements for distributed systems development includi
Micro abstracts away the details of distributed systems. Here are the main features. Micro abstracts away the details of distributed systems. Here are the main features.
- **Authentication** - Auth is built in as a first class citizen. Authentication and authorization enable secure
zero trust networking by providing every service an identity and certificates. This additionally includes rule
based access control.
- **Dynamic Config** - Load and hot reload dynamic config from anywhere. The config interface provides a way to load application - **Dynamic Config** - Load and hot reload dynamic config from anywhere. The config interface provides a way to load application
level config from any source such as env vars, cmdline, file, consul, vault... You can merge the sources and even define fallbacks. level config from any source such as env vars, file, etcd. You can merge the sources and even define fallbacks.
- **Data Storage** - A simple data store interface to read, write and delete records. It includes support for memory, file and - **Data Storage** - A simple data store interface to read, write and delete records. It includes support for memory, file and
s3. State and persistence becomes a core requirement beyond prototyping and Micro looks to build that into the framework. CockroachDB by default. State and persistence becomes a core requirement beyond prototyping and Micro looks to build that into the framework.
- **Service Discovery** - Automatic service registration and name resolution. Service discovery is at the core of micro service - **Service Discovery** - Automatic service registration and name resolution. Service discovery is at the core of micro service
development. When service A needs to speak to service B it needs the location of that service. development. When service A needs to speak to service B it needs the location of that service.
- **Load Balancing** - Client side load balancing built on service discovery. Once we have the addresses of any number of instances
of a service we now need a way to decide which node to route to. We use random hashed load balancing to provide even distribution
across the services and retry a different node if there's a problem.
- **Message Encoding** - Dynamic message encoding based on content-type. The client and server will use codecs along with content-type - **Message Encoding** - Dynamic message encoding based on content-type. The client and server will use codecs along with content-type
to seamlessly encode and decode Go types for you. Any variety of messages could be encoded and sent from different clients. The client to seamlessly encode and decode Go types for you. Any variety of messages could be encoded and sent from different clients. The client
and server handle this by default. and server handle this by default.
- **Async Messaging** - Pub/Sub is built in as a first class citizen for asynchronous communication and event driven architectures. - **Transport** - gRPC or http based request/response with support for bidirectional streaming. We provide an abstraction for synchronous communication. A request made to a service will be automatically resolved, load balanced, dialled and streamed.
- **Async Messaging** - PubSub is built in as a first class citizen for asynchronous communication and event driven architectures.
Event notifications are a core pattern in micro service development. Event notifications are a core pattern in micro service development.
- **Synchronization** - Distributed systems are often built in an eventually consistent manner. Support for distributed locking and - **Synchronization** - Distributed systems are often built in an eventually consistent manner. Support for distributed locking and
@@ -37,6 +42,10 @@ leadership are built in as a Sync interface. When using an eventually consistent
- **Pluggable Interfaces** - Micro makes use of Go interfaces for each system abstraction. Because of this these interfaces - **Pluggable Interfaces** - Micro makes use of Go interfaces for each system abstraction. Because of this these interfaces
are pluggable and allows Micro to be runtime agnostic. are pluggable and allows Micro to be runtime agnostic.
## Getting Started
To be created.
## License ## License
Micro is Apache 2.0 licensed. Micro is Apache 2.0 licensed.

15
SECURITY.md Normal file
View File

@@ -0,0 +1,15 @@
# Security Policy
## Supported Versions
Use this section to tell people about which versions of your project are
currently being supported with security updates.
| Version | Supported |
| ------- | ------------------ |
| 3.7.x | :white_check_mark: |
| < 3.7.0 | :x: |
## Reporting a Vulnerability
If you find any issue, please create github issue in this repo

View File

@@ -1,13 +1,12 @@
// Package broker is an interface used for asynchronous messaging // Package broker is an interface used for asynchronous messaging
package broker package broker // import "go.unistack.org/micro/v4/broker"
import ( import (
"context" "context"
"errors" "errors"
"time"
"go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/metadata" "go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/options"
) )
// DefaultBroker default memory broker // DefaultBroker default memory broker
@@ -18,12 +17,8 @@ var (
ErrNotConnected = errors.New("broker not connected") ErrNotConnected = errors.New("broker not connected")
// ErrDisconnected returns when broker disconnected // ErrDisconnected returns when broker disconnected
ErrDisconnected = errors.New("broker disconnected") ErrDisconnected = errors.New("broker disconnected")
// ErrInvalidMessage returns when invalid Message passed // ErrInvalidMessage returns when message has nvalid format
ErrInvalidMessage = errors.New("invalid message") ErrInvalidMessage = errors.New("broker message has invalid format")
// ErrInvalidHandler returns when subscriber passed to Subscribe
ErrInvalidHandler = errors.New("invalid handler, ony func(Message) error and func([]Message) error supported")
// DefaultGracefulTimeout
DefaultGracefulTimeout = 5 * time.Second
) )
// Broker is an interface used for asynchronous messaging. // Broker is an interface used for asynchronous messaging.
@@ -31,7 +26,7 @@ type Broker interface {
// Name returns broker instance name // Name returns broker instance name
Name() string Name() string
// Init initilize broker // Init initilize broker
Init(opts ...Option) error Init(opts ...options.Option) error
// Options returns broker options // Options returns broker options
Options() Options Options() Options
// Address return configured address // Address return configured address
@@ -40,44 +35,29 @@ type Broker interface {
Connect(ctx context.Context) error Connect(ctx context.Context) error
// Disconnect disconnect from broker // Disconnect disconnect from broker
Disconnect(ctx context.Context) error Disconnect(ctx context.Context) error
// NewMessage create new broker message to publish. // Publish message, msg can be single broker.Message or []broker.Message
NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...PublishOption) (Message, error) Publish(ctx context.Context, msg interface{}, opts ...options.Option) error
// Publish message to broker topic
Publish(ctx context.Context, topic string, messages ...Message) error
// Subscribe subscribes to topic message via handler // Subscribe subscribes to topic message via handler
Subscribe(ctx context.Context, topic string, handler interface{}, opts ...SubscribeOption) (Subscriber, error) Subscribe(ctx context.Context, topic string, handler interface{}, opts ...options.Option) (Subscriber, error)
// String type of broker // String type of broker
String() string String() string
// Live returns broker liveness
Live() bool
// Ready returns broker readiness
Ready() bool
// Health returns broker health
Health() bool
} }
type (
FuncPublish func(ctx context.Context, topic string, messages ...Message) error
HookPublish func(next FuncPublish) FuncPublish
FuncSubscribe func(ctx context.Context, topic string, handler interface{}, opts ...SubscribeOption) (Subscriber, error)
HookSubscribe func(next FuncSubscribe) FuncSubscribe
)
// Message is given to a subscription handler for processing // Message is given to a subscription handler for processing
type Message interface { type Message interface {
// Context for the message. // Context for the message
Context() context.Context Context() context.Context
// Topic returns message destination topic. // Topic
Topic() string Topic() string
// Header returns message headers. // Header returns message headers
Header() metadata.Metadata Header() metadata.Metadata
// Body returns broker message []byte slice // Body returns broker message may be []byte slice or some go struct
Body() []byte Body() interface{}
// Unmarshal try to decode message body to dst. // Ack acknowledge message
// This is helper method that uses codec.Unmarshal.
Unmarshal(dst interface{}, opts ...codec.Option) error
// Ack acknowledge message if supported.
Ack() error Ack() error
// Error returns message error (like decoding errors or some other)
// In this case Body contains raw []byte from broker
Error() error
} }
// Subscriber is a convenience return type for the Subscribe method // Subscriber is a convenience return type for the Subscribe method
@@ -89,3 +69,9 @@ type Subscriber interface {
// Unsubscribe from topic // Unsubscribe from topic
Unsubscribe(ctx context.Context) error Unsubscribe(ctx context.Context) error
} }
// MessageHandler func signature for single message processing
type MessageHandler func(Message) error
// MessagesHandler func signature for batch message processing
type MessagesHandler func([]Message) error

View File

@@ -15,15 +15,6 @@ func FromContext(ctx context.Context) (Broker, bool) {
return c, ok return c, ok
} }
// MustContext returns broker from passed context
func MustContext(ctx context.Context) Broker {
b, ok := FromContext(ctx)
if !ok {
panic("missing broker")
}
return b
}
// NewContext savess broker in context // NewContext savess broker in context
func NewContext(ctx context.Context, s Broker) context.Context { func NewContext(ctx context.Context, s Broker) context.Context {
if ctx == nil { if ctx == nil {
@@ -31,33 +22,3 @@ func NewContext(ctx context.Context, s Broker) context.Context {
} }
return context.WithValue(ctx, brokerKey{}, s) return context.WithValue(ctx, brokerKey{}, s)
} }
// SetSubscribeOption returns a function to setup a context with given value
func SetSubscribeOption(k, v interface{}) SubscribeOption {
return func(o *SubscribeOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetPublishOption returns a function to setup a context with given value
func SetPublishOption(k, v interface{}) PublishOption {
return func(o *PublishOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option {
return func(o *Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}

View File

@@ -37,25 +37,3 @@ func TestNewNilContext(t *testing.T) {
t.Fatal("NewContext not works") t.Fatal("NewContext not works")
} }
} }
func TestSetSubscribeOption(t *testing.T) {
type key struct{}
o := SetSubscribeOption(key{}, "test")
opts := &SubscribeOptions{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetSubscribeOption not works")
}
}
func TestSetOption(t *testing.T) {
type key struct{}
o := SetOption(key{}, "test")
opts := &Options{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetOption not works")
}
}

334
broker/memory.go Normal file
View File

@@ -0,0 +1,334 @@
package broker
import (
"context"
"fmt"
"sync"
"time"
"go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/semconv"
maddr "go.unistack.org/micro/v4/util/addr"
"go.unistack.org/micro/v4/util/id"
mnet "go.unistack.org/micro/v4/util/net"
"go.unistack.org/micro/v4/util/rand"
)
type MemoryBroker struct {
subscribers map[string][]*memorySubscriber
addr string
opts Options
sync.RWMutex
connected bool
}
func (m *MemoryBroker) Options() Options {
return m.opts
}
func (m *MemoryBroker) Address() string {
return m.addr
}
func (m *MemoryBroker) Connect(ctx context.Context) error {
m.Lock()
defer m.Unlock()
if m.connected {
return nil
}
// use 127.0.0.1 to avoid scan of all network interfaces
addr, err := maddr.Extract("127.0.0.1")
if err != nil {
return err
}
var rng rand.Rand
i := rng.Intn(20000)
// set addr with port
addr = mnet.HostPort(addr, 10000+i)
m.addr = addr
m.connected = true
return nil
}
func (m *MemoryBroker) Disconnect(ctx context.Context) error {
m.Lock()
defer m.Unlock()
select {
case <-ctx.Done():
return ctx.Err()
default:
if m.connected {
m.connected = false
}
}
return nil
}
func (m *MemoryBroker) Init(opts ...options.Option) error {
var err error
for _, o := range opts {
if err = o(&m.opts); err != nil {
return err
}
}
return nil
}
func (m *MemoryBroker) Publish(ctx context.Context, message interface{}, opts ...options.Option) error {
m.RLock()
if !m.connected {
m.RUnlock()
return ErrNotConnected
}
m.RUnlock()
var err error
select {
case <-ctx.Done():
return ctx.Err()
default:
options := NewPublishOptions(opts...)
var msgs []Message
switch v := message.(type) {
case []Message:
msgs = v
case Message:
msgs = append(msgs, v)
default:
return ErrInvalidMessage
}
msgTopicMap := make(map[string][]*memoryMessage)
for _, msg := range msgs {
p := &memoryMessage{opts: options}
p.topic, _ = msg.Header().Get(metadata.HeaderTopic)
if v, ok := msg.Body().(*codec.Frame); ok {
p.body = msg.Body()
} else if len(m.opts.Codecs) == 0 {
p.body = msg.Body()
} else {
cf, ok := m.opts.Codecs[options.ContentType]
if !ok {
return fmt.Errorf("%s: %s", codec.ErrUnknownContentType, options.ContentType)
}
p.body, err = cf.Marshal(v)
if err != nil {
return err
}
}
msgTopicMap[p.topic] = append(msgTopicMap[p.topic], p)
}
eh := m.opts.ErrorHandler
for t, ms := range msgTopicMap {
ts := time.Now()
m.opts.Meter.Counter(semconv.PublishMessageInflight, "endpoint", t).Add(len(ms))
m.opts.Meter.Counter(semconv.SubscribeMessageInflight, "endpoint", t).Add(len(ms))
m.RLock()
subs, ok := m.subscribers[t]
m.RUnlock()
if !ok {
m.opts.Meter.Counter(semconv.PublishMessageTotal, "endpoint", t, "status", "failure").Add(len(ms))
m.opts.Meter.Counter(semconv.PublishMessageInflight, "endpoint", t).Add(-len(ms))
m.opts.Meter.Counter(semconv.SubscribeMessageInflight, "endpoint", t).Add(-len(ms))
continue
}
m.opts.Meter.Counter(semconv.PublishMessageTotal, "endpoint", t, "status", "success").Add(len(ms))
for _, sub := range subs {
if sub.opts.ErrorHandler != nil {
eh = sub.opts.ErrorHandler
}
switch mh := sub.handler.(type) {
case MessagesHandler:
mhs := make([]Message, 0, len(ms))
for _, m := range ms {
mhs = append(mhs, m)
}
if err = mh(mhs); err != nil {
m.opts.Meter.Counter(semconv.SubscribeMessageTotal, "endpoint", t, "status", "failure").Add(len(ms))
if eh != nil {
switch meh := eh.(type) {
case MessagesHandler:
_ = meh(mhs)
case MessageHandler:
for _, me := range mhs {
_ = meh(me)
}
}
} else if m.opts.Logger.V(logger.ErrorLevel) {
m.opts.Logger.Error(m.opts.Context, err.Error())
}
}
case MessageHandler:
for _, p := range ms {
if err = mh(p); err != nil {
m.opts.Meter.Counter(semconv.SubscribeMessageTotal, "endpoint", t, "status", "failure").Inc()
if eh != nil {
switch meh := eh.(type) {
case MessageHandler:
_ = meh(p)
case MessagesHandler:
_ = meh([]Message{p})
}
} else if m.opts.Logger.V(logger.ErrorLevel) {
m.opts.Logger.Error(m.opts.Context, err.Error())
}
} else {
if sub.opts.AutoAck {
if err = p.Ack(); err != nil {
m.opts.Logger.Error(m.opts.Context, "ack failed: "+err.Error())
m.opts.Meter.Counter(semconv.SubscribeMessageTotal, "endpoint", t, "status", "failure").Inc()
} else {
m.opts.Meter.Counter(semconv.SubscribeMessageTotal, "endpoint", t, "status", "success").Inc()
}
} else {
m.opts.Meter.Counter(semconv.SubscribeMessageTotal, "endpoint", t, "status", "success").Inc()
}
}
m.opts.Meter.Counter(semconv.PublishMessageInflight, "endpoint", t).Add(-1)
m.opts.Meter.Counter(semconv.SubscribeMessageInflight, "endpoint", t).Add(-1)
}
}
}
te := time.Since(ts)
m.opts.Meter.Summary(semconv.PublishMessageLatencyMicroseconds, "endpoint", t).Update(te.Seconds())
m.opts.Meter.Histogram(semconv.PublishMessageDurationSeconds, "endpoint", t).Update(te.Seconds())
m.opts.Meter.Summary(semconv.SubscribeMessageLatencyMicroseconds, "endpoint", t).Update(te.Seconds())
m.opts.Meter.Histogram(semconv.SubscribeMessageDurationSeconds, "endpoint", t).Update(te.Seconds())
}
}
return nil
}
func (m *MemoryBroker) Subscribe(ctx context.Context, topic string, handler interface{}, opts ...options.Option) (Subscriber, error) {
m.RLock()
if !m.connected {
m.RUnlock()
return nil, ErrNotConnected
}
m.RUnlock()
sid, err := id.New()
if err != nil {
return nil, err
}
options := NewSubscribeOptions(opts...)
sub := &memorySubscriber{
exit: make(chan bool, 1),
id: sid,
topic: topic,
handler: handler,
opts: options,
ctx: ctx,
}
m.Lock()
m.subscribers[topic] = append(m.subscribers[topic], sub)
m.Unlock()
go func() {
<-sub.exit
m.Lock()
newSubscribers := make([]*memorySubscriber, 0, len(m.subscribers)-1)
for _, sb := range m.subscribers[topic] {
if sb.id == sub.id {
continue
}
newSubscribers = append(newSubscribers, sb)
}
m.subscribers[topic] = newSubscribers
m.Unlock()
}()
return sub, nil
}
func (m *MemoryBroker) String() string {
return "memory"
}
func (m *MemoryBroker) Name() string {
return m.opts.Name
}
type memoryMessage struct {
err error
body interface{}
topic string
header metadata.Metadata
opts PublishOptions
ctx context.Context
}
func (m *memoryMessage) Topic() string {
return m.topic
}
func (m *memoryMessage) Header() metadata.Metadata {
return m.header
}
func (m *memoryMessage) Body() interface{} {
return m.body
}
func (m *memoryMessage) Ack() error {
return nil
}
func (m *memoryMessage) Error() error {
return m.err
}
func (m *memoryMessage) Context() context.Context {
return m.ctx
}
type memorySubscriber struct {
ctx context.Context
exit chan bool
handler interface{}
id string
topic string
opts SubscribeOptions
}
func (m *memorySubscriber) Options() SubscribeOptions {
return m.opts
}
func (m *memorySubscriber) Topic() string {
return m.topic
}
func (m *memorySubscriber) Unsubscribe(ctx context.Context) error {
m.exit <- true
return nil
}
// NewBroker return new memory broker
func NewBroker(opts ...options.Option) *MemoryBroker {
return &MemoryBroker{
opts: NewOptions(opts...),
subscribers: make(map[string][]*memorySubscriber),
}
}

View File

@@ -1,341 +0,0 @@
package broker
import (
"context"
"strings"
"sync"
"go.unistack.org/micro/v4/broker"
"go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/options"
maddr "go.unistack.org/micro/v4/util/addr"
"go.unistack.org/micro/v4/util/id"
mnet "go.unistack.org/micro/v4/util/net"
"go.unistack.org/micro/v4/util/rand"
)
type Broker struct {
funcPublish broker.FuncPublish
funcSubscribe broker.FuncSubscribe
subscribers map[string][]*Subscriber
addr string
opts broker.Options
sync.RWMutex
connected bool
}
type memoryMessage struct {
c codec.Codec
topic string
ctx context.Context
body []byte
hdr metadata.Metadata
opts broker.PublishOptions
}
func (m *memoryMessage) Ack() error {
return nil
}
func (m *memoryMessage) Body() []byte {
return m.body
}
func (m *memoryMessage) Header() metadata.Metadata {
return m.hdr
}
func (m *memoryMessage) Context() context.Context {
return m.ctx
}
func (m *memoryMessage) Topic() string {
return ""
}
func (m *memoryMessage) Unmarshal(dst interface{}, opts ...codec.Option) error {
return m.c.Unmarshal(m.body, dst)
}
type Subscriber struct {
ctx context.Context
exit chan bool
handler interface{}
id string
topic string
opts broker.SubscribeOptions
}
func (b *Broker) newCodec(ct string) (codec.Codec, error) {
if idx := strings.IndexRune(ct, ';'); idx >= 0 {
ct = ct[:idx]
}
b.RLock()
c, ok := b.opts.Codecs[ct]
b.RUnlock()
if ok {
return c, nil
}
return nil, codec.ErrUnknownContentType
}
func (b *Broker) Options() broker.Options {
return b.opts
}
func (b *Broker) Address() string {
return b.addr
}
func (b *Broker) Connect(ctx context.Context) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
b.Lock()
defer b.Unlock()
if b.connected {
return nil
}
// use 127.0.0.1 to avoid scan of all network interfaces
addr, err := maddr.Extract("127.0.0.1")
if err != nil {
return err
}
var rng rand.Rand
i := rng.Intn(20000)
// set addr with port
addr = mnet.HostPort(addr, 10000+i)
b.addr = addr
b.connected = true
return nil
}
func (b *Broker) Disconnect(ctx context.Context) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
b.Lock()
defer b.Unlock()
if !b.connected {
return nil
}
b.connected = false
return nil
}
func (b *Broker) Init(opts ...broker.Option) error {
for _, o := range opts {
o(&b.opts)
}
b.funcPublish = b.fnPublish
b.funcSubscribe = b.fnSubscribe
b.opts.Hooks.EachPrev(func(hook options.Hook) {
switch h := hook.(type) {
case broker.HookPublish:
b.funcPublish = h(b.funcPublish)
case broker.HookSubscribe:
b.funcSubscribe = h(b.funcSubscribe)
}
})
return nil
}
func (b *Broker) NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...broker.PublishOption) (broker.Message, error) {
options := broker.NewPublishOptions(opts...)
if options.ContentType == "" {
options.ContentType = b.opts.ContentType
}
m := &memoryMessage{ctx: ctx, hdr: hdr, opts: options}
c, err := b.newCodec(m.opts.ContentType)
if err == nil {
m.body, err = c.Marshal(body)
}
if err != nil {
return nil, err
}
return m, nil
}
func (b *Broker) Publish(ctx context.Context, topic string, messages ...broker.Message) error {
return b.funcPublish(ctx, topic, messages...)
}
func (b *Broker) fnPublish(ctx context.Context, topic string, messages ...broker.Message) error {
return b.publish(ctx, topic, messages...)
}
func (b *Broker) publish(ctx context.Context, topic string, messages ...broker.Message) error {
b.RLock()
if !b.connected {
b.RUnlock()
return broker.ErrNotConnected
}
b.RUnlock()
select {
case <-ctx.Done():
return ctx.Err()
default:
}
b.RLock()
subs, ok := b.subscribers[topic]
b.RUnlock()
if !ok {
return nil
}
var err error
for _, sub := range subs {
switch s := sub.handler.(type) {
default:
if b.opts.Logger.V(logger.ErrorLevel) {
b.opts.Logger.Error(ctx, "broker handler error", broker.ErrInvalidHandler)
}
case func(broker.Message) error:
for _, message := range messages {
msg, ok := message.(*memoryMessage)
if !ok {
if b.opts.Logger.V(logger.ErrorLevel) {
b.opts.Logger.Error(ctx, "broker handler error", broker.ErrInvalidMessage)
}
}
msg.topic = topic
if err = s(msg); err == nil && sub.opts.AutoAck {
err = msg.Ack()
}
if err != nil {
if b.opts.Logger.V(logger.ErrorLevel) {
b.opts.Logger.Error(ctx, "broker handler error", err)
}
}
}
case func([]broker.Message) error:
if err = s(messages); err == nil && sub.opts.AutoAck {
for _, message := range messages {
err = message.Ack()
if err != nil {
if b.opts.Logger.V(logger.ErrorLevel) {
b.opts.Logger.Error(ctx, "broker handler error", err)
}
}
}
}
}
}
return nil
}
func (b *Broker) Subscribe(ctx context.Context, topic string, handler interface{}, opts ...broker.SubscribeOption) (broker.Subscriber, error) {
return b.funcSubscribe(ctx, topic, handler, opts...)
}
func (b *Broker) fnSubscribe(ctx context.Context, topic string, handler interface{}, opts ...broker.SubscribeOption) (broker.Subscriber, error) {
if err := broker.IsValidHandler(handler); err != nil {
return nil, err
}
b.RLock()
if !b.connected {
b.RUnlock()
return nil, broker.ErrNotConnected
}
b.RUnlock()
sid, err := id.New()
if err != nil {
return nil, err
}
options := broker.NewSubscribeOptions(opts...)
sub := &Subscriber{
exit: make(chan bool, 1),
id: sid,
topic: topic,
handler: handler,
opts: options,
ctx: ctx,
}
b.Lock()
b.subscribers[topic] = append(b.subscribers[topic], sub)
b.Unlock()
go func() {
<-sub.exit
b.Lock()
newSubscribers := make([]*Subscriber, 0, len(b.subscribers)-1)
for _, sb := range b.subscribers[topic] {
if sb.id == sub.id {
continue
}
newSubscribers = append(newSubscribers, sb)
}
b.subscribers[topic] = newSubscribers
b.Unlock()
}()
return sub, nil
}
func (b *Broker) String() string {
return "memory"
}
func (b *Broker) Name() string {
return b.opts.Name
}
func (b *Broker) Live() bool {
return true
}
func (b *Broker) Ready() bool {
return true
}
func (b *Broker) Health() bool {
return true
}
func (m *Subscriber) Options() broker.SubscribeOptions {
return m.opts
}
func (m *Subscriber) Topic() string {
return m.topic
}
func (m *Subscriber) Unsubscribe(ctx context.Context) error {
m.exit <- true
return nil
}
// NewBroker return new memory broker
func NewBroker(opts ...broker.Option) broker.Broker {
return &Broker{
opts: broker.NewOptions(opts...),
subscribers: make(map[string][]*Subscriber),
}
}

View File

@@ -1,74 +0,0 @@
package broker
import (
"context"
"fmt"
"testing"
"go.uber.org/atomic"
"go.unistack.org/micro/v4/broker"
"go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/metadata"
)
type hldr struct {
c atomic.Int64
}
func (h *hldr) Handler(m broker.Message) error {
h.c.Add(1)
return nil
}
func TestMemoryBroker(t *testing.T) {
b := NewBroker(broker.Codec("application/octet-stream", codec.NewCodec()))
ctx := context.Background()
if err := b.Init(); err != nil {
t.Fatalf("Unexpected init error %v", err)
}
if err := b.Connect(ctx); err != nil {
t.Fatalf("Unexpected connect error %v", err)
}
topic := "test"
count := int64(10)
h := &hldr{}
sub, err := b.Subscribe(ctx, topic, h.Handler)
if err != nil {
t.Fatalf("Unexpected error subscribing %v", err)
}
for i := int64(0); i < count; i++ {
message, err := b.NewMessage(ctx,
metadata.Pairs(
"foo", "bar",
"id", fmt.Sprintf("%d", i),
),
[]byte(`"hello world"`),
broker.PublishContentType("application/octet-stream"),
)
if err != nil {
t.Fatal(err)
}
if err := b.Publish(ctx, topic, message); err != nil {
t.Fatalf("Unexpected error publishing %d err: %v", i, err)
}
}
if err := sub.Unsubscribe(ctx); err != nil {
t.Fatalf("Unexpected error unsubscribing from %s: %v", topic, err)
}
if err := b.Disconnect(ctx); err != nil {
t.Fatalf("Unexpected connect error %v", err)
}
if h.c.Load() != count {
t.Fatal("invalid messages count received")
}
}

107
broker/memory_test.go Normal file
View File

@@ -0,0 +1,107 @@
package broker
import (
"context"
"fmt"
"testing"
"go.unistack.org/micro/v4/metadata"
)
func TestMemoryBatchBroker(t *testing.T) {
b := NewBroker()
ctx := context.Background()
if err := b.Connect(ctx); err != nil {
t.Fatalf("Unexpected connect error %v", err)
}
topic := "test"
count := 10
fn := func(evts []Message) error {
var err error
for _, evt := range evts {
if err = evt.Ack(); err != nil {
return err
}
}
return nil
}
sub, err := b.Subscribe(ctx, topic, fn)
if err != nil {
t.Fatalf("Unexpected error subscribing %v", err)
}
msgs := make([]Message, 0, count)
for i := 0; i < count; i++ {
message := &memoryMessage{
header: map[string]string{
metadata.HeaderTopic: topic,
"foo": "bar",
"id": fmt.Sprintf("%d", i),
},
body: []byte(`"hello world"`),
}
msgs = append(msgs, message)
}
if err := b.Publish(ctx, msgs); err != nil {
t.Fatalf("Unexpected error publishing %v", err)
}
if err := sub.Unsubscribe(ctx); err != nil {
t.Fatalf("Unexpected error unsubscribing from %s: %v", topic, err)
}
if err := b.Disconnect(ctx); err != nil {
t.Fatalf("Unexpected connect error %v", err)
}
}
func TestMemoryBroker(t *testing.T) {
b := NewBroker()
ctx := context.Background()
if err := b.Connect(ctx); err != nil {
t.Fatalf("Unexpected connect error %v", err)
}
topic := "test"
count := 10
fn := func(p Message) error {
return p.Ack()
}
sub, err := b.Subscribe(ctx, topic, fn)
if err != nil {
t.Fatalf("Unexpected error subscribing %v", err)
}
msgs := make([]Message, 0, count)
for i := 0; i < count; i++ {
message := &memoryMessage{
header: map[string]string{
metadata.HeaderTopic: topic,
"foo": "bar",
"id": fmt.Sprintf("%d", i),
},
body: []byte(`"hello world"`),
}
msgs = append(msgs, message)
}
if err := b.Publish(ctx, msgs); err != nil {
t.Fatalf("Unexpected error publishing %v", err)
}
if err := sub.Unsubscribe(ctx); err != nil {
t.Fatalf("Unexpected error unsubscribing from %s: %v", topic, err)
}
if err := b.Disconnect(ctx); err != nil {
t.Fatalf("Unexpected connect error %v", err)
}
}

View File

@@ -1,179 +0,0 @@
package broker
import (
"context"
"strings"
"sync"
"go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/options"
)
type NoopBroker struct {
funcPublish FuncPublish
funcSubscribe FuncSubscribe
opts Options
sync.RWMutex
}
func (b *NoopBroker) newCodec(ct string) (codec.Codec, error) {
if idx := strings.IndexRune(ct, ';'); idx >= 0 {
ct = ct[:idx]
}
b.RLock()
c, ok := b.opts.Codecs[ct]
b.RUnlock()
if ok {
return c, nil
}
return nil, codec.ErrUnknownContentType
}
func NewBroker(opts ...Option) *NoopBroker {
b := &NoopBroker{opts: NewOptions(opts...)}
b.funcPublish = b.fnPublish
b.funcSubscribe = b.fnSubscribe
return b
}
func (b *NoopBroker) Health() bool {
return true
}
func (b *NoopBroker) Live() bool {
return true
}
func (b *NoopBroker) Ready() bool {
return true
}
func (b *NoopBroker) Name() string {
return b.opts.Name
}
func (b *NoopBroker) String() string {
return "noop"
}
func (b *NoopBroker) Options() Options {
return b.opts
}
func (b *NoopBroker) Init(opts ...Option) error {
for _, opt := range opts {
opt(&b.opts)
}
b.funcPublish = b.fnPublish
b.funcSubscribe = b.fnSubscribe
b.opts.Hooks.EachPrev(func(hook options.Hook) {
switch h := hook.(type) {
case HookPublish:
b.funcPublish = h(b.funcPublish)
case HookSubscribe:
b.funcSubscribe = h(b.funcSubscribe)
}
})
return nil
}
func (b *NoopBroker) Connect(_ context.Context) error {
return nil
}
func (b *NoopBroker) Disconnect(_ context.Context) error {
return nil
}
func (b *NoopBroker) Address() string {
return strings.Join(b.opts.Addrs, ",")
}
type noopMessage struct {
c codec.Codec
ctx context.Context
body []byte
hdr metadata.Metadata
opts PublishOptions
}
func (m *noopMessage) Ack() error {
return nil
}
func (m *noopMessage) Body() []byte {
return m.body
}
func (m *noopMessage) Header() metadata.Metadata {
return m.hdr
}
func (m *noopMessage) Context() context.Context {
return m.ctx
}
func (m *noopMessage) Topic() string {
return ""
}
func (m *noopMessage) Unmarshal(dst interface{}, opts ...codec.Option) error {
return m.c.Unmarshal(m.body, dst)
}
func (b *NoopBroker) NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...PublishOption) (Message, error) {
options := NewPublishOptions(opts...)
if options.ContentType == "" {
options.ContentType = b.opts.ContentType
}
m := &noopMessage{ctx: ctx, hdr: hdr, opts: options}
c, err := b.newCodec(m.opts.ContentType)
if err == nil {
m.body, err = c.Marshal(body)
}
if err != nil {
return nil, err
}
return m, nil
}
func (b *NoopBroker) fnPublish(_ context.Context, _ string, _ ...Message) error {
return nil
}
func (b *NoopBroker) Publish(ctx context.Context, topic string, msg ...Message) error {
return b.funcPublish(ctx, topic, msg...)
}
type NoopSubscriber struct {
ctx context.Context
topic string
handler interface{}
opts SubscribeOptions
}
func (b *NoopBroker) fnSubscribe(ctx context.Context, topic string, handler interface{}, opts ...SubscribeOption) (Subscriber, error) {
return &NoopSubscriber{ctx: ctx, topic: topic, opts: NewSubscribeOptions(opts...), handler: handler}, nil
}
func (b *NoopBroker) Subscribe(ctx context.Context, topic string, handler interface{}, opts ...SubscribeOption) (Subscriber, error) {
return b.funcSubscribe(ctx, topic, handler, opts...)
}
func (s *NoopSubscriber) Options() SubscribeOptions {
return s.opts
}
func (s *NoopSubscriber) Topic() string {
return s.topic
}
func (s *NoopSubscriber) Unsubscribe(_ context.Context) error {
return nil
}

View File

@@ -1,35 +0,0 @@
package broker
import (
"context"
"testing"
)
type testHook struct {
f bool
}
func (t *testHook) Publish1(fn FuncPublish) FuncPublish {
return func(ctx context.Context, topic string, messages ...Message) error {
t.f = true
return fn(ctx, topic, messages...)
}
}
func TestNoopHook(t *testing.T) {
h := &testHook{}
b := NewBroker(Hooks(HookPublish(h.Publish1)))
if err := b.Init(); err != nil {
t.Fatal(err)
}
if err := b.Publish(context.TODO(), "", nil); err != nil {
t.Fatal(err)
}
if !h.f {
t.Fatal("hook not works")
}
}

View File

@@ -7,23 +7,20 @@ import (
"go.unistack.org/micro/v4/codec" "go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/meter" "go.unistack.org/micro/v4/meter"
"go.unistack.org/micro/v4/options" "go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/register" "go.unistack.org/micro/v4/register"
"go.unistack.org/micro/v4/sync"
"go.unistack.org/micro/v4/tracer" "go.unistack.org/micro/v4/tracer"
) )
// Options struct // Options struct
type Options struct { type Options struct {
// Name holds the broker name
Name string
// Tracer used for tracing // Tracer used for tracing
Tracer tracer.Tracer Tracer tracer.Tracer
// Register can be used for clustering // Register can be used for clustering
Register register.Register Register register.Register
// Codecs holds the codecs for marshal/unmarshal based on content-type // Codecs holds the codec for marshal/unmarshal
Codecs map[string]codec.Codec Codecs map[string]codec.Codec
// Logger used for logging // Logger used for logging
Logger logger.Logger Logger logger.Logger
@@ -31,75 +28,48 @@ type Options struct {
Meter meter.Meter Meter meter.Meter
// Context holds external options // Context holds external options
Context context.Context Context context.Context
// Wait waits for a collection of goroutines to finish
Wait *sync.WaitGroup
// TLSConfig holds tls.TLSConfig options // TLSConfig holds tls.TLSConfig options
TLSConfig *tls.Config TLSConfig *tls.Config
// ErrorHandler used when broker have error while processing message
// Addrs holds the broker address ErrorHandler interface{}
Addrs []string // Name holds the broker name
// Hooks can be run before broker Publish/BatchPublish and Name string
// Subscribe/BatchSubscribe methods // Address holds the broker address
Hooks options.Hooks Address []string
// GracefulTimeout contains time to wait to finish in flight requests
GracefulTimeout time.Duration
// ContentType will be used if no content-type set when creating message
ContentType string
} }
// NewOptions create new Options // NewOptions create new Options
func NewOptions(opts ...Option) Options { func NewOptions(opts ...options.Option) Options {
options := Options{ options := Options{
Register: register.DefaultRegister, Register: register.DefaultRegister,
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
Context: context.Background(), Context: context.Background(),
Meter: meter.DefaultMeter, Meter: meter.DefaultMeter,
Codecs: make(map[string]codec.Codec), Codecs: make(map[string]codec.Codec),
Tracer: tracer.DefaultTracer, Tracer: tracer.DefaultTracer,
GracefulTimeout: DefaultGracefulTimeout,
ContentType: DefaultContentType,
} }
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
} }
return options return options
} }
// DefaultContentType is the default content-type if not specified
var DefaultContentType = ""
// Context sets the context option
func Context(ctx context.Context) Option {
return func(o *Options) {
o.Context = ctx
}
}
// ContentType used by default if not specified
func ContentType(ct string) Option {
return func(o *Options) {
o.ContentType = ct
}
}
// PublishOptions struct // PublishOptions struct
type PublishOptions struct { type PublishOptions struct {
// ContentType for message body // Context holds external options
ContentType string
// BodyOnly flag says the message contains raw body bytes and don't need
// codec Marshal method
BodyOnly bool
// Context holds custom options
Context context.Context Context context.Context
// BodyOnly flag says the message contains raw body bytes
BodyOnly bool
// Message metadata usually passed as message headers
Metadata metadata.Metadata
// Content-Type of message for marshal
ContentType string
// Topic destination
Topic string
} }
// NewPublishOptions creates PublishOptions struct // NewPublishOptions creates PublishOptions struct
func NewPublishOptions(opts ...PublishOption) PublishOptions { func NewPublishOptions(opts ...options.Option) PublishOptions {
options := PublishOptions{ options := PublishOptions{
Context: context.Background(), Context: context.Background(),
} }
@@ -109,12 +79,21 @@ func NewPublishOptions(opts ...PublishOption) PublishOptions {
return options return options
} }
// PublishTopic pass topic for messages
func PublishTopic(t string) options.Option {
return func(src interface{}) error {
return options.Set(src, t, ".Topic")
}
}
// SubscribeOptions struct // SubscribeOptions struct
type SubscribeOptions struct { type SubscribeOptions struct {
// Context holds external options // Context holds external options
Context context.Context Context context.Context
// Group holds consumer group // ErrorHandler used when broker have error while processing message
Group string ErrorHandler interface{}
// QueueGroup holds consumer group
QueueGroup string
// AutoAck flag specifies auto ack of incoming message when no error happens // AutoAck flag specifies auto ack of incoming message when no error happens
AutoAck bool AutoAck bool
// BodyOnly flag specifies that message contains only body bytes without header // BodyOnly flag specifies that message contains only body bytes without header
@@ -125,137 +104,16 @@ type SubscribeOptions struct {
BatchWait time.Duration BatchWait time.Duration
} }
// Option func // ErrorHandler will catch all broker errors that cant be handled
type Option func(*Options) // in normal way, for example Codec errors
func ErrorHandler(h interface{}) options.Option {
// PublishOption func return func(src interface{}) error {
type PublishOption func(*PublishOptions) return options.Set(src, h, ".ErrorHandler")
// PublishContentType sets message content-type that used to Marshal
func PublishContentType(ct string) PublishOption {
return func(o *PublishOptions) {
o.ContentType = ct
} }
} }
// PublishBodyOnly publish only body of the message
func PublishBodyOnly(b bool) PublishOption {
return func(o *PublishOptions) {
o.BodyOnly = b
}
}
// Addrs sets the host addresses to be used by the broker
func Addrs(addrs ...string) Option {
return func(o *Options) {
o.Addrs = addrs
}
}
// Codec sets the codec used for encoding/decoding messages
func Codec(ct string, c codec.Codec) Option {
return func(o *Options) {
o.Codecs[ct] = c
}
}
// SubscribeGroup sets the name of the queue to share messages on
func SubscribeGroup(name string) SubscribeOption {
return func(o *SubscribeOptions) {
o.Group = name
}
}
// Register sets register option
func Register(r register.Register) Option {
return func(o *Options) {
o.Register = r
}
}
// TLSConfig sets the TLS Config
func TLSConfig(t *tls.Config) Option {
return func(o *Options) {
o.TLSConfig = t
}
}
// Logger sets the logger
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// Tracer to be used for tracing
func Tracer(t tracer.Tracer) Option {
return func(o *Options) {
o.Tracer = t
}
}
// Meter sets the meter
func Meter(m meter.Meter) Option {
return func(o *Options) {
o.Meter = m
}
}
// Name sets the name
func Name(n string) Option {
return func(o *Options) {
o.Name = n
}
}
// Hooks sets hook runs before action
func Hooks(h ...options.Hook) Option {
return func(o *Options) {
o.Hooks = append(o.Hooks, h...)
}
}
// SubscribeContext set context
func SubscribeContext(ctx context.Context) SubscribeOption {
return func(o *SubscribeOptions) {
o.Context = ctx
}
}
// SubscribeAutoAck contol auto acking of messages
// after they have been handled.
func SubscribeAutoAck(b bool) SubscribeOption {
return func(o *SubscribeOptions) {
o.AutoAck = b
}
}
// SubscribeBodyOnly consumes only body of the message
func SubscribeBodyOnly(b bool) SubscribeOption {
return func(o *SubscribeOptions) {
o.BodyOnly = b
}
}
// SubscribeBatchSize specifies max batch size
func SubscribeBatchSize(n int) SubscribeOption {
return func(o *SubscribeOptions) {
o.BatchSize = n
}
}
// SubscribeBatchWait specifies max batch wait time
func SubscribeBatchWait(td time.Duration) SubscribeOption {
return func(o *SubscribeOptions) {
o.BatchWait = td
}
}
// SubscribeOption func
type SubscribeOption func(*SubscribeOptions)
// NewSubscribeOptions creates new SubscribeOptions // NewSubscribeOptions creates new SubscribeOptions
func NewSubscribeOptions(opts ...SubscribeOption) SubscribeOptions { func NewSubscribeOptions(opts ...options.Option) SubscribeOptions {
options := SubscribeOptions{ options := SubscribeOptions{
AutoAck: true, AutoAck: true,
Context: context.Background(), Context: context.Background(),
@@ -265,3 +123,39 @@ func NewSubscribeOptions(opts ...SubscribeOption) SubscribeOptions {
} }
return options return options
} }
// SubscribeAutoAck contol auto acking of messages
// after they have been handled.
func SubscribeAutoAck(b bool) options.Option {
return func(src interface{}) error {
return options.Set(src, b, ".AutoAck")
}
}
// BodyOnly transfer only body without
func BodyOnly(b bool) options.Option {
return func(src interface{}) error {
return options.Set(src, b, ".BodyOnly")
}
}
// SubscribeBatchSize specifies max batch size
func SubscribeBatchSize(n int) options.Option {
return func(src interface{}) error {
return options.Set(src, n, ".BatchSize")
}
}
// SubscribeBatchWait specifies max batch wait time
func SubscribeBatchWait(td time.Duration) options.Option {
return func(src interface{}) error {
return options.Set(src, td, ".BatchWait")
}
}
// SubscribeQueueGroup sets the shared queue name distributed messages across subscribers
func SubscribeQueueGroup(n string) options.Option {
return func(src interface{}) error {
return options.Set(src, n, ".QueueGroup")
}
}

View File

@@ -1,14 +1,98 @@
package broker package broker
// IsValidHandler func signature import (
func IsValidHandler(sub interface{}) error { "fmt"
switch sub.(type) { "reflect"
default: "strings"
return ErrInvalidHandler "unicode"
case func(Message) error: "unicode/utf8"
break )
case func([]Message) error:
break const (
subSig = "func(context.Context, interface{}) error"
batchSubSig = "func([]context.Context, []interface{}) error"
)
// Precompute the reflect type for error. Can't use error directly
// because Typeof takes an empty interface value. This is annoying.
var typeOfError = reflect.TypeOf((*error)(nil)).Elem()
// Is this an exported - upper case - name?
func isExported(name string) bool {
rune, _ := utf8.DecodeRuneInString(name)
return unicode.IsUpper(rune)
}
// Is this type exported or a builtin?
func isExportedOrBuiltinType(t reflect.Type) bool {
for t.Kind() == reflect.Ptr {
t = t.Elem()
} }
// PkgPath will be non-empty even for an exported type,
// so we need to check the type name as well.
return isExported(t.Name()) || t.PkgPath() == ""
}
// ValidateSubscriber func signature
func ValidateSubscriber(sub interface{}) error {
typ := reflect.TypeOf(sub)
var argType reflect.Type
switch typ.Kind() {
case reflect.Func:
name := "Func"
switch typ.NumIn() {
case 1: // func(Message) error
case 2: // func(context.Context, Message) error or func(context.Context, []Message) error
argType = typ.In(2)
// if sub.Options().Batch {
if argType.Kind() != reflect.Slice {
return fmt.Errorf("subscriber %v dont have required signature %s", name, batchSubSig)
}
if strings.Compare(fmt.Sprintf("%v", argType), "[]interface{}") == 0 {
return fmt.Errorf("subscriber %v dont have required signaure %s", name, batchSubSig)
}
// }
default:
return fmt.Errorf("subscriber %v takes wrong number of args: %v required signature %s or %s", name, typ.NumIn(), subSig, batchSubSig)
}
if !isExportedOrBuiltinType(argType) {
return fmt.Errorf("subscriber %v argument type not exported: %v", name, argType)
}
if typ.NumOut() != 1 {
return fmt.Errorf("subscriber %v has wrong number of return values: %v require signature %s or %s",
name, typ.NumOut(), subSig, batchSubSig)
}
if returnType := typ.Out(0); returnType != typeOfError {
return fmt.Errorf("subscriber %v returns %v not error", name, returnType.String())
}
default:
hdlr := reflect.ValueOf(sub)
name := reflect.Indirect(hdlr).Type().Name()
for m := 0; m < typ.NumMethod(); m++ {
method := typ.Method(m)
switch method.Type.NumIn() {
case 3:
argType = method.Type.In(2)
default:
return fmt.Errorf("subscriber %v.%v takes wrong number of args: %v required signature %s or %s",
name, method.Name, method.Type.NumIn(), subSig, batchSubSig)
}
if !isExportedOrBuiltinType(argType) {
return fmt.Errorf("%v argument type not exported: %v", name, argType)
}
if method.Type.NumOut() != 1 {
return fmt.Errorf(
"subscriber %v.%v has wrong number of return values: %v require signature %s or %s",
name, method.Name, method.Type.NumOut(), subSig, batchSubSig)
}
if returnType := method.Type.Out(0); returnType != typeOfError {
return fmt.Errorf("subscriber %v.%v returns %v not error", name, method.Name, returnType.String())
}
}
}
return nil return nil
} }

View File

@@ -17,13 +17,13 @@ func BackoffExp(_ context.Context, _ Request, attempts int) (time.Duration, erro
} }
// BackoffInterval specifies randomization interval for backoff func // BackoffInterval specifies randomization interval for backoff func
func BackoffInterval(minTime time.Duration, maxTime time.Duration) BackoffFunc { func BackoffInterval(min time.Duration, max time.Duration) BackoffFunc {
return func(_ context.Context, _ Request, attempts int) (time.Duration, error) { return func(_ context.Context, _ Request, attempts int) (time.Duration, error) {
td := time.Duration(math.Pow(float64(attempts), math.E)) * time.Millisecond * 100 td := time.Duration(math.Pow(float64(attempts), math.E)) * time.Millisecond * 100
if td < minTime { if td < min {
return minTime, nil return min, nil
} else if td > maxTime { } else if td > max {
return maxTime, nil return max, nil
} }
return td, nil return td, nil
} }

View File

@@ -34,23 +34,23 @@ func TestBackoffExp(t *testing.T) {
} }
func TestBackoffInterval(t *testing.T) { func TestBackoffInterval(t *testing.T) {
minTime := 100 * time.Millisecond min := 100 * time.Millisecond
maxTime := 300 * time.Millisecond max := 300 * time.Millisecond
r := &testRequest{ r := &testRequest{
service: "test", service: "test",
method: "test", method: "test",
} }
fn := BackoffInterval(minTime, maxTime) fn := BackoffInterval(min, max)
for i := 0; i < 5; i++ { for i := 0; i < 5; i++ {
d, err := fn(context.TODO(), r, i) d, err := fn(context.TODO(), r, i)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if d < minTime || d > maxTime { if d < min || d > max {
t.Fatalf("Expected %v < %v < %v", minTime, d, maxTime) t.Fatalf("Expected %v < %v < %v", min, d, max)
} }
} }
} }

View File

@@ -1,12 +1,12 @@
// Package client is an interface for an RPC client // Package client is an interface for an RPC client
package client package client // import "go.unistack.org/micro/v4/client"
import ( import (
"context" "context"
"time" "time"
"go.unistack.org/micro/v4/codec" "go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/metadata" "go.unistack.org/micro/v4/options"
) )
var ( var (
@@ -22,6 +22,8 @@ var (
DefaultRetries = 0 DefaultRetries = 0
// DefaultRequestTimeout is the default request timeout // DefaultRequestTimeout is the default request timeout
DefaultRequestTimeout = time.Second * 5 DefaultRequestTimeout = time.Second * 5
// DefaultDialTimeout the default dial timeout
DefaultDialTimeout = time.Second * 5
// DefaultPoolSize sets the connection pool size // DefaultPoolSize sets the connection pool size
DefaultPoolSize = 100 DefaultPoolSize = 100
// DefaultPoolTTL sets the connection pool ttl // DefaultPoolTTL sets the connection pool ttl
@@ -29,24 +31,18 @@ var (
) )
// Client is the interface used to make requests to services. // Client is the interface used to make requests to services.
// It supports Request/Response via Transport and Publishing via the Broker.
// It also supports bidirectional streaming of requests. // It also supports bidirectional streaming of requests.
type Client interface { type Client interface {
Name() string Name() string
Init(opts ...Option) error Init(opts ...options.Option) error
Options() Options Options() Options
NewRequest(service string, endpoint string, req interface{}, opts ...RequestOption) Request NewRequest(service string, endpoint string, req interface{}, opts ...options.Option) Request
Call(ctx context.Context, req Request, rsp interface{}, opts ...CallOption) error Call(ctx context.Context, req Request, rsp interface{}, opts ...options.Option) error
Stream(ctx context.Context, req Request, opts ...CallOption) (Stream, error) Stream(ctx context.Context, req Request, opts ...options.Option) (Stream, error)
String() string String() string
} }
type (
FuncCall func(ctx context.Context, req Request, rsp interface{}, opts ...CallOption) error
HookCall func(next FuncCall) FuncCall
FuncStream func(ctx context.Context, req Request, opts ...CallOption) (Stream, error)
HookStream func(next FuncStream) FuncStream
)
// Request is the interface for a synchronous request used by Call or Stream // Request is the interface for a synchronous request used by Call or Stream
type Request interface { type Request interface {
// The service to call // The service to call
@@ -63,16 +59,22 @@ type Request interface {
Codec() codec.Codec Codec() codec.Codec
// indicates whether the request will be a streaming one rather than unary // indicates whether the request will be a streaming one rather than unary
Stream() bool Stream() bool
// Header data
// Header() metadata.Metadata
} }
// Response is the response received from a service // Response is the response received from a service
type Response interface { type Response interface {
// Read the response // Read the response
Codec() codec.Codec Codec() codec.Codec
// The content type
// ContentType() string
// Header data // Header data
Header() metadata.Metadata // Header() metadata.Metadata
// Read the undecoded response // Read the undecoded response
Read() ([]byte, error) Read() ([]byte, error)
// The unencoded request body
// Body() interface{}
} }
// Stream is the interface for a bidirectional synchronous stream // Stream is the interface for a bidirectional synchronous stream
@@ -98,12 +100,3 @@ type Stream interface {
// CloseSend closes the send direction of the stream // CloseSend closes the send direction of the stream
CloseSend() error CloseSend() error
} }
// Option used by the Client
type Option func(*Options)
// CallOption used by Call or Stream
type CallOption func(*CallOptions)
// RequestOption used by NewRequest
type RequestOption func(*RequestOptions)

View File

@@ -2,22 +2,24 @@ package client
import ( import (
"context" "context"
"go.unistack.org/micro/v4/options"
) )
type clientCallOptions struct { type clientCallOptions struct {
Client Client
opts []CallOption opts []options.Option
} }
func (s *clientCallOptions) Call(ctx context.Context, req Request, rsp interface{}, opts ...CallOption) error { func (s *clientCallOptions) Call(ctx context.Context, req Request, rsp interface{}, opts ...options.Option) error {
return s.Client.Call(ctx, req, rsp, append(s.opts, opts...)...) return s.Client.Call(ctx, req, rsp, append(s.opts, opts...)...)
} }
func (s *clientCallOptions) Stream(ctx context.Context, req Request, opts ...CallOption) (Stream, error) { func (s *clientCallOptions) Stream(ctx context.Context, req Request, opts ...options.Option) (Stream, error) {
return s.Client.Stream(ctx, req, append(s.opts, opts...)...) return s.Client.Stream(ctx, req, append(s.opts, opts...)...)
} }
// NewClientCallOptions add CallOption to every call // NewClientCallOptions add CallOption to every call
func NewClientCallOptions(c Client, opts ...CallOption) Client { func NewClientCallOptions(c Client, opts ...options.Option) Client {
return &clientCallOptions{c, opts} return &clientCallOptions{c, opts}
} }

View File

@@ -0,0 +1,28 @@
package client
import (
"context"
"testing"
"time"
"go.unistack.org/micro/v4/options"
)
func TestNewClientCallOptions(t *testing.T) {
var flag bool
w := func(fn CallFunc) CallFunc {
flag = true
return fn
}
c := NewClientCallOptions(NewClient(),
options.Address("127.0.0.1"),
WithCallWrapper(w),
RequestTimeout(1*time.Millisecond),
Retries(0),
Backoff(BackoffInterval(10*time.Millisecond, 100*time.Millisecond)),
)
_ = c.Call(context.TODO(), c.NewRequest("service", "endpoint", nil), nil)
if !flag {
t.Fatalf("NewClientCallOptions not works")
}
}

View File

@@ -15,15 +15,6 @@ func FromContext(ctx context.Context) (Client, bool) {
return c, ok return c, ok
} }
// MustContext get client from context
func MustContext(ctx context.Context) Client {
c, ok := FromContext(ctx)
if !ok {
panic("missing client")
}
return c
}
// NewContext put client in context // NewContext put client in context
func NewContext(ctx context.Context, c Client) context.Context { func NewContext(ctx context.Context, c Client) context.Context {
if ctx == nil { if ctx == nil {
@@ -31,23 +22,3 @@ func NewContext(ctx context.Context, c Client) context.Context {
} }
return context.WithValue(ctx, clientKey{}, c) return context.WithValue(ctx, clientKey{}, c)
} }
// SetCallOption returns a function to setup a context with given value
func SetCallOption(k, v interface{}) CallOption {
return func(o *CallOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option {
return func(o *Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}

View File

@@ -38,25 +38,3 @@ func TestNewNilContext(t *testing.T) {
t.Fatal("NewContext not works") t.Fatal("NewContext not works")
} }
} }
func TestSetCallOption(t *testing.T) {
type key struct{}
o := SetCallOption(key{}, "test")
opts := &CallOptions{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetCallOption not works")
}
}
func TestSetOption(t *testing.T) {
type key struct{}
o := SetOption(key{}, "test")
opts := &Options{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetOption not works")
}
}

View File

@@ -3,7 +3,6 @@ package client
import ( import (
"context" "context"
"fmt" "fmt"
"strconv"
"time" "time"
"go.unistack.org/micro/v4/codec" "go.unistack.org/micro/v4/codec"
@@ -12,7 +11,6 @@ import (
"go.unistack.org/micro/v4/options" "go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/selector" "go.unistack.org/micro/v4/selector"
"go.unistack.org/micro/v4/semconv" "go.unistack.org/micro/v4/semconv"
"go.unistack.org/micro/v4/tracer"
) )
// DefaultCodecs will be used to encode/decode data // DefaultCodecs will be used to encode/decode data
@@ -21,9 +19,7 @@ var DefaultCodecs = map[string]codec.Codec{
} }
type noopClient struct { type noopClient struct {
funcCall FuncCall opts Options
funcStream FuncStream
opts Options
} }
type noopRequest struct { type noopRequest struct {
@@ -37,13 +33,13 @@ type noopRequest struct {
} }
// NewClient returns new noop client // NewClient returns new noop client
func NewClient(opts ...Option) Client { func NewClient(opts ...options.Option) Client {
n := &noopClient{opts: NewOptions(opts...)} nc := &noopClient{opts: NewOptions(opts...)}
// wrap in reverse
n.funcCall = n.fnCall c := Client(nc)
n.funcStream = n.fnStream
return n return c
} }
func (n *noopClient) Name() string { func (n *noopClient) Name() string {
@@ -95,13 +91,10 @@ func (n *noopResponse) Read() ([]byte, error) {
return nil, nil return nil, nil
} }
type noopStream struct { type noopStream struct{}
err error
ctx context.Context
}
func (n *noopStream) Context() context.Context { func (n *noopStream) Context() context.Context {
return n.ctx return context.Background()
} }
func (n *noopStream) Request() Request { func (n *noopStream) Request() Request {
@@ -129,40 +122,31 @@ func (n *noopStream) RecvMsg(interface{}) error {
} }
func (n *noopStream) Error() error { func (n *noopStream) Error() error {
return n.err return nil
} }
func (n *noopStream) Close() error { func (n *noopStream) Close() error {
if sp, ok := tracer.SpanFromContext(n.ctx); ok && sp != nil { return nil
if n.err != nil {
sp.SetStatus(tracer.SpanStatusError, n.err.Error())
}
sp.Finish()
}
return n.err
} }
func (n *noopStream) CloseSend() error { func (n *noopStream) CloseSend() error {
return n.err return nil
} }
func (n *noopClient) Init(opts ...Option) error { func (n *noopClient) newCodec(contentType string) (codec.Codec, error) {
if cf, ok := n.opts.Codecs[contentType]; ok {
return cf, nil
}
if cf, ok := DefaultCodecs[contentType]; ok {
return cf, nil
}
return nil, codec.ErrUnknownContentType
}
func (n *noopClient) Init(opts ...options.Option) error {
for _, o := range opts { for _, o := range opts {
o(&n.opts) o(&n.opts)
} }
n.funcCall = n.fnCall
n.funcStream = n.fnStream
n.opts.Hooks.EachPrev(func(hook options.Hook) {
switch h := hook.(type) {
case HookCall:
n.funcCall = h(n.funcCall)
case HookStream:
n.funcStream = h(n.funcStream)
}
})
return nil return nil
} }
@@ -174,32 +158,7 @@ func (n *noopClient) String() string {
return "noop" return "noop"
} }
func (n *noopClient) Call(ctx context.Context, req Request, rsp interface{}, opts ...CallOption) error { func (n *noopClient) Call(ctx context.Context, req Request, rsp interface{}, opts ...options.Option) error {
ts := time.Now()
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", req.Endpoint()).Inc()
var sp tracer.Span
ctx, sp = n.opts.Tracer.Start(ctx, "rpc-client",
tracer.WithSpanKind(tracer.SpanKindClient),
tracer.WithSpanLabels("endpoint", req.Endpoint()),
)
err := n.funcCall(ctx, req, rsp, opts...)
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", req.Endpoint()).Dec()
te := time.Since(ts)
n.opts.Meter.Summary(semconv.ClientRequestLatencyMicroseconds, "endpoint", req.Endpoint()).Update(te.Seconds())
n.opts.Meter.Histogram(semconv.ClientRequestDurationSeconds, "endpoint", req.Endpoint()).Update(te.Seconds())
if me := errors.FromError(err); me == nil {
sp.Finish()
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", req.Endpoint(), "status", "success", "code", strconv.Itoa(int(200))).Inc()
} else {
sp.SetStatus(tracer.SpanStatusError, err.Error())
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", req.Endpoint(), "status", "failure", "code", strconv.Itoa(int(me.Code))).Inc()
}
return err
}
func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, opts ...CallOption) error {
// make a copy of call opts // make a copy of call opts
callOpts := n.opts.CallOptions callOpts := n.opts.CallOptions
for _, opt := range opts { for _, opt := range opts {
@@ -216,7 +175,7 @@ func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, o
} else { } else {
// got a deadline so no need to setup context // got a deadline so no need to setup context
// but we need to set the timeout we pass along // but we need to set the timeout we pass along
opt := WithRequestTimeout(time.Until(d)) opt := RequestTimeout(time.Until(d))
opt(&callOpts) opt(&callOpts)
} }
@@ -228,8 +187,11 @@ func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, o
} }
// make copy of call method // make copy of call method
hcall := func(ctx context.Context, addr string, req Request, rsp interface{}, opts CallOptions) error { hcall := n.call
return nil
// wrap the call in reverse
for i := len(callOpts.CallWrappers); i > 0; i-- {
hcall = callOpts.CallWrappers[i-1](hcall)
} }
// use the router passed as a call option, or fallback to the rpc clients router // use the router passed as a call option, or fallback to the rpc clients router
@@ -254,7 +216,7 @@ func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, o
// call backoff first. Someone may want an initial start delay // call backoff first. Someone may want an initial start delay
t, err := callOpts.Backoff(ctx, req, i) t, err := callOpts.Backoff(ctx, req, i)
if err != nil { if err != nil {
return errors.InternalServerError("go.micro.client", "%s", err) return errors.InternalServerError("go.micro.client", err.Error())
} }
// only sleep if greater than 0 // only sleep if greater than 0
@@ -268,7 +230,7 @@ func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, o
// TODO apply any filtering here // TODO apply any filtering here
routes, err = n.opts.Lookup(ctx, req, callOpts) routes, err = n.opts.Lookup(ctx, req, callOpts)
if err != nil { if err != nil {
return errors.InternalServerError("go.micro.client", "%s", err) return errors.InternalServerError("go.micro.client", err.Error())
} }
// balance the list of nodes // balance the list of nodes
@@ -298,6 +260,9 @@ func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, o
ch := make(chan error, callOpts.Retries) ch := make(chan error, callOpts.Retries)
var gerr error var gerr error
ts := time.Now()
endpoint := fmt.Sprintf("%s.%s", req.Service(), req.Endpoint())
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", endpoint).Inc()
for i := 0; i <= callOpts.Retries; i++ { for i := 0; i <= callOpts.Retries; i++ {
go func() { go func() {
ch <- call(i) ch <- call(i)
@@ -325,39 +290,28 @@ func (n *noopClient) fnCall(ctx context.Context, req Request, rsp interface{}, o
} }
} }
if gerr != nil {
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", endpoint, "status", "failure").Inc()
} else {
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", endpoint, "status", "success").Inc()
}
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", endpoint).Dec()
te := time.Since(ts)
n.opts.Meter.Summary(semconv.ClientRequestLatencyMicroseconds, "endpoint", endpoint).Update(te.Seconds())
n.opts.Meter.Histogram(semconv.ClientRequestDurationSeconds, "endpoint", endpoint).Update(te.Seconds())
return gerr return gerr
} }
func (n *noopClient) NewRequest(service, endpoint string, _ interface{}, _ ...RequestOption) Request { func (n *noopClient) call(ctx context.Context, addr string, req Request, rsp interface{}, opts CallOptions) error {
return nil
}
func (n *noopClient) NewRequest(service, endpoint string, req interface{}, opts ...options.Option) Request {
return &noopRequest{service: service, endpoint: endpoint} return &noopRequest{service: service, endpoint: endpoint}
} }
func (n *noopClient) Stream(ctx context.Context, req Request, opts ...CallOption) (Stream, error) { func (n *noopClient) Stream(ctx context.Context, req Request, opts ...options.Option) (Stream, error) {
ts := time.Now()
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", req.Endpoint()).Inc()
var sp tracer.Span
ctx, sp = n.opts.Tracer.Start(ctx, "rpc-client",
tracer.WithSpanKind(tracer.SpanKindClient),
tracer.WithSpanLabels("endpoint", req.Endpoint()),
)
stream, err := n.funcStream(ctx, req, opts...)
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", req.Endpoint()).Dec()
te := time.Since(ts)
n.opts.Meter.Summary(semconv.ClientRequestLatencyMicroseconds, "endpoint", req.Endpoint()).Update(te.Seconds())
n.opts.Meter.Histogram(semconv.ClientRequestDurationSeconds, "endpoint", req.Endpoint()).Update(te.Seconds())
if me := errors.FromError(err); me == nil {
sp.Finish()
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", req.Endpoint(), "status", "success", "code", strconv.Itoa(int(200))).Inc()
} else {
sp.SetStatus(tracer.SpanStatusError, err.Error())
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", req.Endpoint(), "status", "failure", "code", strconv.Itoa(int(me.Code))).Inc()
}
return stream, err
}
func (n *noopClient) fnStream(ctx context.Context, req Request, opts ...CallOption) (Stream, error) {
var err error var err error
// make a copy of call opts // make a copy of call opts
@@ -376,7 +330,7 @@ func (n *noopClient) fnStream(ctx context.Context, req Request, opts ...CallOpti
} else { } else {
// got a deadline so no need to setup context // got a deadline so no need to setup context
// but we need to set the timeout we pass along // but we need to set the timeout we pass along
o := WithStreamTimeout(time.Until(d)) o := StreamTimeout(time.Until(d))
o(&callOpts) o(&callOpts)
} }
@@ -417,7 +371,7 @@ func (n *noopClient) fnStream(ctx context.Context, req Request, opts ...CallOpti
// call backoff first. Someone may want an initial start delay // call backoff first. Someone may want an initial start delay
t, cerr := callOpts.Backoff(ctx, req, i) t, cerr := callOpts.Backoff(ctx, req, i)
if cerr != nil { if cerr != nil {
return nil, errors.InternalServerError("go.micro.client", "%s", cerr) return nil, errors.InternalServerError("go.micro.client", cerr.Error())
} }
// only sleep if greater than 0 // only sleep if greater than 0
@@ -431,7 +385,7 @@ func (n *noopClient) fnStream(ctx context.Context, req Request, opts ...CallOpti
// TODO apply any filtering here // TODO apply any filtering here
routes, err = n.opts.Lookup(ctx, req, callOpts) routes, err = n.opts.Lookup(ctx, req, callOpts)
if err != nil { if err != nil {
return nil, errors.InternalServerError("go.micro.client", "%s", err) return nil, errors.InternalServerError("go.micro.client", err.Error())
} }
// balance the list of nodes // balance the list of nodes
@@ -443,7 +397,15 @@ func (n *noopClient) fnStream(ctx context.Context, req Request, opts ...CallOpti
node := next() node := next()
// ts := time.Now()
endpoint := fmt.Sprintf("%s.%s", req.Service(), req.Endpoint())
n.opts.Meter.Counter(semconv.ClientRequestInflight, "endpoint", endpoint).Inc()
stream, cerr := n.stream(ctx, node, req, callOpts) stream, cerr := n.stream(ctx, node, req, callOpts)
if cerr != nil {
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", endpoint, "status", "failure").Inc()
} else {
n.opts.Meter.Counter(semconv.ClientRequestTotal, "endpoint", endpoint, "status", "success").Inc()
}
// record the result of the call to inform future routing decisions // record the result of the call to inform future routing decisions
if verr := n.opts.Selector.Record(node, cerr); verr != nil { if verr := n.opts.Selector.Record(node, cerr); verr != nil {
@@ -497,6 +459,6 @@ func (n *noopClient) fnStream(ctx context.Context, req Request, opts ...CallOpti
return nil, grr return nil, grr
} }
func (n *noopClient) stream(ctx context.Context, _ string, _ Request, _ CallOptions) (Stream, error) { func (n *noopClient) stream(ctx context.Context, addr string, req Request, opts CallOptions) (*noopStream, error) {
return &noopStream{ctx: ctx}, nil return &noopStream{}, nil
} }

View File

@@ -6,13 +6,11 @@ import (
"net" "net"
"time" "time"
"go.unistack.org/micro/v4/broker"
"go.unistack.org/micro/v4/codec" "go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/metadata" "go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/meter" "go.unistack.org/micro/v4/meter"
"go.unistack.org/micro/v4/options" "go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/register"
"go.unistack.org/micro/v4/router" "go.unistack.org/micro/v4/router"
"go.unistack.org/micro/v4/selector" "go.unistack.org/micro/v4/selector"
"go.unistack.org/micro/v4/selector/random" "go.unistack.org/micro/v4/selector/random"
@@ -21,56 +19,44 @@ import (
// Options holds client options // Options holds client options
type Options struct { type Options struct {
// Codecs map
Codecs map[string]codec.Codec
// Proxy is used for proxy requests
Proxy string
// ContentType is used to select codec
ContentType string
// Name is the client name
Name string
// Selector used to select needed address // Selector used to select needed address
Selector selector.Selector Selector selector.Selector
// Logger used to log messages // Logger used to log messages
Logger logger.Logger Logger logger.Logger
// Tracer used for tracing // Tracer used for tracing
Tracer tracer.Tracer Tracer tracer.Tracer
// Broker used to publish messages
Broker broker.Broker
// Meter used for metrics // Meter used for metrics
Meter meter.Meter Meter meter.Meter
// Context is used for external options // Context is used for external options
Context context.Context Context context.Context
// Router used to get route // Router used to get route
Router router.Router Router router.Router
// TLSConfig specifies tls.Config for secure connection // TLSConfig specifies tls.Config for secure connection
TLSConfig *tls.Config TLSConfig *tls.Config
// Codecs map
Codecs map[string]codec.Codec
// Lookup func used to get destination addr // Lookup func used to get destination addr
Lookup LookupFunc Lookup LookupFunc
// ContextDialer used to connect // Proxy is used for proxy requests
ContextDialer func(context.Context, string) (net.Conn, error) Proxy string
// ContentType is used to select codec
// Wrappers contains wrappers ContentType string
Wrappers []Wrapper // Name is the client name
// Hooks can be run before broker Publish/BatchPublish and Name string
// Subscribe/BatchSubscribe methods
Hooks options.Hooks
// CallOptions contains default CallOptions // CallOptions contains default CallOptions
CallOptions CallOptions CallOptions CallOptions
// PoolSize connection pool size // PoolSize connection pool size
PoolSize int PoolSize int
// PoolTTL connection pool ttl // PoolTTL connection pool ttl
PoolTTL time.Duration PoolTTL time.Duration
// ContextDialer used to connect
ContextDialer func(context.Context, string) (net.Conn, error)
// Hooks may contains Client func wrapper
Hooks options.Hooks
} }
// NewCallOptions creates new call options struct // NewCallOptions creates new call options struct
func NewCallOptions(opts ...CallOption) CallOptions { func NewCallOptions(opts ...options.Option) CallOptions {
options := CallOptions{} options := CallOptions{}
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
@@ -80,16 +66,6 @@ func NewCallOptions(opts ...CallOption) CallOptions {
// CallOptions holds client call options // CallOptions holds client call options
type CallOptions struct { type CallOptions struct {
// RequestMetadata holds additional metadata for call
RequestMetadata metadata.Metadata
// Network name
Network string
// Content-Type
ContentType string
// AuthToken string
AuthToken string
// Selector selects addr // Selector selects addr
Selector selector.Selector Selector selector.Selector
// Context used for deadline // Context used for deadline
@@ -97,48 +73,46 @@ type CallOptions struct {
// Router used for route // Router used for route
Router router.Router Router router.Router
// Retry func used for retries // Retry func used for retries
// ResponseMetadata holds additional metadata from call
ResponseMetadata *metadata.Metadata
Retry RetryFunc Retry RetryFunc
// Backoff func used for backoff when retry // Backoff func used for backoff when retry
Backoff BackoffFunc Backoff BackoffFunc
// ContextDialer used to connect // Network name
ContextDialer func(context.Context, string) (net.Conn, error) Network string
// Content-Type
ContentType string
// AuthToken string
AuthToken string
// Address specifies static addr list // Address specifies static addr list
Address []string Address []string
// SelectOptions selector options // SelectOptions selector options
SelectOptions []selector.SelectOption SelectOptions []selector.SelectOption
// CallWrappers call wrappers
CallWrappers []CallWrapper
// StreamTimeout stream timeout // StreamTimeout stream timeout
StreamTimeout time.Duration StreamTimeout time.Duration
// RequestTimeout request timeout // RequestTimeout request timeout
RequestTimeout time.Duration RequestTimeout time.Duration
// RequestMetadata holds additional metadata for call
RequestMetadata metadata.Metadata
// ResponseMetadata holds additional metadata from call
ResponseMetadata *metadata.Metadata
// DialTimeout dial timeout // DialTimeout dial timeout
DialTimeout time.Duration DialTimeout time.Duration
// Retries specifies retries num // Retries specifies retries num
Retries int Retries int
// ContextDialer used to connect
ContextDialer func(context.Context, string) (net.Conn, error)
} }
// ContextDialer pass ContextDialer to client // ContextDialer pass ContextDialer to client
func ContextDialer(fn func(context.Context, string) (net.Conn, error)) Option { func ContextDialer(fn func(context.Context, string) (net.Conn, error)) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.ContextDialer = fn return options.Set(src, fn, ".ContextDialer")
}
}
// Context pass context to client
func Context(ctx context.Context) Option {
return func(o *Options) {
o.Context = ctx
} }
} }
// NewRequestOptions creates new RequestOptions struct // NewRequestOptions creates new RequestOptions struct
func NewRequestOptions(opts ...RequestOption) RequestOptions { func NewRequestOptions(opts ...options.Option) RequestOptions {
options := RequestOptions{} options := RequestOptions{}
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
@@ -157,24 +131,24 @@ type RequestOptions struct {
} }
// NewOptions creates new options struct // NewOptions creates new options struct
func NewOptions(opts ...Option) Options { func NewOptions(opts ...options.Option) Options {
options := Options{ options := Options{
Context: context.Background(), Context: context.Background(),
ContentType: DefaultContentType, ContentType: DefaultContentType,
Codecs: DefaultCodecs, Codecs: make(map[string]codec.Codec),
CallOptions: CallOptions{ CallOptions: CallOptions{
Context: context.Background(), Context: context.Background(),
Backoff: DefaultBackoff, Backoff: DefaultBackoff,
Retry: DefaultRetry, Retry: DefaultRetry,
Retries: DefaultRetries, Retries: DefaultRetries,
RequestTimeout: DefaultRequestTimeout, RequestTimeout: DefaultRequestTimeout,
DialTimeout: DefaultDialTimeout,
}, },
Lookup: LookupRoute, Lookup: LookupRoute,
PoolSize: DefaultPoolSize, PoolSize: DefaultPoolSize,
PoolTTL: DefaultPoolTTL, PoolTTL: DefaultPoolTTL,
Selector: random.NewSelector(), Selector: random.NewSelector(),
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
Broker: broker.DefaultBroker,
Meter: meter.DefaultMeter, Meter: meter.DefaultMeter,
Tracer: tracer.DefaultTracer, Tracer: tracer.DefaultTracer,
Router: router.DefaultRouter, Router: router.DefaultRouter,
@@ -187,284 +161,131 @@ func NewOptions(opts ...Option) Options {
return options return options
} }
// Broker to be used for pub/sub
func Broker(b broker.Broker) Option {
return func(o *Options) {
o.Broker = b
}
}
// Tracer to be used for tracing
func Tracer(t tracer.Tracer) Option {
return func(o *Options) {
o.Tracer = t
}
}
// Logger to be used for log mesages
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// Meter to be used for metrics
func Meter(m meter.Meter) Option {
return func(o *Options) {
o.Meter = m
}
}
// Codec to be used to encode/decode requests for a given content type
func Codec(contentType string, c codec.Codec) Option {
return func(o *Options) {
o.Codecs[contentType] = c
}
}
// ContentType used by default if not specified
func ContentType(ct string) Option {
return func(o *Options) {
o.ContentType = ct
}
}
// Proxy sets the proxy address // Proxy sets the proxy address
func Proxy(addr string) Option { func Proxy(addr string) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Proxy = addr return options.Set(src, addr, ".Proxy")
} }
} }
// PoolSize sets the connection pool size // PoolSize sets the connection pool size
func PoolSize(d int) Option { func PoolSize(d int) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.PoolSize = d return options.Set(src, d, ".PoolSize")
} }
} }
// PoolTTL sets the connection pool ttl // PoolTTL sets the connection pool ttl
func PoolTTL(d time.Duration) Option { func PoolTTL(td time.Duration) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.PoolTTL = d return options.Set(src, td, ".PoolTTL")
}
}
// Register sets the routers register
func Register(r register.Register) Option {
return func(o *Options) {
if o.Router != nil {
_ = o.Router.Init(router.Register(r))
}
}
}
// Router is used to lookup routes for a service
func Router(r router.Router) Option {
return func(o *Options) {
o.Router = r
} }
} }
// Selector is used to select a route // Selector is used to select a route
func Selector(s selector.Selector) Option { func Selector(s selector.Selector) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Selector = s return options.Set(src, s, ".Selector")
} }
} }
// Backoff is used to set the backoff function used when retrying Calls // Backoff is used to set the backoff function used when retrying Calls
func Backoff(fn BackoffFunc) Option { func Backoff(fn BackoffFunc) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.CallOptions.Backoff = fn return options.Set(src, fn, ".Backoff")
}
}
// Name sets the client name
func Name(n string) Option {
return func(o *Options) {
o.Name = n
} }
} }
// Lookup sets the lookup function to use for resolving service names // Lookup sets the lookup function to use for resolving service names
func Lookup(l LookupFunc) Option { func Lookup(fn LookupFunc) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Lookup = l return options.Set(src, fn, ".Lookup")
} }
} }
// TLSConfig specifies a *tls.Config // WithCallWrapper sets the retry function to be used when re-trying.
func TLSConfig(t *tls.Config) Option { func WithCallWrapper(fn CallWrapper) options.Option {
return func(o *Options) { return func(src interface{}) error {
// set the internal tls return options.Set(src, fn, ".CallWrappers")
o.TLSConfig = t
} }
} }
// Retries sets the retry count when making the request. // Retries sets the retry count when making the request.
func Retries(i int) Option { func Retries(n int) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.CallOptions.Retries = i return options.Set(src, n, ".Retries")
} }
} }
// Retry sets the retry function to be used when re-trying. // Retry sets the retry function to be used when re-trying.
func Retry(fn RetryFunc) Option { func Retry(fn RetryFunc) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.CallOptions.Retry = fn return options.Set(src, fn, ".Retry")
} }
} }
// RequestTimeout is the request timeout. // RequestTimeout is the request timeout.
func RequestTimeout(d time.Duration) Option { func RequestTimeout(td time.Duration) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.CallOptions.RequestTimeout = d return options.Set(src, td, ".RequestTimeout")
} }
} }
// StreamTimeout sets the stream timeout // StreamTimeout sets the stream timeout
func StreamTimeout(d time.Duration) Option { func StreamTimeout(td time.Duration) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.CallOptions.StreamTimeout = d return options.Set(src, td, ".StreamTimeout")
} }
} }
// DialTimeout sets the dial timeout // DialTimeout sets the dial timeout
func DialTimeout(d time.Duration) Option { func DialTimeout(td time.Duration) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.CallOptions.DialTimeout = d return options.Set(src, td, ".DialTimeout")
}
}
// WithContextDialer pass ContextDialer to client call
func WithContextDialer(fn func(context.Context, string) (net.Conn, error)) CallOption {
return func(o *CallOptions) {
o.ContextDialer = fn
}
}
// WithContentType specifies call content type
func WithContentType(ct string) CallOption {
return func(o *CallOptions) {
o.ContentType = ct
}
}
// WithAddress sets the remote addresses to use rather than using service discovery
func WithAddress(a ...string) CallOption {
return func(o *CallOptions) {
o.Address = a
}
}
// WithBackoff is a CallOption which overrides that which
// set in Options.CallOptions
func WithBackoff(fn BackoffFunc) CallOption {
return func(o *CallOptions) {
o.Backoff = fn
}
}
// WithRetry is a CallOption which overrides that which
// set in Options.CallOptions
func WithRetry(fn RetryFunc) CallOption {
return func(o *CallOptions) {
o.Retry = fn
}
}
// WithRetries is a CallOption which overrides that which
// set in Options.CallOptions
func WithRetries(i int) CallOption {
return func(o *CallOptions) {
o.Retries = i
} }
} }
// WithResponseMetadata is a CallOption which adds metadata.Metadata to Options.CallOptions // WithResponseMetadata is a CallOption which adds metadata.Metadata to Options.CallOptions
func WithResponseMetadata(md *metadata.Metadata) CallOption { func ResponseMetadata(md *metadata.Metadata) options.Option {
return func(o *CallOptions) { return func(src interface{}) error {
o.ResponseMetadata = md return options.Set(src, md, ".ResponseMetadata")
} }
} }
// WithRequestMetadata is a CallOption which adds metadata.Metadata to Options.CallOptions // WithRequestMetadata is a CallOption which adds metadata.Metadata to Options.CallOptions
func WithRequestMetadata(md metadata.Metadata) CallOption { func RequestMetadata(md metadata.Metadata) options.Option {
return func(o *CallOptions) { return func(src interface{}) error {
o.RequestMetadata = md return options.Set(src, metadata.Copy(md), ".RequestMetadata")
} }
} }
// WithRequestTimeout is a CallOption which overrides that which // AuthToken is a CallOption which overrides the
// set in Options.CallOptions
func WithRequestTimeout(d time.Duration) CallOption {
return func(o *CallOptions) {
o.RequestTimeout = d
}
}
// WithStreamTimeout sets the stream timeout
func WithStreamTimeout(d time.Duration) CallOption {
return func(o *CallOptions) {
o.StreamTimeout = d
}
}
// WithDialTimeout is a CallOption which overrides that which
// set in Options.CallOptions
func WithDialTimeout(d time.Duration) CallOption {
return func(o *CallOptions) {
o.DialTimeout = d
}
}
// WithAuthToken is a CallOption which overrides the
// authorization header with the services own auth token // authorization header with the services own auth token
func WithAuthToken(t string) CallOption { func AuthToken(t string) options.Option {
return func(o *CallOptions) { return func(src interface{}) error {
o.AuthToken = t return options.Set(src, t, ".AuthToken")
} }
} }
// WithRouter sets the router to use for this call // Network is a CallOption which sets the network attribute
func WithRouter(r router.Router) CallOption { func Network(n string) options.Option {
return func(o *CallOptions) { return func(src interface{}) error {
o.Router = r return options.Set(src, n, ".Network")
}
}
// WithSelector sets the selector to use for this call
func WithSelector(s selector.Selector) CallOption {
return func(o *CallOptions) {
o.Selector = s
} }
} }
/*
// WithSelectOptions sets the options to pass to the selector for this call // WithSelectOptions sets the options to pass to the selector for this call
func WithSelectOptions(sops ...selector.SelectOption) CallOption { func WithSelectOptions(sops ...selector.SelectOption) options.Option {
return func(o *CallOptions) { return func(o *CallOptions) {
o.SelectOptions = sops o.SelectOptions = sops
} }
} }
*/
// StreamingRequest specifies that request is streaming // StreamingRequest specifies that request is streaming
func StreamingRequest(b bool) RequestOption { func StreamingRequest(b bool) options.Option {
return func(o *RequestOptions) { return func(src interface{}) error {
o.Stream = b return options.Set(src, b, ".Stream")
}
}
// RequestContentType specifies request content type
func RequestContentType(ct string) RequestOption {
return func(o *RequestOptions) {
o.ContentType = ct
}
}
// Hooks sets hook runs before action
func Hooks(h ...options.Hook) Option {
return func(o *Options) {
o.Hooks = append(o.Hooks, h...)
} }
} }

View File

@@ -1,47 +0,0 @@
package cluster
import (
"context"
"go.unistack.org/micro/v4/metadata"
)
// Message sent to member in cluster
type Message interface {
// Header returns message headers
Header() metadata.Metadata
// Body returns broker message may be []byte slice or some go struct or interface
Body() interface{}
}
type Node interface {
// Name returns node name
Name() string
// Address returns node address
Address() string
// Metadata returns node metadata
Metadata() metadata.Metadata
}
// Cluster interface used for cluster communication across nodes
type Cluster interface {
// Join is used to take an existing members and performing state sync
Join(ctx context.Context, addr ...string) error
// Leave broadcast a leave message and stop listeners
Leave(ctx context.Context) error
// Ping is used to probe live status of the node
Ping(ctx context.Context, node Node, payload []byte) error
// Members returns the cluster members
Members() ([]Node, error)
// Broadcast send message for all members in cluster, if filter is not nil, nodes may be filtered
// by key/value pairs
Broadcast(ctx context.Context, msg Message, filter ...string) error
// Unicast send message to single member in cluster
Unicast(ctx context.Context, node Node, msg Message) error
// Live returns cluster liveness
Live() bool
// Ready returns cluster readiness
Ready() bool
// Health returns cluster health
Health() bool
}

View File

@@ -1,8 +1,19 @@
// Package codec is an interface for encoding messages // Package codec is an interface for encoding messages
package codec package codec // import "go.unistack.org/micro/v4/codec"
import ( import (
"errors" "errors"
"io"
"go.unistack.org/micro/v4/metadata"
)
// Message types
const (
Error MessageType = iota
Request
Response
Event
) )
var ( var (
@@ -13,23 +24,65 @@ var (
) )
var ( var (
// DefaultMaxMsgSize specifies how much data codec can handle
DefaultMaxMsgSize = 1024 * 1024 * 4 // 4Mb
// DefaultCodec is the global default codec // DefaultCodec is the global default codec
DefaultCodec = NewCodec() DefaultCodec = NewCodec()
// DefaultTagName specifies struct tag name to control codec Marshal/Unmarshal // DefaultTagName specifies struct tag name to control codec Marshal/Unmarshal
DefaultTagName = "codec" DefaultTagName = "codec"
) )
// Codec encodes/decodes various types of messages. // MessageType specifies message type for codec
type MessageType int
// Codec encodes/decodes various types of messages used within micro.
// ReadHeader and ReadBody are called in pairs to read requests/responses
// from the connection. Close is called when finished with the
// connection. ReadBody may be called with a nil argument to force the
// body to be read and discarded.
type Codec interface { type Codec interface {
ReadHeader(r io.Reader, m *Message, mt MessageType) error
ReadBody(r io.Reader, v interface{}) error
Write(w io.Writer, m *Message, v interface{}) error
Marshal(v interface{}, opts ...Option) ([]byte, error) Marshal(v interface{}, opts ...Option) ([]byte, error)
Unmarshal(b []byte, v interface{}, opts ...Option) error Unmarshal(b []byte, v interface{}, opts ...Option) error
String() string String() string
} }
type CodecV2 interface { // Message represents detailed information about
Marshal(buf []byte, v interface{}, opts ...Option) ([]byte, error) // the communication, likely followed by the body.
Unmarshal(buf []byte, v interface{}, opts ...Option) error // In the case of an error, body may be nil.
String() string type Message struct {
Header metadata.Metadata
Target string
Method string
Endpoint string
Error string
ID string
Body []byte
Type MessageType
}
// NewMessage creates new codec message
func NewMessage(t MessageType) *Message {
return &Message{Type: t, Header: metadata.New(0)}
}
// MarshalAppend calls codec.Marshal(v) and returns the data appended to buf.
// If codec implements MarshalAppend, that is called instead.
func MarshalAppend(buf []byte, c Codec, v interface{}, opts ...Option) ([]byte, error) {
if nc, ok := c.(interface {
MarshalAppend([]byte, interface{}, ...Option) ([]byte, error)
}); ok {
return nc.MarshalAppend(buf, v, opts...)
}
mbuf, err := c.Marshal(v, opts...)
if err != nil {
return nil, err
}
return append(buf, mbuf...), nil
} }
// RawMessage is a raw encoded JSON value. // RawMessage is a raw encoded JSON value.
@@ -40,8 +93,6 @@ type RawMessage []byte
func (m *RawMessage) MarshalJSON() ([]byte, error) { func (m *RawMessage) MarshalJSON() ([]byte, error) {
if m == nil { if m == nil {
return []byte("null"), nil return []byte("null"), nil
} else if len(*m) == 0 {
return []byte("null"), nil
} }
return *m, nil return *m, nil
} }
@@ -54,22 +105,3 @@ func (m *RawMessage) UnmarshalJSON(data []byte) error {
*m = append((*m)[0:0], data...) *m = append((*m)[0:0], data...)
return nil return nil
} }
// MarshalYAML returns m as the JSON encoding of m.
func (m *RawMessage) MarshalYAML() ([]byte, error) {
if m == nil {
return []byte("null"), nil
} else if len(*m) == 0 {
return []byte("null"), nil
}
return *m, nil
}
// UnmarshalYAML sets *m to a copy of data.
func (m *RawMessage) UnmarshalYAML(data []byte) error {
if m == nil {
return errors.New("RawMessage UnmarshalYAML on nil pointer")
}
*m = append((*m)[0:0], data...)
return nil
}

View File

@@ -15,15 +15,6 @@ func FromContext(ctx context.Context) (Codec, bool) {
return c, ok return c, ok
} }
// MustContext returns codec from context
func MustContext(ctx context.Context) Codec {
c, ok := FromContext(ctx)
if !ok {
panic("missing codec")
}
return c
}
// NewContext put codec in context // NewContext put codec in context
func NewContext(ctx context.Context, c Codec) context.Context { func NewContext(ctx context.Context, c Codec) context.Context {
if ctx == nil { if ctx == nil {

View File

@@ -20,17 +20,6 @@ func (m *Frame) UnmarshalJSON(data []byte) error {
return m.Unmarshal(data) return m.Unmarshal(data)
} }
// MarshalYAML returns frame data
func (m *Frame) MarshalYAML() ([]byte, error) {
return m.Marshal()
}
// UnmarshalYAML set frame data
func (m *Frame) UnmarshalYAML(data []byte) error {
m.Data = append((m.Data)[0:0], data...)
return nil
}
// ProtoMessage noop func // ProtoMessage noop func
func (m *Frame) ProtoMessage() {} func (m *Frame) ProtoMessage() {}

View File

@@ -1,4 +1,4 @@
// Copyright 2021 Unistack LLC // Copyright 2021-2023 Unistack LLC
// //
// Licensed under the Apache License, Version 2.0 (the "License"); // Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License. // you may not use this file except in compliance with the License.

View File

@@ -2,14 +2,70 @@ package codec
import ( import (
"encoding/json" "encoding/json"
"io"
codecpb "go.unistack.org/micro-proto/v4/codec"
) )
type noopCodec struct { type noopCodec struct {
opts Options opts Options
} }
func (c *noopCodec) ReadHeader(conn io.Reader, m *Message, t MessageType) error {
return nil
}
func (c *noopCodec) ReadBody(conn io.Reader, b interface{}) error {
// read bytes
buf, err := io.ReadAll(conn)
if err != nil {
return err
}
if b == nil {
return nil
}
switch v := b.(type) {
case *string:
*v = string(buf)
case *[]byte:
*v = buf
case *Frame:
v.Data = buf
default:
return json.Unmarshal(buf, v)
}
return nil
}
func (c *noopCodec) Write(conn io.Writer, m *Message, b interface{}) error {
if b == nil {
return nil
}
var v []byte
switch vb := b.(type) {
case *Frame:
v = vb.Data
case string:
v = []byte(vb)
case *string:
v = []byte(*vb)
case *[]byte:
v = *vb
case []byte:
v = vb
default:
var err error
v, err = json.Marshal(vb)
if err != nil {
return err
}
}
_, err := conn.Write(v)
return err
}
func (c *noopCodec) String() string { func (c *noopCodec) String() string {
return "noop" return "noop"
} }
@@ -35,8 +91,8 @@ func (c *noopCodec) Marshal(v interface{}, opts ...Option) ([]byte, error) {
return ve, nil return ve, nil
case *Frame: case *Frame:
return ve.Data, nil return ve.Data, nil
case *codecpb.Frame: case *Message:
return ve.Data, nil return ve.Body, nil
} }
return json.Marshal(v) return json.Marshal(v)
@@ -59,8 +115,8 @@ func (c *noopCodec) Unmarshal(d []byte, v interface{}, opts ...Option) error {
case *Frame: case *Frame:
ve.Data = d ve.Data = d
return nil return nil
case *codecpb.Frame: case *Message:
ve.Data = d ve.Body = d
return nil return nil
} }

View File

@@ -23,8 +23,15 @@ type Options struct {
Context context.Context Context context.Context
// TagName specifies tag name in struct to control codec // TagName specifies tag name in struct to control codec
TagName string TagName string
// Flatten specifies that struct must be analyzed for flatten tag // MaxMsgSize specifies max messages size that reads by codec
Flatten bool MaxMsgSize int
}
// MaxMsgSize sets the max message size
func MaxMsgSize(n int) Option {
return func(o *Options) {
o.MaxMsgSize = n
}
} }
// TagName sets the codec tag name in struct // TagName sets the codec tag name in struct
@@ -34,13 +41,6 @@ func TagName(n string) Option {
} }
} }
// Flatten enables checking for flatten tag name
func Flatten(b bool) Option {
return func(o *Options) {
o.Flatten = b
}
}
// Logger sets the logger // Logger sets the logger
func Logger(l logger.Logger) Option { func Logger(l logger.Logger) Option {
return func(o *Options) { return func(o *Options) {
@@ -65,12 +65,12 @@ func Meter(m meter.Meter) Option {
// NewOptions returns new options // NewOptions returns new options
func NewOptions(opts ...Option) Options { func NewOptions(opts ...Option) Options {
options := Options{ options := Options{
Context: context.Background(), Context: context.Background(),
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
Meter: meter.DefaultMeter, Meter: meter.DefaultMeter,
Tracer: tracer.DefaultTracer, Tracer: tracer.DefaultTracer,
TagName: DefaultTagName, MaxMsgSize: DefaultMaxMsgSize,
Flatten: false, TagName: DefaultTagName,
} }
for _, o := range opts { for _, o := range opts {

View File

@@ -1,11 +1,13 @@
// Package config is an interface for dynamic configuration. // Package config is an interface for dynamic configuration.
package config package config // import "go.unistack.org/micro/v4/config"
import ( import (
"context" "context"
"errors" "errors"
"reflect" "reflect"
"time" "time"
"go.unistack.org/micro/v4/options"
) )
type Validator interface { type Validator interface {
@@ -37,26 +39,19 @@ type Config interface {
// Name returns name of config // Name returns name of config
Name() string Name() string
// Init the config // Init the config
Init(opts ...Option) error Init(opts ...options.Option) error
// Options in the config // Options in the config
Options() Options Options() Options
// Load config from sources // Load config from sources
Load(context.Context, ...LoadOption) error Load(context.Context, ...options.Option) error
// Save config to sources // Save config to sources
Save(context.Context, ...SaveOption) error Save(context.Context, ...options.Option) error
// Watch a config for changes // Watch a config for changes
Watch(context.Context, ...WatchOption) (Watcher, error) Watch(context.Context, ...options.Option) (Watcher, error)
// String returns config type name // String returns config type name
String() string String() string
} }
type (
FuncLoad func(ctx context.Context, opts ...LoadOption) error
HookLoad func(next FuncLoad) FuncLoad
FuncSave func(ctx context.Context, opts ...SaveOption) error
HookSave func(next FuncSave) FuncSave
)
// Watcher is the config watcher // Watcher is the config watcher
type Watcher interface { type Watcher interface {
// Next blocks until update happens or error returned // Next blocks until update happens or error returned
@@ -66,7 +61,7 @@ type Watcher interface {
} }
// Load loads config from config sources // Load loads config from config sources
func Load(ctx context.Context, cs []Config, opts ...LoadOption) error { func Load(ctx context.Context, cs []Config, opts ...options.Option) error {
var err error var err error
for _, c := range cs { for _, c := range cs {
if err = c.Init(); err != nil { if err = c.Init(); err != nil {
@@ -138,7 +133,7 @@ var (
return nil return nil
} }
if err := fn(ctx, c); err != nil { if err := fn(ctx, c); err != nil {
c.Options().Logger.Error(ctx, c.String()+" BeforeLoad error", err) c.Options().Logger.Error(ctx, c.String()+" BeforeLoad error "+err.Error())
if !c.Options().AllowFail { if !c.Options().AllowFail {
return err return err
} }
@@ -153,7 +148,7 @@ var (
return nil return nil
} }
if err := fn(ctx, c); err != nil { if err := fn(ctx, c); err != nil {
c.Options().Logger.Error(ctx, c.String()+" AfterLoad error", err) c.Options().Logger.Error(ctx, c.String()+" AfterLoad error "+err.Error())
if !c.Options().AllowFail { if !c.Options().AllowFail {
return err return err
} }
@@ -168,7 +163,7 @@ var (
return nil return nil
} }
if err := fn(ctx, c); err != nil { if err := fn(ctx, c); err != nil {
c.Options().Logger.Error(ctx, c.String()+" BeforeSave error", err) c.Options().Logger.Error(ctx, c.String()+" BeforeSave error "+err.Error())
if !c.Options().AllowFail { if !c.Options().AllowFail {
return err return err
} }
@@ -183,7 +178,7 @@ var (
return nil return nil
} }
if err := fn(ctx, c); err != nil { if err := fn(ctx, c); err != nil {
c.Options().Logger.Error(ctx, c.String()+" AfterSave error", err) c.Options().Logger.Error(ctx, c.String()+" AfterSave error "+err.Error())
if !c.Options().AllowFail { if !c.Options().AllowFail {
return err return err
} }
@@ -198,7 +193,7 @@ var (
return nil return nil
} }
if err := fn(ctx, c); err != nil { if err := fn(ctx, c); err != nil {
c.Options().Logger.Error(ctx, c.String()+" BeforeInit error", err) c.Options().Logger.Error(ctx, c.String()+" BeforeInit error "+err.Error())
if !c.Options().AllowFail { if !c.Options().AllowFail {
return err return err
} }
@@ -213,7 +208,7 @@ var (
return nil return nil
} }
if err := fn(ctx, c); err != nil { if err := fn(ctx, c); err != nil {
c.Options().Logger.Error(ctx, c.String()+" AfterInit error", err) c.Options().Logger.Error(ctx, c.String()+" AfterInit error "+err.Error())
if !c.Options().AllowFail { if !c.Options().AllowFail {
return err return err
} }

View File

@@ -15,15 +15,6 @@ func FromContext(ctx context.Context) (Config, bool) {
return c, ok return c, ok
} }
// MustContext returns store from context
func MustContext(ctx context.Context) Config {
c, ok := FromContext(ctx)
if !ok {
panic("missing config")
}
return c
}
// NewContext put store in context // NewContext put store in context
func NewContext(ctx context.Context, c Config) context.Context { func NewContext(ctx context.Context, c Config) context.Context {
if ctx == nil { if ctx == nil {
@@ -31,43 +22,3 @@ func NewContext(ctx context.Context, c Config) context.Context {
} }
return context.WithValue(ctx, configKey{}, c) return context.WithValue(ctx, configKey{}, c)
} }
// SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option {
return func(o *Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetSaveOption returns a function to setup a context with given value
func SetSaveOption(k, v interface{}) SaveOption {
return func(o *SaveOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetLoadOption returns a function to setup a context with given value
func SetLoadOption(k, v interface{}) LoadOption {
return func(o *LoadOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetWatchOption returns a function to setup a context with given value
func SetWatchOption(k, v interface{}) WatchOption {
return func(o *WatchOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}

View File

@@ -40,47 +40,3 @@ func TestNewContext(t *testing.T) {
t.Fatal("NewContext not works") t.Fatal("NewContext not works")
} }
} }
func TestSetOption(t *testing.T) {
type key struct{}
o := SetOption(key{}, "test")
opts := &Options{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetOption not works")
}
}
func TestSetSaveOption(t *testing.T) {
type key struct{}
o := SetSaveOption(key{}, "test")
opts := &SaveOptions{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetSaveOption not works")
}
}
func TestSetLoadOption(t *testing.T) {
type key struct{}
o := SetLoadOption(key{}, "test")
opts := &LoadOptions{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetLoadOption not works")
}
}
func TestSetWatchOption(t *testing.T) {
type key struct{}
o := SetWatchOption(key{}, "test")
opts := &WatchOptions{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetWatchOption not works")
}
}

View File

@@ -16,16 +16,14 @@ import (
) )
type defaultConfig struct { type defaultConfig struct {
funcLoad FuncLoad opts Options
funcSave FuncSave
opts Options
} }
func (c *defaultConfig) Options() Options { func (c *defaultConfig) Options() Options {
return c.opts return c.opts
} }
func (c *defaultConfig) Init(opts ...Option) error { func (c *defaultConfig) Init(opts ...options.Option) error {
for _, o := range opts { for _, o := range opts {
o(&c.opts) o(&c.opts)
} }
@@ -34,18 +32,6 @@ func (c *defaultConfig) Init(opts ...Option) error {
return err return err
} }
c.funcLoad = c.fnLoad
c.funcSave = c.fnSave
c.opts.Hooks.EachPrev(func(hook options.Hook) {
switch h := hook.(type) {
case HookLoad:
c.funcLoad = h(c.funcLoad)
case HookSave:
c.funcSave = h(c.funcSave)
}
})
if err := DefaultAfterInit(c.opts.Context, c); err != nil && !c.opts.AllowFail { if err := DefaultAfterInit(c.opts.Context, c); err != nil && !c.opts.AllowFail {
return err return err
} }
@@ -53,18 +39,12 @@ func (c *defaultConfig) Init(opts ...Option) error {
return nil return nil
} }
func (c *defaultConfig) Load(ctx context.Context, opts ...LoadOption) error { func (c *defaultConfig) Load(ctx context.Context, opts ...options.Option) error {
return c.funcLoad(ctx, opts...)
}
func (c *defaultConfig) fnLoad(ctx context.Context, opts ...LoadOption) error {
var err error
if c.opts.SkipLoad != nil && c.opts.SkipLoad(ctx, c) { if c.opts.SkipLoad != nil && c.opts.SkipLoad(ctx, c) {
return nil return nil
} }
if err = DefaultBeforeLoad(ctx, c); err != nil && !c.opts.AllowFail { if err := DefaultBeforeLoad(ctx, c); err != nil && !c.opts.AllowFail {
return err return err
} }
@@ -254,7 +234,6 @@ func fillValue(value reflect.Value, val string) error {
} }
value.Set(reflect.ValueOf(v)) value.Set(reflect.ValueOf(v))
} }
return nil return nil
} }
@@ -317,11 +296,7 @@ func fillValues(valueOf reflect.Value, tname string) error {
return nil return nil
} }
func (c *defaultConfig) Save(ctx context.Context, opts ...SaveOption) error { func (c *defaultConfig) Save(ctx context.Context, _ ...options.Option) error {
return c.funcSave(ctx, opts...)
}
func (c *defaultConfig) fnSave(ctx context.Context, opts ...SaveOption) error {
if c.opts.SkipSave != nil && c.opts.SkipSave(ctx, c) { if c.opts.SkipSave != nil && c.opts.SkipSave(ctx, c) {
return nil return nil
} }
@@ -345,19 +320,15 @@ func (c *defaultConfig) Name() string {
return c.opts.Name return c.opts.Name
} }
func (c *defaultConfig) Watch(_ context.Context, _ ...WatchOption) (Watcher, error) { func (c *defaultConfig) Watch(ctx context.Context, opts ...options.Option) (Watcher, error) {
return nil, ErrWatcherNotImplemented return nil, ErrWatcherNotImplemented
} }
// NewConfig returns new default config source // NewConfig returns new default config source
func NewConfig(opts ...Option) Config { func NewConfig(opts ...options.Option) Config {
options := NewOptions(opts...) options := NewOptions(opts...)
if len(options.StructTag) == 0 { if len(options.StructTag) == 0 {
options.StructTag = "default" options.StructTag = "default"
} }
c := &defaultConfig{opts: options} return &defaultConfig{opts: options}
c.funcLoad = c.fnLoad
c.funcSave = c.fnSave
return c
} }

View File

@@ -3,26 +3,24 @@ package config_test
import ( import (
"context" "context"
"fmt" "fmt"
"reflect"
"testing" "testing"
"time" "time"
"go.unistack.org/micro/v4/config" "go.unistack.org/micro/v4/config"
mid "go.unistack.org/micro/v4/util/id"
mtime "go.unistack.org/micro/v4/util/time" mtime "go.unistack.org/micro/v4/util/time"
) )
type cfg struct { type cfg struct {
MapValue map[string]bool `default:"key1=true,key2=false"` StringValue string `default:"string_value"`
StructValue *cfgStructValue IgnoreValue string `json:"-"`
StructValue *cfgStructValue
StringValue string `default:"string_value"` IntValue int `default:"99"`
IgnoreValue string `json:"-"` DurationValue time.Duration `default:"10s"`
UUIDValue string `default:"micro:generate uuid"` MDurationValue mtime.Duration `default:"10s"`
IDValue string `default:"micro:generate id"` MapValue map[string]bool `default:"key1=true,key2=false"`
UUIDValue string `default:"micro:generate uuid"`
DurationValue time.Duration `default:"10s"` IDValue string `default:"micro:generate id"`
MDurationValue mtime.Duration `default:"10s"`
IntValue int `default:"99"`
} }
type cfgStructValue struct { type cfgStructValue struct {
@@ -43,35 +41,6 @@ func (c *cfgStructValue) Validate() error {
return nil return nil
} }
type testHook struct {
f bool
}
func (t *testHook) Load(fn config.FuncLoad) config.FuncLoad {
return func(ctx context.Context, opts ...config.LoadOption) error {
t.f = true
return fn(ctx, opts...)
}
}
func TestHook(t *testing.T) {
h := &testHook{}
c := config.NewConfig(config.Struct(h), config.Hooks(config.HookLoad(h.Load)))
if err := c.Init(); err != nil {
t.Fatal(err)
}
if err := c.Load(context.TODO()); err != nil {
t.Fatal(err)
}
if !h.f {
t.Fatal("hook not works")
}
}
func TestDefault(t *testing.T) { func TestDefault(t *testing.T) {
ctx := context.Background() ctx := context.Background()
conf := &cfg{IntValue: 10} conf := &cfg{IntValue: 10}
@@ -114,6 +83,8 @@ func TestDefault(t *testing.T) {
if conf.IDValue == "" { if conf.IDValue == "" {
t.Fatalf("id value empty") t.Fatalf("id value empty")
} else if len(conf.IDValue) != mid.DefaultSize {
t.Fatalf("id value invalid: %s", conf.IDValue)
} }
_ = conf _ = conf
// t.Logf("%#+v\n", conf) // t.Logf("%#+v\n", conf)
@@ -134,13 +105,3 @@ func TestValidate(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
} }
func Test_SizeOf(t *testing.T) {
st := cfg{}
tVal := reflect.TypeOf(st)
for i := 0; i < tVal.NumField(); i++ {
field := tVal.Field(i)
fmt.Printf("Field: %s, Offset: %d, Size: %d\n", field.Name, field.Offset, field.Type.Size())
}
}

View File

@@ -41,23 +41,16 @@ type Options struct {
BeforeInit []func(context.Context, Config) error BeforeInit []func(context.Context, Config) error
// AfterInit contains slice of funcs that runs after Init // AfterInit contains slice of funcs that runs after Init
AfterInit []func(context.Context, Config) error AfterInit []func(context.Context, Config) error
// AllowFail flag to allow fail in config source
AllowFail bool
// SkipLoad runs only if condition returns true // SkipLoad runs only if condition returns true
SkipLoad func(context.Context, Config) bool SkipLoad func(context.Context, Config) bool
// SkipSave runs only if condition returns true // SkipSave runs only if condition returns true
SkipSave func(context.Context, Config) bool SkipSave func(context.Context, Config) bool
// Hooks can be run before/after config Save/Load
Hooks options.Hooks
// AllowFail flag to allow fail in config source
AllowFail bool
} }
// Option function signature
type Option func(o *Options)
// NewOptions new options struct with filed values // NewOptions new options struct with filed values
func NewOptions(opts ...Option) Options { func NewOptions(opts ...options.Option) Options {
options := Options{ options := Options{
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
Meter: meter.DefaultMeter, Meter: meter.DefaultMeter,
@@ -71,9 +64,6 @@ func NewOptions(opts ...Option) Options {
return options return options
} }
// LoadOption function signature
type LoadOption func(o *LoadOptions)
// LoadOptions struct // LoadOptions struct
type LoadOptions struct { type LoadOptions struct {
Struct interface{} Struct interface{}
@@ -83,7 +73,7 @@ type LoadOptions struct {
} }
// NewLoadOptions create LoadOptions struct with provided opts // NewLoadOptions create LoadOptions struct with provided opts
func NewLoadOptions(opts ...LoadOption) LoadOptions { func NewLoadOptions(opts ...options.Option) LoadOptions {
options := LoadOptions{} options := LoadOptions{}
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
@@ -92,44 +82,27 @@ func NewLoadOptions(opts ...LoadOption) LoadOptions {
} }
// LoadOverride override values when load // LoadOverride override values when load
func LoadOverride(b bool) LoadOption { func LoadOverride(b bool) options.Option {
return func(o *LoadOptions) { return func(src interface{}) error {
o.Override = b return options.Set(src, b, ".Override")
} }
} }
// LoadAppend override values when load // LoadAppend override values when load
func LoadAppend(b bool) LoadOption { func LoadAppend(b bool) options.Option {
return func(o *LoadOptions) { return func(src interface{}) error {
o.Append = b return options.Set(src, b, ".Append")
} }
} }
// LoadStruct override struct for loading
func LoadStruct(src interface{}) LoadOption {
return func(o *LoadOptions) {
o.Struct = src
}
}
// SaveOption function signature
type SaveOption func(o *SaveOptions)
// SaveOptions struct // SaveOptions struct
type SaveOptions struct { type SaveOptions struct {
Struct interface{} Struct interface{}
Context context.Context Context context.Context
} }
// SaveStruct override struct for save to config
func SaveStruct(src interface{}) SaveOption {
return func(o *SaveOptions) {
o.Struct = src
}
}
// NewSaveOptions fill SaveOptions struct // NewSaveOptions fill SaveOptions struct
func NewSaveOptions(opts ...SaveOption) SaveOptions { func NewSaveOptions(opts ...options.Option) SaveOptions {
options := SaveOptions{} options := SaveOptions{}
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
@@ -138,100 +111,65 @@ func NewSaveOptions(opts ...SaveOption) SaveOptions {
} }
// AllowFail allows config source to fail // AllowFail allows config source to fail
func AllowFail(b bool) Option { func AllowFail(b bool) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.AllowFail = b return options.Set(src, b, ".AllowFail")
} }
} }
// BeforeInit run funcs before config Init // BeforeInit run funcs before config Init
func BeforeInit(fn ...func(context.Context, Config) error) Option { func BeforeInit(fn ...func(context.Context, Config) error) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.BeforeInit = fn return options.Set(src, fn, ".BeforeInit")
} }
} }
// AfterInit run funcs after config Init // AfterInit run funcs after config Init
func AfterInit(fn ...func(context.Context, Config) error) Option { func AfterInit(fn ...func(context.Context, Config) error) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.AfterInit = fn return options.Set(src, fn, ".AfterInit")
} }
} }
// BeforeLoad run funcs before config load // BeforeLoad run funcs before config load
func BeforeLoad(fn ...func(context.Context, Config) error) Option { func BeforeLoad(fn ...func(context.Context, Config) error) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.BeforeLoad = fn return options.Set(src, fn, ".BeforeLoad")
} }
} }
// AfterLoad run funcs after config load // AfterLoad run funcs after config load
func AfterLoad(fn ...func(context.Context, Config) error) Option { func AfterLoad(fn ...func(context.Context, Config) error) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.AfterLoad = fn return options.Set(src, fn, ".AfterLoad")
} }
} }
// BeforeSave run funcs before save // BeforeSave run funcs before save
func BeforeSave(fn ...func(context.Context, Config) error) Option { func BeforeSave(fn ...func(context.Context, Config) error) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.BeforeSave = fn return options.Set(src, fn, ".BeforeSave")
} }
} }
// AfterSave run fncs after save // AfterSave run fncs after save
func AfterSave(fn ...func(context.Context, Config) error) Option { func AfterSave(fn ...func(context.Context, Config) error) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.AfterSave = fn return options.Set(src, fn, ".AfterSave")
}
}
// Context pass context
func Context(ctx context.Context) Option {
return func(o *Options) {
o.Context = ctx
}
}
// Codec sets the source codec
func Codec(c codec.Codec) Option {
return func(o *Options) {
o.Codec = c
}
}
// Logger sets the logger
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// Tracer to be used for tracing
func Tracer(t tracer.Tracer) Option {
return func(o *Options) {
o.Tracer = t
} }
} }
// Struct used as config // Struct used as config
func Struct(v interface{}) Option { func Struct(v interface{}) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Struct = v return options.Set(src, v, ".Struct")
} }
} }
// StructTag sets the struct tag that used for filling // StructTag sets the struct tag that used for filling
func StructTag(name string) Option { func StructTag(name string) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.StructTag = name return options.Set(src, name, ".StructTag")
}
}
// Name sets the name
func Name(n string) Option {
return func(o *Options) {
o.Name = n
} }
} }
@@ -249,11 +187,8 @@ type WatchOptions struct {
Coalesce bool Coalesce bool
} }
// WatchOption func signature
type WatchOption func(*WatchOptions)
// NewWatchOptions create WatchOptions struct with provided opts // NewWatchOptions create WatchOptions struct with provided opts
func NewWatchOptions(opts ...WatchOption) WatchOptions { func NewWatchOptions(opts ...options.Option) WatchOptions {
options := WatchOptions{ options := WatchOptions{
Context: context.Background(), Context: context.Background(),
MinInterval: DefaultWatcherMinInterval, MinInterval: DefaultWatcherMinInterval,
@@ -265,38 +200,20 @@ func NewWatchOptions(opts ...WatchOption) WatchOptions {
return options return options
} }
// WatchContext pass context // Coalesce controls watch event combining
func WatchContext(ctx context.Context) WatchOption { func WatchCoalesce(b bool) options.Option {
return func(o *WatchOptions) { return func(src interface{}) error {
o.Context = ctx return options.Set(src, b, ".Coalesce")
}
}
// WatchCoalesce controls watch event combining
func WatchCoalesce(b bool) WatchOption {
return func(o *WatchOptions) {
o.Coalesce = b
} }
} }
// WatchInterval specifies min and max time.Duration for pulling changes // WatchInterval specifies min and max time.Duration for pulling changes
func WatchInterval(minTime, maxTime time.Duration) WatchOption { func WatchInterval(min, max time.Duration) options.Option {
return func(o *WatchOptions) { return func(src interface{}) error {
o.MinInterval = minTime var err error
o.MaxInterval = maxTime if err = options.Set(src, min, ".MinInterval"); err == nil {
} err = options.Set(src, max, ".MaxInterval")
} }
return err
// WatchStruct overrides struct for fill
func WatchStruct(src interface{}) WatchOption {
return func(o *WatchOptions) {
o.Struct = src
}
}
// Hooks sets hook runs before action
func Hooks(h ...options.Hook) Option {
return func(o *Options) {
o.Hooks = append(o.Hooks, h...)
} }
} }

View File

@@ -1,20 +1,14 @@
// Package errors provides a way to return detailed information // Package errors provides a way to return detailed information
// for an RPC request error. The error is normally JSON encoded. // for an RPC request error. The error is normally JSON encoded.
package errors package errors // import "go.unistack.org/micro/v4/errors"
import ( import (
"bytes" "bytes"
"context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io"
"net/http" "net/http"
"strconv" "strconv"
"strings" "strings"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
) )
var ( var (
@@ -44,20 +38,6 @@ var (
ErrGatewayTimeout = &Error{Code: 504} ErrGatewayTimeout = &Error{Code: 504}
) )
const ProblemContentType = "application/problem+json"
type Problem struct {
Type string `json:"type,omitempty"`
Title string `json:"title,omitempty"`
Detail string `json:"detail,omitempty"`
Instance string `json:"instance,omitempty"`
Errors []struct {
Title string `json:"title,omitempty"`
Detail string `json:"detail,omitempty"`
} `json:"errors,omitempty"`
Status int `json:"status,omitempty"`
}
// Error type // Error type
type Error struct { type Error struct {
// ID holds error id or service, usually someting like my_service or id // ID holds error id or service, usually someting like my_service or id
@@ -276,10 +256,6 @@ func CodeIn(err interface{}, codes ...int32) bool {
// FromError try to convert go error to *Error // FromError try to convert go error to *Error
func FromError(err error) *Error { func FromError(err error) *Error {
if err == nil {
return nil
}
if verr, ok := err.(*Error); ok && verr != nil { if verr, ok := err.(*Error); ok && verr != nil {
return verr return verr
} }
@@ -364,135 +340,3 @@ func addslashes(str string) string {
} }
return buf.String() return buf.String()
} }
type retryableError struct {
err error
}
// Retryable returns error that can be retried later
func Retryable(err error) error {
return &retryableError{err: err}
}
type IsRetryableFunc func(error) bool
var (
RetrayableOracleErrors = []IsRetryableFunc{
func(err error) bool {
errmsg := err.Error()
switch {
case strings.Contains(errmsg, `ORA-`):
return true
case strings.Contains(errmsg, `can not assign`):
return true
case strings.Contains(errmsg, `can't assign`):
return true
}
return false
},
}
RetrayablePostgresErrors = []IsRetryableFunc{
func(err error) bool {
errmsg := err.Error()
switch {
case strings.Contains(errmsg, `number of field descriptions must equal number of`):
return true
case strings.Contains(errmsg, `not a pointer`):
return true
case strings.Contains(errmsg, `values, but dst struct has only`):
return true
case strings.Contains(errmsg, `struct doesn't have corresponding row field`):
return true
case strings.Contains(errmsg, `cannot find field`):
return true
case strings.Contains(errmsg, `cannot scan`) || strings.Contains(errmsg, `cannot convert`):
return true
case strings.Contains(errmsg, `failed to connect to`):
return true
}
return false
},
}
RetryableMicroErrors = []IsRetryableFunc{
func(err error) bool {
switch verr := err.(type) {
case *Error:
switch verr.Code {
case 401, 403, 408, 500, 501, 502, 503, 504:
return true
default:
return false
}
case *retryableError:
return true
}
return false
},
}
RetryableGoErrors = []IsRetryableFunc{
func(err error) bool {
switch verr := err.(type) {
case interface{ SafeToRetry() bool }:
return verr.SafeToRetry()
case interface{ Timeout() bool }:
return verr.Timeout()
}
switch {
case errors.Is(err, io.EOF), errors.Is(err, io.ErrUnexpectedEOF):
return true
case errors.Is(err, context.DeadlineExceeded):
return true
case errors.Is(err, io.ErrClosedPipe), errors.Is(err, io.ErrShortBuffer), errors.Is(err, io.ErrShortWrite):
return true
}
return false
},
}
RetryableGrpcErrors = []IsRetryableFunc{
func(err error) bool {
st, ok := status.FromError(err)
if !ok {
return false
}
switch st.Code() {
case codes.Unavailable, codes.ResourceExhausted:
return true
case codes.DeadlineExceeded:
return true
case codes.Internal:
switch {
case strings.Contains(st.Message(), `transport: received the unexpected content-type "text/html; charset=UTF-8"`):
return true
case strings.Contains(st.Message(), io.ErrUnexpectedEOF.Error()):
return true
case strings.Contains(st.Message(), `stream terminated by RST_STREAM with error code: INTERNAL_ERROR`):
return true
}
}
return false
},
}
)
// Unwrap provides error wrapping
func (e *retryableError) Unwrap() error {
return e.err
}
// Error returns the error string
func (e *retryableError) Error() string {
if e.err == nil {
return ""
}
return e.err.Error()
}
// IsRetryable checks error for ability to retry later
func IsRetryable(err error, fns ...IsRetryableFunc) bool {
for _, fn := range fns {
if ok := fn(err); ok {
return true
}
}
return false
}

View File

@@ -1,4 +1,4 @@
// Copyright 2021 Unistack LLC // Copyright 2021-2023 Unistack LLC
// //
// Licensed under the Apache License, Version 2.0 (the "License"); // Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License. // you may not use this file except in compliance with the License.

View File

@@ -2,19 +2,12 @@ package errors
import ( import (
"encoding/json" "encoding/json"
"errors" er "errors"
"fmt" "fmt"
"net/http" "net/http"
"testing" "testing"
) )
func TestIsRetrayable(t *testing.T) {
err := fmt.Errorf("ORA-")
if !IsRetryable(err, RetrayableOracleErrors...) {
t.Fatalf("IsRetrayable not works")
}
}
func TestMarshalJSON(t *testing.T) { func TestMarshalJSON(t *testing.T) {
e := InternalServerError("id", "err: %v", fmt.Errorf("err: %v", `xxx: "UNIX_TIMESTAMP": invalid identifier`)) e := InternalServerError("id", "err: %v", fmt.Errorf("err: %v", `xxx: "UNIX_TIMESTAMP": invalid identifier`))
_, err := json.Marshal(e) _, err := json.Marshal(e)
@@ -26,7 +19,7 @@ func TestMarshalJSON(t *testing.T) {
func TestEmpty(t *testing.T) { func TestEmpty(t *testing.T) {
msg := "test" msg := "test"
var err *Error var err *Error
err = FromError(errors.New(msg)) err = FromError(fmt.Errorf(msg))
if err.Detail != msg { if err.Detail != msg {
t.Fatalf("invalid error %v", err) t.Fatalf("invalid error %v", err)
} }
@@ -42,7 +35,7 @@ func TestFromError(t *testing.T) {
if merr.ID != "go.micro.test" || merr.Code != 404 { if merr.ID != "go.micro.test" || merr.Code != 404 {
t.Fatalf("invalid conversation %v != %v", err, merr) t.Fatalf("invalid conversation %v != %v", err, merr)
} }
err = errors.New(err.Error()) err = er.New(err.Error())
merr = FromError(err) merr = FromError(err)
if merr.ID != "go.micro.test" || merr.Code != 404 { if merr.ID != "go.micro.test" || merr.Code != 404 {
t.Fatalf("invalid conversation %v != %v", err, merr) t.Fatalf("invalid conversation %v != %v", err, merr)
@@ -57,7 +50,7 @@ func TestEqual(t *testing.T) {
t.Fatal("errors must be equal") t.Fatal("errors must be equal")
} }
err3 := errors.New("my test err") err3 := er.New("my test err")
if Equal(err1, err3) { if Equal(err1, err3) {
t.Fatal("errors must be not equal") t.Fatal("errors must be not equal")
} }

View File

@@ -22,13 +22,3 @@ func NewContext(ctx context.Context, f Flow) context.Context {
} }
return context.WithValue(ctx, flowKey{}, f) return context.WithValue(ctx, flowKey{}, f)
} }
// SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option {
return func(o *Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}

View File

@@ -1,5 +1,3 @@
//go:build ignore
package flow package flow
import ( import (
@@ -42,14 +40,3 @@ func TestNewContext(t *testing.T) {
t.Fatal("NewContext not works") t.Fatal("NewContext not works")
} }
} }
func TestSetOption(t *testing.T) {
type key struct{}
o := SetOption(key{}, "test")
opts := &Options{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetOption not works")
}
}

View File

@@ -1,17 +1,17 @@
//go:build ignore
package flow package flow
import ( import (
"context" "context"
"fmt" "fmt"
"path/filepath"
"sync" "sync"
"github.com/heimdalr/dag" "github.com/silas/dag"
"go.unistack.org/micro/v4/client" "go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/codec" "go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/metadata" "go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/options"
moptions "go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/store" "go.unistack.org/micro/v4/store"
"go.unistack.org/micro/v4/util/id" "go.unistack.org/micro/v4/util/id"
) )
@@ -22,7 +22,7 @@ type microFlow struct {
type microWorkflow struct { type microWorkflow struct {
opts Options opts Options
g *dag.DAG g *dag.AcyclicGraph
steps map[string]Step steps map[string]Step
id string id string
status Status status Status
@@ -34,20 +34,20 @@ func (w *microWorkflow) ID() string {
return w.id return w.id
} }
func (w *microWorkflow) Steps() ([][]Step, error) {
return w.getSteps("", false)
}
func (w *microWorkflow) Status() Status { func (w *microWorkflow) Status() Status {
return w.status return w.status
} }
func (w *microWorkflow) AppendSteps(steps ...Step) error { func (w *microWorkflow) AppendSteps(steps ...Step) error {
var err error
w.Lock() w.Lock()
defer w.Unlock()
for _, s := range steps { for _, s := range steps {
w.steps[s.String()] = s w.steps[s.String()] = s
if _, err = w.g.AddVertex(s); err != nil { w.g.Add(s)
return err
}
} }
for _, dst := range steps { for _, dst := range steps {
@@ -56,13 +56,18 @@ func (w *microWorkflow) AppendSteps(steps ...Step) error {
if !ok { if !ok {
return ErrStepNotExists return ErrStepNotExists
} }
if err = w.g.AddEdge(src.String(), dst.String()); err != nil { w.g.Connect(dag.BasicEdge(src, dst))
return err
}
} }
} }
w.g.ReduceTransitively() if err := w.g.Validate(); err != nil {
w.Unlock()
return err
}
w.g.TransitiveReduction()
w.Unlock()
return nil return nil
} }
@@ -71,11 +76,10 @@ func (w *microWorkflow) RemoveSteps(steps ...Step) error {
// TODO: handle case when some step requires or required by removed step // TODO: handle case when some step requires or required by removed step
w.Lock() w.Lock()
defer w.Unlock()
for _, s := range steps { for _, s := range steps {
delete(w.steps, s.String()) delete(w.steps, s.String())
w.g.DeleteVertex(s.String()) w.g.Remove(s)
} }
for _, dst := range steps { for _, dst := range steps {
@@ -84,34 +88,91 @@ func (w *microWorkflow) RemoveSteps(steps ...Step) error {
if !ok { if !ok {
return ErrStepNotExists return ErrStepNotExists
} }
w.g.AddEdge(src.String(), dst.String()) w.g.Connect(dag.BasicEdge(src, dst))
} }
} }
w.g.ReduceTransitively() if err := w.g.Validate(); err != nil {
w.Unlock()
return err
}
w.g.TransitiveReduction()
w.Unlock()
return nil return nil
} }
func (w *microWorkflow) getSteps(start string, reverse bool) ([][]Step, error) {
var steps [][]Step
var root dag.Vertex
var err error
fn := func(n dag.Vertex, idx int) error {
if idx == 0 {
steps = make([][]Step, 1)
steps[0] = make([]Step, 0, 1)
} else if idx >= len(steps) {
tsteps := make([][]Step, idx+1)
copy(tsteps, steps)
steps = tsteps
steps[idx] = make([]Step, 0, 1)
}
steps[idx] = append(steps[idx], n.(Step))
return nil
}
if start != "" {
var ok bool
w.RLock()
root, ok = w.steps[start]
w.RUnlock()
if !ok {
return nil, ErrStepNotExists
}
} else {
root, err = w.g.Root()
if err != nil {
return nil, err
}
}
if reverse {
err = w.g.SortedReverseDepthFirstWalk([]dag.Vertex{root}, fn)
} else {
err = w.g.SortedDepthFirstWalk([]dag.Vertex{root}, fn)
}
if err != nil {
return nil, err
}
return steps, nil
}
func (w *microWorkflow) Abort(ctx context.Context, id string) error { func (w *microWorkflow) Abort(ctx context.Context, id string) error {
workflowStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("workflows", id)) workflowStore := store.NewNamespaceStore(w.opts.Store, "workflows"+w.opts.Store.Options().Separator+id)
return workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusAborted.String())}) return workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusAborted.String())})
} }
func (w *microWorkflow) Suspend(ctx context.Context, id string) error { func (w *microWorkflow) Suspend(ctx context.Context, id string) error {
workflowStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("workflows", id)) workflowStore := store.NewNamespaceStore(w.opts.Store, "workflows"+w.opts.Store.Options().Separator+id)
return workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusSuspend.String())}) return workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusSuspend.String())})
} }
func (w *microWorkflow) Resume(ctx context.Context, id string) error { func (w *microWorkflow) Resume(ctx context.Context, id string) error {
workflowStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("workflows", id)) workflowStore := store.NewNamespaceStore(w.opts.Store, "workflows"+w.opts.Store.Options().Separator+id)
return workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusRunning.String())}) return workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusRunning.String())})
} }
func (w *microWorkflow) Execute(ctx context.Context, req *Message, opts ...ExecuteOption) (string, error) { func (w *microWorkflow) Execute(ctx context.Context, req *Message, opts ...options.Option) (string, error) {
w.Lock() w.Lock()
if !w.init { if !w.init {
w.g.ReduceTransitively() if err := w.g.Validate(); err != nil {
w.Unlock()
return "", err
}
w.g.TransitiveReduction()
w.init = true w.init = true
} }
w.Unlock() w.Unlock()
@@ -121,292 +182,176 @@ func (w *microWorkflow) Execute(ctx context.Context, req *Message, opts ...Execu
return "", err return "", err
} }
// stepStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("steps", eid)) stepStore := store.NewNamespaceStore(w.opts.Store, "steps"+w.opts.Store.Options().Separator+eid)
workflowStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("workflows", eid)) workflowStore := store.NewNamespaceStore(w.opts.Store, "workflows"+w.opts.Store.Options().Separator+eid)
options := NewExecuteOptions(opts...) options := NewExecuteOptions(opts...)
nopts := make([]ExecuteOption, 0, len(opts)+5) steps, err := w.getSteps(options.Start, options.Reverse)
if err != nil {
if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusPending.String())}); werr != nil {
w.opts.Logger.Error(w.opts.Context, "store write error", "error", werr.Error())
}
return "", err
}
var wg sync.WaitGroup
cherr := make(chan error, 1)
chstatus := make(chan Status, 1)
nctx, cancel := context.WithCancel(ctx)
defer cancel()
nopts := make([]moptions.Option, 0, len(opts)+5)
nopts = append(nopts, nopts = append(nopts,
ExecuteClient(w.opts.Client), moptions.Client(w.opts.Client),
ExecuteTracer(w.opts.Tracer), moptions.Tracer(w.opts.Tracer),
ExecuteLogger(w.opts.Logger), moptions.Logger(w.opts.Logger),
ExecuteMeter(w.opts.Meter), moptions.Meter(w.opts.Meter),
) )
nopts = append(nopts, opts...) nopts = append(nopts, opts...)
done := make(chan struct{})
if werr := workflowStore.Write(ctx, "status", &codec.Frame{Data: []byte(StatusRunning.String())}); werr != nil { if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusRunning.String())}); werr != nil {
w.opts.Logger.Error(ctx, "store error: %v", werr) w.opts.Logger.Error(w.opts.Context, "store write error", "error", werr.Error())
return eid, werr return eid, werr
} }
for idx := range steps {
var startID string for nidx := range steps[idx] {
if options.Start == "" { cstep := steps[idx][nidx]
mp := w.g.GetRoots() if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusPending.String())}); werr != nil {
if len(mp) != 1 { return eid, werr
return eid, ErrStepNotExists
}
for k := range mp {
startID = k
}
} else {
for k, v := range w.g.GetVertices() {
if v == options.Start {
startID = k
} }
} }
} }
if startID == "" { go func() {
return eid, ErrStepNotExists for idx := range steps {
} for nidx := range steps[idx] {
wStatus := &codec.Frame{}
if options.Async { if werr := workflowStore.Read(w.opts.Context, "status", wStatus); werr != nil {
go w.handleWorkflow(startID, nopts...) cherr <- werr
return eid, nil return
}
return eid, w.handleWorkflow(startID, nopts...)
}
func (w *microWorkflow) handleWorkflow(startID string, opts ...ExecuteOption) error {
w.RLock()
defer w.RUnlock()
// stepStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("steps", eid))
// workflowStore := store.NewNamespaceStore(w.opts.Store, filepath.Join("workflows", eid))
// Get IDs of all descendant vertices.
flowIDs, errDes := w.g.GetDescendants(startID)
if errDes != nil {
return errDes
}
// inputChannels provides for input channels for each of the descendant vertices (+ the start-vertex).
inputChannels := make(map[string]chan FlowResult, len(flowIDs)+1)
// Iterate vertex IDs and create an input channel for each of them and a single
// output channel for leaves. Note, this "pre-flight" is needed to ensure we
// really have an input channel regardless of how we traverse the tree and spawn
// workers.
leafCount := 0
for id := range flowIDs {
// Get all parents of this vertex.
parents, errPar := w.g.GetParents(id)
if errPar != nil {
return errPar
}
// Create a buffered input channel that has capacity for all parent results.
inputChannels[id] = make(chan FlowResult, len(parents))
if ok, err := w.g.IsLeaf(id); ok && err == nil {
leafCount += 1
}
}
// outputChannel caries the results of leaf vertices.
outputChannel := make(chan FlowResult, leafCount)
// To also process the start vertex and to have its results being passed to its
// children, add it to the vertex IDs. Also add an input channel for the start
// vertex and feed the inputs to this channel.
flowIDs[startID] = struct{}{}
inputChannels[startID] = make(chan FlowResult, len(inputs))
for _, i := range inputs {
inputChannels[startID] <- i
}
wg := sync.WaitGroup{}
// Iterate all vertex IDs (now incl. start vertex) and handle each worker (incl.
// inputs and outputs) in a separate goroutine.
for id := range flowIDs {
// Get all children of this vertex that later need to be notified. Note, we
// collect all children before the goroutine to be able to release the read
// lock as early as possible.
children, errChildren := w.g.GetChildren(id)
if errChildren != nil {
return errChildren
}
// Remember to wait for this goroutine.
wg.Add(1)
go func(id string) {
// Get this vertex's input channel.
// Note, only concurrent read here, which is fine.
c := inputChannels[id]
// Await all parent inputs and stuff them into a slice.
parentCount := cap(c)
parentResults := make([]FlowResult, parentCount)
for i := 0; i < parentCount; i++ {
parentResults[i] = <-c
}
// Execute the worker.
errWorker := callback(w.g, id, parentResults)
if errWorker != nil {
return errWorker
}
// Send this worker's FlowResult onto all children's input channels or, if it is
// a leaf (i.e. no children), send the result onto the output channel.
if len(children) > 0 {
for child := range children {
inputChannels[child] <- flowResult
} }
} else { if status := StringStatus[string(wStatus.Data)]; status != StatusRunning {
outputChannel <- flowResult chstatus <- status
} return
}
// "Sign off". if w.opts.Logger.V(logger.TraceLevel) {
wg.Done() w.opts.Logger.Trace(nctx, fmt.Sprintf("step will be executed %v", steps[idx][nidx]))
}(id) }
} cstep := steps[idx][nidx]
// nolint: nestif
// Wait for all go routines to finish. if len(cstep.Requires()) == 0 {
wg.Wait() wg.Add(1)
go func(step Step) {
// Await all leaf vertex results and stuff them into a slice. defer wg.Done()
resultCount := cap(outputChannel) if werr := stepStore.Write(ctx, step.ID()+w.opts.Store.Options().Separator+"req", req); werr != nil {
results := make([]FlowResult, resultCount)
for i := 0; i < resultCount; i++ {
results[i] = <-outputChannel
}
/*
go func() {
for idx := range steps {
for nidx := range steps[idx] {
wStatus := &codec.Frame{}
if werr := workflowStore.Read(w.opts.Context, "status", wStatus); werr != nil {
cherr <- werr
return
}
if status := StringStatus[string(wStatus.Data)]; status != StatusRunning {
chstatus <- status
return
}
if w.opts.Logger.V(logger.TraceLevel) {
w.opts.Logger.Tracef(nctx, "will be executed %v", steps[idx][nidx])
}
cstep := steps[idx][nidx]
// nolint: nestif
if len(cstep.Requires()) == 0 {
wg.Add(1)
go func(step Step) {
defer wg.Done()
if werr := stepStore.Write(ctx, filepath.Join(step.ID(), "req"), req); werr != nil {
cherr <- werr
return
}
if werr := stepStore.Write(ctx, filepath.Join(step.ID(), "status"), &codec.Frame{Data: []byte(StatusRunning.String())}); werr != nil {
cherr <- werr
return
}
rsp, serr := step.Execute(nctx, req, nopts...)
if serr != nil {
step.SetStatus(StatusFailure)
if werr := stepStore.Write(ctx, filepath.Join(step.ID(), "rsp"), serr); werr != nil && w.opts.Logger.V(logger.ErrorLevel) {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr)
}
if werr := stepStore.Write(ctx, filepath.Join(step.ID(), "status"), &codec.Frame{Data: []byte(StatusFailure.String())}); werr != nil && w.opts.Logger.V(logger.ErrorLevel) {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr)
}
cherr <- serr
return
}
if werr := stepStore.Write(ctx, filepath.Join(step.ID(), "rsp"), rsp); werr != nil {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr)
cherr <- werr
return
}
if werr := stepStore.Write(ctx, filepath.Join(step.ID(), "status"), &codec.Frame{Data: []byte(StatusSuccess.String())}); werr != nil {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr)
cherr <- werr
return
}
}(cstep)
wg.Wait()
} else {
if werr := stepStore.Write(ctx, filepath.Join(cstep.ID(), "req"), req); werr != nil {
cherr <- werr cherr <- werr
return return
} }
if werr := stepStore.Write(ctx, filepath.Join(cstep.ID(), "status"), &codec.Frame{Data: []byte(StatusRunning.String())}); werr != nil { if werr := stepStore.Write(ctx, step.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusRunning.String())}); werr != nil {
cherr <- werr cherr <- werr
return return
} }
rsp, serr := cstep.Execute(nctx, req, nopts...) rsp, serr := step.Execute(nctx, req, nopts...)
if serr != nil { if serr != nil {
cstep.SetStatus(StatusFailure) step.SetStatus(StatusFailure)
if werr := stepStore.Write(ctx, filepath.Join(cstep.ID(), "rsp"), serr); werr != nil && w.opts.Logger.V(logger.ErrorLevel) { if werr := stepStore.Write(ctx, step.ID()+w.opts.Store.Options().Separator+"rsp", serr); werr != nil && w.opts.Logger.V(logger.ErrorLevel) {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr) w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
} }
if werr := stepStore.Write(ctx, filepath.Join(cstep.ID(), "status"), &codec.Frame{Data: []byte(StatusFailure.String())}); werr != nil && w.opts.Logger.V(logger.ErrorLevel) { if werr := stepStore.Write(ctx, step.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusFailure.String())}); werr != nil && w.opts.Logger.V(logger.ErrorLevel) {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr) w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
} }
cherr <- serr cherr <- serr
return return
} }
if werr := stepStore.Write(ctx, filepath.Join(cstep.ID(), "rsp"), rsp); werr != nil { if werr := stepStore.Write(ctx, step.ID()+w.opts.Store.Options().Separator+"rsp", rsp); werr != nil {
w.opts.Logger.Errorf(ctx, "store write error: %v", werr) w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
cherr <- werr cherr <- werr
return return
} }
if werr := stepStore.Write(ctx, filepath.Join(cstep.ID(), "status"), &codec.Frame{Data: []byte(StatusSuccess.String())}); werr != nil { if werr := stepStore.Write(ctx, step.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusSuccess.String())}); werr != nil {
w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
cherr <- werr cherr <- werr
return return
} }
}(cstep)
wg.Wait()
} else {
if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"req", req); werr != nil {
cherr <- werr
return
}
if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusRunning.String())}); werr != nil {
cherr <- werr
return
}
rsp, serr := cstep.Execute(nctx, req, nopts...)
if serr != nil {
cstep.SetStatus(StatusFailure)
if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"rsp", serr); werr != nil && w.opts.Logger.V(logger.ErrorLevel) {
w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
}
if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusFailure.String())}); werr != nil && w.opts.Logger.V(logger.ErrorLevel) {
w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
}
cherr <- serr
return
}
if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"rsp", rsp); werr != nil {
w.opts.Logger.Error(ctx, "store write error", "error", werr.Error())
cherr <- werr
return
}
if werr := stepStore.Write(ctx, cstep.ID()+w.opts.Store.Options().Separator+"status", &codec.Frame{Data: []byte(StatusSuccess.String())}); werr != nil {
cherr <- werr
return
} }
} }
} }
close(done)
}()
if options.Async {
return eid, nil
} }
close(done)
}()
logger.Tracef(ctx, "wait for finish or error") if options.Async {
select { return eid, nil
case <-nctx.Done(): }
err = nctx.Err()
case cerr := <-cherr:
err = cerr
case <-done:
close(cherr)
case <-chstatus:
close(chstatus)
return eid, nil
}
switch { w.opts.Logger.Trace(ctx, "wait for finish or error")
case nctx.Err() != nil: select {
if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusAborted.String())}); werr != nil { case <-nctx.Done():
w.opts.Logger.Errorf(w.opts.Context, "store error: %v", werr) err = nctx.Err()
} case cerr := <-cherr:
case err == nil: err = cerr
if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusSuccess.String())}); werr != nil { case <-done:
w.opts.Logger.Errorf(w.opts.Context, "store error: %v", werr) close(cherr)
} case <-chstatus:
case err != nil: close(chstatus)
if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusFailure.String())}); werr != nil { return eid, nil
w.opts.Logger.Errorf(w.opts.Context, "store error: %v", werr) }
}
switch {
case nctx.Err() != nil:
if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusAborted.String())}); werr != nil {
w.opts.Logger.Error(w.opts.Context, "store write error", "error", werr.Error())
} }
*/ case err == nil:
return err if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusSuccess.String())}); werr != nil {
w.opts.Logger.Error(w.opts.Context, "store write error", "error", werr.Error())
}
case err != nil:
if werr := workflowStore.Write(w.opts.Context, "status", &codec.Frame{Data: []byte(StatusFailure.String())}); werr != nil {
w.opts.Logger.Error(w.opts.Context, "store write error", "error", werr.Error())
}
}
return eid, err
} }
// NewFlow create new flow // NewFlow create new flow
func NewFlow(opts ...Option) Flow { func NewFlow(opts ...options.Option) Flow {
options := NewOptions(opts...) options := NewOptions(opts...)
return &microFlow{opts: options} return &microFlow{opts: options}
} }
@@ -415,7 +360,7 @@ func (f *microFlow) Options() Options {
return f.opts return f.opts
} }
func (f *microFlow) Init(opts ...Option) error { func (f *microFlow) Init(opts ...options.Option) error {
for _, o := range opts { for _, o := range opts {
o(&f.opts) o(&f.opts)
} }
@@ -442,11 +387,11 @@ func (f *microFlow) WorkflowList(ctx context.Context) ([]Workflow, error) {
} }
func (f *microFlow) WorkflowCreate(ctx context.Context, id string, steps ...Step) (Workflow, error) { func (f *microFlow) WorkflowCreate(ctx context.Context, id string, steps ...Step) (Workflow, error) {
w := &microWorkflow{opts: f.opts, id: id, g: &dag.DAG{}, steps: make(map[string]Step, len(steps))} w := &microWorkflow{opts: f.opts, id: id, g: &dag.AcyclicGraph{}, steps: make(map[string]Step, len(steps))}
for _, s := range steps { for _, s := range steps {
w.steps[s.String()] = s w.steps[s.String()] = s
w.g.AddVertex(s) w.g.Add(s)
} }
for _, dst := range steps { for _, dst := range steps {
@@ -455,11 +400,14 @@ func (f *microFlow) WorkflowCreate(ctx context.Context, id string, steps ...Step
if !ok { if !ok {
return nil, ErrStepNotExists return nil, ErrStepNotExists
} }
w.g.AddEdge(src.String(), dst.String()) w.g.Connect(dag.BasicEdge(src, dst))
} }
} }
w.g.ReduceTransitively() if err := w.g.Validate(); err != nil {
return nil, err
}
w.g.TransitiveReduction()
w.init = true w.init = true
@@ -541,17 +489,17 @@ func (s *microCallStep) SetStatus(status Status) {
s.status = status s.status = status
} }
func (s *microCallStep) Execute(ctx context.Context, req *Message, opts ...ExecuteOption) (*Message, error) { func (s *microCallStep) Execute(ctx context.Context, req *Message, opts ...options.Option) (*Message, error) {
options := NewExecuteOptions(opts...) options := NewExecuteOptions(opts...)
if options.Client == nil { if options.Client == nil {
return nil, ErrMissingClient return nil, ErrMissingClient
} }
rsp := &codec.Frame{} rsp := &codec.Frame{}
copts := []client.CallOption{client.WithRetries(0)} copts := []moptions.Option{client.Retries(0)}
if options.Timeout > 0 { if options.Timeout > 0 {
copts = append(copts, copts = append(copts,
client.WithRequestTimeout(options.Timeout), client.RequestTimeout(options.Timeout),
client.WithDialTimeout(options.Timeout)) client.DialTimeout(options.Timeout))
} }
nctx := metadata.NewOutgoingContext(ctx, req.Header) nctx := metadata.NewOutgoingContext(ctx, req.Header)
err := options.Client.Call(nctx, options.Client.NewRequest(s.service, s.method, &codec.Frame{Data: req.Body}), rsp, copts...) err := options.Client.Call(nctx, options.Client.NewRequest(s.service, s.method, &codec.Frame{Data: req.Body}), rsp, copts...)
@@ -624,18 +572,18 @@ func (s *microPublishStep) SetStatus(status Status) {
s.status = status s.status = status
} }
func (s *microPublishStep) Execute(ctx context.Context, req *Message, opts ...ExecuteOption) (*Message, error) { func (s *microPublishStep) Execute(ctx context.Context, req *Message, opts ...options.Option) (*Message, error) {
return nil, nil return nil, nil
} }
// NewCallStep create new step with client.Call // NewCallStep create new step with client.Call
func NewCallStep(service string, name string, method string, opts ...StepOption) Step { func NewCallStep(service string, name string, method string, opts ...options.Option) Step {
options := NewStepOptions(opts...) options := NewStepOptions(opts...)
return &microCallStep{service: service, method: name + "." + method, opts: options} return &microCallStep{service: service, method: name + "." + method, opts: options}
} }
// NewPublishStep create new step with client.Publish // NewPublishStep create new step with client.Publish
func NewPublishStep(topic string, opts ...StepOption) Step { func NewPublishStep(topic string, opts ...options.Option) Step {
options := NewStepOptions(opts...) options := NewStepOptions(opts...)
return &microPublishStep{topic: topic, opts: options} return &microPublishStep{topic: topic, opts: options}
} }

View File

@@ -8,6 +8,7 @@ import (
"sync/atomic" "sync/atomic"
"go.unistack.org/micro/v4/metadata" "go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/options"
) )
var ( var (
@@ -51,7 +52,7 @@ type Step interface {
// Endpoint returns rpc endpoint service_name.service_method or broker topic // Endpoint returns rpc endpoint service_name.service_method or broker topic
Endpoint() string Endpoint() string
// Execute step run // Execute step run
Execute(ctx context.Context, req *Message, opts ...ExecuteOption) (*Message, error) Execute(ctx context.Context, req *Message, opts ...options.Option) (*Message, error)
// Requires returns dependent steps // Requires returns dependent steps
Requires() []string Requires() []string
// Options returns step options // Options returns step options
@@ -118,13 +119,15 @@ type Workflow interface {
// ID returns id of the workflow // ID returns id of the workflow
ID() string ID() string
// Execute workflow with args, return execution id and error // Execute workflow with args, return execution id and error
Execute(ctx context.Context, req *Message, opts ...ExecuteOption) (string, error) Execute(ctx context.Context, req *Message, opts ...options.Option) (string, error)
// RemoveSteps remove steps from workflow // RemoveSteps remove steps from workflow
RemoveSteps(steps ...Step) error RemoveSteps(steps ...Step) error
// AppendSteps append steps to workflow // AppendSteps append steps to workflow
AppendSteps(steps ...Step) error AppendSteps(steps ...Step) error
// Status returns workflow status // Status returns workflow status
Status() Status Status() Status
// Steps returns steps slice where parallel steps returned on the same level
Steps() ([][]Step, error)
// Suspend suspends execution // Suspend suspends execution
Suspend(ctx context.Context, id string) error Suspend(ctx context.Context, id string) error
// Resume resumes execution // Resume resumes execution
@@ -138,7 +141,7 @@ type Flow interface {
// Options returns options // Options returns options
Options() Options Options() Options
// Init initialize // Init initialize
Init(...Option) error Init(...options.Option) error
// WorkflowCreate creates new workflow with specific id and steps // WorkflowCreate creates new workflow with specific id and steps
WorkflowCreate(ctx context.Context, id string, steps ...Step) (Workflow, error) WorkflowCreate(ctx context.Context, id string, steps ...Step) (Workflow, error)
// WorkflowSave saves workflow // WorkflowSave saves workflow

View File

@@ -7,13 +7,11 @@ import (
"go.unistack.org/micro/v4/client" "go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/meter" "go.unistack.org/micro/v4/meter"
"go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/store" "go.unistack.org/micro/v4/store"
"go.unistack.org/micro/v4/tracer" "go.unistack.org/micro/v4/tracer"
) )
// Option func
type Option func(*Options)
// Options server struct // Options server struct
type Options struct { type Options struct {
// Context holds the external options and can be used for flow shutdown // Context holds the external options and can be used for flow shutdown
@@ -31,7 +29,7 @@ type Options struct {
} }
// NewOptions returns new options struct with default or passed values // NewOptions returns new options struct with default or passed values
func NewOptions(opts ...Option) Options { func NewOptions(opts ...options.Option) Options {
options := Options{ options := Options{
Context: context.Background(), Context: context.Background(),
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
@@ -47,66 +45,12 @@ func NewOptions(opts ...Option) Options {
return options return options
} }
// Logger sets the logger option
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// Meter sets the meter option
func Meter(m meter.Meter) Option {
return func(o *Options) {
o.Meter = m
}
}
// Client to use for sync/async communication
func Client(c client.Client) Option {
return func(o *Options) {
o.Client = c
}
}
// Context specifies a context for the service.
// Can be used to signal shutdown of the flow
// or can be used for extra option values.
func Context(ctx context.Context) Option {
return func(o *Options) {
o.Context = ctx
}
}
// Tracer mechanism for distributed tracking
func Tracer(t tracer.Tracer) Option {
return func(o *Options) {
o.Tracer = t
}
}
// Store used for intermediate results
func Store(s store.Store) Option {
return func(o *Options) {
o.Store = s
}
}
// WorkflowOption func signature
type WorkflowOption func(*WorkflowOptions)
// WorkflowOptions holds workflow options // WorkflowOptions holds workflow options
type WorkflowOptions struct { type WorkflowOptions struct {
Context context.Context Context context.Context
ID string ID string
} }
// WorkflowID set workflow id
func WorkflowID(id string) WorkflowOption {
return func(o *WorkflowOptions) {
o.ID = id
}
}
// ExecuteOptions holds execute options // ExecuteOptions holds execute options
type ExecuteOptions struct { type ExecuteOptions struct {
// Client holds the client.Client // Client holds the client.Client
@@ -123,64 +67,28 @@ type ExecuteOptions struct {
Start string Start string
// Timeout for execution // Timeout for execution
Timeout time.Duration Timeout time.Duration
// Reverse execution
Reverse bool
// Async enables async execution // Async enables async execution
Async bool Async bool
} }
// ExecuteOption func signature // Reverse says that dag must be run in reverse order
type ExecuteOption func(*ExecuteOptions) func Reverse(b bool) options.Option {
return func(src interface{}) error {
// ExecuteClient pass client.Client to ExecuteOption return options.Set(src, b, ".Reverse")
func ExecuteClient(c client.Client) ExecuteOption {
return func(o *ExecuteOptions) {
o.Client = c
} }
} }
// ExecuteTracer pass tracer.Tracer to ExecuteOption // Async says that caller does not wait for execution complete
func ExecuteTracer(t tracer.Tracer) ExecuteOption { func Async(b bool) options.Option {
return func(o *ExecuteOptions) { return func(src interface{}) error {
o.Tracer = t return options.Set(src, b, ".Async")
}
}
// ExecuteLogger pass logger.Logger to ExecuteOption
func ExecuteLogger(l logger.Logger) ExecuteOption {
return func(o *ExecuteOptions) {
o.Logger = l
}
}
// ExecuteMeter pass meter.Meter to ExecuteOption
func ExecuteMeter(m meter.Meter) ExecuteOption {
return func(o *ExecuteOptions) {
o.Meter = m
}
}
// ExecuteContext pass context.Context ot ExecuteOption
func ExecuteContext(ctx context.Context) ExecuteOption {
return func(o *ExecuteOptions) {
o.Context = ctx
}
}
// ExecuteTimeout pass timeout time.Duration for execution
func ExecuteTimeout(td time.Duration) ExecuteOption {
return func(o *ExecuteOptions) {
o.Timeout = td
}
}
// ExecuteAsync says that caller does not wait for execution complete
func ExecuteAsync(b bool) ExecuteOption {
return func(o *ExecuteOptions) {
o.Async = b
} }
} }
// NewExecuteOptions create new ExecuteOptions struct // NewExecuteOptions create new ExecuteOptions struct
func NewExecuteOptions(opts ...ExecuteOption) ExecuteOptions { func NewExecuteOptions(opts ...options.Option) ExecuteOptions {
options := ExecuteOptions{ options := ExecuteOptions{
Client: client.DefaultClient, Client: client.DefaultClient,
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
@@ -202,11 +110,8 @@ type StepOptions struct {
Requires []string Requires []string
} }
// StepOption func signature
type StepOption func(*StepOptions)
// NewStepOptions create new StepOptions struct // NewStepOptions create new StepOptions struct
func NewStepOptions(opts ...StepOption) StepOptions { func NewStepOptions(opts ...options.Option) StepOptions {
options := StepOptions{ options := StepOptions{
Context: context.Background(), Context: context.Background(),
} }
@@ -216,23 +121,23 @@ func NewStepOptions(opts ...StepOption) StepOptions {
return options return options
} }
// StepID sets the step id for dag // Requires specifies required steps
func StepID(id string) StepOption { func Requires(steps ...string) options.Option {
return func(o *StepOptions) { return func(src interface{}) error {
o.ID = id return options.Set(src, steps, ".Requires")
} }
} }
// StepRequires specifies required steps // Fallback set the step to run on error
func StepRequires(steps ...string) StepOption { func Fallback(step string) options.Option {
return func(o *StepOptions) { return func(src interface{}) error {
o.Requires = steps return options.Set(src, step, ".Fallback")
} }
} }
// StepFallback set the step to run on error // ID sets the step ID
func StepFallback(step string) StepOption { func StepID(id string) options.Option {
return func(o *StepOptions) { return func(src interface{}) error {
o.Fallback = step return options.Set(src, id, ".ID")
} }
} }

View File

@@ -32,7 +32,7 @@ type fsm struct {
// NewFSM creates a new finite state machine having the specified initial state // NewFSM creates a new finite state machine having the specified initial state
// with specified options // with specified options
func NewFSM(opts ...Option) FSM { func NewFSM(opts ...Option) *fsm {
return &fsm{ return &fsm{
statesMap: map[string]StateFunc{}, statesMap: map[string]StateFunc{},
opts: NewOptions(opts...), opts: NewOptions(opts...),

View File

@@ -1,4 +1,4 @@
package fsm package fsm // import "go.unistack.org/micro/v4/fsm"
import ( import (
"context" "context"

View File

@@ -17,7 +17,7 @@ func TestFSMStart(t *testing.T) {
wrapper := func(next StateFunc) StateFunc { wrapper := func(next StateFunc) StateFunc {
return func(sctx context.Context, s State, opts ...StateOption) (State, error) { return func(sctx context.Context, s State, opts ...StateOption) (State, error) {
sctx = logger.NewContext(sctx, logger.DefaultLogger.Fields("state", s.Name())) sctx = logger.NewContext(sctx, logger.Attrs("state", s.Name()))
return next(sctx, s, opts...) return next(sctx, s, opts...)
} }
} }

35
go.mod
View File

@@ -1,34 +1,21 @@
module go.unistack.org/micro/v4 module go.unistack.org/micro/v4
go 1.22.0 go 1.20
require ( require (
dario.cat/mergo v1.0.1 dario.cat/mergo v1.0.0
github.com/DATA-DOG/go-sqlmock v1.5.2 github.com/DATA-DOG/go-sqlmock v1.5.0
github.com/KimMachineGun/automemlimit v0.7.0 github.com/google/uuid v1.3.1
github.com/goccy/go-yaml v1.17.1
github.com/google/uuid v1.6.0
github.com/matoous/go-nanoid v1.5.1
github.com/patrickmn/go-cache v2.1.0+incompatible github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5 github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5
github.com/spf13/cast v1.7.1 golang.org/x/sync v0.3.0
github.com/stretchr/testify v1.10.0 golang.org/x/sys v0.12.0
go.uber.org/atomic v1.11.0 google.golang.org/grpc v1.58.2
go.uber.org/automaxprocs v1.6.0 google.golang.org/protobuf v1.31.0
go.unistack.org/micro-proto/v4 v4.1.0
golang.org/x/sync v0.10.0
google.golang.org/grpc v1.69.4
google.golang.org/protobuf v1.36.3
) )
require ( require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/golang/protobuf v1.5.3 // indirect
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect golang.org/x/net v0.15.0 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 // indirect
github.com/rogpeppe/go-internal v1.13.1 // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sys v0.29.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
) )

89
go.sum
View File

@@ -1,69 +1,34 @@
dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s= dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk= dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU= github.com/DATA-DOG/go-sqlmock v1.5.0 h1:Shsta01QNfFxHCfpW6YH2STWB0MudeXXEWMr20OEh60=
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU= github.com/DATA-DOG/go-sqlmock v1.5.0/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/KimMachineGun/automemlimit v0.7.0 h1:7G06p/dMSf7G8E6oq+f2uOPuVncFyIlDI/pBWK49u88= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/KimMachineGun/automemlimit v0.7.0/go.mod h1:QZxpHaGOQoYvFhv/r4u3U0JTC2ZcOwbSr11UZF46UBM= github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
github.com/goccy/go-yaml v1.17.1 h1:LI34wktB2xEE3ONG/2Ar54+/HJVBriAGJ55PHls4YuY= github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/goccy/go-yaml v1.17.1/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/matoous/go-nanoid v1.5.1 h1:aCjdvTyO9LLnTIi0fgdXhOPPvOHjpXN6Ik9DaNjIct4=
github.com/matoous/go-nanoid v1.5.1/go.mod h1:zyD2a71IubI24efhpvkJz+ZwfwagzgSO6UNiFsZKN7U=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc= github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ= github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0=
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g=
github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5 h1:G/FZtUu7a6NTWl3KUHMV9jkLAh/Rvtf03NWMHaEDl+E= github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5 h1:G/FZtUu7a6NTWl3KUHMV9jkLAh/Rvtf03NWMHaEDl+E=
github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5/go.mod h1:7RTUFBdIRC9nZ7/3RyRNH1bdqIShrDejd1YbLwgPS+I= github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5/go.mod h1:7RTUFBdIRC9nZ7/3RyRNH1bdqIShrDejd1YbLwgPS+I=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y= golang.org/x/net v0.15.0 h1:ugBLEUaxABaB5AJqW9enI0ACdci2RUd4eP51NTBvuJ8=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo= golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= golang.org/x/sys v0.12.0 h1:CM0HF96J0hcLAwsHPJZjfdNzs0gftsLfgKt57wWHJ0o=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs= golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
go.unistack.org/micro-proto/v4 v4.1.0 h1:qPwL2n/oqh9RE3RTTDgt28XK3QzV597VugQPaw9lKUk= google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 h1:bVf09lpb+OJbByTj913DRJioFFAjf/ZGxEz7MajTp2U=
go.unistack.org/micro-proto/v4 v4.1.0/go.mod h1:ArmK7o+uFvxSY3dbJhKBBX4Pm1rhWdLEFf3LxBrMtec= google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98/go.mod h1:TUfxEVdsvPg18p6AslUXFoLdpED4oBnGwyqk3dV1XzM=
golang.org/x/net v0.34.0 h1:Mb7Mrk043xzHgnRM88suvJFwzVrRfHEHJEl5/71CKw0= google.golang.org/grpc v1.58.2 h1:SXUpjxeVF3FKrTYQI4f4KvbGD5u2xccdYdurwowix5I=
golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k= google.golang.org/grpc v1.58.2/go.mod h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484 h1:Z7FRVJPSMaHQxD0uXU8WdgFh8PseLM8Q8NzhnpMrBhQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484/go.mod h1:lcTa1sDdWEIHMWlITnIczmw5w60CF9ffkb8Z+DVmmjA=
google.golang.org/grpc v1.69.4 h1:MF5TftSMkd8GLw/m0KM6V8CMOCY6NZ1NQDPGFgbTt4A=
google.golang.org/grpc v1.69.4/go.mod h1:vyjdE6jLBI76dgpDojsFGNaHlxdjXN9ghpnd2o7JGZ4=
google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=
google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -1,117 +0,0 @@
package metadata
import (
"context"
"go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/server"
)
type wrapper struct {
keys []string
client.Client
}
func NewClientWrapper(keys ...string) client.Wrapper {
return func(c client.Client) client.Client {
handler := &wrapper{
Client: c,
keys: keys,
}
return handler
}
}
func NewClientCallWrapper(keys ...string) client.CallWrapper {
return func(fn client.CallFunc) client.CallFunc {
return func(ctx context.Context, addr string, req client.Request, rsp interface{}, opts client.CallOptions) error {
if keys == nil {
return fn(ctx, addr, req, rsp, opts)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return fn(ctx, addr, req, rsp, opts)
}
}
}
func (w *wrapper) Call(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
if w.keys == nil {
return w.Client.Call(ctx, req, rsp, opts...)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range w.keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return w.Client.Call(ctx, req, rsp, opts...)
}
func (w *wrapper) Stream(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
if w.keys == nil {
return w.Client.Stream(ctx, req, opts...)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range w.keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return w.Client.Stream(ctx, req, opts...)
}
func NewServerHandlerWrapper(keys ...string) server.HandlerWrapper {
return func(fn server.HandlerFunc) server.HandlerFunc {
return func(ctx context.Context, req server.Request, rsp interface{}) error {
if keys == nil {
return fn(ctx, req, rsp)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return fn(ctx, req, rsp)
}
}
}

View File

@@ -1,63 +0,0 @@
package recovery
import (
"context"
"fmt"
"go.unistack.org/micro/v4/errors"
"go.unistack.org/micro/v4/server"
)
func NewOptions(opts ...Option) Options {
options := Options{
ServerHandlerFn: DefaultServerHandlerFn,
}
for _, o := range opts {
o(&options)
}
return options
}
type Options struct {
ServerHandlerFn func(context.Context, server.Request, interface{}, error) error
}
type Option func(*Options)
func ServerHandlerFunc(fn func(context.Context, server.Request, interface{}, error) error) Option {
return func(o *Options) {
o.ServerHandlerFn = fn
}
}
var DefaultServerHandlerFn = func(ctx context.Context, req server.Request, rsp interface{}, err error) error {
return errors.BadRequest("", "%v", err)
}
var Hook = NewHook()
type hook struct {
opts Options
}
func NewHook(opts ...Option) *hook {
return &hook{opts: NewOptions(opts...)}
}
func (w *hook) ServerHandler(next server.FuncHandler) server.FuncHandler {
return func(ctx context.Context, req server.Request, rsp interface{}) (err error) {
defer func() {
r := recover()
switch verr := r.(type) {
case nil:
return
case error:
err = w.opts.ServerHandlerFn(ctx, req, rsp, verr)
default:
err = w.opts.ServerHandlerFn(ctx, req, rsp, fmt.Errorf("%v", r))
}
}()
err = next(ctx, req, rsp)
return err
}
}

View File

@@ -1,103 +0,0 @@
package requestid
import (
"context"
"net/textproto"
"go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/server"
"go.unistack.org/micro/v4/util/id"
)
type XRequestIDKey struct{}
// DefaultMetadataKey contains metadata key
var DefaultMetadataKey = textproto.CanonicalMIMEHeaderKey("x-request-id")
// DefaultMetadataFunc wil be used if user not provide own func to fill metadata
var DefaultMetadataFunc = func(ctx context.Context) (context.Context, error) {
var xid string
cid, cok := ctx.Value(XRequestIDKey{}).(string)
if cok && cid != "" {
xid = cid
}
imd, iok := metadata.FromIncomingContext(ctx)
if !iok || imd == nil {
imd = metadata.New(1)
ctx = metadata.NewIncomingContext(ctx, imd)
}
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(1)
ctx = metadata.NewOutgoingContext(ctx, omd)
}
if xid == "" {
xid = imd.GetJoined(DefaultMetadataKey)
if xid == "" {
xid = omd.GetJoined(DefaultMetadataKey)
}
}
if xid == "" {
var err error
xid, err = id.New()
if err != nil {
return ctx, err
}
}
if !cok {
ctx = context.WithValue(ctx, XRequestIDKey{}, xid)
}
if !iok {
imd.Set(DefaultMetadataKey, xid)
}
if !ook {
omd.Set(DefaultMetadataKey, xid)
}
return ctx, nil
}
type hook struct{}
func NewHook() *hook {
return &hook{}
}
func (w *hook) ServerHandler(next server.FuncHandler) server.FuncHandler {
return func(ctx context.Context, req server.Request, rsp interface{}) error {
var err error
if ctx, err = DefaultMetadataFunc(ctx); err != nil {
return err
}
return next(ctx, req, rsp)
}
}
func (w *hook) ClientCall(next client.FuncCall) client.FuncCall {
return func(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
var err error
if ctx, err = DefaultMetadataFunc(ctx); err != nil {
return err
}
return next(ctx, req, rsp, opts...)
}
}
func (w *hook) ClientStream(next client.FuncStream) client.FuncStream {
return func(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
var err error
if ctx, err = DefaultMetadataFunc(ctx); err != nil {
return nil, err
}
return next(ctx, req, opts...)
}
}

View File

@@ -1,34 +0,0 @@
package requestid
import (
"context"
"slices"
"testing"
"go.unistack.org/micro/v4/metadata"
)
func TestDefaultMetadataFunc(t *testing.T) {
ctx := context.TODO()
nctx, err := DefaultMetadataFunc(ctx)
if err != nil {
t.Fatalf("%v", err)
}
imd, ok := metadata.FromIncomingContext(nctx)
if !ok {
t.Fatalf("md missing in incoming context")
}
omd, ok := metadata.FromOutgoingContext(nctx)
if !ok {
t.Fatalf("md missing in outgoing context")
}
iv := imd.Get(DefaultMetadataKey)
ov := omd.Get(DefaultMetadataKey)
if !slices.Equal(iv, ov) {
t.Fatalf("missing metadata key value %v != %v", iv, ov)
}
}

View File

@@ -1,51 +0,0 @@
package sql
import (
"database/sql/driver"
"errors"
"fmt"
"runtime"
)
//go:generate sh -c "go run gen.go > wrap_gen.go"
// namedValueToValue converts driver arguments of NamedValue format to Value format. Implemented in the same way as in
// database/sql ctxutil.go.
func namedValueToValue(named []driver.NamedValue) ([]driver.Value, error) {
dargs := make([]driver.Value, len(named))
for n, param := range named {
if len(param.Name) > 0 {
return nil, errors.New("sql: driver does not support the use of Named Parameters")
}
dargs[n] = param.Value
}
return dargs, nil
}
// namedValueToLabels convert driver arguments to interface{} slice
func namedValueToLabels(named []driver.NamedValue) []interface{} {
largs := make([]interface{}, 0, len(named)*2)
var name string
for _, param := range named {
if param.Name != "" {
name = param.Name
} else {
name = fmt.Sprintf("$%d", param.Ordinal)
}
largs = append(largs, fmt.Sprintf("%s=%v", name, param.Value))
}
return largs
}
// getCallerName get the name of the function A where A() -> B() -> GetFunctionCallerName()
func getCallerName() string {
pc, _, _, ok := runtime.Caller(3)
details := runtime.FuncForPC(pc)
var callerName string
if ok && details != nil {
callerName = details.Name()
} else {
callerName = labelUnknown
}
return callerName
}

View File

@@ -1,467 +0,0 @@
package sql
import (
"context"
"database/sql/driver"
"fmt"
"time"
"go.unistack.org/micro/v4/hooks/requestid"
"go.unistack.org/micro/v4/tracer"
)
var (
_ driver.Conn = (*wrapperConn)(nil)
_ driver.ConnBeginTx = (*wrapperConn)(nil)
_ driver.ConnPrepareContext = (*wrapperConn)(nil)
_ driver.Pinger = (*wrapperConn)(nil)
_ driver.Validator = (*wrapperConn)(nil)
_ driver.Queryer = (*wrapperConn)(nil) // nolint:staticcheck
_ driver.QueryerContext = (*wrapperConn)(nil)
_ driver.Execer = (*wrapperConn)(nil) // nolint:staticcheck
_ driver.ExecerContext = (*wrapperConn)(nil)
// _ driver.Connector
// _ driver.Driver
// _ driver.DriverContext
)
// wrapperConn defines a wrapper for driver.Conn
type wrapperConn struct {
d *wrapperDriver
dname string
conn driver.Conn
opts Options
ctx context.Context
//span tracer.Span
}
// Close implements driver.Conn Close
func (w *wrapperConn) Close() error {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Close"}
ts := time.Now()
err := w.conn.Close()
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Close", getCallerName(), td, err)...)
}
*/
return err
}
// Begin implements driver.Conn Begin
func (w *wrapperConn) Begin() (driver.Tx, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
labels := []string{labelMethod, "Begin"}
ts := time.Now()
tx, err := w.conn.Begin() // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Begin", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Begin", getCallerName(), td, err)...)
}
*/
return &wrapperTx{tx: tx, opts: w.opts, ctx: ctx}, nil
}
// BeginTx implements driver.ConnBeginTx BeginTx
func (w *wrapperConn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
name := getQueryName(ctx)
nctx, span := w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
span.AddLabels("db.method", "BeginTx")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "BeginTx", labelQuery, name}
connBeginTx, ok := w.conn.(driver.ConnBeginTx)
if !ok {
return w.Begin()
}
ts := time.Now()
tx, err := connBeginTx.BeginTx(nctx, opts)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "BeginTx", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "BeginTx", getCallerName(), td, err)...)
}
*/
return &wrapperTx{tx: tx, opts: w.opts, ctx: ctx, span: span}, nil
}
// Prepare implements driver.Conn Prepare
func (w *wrapperConn) Prepare(query string) (driver.Stmt, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Prepare", labelQuery, getCallerName()}
ts := time.Now()
stmt, err := w.conn.Prepare(query)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Prepare", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Prepare", getCallerName(), td, err)...)
}
*/
return wrapStmt(stmt, query, w.opts), nil
}
// PrepareContext implements driver.ConnPrepareContext PrepareContext
func (w *wrapperConn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "PrepareContext")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "PrepareContext", labelQuery, name}
conn, ok := w.conn.(driver.ConnPrepareContext)
if !ok {
return w.Prepare(query)
}
ts := time.Now()
stmt, err := conn.PrepareContext(nctx, query)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "PrepareContext", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "PrepareContext", getCallerName(), td, err)...)
}
*/
return wrapStmt(stmt, query, w.opts), nil
}
// Exec implements driver.Execer Exec
func (w *wrapperConn) Exec(query string, args []driver.Value) (driver.Result, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Exec", labelQuery, getCallerName()}
// nolint:staticcheck
conn, ok := w.conn.(driver.Execer)
if !ok {
return nil, driver.ErrSkip
}
ts := time.Now()
res, err := conn.Exec(query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Exec", getCallerName(), td, err)...)
}
*/
return res, err
}
// Exec implements driver.StmtExecContext ExecContext
func (w *wrapperConn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "ExecContext")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
labels := []string{labelMethod, "ExecContext", labelQuery, name}
conn, ok := w.conn.(driver.ExecerContext)
if !ok {
// nolint:staticcheck
return nil, driver.ErrSkip
}
ts := time.Now()
res, err := conn.ExecContext(nctx, query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", getCallerName(), td, err)...)
}
*/
return res, err
}
// Ping implements driver.Pinger Ping
func (w *wrapperConn) Ping(ctx context.Context) error {
conn, ok := w.conn.(driver.Pinger)
if !ok {
// fallback path to check db alive
pc, err := w.d.Open(w.dname)
if err != nil {
return err
}
return pc.Close()
}
var nctx context.Context //nolint:gosimple
nctx = ctx
/*
var span tracer.Span
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "Ping")
defer span.Finish()
*/
labels := []string{labelMethod, "Ping"}
ts := time.Now()
err := conn.Ping(nctx)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
// span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Ping", getCallerName(), td, err)...)
}
*/
return err
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
return nil
}
// Query implements driver.Queryer Query
func (w *wrapperConn) Query(query string, args []driver.Value) (driver.Rows, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
// nolint:staticcheck
conn, ok := w.conn.(driver.Queryer)
if !ok {
return nil, driver.ErrSkip
}
labels := []string{labelMethod, "Query", labelQuery, getCallerName()}
ts := time.Now()
rows, err := conn.Query(query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Query", getCallerName(), td, err)...)
}
*/
return rows, err
}
// QueryContext implements Driver.QueryerContext QueryContext
func (w *wrapperConn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "QueryContext")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
labels := []string{labelMethod, "QueryContext", labelQuery, name}
conn, ok := w.conn.(driver.QueryerContext)
if !ok {
return nil, driver.ErrSkip
}
ts := time.Now()
rows, err := conn.QueryContext(nctx, query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", getCallerName(), td, err)...)
}
*/
return rows, err
}
// CheckNamedValue implements driver.NamedValueChecker
func (w *wrapperConn) CheckNamedValue(v *driver.NamedValue) error {
s, ok := w.conn.(driver.NamedValueChecker)
if !ok {
return driver.ErrSkip
}
return s.CheckNamedValue(v)
}
// IsValid implements driver.Validator
func (w *wrapperConn) IsValid() bool {
v, ok := w.conn.(driver.Validator)
if !ok {
return w.conn != nil
}
return v.IsValid()
}
func (w *wrapperConn) ResetSession(ctx context.Context) error {
s, ok := w.conn.(driver.SessionResetter)
if !ok {
return driver.ErrSkip
}
return s.ResetSession(ctx)
}

View File

@@ -1,94 +0,0 @@
package sql
import (
"context"
"database/sql/driver"
"time"
)
var (
// _ driver.DriverContext = (*wrapperDriver)(nil)
// _ driver.Connector = (*wrapperDriver)(nil)
)
/*
type conn interface {
driver.Pinger
driver.Execer
driver.ExecerContext
driver.Queryer
driver.QueryerContext
driver.Conn
driver.ConnPrepareContext
driver.ConnBeginTx
}
*/
// wrapperDriver defines a wrapper for driver.Driver
type wrapperDriver struct {
driver driver.Driver
opts Options
ctx context.Context
}
// NewWrapper creates and returns a new SQL driver with passed capabilities
func NewWrapper(d driver.Driver, opts ...Option) driver.Driver {
return &wrapperDriver{driver: d, opts: NewOptions(opts...), ctx: context.Background()}
}
type wrappedConnector struct {
connector driver.Connector
// name string
opts Options
ctx context.Context
}
func NewWrapperConnector(c driver.Connector, opts ...Option) driver.Connector {
return &wrappedConnector{connector: c, opts: NewOptions(opts...), ctx: context.Background()}
}
// Connect implements driver.Driver Connect
func (w *wrappedConnector) Connect(ctx context.Context) (driver.Conn, error) {
return w.connector.Connect(ctx)
}
// Driver implements driver.Driver Driver
func (w *wrappedConnector) Driver() driver.Driver {
return w.connector.Driver()
}
/*
// Connect implements driver.Driver OpenConnector
func (w *wrapperDriver) OpenConnector(name string) (driver.Conn, error) {
return &wrapperConnector{driver: w.driver, name: name, opts: w.opts}, nil
}
*/
// Open implements driver.Driver Open
func (w *wrapperDriver) Open(name string) (driver.Conn, error) {
// ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second) // Ensure eventual timeout
// defer cancel()
/*
connector, err := w.OpenConnector(name)
if err != nil {
return nil, err
}
return connector.Connect(ctx)
*/
ts := time.Now()
c, err := w.driver.Open(name)
td := time.Since(ts)
/*
if w.opts.LoggerEnabled {
w.opts.Logger.Log(w.ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(w.ctx, "Open", getCallerName(), td, err)...)
}
*/
_ = td
if err != nil {
return nil, err
}
return wrapConn(c, w.opts), nil
}

View File

@@ -1,167 +0,0 @@
//go:build ignore
package main
import (
"bytes"
"crypto/md5"
"fmt"
"io"
"sort"
"strings"
)
var connIfaces = []string{
"driver.ConnBeginTx",
"driver.ConnPrepareContext",
"driver.Execer",
"driver.ExecerContext",
"driver.NamedValueChecker",
"driver.Pinger",
"driver.Queryer",
"driver.QueryerContext",
"driver.SessionResetter",
"driver.Validator",
}
var stmtIfaces = []string{
"driver.StmtExecContext",
"driver.StmtQueryContext",
"driver.ColumnConverter",
"driver.NamedValueChecker",
}
func getHash(s []string) string {
h := md5.New()
io.WriteString(h, strings.Join(s, "|"))
return fmt.Sprintf("%x", h.Sum(nil))
}
func main() {
comboConn := all(connIfaces)
sort.Slice(comboConn, func(i, j int) bool {
return len(comboConn[i]) < len(comboConn[j])
})
comboStmt := all(stmtIfaces)
sort.Slice(comboStmt, func(i, j int) bool {
return len(comboStmt[i]) < len(comboStmt[j])
})
b := bytes.NewBuffer(nil)
b.WriteString("// Code generated. DO NOT EDIT.\n\n")
b.WriteString("package sql\n\n")
b.WriteString(`import "database/sql/driver"`)
b.WriteString("\n\n")
b.WriteString("func wrapConn(dc driver.Conn, opts Options) driver.Conn {\n")
b.WriteString("\tc := &wrapperConn{conn: dc, opts: opts}\n")
for idx := len(comboConn) - 1; idx >= 0; idx-- {
ifaces := comboConn[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("\tif _, ok := dc.(wrapConn%04d_%s); ok {\n", n, h))
b.WriteString("\treturn struct {\n")
b.WriteString("\t\tdriver.Conn\n")
b.WriteString(fmt.Sprintf("\t\t\t%s", strings.Join(ifaces, "\n\t\t\t")))
b.WriteString("\t\t\n}{")
for idx := range ifaces {
if idx > 0 {
b.WriteString(", ")
b.WriteString("c")
} else if idx == 0 {
b.WriteString("c")
} else {
b.WriteString("c")
}
}
b.WriteString(", c}\n")
b.WriteString("}\n\n")
}
b.WriteString("return c\n")
b.WriteString("}\n")
for idx := len(comboConn) - 1; idx >= 0; idx-- {
ifaces := comboConn[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("// %s\n", strings.Join(ifaces, "|")))
b.WriteString(fmt.Sprintf("type wrapConn%04d_%s interface {\n", n, h))
for _, iface := range ifaces {
b.WriteString(fmt.Sprintf("\t%s\n", iface))
}
b.WriteString("}\n\n")
}
b.WriteString("func wrapStmt(stmt driver.Stmt, query string, opts Options) driver.Stmt {\n")
b.WriteString("\tc := &wrapperStmt{stmt: stmt, query: query, opts: opts}\n")
for idx := len(comboStmt) - 1; idx >= 0; idx-- {
ifaces := comboStmt[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("\tif _, ok := stmt.(wrapStmt%04d_%s); ok {\n", n, h))
b.WriteString("\treturn struct {\n")
b.WriteString("\t\tdriver.Stmt\n")
b.WriteString(fmt.Sprintf("\t\t\t%s", strings.Join(ifaces, "\n\t\t\t")))
b.WriteString("\t\t\n}{")
for idx := range ifaces {
if idx > 0 {
b.WriteString(", ")
b.WriteString("c")
} else if idx == 0 {
b.WriteString("c")
} else {
b.WriteString("c")
}
}
b.WriteString(", c}\n")
b.WriteString("}\n\n")
}
b.WriteString("return c\n")
b.WriteString("}\n")
for idx := len(comboStmt) - 1; idx >= 0; idx-- {
ifaces := comboStmt[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("// %s\n", strings.Join(ifaces, "|")))
b.WriteString(fmt.Sprintf("type wrapStmt%04d_%s interface {\n", n, h))
for _, iface := range ifaces {
b.WriteString(fmt.Sprintf("\t%s\n", iface))
}
b.WriteString("}\n\n")
}
fmt.Printf("%s\n", b.String())
}
// all returns all combinations for a given string array.
func all[T any](set []T) (subsets [][]T) {
length := uint(len(set))
for subsetBits := 1; subsetBits < (1 << length); subsetBits++ {
var subset []T
for object := uint(0); object < length; object++ {
if (subsetBits>>object)&1 == 1 {
subset = append(subset, set[object])
}
}
subsets = append(subsets, subset)
}
return subsets
}

View File

@@ -1,172 +0,0 @@
package sql
import (
"context"
"fmt"
"time"
"go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/meter"
"go.unistack.org/micro/v4/tracer"
)
var (
// DefaultMeterStatsInterval holds default stats interval
DefaultMeterStatsInterval = 5 * time.Second
// DefaultLoggerObserver used to prepare labels for logger
DefaultLoggerObserver = func(ctx context.Context, method string, query string, td time.Duration, err error) []interface{} {
labels := []interface{}{"db.method", method, "took", fmt.Sprintf("%v", td)}
if err != nil {
labels = append(labels, "error", err.Error())
}
if query != labelUnknown {
labels = append(labels, "query", query)
}
return labels
}
)
var (
MaxOpenConnections = "micro_sql_max_open_conn"
OpenConnections = "micro_sql_open_conn"
InuseConnections = "micro_sql_inuse_conn"
IdleConnections = "micro_sql_idle_conn"
WaitConnections = "micro_sql_waited_conn"
BlockedSeconds = "micro_sql_blocked_seconds"
MaxIdleClosed = "micro_sql_max_idle_closed"
MaxIdletimeClosed = "micro_sql_closed_max_idle"
MaxLifetimeClosed = "micro_sql_closed_max_lifetime"
meterRequestTotal = "micro_sql_request_total"
meterRequestLatencyMicroseconds = "micro_sql_latency_microseconds"
meterRequestDurationSeconds = "micro_sql_request_duration_seconds"
labelUnknown = "unknown"
labelQuery = "db_statement"
labelMethod = "db_method"
labelStatus = "status"
labelSuccess = "success"
labelFailure = "failure"
labelHost = "db_host"
labelDatabase = "db_name"
)
// Options struct holds wrapper options
type Options struct {
Logger logger.Logger
Meter meter.Meter
Tracer tracer.Tracer
DatabaseHost string
DatabaseName string
MeterStatsInterval time.Duration
LoggerLevel logger.Level
LoggerEnabled bool
LoggerObserver func(ctx context.Context, method string, name string, td time.Duration, err error) []interface{}
}
// Option func signature
type Option func(*Options)
// NewOptions create new Options struct from provided option slice
func NewOptions(opts ...Option) Options {
options := Options{
Logger: logger.DefaultLogger,
Meter: meter.DefaultMeter,
Tracer: tracer.DefaultTracer,
MeterStatsInterval: DefaultMeterStatsInterval,
LoggerLevel: logger.ErrorLevel,
LoggerObserver: DefaultLoggerObserver,
}
for _, o := range opts {
o(&options)
}
options.Meter = options.Meter.Clone(
meter.Labels(
labelHost, options.DatabaseHost,
labelDatabase, options.DatabaseName,
),
)
options.Logger = options.Logger.Clone(logger.WithAddCallerSkipCount(1))
return options
}
// MetricInterval specifies stats interval for *sql.DB
func MetricInterval(td time.Duration) Option {
return func(o *Options) {
o.MeterStatsInterval = td
}
}
func DatabaseHost(host string) Option {
return func(o *Options) {
o.DatabaseHost = host
}
}
func DatabaseName(name string) Option {
return func(o *Options) {
o.DatabaseName = name
}
}
// Meter passes meter.Meter to wrapper
func Meter(m meter.Meter) Option {
return func(o *Options) {
o.Meter = m
}
}
// Logger passes logger.Logger to wrapper
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// LoggerEnabled enable sql logging
func LoggerEnabled(b bool) Option {
return func(o *Options) {
o.LoggerEnabled = b
}
}
// LoggerLevel passes logger.Level option
func LoggerLevel(lvl logger.Level) Option {
return func(o *Options) {
o.LoggerLevel = lvl
}
}
// LoggerObserver passes observer to fill logger fields
func LoggerObserver(obs func(context.Context, string, string, time.Duration, error) []interface{}) Option {
return func(o *Options) {
o.LoggerObserver = obs
}
}
// Tracer passes tracer.Tracer to wrapper
func Tracer(t tracer.Tracer) Option {
return func(o *Options) {
o.Tracer = t
}
}
type queryNameKey struct{}
// QueryName passes query name to wrapper func
func QueryName(ctx context.Context, name string) context.Context {
if ctx == nil {
ctx = context.Background()
}
return context.WithValue(ctx, queryNameKey{}, name)
}
func getQueryName(ctx context.Context) string {
if v, ok := ctx.Value(queryNameKey{}).(string); ok && v != labelUnknown {
return v
}
return getCallerName()
}

View File

@@ -1,41 +0,0 @@
package sql
import (
"context"
"database/sql"
"time"
)
type Statser interface {
Stats() sql.DBStats
}
func NewStatsMeter(ctx context.Context, db Statser, opts ...Option) {
options := NewOptions(opts...)
go func() {
ticker := time.NewTicker(options.MeterStatsInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if db == nil {
return
}
stats := db.Stats()
options.Meter.Counter(MaxOpenConnections).Set(uint64(stats.MaxOpenConnections))
options.Meter.Counter(OpenConnections).Set(uint64(stats.OpenConnections))
options.Meter.Counter(InuseConnections).Set(uint64(stats.InUse))
options.Meter.Counter(IdleConnections).Set(uint64(stats.Idle))
options.Meter.Counter(WaitConnections).Set(uint64(stats.WaitCount))
options.Meter.FloatCounter(BlockedSeconds).Set(stats.WaitDuration.Seconds())
options.Meter.Counter(MaxIdleClosed).Set(uint64(stats.MaxIdleClosed))
options.Meter.Counter(MaxIdletimeClosed).Set(uint64(stats.MaxIdleTimeClosed))
options.Meter.Counter(MaxLifetimeClosed).Set(uint64(stats.MaxLifetimeClosed))
}
}
}()
}

View File

@@ -1,287 +0,0 @@
package sql
import (
"context"
"database/sql/driver"
"fmt"
"time"
requestid "go.unistack.org/micro/v4/hooks/requestid"
"go.unistack.org/micro/v4/tracer"
)
var (
_ driver.Stmt = (*wrapperStmt)(nil)
_ driver.StmtQueryContext = (*wrapperStmt)(nil)
_ driver.StmtExecContext = (*wrapperStmt)(nil)
_ driver.NamedValueChecker = (*wrapperStmt)(nil)
)
// wrapperStmt defines a wrapper for driver.Stmt
type wrapperStmt struct {
stmt driver.Stmt
opts Options
query string
ctx context.Context
}
// Close implements driver.Stmt Close
func (w *wrapperStmt) Close() error {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Close"}
ts := time.Now()
err := w.stmt.Close()
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Close", getCallerName(), td, err)...)
}
*/
return err
}
// NumInput implements driver.Stmt NumInput
func (w *wrapperStmt) NumInput() int {
return w.stmt.NumInput()
}
// CheckNamedValue implements driver.NamedValueChecker
func (w *wrapperStmt) CheckNamedValue(v *driver.NamedValue) error {
s, ok := w.stmt.(driver.NamedValueChecker)
if !ok {
return driver.ErrSkip
}
return s.CheckNamedValue(v)
}
// Exec implements driver.Stmt Exec
func (w *wrapperStmt) Exec(args []driver.Value) (driver.Result, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Exec"}
ts := time.Now()
res, err := w.stmt.Exec(args) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Exec", getCallerName(), td, err)...)
}
*/
return res, err
}
// Query implements driver.Stmt Query
func (w *wrapperStmt) Query(args []driver.Value) (driver.Rows, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Query"}
ts := time.Now()
rows, err := w.stmt.Query(args) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Query", getCallerName(), td, err)...)
}
*/
return rows, err
}
// ColumnConverter implements driver.ColumnConverter
func (w *wrapperStmt) ColumnConverter(idx int) driver.ValueConverter {
s, ok := w.stmt.(driver.ColumnConverter) // nolint:staticcheck
if !ok {
return nil
}
return s.ColumnConverter(idx)
}
// ExecContext implements driver.StmtExecContext ExecContext
func (w *wrapperStmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "ExecContext")
span.AddLabels("db.statement", name)
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "ExecContext", labelQuery, name}
if conn, ok := w.stmt.(driver.StmtExecContext); ok {
ts := time.Now()
res, err := conn.ExecContext(nctx, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", name, td, err)...)
}
*/
return res, err
}
values, err := namedValueToValue(args)
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", name, 0, err)...)
}
*/
return nil, err
}
ts := time.Now()
res, err := w.Exec(values) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", name, td, err)...)
}
*/
return res, err
}
// QueryContext implements driver.StmtQueryContext StmtQueryContext
func (w *wrapperStmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "QueryContext")
span.AddLabels("db.statement", name)
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "QueryContext", labelQuery, name}
if conn, ok := w.stmt.(driver.StmtQueryContext); ok {
ts := time.Now()
rows, err := conn.QueryContext(nctx, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", name, td, err)...)
}
*/
return rows, err
}
values, err := namedValueToValue(args)
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", name, 0, err)...)
}
*/
return nil, err
}
ts := time.Now()
rows, err := w.Query(values) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", name, td, err)...)
}
*/
return rows, err
}

View File

@@ -1,63 +0,0 @@
package sql
import (
"context"
"database/sql/driver"
"time"
"go.unistack.org/micro/v4/tracer"
)
var _ driver.Tx = (*wrapperTx)(nil)
// wrapperTx defines a wrapper for driver.Tx
type wrapperTx struct {
tx driver.Tx
span tracer.Span
opts Options
ctx context.Context
}
// Commit implements driver.Tx Commit
func (w *wrapperTx) Commit() error {
ts := time.Now()
err := w.tx.Commit()
td := time.Since(ts)
_ = td
if w.span != nil {
if err != nil {
w.span.SetStatus(tracer.SpanStatusError, err.Error())
}
w.span.Finish()
}
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(w.ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(w.ctx, "Commit", getCallerName(), td, err)...)
}
*/
w.ctx = nil
return err
}
// Rollback implements driver.Tx Rollback
func (w *wrapperTx) Rollback() error {
ts := time.Now()
err := w.tx.Rollback()
td := time.Since(ts)
_ = td
if w.span != nil {
if err != nil {
w.span.SetStatus(tracer.SpanStatusError, err.Error())
}
w.span.Finish()
}
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(w.ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(w.ctx, "Rollback", getCallerName(), td, err)...)
}
*/
w.ctx = nil
return err
}

View File

@@ -1,19 +0,0 @@
package sql
import (
"database/sql/driver"
)
/*
func wrapDriver(d driver.Driver, opts Options) driver.Driver {
if _, ok := d.(driver.DriverContext); ok {
return &wrapperDriver{driver: d, opts: opts}
}
return struct{ driver.Driver }{&wrapperDriver{driver: d, opts: opts}}
}
*/
// WrapConn allows an existing driver.Conn to be wrapped.
func WrapConn(c driver.Conn, opts ...Option) driver.Conn {
return wrapConn(c, NewOptions(opts...))
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,133 +0,0 @@
package validator
import (
"context"
"go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/errors"
"go.unistack.org/micro/v4/server"
)
var (
DefaultClientErrorFunc = func(req client.Request, rsp interface{}, err error) error {
if rsp != nil {
return errors.BadGateway(req.Service(), "%v", err)
}
return errors.BadRequest(req.Service(), "%v", err)
}
DefaultServerErrorFunc = func(req server.Request, rsp interface{}, err error) error {
if rsp != nil {
return errors.BadGateway(req.Service(), "%v", err)
}
return errors.BadRequest(req.Service(), "%v", err)
}
)
type (
ClientErrorFunc func(client.Request, interface{}, error) error
ServerErrorFunc func(server.Request, interface{}, error) error
)
// Options struct holds wrapper options
type Options struct {
ClientErrorFn ClientErrorFunc
ServerErrorFn ServerErrorFunc
ClientValidateResponse bool
ServerValidateResponse bool
}
// Option func signature
type Option func(*Options)
func ClientValidateResponse(b bool) Option {
return func(o *Options) {
o.ClientValidateResponse = b
}
}
func ServerValidateResponse(b bool) Option {
return func(o *Options) {
o.ClientValidateResponse = b
}
}
func ClientReqErrorFn(fn ClientErrorFunc) Option {
return func(o *Options) {
o.ClientErrorFn = fn
}
}
func ServerErrorFn(fn ServerErrorFunc) Option {
return func(o *Options) {
o.ServerErrorFn = fn
}
}
func NewOptions(opts ...Option) Options {
options := Options{
ClientErrorFn: DefaultClientErrorFunc,
ServerErrorFn: DefaultServerErrorFunc,
}
for _, o := range opts {
o(&options)
}
return options
}
func NewHook(opts ...Option) *hook {
return &hook{opts: NewOptions(opts...)}
}
type validator interface {
Validate() error
}
type hook struct {
opts Options
}
func (w *hook) ClientCall(next client.FuncCall) client.FuncCall {
return func(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
if v, ok := req.Body().(validator); ok {
if err := v.Validate(); err != nil {
return w.opts.ClientErrorFn(req, nil, err)
}
}
err := next(ctx, req, rsp, opts...)
if v, ok := rsp.(validator); ok && w.opts.ClientValidateResponse {
if verr := v.Validate(); verr != nil {
return w.opts.ClientErrorFn(req, rsp, verr)
}
}
return err
}
}
func (w *hook) ClientStream(next client.FuncStream) client.FuncStream {
return func(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
if v, ok := req.Body().(validator); ok {
if err := v.Validate(); err != nil {
return nil, w.opts.ClientErrorFn(req, nil, err)
}
}
return next(ctx, req, opts...)
}
}
func (w *hook) ServerHandler(next server.FuncHandler) server.FuncHandler {
return func(ctx context.Context, req server.Request, rsp interface{}) error {
if v, ok := req.Body().(validator); ok {
if err := v.Validate(); err != nil {
return w.opts.ServerErrorFn(req, nil, err)
}
}
err := next(ctx, req, rsp)
if v, ok := rsp.(validator); ok && w.opts.ServerValidateResponse {
if verr := v.Validate(); verr != nil {
return w.opts.ServerErrorFn(req, rsp, verr)
}
}
return err
}
}

View File

@@ -13,15 +13,6 @@ func FromContext(ctx context.Context) (Logger, bool) {
return l, ok return l, ok
} }
// MustContext returns logger from passed context or DefaultLogger if empty
func MustContext(ctx context.Context) Logger {
l, ok := FromContext(ctx)
if !ok {
panic("missing logger")
}
return l
}
// NewContext stores logger into passed context // NewContext stores logger into passed context
func NewContext(ctx context.Context, l Logger) context.Context { func NewContext(ctx context.Context, l Logger) context.Context {
if ctx == nil { if ctx == nil {
@@ -29,13 +20,3 @@ func NewContext(ctx context.Context, l Logger) context.Context {
} }
return context.WithValue(ctx, loggerKey{}, l) return context.WithValue(ctx, loggerKey{}, l)
} }
// SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option {
return func(o *Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}

View File

@@ -40,14 +40,3 @@ func TestNewContext(t *testing.T) {
t.Fatal("NewContext not works") t.Fatal("NewContext not works")
} }
} }
func TestSetOption(t *testing.T) {
type key struct{}
o := SetOption(key{}, "test")
opts := &Options{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetOption not works")
}
}

View File

@@ -4,20 +4,18 @@ package logger
type Level int8 type Level int8
const ( const (
// TraceLevel usually used to find bugs, very verbose // TraceLevel level usually used to find bugs, very verbose
TraceLevel Level = iota - 2 TraceLevel Level = iota - 2
// DebugLevel used only when enabled debugging // DebugLevel level used only when enabled debugging
DebugLevel DebugLevel
// InfoLevel used for general info about what's going on inside the application // InfoLevel level used for general info about what's going on inside the application
InfoLevel InfoLevel
// WarnLevel used for non-critical entries // WarnLevel level used for non-critical entries
WarnLevel WarnLevel
// ErrorLevel used for errors that should definitely be noted // ErrorLevel level used for errors that should definitely be noted
ErrorLevel ErrorLevel
// FatalLevel used for critical errors and then calls `os.Exit(1)` // FatalLevel level used for critical errors and then calls `os.Exit(1)`
FatalLevel FatalLevel
// NoneLevel used to disable logging
NoneLevel
) )
// String returns logger level string representation // String returns logger level string representation
@@ -35,8 +33,6 @@ func (l Level) String() string {
return "error" return "error"
case FatalLevel: case FatalLevel:
return "fatal" return "fatal"
case NoneLevel:
return "none"
} }
return "info" return "info"
} }
@@ -62,8 +58,6 @@ func ParseLevel(lvl string) Level {
return ErrorLevel return ErrorLevel
case FatalLevel.String(): case FatalLevel.String():
return FatalLevel return FatalLevel
case NoneLevel.String():
return NoneLevel
} }
return InfoLevel return InfoLevel
} }

View File

@@ -3,6 +3,8 @@ package logger
import ( import (
"context" "context"
"go.unistack.org/micro/v4/options"
) )
type ContextAttrFunc func(ctx context.Context) []interface{} type ContextAttrFunc func(ctx context.Context) []interface{}
@@ -14,41 +16,83 @@ var (
DefaultLogger = NewLogger() DefaultLogger = NewLogger()
// DefaultLevel used by logger // DefaultLevel used by logger
DefaultLevel = InfoLevel DefaultLevel = InfoLevel
// DefaultCallerSkipCount used by logger
DefaultCallerSkipCount = 2
) )
// Logger is a generic logging interface // Logger is a generic logging interface
type Logger interface { type Logger interface {
// Init initialises options // Init initialises options
Init(opts ...Option) error Init(opts ...options.Option) error
// Clone create logger copy with new options // Clone create logger copy with new options
Clone(opts ...Option) Logger Clone(opts ...options.Option) Logger
// V compare provided verbosity level with current log level // V compare provided verbosity level with current log level
V(level Level) bool V(level Level) bool
// Level sets the log level for logger // Level sets the log level for logger
Level(level Level) Level(level Level)
// The Logger options // The Logger options
Options() Options Options() Options
// Fields set fields to always be logged with keyval pairs // Attrs set attrs to always be logged with keyval pairs
Fields(fields ...interface{}) Logger Attrs(attrs ...interface{}) Logger
// Info level message // Info level message
Info(ctx context.Context, msg string, args ...interface{}) Info(ctx context.Context, msg string, attrs ...interface{})
// Trace level message // Tracef level message
Trace(ctx context.Context, msg string, args ...interface{}) Trace(ctx context.Context, msg string, attrs ...interface{})
// Debug level message // Debug level message
Debug(ctx context.Context, msg string, args ...interface{}) Debug(ctx context.Context, msg string, attrs ...interface{})
// Warn level message // Warn level message
Warn(ctx context.Context, msg string, args ...interface{}) Warn(ctx context.Context, msg string, attrs ...interface{})
// Error level message // Error level message
Error(ctx context.Context, msg string, args ...interface{}) Error(ctx context.Context, msg string, attrs ...interface{})
// Fatal level message // Fatal level message
Fatal(ctx context.Context, msg string, args ...interface{}) Fatal(ctx context.Context, msg string, attrs ...interface{})
// Log logs message with needed level // Log logs message with needed level
Log(ctx context.Context, level Level, msg string, args ...interface{}) Log(ctx context.Context, level Level, msg string, attrs ...interface{})
// Name returns broker instance name // String returns the name of logger
Name() string
// String returns the type of logger
String() string String() string
} }
// Field contains keyval pair // Info writes formatted msg to default logger on info level
type Field interface{} func Info(ctx context.Context, msg string, attrs ...interface{}) {
DefaultLogger.Info(ctx, msg, attrs...)
}
// Error writes formatted msg to default logger on error level
func Error(ctx context.Context, msg string, attrs ...interface{}) {
DefaultLogger.Error(ctx, msg, attrs...)
}
// Debugf writes formatted msg to default logger on debug level
func Debugf(ctx context.Context, msg string, attrs ...interface{}) {
DefaultLogger.Debug(ctx, msg, attrs...)
}
// Warn writes formatted msg to default logger on warn level
func Warn(ctx context.Context, msg string, attrs ...interface{}) {
DefaultLogger.Warn(ctx, msg, attrs...)
}
// Trace writes formatted msg to default logger on trace level
func Trace(ctx context.Context, msg string, attrs ...interface{}) {
DefaultLogger.Trace(ctx, msg, attrs...)
}
// Fatal writes formatted msg to default logger on fatal level
func Fatal(ctx context.Context, msg string, attrs ...interface{}) {
DefaultLogger.Fatal(ctx, msg, attrs...)
}
// V returns true if passed level enabled in default logger
func V(level Level) bool {
return DefaultLogger.V(level)
}
// Init initialize default logger
func Init(opts ...options.Option) error {
return DefaultLogger.Init(opts...)
}
// Attrs create default logger with specific attrs
func Attrs(attrs ...interface{}) Logger {
return DefaultLogger.Attrs(attrs...)
}

View File

@@ -2,41 +2,34 @@ package logger
import ( import (
"context" "context"
)
const ( "go.unistack.org/micro/v4/options"
defaultCallerSkipCount = 2
) )
type noopLogger struct { type noopLogger struct {
opts Options opts Options
} }
func NewLogger(opts ...Option) Logger { func NewLogger(opts ...options.Option) Logger {
options := NewOptions(opts...) options := NewOptions(opts...)
options.CallerSkipCount = defaultCallerSkipCount
return &noopLogger{opts: options} return &noopLogger{opts: options}
} }
func (l *noopLogger) V(_ Level) bool { func (l *noopLogger) V(lvl Level) bool {
return false return false
} }
func (l *noopLogger) Level(_ Level) { func (l *noopLogger) Level(lvl Level) {
} }
func (l *noopLogger) Name() string { func (l *noopLogger) Init(opts ...options.Option) error {
return l.opts.Name
}
func (l *noopLogger) Init(opts ...Option) error {
for _, o := range opts { for _, o := range opts {
o(&l.opts) o(&l.opts)
} }
return nil return nil
} }
func (l *noopLogger) Clone(opts ...Option) Logger { func (l *noopLogger) Clone(opts ...options.Option) Logger {
nl := &noopLogger{opts: l.opts} nl := &noopLogger{opts: l.opts}
for _, o := range opts { for _, o := range opts {
o(&nl.opts) o(&nl.opts)
@@ -44,7 +37,7 @@ func (l *noopLogger) Clone(opts ...Option) Logger {
return nl return nl
} }
func (l *noopLogger) Fields(_ ...interface{}) Logger { func (l *noopLogger) Attrs(attrs ...interface{}) Logger {
return l return l
} }

View File

@@ -5,66 +5,47 @@ import (
"io" "io"
"log/slog" "log/slog"
"os" "os"
"slices" "reflect"
"time"
"go.unistack.org/micro/v4/meter" "go.unistack.org/micro/v4/options"
rutil "go.unistack.org/micro/v4/util/reflect"
) )
// Option func signature
type Option func(*Options)
// Options holds logger options // Options holds logger options
type Options struct { type Options struct {
// TimeKey is the key used for the time of the log call
TimeKey string
// LevelKey is the key used for the level of the log call
LevelKey string
// ErroreKey is the key used for the error of the log call
ErrorKey string
// MessageKey is the key used for the message of the log call
MessageKey string
// SourceKey is the key used for the source file and line of the log call
SourceKey string
// StacktraceKey is the key used for the stacktrace
StacktraceKey string
// Name holds the logger name
Name string
// Out holds the output writer // Out holds the output writer
Out io.Writer Out io.Writer
// Context holds exernal options // Context holds exernal options
Context context.Context Context context.Context
// Meter used to count logs for specific level // Attrs holds additional attributes
Meter meter.Meter Attrs []interface{}
// TimeFunc used to obtain current time // Name holds the logger name
TimeFunc func() time.Time Name string
// Fields holds additional metadata
Fields []interface{}
// ContextAttrFuncs contains funcs that executed before log func on context
ContextAttrFuncs []ContextAttrFunc
// callerSkipCount number of frmaes to skip
CallerSkipCount int
// The logging level the logger should log // The logging level the logger should log
Level Level Level Level
// AddSource enabled writing source file and position in log // CallerSkipCount number of frmaes to skip
AddSource bool CallerSkipCount int
// AddStacktrace controls writing of stacktaces on error // ContextAttrFuncs contains funcs that executed before log func on context
AddStacktrace bool ContextAttrFuncs []ContextAttrFunc
// DedupKeys deduplicate keys in log output // TimeKey is the key used for the time of the log call
DedupKeys bool TimeKey string
// LevelKey is the key used for the level of the log call
LevelKey string
// MessageKey is the key used for the message of the log call
MessageKey string
// SourceKey is the key used for the source file and line of the log call
SourceKey string
} }
// NewOptions creates new options struct // NewOptions creates new options struct
func NewOptions(opts ...Option) Options { func NewOptions(opts ...options.Option) Options {
options := Options{ options := Options{
Level: DefaultLevel, Level: DefaultLevel,
Fields: make([]interface{}, 0, 6), Attrs: make([]interface{}, 0, 6),
Out: os.Stderr, Out: os.Stderr,
CallerSkipCount: DefaultCallerSkipCount,
Context: context.Background(), Context: context.Background(),
ContextAttrFuncs: DefaultContextAttrFuncs, ContextAttrFuncs: DefaultContextAttrFuncs,
AddSource: true,
TimeFunc: time.Now,
Meter: meter.DefaultMeter,
} }
WithMicroKeys()(&options) WithMicroKeys()(&options)
@@ -72,159 +53,126 @@ func NewOptions(opts ...Option) Options {
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
} }
return options return options
} }
// WithContextAttrFuncs appends default funcs for the context attrs filler // WithContextAttrFuncs appends default funcs for the context arrts filler
func WithContextAttrFuncs(fncs ...ContextAttrFunc) Option { func WithContextAttrFuncs(fncs ...ContextAttrFunc) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.ContextAttrFuncs = append(o.ContextAttrFuncs, fncs...) v, err := options.Get(src, ".ContextAttrFuncs")
} if err != nil {
} return err
} else if rutil.IsZero(v) {
// WithDedupKeys dont log duplicate keys v = reflect.MakeSlice(reflect.TypeOf(v), 0, len(fncs)).Interface()
func WithDedupKeys(b bool) Option {
return func(o *Options) {
o.DedupKeys = b
}
}
// WithAddFields add fields for the logger
func WithAddFields(fields ...interface{}) Option {
return func(o *Options) {
if o.DedupKeys {
for i := 0; i < len(o.Fields); i += 2 {
for j := 0; j < len(fields); j += 2 {
iv, iok := o.Fields[i].(string)
jv, jok := fields[j].(string)
if iok && jok && iv == jv {
o.Fields[i+1] = fields[j+1]
fields = slices.Delete(fields, j, j+2)
}
}
}
if len(fields) > 0 {
o.Fields = append(o.Fields, fields...)
}
} else {
o.Fields = append(o.Fields, fields...)
} }
cv := reflect.ValueOf(v)
for _, l := range fncs {
cv = reflect.Append(cv, reflect.ValueOf(l))
}
return options.Set(src, cv.Interface(), ".ContextAttrFuncs")
} }
} }
// WithFields set default fields for the logger // WithAttrs set default fields for the logger
func WithFields(fields ...interface{}) Option { func WithAttrs(attrs ...interface{}) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Fields = fields return options.Set(src, attrs, ".Attrs")
} }
} }
// WithLevel set default level for the logger // WithLevel set default level for the logger
func WithLevel(level Level) Option { func WithLevel(lvl Level) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Level = level return options.Set(src, lvl, ".Level")
} }
} }
// WithOutput set default output writer for the logger // WithOutput set default output writer for the logger
func WithOutput(out io.Writer) Option { func WithOutput(out io.Writer) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.Out = out return options.Set(src, out, ".Out")
} }
} }
// WithAddStacktrace controls writing stacktrace on error // WithCallerSkipCount set frame count to skip
func WithAddStacktrace(v bool) Option { func WithCallerSkipCount(c int) options.Option {
return func(o *Options) { return func(src interface{}) error {
o.AddStacktrace = v return options.Set(src, c, ".CallerSkipCount")
} }
} }
// WithAddSource controls writing source file and pos in log func WithZapKeys() options.Option {
func WithAddSource(v bool) Option { return func(src interface{}) error {
return func(o *Options) { var err error
o.AddSource = v if err = options.Set(src, "@timestamp", ".TimeKey"); err != nil {
} return err
}
// WithContext set context
func WithContext(ctx context.Context) Option {
return func(o *Options) {
o.Context = ctx
}
}
// WithName sets the name
func WithName(n string) Option {
return func(o *Options) {
o.Name = n
}
}
// WithMeter sets the meter
func WithMeter(m meter.Meter) Option {
return func(o *Options) {
o.Meter = m
}
}
// WithTimeFunc sets the func to obtain current time
func WithTimeFunc(fn func() time.Time) Option {
return func(o *Options) {
o.TimeFunc = fn
}
}
func WithZapKeys() Option {
return func(o *Options) {
o.TimeKey = "@timestamp"
o.LevelKey = slog.LevelKey
o.MessageKey = slog.MessageKey
o.SourceKey = "caller"
o.StacktraceKey = "stacktrace"
o.ErrorKey = "error"
}
}
func WithZerologKeys() Option {
return func(o *Options) {
o.TimeKey = slog.TimeKey
o.LevelKey = slog.LevelKey
o.MessageKey = "message"
o.SourceKey = "caller"
o.StacktraceKey = "stacktrace"
o.ErrorKey = "error"
}
}
func WithSlogKeys() Option {
return func(o *Options) {
o.TimeKey = slog.TimeKey
o.LevelKey = slog.LevelKey
o.MessageKey = slog.MessageKey
o.SourceKey = slog.SourceKey
o.StacktraceKey = "stacktrace"
o.ErrorKey = "error"
}
}
func WithMicroKeys() Option {
return func(o *Options) {
o.TimeKey = "timestamp"
o.LevelKey = slog.LevelKey
o.MessageKey = slog.MessageKey
o.SourceKey = "caller"
o.StacktraceKey = "stacktrace"
o.ErrorKey = "error"
}
}
// WithAddCallerSkipCount add skip count for copy logger
func WithAddCallerSkipCount(n int) Option {
return func(o *Options) {
if n > 0 {
o.CallerSkipCount += n
} }
if err = options.Set(src, "level", ".LevelKey"); err != nil {
return err
}
if err = options.Set(src, "msg", ".MessageKey"); err != nil {
return err
}
if err = options.Set(src, "caller", ".SourceKey"); err != nil {
return err
}
return nil
}
}
func WithZerologKeys() options.Option {
return func(src interface{}) error {
var err error
if err = options.Set(src, "time", ".TimeKey"); err != nil {
return err
}
if err = options.Set(src, "level", ".LevelKey"); err != nil {
return err
}
if err = options.Set(src, "message", ".MessageKey"); err != nil {
return err
}
if err = options.Set(src, "caller", ".SourceKey"); err != nil {
return err
}
return nil
}
}
func WithSlogKeys() options.Option {
return func(src interface{}) error {
var err error
if err = options.Set(src, slog.TimeKey, ".TimeKey"); err != nil {
return err
}
if err = options.Set(src, slog.LevelKey, ".LevelKey"); err != nil {
return err
}
if err = options.Set(src, slog.MessageKey, ".MessageKey"); err != nil {
return err
}
if err = options.Set(src, slog.SourceKey, ".SourceKey"); err != nil {
return err
}
return nil
}
}
func WithMicroKeys() options.Option {
return func(src interface{}) error {
var err error
if err = options.Set(src, "timestamp", ".TimeKey"); err != nil {
return err
}
if err = options.Set(src, "level", ".LevelKey"); err != nil {
return err
}
if err = options.Set(src, "msg", ".MessageKey"); err != nil {
return err
}
if err = options.Set(src, "caller", ".SourceKey"); err != nil {
return err
}
return nil
} }
} }

View File

@@ -2,31 +2,17 @@ package slog
import ( import (
"context" "context"
"io"
"log/slog" "log/slog"
"os" "os"
"reflect"
"regexp"
"runtime" "runtime"
"strconv" "strconv"
"sync"
"sync/atomic"
"time" "time"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/semconv" "go.unistack.org/micro/v4/options"
"go.unistack.org/micro/v4/tracer" "go.unistack.org/micro/v4/tracer"
) )
const (
badKey = "!BADKEY"
// defaultCallerSkipCount used by logger
defaultCallerSkipCount = 3
timeFormat = "2006-01-02T15:04:05.000000000Z07:00"
)
var reTrace = regexp.MustCompile(`.*/slog/logger\.go.*\n`)
var ( var (
traceValue = slog.StringValue("trace") traceValue = slog.StringValue("trace")
debugValue = slog.StringValue("debug") debugValue = slog.StringValue("debug")
@@ -34,30 +20,8 @@ var (
warnValue = slog.StringValue("warn") warnValue = slog.StringValue("warn")
errorValue = slog.StringValue("error") errorValue = slog.StringValue("error")
fatalValue = slog.StringValue("fatal") fatalValue = slog.StringValue("fatal")
noneValue = slog.StringValue("none")
) )
type wrapper struct {
h slog.Handler
level atomic.Int64
}
func (h *wrapper) Enabled(ctx context.Context, level slog.Level) bool {
return level >= slog.Level(int(h.level.Load()))
}
func (h *wrapper) Handle(ctx context.Context, rec slog.Record) error {
return h.h.Handle(ctx, rec)
}
func (h *wrapper) WithAttrs(attrs []slog.Attr) slog.Handler {
return h.h.WithAttrs(attrs)
}
func (h *wrapper) WithGroup(name string) slog.Handler {
return h.h.WithGroup(name)
}
func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr { func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr {
switch a.Key { switch a.Key {
case slog.SourceKey: case slog.SourceKey:
@@ -66,7 +30,6 @@ func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr {
a.Key = s.opts.SourceKey a.Key = s.opts.SourceKey
case slog.TimeKey: case slog.TimeKey:
a.Key = s.opts.TimeKey a.Key = s.opts.TimeKey
a.Value = slog.StringValue(a.Value.Time().Format(timeFormat))
case slog.MessageKey: case slog.MessageKey:
a.Key = s.opts.MessageKey a.Key = s.opts.MessageKey
case slog.LevelKey: case slog.LevelKey:
@@ -86,8 +49,6 @@ func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr {
a.Value = errorValue a.Value = errorValue
case lvl >= logger.FatalLevel: case lvl >= logger.FatalLevel:
a.Value = fatalValue a.Value = fatalValue
case lvl >= logger.NoneLevel:
a.Value = noneValue
default: default:
a.Value = infoValue a.Value = infoValue
} }
@@ -97,214 +58,218 @@ func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr {
} }
type slogLogger struct { type slogLogger struct {
handler *wrapper slog *slog.Logger
leveler *slog.LevelVar
opts logger.Options opts logger.Options
mu sync.RWMutex
} }
func (s *slogLogger) Clone(opts ...logger.Option) logger.Logger { func (s *slogLogger) Clone(opts ...options.Option) logger.Logger {
s.mu.RLock()
options := s.opts options := s.opts
s.mu.RUnlock()
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
} }
if len(options.ContextAttrFuncs) == 0 { l := &slogLogger{
options.ContextAttrFuncs = logger.DefaultContextAttrFuncs opts: options,
} }
attrs, _ := s.argsAttrs(options.Fields) /*
l := &slogLogger{ if slog, ok := s.opts.Context.Value(loggerKey{}).(*slog.Logger); ok {
handler: &wrapper{h: s.handler.h.WithAttrs(attrs)}, l.slog = slog
opts: options, return nil
}
*/
l.leveler = new(slog.LevelVar)
handleOpt := &slog.HandlerOptions{
ReplaceAttr: s.renameAttr,
Level: l.leveler,
AddSource: true,
} }
l.handler.level.Store(int64(loggerToSlogLevel(options.Level))) l.leveler.Set(loggerToSlogLevel(l.opts.Level))
handler := slog.NewJSONHandler(options.Out, handleOpt)
l.slog = slog.New(handler).With(options.Attrs...)
return l return l
} }
func (s *slogLogger) V(level logger.Level) bool { func (s *slogLogger) V(level logger.Level) bool {
s.mu.Lock() return s.opts.Level.Enabled(level)
v := s.opts.Level.Enabled(level)
s.mu.Unlock()
return v
} }
func (s *slogLogger) Level(level logger.Level) { func (s *slogLogger) Level(level logger.Level) {
s.mu.Lock() s.leveler.Set(loggerToSlogLevel(level))
s.opts.Level = level
s.handler.level.Store(int64(loggerToSlogLevel(level)))
s.mu.Unlock()
} }
func (s *slogLogger) Options() logger.Options { func (s *slogLogger) Options() logger.Options {
return s.opts return s.opts
} }
func (s *slogLogger) Fields(fields ...interface{}) logger.Logger { func (s *slogLogger) Attrs(attrs ...interface{}) logger.Logger {
s.mu.RLock() nl := &slogLogger{opts: s.opts}
options := s.opts nl.leveler = new(slog.LevelVar)
s.mu.RUnlock() nl.leveler.Set(s.leveler.Level())
l := &slogLogger{opts: options} handleOpt := &slog.HandlerOptions{
logger.WithAddFields(fields...)(&l.opts) ReplaceAttr: nl.renameAttr,
Level: s.leveler,
if len(options.ContextAttrFuncs) == 0 { AddSource: true,
options.ContextAttrFuncs = logger.DefaultContextAttrFuncs
} }
attrs, _ := s.argsAttrs(fields) handler := slog.NewJSONHandler(s.opts.Out, handleOpt)
l.handler = &wrapper{h: s.handler.h.WithAttrs(attrs)} nl.slog = slog.New(handler).With(attrs...)
l.handler.level.Store(int64(loggerToSlogLevel(l.opts.Level)))
return l return nl
} }
func (s *slogLogger) Init(opts ...logger.Option) error { func (s *slogLogger) Init(opts ...options.Option) error {
s.mu.Lock()
for _, o := range opts {
o(&s.opts)
}
if len(s.opts.ContextAttrFuncs) == 0 { if len(s.opts.ContextAttrFuncs) == 0 {
s.opts.ContextAttrFuncs = logger.DefaultContextAttrFuncs s.opts.ContextAttrFuncs = logger.DefaultContextAttrFuncs
} }
for _, o := range opts {
if err := o(&s.opts); err != nil {
return err
}
}
/*
if slog, ok := s.opts.Context.Value(loggerKey{}).(*slog.Logger); ok {
s.slog = slog
return nil
}
*/
s.leveler = new(slog.LevelVar)
handleOpt := &slog.HandlerOptions{ handleOpt := &slog.HandlerOptions{
ReplaceAttr: s.renameAttr, ReplaceAttr: s.renameAttr,
Level: loggerToSlogLevel(logger.TraceLevel), Level: s.leveler,
AddSource: s.opts.AddSource, AddSource: true,
} }
s.leveler.Set(loggerToSlogLevel(s.opts.Level))
attrs, _ := s.argsAttrs(s.opts.Fields) handler := slog.NewJSONHandler(s.opts.Out, handleOpt)
s.slog = slog.New(handler).With(s.opts.Attrs...)
var h slog.Handler
if s.opts.Context != nil {
if v, ok := s.opts.Context.Value(handlerKey{}).(slog.Handler); ok && v != nil {
h = v
}
if fn := s.opts.Context.Value(handlerFnKey{}); fn != nil {
if rfn := reflect.ValueOf(fn); rfn.Kind() == reflect.Func {
if ret := rfn.Call([]reflect.Value{reflect.ValueOf(s.opts.Out), reflect.ValueOf(handleOpt)}); len(ret) == 1 {
if iface, ok := ret[0].Interface().(slog.Handler); ok && iface != nil {
h = iface
}
}
}
}
}
if h == nil {
h = slog.NewJSONHandler(s.opts.Out, handleOpt)
}
s.handler = &wrapper{h: h.WithAttrs(attrs)}
s.handler.level.Store(int64(loggerToSlogLevel(s.opts.Level)))
s.mu.Unlock()
return nil return nil
} }
func (s *slogLogger) Log(ctx context.Context, lvl logger.Level, msg string, attrs ...interface{}) { func (s *slogLogger) Log(ctx context.Context, lvl logger.Level, msg string, attrs ...interface{}) {
s.printLog(ctx, lvl, msg, attrs...) if !s.V(lvl) {
return
}
var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), loggerToSlogLevel(lvl), msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
_ = s.slog.Handler().Handle(ctx, r)
} }
func (s *slogLogger) Info(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Info(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.InfoLevel, msg, attrs...) if !s.V(logger.InfoLevel) {
return
}
var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), slog.LevelInfo, msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
_ = s.slog.Handler().Handle(ctx, r)
} }
func (s *slogLogger) Debug(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Debug(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.DebugLevel, msg, attrs...) if !s.V(logger.DebugLevel) {
return
}
var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), slog.LevelDebug, msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
_ = s.slog.Handler().Handle(ctx, r)
} }
func (s *slogLogger) Trace(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Trace(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.TraceLevel, msg, attrs...) if !s.V(logger.TraceLevel) {
return
}
var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), slog.LevelDebug-1, msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
_ = s.slog.Handler().Handle(ctx, r)
} }
func (s *slogLogger) Error(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Error(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.ErrorLevel, msg, attrs...) if !s.V(logger.ErrorLevel) {
return
}
var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), slog.LevelError, msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
r.Attrs(func(a slog.Attr) bool {
if a.Key == "error" {
if span, ok := tracer.SpanFromContext(ctx); ok {
span.SetStatus(tracer.SpanStatusError, a.Value.String())
return false
}
}
return true
})
_ = s.slog.Handler().Handle(ctx, r)
} }
func (s *slogLogger) Fatal(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Fatal(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.FatalLevel, msg, attrs...) if !s.V(logger.FatalLevel) {
if closer, ok := s.opts.Out.(io.Closer); ok { return
closer.Close()
} }
time.Sleep(1 * time.Second) var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), slog.LevelError+1, msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
_ = s.slog.Handler().Handle(ctx, r)
os.Exit(1) os.Exit(1)
} }
func (s *slogLogger) Warn(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Warn(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.WarnLevel, msg, attrs...) if !s.V(logger.WarnLevel) {
} return
}
func (s *slogLogger) Name() string { var pcs [1]uintptr
return s.opts.Name runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, Infof]
r := slog.NewRecord(time.Now(), slog.LevelWarn, msg, pcs[0])
for _, fn := range s.opts.ContextAttrFuncs {
attrs = append(attrs, fn(ctx)...)
}
r.Add(attrs...)
_ = s.slog.Handler().Handle(ctx, r)
} }
func (s *slogLogger) String() string { func (s *slogLogger) String() string {
return "slog" return "slog"
} }
func (s *slogLogger) printLog(ctx context.Context, lvl logger.Level, msg string, args ...interface{}) { func NewLogger(opts ...options.Option) logger.Logger {
if !s.V(lvl) { l := &slogLogger{
return
}
var argError error
s.opts.Meter.Counter(semconv.LoggerMessageTotal, "level", lvl.String()).Inc()
attrs, err := s.argsAttrs(args)
if err != nil {
argError = err
}
if argError != nil {
if span, ok := tracer.SpanFromContext(ctx); ok {
span.SetStatus(tracer.SpanStatusError, argError.Error())
}
}
for _, fn := range s.opts.ContextAttrFuncs {
ctxAttrs, err := s.argsAttrs(fn(ctx))
if err != nil {
argError = err
}
attrs = append(attrs, ctxAttrs...)
}
if argError != nil {
if span, ok := tracer.SpanFromContext(ctx); ok {
span.SetStatus(tracer.SpanStatusError, argError.Error())
}
}
if s.opts.AddStacktrace && (lvl == logger.FatalLevel || lvl == logger.ErrorLevel) {
stackInfo := make([]byte, 1024*1024)
if stackSize := runtime.Stack(stackInfo, false); stackSize > 0 {
traceLines := reTrace.Split(string(stackInfo[:stackSize]), -1)
if len(traceLines) != 0 {
attrs = append(attrs, slog.String(s.opts.StacktraceKey, traceLines[len(traceLines)-1]))
}
}
}
var pcs [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, printLog, LogLvlMethod]
r := slog.NewRecord(s.opts.TimeFunc(), loggerToSlogLevel(lvl), msg, pcs[0])
r.AddAttrs(attrs...)
_ = s.handler.Handle(ctx, r)
}
func NewLogger(opts ...logger.Option) logger.Logger {
s := &slogLogger{
opts: logger.NewOptions(opts...), opts: logger.NewOptions(opts...),
} }
s.opts.CallerSkipCount = defaultCallerSkipCount return l
return s
} }
func loggerToSlogLevel(level logger.Level) slog.Level { func loggerToSlogLevel(level logger.Level) slog.Level {
@@ -319,8 +284,6 @@ func loggerToSlogLevel(level logger.Level) slog.Level {
return slog.LevelDebug - 1 return slog.LevelDebug - 1
case logger.FatalLevel: case logger.FatalLevel:
return slog.LevelError + 1 return slog.LevelError + 1
case logger.NoneLevel:
return slog.LevelError + 2
default: default:
return slog.LevelInfo return slog.LevelInfo
} }
@@ -338,45 +301,7 @@ func slogToLoggerLevel(level slog.Level) logger.Level {
return logger.TraceLevel return logger.TraceLevel
case slog.LevelError + 1: case slog.LevelError + 1:
return logger.FatalLevel return logger.FatalLevel
case slog.LevelError + 2:
return logger.NoneLevel
default: default:
return logger.InfoLevel return logger.InfoLevel
} }
} }
func (s *slogLogger) argsAttrs(args []interface{}) ([]slog.Attr, error) {
attrs := make([]slog.Attr, 0, len(args))
var err error
for idx := 0; idx < len(args); idx++ {
switch arg := args[idx].(type) {
case slog.Attr:
attrs = append(attrs, arg)
case string:
if idx+1 < len(args) {
attrs = append(attrs, slog.Any(arg, args[idx+1]))
idx++
} else {
attrs = append(attrs, slog.String(badKey, arg))
}
case error:
attrs = append(attrs, slog.String(s.opts.ErrorKey, arg.Error()))
err = arg
}
}
return attrs, err
}
type handlerKey struct{}
func WithHandler(h slog.Handler) logger.Option {
return logger.SetOption(handlerKey{}, h)
}
type handlerFnKey struct{}
func WithHandlerFunc(fn any) logger.Option {
return logger.SetOption(handlerFnKey{}, fn)
}

View File

@@ -3,288 +3,12 @@ package slog
import ( import (
"bytes" "bytes"
"context" "context"
"errors"
"fmt"
"log" "log"
"log/slog"
"strings"
"testing" "testing"
"time"
"github.com/google/uuid"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/util/buffer"
) )
// always first to have proper check
func TestStacktrace(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.DebugLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithAddStacktrace(true),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Error(ctx, "msg1", errors.New("err"))
if !bytes.Contains(buf.Bytes(), []byte(`slog_test.go:32`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestNoneLevel(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.NoneLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithAddStacktrace(true),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Error(ctx, "msg1", errors.New("err"))
if buf.Len() != 0 {
t.Fatalf("logger none level not works, buf contains: %s", buf.Bytes())
}
}
func TestDelayedBuffer(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
dbuf := buffer.NewDelayedBuffer(100, 100*time.Millisecond, buf)
l := NewLogger(logger.WithLevel(logger.ErrorLevel), logger.WithOutput(dbuf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithAddStacktrace(true),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Error(ctx, "msg1", errors.New("err"))
time.Sleep(120 * time.Millisecond)
if !bytes.Contains(buf.Bytes(), []byte(`key1=val1`)) {
t.Fatalf("logger delayed buffer not works, buf contains: %s", buf.Bytes())
}
}
func TestTime(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.ErrorLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithAddStacktrace(true),
logger.WithTimeFunc(func() time.Time {
return time.Unix(0, 0)
}),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Error(ctx, "msg1", errors.New("err"))
if !bytes.Contains(buf.Bytes(), []byte(`timestamp=1970-01-01T03:00:00.000000000+03:00`)) &&
!bytes.Contains(buf.Bytes(), []byte(`timestamp=1970-01-01T00:00:00.000000000Z`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestWithFields(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.InfoLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithDedupKeys(true),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg1")
l = l.Fields("key1", "val2")
l.Info(ctx, "msg2")
if !bytes.Contains(buf.Bytes(), []byte(`msg=msg2 key1=val1`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestWithDedupKeysWithAddFields(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.InfoLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithDedupKeys(true),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg1")
if err := l.Init(logger.WithAddFields("key2", "val2")); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg2")
if err := l.Init(logger.WithAddFields("key2", "val3", "key1", "val4")); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg3")
if !bytes.Contains(buf.Bytes(), []byte(`msg=msg3 key1=val4 key2=val3`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestWithHandlerFunc(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.InfoLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
)
if err := l.Init(); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg1")
if !bytes.Contains(buf.Bytes(), []byte(`msg=msg1`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestWithAddFields(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.InfoLevel), logger.WithOutput(buf))
if err := l.Init(); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg1")
if err := l.Init(logger.WithAddFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg2")
if err := l.Init(logger.WithAddFields("key2", "val2")); err != nil {
t.Fatal(err)
}
l.Info(ctx, "msg3")
if !bytes.Contains(buf.Bytes(), []byte(`"key1"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"key2"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestMultipleFieldsWithLevel(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.InfoLevel), logger.WithOutput(buf))
if err := l.Init(); err != nil {
t.Fatal(err)
}
l = l.Fields("key", "val")
l.Info(ctx, "msg1")
nl := l.Clone(logger.WithLevel(logger.DebugLevel))
nl.Debug(ctx, "msg2")
l.Debug(ctx, "msg3")
if !bytes.Contains(buf.Bytes(), []byte(`"key":"val"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"msg1"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"msg2"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
if bytes.Contains(buf.Bytes(), []byte(`"msg3"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestMultipleFields(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.InfoLevel), logger.WithOutput(buf))
if err := l.Init(); err != nil {
t.Fatal(err)
}
l = l.Fields("key", "val")
l = l.Fields("key1", "val1")
l.Info(ctx, "msg")
if !bytes.Contains(buf.Bytes(), []byte(`"key":"val"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"key1":"val1"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestError(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.ErrorLevel), logger.WithOutput(buf), logger.WithAddStacktrace(true))
if err := l.Init(); err != nil {
t.Fatal(err)
}
l.Error(ctx, "message", fmt.Errorf("error message"))
if !bytes.Contains(buf.Bytes(), []byte(`"stacktrace":"`)) {
t.Fatalf("logger stacktrace not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"error":"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestErrorf(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.ErrorLevel), logger.WithOutput(buf), logger.WithAddStacktrace(true))
if err := l.Init(logger.WithContextAttrFuncs(func(_ context.Context) []interface{} {
return nil
})); err != nil {
t.Fatal(err)
}
l.Log(ctx, logger.ErrorLevel, "message", errors.New("error msg"))
l.Log(ctx, logger.ErrorLevel, "", errors.New("error msg"))
if !bytes.Contains(buf.Bytes(), []byte(`"error":"error msg"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"stacktrace":"`)) {
t.Fatalf("logger stacktrace not works, buf contains: %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte(`"error":"`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
}
}
func TestContext(t *testing.T) { func TestContext(t *testing.T) {
ctx := context.TODO() ctx := context.TODO()
buf := bytes.NewBuffer(nil) buf := bytes.NewBuffer(nil)
@@ -293,7 +17,7 @@ func TestContext(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
nl, ok := logger.FromContext(logger.NewContext(ctx, l.Fields("key", "val"))) nl, ok := logger.FromContext(logger.NewContext(ctx, l.Attrs("key", "val")))
if !ok { if !ok {
t.Fatal("context without logger") t.Fatal("context without logger")
} }
@@ -303,7 +27,7 @@ func TestContext(t *testing.T) {
} }
} }
func TestFields(t *testing.T) { func TestAttrs(t *testing.T) {
ctx := context.TODO() ctx := context.TODO()
buf := bytes.NewBuffer(nil) buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.TraceLevel), logger.WithOutput(buf)) l := NewLogger(logger.WithLevel(logger.TraceLevel), logger.WithOutput(buf))
@@ -311,7 +35,7 @@ func TestFields(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
nl := l.Fields("key", "val") nl := l.Attrs("key", "val")
nl.Info(ctx, "message") nl.Info(ctx, "message")
if !bytes.Contains(buf.Bytes(), []byte(`"key":"val"`)) { if !bytes.Contains(buf.Bytes(), []byte(`"key":"val"`)) {
@@ -327,7 +51,7 @@ func TestFromContextWithFields(t *testing.T) {
if err := l.Init(); err != nil { if err := l.Init(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
nl := l.Fields("key", "val") nl := l.Attrs("key", "val")
ctx = logger.NewContext(ctx, nl) ctx = logger.NewContext(ctx, nl)
@@ -340,11 +64,6 @@ func TestFromContextWithFields(t *testing.T) {
if !bytes.Contains(buf.Bytes(), []byte(`"key":"val"`)) { if !bytes.Contains(buf.Bytes(), []byte(`"key":"val"`)) {
t.Fatalf("logger fields not works, buf contains: %s", buf.Bytes()) t.Fatalf("logger fields not works, buf contains: %s", buf.Bytes())
} }
l.Info(ctx, "test", "uncorrected number attributes")
if !bytes.Contains(buf.Bytes(), []byte(`"!BADKEY":"uncorrected number attributes"`)) {
t.Fatalf("logger fields not works, buf contains: %s", buf.Bytes())
}
} }
func TestClone(t *testing.T) { func TestClone(t *testing.T) {
@@ -404,7 +123,7 @@ func TestLogger(t *testing.T) {
} }
l.Trace(ctx, "trace_msg1") l.Trace(ctx, "trace_msg1")
l.Warn(ctx, "warn_msg1") l.Warn(ctx, "warn_msg1")
l.Fields("error", "test").Info(ctx, "error message") l.Attrs("error", "test").Info(ctx, "error message")
l.Warn(ctx, "first second") l.Warn(ctx, "first second")
if !(bytes.Contains(buf.Bytes(), []byte(`"level":"trace"`)) && bytes.Contains(buf.Bytes(), []byte(`"msg":"trace_msg1"`))) { if !(bytes.Contains(buf.Bytes(), []byte(`"level":"trace"`)) && bytes.Contains(buf.Bytes(), []byte(`"msg":"trace_msg1"`))) {
@@ -420,53 +139,3 @@ func TestLogger(t *testing.T) {
t.Fatalf("logger warn, buf %s", buf.Bytes()) t.Fatalf("logger warn, buf %s", buf.Bytes())
} }
} }
func Test_WithContextAttrFunc(t *testing.T) {
loggerContextAttrFuncs := []logger.ContextAttrFunc{
func(ctx context.Context) []interface{} {
md, ok := metadata.FromOutgoingContext(ctx)
if !ok {
return nil
}
attrs := make([]interface{}, 0, 10)
for k, v := range md {
key := strings.ToLower(k)
switch key {
case "x-request-id", "phone", "external-Id", "source-service", "x-app-install-id", "client-id", "client-ip":
attrs = append(attrs, key, v[0])
}
}
return attrs
},
}
logger.DefaultContextAttrFuncs = append(logger.DefaultContextAttrFuncs, loggerContextAttrFuncs...)
ctx := context.TODO()
ctx = metadata.AppendOutgoingContext(ctx, "X-Request-Id", uuid.New().String(),
"Source-Service", "Test-System")
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.TraceLevel), logger.WithOutput(buf))
if err := l.Init(); err != nil {
t.Fatal(err)
}
l.Info(ctx, "test message")
if !(bytes.Contains(buf.Bytes(), []byte(`"level":"info"`)) && bytes.Contains(buf.Bytes(), []byte(`"msg":"test message"`))) {
t.Fatalf("logger info, buf %s", buf.Bytes())
}
if !(bytes.Contains(buf.Bytes(), []byte(`"x-request-id":"`))) {
t.Fatalf("logger info, buf %s", buf.Bytes())
}
if !(bytes.Contains(buf.Bytes(), []byte(`"source-service":"Test-System"`))) {
t.Fatalf("logger info, buf %s", buf.Bytes())
}
buf.Reset()
omd, _ := metadata.FromOutgoingContext(ctx)
l.Info(ctx, "test message1")
omd.Set("Source-Service", "Test-System2")
l.Info(ctx, "test message2")
// t.Logf("xxx %s", buf.Bytes())
}

View File

@@ -12,7 +12,7 @@ type stdLogger struct {
// NewStdLogger returns new *log.Logger baked by logger.Logger implementation // NewStdLogger returns new *log.Logger baked by logger.Logger implementation
func NewStdLogger(l Logger, level Level) *log.Logger { func NewStdLogger(l Logger, level Level) *log.Logger {
return log.New(&stdLogger{l: l, level: level}, "" /* prefix */, 0 /* flags */) return log.New(&stdLogger{l: l.Clone(WithCallerSkipCount(l.Options().CallerSkipCount + 1)), level: level}, "" /* prefix */, 0 /* flags */)
} }
func (sl *stdLogger) Write(p []byte) (int, error) { func (sl *stdLogger) Write(p []byte) (int, error) {

View File

@@ -36,14 +36,14 @@ var (
circularShortBytes = []byte("<shown>") circularShortBytes = []byte("<shown>")
invalidAngleBytes = []byte("<invalid>") invalidAngleBytes = []byte("<invalid>")
filteredBytes = []byte("<filtered>") filteredBytes = []byte("<filtered>")
// openBracketBytes = []byte("[") openBracketBytes = []byte("[")
// closeBracketBytes = []byte("]") closeBracketBytes = []byte("]")
percentBytes = []byte("%") percentBytes = []byte("%")
precisionBytes = []byte(".") precisionBytes = []byte(".")
openAngleBytes = []byte("<") openAngleBytes = []byte("<")
closeAngleBytes = []byte(">") closeAngleBytes = []byte(">")
openMapBytes = []byte("{") openMapBytes = []byte("{")
closeMapBytes = []byte("}") closeMapBytes = []byte("}")
) )
type protoMessage interface { type protoMessage interface {
@@ -52,15 +52,13 @@ type protoMessage interface {
} }
type Wrapper struct { type Wrapper struct {
pointers map[uintptr]int val interface{}
takeMap map[int]bool s fmt.State
pointers map[uintptr]int
val interface{} opts *Options
s fmt.State
opts *Options
depth int depth int
ignoreNextType bool ignoreNextType bool
takeMap map[int]bool
protoWrapperType bool protoWrapperType bool
sqlWrapperType bool sqlWrapperType bool
} }

View File

@@ -82,12 +82,12 @@ func TestTagged(t *testing.T) {
func TestTaggedNested(t *testing.T) { func TestTaggedNested(t *testing.T) {
type val struct { type val struct {
key string `logger:"take"` key string `logger:"take"`
// val string `logger:"omit"` val string `logger:"omit"`
unk string unk string
} }
type str struct { type str struct {
// key string `logger:"omit"` key string `logger:"omit"`
val *val `logger:"take"` val *val `logger:"take"`
} }
var iface interface{} var iface interface{}

View File

@@ -1,294 +1,142 @@
// Package metadata is a way of defining message headers
package metadata package metadata
import ( import (
"context" "context"
"fmt"
"strings"
) )
// In the metadata package, context and metadata are treated as immutable.
// Deep copies of metadata are made to keep things safe and correct.
// If a user takes a map and changes it across threads, it's their responsibility.
//
// 1. Incoming Context
//
// This context is provided by an external system and populated by the server or broker of the micro framework.
// It should not be modified. The idea is to extract all necessary data from it,
// validate the data, and transfer it into the current context.
// After that, only the current context should be used throughout the code.
//
// 2. Current Context
//
// This is the context used during the execution flow.
// You can add any needed metadata to it and pass it through your code.
//
// 3. Outgoing Context
//
// This context is for sending data to external systems.
// You can add what you need before sending it out.
// But its usually better to build and prepare this context right before making the external call,
// instead of changing it in many places.
//
// Execution Flow:
//
// [External System]
// ↓
// [Incoming Context]
// ↓
// [Extract & Validate Metadata from Incoming Context]
// ↓
// [Prepare Current Context]
// ↓
// [Enrich Current Context]
// ↓
// [Business Logic]
// ↓
// [Prepare Outgoing Context]
// ↓
// [External System Call]
type ( type (
metadataCurrentKey struct{} mdIncomingKey struct{}
metadataIncomingKey struct{} mdOutgoingKey struct{}
metadataOutgoingKey struct{} mdKey struct{}
rawMetadata struct {
md Metadata
added [][]string
}
) )
// NewContext creates a new context with the provided Metadata attached. // FromIncomingContext returns metadata from incoming ctx
// The Metadata must not be modified after calling this function. // returned metadata shoud not be modified or race condition happens
func NewContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataCurrentKey{}, rawMetadata{md: md})
}
// NewIncomingContext creates a new context with the provided incoming Metadata attached.
// The Metadata must not be modified after calling this function.
func NewIncomingContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataIncomingKey{}, rawMetadata{md: md})
}
// NewOutgoingContext creates a new context with the provided outgoing Metadata attached.
// The Metadata must not be modified after calling this function.
func NewOutgoingContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataOutgoingKey{}, rawMetadata{md: md})
}
// AppendContext returns a new context with the provided key-value pairs (kv)
// merged with any existing metadata in the context. For a description of kv,
// please refer to the Pairs documentation.
func AppendContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataCurrentKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataCurrentKey{}, rawMetadata{md: md.md, added: added})
}
// AppendOutgoingContext returns a new context with the provided key-value pairs (kv)
// merged with any existing metadata in the context. For a description of kv,
// please refer to the Pairs documentation.
func AppendOutgoingContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendOutgoingContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataOutgoingKey{}, rawMetadata{md: md.md, added: added})
}
// FromContext retrieves a deep copy of the metadata from the context and returns it
// with a boolean indicating if it was found.
func FromContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataCurrentKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, true
}
// MustContext retrieves a deep copy of the metadata from the context and panics
// if the metadata is not found.
func MustContext(ctx context.Context) Metadata {
md, ok := FromContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// FromIncomingContext retrieves a deep copy of the metadata from the context and returns it
// with a boolean indicating if it was found.
func FromIncomingContext(ctx context.Context) (Metadata, bool) { func FromIncomingContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataIncomingKey{}).(rawMetadata) if ctx == nil {
if !ok {
return nil, false return nil, false
} }
metadataSize := len(raw.md) md, ok := ctx.Value(mdIncomingKey{}).(*rawMetadata)
for i := range raw.added { if !ok || md.md == nil {
metadataSize += len(raw.added[i]) / 2 return nil, false
} }
return md.md, ok
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromIncomingContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, true
} }
// MustIncomingContext retrieves a deep copy of the metadata from the context and panics // FromOutgoingContext returns metadata from outgoing ctx
// if the metadata is not found. // returned metadata shoud not be modified or race condition happens
func MustIncomingContext(ctx context.Context) Metadata {
md, ok := FromIncomingContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// FromOutgoingContext retrieves a deep copy of the metadata from the context and returns it
// with a boolean indicating if it was found.
func FromOutgoingContext(ctx context.Context) (Metadata, bool) { func FromOutgoingContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataOutgoingKey{}).(rawMetadata) if ctx == nil {
if !ok {
return nil, false return nil, false
} }
md, ok := ctx.Value(mdOutgoingKey{}).(*rawMetadata)
metadataSize := len(raw.md) if !ok || md.md == nil {
for i := range raw.added { return nil, false
metadataSize += len(raw.added[i]) / 2
} }
return md.md, ok
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromOutgoingContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, ok
} }
// MustOutgoingContext retrieves a deep copy of the metadata from the context and panics // FromContext returns metadata from the given context
// if the metadata is not found. // returned metadata shoud not be modified or race condition happens
func MustOutgoingContext(ctx context.Context) Metadata { func FromContext(ctx context.Context) (Metadata, bool) {
md, ok := FromOutgoingContext(ctx) if ctx == nil {
return nil, false
}
md, ok := ctx.Value(mdKey{}).(*rawMetadata)
if !ok || md.md == nil {
return nil, false
}
return md.md, ok
}
// NewContext creates a new context with the given metadata
func NewContext(ctx context.Context, md Metadata) context.Context {
if ctx == nil {
ctx = context.Background()
}
ctx = context.WithValue(ctx, mdKey{}, &rawMetadata{md})
ctx = context.WithValue(ctx, mdIncomingKey{}, &rawMetadata{})
ctx = context.WithValue(ctx, mdOutgoingKey{}, &rawMetadata{})
return ctx
}
// SetOutgoingContext modify outgoing context with given metadata
func SetOutgoingContext(ctx context.Context, md Metadata) bool {
if ctx == nil {
return false
}
if omd, ok := ctx.Value(mdOutgoingKey{}).(*rawMetadata); ok {
omd.md = md
return true
}
return false
}
// SetIncomingContext modify incoming context with given metadata
func SetIncomingContext(ctx context.Context, md Metadata) bool {
if ctx == nil {
return false
}
if omd, ok := ctx.Value(mdIncomingKey{}).(*rawMetadata); ok {
omd.md = md
return true
}
return false
}
// NewIncomingContext creates a new context with incoming metadata attached
func NewIncomingContext(ctx context.Context, md Metadata) context.Context {
if ctx == nil {
ctx = context.Background()
}
ctx = context.WithValue(ctx, mdIncomingKey{}, &rawMetadata{md})
if v, ok := ctx.Value(mdOutgoingKey{}).(*rawMetadata); !ok || v == nil {
ctx = context.WithValue(ctx, mdOutgoingKey{}, &rawMetadata{})
}
return ctx
}
// NewOutgoingContext creates a new context with outcoming metadata attached
func NewOutgoingContext(ctx context.Context, md Metadata) context.Context {
if ctx == nil {
ctx = context.Background()
}
ctx = context.WithValue(ctx, mdOutgoingKey{}, &rawMetadata{md})
if v, ok := ctx.Value(mdIncomingKey{}).(*rawMetadata); !ok || v == nil {
ctx = context.WithValue(ctx, mdIncomingKey{}, &rawMetadata{})
}
return ctx
}
// AppendOutgoingContext apends new md to context
func AppendOutgoingContext(ctx context.Context, kv ...string) context.Context {
md, ok := Pairs(kv...)
if !ok { if !ok {
panic("missing metadata") return ctx
} }
return md omd, ok := FromOutgoingContext(ctx)
}
// ValueFromCurrentContext retrieves a deep copy of the metadata for the given key
// from the context, performing a case-insensitive search if needed. Returns nil if not found.
func ValueFromCurrentContext(ctx context.Context, key string) []string {
md, ok := ctx.Value(metadataCurrentKey{}).(rawMetadata)
if !ok { if !ok {
return nil return NewOutgoingContext(ctx, md)
} }
for k, v := range md {
if v, ok := md.md[key]; ok { omd.Set(k, v)
return copyOf(v)
} }
for k, v := range md.md { return NewOutgoingContext(ctx, omd)
// Case-insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
} }
// ValueFromIncomingContext retrieves a deep copy of the metadata for the given key // AppendIncomingContext apends new md to context
// from the context, performing a case-insensitive search if needed. Returns nil if not found. func AppendIncomingContext(ctx context.Context, kv ...string) context.Context {
func ValueFromIncomingContext(ctx context.Context, key string) []string { md, ok := Pairs(kv...)
raw, ok := ctx.Value(metadataIncomingKey{}).(rawMetadata)
if !ok { if !ok {
return nil return ctx
} }
omd, ok := FromIncomingContext(ctx)
if v, ok := raw.md[key]; ok {
return copyOf(v)
}
for k, v := range raw.md {
// Case-insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}
// ValueFromOutgoingContext retrieves a deep copy of the metadata for the given key
// from the context, performing a case-insensitive search if needed. Returns nil if not found.
func ValueFromOutgoingContext(ctx context.Context, key string) []string {
md, ok := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
if !ok { if !ok {
return nil return NewIncomingContext(ctx, md)
} }
for k, v := range md {
if v, ok := md.md[key]; ok { omd.Set(k, v)
return copyOf(v)
} }
for k, v := range md.md { return NewIncomingContext(ctx, omd)
// Case-insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
} }

140
metadata/context_test.go Normal file
View File

@@ -0,0 +1,140 @@
package metadata
import (
"context"
"testing"
)
func TestFromNilContext(t *testing.T) {
// nolint: staticcheck
c, ok := FromContext(nil)
if ok || c != nil {
t.Fatal("FromContext not works")
}
}
func TestNewNilContext(t *testing.T) {
// nolint: staticcheck
ctx := NewContext(nil, New(0))
c, ok := FromContext(ctx)
if c == nil || !ok {
t.Fatal("NewContext not works")
}
}
func TestFromContext(t *testing.T) {
ctx := context.WithValue(context.TODO(), mdKey{}, &rawMetadata{New(0)})
c, ok := FromContext(ctx)
if c == nil || !ok {
t.Fatal("FromContext not works")
}
}
func TestNewContext(t *testing.T) {
ctx := NewContext(context.TODO(), New(0))
c, ok := FromContext(ctx)
if c == nil || !ok {
t.Fatal("NewContext not works")
}
}
func TestFromIncomingContext(t *testing.T) {
ctx := context.WithValue(context.TODO(), mdIncomingKey{}, &rawMetadata{New(0)})
c, ok := FromIncomingContext(ctx)
if c == nil || !ok {
t.Fatal("FromIncomingContext not works")
}
}
func TestFromOutgoingContext(t *testing.T) {
ctx := context.WithValue(context.TODO(), mdOutgoingKey{}, &rawMetadata{New(0)})
c, ok := FromOutgoingContext(ctx)
if c == nil || !ok {
t.Fatal("FromOutgoingContext not works")
}
}
func TestSetIncomingContext(t *testing.T) {
md := New(1)
md.Set("key", "val")
ctx := context.WithValue(context.TODO(), mdIncomingKey{}, &rawMetadata{})
if !SetIncomingContext(ctx, md) {
t.Fatal("SetIncomingContext not works")
}
md, ok := FromIncomingContext(ctx)
if md == nil || !ok {
t.Fatal("SetIncomingContext not works")
} else if v, ok := md.Get("key"); !ok || v != "val" {
t.Fatal("SetIncomingContext not works")
}
}
func TestSetOutgoingContext(t *testing.T) {
md := New(1)
md.Set("key", "val")
ctx := context.WithValue(context.TODO(), mdOutgoingKey{}, &rawMetadata{})
if !SetOutgoingContext(ctx, md) {
t.Fatal("SetOutgoingContext not works")
}
md, ok := FromOutgoingContext(ctx)
if md == nil || !ok {
t.Fatal("SetOutgoingContext not works")
} else if v, ok := md.Get("key"); !ok || v != "val" {
t.Fatal("SetOutgoingContext not works")
}
}
func TestNewIncomingContext(t *testing.T) {
md := New(1)
md.Set("key", "val")
ctx := NewIncomingContext(context.TODO(), md)
c, ok := FromIncomingContext(ctx)
if c == nil || !ok {
t.Fatal("NewIncomingContext not works")
}
}
func TestNewOutgoingContext(t *testing.T) {
md := New(1)
md.Set("key", "val")
ctx := NewOutgoingContext(context.TODO(), md)
c, ok := FromOutgoingContext(ctx)
if c == nil || !ok {
t.Fatal("NewOutgoingContext not works")
}
}
func TestAppendIncomingContext(t *testing.T) {
md := New(1)
md.Set("key1", "val1")
ctx := AppendIncomingContext(context.TODO(), "key2", "val2")
nmd, ok := FromIncomingContext(ctx)
if nmd == nil || !ok {
t.Fatal("AppendIncomingContext not works")
}
if v, ok := nmd.Get("key2"); !ok || v != "val2" {
t.Fatal("AppendIncomingContext not works")
}
}
func TestAppendOutgoingContext(t *testing.T) {
md := New(1)
md.Set("key1", "val1")
ctx := AppendOutgoingContext(context.TODO(), "key2", "val2")
nmd, ok := FromOutgoingContext(ctx)
if nmd == nil || !ok {
t.Fatal("AppendOutgoingContext not works")
}
if v, ok := nmd.Get("key2"); !ok || v != "val2" {
t.Fatal("AppendOutgoingContext not works")
}
}

View File

@@ -1,19 +0,0 @@
// Package metadata is a way of defining message headers
package metadata
var (
// HeaderTopic is the header name that contains topic name.
HeaderTopic = "Micro-Topic"
// HeaderContentType specifies content type of message.
HeaderContentType = "Content-Type"
// HeaderEndpoint specifies endpoint in service.
HeaderEndpoint = "Micro-Endpoint"
// HeaderService specifies service.
HeaderService = "Micro-Service"
// HeaderTimeout specifies timeout of operation.
HeaderTimeout = "Micro-Timeout"
// HeaderAuthorization specifies Authorization header.
HeaderAuthorization = "Authorization"
// HeaderXRequestID specifies request id.
HeaderXRequestID = "X-Request-Id"
)

View File

@@ -1,7 +0,0 @@
package metadata
func copyOf(v []string) []string {
vals := make([]string, len(v))
copy(vals, v)
return vals
}

View File

@@ -1,37 +0,0 @@
package metadata
import "sort"
type Iterator struct {
md Metadata
keys []string
cur int
cnt int
}
// Next advances the iterator to the next element.
func (iter *Iterator) Next(k *string, v *[]string) bool {
if iter.cur+1 > iter.cnt {
return false
}
if k != nil && v != nil {
*k = iter.keys[iter.cur]
vv := iter.md[*k]
*v = make([]string, len(vv))
copy(*v, vv)
iter.cur++
}
return true
}
// Iterator returns an iterator for iterating over metadata in sorted order.
func (md Metadata) Iterator() *Iterator {
iter := &Iterator{md: md, cnt: len(md)}
iter.keys = make([]string, 0, iter.cnt)
for k := range md {
iter.keys = append(iter.keys, k)
}
sort.Strings(iter.keys)
return iter
}

View File

@@ -1,160 +1,143 @@
package metadata // Package metadata is a way of defining message headers
package metadata // import "go.unistack.org/micro/v4/metadata"
import ( import (
"fmt"
"net/textproto" "net/textproto"
"strings" "sort"
) )
// defaultMetadataSize is used when initializing new Metadata. var (
// HeaderTopic is the header name that contains topic name
HeaderTopic = "Micro-Topic"
// HeaderContentType specifies content type of message
HeaderContentType = "Content-Type"
// HeaderEndpoint specifies endpoint in service
HeaderEndpoint = "Micro-Endpoint"
// HeaderService specifies service
HeaderService = "Micro-Service"
// HeaderTimeout specifies timeout of operation
HeaderTimeout = "Micro-Timeout"
// HeaderAuthorization specifies Authorization header
HeaderAuthorization = "Authorization"
// HeaderXRequestID specifies request id
HeaderXRequestID = "X-Request-Id"
)
// Metadata is our way of representing request headers internally.
// They're used at the RPC level and translate back and forth
// from Transport headers.
type Metadata map[string]string
type rawMetadata struct {
md Metadata
}
// defaultMetadataSize used when need to init new Metadata
var defaultMetadataSize = 2 var defaultMetadataSize = 2
// Metadata maps keys to values. Use the New, NewWithMetadata and Pairs functions to create it. // Iterator used to iterate over metadata with order
type Metadata map[string][]string type Iterator struct {
md Metadata
// New creates a zero-value Metadata with the specified size. keys []string
func New(l int) Metadata { cur int
if l == 0 { cnt int
l = defaultMetadataSize
}
md := make(Metadata, l)
return md
} }
// NewWithMetadata creates a Metadata from the provided key-value map. // Next advance iterator to next element
func NewWithMetadata(m map[string]string) Metadata { func (iter *Iterator) Next(k, v *string) bool {
md := make(Metadata, len(m)) if iter.cur+1 > iter.cnt {
for key, val := range m { return false
md[key] = append(md[key], val)
} }
return md
*k = iter.keys[iter.cur]
*v = iter.md[*k]
iter.cur++
return true
} }
// Pairs returns a Metadata formed from the key-value mapping. It panics if the length of kv is odd. // Iterator returns the itarator for metadata in sorted order
func Pairs(kv ...string) Metadata { func (md Metadata) Iterator() *Iterator {
iter := &Iterator{md: md, cnt: len(md)}
iter.keys = make([]string, 0, iter.cnt)
for k := range md {
iter.keys = append(iter.keys, k)
}
sort.Strings(iter.keys)
return iter
}
// Get returns value from metadata by key
func (md Metadata) Get(key string) (string, bool) {
// fast path
val, ok := md[key]
if !ok {
// slow path
val, ok = md[textproto.CanonicalMIMEHeaderKey(key)]
}
return val, ok
}
// Set is used to store value in metadata
func (md Metadata) Set(kv ...string) {
if len(kv)%2 == 1 { if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: Pairs got the odd number of input pairs for metadata: %d", len(kv))) kv = kv[:len(kv)-1]
} }
md := make(Metadata, len(kv)/2) for idx := 0; idx < len(kv); idx += 2 {
for i := 0; i < len(kv); i += 2 { md[textproto.CanonicalMIMEHeaderKey(kv[idx])] = kv[idx+1]
md[kv[i]] = append(md[kv[i]], kv[i+1])
} }
return md
} }
// Join combines multiple Metadatas into a single Metadata. // Del is used to remove value from metadata
// The order of values for each key is determined by the order in which the Metadatas are provided to Join. func (md Metadata) Del(keys ...string) {
func Join(mds ...Metadata) Metadata { for _, key := range keys {
out := Metadata{} // fast path
for _, md := range mds { delete(md, key)
for k, v := range md { // slow path
out[k] = append(out[k], v...) delete(md, textproto.CanonicalMIMEHeaderKey(key))
}
}
// Copy makes a copy of the metadata
func Copy(md Metadata) Metadata {
nmd := New(len(md))
for key, val := range md {
nmd.Set(key, val)
}
return nmd
}
// New return new sized metadata
func New(size int) Metadata {
if size == 0 {
size = defaultMetadataSize
}
return make(Metadata, size)
}
// Merge merges metadata to existing metadata, overwriting if specified
func Merge(omd Metadata, mmd Metadata, overwrite bool) Metadata {
var ok bool
nmd := Copy(omd)
for key, val := range mmd {
_, ok = nmd[key]
switch {
case ok && !overwrite:
continue
case val != "":
nmd.Set(key, val)
case ok && val == "":
nmd.Del(key)
} }
} }
return out return nmd
} }
// Copy returns a deep copy of Metadata. // Pairs from which metadata created
func Copy(src Metadata) Metadata { func Pairs(kv ...string) (Metadata, bool) {
out := make(Metadata, len(src)) if len(kv)%2 == 1 {
for k, v := range src { return nil, false
out[k] = copyOf(v)
}
return out
}
// Copy returns a deep copy of Metadata.
func (md Metadata) Copy() Metadata {
out := make(Metadata, len(md))
for k, v := range md {
out[k] = copyOf(v)
}
return out
}
// CopyTo performs a deep copy of Metadata to the out.
func (md Metadata) CopyTo(out Metadata) {
for k, v := range md {
out[k] = copyOf(v)
}
}
// Len returns the number of items in Metadata.
func (md Metadata) Len() int {
return len(md)
}
// AsMap returns a deep copy of Metadata as a map[string]string
func (md Metadata) AsMap() map[string]string {
out := make(map[string]string, len(md))
for k, v := range md {
out[k] = strings.Join(v, ",")
}
return out
}
// AsHTTP1 returns a deep copy of Metadata with keys converted to canonical MIME header key format.
func (md Metadata) AsHTTP1() map[string][]string {
out := make(map[string][]string, len(md))
for k, v := range md {
out[textproto.CanonicalMIMEHeaderKey(k)] = copyOf(v)
}
return out
}
// AsHTTP2 returns a deep copy of Metadata with keys converted to lowercase.
func (md Metadata) AsHTTP2() map[string][]string {
out := make(map[string][]string, len(md))
for k, v := range md {
out[strings.ToLower(k)] = copyOf(v)
}
return out
}
// Get retrieves the values for a given key, checking the key in three formats:
// - exact case,
// - lower case,
// - canonical MIME header key format.
func (md Metadata) Get(k string) []string {
v, ok := md[k]
if !ok {
v, ok = md[strings.ToLower(k)]
}
if !ok {
v = md[textproto.CanonicalMIMEHeaderKey(k)]
}
return v
}
// GetJoined retrieves the values for a given key and joins them into a single string, separated by commas.
func (md Metadata) GetJoined(k string) string {
return strings.Join(md.Get(k), ",")
}
// Set assigns the values to the given key.
func (md Metadata) Set(key string, vals ...string) {
if len(vals) == 0 {
return
}
md[key] = vals
}
// Append adds values to the existing values for the given key.
func (md Metadata) Append(key string, vals ...string) {
if len(vals) == 0 {
return
}
md[key] = append(md[key], vals...)
}
// Del removes the values for the given keys k. It checks and removes the keys in the following formats:
// - exact case,
// - lower case,
// - canonical MIME header key format.
func (md Metadata) Del(k ...string) {
for i := range k {
delete(md, k[i])
delete(md, strings.ToLower(k[i]))
delete(md, textproto.CanonicalMIMEHeaderKey(k[i]))
} }
md := New(len(kv) / 2)
md.Set(kv...)
return md, true
} }

View File

@@ -5,64 +5,50 @@ import (
"testing" "testing"
) )
func TesSet(t *testing.T) { func TestMetadataSetMultiple(t *testing.T) {
md := Pairs("key1", "val1", "key2", "val2") md := New(4)
md.Set("key1", "val2", "val3") md.Set("key1", "val1", "key2", "val2", "key3")
v := md.GetJoined("X-Request-Id")
if v != "val2, val3" { if v, ok := md.Get("key1"); !ok || v != "val1" {
t.Fatal("set not works") t.Fatalf("invalid kv %#+v", md)
}
if v, ok := md.Get("key2"); !ok || v != "val2" {
t.Fatalf("invalid kv %#+v", md)
}
if _, ok := md.Get("key3"); ok {
t.Fatalf("invalid kv %#+v", md)
} }
} }
/* func TestAppend(t *testing.T) {
func TestAppendOutgoingContextModify(t *testing.T) { ctx := context.Background()
md := Pairs("key1", "val1") ctx = AppendIncomingContext(ctx, "key1", "val1", "key2", "val2")
ctx := NewOutgoingContext(context.TODO(), md) md, ok := FromIncomingContext(ctx)
nctx := AppendOutgoingContext(ctx, "key1", "val3", "key2", "val2") if !ok {
_ = nctx t.Fatal("metadata empty")
omd := MustOutgoingContext(nctx) }
fmt.Printf("%#+v\n", omd) if _, ok := md.Get("key1"); !ok {
} t.Fatal("key1 not found")
*/
func TestLowercase(t *testing.T) {
md := New(1)
md["x-request-id"] = []string{"12345"}
v := md.GetJoined("X-Request-Id")
if v == "" {
t.Fatalf("metadata invalid %#+v", md)
} }
}
func TestMultipleUsage(t *testing.T) {
ctx := context.TODO()
md := New(0)
md.Set("key1_1", "val1_1", "key1_2", "val1_2", "key1_3", "val1_3")
ctx = NewIncomingContext(ctx, Copy(md))
ctx = NewOutgoingContext(ctx, Copy(md))
imd, _ := FromIncomingContext(ctx)
omd, _ := FromOutgoingContext(ctx)
_ = func(x context.Context) context.Context {
m, _ := FromIncomingContext(x)
m.Del("key1_2")
return ctx
}(ctx)
_ = func(x context.Context) context.Context {
m, _ := FromIncomingContext(x)
m.Del("key1_3")
return ctx
}(ctx)
_ = imd
_ = omd
} }
func TestPairs(t *testing.T) { func TestPairs(t *testing.T) {
md := Pairs("key1", "val1", "key2", "val2") md, ok := Pairs("key1", "val1", "key2", "val2")
if v := md.Get("key1"); v == nil { if !ok {
t.Fatal("odd number of kv")
}
if _, ok = md.Get("key1"); !ok {
t.Fatal("key1 not found") t.Fatal("key1 not found")
} }
} }
func testCtx(ctx context.Context) {
md := New(2)
md.Set("Key1", "Val1_new")
md.Set("Key3", "Val3")
SetOutgoingContext(ctx, md)
}
func TestPassing(t *testing.T) { func TestPassing(t *testing.T) {
ctx := context.TODO() ctx := context.TODO()
md1 := New(2) md1 := New(2)
@@ -70,63 +56,63 @@ func TestPassing(t *testing.T) {
md1.Set("Key2", "Val2") md1.Set("Key2", "Val2")
ctx = NewIncomingContext(ctx, md1) ctx = NewIncomingContext(ctx, md1)
testCtx(ctx)
_, ok := FromOutgoingContext(ctx)
if ok {
t.Fatalf("create outgoing context")
}
ctx = NewOutgoingContext(ctx, md1)
md, ok := FromOutgoingContext(ctx) md, ok := FromOutgoingContext(ctx)
if !ok { if !ok {
t.Fatalf("missing metadata from outgoing context") t.Fatalf("missing metadata from outgoing context")
} }
if v := md.Get("Key1"); v == nil || v[0] != "Val1" { if v, ok := md.Get("Key1"); !ok || v != "Val1_new" {
t.Fatalf("invalid metadata value %#+v", md) t.Fatalf("invalid metadata value %#+v", md)
} }
} }
func TestIterator(t *testing.T) { func TestMerge(t *testing.T) {
md := Pairs( omd := Metadata{
"1Last", "last", "key1": "val1",
"2First", "first", }
"3Second", "second", mmd := Metadata{
) "key2": "val2",
iter := md.Iterator()
var k string
var v []string
chk := New(3)
for iter.Next(&k, &v) {
chk[k] = v
} }
for k, v := range chk { nmd := Merge(omd, mmd, true)
if cv, ok := md[k]; !ok || len(cv) != len(v) || cv[0] != v[0] { if len(nmd) != 2 {
t.Fatalf("XXXX %#+v %#+v", chk, md) t.Fatalf("merge failed: %v", nmd)
} }
}
func TestIterator(t *testing.T) {
md := Metadata{
"1Last": "last",
"2First": "first",
"3Second": "second",
}
iter := md.Iterator()
var k, v string
for iter.Next(&k, &v) {
// fmt.Printf("k: %s, v: %s\n", k, v)
} }
} }
func TestMedataCanonicalKey(t *testing.T) { func TestMedataCanonicalKey(t *testing.T) {
md := New(1) md := New(1)
md.Set("x-request-id", "12345") md.Set("x-request-id", "12345")
v := md.GetJoined("x-request-id") v, ok := md.Get("x-request-id")
if v == "" { if !ok {
t.Fatalf("failed to get x-request-id") t.Fatalf("failed to get x-request-id")
} else if v != "12345" { } else if v != "12345" {
t.Fatalf("invalid metadata value: %s != %s", "12345", v) t.Fatalf("invalid metadata value: %s != %s", "12345", v)
} }
v = md.GetJoined("X-Request-Id") v, ok = md.Get("X-Request-Id")
if v == "" { if !ok {
t.Fatalf("failed to get x-request-id") t.Fatalf("failed to get x-request-id")
} else if v != "12345" { } else if v != "12345" {
t.Fatalf("invalid metadata value: %s != %s", "12345", v) t.Fatalf("invalid metadata value: %s != %s", "12345", v)
} }
v = md.GetJoined("X-Request-ID") v, ok = md.Get("X-Request-ID")
if v == "" { if !ok {
t.Fatalf("failed to get x-request-id") t.Fatalf("failed to get x-request-id")
} else if v != "12345" { } else if v != "12345" {
t.Fatalf("invalid metadata value: %s != %s", "12345", v) t.Fatalf("invalid metadata value: %s != %s", "12345", v)
@@ -138,8 +124,8 @@ func TestMetadataSet(t *testing.T) {
md.Set("Key", "val") md.Set("Key", "val")
val := md.GetJoined("Key") val, ok := md.Get("Key")
if val == "" { if !ok {
t.Fatal("key Key not found") t.Fatal("key Key not found")
} }
if val != "val" { if val != "val" {
@@ -149,27 +135,36 @@ func TestMetadataSet(t *testing.T) {
func TestMetadataDelete(t *testing.T) { func TestMetadataDelete(t *testing.T) {
md := Metadata{ md := Metadata{
"Foo": []string{"bar"}, "Foo": "bar",
"Baz": []string{"empty"}, "Baz": "empty",
} }
md.Del("Baz") md.Del("Baz")
v := md.Get("Baz") _, ok := md.Get("Baz")
if v != nil { if ok {
t.Fatal("key Baz not deleted") t.Fatal("key Baz not deleted")
} }
} }
func TestNilContext(t *testing.T) {
var ctx context.Context
_, ok := FromContext(ctx)
if ok {
t.Fatal("nil context")
}
}
func TestMetadataCopy(t *testing.T) { func TestMetadataCopy(t *testing.T) {
md := Metadata{ md := Metadata{
"Foo": []string{"bar"}, "Foo": "bar",
"Bar": []string{"baz"}, "Bar": "baz",
} }
cp := Copy(md) cp := Copy(md)
for k, v := range md { for k, v := range md {
if cv := cp[k]; cv[0] != v[0] { if cv := cp[k]; cv != v {
t.Fatalf("Got %s:%s for %s:%s", k, cv, k, v) t.Fatalf("Got %s:%s for %s:%s", k, cv, k, v)
} }
} }
@@ -177,7 +172,7 @@ func TestMetadataCopy(t *testing.T) {
func TestMetadataContext(t *testing.T) { func TestMetadataContext(t *testing.T) {
md := Metadata{ md := Metadata{
"Foo": []string{"bar"}, "Foo": "bar",
} }
ctx := NewContext(context.TODO(), md) ctx := NewContext(context.TODO(), md)
@@ -187,7 +182,7 @@ func TestMetadataContext(t *testing.T) {
t.Errorf("Unexpected error retrieving metadata, got %t", ok) t.Errorf("Unexpected error retrieving metadata, got %t", ok)
} }
if emd["Foo"][0] != md["Foo"][0] { if emd["Foo"] != md["Foo"] {
t.Errorf("Expected key: %s val: %s, got key: %s val: %s", "Foo", md["Foo"], "Foo", emd["Foo"]) t.Errorf("Expected key: %s val: %s, got key: %s val: %s", "Foo", md["Foo"], "Foo", emd["Foo"])
} }
@@ -195,75 +190,3 @@ func TestMetadataContext(t *testing.T) {
t.Errorf("Expected metadata length 1 got %d", i) t.Errorf("Expected metadata length 1 got %d", i)
} }
} }
func TestFromContext(t *testing.T) {
ctx := context.WithValue(context.TODO(), metadataCurrentKey{}, rawMetadata{md: New(0)})
c, ok := FromContext(ctx)
if c == nil || !ok {
t.Fatal("FromContext not works")
}
}
func TestNewContext(t *testing.T) {
ctx := NewContext(context.TODO(), New(0))
c, ok := FromContext(ctx)
if c == nil || !ok {
t.Fatal("NewContext not works")
}
}
func TestFromIncomingContext(t *testing.T) {
ctx := context.WithValue(context.TODO(), metadataIncomingKey{}, rawMetadata{md: New(0)})
c, ok := FromIncomingContext(ctx)
if c == nil || !ok {
t.Fatal("FromIncomingContext not works")
}
}
func TestFromOutgoingContext(t *testing.T) {
ctx := context.WithValue(context.TODO(), metadataOutgoingKey{}, rawMetadata{md: New(0)})
c, ok := FromOutgoingContext(ctx)
if c == nil || !ok {
t.Fatal("FromOutgoingContext not works")
}
}
func TestNewIncomingContext(t *testing.T) {
md := New(1)
md.Set("key", "val")
ctx := NewIncomingContext(context.TODO(), md)
c, ok := FromIncomingContext(ctx)
if c == nil || !ok {
t.Fatal("NewIncomingContext not works")
}
}
func TestNewOutgoingContext(t *testing.T) {
md := New(1)
md.Set("key", "val")
ctx := NewOutgoingContext(context.TODO(), md)
c, ok := FromOutgoingContext(ctx)
if c == nil || !ok {
t.Fatal("NewOutgoingContext not works")
}
}
func TestAppendOutgoingContext(t *testing.T) {
md := New(1)
md.Set("key1", "val1")
ctx := AppendOutgoingContext(context.TODO(), "key2", "val2")
nmd, ok := FromOutgoingContext(ctx)
if nmd == nil || !ok {
t.Fatal("AppendOutgoingContext not works")
}
if v := nmd.GetJoined("key2"); v != "val2" {
t.Fatal("AppendOutgoingContext not works")
}
}

View File

@@ -15,15 +15,6 @@ func FromContext(ctx context.Context) (Meter, bool) {
return c, ok return c, ok
} }
// MustContext get meter from context
func MustContext(ctx context.Context) Meter {
m, ok := FromContext(ctx)
if !ok {
panic("missing meter")
}
return m
}
// NewContext put meter in context // NewContext put meter in context
func NewContext(ctx context.Context, c Meter) context.Context { func NewContext(ctx context.Context, c Meter) context.Context {
if ctx == nil { if ctx == nil {
@@ -31,13 +22,3 @@ func NewContext(ctx context.Context, c Meter) context.Context {
} }
return context.WithValue(ctx, meterKey{}, c) return context.WithValue(ctx, meterKey{}, c)
} }
// SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option {
return func(o *Options) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}

View File

@@ -40,14 +40,3 @@ func TestNewContext(t *testing.T) {
t.Fatal("NewContext not works") t.Fatal("NewContext not works")
} }
} }
func TestSetOption(t *testing.T) {
type key struct{}
o := SetOption(key{}, "test")
opts := &Options{}
o(opts)
if v, ok := opts.Context.Value(key{}).(string); !ok || v == "" {
t.Fatal("SetOption not works")
}
}

View File

@@ -1,5 +1,5 @@
// Package meter is for instrumentation // Package meter is for instrumentation
package meter package meter // import "go.unistack.org/micro/v4/meter"
import ( import (
"io" "io"
@@ -7,6 +7,8 @@ import (
"strconv" "strconv"
"strings" "strings"
"time" "time"
"go.unistack.org/micro/v4/options"
) )
var ( var (
@@ -16,19 +18,14 @@ var (
DefaultAddress = ":9090" DefaultAddress = ":9090"
// DefaultPath the meter endpoint where the Meter data will be made available // DefaultPath the meter endpoint where the Meter data will be made available
DefaultPath = "/metrics" DefaultPath = "/metrics"
// DefaultMeterStatsInterval specifies interval for meter updating // DefaultMetricPrefix holds the string that prepends to all metrics
DefaultMeterStatsInterval = 5 * time.Second DefaultMetricPrefix = "micro_"
// DefaultLabelPrefix holds the string that prepends to all labels
DefaultLabelPrefix = "micro_"
// DefaultSummaryQuantiles is the default spread of stats for summary // DefaultSummaryQuantiles is the default spread of stats for summary
DefaultSummaryQuantiles = []float64{0.5, 0.9, 0.97, 0.99, 1} DefaultSummaryQuantiles = []float64{0.5, 0.9, 0.97, 0.99, 1}
// DefaultSummaryWindow is the default window for summary // DefaultSummaryWindow is the default window for summary
DefaultSummaryWindow = 5 * time.Minute DefaultSummaryWindow = 5 * time.Minute
// DefaultSkipEndpoints is the slice of endpoint that must not be metered
DefaultSkipEndpoints = []string{
"MeterService.Metrics",
"HealthService.Live",
"HealthService.Ready",
"HealthService.Version",
}
) )
// Meter is an interface for collecting and instrumenting metrics // Meter is an interface for collecting and instrumenting metrics
@@ -36,9 +33,9 @@ type Meter interface {
// Name returns meter name // Name returns meter name
Name() string Name() string
// Init initialize meter // Init initialize meter
Init(opts ...Option) error Init(opts ...options.Option) error
// Clone create meter copy with new options // Clone create meter copy with new options
Clone(opts ...Option) Meter Clone(opts ...options.Option) Meter
// Counter get or create counter // Counter get or create counter
Counter(name string, labels ...string) Counter Counter(name string, labels ...string) Counter
// FloatCounter get or create float counter // FloatCounter get or create float counter
@@ -46,7 +43,7 @@ type Meter interface {
// Gauge get or create gauge // Gauge get or create gauge
Gauge(name string, fn func() float64, labels ...string) Gauge Gauge(name string, fn func() float64, labels ...string) Gauge
// Set create new meter metrics set // Set create new meter metrics set
Set(opts ...Option) Meter Set(opts ...options.Option) Meter
// Histogram get or create histogram // Histogram get or create histogram
Histogram(name string, labels ...string) Histogram Histogram(name string, labels ...string) Histogram
// Summary get or create summary // Summary get or create summary
@@ -54,7 +51,7 @@ type Meter interface {
// SummaryExt get or create summary with spcified quantiles and window time // SummaryExt get or create summary with spcified quantiles and window time
SummaryExt(name string, window time.Duration, quantiles []float64, labels ...string) Summary SummaryExt(name string, window time.Duration, quantiles []float64, labels ...string) Summary
// Write writes metrics to io.Writer // Write writes metrics to io.Writer
Write(w io.Writer, opts ...Option) error Write(w io.Writer, opts ...options.Option) error
// Options returns meter options // Options returns meter options
Options() Options Options() Options
// String return meter type // String return meter type

Some files were not shown because too many files have changed in this diff Show More