Compare commits

...

121 Commits

Author SHA1 Message Date
46eb739dff broker: add ErrorHandler
Some checks failed
coverage / build (push) Failing after 4m49s
test / test (push) Failing after 16m1s
sync / sync (push) Failing after 20s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-12-16 08:34:55 +03:00
13b01f59ee logger: conditional caller field
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-12-16 08:34:55 +03:00
52c8f3da86 Merge pull request 'add opt gracefultimeout broker' (#410) from devstigneev/micro:v4_new_opts into v4
Some checks failed
test / test (push) Failing after 12m37s
coverage / build (push) Failing after 12m51s
sync / sync (push) Failing after 16s
Reviewed-on: #410
2025-12-10 15:22:35 +03:00
Evstigneev Denis
e7f9f638bd add opt gracefultimeout broker
Some checks failed
test / test (pull_request) Failing after 13m43s
lint / lint (pull_request) Failing after 14m9s
coverage / build (pull_request) Failing after 14m25s
2025-12-10 15:20:14 +03:00
d9afc9ce4f update all
Some checks failed
coverage / build (push) Failing after 3m9s
test / test (push) Failing after 18m22s
sync / sync (push) Successful in 9s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-10-31 21:52:22 +03:00
7a325e2c9e remove using global map for default codecs (#223)
Some checks failed
test / test (push) Failing after 15m6s
coverage / build (push) Failing after 15m16s
sync / sync (push) Failing after 8s
2025-10-15 21:32:52 +03:00
7daa927e70 add HistogramExt method with custom quantiles
Some checks failed
coverage / build (push) Successful in 4m4s
test / test (push) Failing after 18m1s
sync / sync (push) Successful in 26s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-10-12 15:55:00 +03:00
vtolstov
54bb7f7acb Apply Code Coverage Badge 2025-10-12 11:27:04 +00:00
9eaab95519 meter: improve Gauge
All checks were successful
sync / sync (push) Successful in 1m56s
coverage / build (push) Successful in 3m55s
test / test (push) Successful in 4m12s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-10-12 14:24:44 +03:00
vtolstov
9219dc6b2a Apply Code Coverage Badge 2025-10-11 15:49:04 +00:00
52607b38f1 logger: fixup Fatal finalizers
All checks were successful
coverage / build (push) Successful in 2m0s
test / test (push) Successful in 3m15s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-10-11 18:46:42 +03:00
vtolstov
886f046409 Apply Code Coverage Badge 2025-10-10 12:30:04 +00:00
4d6d469d40 logger: add Fatal finalizers
All checks were successful
coverage / build (push) Successful in 2m37s
test / test (push) Successful in 4m49s
* closes #222

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-10-10 15:28:10 +03:00
vtolstov
4a944274f4 Apply Code Coverage Badge 2025-10-07 20:56:10 +00:00
b0cbddcfdd meter: improve meter usage across micro framework (#409)
All checks were successful
sync / sync (push) Successful in 1m41s
coverage / build (push) Successful in 3m13s
test / test (push) Successful in 4m2s
Reviewed-on: #409
Co-authored-by: Vasiliy Tolstov <v.tolstov@unistack.org>
Co-committed-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-10-07 23:54:20 +03:00
vtolstov
d0534a7d05 Apply Code Coverage Badge 2025-09-20 19:59:32 +00:00
ab051405c5 initial hasql support (#407)
Some checks failed
coverage / build (push) Successful in 3m47s
test / test (push) Failing after 17m14s
closes #403

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
Reviewed-on: #407
Co-authored-by: Vasiliy Tolstov <v.tolstov@unistack.org>
Co-committed-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-09-20 22:57:39 +03:00
vtolstov
268b3dbff4 Apply Code Coverage Badge 2025-07-12 21:20:05 +00:00
f9d2c14597 fixup tests
Some checks failed
sync / sync (push) Successful in 1m8s
coverage / build (push) Successful in 2m3s
test / test (push) Failing after 2m55s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-07-13 00:11:08 +03:00
e6bf914dd9 tracer: write log fields only if span exists and recording
Some checks failed
coverage / build (push) Failing after 1m14s
test / test (push) Has been cancelled
sync / sync (push) Successful in 1m37s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-07-13 00:08:30 +03:00
b59f4a16f0 meter: disable auto sorting labels
Some checks failed
coverage / build (push) Failing after 1m39s
test / test (push) Successful in 4m37s
sync / sync (push) Successful in 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-06-17 19:02:06 +03:00
3deb572f72 [v4] fix out-of-bounds behavior in seeker buffer and add tests (#219)
Some checks failed
coverage / build (push) Failing after 2m12s
test / test (push) Successful in 4m27s
sync / sync (push) Successful in 7s
* add check negative position to Read() and write tests

* add tests for Write() method

* add tests for Write() method

* add checks of whence and negative position to Seek() and write tests

* add tests for Rewind()

* add tests for Close()

* add tests for Reset()

* add tests for Len()

* add tests for Bytes()

* tests polishing

* tests polishing

* tests polishing

* tests polishing
2025-06-15 17:24:48 +03:00
0e668c0f0f fixup tests
Some checks failed
coverage / build (push) Failing after 2m13s
test / test (push) Failing after 19m18s
sync / sync (push) Successful in 19s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-06-09 17:36:11 +03:00
2bac878845 broker: fix message options
Some checks failed
coverage / build (push) Failing after 1m58s
test / test (push) Has started running
sync / sync (push) Successful in 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-06-09 17:23:30 +03:00
9ee31fb5a6 fixup compile
Some checks failed
coverage / build (push) Has been cancelled
test / test (push) Has been cancelled
sync / sync (push) Successful in 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-29 12:46:23 +03:00
ed5d30a58e store/noop: fixup Exists
Some checks failed
coverage / build (push) Has been cancelled
test / test (push) Has been cancelled
sync / sync (push) Has been cancelled
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-29 12:43:39 +03:00
vtolstov
b4b67a8b41 Apply Code Coverage Badge 2025-05-25 02:41:23 +00:00
13f90ff716 changed embedded mutex to private field (#217)
Some checks failed
sync / sync (push) Failing after 16m12s
test / test (push) Failing after 17m28s
coverage / build (push) Failing after 17m40s
2025-05-25 01:15:03 +03:00
0f8f12aee0 add tracer enabled status
Some checks failed
coverage / build (push) Successful in 2m52s
test / test (push) Failing after 18m53s
sync / sync (push) Successful in 26s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-19 09:33:01 +03:00
8b406cf963 util/buffer: add Reset() method
Some checks failed
coverage / build (push) Failing after 1m36s
test / test (push) Successful in 3m35s
sync / sync (push) Successful in 7s
closes #402

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-12 19:18:45 +03:00
029a434a2b broker: pass broker content type if message options not pass it
All checks were successful
coverage / build (push) Successful in 1m44s
test / test (push) Successful in 3m5s
sync / sync (push) Successful in 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-09 13:51:35 +03:00
vtolstov
847259bc39 Apply Code Coverage Badge 2025-05-09 09:36:02 +00:00
a1ee8728ad broker: add Content-Type and DefaultContentType
All checks were successful
coverage / build (push) Successful in 1m58s
sync / sync (push) Successful in 1m37s
test / test (push) Successful in 3m47s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-09 12:34:46 +03:00
88a5875cfb switch yaml package to maintained one
All checks were successful
coverage / build (push) Successful in 2m16s
test / test (push) Successful in 4m37s
sync / sync (push) Successful in 8s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-09 12:18:49 +03:00
03ee33040c util/id: switch to default uuid package
All checks were successful
coverage / build (push) Successful in 2m21s
test / test (push) Successful in 4m50s
sync / sync (push) Successful in 6s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-08 19:07:00 +03:00
0144f175f0 add comment
All checks were successful
coverage / build (push) Successful in 1m33s
test / test (push) Successful in 3m41s
sync / sync (push) Successful in 8s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-06 23:00:15 +03:00
b3539a32ab logger: add none level
closes #399

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-06 23:00:15 +03:00
vtolstov
6a7223ea4a Apply Code Coverage Badge 2025-05-06 08:09:12 +00:00
1a1b67866a [v4] improve metadata documentation (#216)
All checks were successful
coverage / build (push) Successful in 1m47s
test / test (push) Successful in 2m50s
* add usage docs for context types and metadata, improve comments

* changes after review
2025-05-06 11:02:27 +03:00
b7c98da6d1 added commit hash check to avoid unnecessary repository cloning (#215)
All checks were successful
sync / sync (push) Successful in 16s
2025-05-05 14:53:28 +03:00
2c21cce076 tracer/noop: disable allocation for trace and span id
All checks were successful
coverage / build (push) Successful in 2m22s
test / test (push) Successful in 4m22s
sync / sync (push) Successful in 26s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-05 09:48:22 +03:00
c8946dcdc8 fix sync
All checks were successful
sync / sync (push) Successful in 24s
2025-05-04 15:03:22 +03:00
vtolstov
d342ff2626 Apply Code Coverage Badge 2025-05-02 06:23:44 +00:00
f2d0d67d4c hooks/requestid: fixup panic
All checks were successful
coverage / build (push) Successful in 2m0s
test / test (push) Successful in 2m33s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-05-02 09:22:19 +03:00
677dc30af0 [v4] update ci (#213)
All checks were successful
sync / sync (push) Successful in 41s
* update ci

* cleanup
2025-05-01 19:15:06 +03:00
vtolstov
7122cc873c Apply Code Coverage Badge 2025-05-01 15:58:29 +00:00
77e370ffdc move wrapper.sql to hook/sql (#401)
All checks were successful
coverage / build (push) Successful in 2m3s
test / test (push) Successful in 2m42s
move micro-wrapper-sql to core repo

Co-authored-by: Vasiliy Tolstov <v.tolstov@unistack.org>
Reviewed-on: #401
Co-authored-by: Evstigneev Denis <danteevstigneev@yandex.ru>
Co-committed-by: Evstigneev Denis <danteevstigneev@yandex.ru>
2025-05-01 18:56:34 +03:00
vtolstov
7b1c42e50b Apply Code Coverage Badge 2025-04-29 20:18:49 +00:00
f3b9493ac3 hooks/metadata: fix
All checks were successful
coverage / build (push) Successful in 1m20s
test / test (push) Successful in 1m57s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 23:18:10 +03:00
e4ee705eb2 metadata: sync with grpc
Some checks failed
coverage / build (push) Failing after 41s
test / test (push) Successful in 2m31s
sync / sync (push) Successful in 46s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 23:13:57 +03:00
7ff7a3dbe0 update all
All checks were successful
sync / sync (push) Successful in 41s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 18:29:26 +03:00
7af5147f4b update all
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 18:28:33 +03:00
394fd16243 update all
All checks were successful
sync / sync (push) Successful in 36s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 18:16:16 +03:00
2b08c8f682 fixup coverage job
All checks were successful
sync / sync (push) Successful in 43s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 14:13:56 +03:00
f9a7f62d02 fixup workflow
Some checks failed
sync / sync (push) Failing after 1m9s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 14:06:43 +03:00
vtolstov
f5aedf5951 Apply Code Coverage Badge 2025-04-29 10:54:56 +00:00
a5ef231171 cleanup metadata
All checks were successful
sync / sync (push) Successful in 1m20s
coverage / build (push) Successful in 1m23s
test / test (push) Successful in 2m42s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 13:54:16 +03:00
23f2ee9bb7 fixup hooks
All checks were successful
coverage / build (push) Successful in 3m12s
test / test (push) Successful in 5m4s
sync / sync (push) Successful in 31s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 13:09:56 +03:00
88606e89ca fixup metadata
Some checks are pending
coverage / build (push) Waiting to run
test / test (push) Waiting to run
sync / sync (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-29 13:01:37 +03:00
vtolstov
24efbb68bf Apply Code Coverage Badge
All checks were successful
coverage / build (push) Successful in 2m4s
test / test (push) Successful in 3m28s
sync / sync (push) Successful in 25s
2025-04-27 19:16:46 +00:00
vtolstov
cecdaa0fed Apply Code Coverage Badge 2025-04-27 19:13:06 +00:00
vtolstov
9627995cee Apply Code Coverage Badge
All checks were successful
sync / sync (push) Successful in 1m54s
coverage / build (push) Successful in 1m57s
test / test (push) Successful in 2m27s
2025-04-27 19:07:59 +00:00
vtolstov
0f3539dc7b Apply Code Coverage Badge 2025-04-27 19:06:03 +00:00
ff414eff2e [v4] fix flatten map util function (#211)
All checks were successful
sync / sync (push) Successful in 1m51s
coverage / build (push) Successful in 2m17s
test / test (push) Successful in 4m29s
* add the fixed version of FlattenMap() and corresponding tests
* replaced the old FlattenMap() implementation with a new one
2025-04-27 21:44:24 +03:00
vtolstov
fbf6832738 Apply Code Coverage Badge
Some checks are pending
coverage / build (push) Successful in 6m34s
test / test (push) Successful in 8m23s
sync / sync (push) Has started running
2025-04-27 18:21:57 +00:00
vtolstov
59ff1f931b Apply Code Coverage Badge 2025-04-27 18:19:31 +00:00
2030bd2803 attempt to fix coverage/lint/test job (#210) 2025-04-27 21:12:16 +03:00
bb87a87ae5 improve sync
All checks were successful
sync / sync (push) Successful in 13s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 21:02:19 +03:00
0bd5aed7cc rename workflow
All checks were successful
sync / sync (push) Successful in 10s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 16:08:57 +03:00
434798a574 check actions env
All checks were successful
sync / sync (push) Successful in 9s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 16:00:00 +03:00
459a951115 check actions env
All checks were successful
syncpull / pull (push) Successful in 9s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 15:49:50 +03:00
770c2715d4 check actions env
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 15:48:57 +03:00
c93286afd5 [v4] rename .gitea to .github
All checks were successful
syncpull / pull (push) Successful in 10s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 14:10:57 +03:00
vtolstov
6bf118d978 Apply Code Coverage Badge 2025-04-27 11:05:07 +00:00
7493de1168 move hooks (#398)
All checks were successful
coverage / build (push) Successful in 1m18s
test / test (push) Successful in 2m5s
## Pull Request template
Please, go through these steps before clicking submit on this PR.

1. Give a descriptive title to your PR.
2. Provide a description of your changes.
3. Make sure you have some relevant tests.
4. Put `closes #XXXX` in your comment to auto-close the issue that your PR fixes (if applicable).

**PLEASE REMOVE THIS TEMPLATE BEFORE SUBMITTING**

Reviewed-on: #398
Co-authored-by: Evstigneev Denis <danteevstigneev@yandex.ru>
Co-committed-by: Evstigneev Denis <danteevstigneev@yandex.ru>
2025-04-27 14:04:33 +03:00
vtolstov
212a685b50 Apply Code Coverage Badge 2025-04-27 10:58:10 +00:00
3f21bafc2f fixup lint
All checks were successful
coverage / build (push) Successful in 1m17s
test / test (push) Successful in 2m51s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 13:57:42 +03:00
a9ed8b16c1 skip on needed changes
All checks were successful
syncpull / pull (push) Successful in 10s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 13:40:33 +03:00
vtolstov
740cd5931d Apply Code Coverage Badge 2025-04-27 10:39:03 +00:00
85a78063d0 fix panic on shutdown caused by double channel close (#209)
All checks were successful
coverage / build (push) Successful in 2m35s
test / test (push) Successful in 3m43s
2025-04-27 13:37:18 +03:00
604ad9cd9d check sync action
All checks were successful
syncpull / pull (push) Successful in 18s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 13:36:11 +03:00
91137537a2 check sync action
All checks were successful
syncpull / pull (push) Successful in 11s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 13:23:33 +03:00
950e2352fd check sync action
Some checks failed
syncpull / pull (push) Failing after 10s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 13:17:27 +03:00
0bb29b29cf check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:38:02 +03:00
17bcd0b0ab check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:37:17 +03:00
20f9f4da3b check sync action
Some checks failed
syncpull / pull (push) Failing after 8s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:34:28 +03:00
66fa04b8dc check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:33:09 +03:00
1ef3ad6531 check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:32:22 +03:00
c95a91349d check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:30:52 +03:00
fdcf8e6ca4 check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 12:25:27 +03:00
8cb2d9db4a check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:28:50 +03:00
04da4388ac check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:27:26 +03:00
79fb23e644 check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:24:40 +03:00
f8fe923ab1 check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:21:25 +03:00
105f56dbfe check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:20:21 +03:00
9fed5a368b check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:19:58 +03:00
7374d41cf8 check sync action
Some checks failed
syncpull / pull (push) Failing after 6s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:15:32 +03:00
a4a8935c1f check sync action
Some checks failed
syncpull / pull (push) Failing after 6s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:11:39 +03:00
5f498c8232 check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:10:06 +03:00
a00fdf679b check sync action
Some checks failed
syncpull / pull (push) Failing after 6s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 11:05:22 +03:00
dc9ebe4155 check sync action
Some checks failed
syncpull / pull (push) Failing after 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:57:51 +03:00
87ced484b7 check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:56:34 +03:00
af99b11a59 check sync action
Some checks failed
syncpull / pull (push) Failing after 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:52:20 +03:00
2724b51f7c check sync action
Some checks failed
syncpull / pull (push) Failing after 4s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:41:22 +03:00
5b5d0e02b9 check sync action
Some checks failed
syncpull / pull (push) Failing after 7s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:38:36 +03:00
afc2de6819 check sync action
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:37:01 +03:00
32a8ab9c05 check sync action
Some checks failed
syncpull / pull (push) Failing after 5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 10:34:25 +03:00
vtolstov
7e5401bded Apply Code Coverage Badge 2025-04-27 06:30:32 +00:00
64b91cea06 check sync action
All checks were successful
coverage / build (push) Successful in 1m18s
test / test (push) Successful in 2m5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 09:29:33 +03:00
vtolstov
0f59fdcbde Apply Code Coverage Badge 2025-04-27 06:29:19 +00:00
50979e6708 check sync action
Some checks failed
coverage / build (push) Has started running
test / test (push) Has been cancelled
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 09:28:17 +03:00
46f3108870 check sync action
Some checks failed
coverage / build (push) Has been cancelled
test / test (push) Has started running
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 09:27:35 +03:00
vtolstov
5fed91a65f Apply Code Coverage Badge 2025-04-27 06:25:16 +00:00
1c5bba908d check sync action
All checks were successful
coverage / build (push) Successful in 1m14s
test / test (push) Successful in 2m3s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-27 09:24:41 +03:00
vtolstov
bc8ebdcad5 Apply Code Coverage Badge 2025-04-24 11:55:11 +00:00
fc24f3af92 metadata: add AsMap func
All checks were successful
coverage / build (push) Successful in 1m18s
test / test (push) Successful in 2m3s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-24 14:54:37 +03:00
1058177d1c Delete SECURITY.md 2025-04-22 15:54:19 +03:00
vtolstov
fa53fac085 Apply Code Coverage Badge 2025-04-13 21:03:53 +00:00
8c060df5e3 tracer: add IsRecording to span interface
All checks were successful
coverage / build (push) Successful in 2m0s
test / test (push) Successful in 4m29s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-04-14 00:02:45 +03:00
e1f8c62685 broker: add SetPublishOption
All checks were successful
coverage / build (push) Successful in 1m33s
test / test (push) Successful in 2m15s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-03-07 18:21:57 +03:00
562b1ab9b7 broker: simplify handler check
All checks were successful
coverage / build (push) Successful in 1m19s
test / test (push) Successful in 2m5s
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2025-03-07 15:26:20 +03:00
91 changed files with 25588 additions and 1075 deletions

View File

@@ -3,14 +3,16 @@ name: coverage
on: on:
push: push:
branches: [ main, v3, v4 ] branches: [ main, v3, v4 ]
paths-ignore:
- '.github/**'
- '.gitea/**'
pull_request: pull_request:
branches: [ main, v3, v4 ] branches: [ main, v3, v4 ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs: jobs:
build: build:
if: github.server_url != 'https://github.com'
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: checkout code - name: checkout code
@@ -22,7 +24,7 @@ jobs:
uses: actions/setup-go@v5 uses: actions/setup-go@v5
with: with:
cache-dependency-path: "**/*.sum" cache-dependency-path: "**/*.sum"
go-version: 'stable' go-version: 'stable'
- name: test coverage - name: test coverage
run: | run: |
@@ -39,8 +41,8 @@ jobs:
name: autocommit name: autocommit
with: with:
commit_message: Apply Code Coverage Badge commit_message: Apply Code Coverage Badge
skip_fetch: true skip_fetch: false
skip_checkout: true skip_checkout: false
file_pattern: ./README.md file_pattern: ./README.md
- name: push - name: push
@@ -48,4 +50,4 @@ jobs:
uses: ad-m/github-push-action@master uses: ad-m/github-push-action@master
with: with:
github_token: ${{ github.token }} github_token: ${{ github.token }}
branch: ${{ github.ref }} branch: ${{ github.ref }}

View File

@@ -3,10 +3,10 @@ name: lint
on: on:
pull_request: pull_request:
types: [opened, reopened, synchronize] types: [opened, reopened, synchronize]
branches: branches: [ master, v3, v4 ]
- master paths-ignore:
- v3 - '.github/**'
- v4 - '.gitea/**'
jobs: jobs:
lint: lint:
@@ -20,10 +20,10 @@ jobs:
uses: actions/setup-go@v5 uses: actions/setup-go@v5
with: with:
cache-dependency-path: "**/*.sum" cache-dependency-path: "**/*.sum"
go-version: 'stable' go-version: 'stable'
- name: setup deps - name: setup deps
run: go get -v ./... run: go get -v ./...
- name: run lint - name: run lint
uses: https://github.com/golangci/golangci-lint-action@v6 uses: golangci/golangci-lint-action@v6
with: with:
version: 'latest' version: 'latest'

94
.github/workflows/job_sync.yml vendored Normal file
View File

@@ -0,0 +1,94 @@
name: sync
on:
schedule:
- cron: '*/5 * * * *'
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
sync:
if: github.server_url != 'https://github.com'
runs-on: ubuntu-latest
steps:
- name: init
run: |
git config --global user.email "vtolstov <vtolstov@users.noreply.github.com>"
git config --global user.name "github-actions[bot]"
echo "machine git.unistack.org login vtolstov password ${{ secrets.TOKEN_GITEA }}" >> /root/.netrc
echo "machine github.com login vtolstov password ${{ secrets.TOKEN_GITHUB }}" >> /root/.netrc
- name: check master
id: check_master
run: |
src_hash=$(git ls-remote https://github.com/${GITHUB_REPOSITORY} refs/heads/master | cut -f1)
dst_hash=$(git ls-remote ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} refs/heads/master | cut -f1)
echo "src_hash=$src_hash"
echo "dst_hash=$dst_hash"
if [ "$src_hash" != "$dst_hash" -a "$src_hash" != "" -a "$dst_hash" != "" ]; then
echo "sync_needed=true" >> $GITHUB_OUTPUT
else
echo "sync_needed=false" >> $GITHUB_OUTPUT
fi
- name: sync master
if: steps.check_master.outputs.sync_needed == 'true'
run: |
git clone --filter=blob:none --filter=tree:0 --branch master --single-branch ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} repo
cd repo
git remote add --no-tags --fetch --track master upstream https://github.com/${GITHUB_REPOSITORY}
git pull --rebase upstream master
git push upstream master --progress
git push origin master --progress
cd ../
rm -rf repo
- name: check v3
id: check_v3
run: |
src_hash=$(git ls-remote https://github.com/${GITHUB_REPOSITORY} refs/heads/v3 | cut -f1)
dst_hash=$(git ls-remote ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} refs/heads/v3 | cut -f1)
echo "src_hash=$src_hash"
echo "dst_hash=$dst_hash"
if [ "$src_hash" != "$dst_hash" ]; then
echo "sync_needed=true" >> $GITHUB_OUTPUT
else
echo "sync_needed=false" >> $GITHUB_OUTPUT
fi
- name: sync v3
if: steps.check_v3.outputs.sync_needed == 'true'
run: |
git clone --filter=blob:none --filter=tree:0 --branch v3 --single-branch ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} repo
cd repo
git remote add --no-tags --fetch --track v3 upstream https://github.com/${GITHUB_REPOSITORY}
git pull --rebase upstream v3
git push upstream v3 --progress
git push origin v3 --progress
cd ../
rm -rf repo
- name: check v4
id: check_v4
run: |
src_hash=$(git ls-remote https://github.com/${GITHUB_REPOSITORY} refs/heads/v4 | cut -f1)
dst_hash=$(git ls-remote ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} refs/heads/v4 | cut -f1)
echo "src_hash=$src_hash"
echo "dst_hash=$dst_hash"
if [ "$src_hash" != "$dst_hash" ]; then
echo "sync_needed=true" >> $GITHUB_OUTPUT
else
echo "sync_needed=false" >> $GITHUB_OUTPUT
fi
- name: sync v4
if: steps.check_v4.outputs.sync_needed == 'true'
run: |
git clone --filter=blob:none --filter=tree:0 --branch v4 --single-branch ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY} repo
cd repo
git remote add --no-tags --fetch --track v4 upstream https://github.com/${GITHUB_REPOSITORY}
git pull --rebase upstream v4
git push upstream v4 --progress
git push origin v4 --progress
cd ../
rm -rf repo

View File

@@ -3,15 +3,12 @@ name: test
on: on:
pull_request: pull_request:
types: [opened, reopened, synchronize] types: [opened, reopened, synchronize]
branches: branches: [ master, v3, v4 ]
- master
- v3
- v4
push: push:
branches: branches: [ master, v3, v4 ]
- master paths-ignore:
- v3 - '.github/**'
- v4 - '.gitea/**'
jobs: jobs:
test: test:

View File

@@ -3,15 +3,12 @@ name: test
on: on:
pull_request: pull_request:
types: [opened, reopened, synchronize] types: [opened, reopened, synchronize]
branches: branches: [ master, v3, v4 ]
- master
- v3
- v4
push: push:
branches: branches: [ master, v3, v4 ]
- master paths-ignore:
- v3 - '.github/**'
- v4 - '.gitea/**'
jobs: jobs:
test: test:
@@ -35,19 +32,19 @@ jobs:
go-version: 'stable' go-version: 'stable'
- name: setup go work - name: setup go work
env: env:
GOWORK: /workspace/${{ github.repository_owner }}/go.work GOWORK: ${{ github.workspace }}/go.work
run: | run: |
go work init go work init
go work use . go work use .
go work use micro-tests go work use micro-tests
- name: setup deps - name: setup deps
env: env:
GOWORK: /workspace/${{ github.repository_owner }}/go.work GOWORK: ${{ github.workspace }}/go.work
run: go get -v ./... run: go get -v ./...
- name: run tests - name: run tests
env: env:
INTEGRATION_TESTS: yes INTEGRATION_TESTS: yes
GOWORK: /workspace/${{ github.repository_owner }}/go.work GOWORK: ${{ github.workspace }}/go.work
run: | run: |
cd micro-tests cd micro-tests
go test -mod readonly -v ./... || true go test -mod readonly -v ./... || true

View File

@@ -1,5 +1,5 @@
run: run:
concurrency: 8 concurrency: 8
deadline: 5m timeout: 5m
issues-exit-code: 1 issues-exit-code: 1
tests: true tests: true

View File

@@ -1,5 +1,5 @@
# Micro # Micro
![Coverage](https://img.shields.io/badge/Coverage-44.2%25-yellow) ![Coverage](https://img.shields.io/badge/Coverage-33.6%25-yellow)
[![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Doc](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go.unistack.org/micro/v4?tab=overview) [![Doc](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/go.unistack.org/micro/v4?tab=overview)
[![Status](https://git.unistack.org/unistack-org/micro/actions/workflows/job_tests.yml/badge.svg?branch=v4)](https://git.unistack.org/unistack-org/micro/actions?query=workflow%3Abuild+branch%3Av4+event%3Apush) [![Status](https://git.unistack.org/unistack-org/micro/actions/workflows/job_tests.yml/badge.svg?branch=v4)](https://git.unistack.org/unistack-org/micro/actions?query=workflow%3Abuild+branch%3Av4+event%3Apush)

View File

@@ -1,15 +0,0 @@
# Security Policy
## Supported Versions
Use this section to tell people about which versions of your project are
currently being supported with security updates.
| Version | Supported |
| ------- | ------------------ |
| 3.7.x | :white_check_mark: |
| < 3.7.0 | :x: |
## Reporting a Vulnerability
If you find any issue, please create github issue in this repo

View File

@@ -21,7 +21,7 @@ var (
// ErrInvalidMessage returns when invalid Message passed // ErrInvalidMessage returns when invalid Message passed
ErrInvalidMessage = errors.New("invalid message") ErrInvalidMessage = errors.New("invalid message")
// ErrInvalidHandler returns when subscriber passed to Subscribe // ErrInvalidHandler returns when subscriber passed to Subscribe
ErrInvalidHandler = errors.New("invalid handler") ErrInvalidHandler = errors.New("invalid handler, ony func(Message) error and func([]Message) error supported")
// DefaultGracefulTimeout // DefaultGracefulTimeout
DefaultGracefulTimeout = 5 * time.Second DefaultGracefulTimeout = 5 * time.Second
) )
@@ -41,11 +41,11 @@ type Broker interface {
// Disconnect disconnect from broker // Disconnect disconnect from broker
Disconnect(ctx context.Context) error Disconnect(ctx context.Context) error
// NewMessage create new broker message to publish. // NewMessage create new broker message to publish.
NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...PublishOption) (Message, error) NewMessage(ctx context.Context, hdr metadata.Metadata, body any, opts ...MessageOption) (Message, error)
// Publish message to broker topic // Publish message to broker topic
Publish(ctx context.Context, topic string, messages ...Message) error Publish(ctx context.Context, topic string, messages ...Message) error
// Subscribe subscribes to topic message via handler // Subscribe subscribes to topic message via handler
Subscribe(ctx context.Context, topic string, handler interface{}, opts ...SubscribeOption) (Subscriber, error) Subscribe(ctx context.Context, topic string, handler any, opts ...SubscribeOption) (Subscriber, error)
// String type of broker // String type of broker
String() string String() string
// Live returns broker liveness // Live returns broker liveness
@@ -59,7 +59,7 @@ type Broker interface {
type ( type (
FuncPublish func(ctx context.Context, topic string, messages ...Message) error FuncPublish func(ctx context.Context, topic string, messages ...Message) error
HookPublish func(next FuncPublish) FuncPublish HookPublish func(next FuncPublish) FuncPublish
FuncSubscribe func(ctx context.Context, topic string, handler interface{}, opts ...SubscribeOption) (Subscriber, error) FuncSubscribe func(ctx context.Context, topic string, handler any, opts ...SubscribeOption) (Subscriber, error)
HookSubscribe func(next FuncSubscribe) FuncSubscribe HookSubscribe func(next FuncSubscribe) FuncSubscribe
) )
@@ -75,7 +75,7 @@ type Message interface {
Body() []byte Body() []byte
// Unmarshal try to decode message body to dst. // Unmarshal try to decode message body to dst.
// This is helper method that uses codec.Unmarshal. // This is helper method that uses codec.Unmarshal.
Unmarshal(dst interface{}, opts ...codec.Option) error Unmarshal(dst any, opts ...codec.Option) error
// Ack acknowledge message if supported. // Ack acknowledge message if supported.
Ack() error Ack() error
} }

View File

@@ -42,6 +42,16 @@ func SetSubscribeOption(k, v interface{}) SubscribeOption {
} }
} }
// SetMessageOption returns a function to setup a context with given value
func SetMessageOption(k, v interface{}) MessageOption {
return func(o *MessageOptions) {
if o.Context == nil {
o.Context = context.Background()
}
o.Context = context.WithValue(o.Context, k, v)
}
}
// SetOption returns a function to setup a context with given value // SetOption returns a function to setup a context with given value
func SetOption(k, v interface{}) Option { func SetOption(k, v interface{}) Option {
return func(o *Options) { return func(o *Options) {

View File

@@ -22,8 +22,8 @@ type Broker struct {
subscribers map[string][]*Subscriber subscribers map[string][]*Subscriber
addr string addr string
opts broker.Options opts broker.Options
sync.RWMutex mu sync.RWMutex
connected bool connected bool
} }
type memoryMessage struct { type memoryMessage struct {
@@ -32,7 +32,7 @@ type memoryMessage struct {
ctx context.Context ctx context.Context
body []byte body []byte
hdr metadata.Metadata hdr metadata.Metadata
opts broker.PublishOptions opts broker.MessageOptions
} }
func (m *memoryMessage) Ack() error { func (m *memoryMessage) Ack() error {
@@ -72,9 +72,9 @@ func (b *Broker) newCodec(ct string) (codec.Codec, error) {
if idx := strings.IndexRune(ct, ';'); idx >= 0 { if idx := strings.IndexRune(ct, ';'); idx >= 0 {
ct = ct[:idx] ct = ct[:idx]
} }
b.RLock() b.mu.RLock()
c, ok := b.opts.Codecs[ct] c, ok := b.opts.Codecs[ct]
b.RUnlock() b.mu.RUnlock()
if ok { if ok {
return c, nil return c, nil
} }
@@ -96,8 +96,8 @@ func (b *Broker) Connect(ctx context.Context) error {
default: default:
} }
b.Lock() b.mu.Lock()
defer b.Unlock() defer b.mu.Unlock()
if b.connected { if b.connected {
return nil return nil
@@ -126,8 +126,8 @@ func (b *Broker) Disconnect(ctx context.Context) error {
default: default:
} }
b.Lock() b.mu.Lock()
defer b.Unlock() defer b.mu.Unlock()
if !b.connected { if !b.connected {
return nil return nil
@@ -157,8 +157,11 @@ func (b *Broker) Init(opts ...broker.Option) error {
return nil return nil
} }
func (b *Broker) NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...broker.PublishOption) (broker.Message, error) { func (b *Broker) NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...broker.MessageOption) (broker.Message, error) {
options := broker.NewPublishOptions(opts...) options := broker.NewMessageOptions(opts...)
if options.ContentType == "" {
options.ContentType = b.opts.ContentType
}
m := &memoryMessage{ctx: ctx, hdr: hdr, opts: options} m := &memoryMessage{ctx: ctx, hdr: hdr, opts: options}
c, err := b.newCodec(m.opts.ContentType) c, err := b.newCodec(m.opts.ContentType)
if err == nil { if err == nil {
@@ -180,12 +183,12 @@ func (b *Broker) fnPublish(ctx context.Context, topic string, messages ...broker
} }
func (b *Broker) publish(ctx context.Context, topic string, messages ...broker.Message) error { func (b *Broker) publish(ctx context.Context, topic string, messages ...broker.Message) error {
b.RLock() b.mu.RLock()
if !b.connected { if !b.connected {
b.RUnlock() b.mu.RUnlock()
return broker.ErrNotConnected return broker.ErrNotConnected
} }
b.RUnlock() b.mu.RUnlock()
select { select {
case <-ctx.Done(): case <-ctx.Done():
@@ -193,9 +196,9 @@ func (b *Broker) publish(ctx context.Context, topic string, messages ...broker.M
default: default:
} }
b.RLock() b.mu.RLock()
subs, ok := b.subscribers[topic] subs, ok := b.subscribers[topic]
b.RUnlock() b.mu.RUnlock()
if !ok { if !ok {
return nil return nil
} }
@@ -252,12 +255,12 @@ func (b *Broker) fnSubscribe(ctx context.Context, topic string, handler interfac
return nil, err return nil, err
} }
b.RLock() b.mu.RLock()
if !b.connected { if !b.connected {
b.RUnlock() b.mu.RUnlock()
return nil, broker.ErrNotConnected return nil, broker.ErrNotConnected
} }
b.RUnlock() b.mu.RUnlock()
sid, err := id.New() sid, err := id.New()
if err != nil { if err != nil {
@@ -275,13 +278,13 @@ func (b *Broker) fnSubscribe(ctx context.Context, topic string, handler interfac
ctx: ctx, ctx: ctx,
} }
b.Lock() b.mu.Lock()
b.subscribers[topic] = append(b.subscribers[topic], sub) b.subscribers[topic] = append(b.subscribers[topic], sub)
b.Unlock() b.mu.Unlock()
go func() { go func() {
<-sub.exit <-sub.exit
b.Lock() b.mu.Lock()
newSubscribers := make([]*Subscriber, 0, len(b.subscribers)-1) newSubscribers := make([]*Subscriber, 0, len(b.subscribers)-1)
for _, sb := range b.subscribers[topic] { for _, sb := range b.subscribers[topic] {
if sb.id == sub.id { if sb.id == sub.id {
@@ -290,7 +293,7 @@ func (b *Broker) fnSubscribe(ctx context.Context, topic string, handler interfac
newSubscribers = append(newSubscribers, sb) newSubscribers = append(newSubscribers, sb)
} }
b.subscribers[topic] = newSubscribers b.subscribers[topic] = newSubscribers
b.Unlock() b.mu.Unlock()
}() }()
return sub, nil return sub, nil

View File

@@ -49,7 +49,7 @@ func TestMemoryBroker(t *testing.T) {
"id", fmt.Sprintf("%d", i), "id", fmt.Sprintf("%d", i),
), ),
[]byte(`"hello world"`), []byte(`"hello world"`),
broker.PublishContentType("application/octet-stream"), broker.MessageContentType("application/octet-stream"),
) )
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)

View File

@@ -14,16 +14,16 @@ type NoopBroker struct {
funcPublish FuncPublish funcPublish FuncPublish
funcSubscribe FuncSubscribe funcSubscribe FuncSubscribe
opts Options opts Options
sync.RWMutex mu sync.RWMutex
} }
func (b *NoopBroker) newCodec(ct string) (codec.Codec, error) { func (b *NoopBroker) newCodec(ct string) (codec.Codec, error) {
if idx := strings.IndexRune(ct, ';'); idx >= 0 { if idx := strings.IndexRune(ct, ';'); idx >= 0 {
ct = ct[:idx] ct = ct[:idx]
} }
b.RLock() b.mu.RLock()
c, ok := b.opts.Codecs[ct] c, ok := b.opts.Codecs[ct]
b.RUnlock() b.mu.RUnlock()
if ok { if ok {
return c, nil return c, nil
} }
@@ -99,7 +99,7 @@ type noopMessage struct {
ctx context.Context ctx context.Context
body []byte body []byte
hdr metadata.Metadata hdr metadata.Metadata
opts PublishOptions opts MessageOptions
} }
func (m *noopMessage) Ack() error { func (m *noopMessage) Ack() error {
@@ -126,8 +126,11 @@ func (m *noopMessage) Unmarshal(dst interface{}, opts ...codec.Option) error {
return m.c.Unmarshal(m.body, dst) return m.c.Unmarshal(m.body, dst)
} }
func (b *NoopBroker) NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...PublishOption) (Message, error) { func (b *NoopBroker) NewMessage(ctx context.Context, hdr metadata.Metadata, body interface{}, opts ...MessageOption) (Message, error) {
options := NewPublishOptions(opts...) options := NewMessageOptions(opts...)
if options.ContentType == "" {
options.ContentType = b.opts.ContentType
}
m := &noopMessage{ctx: ctx, hdr: hdr, opts: options} m := &noopMessage{ctx: ctx, hdr: hdr, opts: options}
c, err := b.newCodec(m.opts.ContentType) c, err := b.newCodec(m.opts.ContentType)
if err == nil { if err == nil {

View File

@@ -18,7 +18,6 @@ import (
type Options struct { type Options struct {
// Name holds the broker name // Name holds the broker name
Name string Name string
// Tracer used for tracing // Tracer used for tracing
Tracer tracer.Tracer Tracer tracer.Tracer
// Register can be used for clustering // Register can be used for clustering
@@ -31,20 +30,20 @@ type Options struct {
Meter meter.Meter Meter meter.Meter
// Context holds external options // Context holds external options
Context context.Context Context context.Context
// Wait waits for a collection of goroutines to finish // Wait waits for a collection of goroutines to finish
Wait *sync.WaitGroup Wait *sync.WaitGroup
// TLSConfig holds tls.TLSConfig options // TLSConfig holds tls.TLSConfig options
TLSConfig *tls.Config TLSConfig *tls.Config
// Addrs holds the broker address // Addrs holds the broker address
Addrs []string Addrs []string
// Hooks can be run before broker Publish/BatchPublish and // Hooks can be run before broker Publishing and message processing in Subscribe
// Subscribe/BatchSubscribe methods
Hooks options.Hooks Hooks options.Hooks
// GracefulTimeout contains time to wait to finish in flight requests // GracefulTimeout contains time to wait to finish in flight requests
GracefulTimeout time.Duration GracefulTimeout time.Duration
// ContentType will be used if no content-type set when creating message
ContentType string
// ErrorHandler specifies handler for all broker errors handling subscriber
ErrorHandler any
} }
// NewOptions create new Options // NewOptions create new Options
@@ -57,14 +56,19 @@ func NewOptions(opts ...Option) Options {
Codecs: make(map[string]codec.Codec), Codecs: make(map[string]codec.Codec),
Tracer: tracer.DefaultTracer, Tracer: tracer.DefaultTracer,
GracefulTimeout: DefaultGracefulTimeout, GracefulTimeout: DefaultGracefulTimeout,
ContentType: DefaultContentType,
} }
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
} }
return options return options
} }
// DefaultContentType is the default content-type if not specified
var DefaultContentType = ""
// Context sets the context option // Context sets the context option
func Context(ctx context.Context) Option { func Context(ctx context.Context) Option {
return func(o *Options) { return func(o *Options) {
@@ -72,18 +76,42 @@ func Context(ctx context.Context) Option {
} }
} }
// PublishOptions struct func GracefulTimeout(t time.Duration) Option {
type PublishOptions struct { return func(o *Options) {
o.GracefulTimeout = t
}
}
// ContentType used by default if not specified
func ContentType(ct string) Option {
return func(o *Options) {
o.ContentType = ct
}
}
// ErrorHandler handles errors in broker
func ErrorHandler(h any) Option {
return func(o *Options) {
o.ErrorHandler = h
}
}
// MessageOptions struct
type MessageOptions struct {
// ContentType for message body // ContentType for message body
ContentType string ContentType string
// BodyOnly flag says the message contains raw body bytes and don't need // BodyOnly flag says the message contains raw body bytes and don't need
// codec Marshal method // codec Marshal method
BodyOnly bool BodyOnly bool
// Context holds custom options
Context context.Context
} }
// NewPublishOptions creates PublishOptions struct // NewMessageOptions creates MessageOptions struct
func NewPublishOptions(opts ...PublishOption) PublishOptions { func NewMessageOptions(opts ...MessageOption) MessageOptions {
options := PublishOptions{} options := MessageOptions{
Context: context.Background(),
}
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
} }
@@ -109,19 +137,19 @@ type SubscribeOptions struct {
// Option func // Option func
type Option func(*Options) type Option func(*Options)
// PublishOption func // MessageOption func
type PublishOption func(*PublishOptions) type MessageOption func(*MessageOptions)
// PublishContentType sets message content-type that used to Marshal // MessageContentType sets message content-type that used to Marshal
func PublishContentType(ct string) PublishOption { func MessageContentType(ct string) MessageOption {
return func(o *PublishOptions) { return func(o *MessageOptions) {
o.ContentType = ct o.ContentType = ct
} }
} }
// PublishBodyOnly publish only body of the message // MessageBodyOnly publish only body of the message
func PublishBodyOnly(b bool) PublishOption { func MessageBodyOnly(b bool) MessageOption {
return func(o *PublishOptions) { return func(o *MessageOptions) {
o.BodyOnly = b o.BodyOnly = b
} }
} }

View File

@@ -1,87 +1,14 @@
package broker package broker
import (
"fmt"
"reflect"
"unicode"
"unicode/utf8"
)
const (
messageSig = "func(broker.Message) error"
messagesSig = "func([]broker.Message) error"
)
// Precompute the reflect type for error. Can't use error directly
// because Typeof takes an empty interface value. This is annoying.
var typeOfError = reflect.TypeOf((*error)(nil)).Elem()
// Is this an exported - upper case - name?
func isExported(name string) bool {
r, _ := utf8.DecodeRuneInString(name)
return unicode.IsUpper(r)
}
// Is this type exported or a builtin?
func isExportedOrBuiltinType(t reflect.Type) bool {
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
// PkgPath will be non-empty even for an exported type,
// so we need to check the type name as well.
return isExported(t.Name()) || t.PkgPath() == ""
}
// IsValidHandler func signature // IsValidHandler func signature
func IsValidHandler(sub interface{}) error { func IsValidHandler(sub interface{}) error {
typ := reflect.TypeOf(sub) switch sub.(type) {
var argType reflect.Type
switch typ.Kind() {
case reflect.Func:
name := "Func"
switch typ.NumIn() {
case 1:
argType = typ.In(0)
default:
return fmt.Errorf("subscriber %v takes wrong number of args: %v required signature %s", name, typ.NumIn(), messageSig)
}
if !isExportedOrBuiltinType(argType) {
return fmt.Errorf("subscriber %v argument type not exported: %v", name, argType)
}
if typ.NumOut() != 1 {
return fmt.Errorf("subscriber %v has wrong number of return values: %v require signature %s",
name, typ.NumOut(), messageSig)
}
if returnType := typ.Out(0); returnType != typeOfError {
return fmt.Errorf("subscriber %v returns %v not error", name, returnType.String())
}
default: default:
hdlr := reflect.ValueOf(sub) return ErrInvalidHandler
name := reflect.Indirect(hdlr).Type().Name() case func(Message) error:
break
for m := 0; m < typ.NumMethod(); m++ { case func([]Message) error:
method := typ.Method(m) break
switch method.Type.NumIn() {
case 3:
argType = method.Type.In(2)
default:
return fmt.Errorf("subscriber %v.%v takes wrong number of args: %v required signature %s",
name, method.Name, method.Type.NumIn(), messageSig)
}
if !isExportedOrBuiltinType(argType) {
return fmt.Errorf("%v argument type not exported: %v", name, argType)
}
if method.Type.NumOut() != 1 {
return fmt.Errorf(
"subscriber %v.%v has wrong number of return values: %v require signature %s",
name, method.Name, method.Type.NumOut(), messageSig)
}
if returnType := method.Type.Out(0); returnType != typeOfError {
return fmt.Errorf("subscriber %v.%v returns %v not error", name, method.Name, returnType.String())
}
}
} }
return nil return nil
} }

View File

@@ -15,11 +15,6 @@ import (
"go.unistack.org/micro/v4/tracer" "go.unistack.org/micro/v4/tracer"
) )
// DefaultCodecs will be used to encode/decode data
var DefaultCodecs = map[string]codec.Codec{
"application/octet-stream": codec.NewCodec(),
}
type noopClient struct { type noopClient struct {
funcCall FuncCall funcCall FuncCall
funcStream FuncStream funcStream FuncStream

View File

@@ -161,7 +161,7 @@ func NewOptions(opts ...Option) Options {
options := Options{ options := Options{
Context: context.Background(), Context: context.Background(),
ContentType: DefaultContentType, ContentType: DefaultContentType,
Codecs: DefaultCodecs, Codecs: make(map[string]codec.Codec),
CallOptions: CallOptions{ CallOptions: CallOptions{
Context: context.Background(), Context: context.Background(),
Backoff: DefaultBackoff, Backoff: DefaultBackoff,

235
cluster/hasql/cluster.go Normal file
View File

@@ -0,0 +1,235 @@
package sql
import (
"context"
"database/sql"
"reflect"
"unsafe"
"golang.yandex/hasql/v2"
)
func newSQLRowError() *sql.Row {
row := &sql.Row{}
t := reflect.TypeOf(row).Elem()
field, _ := t.FieldByName("err")
rowPtr := unsafe.Pointer(row)
errFieldPtr := unsafe.Pointer(uintptr(rowPtr) + field.Offset)
errPtr := (*error)(errFieldPtr)
*errPtr = ErrorNoAliveNodes
return row
}
type ClusterQuerier interface {
Querier
WaitForNodes(ctx context.Context, criterion ...hasql.NodeStateCriterion) error
}
type Cluster struct {
hasql *hasql.Cluster[Querier]
options ClusterOptions
}
// NewCluster returns [Querier] that provides cluster of nodes
func NewCluster[T Querier](opts ...ClusterOption) (ClusterQuerier, error) {
options := ClusterOptions{Context: context.Background()}
for _, opt := range opts {
opt(&options)
}
if options.NodeChecker == nil {
return nil, ErrClusterChecker
}
if options.NodeDiscoverer == nil {
return nil, ErrClusterDiscoverer
}
if options.NodePicker == nil {
return nil, ErrClusterPicker
}
if options.Retries < 1 {
options.Retries = 1
}
if options.NodeStateCriterion == 0 {
options.NodeStateCriterion = hasql.Primary
}
options.Options = append(options.Options, hasql.WithNodePicker(options.NodePicker))
if p, ok := options.NodePicker.(*CustomPicker[Querier]); ok {
p.opts.Priority = options.NodePriority
}
c, err := hasql.NewCluster(
options.NodeDiscoverer,
options.NodeChecker,
options.Options...,
)
if err != nil {
return nil, err
}
return &Cluster{hasql: c, options: options}, nil
}
func (c *Cluster) BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error) {
var tx *sql.Tx
var err error
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
for ; retries < c.options.Retries; retries++ {
if tx, err = n.DB().BeginTx(ctx, opts); err != nil && retries >= c.options.Retries {
return true
}
}
return false
})
if tx == nil && err == nil {
err = ErrorNoAliveNodes
}
return tx, err
}
func (c *Cluster) Close() error {
return c.hasql.Close()
}
func (c *Cluster) Conn(ctx context.Context) (*sql.Conn, error) {
var conn *sql.Conn
var err error
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
for ; retries < c.options.Retries; retries++ {
if conn, err = n.DB().Conn(ctx); err != nil && retries >= c.options.Retries {
return true
}
}
return false
})
if conn == nil && err == nil {
err = ErrorNoAliveNodes
}
return conn, err
}
func (c *Cluster) ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error) {
var res sql.Result
var err error
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
for ; retries < c.options.Retries; retries++ {
if res, err = n.DB().ExecContext(ctx, query, args...); err != nil && retries >= c.options.Retries {
return true
}
}
return false
})
if res == nil && err == nil {
err = ErrorNoAliveNodes
}
return res, err
}
func (c *Cluster) PrepareContext(ctx context.Context, query string) (*sql.Stmt, error) {
var res *sql.Stmt
var err error
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
for ; retries < c.options.Retries; retries++ {
if res, err = n.DB().PrepareContext(ctx, query); err != nil && retries >= c.options.Retries {
return true
}
}
return false
})
if res == nil && err == nil {
err = ErrorNoAliveNodes
}
return res, err
}
func (c *Cluster) QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error) {
var res *sql.Rows
var err error
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
for ; retries < c.options.Retries; retries++ {
if res, err = n.DB().QueryContext(ctx, query); err != nil && err != sql.ErrNoRows && retries >= c.options.Retries {
return true
}
}
return false
})
if res == nil && err == nil {
err = ErrorNoAliveNodes
}
return res, err
}
func (c *Cluster) QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row {
var res *sql.Row
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
for ; retries < c.options.Retries; retries++ {
res = n.DB().QueryRowContext(ctx, query, args...)
if res.Err() == nil {
return false
} else if res.Err() != nil && retries >= c.options.Retries {
return false
}
}
return true
})
if res == nil {
res = newSQLRowError()
}
return res
}
func (c *Cluster) PingContext(ctx context.Context) error {
var err error
var ok bool
retries := 0
c.hasql.NodesIter(c.getNodeStateCriterion(ctx))(func(n *hasql.Node[Querier]) bool {
ok = true
for ; retries < c.options.Retries; retries++ {
if err = n.DB().PingContext(ctx); err != nil && retries >= c.options.Retries {
return true
}
}
return false
})
if !ok {
err = ErrorNoAliveNodes
}
return err
}
func (c *Cluster) WaitForNodes(ctx context.Context, criterions ...hasql.NodeStateCriterion) error {
for _, criterion := range criterions {
if _, err := c.hasql.WaitForNode(ctx, criterion); err != nil {
return err
}
}
return nil
}

View File

@@ -0,0 +1,171 @@
package sql
import (
"context"
"fmt"
"testing"
"time"
"github.com/DATA-DOG/go-sqlmock"
"golang.yandex/hasql/v2"
)
func TestNewCluster(t *testing.T) {
dbMaster, dbMasterMock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbMaster.Close()
dbMasterMock.MatchExpectationsInOrder(false)
dbMasterMock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(1, 0)).
RowsWillBeClosed().
WithoutArgs()
dbMasterMock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("master-dc1"))
dbDRMaster, dbDRMasterMock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbDRMaster.Close()
dbDRMasterMock.MatchExpectationsInOrder(false)
dbDRMasterMock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(2, 40)).
RowsWillBeClosed().
WithoutArgs()
dbDRMasterMock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("drmaster1-dc2"))
dbDRMasterMock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("drmaster"))
dbSlaveDC1, dbSlaveDC1Mock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbSlaveDC1.Close()
dbSlaveDC1Mock.MatchExpectationsInOrder(false)
dbSlaveDC1Mock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(2, 50)).
RowsWillBeClosed().
WithoutArgs()
dbSlaveDC1Mock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("slave-dc1"))
dbSlaveDC2, dbSlaveDC2Mock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbSlaveDC2.Close()
dbSlaveDC1Mock.MatchExpectationsInOrder(false)
dbSlaveDC2Mock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(2, 50)).
RowsWillBeClosed().
WithoutArgs()
dbSlaveDC2Mock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("slave-dc1"))
tctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
defer cancel()
c, err := NewCluster[Querier](
WithClusterContext(tctx),
WithClusterNodeChecker(hasql.PostgreSQLChecker),
WithClusterNodePicker(NewCustomPicker[Querier](
CustomPickerMaxLag(100),
)),
WithClusterNodes(
ClusterNode{"slave-dc1", dbSlaveDC1, 1},
ClusterNode{"master-dc1", dbMaster, 1},
ClusterNode{"slave-dc2", dbSlaveDC2, 2},
ClusterNode{"drmaster1-dc2", dbDRMaster, 0},
),
WithClusterOptions(
hasql.WithUpdateInterval[Querier](2*time.Second),
hasql.WithUpdateTimeout[Querier](1*time.Second),
),
)
if err != nil {
t.Fatal(err)
}
defer c.Close()
if err = c.WaitForNodes(tctx, hasql.Primary, hasql.Standby); err != nil {
t.Fatal(err)
}
time.Sleep(500 * time.Millisecond)
node1Name := ""
fmt.Printf("check for Standby\n")
if row := c.QueryRowContext(NodeStateCriterion(tctx, hasql.Standby), "SELECT node_name as name"); row.Err() != nil {
t.Fatal(row.Err())
} else if err = row.Scan(&node1Name); err != nil {
t.Fatal(err)
} else if "slave-dc1" != node1Name {
t.Fatalf("invalid node name %s != %s", "slave-dc1", node1Name)
}
dbSlaveDC1Mock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("slave-dc1"))
node2Name := ""
fmt.Printf("check for PreferStandby\n")
if row := c.QueryRowContext(NodeStateCriterion(tctx, hasql.PreferStandby), "SELECT node_name as name"); row.Err() != nil {
t.Fatal(row.Err())
} else if err = row.Scan(&node2Name); err != nil {
t.Fatal(err)
} else if "slave-dc1" != node2Name {
t.Fatalf("invalid node name %s != %s", "slave-dc1", node2Name)
}
node3Name := ""
fmt.Printf("check for PreferPrimary\n")
if row := c.QueryRowContext(NodeStateCriterion(tctx, hasql.PreferPrimary), "SELECT node_name as name"); row.Err() != nil {
t.Fatal(row.Err())
} else if err = row.Scan(&node3Name); err != nil {
t.Fatal(err)
} else if "master-dc1" != node3Name {
t.Fatalf("invalid node name %s != %s", "master-dc1", node3Name)
}
dbSlaveDC1Mock.ExpectQuery(`.*`).WillReturnRows(sqlmock.NewRows([]string{"role"}).RowError(1, fmt.Errorf("row error")))
time.Sleep(2 * time.Second)
fmt.Printf("check for PreferStandby\n")
if row := c.QueryRowContext(NodeStateCriterion(tctx, hasql.PreferStandby), "SELECT node_name as name"); row.Err() == nil {
t.Fatal("must return error")
}
if dbMasterErr := dbMasterMock.ExpectationsWereMet(); dbMasterErr != nil {
t.Error(dbMasterErr)
}
}

25
cluster/hasql/db.go Normal file
View File

@@ -0,0 +1,25 @@
package sql
import (
"context"
"database/sql"
)
type Querier interface {
// Basic connection methods
PingContext(ctx context.Context) error
Close() error
// Query methods with context
ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
// Prepared statements with context
PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
// Transaction management with context
BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error)
Conn(ctx context.Context) (*sql.Conn, error)
}

295
cluster/hasql/driver.go Normal file
View File

@@ -0,0 +1,295 @@
package sql
import (
"context"
"database/sql"
"database/sql/driver"
"io"
"sync"
"time"
)
// OpenDBWithCluster creates a [*sql.DB] that uses the [ClusterQuerier]
func OpenDBWithCluster(db ClusterQuerier) (*sql.DB, error) {
driver := NewClusterDriver(db)
connector, err := driver.OpenConnector("")
if err != nil {
return nil, err
}
return sql.OpenDB(connector), nil
}
// ClusterDriver implements [driver.Driver] and driver.Connector for an existing [Querier]
type ClusterDriver struct {
db ClusterQuerier
}
// NewClusterDriver creates a new [driver.Driver] that uses an existing [ClusterQuerier]
func NewClusterDriver(db ClusterQuerier) *ClusterDriver {
return &ClusterDriver{db: db}
}
// Open implements [driver.Driver.Open]
func (d *ClusterDriver) Open(name string) (driver.Conn, error) {
return d.Connect(context.Background())
}
// OpenConnector implements [driver.DriverContext.OpenConnector]
func (d *ClusterDriver) OpenConnector(name string) (driver.Connector, error) {
return d, nil
}
// Connect implements [driver.Connector.Connect]
func (d *ClusterDriver) Connect(ctx context.Context) (driver.Conn, error) {
conn, err := d.db.Conn(ctx)
if err != nil {
return nil, err
}
return &dbConn{conn: conn}, nil
}
// Driver implements [driver.Connector.Driver]
func (d *ClusterDriver) Driver() driver.Driver {
return d
}
// dbConn implements driver.Conn with both context and legacy methods
type dbConn struct {
conn *sql.Conn
mu sync.Mutex
}
// Prepare implements [driver.Conn.Prepare] (legacy method)
func (c *dbConn) Prepare(query string) (driver.Stmt, error) {
return c.PrepareContext(context.Background(), query)
}
// PrepareContext implements [driver.ConnPrepareContext.PrepareContext]
func (c *dbConn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
c.mu.Lock()
defer c.mu.Unlock()
stmt, err := c.conn.PrepareContext(ctx, query)
if err != nil {
return nil, err
}
return &dbStmt{stmt: stmt}, nil
}
// Exec implements [driver.Execer.Exec] (legacy method)
func (c *dbConn) Exec(query string, args []driver.Value) (driver.Result, error) {
namedArgs := make([]driver.NamedValue, len(args))
for i, value := range args {
namedArgs[i] = driver.NamedValue{Value: value}
}
return c.ExecContext(context.Background(), query, namedArgs)
}
// ExecContext implements [driver.ExecerContext.ExecContext]
func (c *dbConn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {
c.mu.Lock()
defer c.mu.Unlock()
// Convert driver.NamedValue to any
interfaceArgs := make([]any, len(args))
for i, arg := range args {
interfaceArgs[i] = arg.Value
}
return c.conn.ExecContext(ctx, query, interfaceArgs...)
}
// Query implements [driver.Queryer.Query] (legacy method)
func (c *dbConn) Query(query string, args []driver.Value) (driver.Rows, error) {
namedArgs := make([]driver.NamedValue, len(args))
for i, value := range args {
namedArgs[i] = driver.NamedValue{Value: value}
}
return c.QueryContext(context.Background(), query, namedArgs)
}
// QueryContext implements [driver.QueryerContext.QueryContext]
func (c *dbConn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {
c.mu.Lock()
defer c.mu.Unlock()
// Convert driver.NamedValue to any
interfaceArgs := make([]any, len(args))
for i, arg := range args {
interfaceArgs[i] = arg.Value
}
rows, err := c.conn.QueryContext(ctx, query, interfaceArgs...)
if err != nil {
return nil, err
}
return &dbRows{rows: rows}, nil
}
// Begin implements [driver.Conn.Begin] (legacy method)
func (c *dbConn) Begin() (driver.Tx, error) {
return c.BeginTx(context.Background(), driver.TxOptions{})
}
// BeginTx implements [driver.ConnBeginTx.BeginTx]
func (c *dbConn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
c.mu.Lock()
defer c.mu.Unlock()
sqlOpts := &sql.TxOptions{
Isolation: sql.IsolationLevel(opts.Isolation),
ReadOnly: opts.ReadOnly,
}
tx, err := c.conn.BeginTx(ctx, sqlOpts)
if err != nil {
return nil, err
}
return &dbTx{tx: tx}, nil
}
// Ping implements [driver.Pinger.Ping]
func (c *dbConn) Ping(ctx context.Context) error {
return c.conn.PingContext(ctx)
}
// Close implements [driver.Conn.Close]
func (c *dbConn) Close() error {
return c.conn.Close()
}
// IsValid implements [driver.Validator.IsValid]
func (c *dbConn) IsValid() bool {
// Ping with a short timeout to check if the connection is still valid
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
return c.conn.PingContext(ctx) == nil
}
// dbStmt implements [driver.Stmt] with both context and legacy methods
type dbStmt struct {
stmt *sql.Stmt
mu sync.Mutex
}
// Close implements [driver.Stmt.Close]
func (s *dbStmt) Close() error {
s.mu.Lock()
defer s.mu.Unlock()
return s.stmt.Close()
}
// Close implements [driver.Stmt.NumInput]
func (s *dbStmt) NumInput() int {
return -1 // Number of parameters is unknown
}
// Exec implements [driver.Stmt.Exec] (legacy method)
func (s *dbStmt) Exec(args []driver.Value) (driver.Result, error) {
namedArgs := make([]driver.NamedValue, len(args))
for i, value := range args {
namedArgs[i] = driver.NamedValue{Value: value}
}
return s.ExecContext(context.Background(), namedArgs)
}
// ExecContext implements [driver.StmtExecContext.ExecContext]
func (s *dbStmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
s.mu.Lock()
defer s.mu.Unlock()
interfaceArgs := make([]any, len(args))
for i, arg := range args {
interfaceArgs[i] = arg.Value
}
return s.stmt.ExecContext(ctx, interfaceArgs...)
}
// Query implements [driver.Stmt.Query] (legacy method)
func (s *dbStmt) Query(args []driver.Value) (driver.Rows, error) {
namedArgs := make([]driver.NamedValue, len(args))
for i, value := range args {
namedArgs[i] = driver.NamedValue{Value: value}
}
return s.QueryContext(context.Background(), namedArgs)
}
// QueryContext implements [driver.StmtQueryContext.QueryContext]
func (s *dbStmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
s.mu.Lock()
defer s.mu.Unlock()
interfaceArgs := make([]any, len(args))
for i, arg := range args {
interfaceArgs[i] = arg.Value
}
rows, err := s.stmt.QueryContext(ctx, interfaceArgs...)
if err != nil {
return nil, err
}
return &dbRows{rows: rows}, nil
}
// dbRows implements [driver.Rows]
type dbRows struct {
rows *sql.Rows
}
// Columns implements [driver.Rows.Columns]
func (r *dbRows) Columns() []string {
cols, err := r.rows.Columns()
if err != nil {
// This shouldn't happen if the query was successful
return []string{}
}
return cols
}
// Close implements [driver.Rows.Close]
func (r *dbRows) Close() error {
return r.rows.Close()
}
// Next implements [driver.Rows.Next]
func (r *dbRows) Next(dest []driver.Value) error {
if !r.rows.Next() {
if err := r.rows.Err(); err != nil {
return err
}
return io.EOF
}
// Create a slice of interfaces to scan into
scanArgs := make([]any, len(dest))
for i := range scanArgs {
scanArgs[i] = &dest[i]
}
return r.rows.Scan(scanArgs...)
}
// dbTx implements [driver.Tx]
type dbTx struct {
tx *sql.Tx
mu sync.Mutex
}
// Commit implements [driver.Tx.Commit]
func (t *dbTx) Commit() error {
t.mu.Lock()
defer t.mu.Unlock()
return t.tx.Commit()
}
// Rollback implements [driver.Tx.Rollback]
func (t *dbTx) Rollback() error {
t.mu.Lock()
defer t.mu.Unlock()
return t.tx.Rollback()
}

View File

@@ -0,0 +1,141 @@
package sql
import (
"context"
"testing"
"time"
"github.com/DATA-DOG/go-sqlmock"
"golang.yandex/hasql/v2"
)
func TestDriver(t *testing.T) {
dbMaster, dbMasterMock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbMaster.Close()
dbMasterMock.MatchExpectationsInOrder(false)
dbMasterMock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(1, 0)).
RowsWillBeClosed().
WithoutArgs()
dbMasterMock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("master-dc1"))
dbDRMaster, dbDRMasterMock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbDRMaster.Close()
dbDRMasterMock.MatchExpectationsInOrder(false)
dbDRMasterMock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(2, 40)).
RowsWillBeClosed().
WithoutArgs()
dbDRMasterMock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("drmaster1-dc2"))
dbDRMasterMock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("drmaster"))
dbSlaveDC1, dbSlaveDC1Mock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbSlaveDC1.Close()
dbSlaveDC1Mock.MatchExpectationsInOrder(false)
dbSlaveDC1Mock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(2, 50)).
RowsWillBeClosed().
WithoutArgs()
dbSlaveDC1Mock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("slave-dc1"))
dbSlaveDC2, dbSlaveDC2Mock, err := sqlmock.New(sqlmock.MonitorPingsOption(true))
if err != nil {
t.Fatal(err)
}
defer dbSlaveDC2.Close()
dbSlaveDC1Mock.MatchExpectationsInOrder(false)
dbSlaveDC2Mock.ExpectQuery(`.*pg_is_in_recovery.*`).WillReturnRows(
sqlmock.NewRowsWithColumnDefinition(
sqlmock.NewColumn("role").OfType("int8", 0),
sqlmock.NewColumn("replication_lag").OfType("int8", 0)).
AddRow(2, 50)).
RowsWillBeClosed().
WithoutArgs()
dbSlaveDC2Mock.ExpectQuery(`SELECT node_name as name`).WillReturnRows(
sqlmock.NewRows([]string{"name"}).
AddRow("slave-dc1"))
tctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
defer cancel()
c, err := NewCluster[Querier](
WithClusterContext(tctx),
WithClusterNodeChecker(hasql.PostgreSQLChecker),
WithClusterNodePicker(NewCustomPicker[Querier](
CustomPickerMaxLag(100),
)),
WithClusterNodes(
ClusterNode{"slave-dc1", dbSlaveDC1, 1},
ClusterNode{"master-dc1", dbMaster, 1},
ClusterNode{"slave-dc2", dbSlaveDC2, 2},
ClusterNode{"drmaster1-dc2", dbDRMaster, 0},
),
WithClusterOptions(
hasql.WithUpdateInterval[Querier](2*time.Second),
hasql.WithUpdateTimeout[Querier](1*time.Second),
),
)
if err != nil {
t.Fatal(err)
}
defer c.Close()
if err = c.WaitForNodes(tctx, hasql.Primary, hasql.Standby); err != nil {
t.Fatal(err)
}
db, err := OpenDBWithCluster(c)
if err != nil {
t.Fatal(err)
}
// Use context methods
row := db.QueryRowContext(NodeStateCriterion(t.Context(), hasql.Primary), "SELECT node_name as name")
if err = row.Err(); err != nil {
t.Fatal(err)
}
nodeName := ""
if err = row.Scan(&nodeName); err != nil {
t.Fatal(err)
}
if nodeName != "master-dc1" {
t.Fatalf("invalid node_name %s != %s", "master-dc1", nodeName)
}
}

10
cluster/hasql/error.go Normal file
View File

@@ -0,0 +1,10 @@
package sql
import "errors"
var (
ErrClusterChecker = errors.New("cluster node checker required")
ErrClusterDiscoverer = errors.New("cluster node discoverer required")
ErrClusterPicker = errors.New("cluster node picker required")
ErrorNoAliveNodes = errors.New("cluster no alive nodes")
)

110
cluster/hasql/options.go Normal file
View File

@@ -0,0 +1,110 @@
package sql
import (
"context"
"math"
"golang.yandex/hasql/v2"
)
// ClusterOptions contains cluster specific options
type ClusterOptions struct {
NodeChecker hasql.NodeChecker
NodePicker hasql.NodePicker[Querier]
NodeDiscoverer hasql.NodeDiscoverer[Querier]
Options []hasql.ClusterOpt[Querier]
Context context.Context
Retries int
NodePriority map[string]int32
NodeStateCriterion hasql.NodeStateCriterion
}
// ClusterOption apply cluster options to ClusterOptions
type ClusterOption func(*ClusterOptions)
// WithClusterNodeChecker pass hasql.NodeChecker to cluster options
func WithClusterNodeChecker(c hasql.NodeChecker) ClusterOption {
return func(o *ClusterOptions) {
o.NodeChecker = c
}
}
// WithClusterNodePicker pass hasql.NodePicker to cluster options
func WithClusterNodePicker(p hasql.NodePicker[Querier]) ClusterOption {
return func(o *ClusterOptions) {
o.NodePicker = p
}
}
// WithClusterNodeDiscoverer pass hasql.NodeDiscoverer to cluster options
func WithClusterNodeDiscoverer(d hasql.NodeDiscoverer[Querier]) ClusterOption {
return func(o *ClusterOptions) {
o.NodeDiscoverer = d
}
}
// WithRetries retry count on other nodes in case of error
func WithRetries(n int) ClusterOption {
return func(o *ClusterOptions) {
o.Retries = n
}
}
// WithClusterContext pass context.Context to cluster options and used for checks
func WithClusterContext(ctx context.Context) ClusterOption {
return func(o *ClusterOptions) {
o.Context = ctx
}
}
// WithClusterOptions pass hasql.ClusterOpt
func WithClusterOptions(opts ...hasql.ClusterOpt[Querier]) ClusterOption {
return func(o *ClusterOptions) {
o.Options = append(o.Options, opts...)
}
}
// WithClusterNodeStateCriterion pass default hasql.NodeStateCriterion
func WithClusterNodeStateCriterion(c hasql.NodeStateCriterion) ClusterOption {
return func(o *ClusterOptions) {
o.NodeStateCriterion = c
}
}
type ClusterNode struct {
Name string
DB Querier
Priority int32
}
// WithClusterNodes create cluster with static NodeDiscoverer
func WithClusterNodes(cns ...ClusterNode) ClusterOption {
return func(o *ClusterOptions) {
nodes := make([]*hasql.Node[Querier], 0, len(cns))
if o.NodePriority == nil {
o.NodePriority = make(map[string]int32, len(cns))
}
for _, cn := range cns {
nodes = append(nodes, hasql.NewNode(cn.Name, cn.DB))
if cn.Priority == 0 {
cn.Priority = math.MaxInt32
}
o.NodePriority[cn.Name] = cn.Priority
}
o.NodeDiscoverer = hasql.NewStaticNodeDiscoverer(nodes...)
}
}
type nodeStateCriterionKey struct{}
// NodeStateCriterion inject hasql.NodeStateCriterion to context
func NodeStateCriterion(ctx context.Context, c hasql.NodeStateCriterion) context.Context {
return context.WithValue(ctx, nodeStateCriterionKey{}, c)
}
func (c *Cluster) getNodeStateCriterion(ctx context.Context) hasql.NodeStateCriterion {
if v, ok := ctx.Value(nodeStateCriterionKey{}).(hasql.NodeStateCriterion); ok {
return v
}
return c.options.NodeStateCriterion
}

113
cluster/hasql/picker.go Normal file
View File

@@ -0,0 +1,113 @@
package sql
import (
"fmt"
"math"
"time"
"golang.yandex/hasql/v2"
)
// compile time guard
var _ hasql.NodePicker[Querier] = (*CustomPicker[Querier])(nil)
// CustomPickerOptions holds options to pick nodes
type CustomPickerOptions struct {
MaxLag int
Priority map[string]int32
Retries int
}
// CustomPickerOption func apply option to CustomPickerOptions
type CustomPickerOption func(*CustomPickerOptions)
// CustomPickerMaxLag specifies max lag for which node can be used
func CustomPickerMaxLag(n int) CustomPickerOption {
return func(o *CustomPickerOptions) {
o.MaxLag = n
}
}
// NewCustomPicker creates new node picker
func NewCustomPicker[T Querier](opts ...CustomPickerOption) *CustomPicker[Querier] {
options := CustomPickerOptions{}
for _, o := range opts {
o(&options)
}
return &CustomPicker[Querier]{opts: options}
}
// CustomPicker holds node picker options
type CustomPicker[T Querier] struct {
opts CustomPickerOptions
}
// PickNode used to return specific node
func (p *CustomPicker[T]) PickNode(cnodes []hasql.CheckedNode[T]) hasql.CheckedNode[T] {
for _, n := range cnodes {
fmt.Printf("node %s\n", n.Node.String())
}
return cnodes[0]
}
func (p *CustomPicker[T]) getPriority(nodeName string) int32 {
if prio, ok := p.opts.Priority[nodeName]; ok {
return prio
}
return math.MaxInt32 // Default to lowest priority
}
// CompareNodes used to sort nodes
func (p *CustomPicker[T]) CompareNodes(a, b hasql.CheckedNode[T]) int {
// Get replication lag values
aLag := a.Info.(interface{ ReplicationLag() int }).ReplicationLag()
bLag := b.Info.(interface{ ReplicationLag() int }).ReplicationLag()
// First check that lag lower then MaxLag
if aLag > p.opts.MaxLag && bLag > p.opts.MaxLag {
return 0 // both are equal
}
// If one node exceeds MaxLag and the other doesn't, prefer the one that doesn't
if aLag > p.opts.MaxLag {
return 1 // b is better
}
if bLag > p.opts.MaxLag {
return -1 // a is better
}
// Get node priorities
aPrio := p.getPriority(a.Node.String())
bPrio := p.getPriority(b.Node.String())
// if both priority equals
if aPrio == bPrio {
// First compare by replication lag
if aLag < bLag {
return -1
}
if aLag > bLag {
return 1
}
// If replication lag is equal, compare by latency
aLatency := a.Info.(interface{ Latency() time.Duration }).Latency()
bLatency := b.Info.(interface{ Latency() time.Duration }).Latency()
if aLatency < bLatency {
return -1
}
if aLatency > bLatency {
return 1
}
// If lag and latency is equal
return 0
}
// If priorities are different, prefer the node with lower priority value
if aPrio < bPrio {
return -1
}
return 1
}

View File

@@ -3,8 +3,6 @@ package codec
import ( import (
"errors" "errors"
"gopkg.in/yaml.v3"
) )
var ( var (
@@ -68,10 +66,10 @@ func (m *RawMessage) MarshalYAML() ([]byte, error) {
} }
// UnmarshalYAML sets *m to a copy of data. // UnmarshalYAML sets *m to a copy of data.
func (m *RawMessage) UnmarshalYAML(n *yaml.Node) error { func (m *RawMessage) UnmarshalYAML(data []byte) error {
if m == nil { if m == nil {
return errors.New("RawMessage UnmarshalYAML on nil pointer") return errors.New("RawMessage UnmarshalYAML on nil pointer")
} }
*m = append((*m)[0:0], []byte(n.Value)...) *m = append((*m)[0:0], data...)
return nil return nil
} }

View File

@@ -1,7 +1,5 @@
package codec package codec
import "gopkg.in/yaml.v3"
// Frame gives us the ability to define raw data to send over the pipes // Frame gives us the ability to define raw data to send over the pipes
type Frame struct { type Frame struct {
Data []byte Data []byte
@@ -28,8 +26,8 @@ func (m *Frame) MarshalYAML() ([]byte, error) {
} }
// UnmarshalYAML set frame data // UnmarshalYAML set frame data
func (m *Frame) UnmarshalYAML(n *yaml.Node) error { func (m *Frame) UnmarshalYAML(data []byte) error {
m.Data = []byte(n.Value) m.Data = append((m.Data)[0:0], data...)
return nil return nil
} }

33
go.mod
View File

@@ -1,34 +1,33 @@
module go.unistack.org/micro/v4 module go.unistack.org/micro/v4
go 1.22.0 go 1.25
require ( require (
dario.cat/mergo v1.0.1 dario.cat/mergo v1.0.2
github.com/DATA-DOG/go-sqlmock v1.5.2 github.com/DATA-DOG/go-sqlmock v1.5.2
github.com/KimMachineGun/automemlimit v0.7.0 github.com/KimMachineGun/automemlimit v0.7.5
github.com/ash3in/uuidv8 v1.2.0 github.com/goccy/go-yaml v1.18.0
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/matoous/go-nanoid v1.5.1 github.com/matoous/go-nanoid v1.5.1
github.com/patrickmn/go-cache v2.1.0+incompatible github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5 github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5
github.com/spf13/cast v1.7.1 github.com/spf13/cast v1.10.0
github.com/stretchr/testify v1.11.1
go.uber.org/atomic v1.11.0 go.uber.org/atomic v1.11.0
go.uber.org/automaxprocs v1.6.0
go.unistack.org/micro-proto/v4 v4.1.0 go.unistack.org/micro-proto/v4 v4.1.0
golang.org/x/sync v0.10.0 golang.org/x/sync v0.17.0
google.golang.org/grpc v1.69.4 golang.yandex/hasql/v2 v2.1.0
google.golang.org/protobuf v1.36.3 google.golang.org/grpc v1.76.0
gopkg.in/yaml.v3 v3.0.1 google.golang.org/protobuf v1.36.10
) )
require ( require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rogpeppe/go-internal v1.13.1 // indirect github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/stretchr/testify v1.10.0 // indirect golang.org/x/sys v0.37.0 // indirect
golang.org/x/net v0.34.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda // indirect
golang.org/x/sys v0.29.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
) )

70
go.sum
View File

@@ -1,19 +1,19 @@
dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s= dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk= dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU= github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU=
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU= github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
github.com/KimMachineGun/automemlimit v0.7.0 h1:7G06p/dMSf7G8E6oq+f2uOPuVncFyIlDI/pBWK49u88= github.com/KimMachineGun/automemlimit v0.7.5 h1:RkbaC0MwhjL1ZuBKunGDjE/ggwAX43DwZrJqVwyveTk=
github.com/KimMachineGun/automemlimit v0.7.0/go.mod h1:QZxpHaGOQoYvFhv/r4u3U0JTC2ZcOwbSr11UZF46UBM= github.com/KimMachineGun/automemlimit v0.7.5/go.mod h1:QZxpHaGOQoYvFhv/r4u3U0JTC2ZcOwbSr11UZF46UBM=
github.com/ash3in/uuidv8 v1.2.0 h1:2oogGdtCPwaVtyvPPGin4TfZLtOGE5F+W++E880G6SI= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/ash3in/uuidv8 v1.2.0/go.mod h1:BnU0wJBxnzdEKmVg4xckBkD+VZuecTFTUP3M0dWgyY4= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/goccy/go-yaml v1.18.0 h1:8W7wMFS12Pcas7KU+VVkaiCng+kG8QiFeFwzFb+rwuw=
github.com/goccy/go-yaml v1.18.0/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE= github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE=
@@ -30,38 +30,36 @@ github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaR
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ= github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0= github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0=
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y= github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g= github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U= github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5 h1:G/FZtUu7a6NTWl3KUHMV9jkLAh/Rvtf03NWMHaEDl+E= github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5 h1:G/FZtUu7a6NTWl3KUHMV9jkLAh/Rvtf03NWMHaEDl+E=
github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5/go.mod h1:7RTUFBdIRC9nZ7/3RyRNH1bdqIShrDejd1YbLwgPS+I= github.com/silas/dag v0.0.0-20220518035006-a7e85ada93c5/go.mod h1:7RTUFBdIRC9nZ7/3RyRNH1bdqIShrDejd1YbLwgPS+I=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y= github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo= github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
go.unistack.org/micro-proto/v4 v4.1.0 h1:qPwL2n/oqh9RE3RTTDgt28XK3QzV597VugQPaw9lKUk= go.unistack.org/micro-proto/v4 v4.1.0 h1:qPwL2n/oqh9RE3RTTDgt28XK3QzV597VugQPaw9lKUk=
go.unistack.org/micro-proto/v4 v4.1.0/go.mod h1:ArmK7o+uFvxSY3dbJhKBBX4Pm1rhWdLEFf3LxBrMtec= go.unistack.org/micro-proto/v4 v4.1.0/go.mod h1:ArmK7o+uFvxSY3dbJhKBBX4Pm1rhWdLEFf3LxBrMtec=
golang.org/x/net v0.34.0 h1:Mb7Mrk043xzHgnRM88suvJFwzVrRfHEHJEl5/71CKw0= golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k= golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484 h1:Z7FRVJPSMaHQxD0uXU8WdgFh8PseLM8Q8NzhnpMrBhQ= golang.yandex/hasql/v2 v2.1.0 h1:7CaFFWeHoK5TvA+QvZzlKHlIN5sqNpqM8NSrXskZD/k=
google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484/go.mod h1:lcTa1sDdWEIHMWlITnIczmw5w60CF9ffkb8Z+DVmmjA= golang.yandex/hasql/v2 v2.1.0/go.mod h1:3Au1AxuJDCTXmS117BpbI6e+70kGWeyLR1qJAH6HdtA=
google.golang.org/grpc v1.69.4 h1:MF5TftSMkd8GLw/m0KM6V8CMOCY6NZ1NQDPGFgbTt4A= google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda h1:i/Q+bfisr7gq6feoJnS/DlpdwEL4ihp41fvRiM3Ork0=
google.golang.org/grpc v1.69.4/go.mod h1:vyjdE6jLBI76dgpDojsFGNaHlxdjXN9ghpnd2o7JGZ4= google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU= google.golang.org/grpc v1.76.0 h1:UnVkv1+uMLYXoIz6o7chp59WfQUYA2ex/BXQ9rHZu7A=
google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= google.golang.org/grpc v1.76.0/go.mod h1:Ju12QI8M6iQJtbcsV+awF5a4hfJMLi4X0JLo94ULZ6c=
google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=

117
hooks/metadata/metadata.go Normal file
View File

@@ -0,0 +1,117 @@
package metadata
import (
"context"
"go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/server"
)
type wrapper struct {
keys []string
client.Client
}
func NewClientWrapper(keys ...string) client.Wrapper {
return func(c client.Client) client.Client {
handler := &wrapper{
Client: c,
keys: keys,
}
return handler
}
}
func NewClientCallWrapper(keys ...string) client.CallWrapper {
return func(fn client.CallFunc) client.CallFunc {
return func(ctx context.Context, addr string, req client.Request, rsp interface{}, opts client.CallOptions) error {
if keys == nil {
return fn(ctx, addr, req, rsp, opts)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return fn(ctx, addr, req, rsp, opts)
}
}
}
func (w *wrapper) Call(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
if w.keys == nil {
return w.Client.Call(ctx, req, rsp, opts...)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range w.keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return w.Client.Call(ctx, req, rsp, opts...)
}
func (w *wrapper) Stream(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
if w.keys == nil {
return w.Client.Stream(ctx, req, opts...)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range w.keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return w.Client.Stream(ctx, req, opts...)
}
func NewServerHandlerWrapper(keys ...string) server.HandlerWrapper {
return func(fn server.HandlerFunc) server.HandlerFunc {
return func(ctx context.Context, req server.Request, rsp interface{}) error {
if keys == nil {
return fn(ctx, req, rsp)
}
if imd, iok := metadata.FromIncomingContext(ctx); iok && imd != nil {
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(len(imd))
}
for _, k := range keys {
if v := imd.Get(k); v != nil {
omd.Set(k, v...)
}
}
if !ook {
ctx = metadata.NewOutgoingContext(ctx, omd)
}
}
return fn(ctx, req, rsp)
}
}
}

View File

@@ -0,0 +1,63 @@
package recovery
import (
"context"
"fmt"
"go.unistack.org/micro/v4/errors"
"go.unistack.org/micro/v4/server"
)
func NewOptions(opts ...Option) Options {
options := Options{
ServerHandlerFn: DefaultServerHandlerFn,
}
for _, o := range opts {
o(&options)
}
return options
}
type Options struct {
ServerHandlerFn func(context.Context, server.Request, interface{}, error) error
}
type Option func(*Options)
func ServerHandlerFunc(fn func(context.Context, server.Request, interface{}, error) error) Option {
return func(o *Options) {
o.ServerHandlerFn = fn
}
}
var DefaultServerHandlerFn = func(ctx context.Context, req server.Request, rsp interface{}, err error) error {
return errors.BadRequest("", "%v", err)
}
var Hook = NewHook()
type hook struct {
opts Options
}
func NewHook(opts ...Option) *hook {
return &hook{opts: NewOptions(opts...)}
}
func (w *hook) ServerHandler(next server.FuncHandler) server.FuncHandler {
return func(ctx context.Context, req server.Request, rsp interface{}) (err error) {
defer func() {
r := recover()
switch verr := r.(type) {
case nil:
return
case error:
err = w.opts.ServerHandlerFn(ctx, req, rsp, verr)
default:
err = w.opts.ServerHandlerFn(ctx, req, rsp, fmt.Errorf("%v", r))
}
}()
err = next(ctx, req, rsp)
return err
}
}

View File

@@ -0,0 +1,103 @@
package requestid
import (
"context"
"net/textproto"
"go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/metadata"
"go.unistack.org/micro/v4/server"
"go.unistack.org/micro/v4/util/id"
)
type XRequestIDKey struct{}
// DefaultMetadataKey contains metadata key
var DefaultMetadataKey = textproto.CanonicalMIMEHeaderKey("x-request-id")
// DefaultMetadataFunc wil be used if user not provide own func to fill metadata
var DefaultMetadataFunc = func(ctx context.Context) (context.Context, error) {
var xid string
cid, cok := ctx.Value(XRequestIDKey{}).(string)
if cok && cid != "" {
xid = cid
}
imd, iok := metadata.FromIncomingContext(ctx)
if !iok || imd == nil {
imd = metadata.New(1)
ctx = metadata.NewIncomingContext(ctx, imd)
}
omd, ook := metadata.FromOutgoingContext(ctx)
if !ook || omd == nil {
omd = metadata.New(1)
ctx = metadata.NewOutgoingContext(ctx, omd)
}
if xid == "" {
xid = imd.GetJoined(DefaultMetadataKey)
if xid == "" {
xid = omd.GetJoined(DefaultMetadataKey)
}
}
if xid == "" {
var err error
xid, err = id.New()
if err != nil {
return ctx, err
}
}
if !cok {
ctx = context.WithValue(ctx, XRequestIDKey{}, xid)
}
if !iok {
imd.Set(DefaultMetadataKey, xid)
}
if !ook {
omd.Set(DefaultMetadataKey, xid)
}
return ctx, nil
}
type hook struct{}
func NewHook() *hook {
return &hook{}
}
func (w *hook) ServerHandler(next server.FuncHandler) server.FuncHandler {
return func(ctx context.Context, req server.Request, rsp interface{}) error {
var err error
if ctx, err = DefaultMetadataFunc(ctx); err != nil {
return err
}
return next(ctx, req, rsp)
}
}
func (w *hook) ClientCall(next client.FuncCall) client.FuncCall {
return func(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
var err error
if ctx, err = DefaultMetadataFunc(ctx); err != nil {
return err
}
return next(ctx, req, rsp, opts...)
}
}
func (w *hook) ClientStream(next client.FuncStream) client.FuncStream {
return func(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
var err error
if ctx, err = DefaultMetadataFunc(ctx); err != nil {
return nil, err
}
return next(ctx, req, opts...)
}
}

View File

@@ -0,0 +1,34 @@
package requestid
import (
"context"
"slices"
"testing"
"go.unistack.org/micro/v4/metadata"
)
func TestDefaultMetadataFunc(t *testing.T) {
ctx := context.TODO()
nctx, err := DefaultMetadataFunc(ctx)
if err != nil {
t.Fatalf("%v", err)
}
imd, ok := metadata.FromIncomingContext(nctx)
if !ok {
t.Fatalf("md missing in incoming context")
}
omd, ok := metadata.FromOutgoingContext(nctx)
if !ok {
t.Fatalf("md missing in outgoing context")
}
iv := imd.Get(DefaultMetadataKey)
ov := omd.Get(DefaultMetadataKey)
if !slices.Equal(iv, ov) {
t.Fatalf("missing metadata key value %v != %v", iv, ov)
}
}

51
hooks/sql/common.go Normal file
View File

@@ -0,0 +1,51 @@
package sql
import (
"database/sql/driver"
"errors"
"fmt"
"runtime"
)
//go:generate sh -c "go run gen.go > wrap_gen.go"
// namedValueToValue converts driver arguments of NamedValue format to Value format. Implemented in the same way as in
// database/sql ctxutil.go.
func namedValueToValue(named []driver.NamedValue) ([]driver.Value, error) {
dargs := make([]driver.Value, len(named))
for n, param := range named {
if len(param.Name) > 0 {
return nil, errors.New("sql: driver does not support the use of Named Parameters")
}
dargs[n] = param.Value
}
return dargs, nil
}
// namedValueToLabels convert driver arguments to interface{} slice
func namedValueToLabels(named []driver.NamedValue) []interface{} {
largs := make([]interface{}, 0, len(named)*2)
var name string
for _, param := range named {
if param.Name != "" {
name = param.Name
} else {
name = fmt.Sprintf("$%d", param.Ordinal)
}
largs = append(largs, fmt.Sprintf("%s=%v", name, param.Value))
}
return largs
}
// getCallerName get the name of the function A where A() -> B() -> GetFunctionCallerName()
func getCallerName() string {
pc, _, _, ok := runtime.Caller(3)
details := runtime.FuncForPC(pc)
var callerName string
if ok && details != nil {
callerName = details.Name()
} else {
callerName = labelUnknown
}
return callerName
}

467
hooks/sql/conn.go Normal file
View File

@@ -0,0 +1,467 @@
package sql
import (
"context"
"database/sql/driver"
"fmt"
"time"
"go.unistack.org/micro/v4/hooks/requestid"
"go.unistack.org/micro/v4/tracer"
)
var (
_ driver.Conn = (*wrapperConn)(nil)
_ driver.ConnBeginTx = (*wrapperConn)(nil)
_ driver.ConnPrepareContext = (*wrapperConn)(nil)
_ driver.Pinger = (*wrapperConn)(nil)
_ driver.Validator = (*wrapperConn)(nil)
_ driver.Queryer = (*wrapperConn)(nil) // nolint:staticcheck
_ driver.QueryerContext = (*wrapperConn)(nil)
_ driver.Execer = (*wrapperConn)(nil) // nolint:staticcheck
_ driver.ExecerContext = (*wrapperConn)(nil)
// _ driver.Connector
// _ driver.Driver
// _ driver.DriverContext
)
// wrapperConn defines a wrapper for driver.Conn
type wrapperConn struct {
d *wrapperDriver
dname string
conn driver.Conn
opts Options
ctx context.Context
//span tracer.Span
}
// Close implements driver.Conn Close
func (w *wrapperConn) Close() error {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Close"}
ts := time.Now()
err := w.conn.Close()
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Close", getCallerName(), td, err)...)
}
*/
return err
}
// Begin implements driver.Conn Begin
func (w *wrapperConn) Begin() (driver.Tx, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
labels := []string{labelMethod, "Begin"}
ts := time.Now()
tx, err := w.conn.Begin() // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Begin", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Begin", getCallerName(), td, err)...)
}
*/
return &wrapperTx{tx: tx, opts: w.opts, ctx: ctx}, nil
}
// BeginTx implements driver.ConnBeginTx BeginTx
func (w *wrapperConn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
name := getQueryName(ctx)
nctx, span := w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
span.AddLabels("db.method", "BeginTx")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "BeginTx", labelQuery, name}
connBeginTx, ok := w.conn.(driver.ConnBeginTx)
if !ok {
return w.Begin()
}
ts := time.Now()
tx, err := connBeginTx.BeginTx(nctx, opts)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "BeginTx", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "BeginTx", getCallerName(), td, err)...)
}
*/
return &wrapperTx{tx: tx, opts: w.opts, ctx: ctx, span: span}, nil
}
// Prepare implements driver.Conn Prepare
func (w *wrapperConn) Prepare(query string) (driver.Stmt, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Prepare", labelQuery, getCallerName()}
ts := time.Now()
stmt, err := w.conn.Prepare(query)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Prepare", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Prepare", getCallerName(), td, err)...)
}
*/
return wrapStmt(stmt, query, w.opts), nil
}
// PrepareContext implements driver.ConnPrepareContext PrepareContext
func (w *wrapperConn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "PrepareContext")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "PrepareContext", labelQuery, name}
conn, ok := w.conn.(driver.ConnPrepareContext)
if !ok {
return w.Prepare(query)
}
ts := time.Now()
stmt, err := conn.PrepareContext(nctx, query)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "PrepareContext", getCallerName(), td, err)...)
}
*/
return nil, err
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "PrepareContext", getCallerName(), td, err)...)
}
*/
return wrapStmt(stmt, query, w.opts), nil
}
// Exec implements driver.Execer Exec
func (w *wrapperConn) Exec(query string, args []driver.Value) (driver.Result, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Exec", labelQuery, getCallerName()}
// nolint:staticcheck
conn, ok := w.conn.(driver.Execer)
if !ok {
return nil, driver.ErrSkip
}
ts := time.Now()
res, err := conn.Exec(query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Exec", getCallerName(), td, err)...)
}
*/
return res, err
}
// Exec implements driver.StmtExecContext ExecContext
func (w *wrapperConn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "ExecContext")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
labels := []string{labelMethod, "ExecContext", labelQuery, name}
conn, ok := w.conn.(driver.ExecerContext)
if !ok {
// nolint:staticcheck
return nil, driver.ErrSkip
}
ts := time.Now()
res, err := conn.ExecContext(nctx, query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", getCallerName(), td, err)...)
}
*/
return res, err
}
// Ping implements driver.Pinger Ping
func (w *wrapperConn) Ping(ctx context.Context) error {
conn, ok := w.conn.(driver.Pinger)
if !ok {
// fallback path to check db alive
pc, err := w.d.Open(w.dname)
if err != nil {
return err
}
return pc.Close()
}
var nctx context.Context //nolint:gosimple
nctx = ctx
/*
var span tracer.Span
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "Ping")
defer span.Finish()
*/
labels := []string{labelMethod, "Ping"}
ts := time.Now()
err := conn.Ping(nctx)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
// span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Ping", getCallerName(), td, err)...)
}
*/
return err
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
return nil
}
// Query implements driver.Queryer Query
func (w *wrapperConn) Query(query string, args []driver.Value) (driver.Rows, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
// nolint:staticcheck
conn, ok := w.conn.(driver.Queryer)
if !ok {
return nil, driver.ErrSkip
}
labels := []string{labelMethod, "Query", labelQuery, getCallerName()}
ts := time.Now()
rows, err := conn.Query(query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Query", getCallerName(), td, err)...)
}
*/
return rows, err
}
// QueryContext implements Driver.QueryerContext QueryContext
func (w *wrapperConn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "QueryContext")
span.AddLabels("db.statement", name)
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
labels := []string{labelMethod, "QueryContext", labelQuery, name}
conn, ok := w.conn.(driver.QueryerContext)
if !ok {
return nil, driver.ErrSkip
}
ts := time.Now()
rows, err := conn.QueryContext(nctx, query, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", getCallerName(), td, err)...)
}
*/
return rows, err
}
// CheckNamedValue implements driver.NamedValueChecker
func (w *wrapperConn) CheckNamedValue(v *driver.NamedValue) error {
s, ok := w.conn.(driver.NamedValueChecker)
if !ok {
return driver.ErrSkip
}
return s.CheckNamedValue(v)
}
// IsValid implements driver.Validator
func (w *wrapperConn) IsValid() bool {
v, ok := w.conn.(driver.Validator)
if !ok {
return w.conn != nil
}
return v.IsValid()
}
func (w *wrapperConn) ResetSession(ctx context.Context) error {
s, ok := w.conn.(driver.SessionResetter)
if !ok {
return driver.ErrSkip
}
return s.ResetSession(ctx)
}

94
hooks/sql/driver.go Normal file
View File

@@ -0,0 +1,94 @@
package sql
import (
"context"
"database/sql/driver"
"time"
)
var (
// _ driver.DriverContext = (*wrapperDriver)(nil)
// _ driver.Connector = (*wrapperDriver)(nil)
)
/*
type conn interface {
driver.Pinger
driver.Execer
driver.ExecerContext
driver.Queryer
driver.QueryerContext
driver.Conn
driver.ConnPrepareContext
driver.ConnBeginTx
}
*/
// wrapperDriver defines a wrapper for driver.Driver
type wrapperDriver struct {
driver driver.Driver
opts Options
ctx context.Context
}
// NewWrapper creates and returns a new SQL driver with passed capabilities
func NewWrapper(d driver.Driver, opts ...Option) driver.Driver {
return &wrapperDriver{driver: d, opts: NewOptions(opts...), ctx: context.Background()}
}
type wrappedConnector struct {
connector driver.Connector
// name string
opts Options
ctx context.Context
}
func NewWrapperConnector(c driver.Connector, opts ...Option) driver.Connector {
return &wrappedConnector{connector: c, opts: NewOptions(opts...), ctx: context.Background()}
}
// Connect implements driver.Driver Connect
func (w *wrappedConnector) Connect(ctx context.Context) (driver.Conn, error) {
return w.connector.Connect(ctx)
}
// Driver implements driver.Driver Driver
func (w *wrappedConnector) Driver() driver.Driver {
return w.connector.Driver()
}
/*
// Connect implements driver.Driver OpenConnector
func (w *wrapperDriver) OpenConnector(name string) (driver.Conn, error) {
return &wrapperConnector{driver: w.driver, name: name, opts: w.opts}, nil
}
*/
// Open implements driver.Driver Open
func (w *wrapperDriver) Open(name string) (driver.Conn, error) {
// ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second) // Ensure eventual timeout
// defer cancel()
/*
connector, err := w.OpenConnector(name)
if err != nil {
return nil, err
}
return connector.Connect(ctx)
*/
ts := time.Now()
c, err := w.driver.Open(name)
td := time.Since(ts)
/*
if w.opts.LoggerEnabled {
w.opts.Logger.Log(w.ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(w.ctx, "Open", getCallerName(), td, err)...)
}
*/
_ = td
if err != nil {
return nil, err
}
return wrapConn(c, w.opts), nil
}

167
hooks/sql/gen.go Normal file
View File

@@ -0,0 +1,167 @@
//go:build ignore
package main
import (
"bytes"
"crypto/md5"
"fmt"
"io"
"sort"
"strings"
)
var connIfaces = []string{
"driver.ConnBeginTx",
"driver.ConnPrepareContext",
"driver.Execer",
"driver.ExecerContext",
"driver.NamedValueChecker",
"driver.Pinger",
"driver.Queryer",
"driver.QueryerContext",
"driver.SessionResetter",
"driver.Validator",
}
var stmtIfaces = []string{
"driver.StmtExecContext",
"driver.StmtQueryContext",
"driver.ColumnConverter",
"driver.NamedValueChecker",
}
func getHash(s []string) string {
h := md5.New()
io.WriteString(h, strings.Join(s, "|"))
return fmt.Sprintf("%x", h.Sum(nil))
}
func main() {
comboConn := all(connIfaces)
sort.Slice(comboConn, func(i, j int) bool {
return len(comboConn[i]) < len(comboConn[j])
})
comboStmt := all(stmtIfaces)
sort.Slice(comboStmt, func(i, j int) bool {
return len(comboStmt[i]) < len(comboStmt[j])
})
b := bytes.NewBuffer(nil)
b.WriteString("// Code generated. DO NOT EDIT.\n\n")
b.WriteString("package sql\n\n")
b.WriteString(`import "database/sql/driver"`)
b.WriteString("\n\n")
b.WriteString("func wrapConn(dc driver.Conn, opts Options) driver.Conn {\n")
b.WriteString("\tc := &wrapperConn{conn: dc, opts: opts}\n")
for idx := len(comboConn) - 1; idx >= 0; idx-- {
ifaces := comboConn[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("\tif _, ok := dc.(wrapConn%04d_%s); ok {\n", n, h))
b.WriteString("\treturn struct {\n")
b.WriteString("\t\tdriver.Conn\n")
b.WriteString(fmt.Sprintf("\t\t\t%s", strings.Join(ifaces, "\n\t\t\t")))
b.WriteString("\t\t\n}{")
for idx := range ifaces {
if idx > 0 {
b.WriteString(", ")
b.WriteString("c")
} else if idx == 0 {
b.WriteString("c")
} else {
b.WriteString("c")
}
}
b.WriteString(", c}\n")
b.WriteString("}\n\n")
}
b.WriteString("return c\n")
b.WriteString("}\n")
for idx := len(comboConn) - 1; idx >= 0; idx-- {
ifaces := comboConn[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("// %s\n", strings.Join(ifaces, "|")))
b.WriteString(fmt.Sprintf("type wrapConn%04d_%s interface {\n", n, h))
for _, iface := range ifaces {
b.WriteString(fmt.Sprintf("\t%s\n", iface))
}
b.WriteString("}\n\n")
}
b.WriteString("func wrapStmt(stmt driver.Stmt, query string, opts Options) driver.Stmt {\n")
b.WriteString("\tc := &wrapperStmt{stmt: stmt, query: query, opts: opts}\n")
for idx := len(comboStmt) - 1; idx >= 0; idx-- {
ifaces := comboStmt[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("\tif _, ok := stmt.(wrapStmt%04d_%s); ok {\n", n, h))
b.WriteString("\treturn struct {\n")
b.WriteString("\t\tdriver.Stmt\n")
b.WriteString(fmt.Sprintf("\t\t\t%s", strings.Join(ifaces, "\n\t\t\t")))
b.WriteString("\t\t\n}{")
for idx := range ifaces {
if idx > 0 {
b.WriteString(", ")
b.WriteString("c")
} else if idx == 0 {
b.WriteString("c")
} else {
b.WriteString("c")
}
}
b.WriteString(", c}\n")
b.WriteString("}\n\n")
}
b.WriteString("return c\n")
b.WriteString("}\n")
for idx := len(comboStmt) - 1; idx >= 0; idx-- {
ifaces := comboStmt[idx]
n := len(ifaces)
if n == 0 {
continue
}
h := getHash(ifaces)
b.WriteString(fmt.Sprintf("// %s\n", strings.Join(ifaces, "|")))
b.WriteString(fmt.Sprintf("type wrapStmt%04d_%s interface {\n", n, h))
for _, iface := range ifaces {
b.WriteString(fmt.Sprintf("\t%s\n", iface))
}
b.WriteString("}\n\n")
}
fmt.Printf("%s\n", b.String())
}
// all returns all combinations for a given string array.
func all[T any](set []T) (subsets [][]T) {
length := uint(len(set))
for subsetBits := 1; subsetBits < (1 << length); subsetBits++ {
var subset []T
for object := uint(0); object < length; object++ {
if (subsetBits>>object)&1 == 1 {
subset = append(subset, set[object])
}
}
subsets = append(subsets, subset)
}
return subsets
}

172
hooks/sql/options.go Normal file
View File

@@ -0,0 +1,172 @@
package sql
import (
"context"
"fmt"
"time"
"go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/meter"
"go.unistack.org/micro/v4/tracer"
)
var (
// DefaultMeterStatsInterval holds default stats interval
DefaultMeterStatsInterval = 5 * time.Second
// DefaultLoggerObserver used to prepare labels for logger
DefaultLoggerObserver = func(ctx context.Context, method string, query string, td time.Duration, err error) []interface{} {
labels := []interface{}{"db.method", method, "took", fmt.Sprintf("%v", td)}
if err != nil {
labels = append(labels, "error", err.Error())
}
if query != labelUnknown {
labels = append(labels, "query", query)
}
return labels
}
)
var (
MaxOpenConnections = "micro_sql_max_open_conn"
OpenConnections = "micro_sql_open_conn"
InuseConnections = "micro_sql_inuse_conn"
IdleConnections = "micro_sql_idle_conn"
WaitConnections = "micro_sql_waited_conn"
BlockedSeconds = "micro_sql_blocked_seconds"
MaxIdleClosed = "micro_sql_max_idle_closed"
MaxIdletimeClosed = "micro_sql_closed_max_idle"
MaxLifetimeClosed = "micro_sql_closed_max_lifetime"
meterRequestTotal = "micro_sql_request_total"
meterRequestLatencyMicroseconds = "micro_sql_latency_microseconds"
meterRequestDurationSeconds = "micro_sql_request_duration_seconds"
labelUnknown = "unknown"
labelQuery = "db_statement"
labelMethod = "db_method"
labelStatus = "status"
labelSuccess = "success"
labelFailure = "failure"
labelHost = "db_host"
labelDatabase = "db_name"
)
// Options struct holds wrapper options
type Options struct {
Logger logger.Logger
Meter meter.Meter
Tracer tracer.Tracer
DatabaseHost string
DatabaseName string
MeterStatsInterval time.Duration
LoggerLevel logger.Level
LoggerEnabled bool
LoggerObserver func(ctx context.Context, method string, name string, td time.Duration, err error) []interface{}
}
// Option func signature
type Option func(*Options)
// NewOptions create new Options struct from provided option slice
func NewOptions(opts ...Option) Options {
options := Options{
Logger: logger.DefaultLogger,
Meter: meter.DefaultMeter,
Tracer: tracer.DefaultTracer,
MeterStatsInterval: DefaultMeterStatsInterval,
LoggerLevel: logger.ErrorLevel,
LoggerObserver: DefaultLoggerObserver,
}
for _, o := range opts {
o(&options)
}
options.Meter = options.Meter.Clone(
meter.Labels(
labelHost, options.DatabaseHost,
labelDatabase, options.DatabaseName,
),
)
options.Logger = options.Logger.Clone(logger.WithAddCallerSkipCount(1))
return options
}
// MetricInterval specifies stats interval for *sql.DB
func MetricInterval(td time.Duration) Option {
return func(o *Options) {
o.MeterStatsInterval = td
}
}
func DatabaseHost(host string) Option {
return func(o *Options) {
o.DatabaseHost = host
}
}
func DatabaseName(name string) Option {
return func(o *Options) {
o.DatabaseName = name
}
}
// Meter passes meter.Meter to wrapper
func Meter(m meter.Meter) Option {
return func(o *Options) {
o.Meter = m
}
}
// Logger passes logger.Logger to wrapper
func Logger(l logger.Logger) Option {
return func(o *Options) {
o.Logger = l
}
}
// LoggerEnabled enable sql logging
func LoggerEnabled(b bool) Option {
return func(o *Options) {
o.LoggerEnabled = b
}
}
// LoggerLevel passes logger.Level option
func LoggerLevel(lvl logger.Level) Option {
return func(o *Options) {
o.LoggerLevel = lvl
}
}
// LoggerObserver passes observer to fill logger fields
func LoggerObserver(obs func(context.Context, string, string, time.Duration, error) []interface{}) Option {
return func(o *Options) {
o.LoggerObserver = obs
}
}
// Tracer passes tracer.Tracer to wrapper
func Tracer(t tracer.Tracer) Option {
return func(o *Options) {
o.Tracer = t
}
}
type queryNameKey struct{}
// QueryName passes query name to wrapper func
func QueryName(ctx context.Context, name string) context.Context {
if ctx == nil {
ctx = context.Background()
}
return context.WithValue(ctx, queryNameKey{}, name)
}
func getQueryName(ctx context.Context) string {
if v, ok := ctx.Value(queryNameKey{}).(string); ok && v != labelUnknown {
return v
}
return getCallerName()
}

95
hooks/sql/stats.go Normal file
View File

@@ -0,0 +1,95 @@
package sql
import (
"context"
"database/sql"
"sync"
"time"
)
type Statser interface {
Stats() sql.DBStats
}
func NewStatsMeter(ctx context.Context, db Statser, opts ...Option) {
if db == nil {
return
}
options := NewOptions(opts...)
var (
statsMu sync.Mutex
lastUpdated time.Time
maxOpenConnections, openConnections, inUse, idle, waitCount float64
maxIdleClosed, maxIdleTimeClosed, maxLifetimeClosed float64
waitDuration float64
)
updateFn := func() {
statsMu.Lock()
defer statsMu.Unlock()
if time.Since(lastUpdated) < options.MeterStatsInterval {
return
}
stats := db.Stats()
maxOpenConnections = float64(stats.MaxOpenConnections)
openConnections = float64(stats.OpenConnections)
inUse = float64(stats.InUse)
idle = float64(stats.Idle)
waitCount = float64(stats.WaitCount)
maxIdleClosed = float64(stats.MaxIdleClosed)
maxIdleTimeClosed = float64(stats.MaxIdleTimeClosed)
maxLifetimeClosed = float64(stats.MaxLifetimeClosed)
waitDuration = float64(stats.WaitDuration.Seconds())
lastUpdated = time.Now()
}
options.Meter.Gauge(MaxOpenConnections, func() float64 {
updateFn()
return maxOpenConnections
})
options.Meter.Gauge(OpenConnections, func() float64 {
updateFn()
return openConnections
})
options.Meter.Gauge(InuseConnections, func() float64 {
updateFn()
return inUse
})
options.Meter.Gauge(IdleConnections, func() float64 {
updateFn()
return idle
})
options.Meter.Gauge(WaitConnections, func() float64 {
updateFn()
return waitCount
})
options.Meter.Gauge(BlockedSeconds, func() float64 {
updateFn()
return waitDuration
})
options.Meter.Gauge(MaxIdleClosed, func() float64 {
updateFn()
return maxIdleClosed
})
options.Meter.Gauge(MaxIdletimeClosed, func() float64 {
updateFn()
return maxIdleTimeClosed
})
options.Meter.Gauge(MaxLifetimeClosed, func() float64 {
updateFn()
return maxLifetimeClosed
})
}

287
hooks/sql/stmt.go Normal file
View File

@@ -0,0 +1,287 @@
package sql
import (
"context"
"database/sql/driver"
"fmt"
"time"
requestid "go.unistack.org/micro/v4/hooks/requestid"
"go.unistack.org/micro/v4/tracer"
)
var (
_ driver.Stmt = (*wrapperStmt)(nil)
_ driver.StmtQueryContext = (*wrapperStmt)(nil)
_ driver.StmtExecContext = (*wrapperStmt)(nil)
_ driver.NamedValueChecker = (*wrapperStmt)(nil)
)
// wrapperStmt defines a wrapper for driver.Stmt
type wrapperStmt struct {
stmt driver.Stmt
opts Options
query string
ctx context.Context
}
// Close implements driver.Stmt Close
func (w *wrapperStmt) Close() error {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Close"}
ts := time.Now()
err := w.stmt.Close()
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Close", getCallerName(), td, err)...)
}
*/
return err
}
// NumInput implements driver.Stmt NumInput
func (w *wrapperStmt) NumInput() int {
return w.stmt.NumInput()
}
// CheckNamedValue implements driver.NamedValueChecker
func (w *wrapperStmt) CheckNamedValue(v *driver.NamedValue) error {
s, ok := w.stmt.(driver.NamedValueChecker)
if !ok {
return driver.ErrSkip
}
return s.CheckNamedValue(v)
}
// Exec implements driver.Stmt Exec
func (w *wrapperStmt) Exec(args []driver.Value) (driver.Result, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Exec"}
ts := time.Now()
res, err := w.stmt.Exec(args) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Exec", getCallerName(), td, err)...)
}
*/
return res, err
}
// Query implements driver.Stmt Query
func (w *wrapperStmt) Query(args []driver.Value) (driver.Rows, error) {
var ctx context.Context
if w.ctx != nil {
ctx = w.ctx
} else {
ctx = context.Background()
}
_ = ctx
labels := []string{labelMethod, "Query"}
ts := time.Now()
rows, err := w.stmt.Query(args) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "Query", getCallerName(), td, err)...)
}
*/
return rows, err
}
// ColumnConverter implements driver.ColumnConverter
func (w *wrapperStmt) ColumnConverter(idx int) driver.ValueConverter {
s, ok := w.stmt.(driver.ColumnConverter) // nolint:staticcheck
if !ok {
return nil
}
return s.ColumnConverter(idx)
}
// ExecContext implements driver.StmtExecContext ExecContext
func (w *wrapperStmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "ExecContext")
span.AddLabels("db.statement", name)
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "ExecContext", labelQuery, name}
if conn, ok := w.stmt.(driver.StmtExecContext); ok {
ts := time.Now()
res, err := conn.ExecContext(nctx, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", name, td, err)...)
}
*/
return res, err
}
values, err := namedValueToValue(args)
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", name, 0, err)...)
}
*/
return nil, err
}
ts := time.Now()
res, err := w.Exec(values) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "ExecContext", name, td, err)...)
}
*/
return res, err
}
// QueryContext implements driver.StmtQueryContext StmtQueryContext
func (w *wrapperStmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
var nctx context.Context
var span tracer.Span
name := getQueryName(ctx)
if w.ctx != nil {
nctx, span = w.opts.Tracer.Start(w.ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
} else {
nctx, span = w.opts.Tracer.Start(ctx, "sdk.database", tracer.WithSpanKind(tracer.SpanKindClient))
}
span.AddLabels("db.method", "QueryContext")
span.AddLabels("db.statement", name)
defer span.Finish()
if len(args) > 0 {
span.AddLabels("db.args", fmt.Sprintf("%v", namedValueToLabels(args)))
}
if id, ok := ctx.Value(requestid.XRequestIDKey{}).(string); ok {
span.AddLabels("x-request-id", id)
}
labels := []string{labelMethod, "QueryContext", labelQuery, name}
if conn, ok := w.stmt.(driver.StmtQueryContext); ok {
ts := time.Now()
rows, err := conn.QueryContext(nctx, args)
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", name, td, err)...)
}
*/
return rows, err
}
values, err := namedValueToValue(args)
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", name, 0, err)...)
}
*/
return nil, err
}
ts := time.Now()
rows, err := w.Query(values) // nolint:staticcheck
td := time.Since(ts)
te := td.Seconds()
if err != nil {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelFailure)...).Inc()
span.SetStatus(tracer.SpanStatusError, err.Error())
} else {
w.opts.Meter.Counter(meterRequestTotal, append(labels, labelStatus, labelSuccess)...).Inc()
}
w.opts.Meter.Summary(meterRequestLatencyMicroseconds, labels...).Update(te)
w.opts.Meter.Histogram(meterRequestDurationSeconds, labels...).Update(te)
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(ctx, "QueryContext", name, td, err)...)
}
*/
return rows, err
}

63
hooks/sql/tx.go Normal file
View File

@@ -0,0 +1,63 @@
package sql
import (
"context"
"database/sql/driver"
"time"
"go.unistack.org/micro/v4/tracer"
)
var _ driver.Tx = (*wrapperTx)(nil)
// wrapperTx defines a wrapper for driver.Tx
type wrapperTx struct {
tx driver.Tx
span tracer.Span
opts Options
ctx context.Context
}
// Commit implements driver.Tx Commit
func (w *wrapperTx) Commit() error {
ts := time.Now()
err := w.tx.Commit()
td := time.Since(ts)
_ = td
if w.span != nil {
if err != nil {
w.span.SetStatus(tracer.SpanStatusError, err.Error())
}
w.span.Finish()
}
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(w.ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(w.ctx, "Commit", getCallerName(), td, err)...)
}
*/
w.ctx = nil
return err
}
// Rollback implements driver.Tx Rollback
func (w *wrapperTx) Rollback() error {
ts := time.Now()
err := w.tx.Rollback()
td := time.Since(ts)
_ = td
if w.span != nil {
if err != nil {
w.span.SetStatus(tracer.SpanStatusError, err.Error())
}
w.span.Finish()
}
/*
if w.opts.LoggerEnabled && w.opts.Logger.V(w.opts.LoggerLevel) {
w.opts.Logger.Log(w.ctx, w.opts.LoggerLevel, w.opts.LoggerObserver(w.ctx, "Rollback", getCallerName(), td, err)...)
}
*/
w.ctx = nil
return err
}

19
hooks/sql/wrap.go Normal file
View File

@@ -0,0 +1,19 @@
package sql
import (
"database/sql/driver"
)
/*
func wrapDriver(d driver.Driver, opts Options) driver.Driver {
if _, ok := d.(driver.DriverContext); ok {
return &wrapperDriver{driver: d, opts: opts}
}
return struct{ driver.Driver }{&wrapperDriver{driver: d, opts: opts}}
}
*/
// WrapConn allows an existing driver.Conn to be wrapped.
func WrapConn(c driver.Conn, opts ...Option) driver.Conn {
return wrapConn(c, NewOptions(opts...))
}

20699
hooks/sql/wrap_gen.go Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,133 @@
package validator
import (
"context"
"go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/errors"
"go.unistack.org/micro/v4/server"
)
var (
DefaultClientErrorFunc = func(req client.Request, rsp interface{}, err error) error {
if rsp != nil {
return errors.BadGateway(req.Service(), "%v", err)
}
return errors.BadRequest(req.Service(), "%v", err)
}
DefaultServerErrorFunc = func(req server.Request, rsp interface{}, err error) error {
if rsp != nil {
return errors.BadGateway(req.Service(), "%v", err)
}
return errors.BadRequest(req.Service(), "%v", err)
}
)
type (
ClientErrorFunc func(client.Request, interface{}, error) error
ServerErrorFunc func(server.Request, interface{}, error) error
)
// Options struct holds wrapper options
type Options struct {
ClientErrorFn ClientErrorFunc
ServerErrorFn ServerErrorFunc
ClientValidateResponse bool
ServerValidateResponse bool
}
// Option func signature
type Option func(*Options)
func ClientValidateResponse(b bool) Option {
return func(o *Options) {
o.ClientValidateResponse = b
}
}
func ServerValidateResponse(b bool) Option {
return func(o *Options) {
o.ClientValidateResponse = b
}
}
func ClientReqErrorFn(fn ClientErrorFunc) Option {
return func(o *Options) {
o.ClientErrorFn = fn
}
}
func ServerErrorFn(fn ServerErrorFunc) Option {
return func(o *Options) {
o.ServerErrorFn = fn
}
}
func NewOptions(opts ...Option) Options {
options := Options{
ClientErrorFn: DefaultClientErrorFunc,
ServerErrorFn: DefaultServerErrorFunc,
}
for _, o := range opts {
o(&options)
}
return options
}
func NewHook(opts ...Option) *hook {
return &hook{opts: NewOptions(opts...)}
}
type validator interface {
Validate() error
}
type hook struct {
opts Options
}
func (w *hook) ClientCall(next client.FuncCall) client.FuncCall {
return func(ctx context.Context, req client.Request, rsp interface{}, opts ...client.CallOption) error {
if v, ok := req.Body().(validator); ok {
if err := v.Validate(); err != nil {
return w.opts.ClientErrorFn(req, nil, err)
}
}
err := next(ctx, req, rsp, opts...)
if v, ok := rsp.(validator); ok && w.opts.ClientValidateResponse {
if verr := v.Validate(); verr != nil {
return w.opts.ClientErrorFn(req, rsp, verr)
}
}
return err
}
}
func (w *hook) ClientStream(next client.FuncStream) client.FuncStream {
return func(ctx context.Context, req client.Request, opts ...client.CallOption) (client.Stream, error) {
if v, ok := req.Body().(validator); ok {
if err := v.Validate(); err != nil {
return nil, w.opts.ClientErrorFn(req, nil, err)
}
}
return next(ctx, req, opts...)
}
}
func (w *hook) ServerHandler(next server.FuncHandler) server.FuncHandler {
return func(ctx context.Context, req server.Request, rsp interface{}) error {
if v, ok := req.Body().(validator); ok {
if err := v.Validate(); err != nil {
return w.opts.ServerErrorFn(req, nil, err)
}
}
err := next(ctx, req, rsp)
if v, ok := rsp.(validator); ok && w.opts.ServerValidateResponse {
if verr := v.Validate(); verr != nil {
return w.opts.ServerErrorFn(req, rsp, verr)
}
}
return err
}
}

View File

@@ -4,18 +4,20 @@ package logger
type Level int8 type Level int8
const ( const (
// TraceLevel level usually used to find bugs, very verbose // TraceLevel usually used to find bugs, very verbose
TraceLevel Level = iota - 2 TraceLevel Level = iota - 2
// DebugLevel level used only when enabled debugging // DebugLevel used only when enabled debugging
DebugLevel DebugLevel
// InfoLevel level used for general info about what's going on inside the application // InfoLevel used for general info about what's going on inside the application
InfoLevel InfoLevel
// WarnLevel level used for non-critical entries // WarnLevel used for non-critical entries
WarnLevel WarnLevel
// ErrorLevel level used for errors that should definitely be noted // ErrorLevel used for errors that should definitely be noted
ErrorLevel ErrorLevel
// FatalLevel level used for critical errors and then calls `os.Exit(1)` // FatalLevel used for critical errors and then calls `os.Exit(1)`
FatalLevel FatalLevel
// NoneLevel used to disable logging
NoneLevel
) )
// String returns logger level string representation // String returns logger level string representation
@@ -33,6 +35,8 @@ func (l Level) String() string {
return "error" return "error"
case FatalLevel: case FatalLevel:
return "fatal" return "fatal"
case NoneLevel:
return "none"
} }
return "info" return "info"
} }
@@ -58,6 +62,8 @@ func ParseLevel(lvl string) Level {
return ErrorLevel return ErrorLevel
case FatalLevel.String(): case FatalLevel.String():
return FatalLevel return FatalLevel
case NoneLevel.String():
return NoneLevel
} }
return InfoLevel return InfoLevel
} }

View File

@@ -8,6 +8,7 @@ import (
"slices" "slices"
"time" "time"
"go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/meter" "go.unistack.org/micro/v4/meter"
) )
@@ -42,8 +43,10 @@ type Options struct {
Fields []interface{} Fields []interface{}
// ContextAttrFuncs contains funcs that executed before log func on context // ContextAttrFuncs contains funcs that executed before log func on context
ContextAttrFuncs []ContextAttrFunc ContextAttrFuncs []ContextAttrFunc
// callerSkipCount number of frmaes to skip // callerSkipCount number of frames to skip
CallerSkipCount int CallerSkipCount int
// AddCaller enables to get caller
AddCaller bool
// The logging level the logger should log // The logging level the logger should log
Level Level Level Level
// AddSource enabled writing source file and position in log // AddSource enabled writing source file and position in log
@@ -52,6 +55,12 @@ type Options struct {
AddStacktrace bool AddStacktrace bool
// DedupKeys deduplicate keys in log output // DedupKeys deduplicate keys in log output
DedupKeys bool DedupKeys bool
// FatalFinalizers runs in order in [logger.Fatal] method
FatalFinalizers []func(context.Context)
}
var DefaultFatalFinalizer = func(ctx context.Context) {
os.Exit(1)
} }
// NewOptions creates new options struct // NewOptions creates new options struct
@@ -65,6 +74,7 @@ func NewOptions(opts ...Option) Options {
AddSource: true, AddSource: true,
TimeFunc: time.Now, TimeFunc: time.Now,
Meter: meter.DefaultMeter, Meter: meter.DefaultMeter,
FatalFinalizers: []func(context.Context){DefaultFatalFinalizer},
} }
WithMicroKeys()(&options) WithMicroKeys()(&options)
@@ -76,6 +86,19 @@ func NewOptions(opts ...Option) Options {
return options return options
} }
func WithCallerEnabled(b bool) logger.Option {
return func(o *Options) {
o.AddCaller = b
}
}
// WithFatalFinalizers set logger.Fatal finalizers
func WithFatalFinalizers(fncs ...func(context.Context)) Option {
return func(o *Options) {
o.FatalFinalizers = fncs
}
}
// WithContextAttrFuncs appends default funcs for the context attrs filler // WithContextAttrFuncs appends default funcs for the context attrs filler
func WithContextAttrFuncs(fncs ...ContextAttrFunc) Option { func WithContextAttrFuncs(fncs ...ContextAttrFunc) Option {
return func(o *Options) { return func(o *Options) {

View File

@@ -4,14 +4,12 @@ import (
"context" "context"
"io" "io"
"log/slog" "log/slog"
"os"
"reflect" "reflect"
"regexp" "regexp"
"runtime" "runtime"
"strconv" "strconv"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/semconv" "go.unistack.org/micro/v4/semconv"
@@ -34,15 +32,16 @@ var (
warnValue = slog.StringValue("warn") warnValue = slog.StringValue("warn")
errorValue = slog.StringValue("error") errorValue = slog.StringValue("error")
fatalValue = slog.StringValue("fatal") fatalValue = slog.StringValue("fatal")
noneValue = slog.StringValue("none")
) )
type wrapper struct { type wrapper struct {
h slog.Handler h slog.Handler
level atomic.Int64 level int64
} }
func (h *wrapper) Enabled(ctx context.Context, level slog.Level) bool { func (h *wrapper) Enabled(ctx context.Context, level slog.Level) bool {
return level >= slog.Level(int(h.level.Load())) return level >= slog.Level(atomic.LoadInt64(&h.level))
} }
func (h *wrapper) Handle(ctx context.Context, rec slog.Record) error { func (h *wrapper) Handle(ctx context.Context, rec slog.Record) error {
@@ -50,11 +49,17 @@ func (h *wrapper) Handle(ctx context.Context, rec slog.Record) error {
} }
func (h *wrapper) WithAttrs(attrs []slog.Attr) slog.Handler { func (h *wrapper) WithAttrs(attrs []slog.Attr) slog.Handler {
return h.h.WithAttrs(attrs) return &wrapper{
h: h.h.WithAttrs(attrs),
level: atomic.LoadInt64(&h.level),
}
} }
func (h *wrapper) WithGroup(name string) slog.Handler { func (h *wrapper) WithGroup(name string) slog.Handler {
return h.h.WithGroup(name) return &wrapper{
h: h.h.WithGroup(name),
level: atomic.LoadInt64(&h.level),
}
} }
func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr { func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr {
@@ -85,6 +90,8 @@ func (s *slogLogger) renameAttr(_ []string, a slog.Attr) slog.Attr {
a.Value = errorValue a.Value = errorValue
case lvl >= logger.FatalLevel: case lvl >= logger.FatalLevel:
a.Value = fatalValue a.Value = fatalValue
case lvl >= logger.NoneLevel:
a.Value = noneValue
default: default:
a.Value = infoValue a.Value = infoValue
} }
@@ -114,10 +121,13 @@ func (s *slogLogger) Clone(opts ...logger.Option) logger.Logger {
attrs, _ := s.argsAttrs(options.Fields) attrs, _ := s.argsAttrs(options.Fields)
l := &slogLogger{ l := &slogLogger{
handler: &wrapper{h: s.handler.h.WithAttrs(attrs)}, handler: &wrapper{
opts: options, h: s.handler.h.WithAttrs(attrs),
level: atomic.LoadInt64(&s.handler.level),
},
opts: options,
} }
l.handler.level.Store(int64(loggerToSlogLevel(options.Level))) atomic.StoreInt64(&l.handler.level, int64(loggerToSlogLevel(options.Level)))
return l return l
} }
@@ -130,9 +140,9 @@ func (s *slogLogger) V(level logger.Level) bool {
} }
func (s *slogLogger) Level(level logger.Level) { func (s *slogLogger) Level(level logger.Level) {
atomic.StoreInt64(&s.handler.level, int64(loggerToSlogLevel(level)))
s.mu.Lock() s.mu.Lock()
s.opts.Level = level s.opts.Level = level
s.handler.level.Store(int64(loggerToSlogLevel(level)))
s.mu.Unlock() s.mu.Unlock()
} }
@@ -153,8 +163,11 @@ func (s *slogLogger) Fields(fields ...interface{}) logger.Logger {
} }
attrs, _ := s.argsAttrs(fields) attrs, _ := s.argsAttrs(fields)
l.handler = &wrapper{h: s.handler.h.WithAttrs(attrs)} l.handler = &wrapper{
l.handler.level.Store(int64(loggerToSlogLevel(l.opts.Level))) h: s.handler.h.WithAttrs(attrs),
level: atomic.LoadInt64(&s.handler.level),
}
atomic.StoreInt64(&l.handler.level, int64(loggerToSlogLevel(l.opts.Level)))
return l return l
} }
@@ -199,8 +212,11 @@ func (s *slogLogger) Init(opts ...logger.Option) error {
h = slog.NewJSONHandler(s.opts.Out, handleOpt) h = slog.NewJSONHandler(s.opts.Out, handleOpt)
} }
s.handler = &wrapper{h: h.WithAttrs(attrs)} s.handler = &wrapper{
s.handler.level.Store(int64(loggerToSlogLevel(s.opts.Level))) h: h.WithAttrs(attrs),
level: atomic.LoadInt64(&s.handler.level),
}
atomic.StoreInt64(&s.handler.level, int64(loggerToSlogLevel(s.opts.Level)))
s.mu.Unlock() s.mu.Unlock()
return nil return nil
@@ -228,11 +244,12 @@ func (s *slogLogger) Error(ctx context.Context, msg string, attrs ...interface{}
func (s *slogLogger) Fatal(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Fatal(ctx context.Context, msg string, attrs ...interface{}) {
s.printLog(ctx, logger.FatalLevel, msg, attrs...) s.printLog(ctx, logger.FatalLevel, msg, attrs...)
for _, fn := range s.opts.FatalFinalizers {
fn(ctx)
}
if closer, ok := s.opts.Out.(io.Closer); ok { if closer, ok := s.opts.Out.(io.Closer); ok {
closer.Close() closer.Close()
} }
time.Sleep(1 * time.Second)
os.Exit(1)
} }
func (s *slogLogger) Warn(ctx context.Context, msg string, attrs ...interface{}) { func (s *slogLogger) Warn(ctx context.Context, msg string, attrs ...interface{}) {
@@ -288,10 +305,17 @@ func (s *slogLogger) printLog(ctx context.Context, lvl logger.Level, msg string,
} }
} }
var pcs [1]uintptr var pcs uintptr
runtime.Callers(s.opts.CallerSkipCount, pcs[:]) // skip [Callers, printLog, LogLvlMethod]
r := slog.NewRecord(s.opts.TimeFunc(), loggerToSlogLevel(lvl), msg, pcs[0]) if s.opts.AddCaller {
var caller [1]uintptr
runtime.Callers(s.opts.CallerSkipCount, caller[:]) // skip [Callers, printLog, LogLvlMethod]
pcs = caller[0]
}
r := slog.NewRecord(s.opts.TimeFunc(), loggerToSlogLevel(lvl), msg, pcs)
r.AddAttrs(attrs...) r.AddAttrs(attrs...)
_ = s.handler.Handle(ctx, r) _ = s.handler.Handle(ctx, r)
} }
@@ -316,6 +340,8 @@ func loggerToSlogLevel(level logger.Level) slog.Level {
return slog.LevelDebug - 1 return slog.LevelDebug - 1
case logger.FatalLevel: case logger.FatalLevel:
return slog.LevelError + 1 return slog.LevelError + 1
case logger.NoneLevel:
return slog.LevelError + 2
default: default:
return slog.LevelInfo return slog.LevelInfo
} }
@@ -333,6 +359,8 @@ func slogToLoggerLevel(level slog.Level) logger.Level {
return logger.TraceLevel return logger.TraceLevel
case slog.LevelError + 1: case slog.LevelError + 1:
return logger.FatalLevel return logger.FatalLevel
case slog.LevelError + 2:
return logger.NoneLevel
default: default:
return logger.InfoLevel return logger.InfoLevel
} }

View File

@@ -36,6 +36,24 @@ func TestStacktrace(t *testing.T) {
} }
} }
func TestNoneLevel(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(logger.WithLevel(logger.NoneLevel), logger.WithOutput(buf),
WithHandlerFunc(slog.NewTextHandler),
logger.WithAddStacktrace(true),
)
if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
t.Fatal(err)
}
l.Error(ctx, "msg1", errors.New("err"))
if buf.Len() != 0 {
t.Fatalf("logger none level not works, buf contains: %s", buf.Bytes())
}
}
func TestDelayedBuffer(t *testing.T) { func TestDelayedBuffer(t *testing.T) {
ctx := context.TODO() ctx := context.TODO()
buf := bytes.NewBuffer(nil) buf := bytes.NewBuffer(nil)
@@ -62,7 +80,7 @@ func TestTime(t *testing.T) {
WithHandlerFunc(slog.NewTextHandler), WithHandlerFunc(slog.NewTextHandler),
logger.WithAddStacktrace(true), logger.WithAddStacktrace(true),
logger.WithTimeFunc(func() time.Time { logger.WithTimeFunc(func() time.Time {
return time.Unix(0, 0) return time.Unix(0, 0).UTC()
}), }),
) )
if err := l.Init(logger.WithFields("key1", "val1")); err != nil { if err := l.Init(logger.WithFields("key1", "val1")); err != nil {
@@ -71,8 +89,7 @@ func TestTime(t *testing.T) {
l.Error(ctx, "msg1", errors.New("err")) l.Error(ctx, "msg1", errors.New("err"))
if !bytes.Contains(buf.Bytes(), []byte(`timestamp=1970-01-01T03:00:00.000000000+03:00`)) && if !bytes.Contains(buf.Bytes(), []byte(`timestamp=1970-01-01T00:00:00.000000000Z`)) {
!bytes.Contains(buf.Bytes(), []byte(`timestamp=1970-01-01T00:00:00.000000000Z`)) {
t.Fatalf("logger error not works, buf contains: %s", buf.Bytes()) t.Fatalf("logger error not works, buf contains: %s", buf.Bytes())
} }
} }
@@ -406,7 +423,7 @@ func TestLogger(t *testing.T) {
func Test_WithContextAttrFunc(t *testing.T) { func Test_WithContextAttrFunc(t *testing.T) {
loggerContextAttrFuncs := []logger.ContextAttrFunc{ loggerContextAttrFuncs := []logger.ContextAttrFunc{
func(ctx context.Context) []interface{} { func(ctx context.Context) []interface{} {
md, ok := metadata.FromIncomingContext(ctx) md, ok := metadata.FromOutgoingContext(ctx)
if !ok { if !ok {
return nil return nil
} }
@@ -425,7 +442,7 @@ func Test_WithContextAttrFunc(t *testing.T) {
logger.DefaultContextAttrFuncs = append(logger.DefaultContextAttrFuncs, loggerContextAttrFuncs...) logger.DefaultContextAttrFuncs = append(logger.DefaultContextAttrFuncs, loggerContextAttrFuncs...)
ctx := context.TODO() ctx := context.TODO()
ctx = metadata.AppendIncomingContext(ctx, "X-Request-Id", uuid.New().String(), ctx = metadata.AppendOutgoingContext(ctx, "X-Request-Id", uuid.New().String(),
"Source-Service", "Test-System") "Source-Service", "Test-System")
buf := bytes.NewBuffer(nil) buf := bytes.NewBuffer(nil)
@@ -445,10 +462,32 @@ func Test_WithContextAttrFunc(t *testing.T) {
t.Fatalf("logger info, buf %s", buf.Bytes()) t.Fatalf("logger info, buf %s", buf.Bytes())
} }
buf.Reset() buf.Reset()
imd, _ := metadata.FromIncomingContext(ctx) omd, _ := metadata.FromOutgoingContext(ctx)
l.Info(ctx, "test message1") l.Info(ctx, "test message1")
imd.Set("Source-Service", "Test-System2") omd.Set("Source-Service", "Test-System2")
l.Info(ctx, "test message2") l.Info(ctx, "test message2")
// t.Logf("xxx %s", buf.Bytes()) // t.Logf("xxx %s", buf.Bytes())
} }
func TestFatalFinalizers(t *testing.T) {
ctx := context.TODO()
buf := bytes.NewBuffer(nil)
l := NewLogger(
logger.WithLevel(logger.TraceLevel),
logger.WithOutput(buf),
)
if err := l.Init(
logger.WithFatalFinalizers(func(ctx context.Context) {
l.Info(ctx, "fatal finalizer")
})); err != nil {
t.Fatal(err)
}
l.Fatal(ctx, "info_msg1")
if !bytes.Contains(buf.Bytes(), []byte("fatal finalizer")) {
t.Fatalf("logger dont have fatal message, buf %s", buf.Bytes())
}
if !bytes.Contains(buf.Bytes(), []byte("info_msg1")) {
t.Fatalf("logger dont have info_msg1 message, buf %s", buf.Bytes())
}
}

294
metadata/context.go Normal file
View File

@@ -0,0 +1,294 @@
package metadata
import (
"context"
"fmt"
"strings"
)
// In the metadata package, context and metadata are treated as immutable.
// Deep copies of metadata are made to keep things safe and correct.
// If a user takes a map and changes it across threads, it's their responsibility.
//
// 1. Incoming Context
//
// This context is provided by an external system and populated by the server or broker of the micro framework.
// It should not be modified. The idea is to extract all necessary data from it,
// validate the data, and transfer it into the current context.
// After that, only the current context should be used throughout the code.
//
// 2. Current Context
//
// This is the context used during the execution flow.
// You can add any needed metadata to it and pass it through your code.
//
// 3. Outgoing Context
//
// This context is for sending data to external systems.
// You can add what you need before sending it out.
// But its usually better to build and prepare this context right before making the external call,
// instead of changing it in many places.
//
// Execution Flow:
//
// [External System]
// ↓
// [Incoming Context]
// ↓
// [Extract & Validate Metadata from Incoming Context]
// ↓
// [Prepare Current Context]
// ↓
// [Enrich Current Context]
// ↓
// [Business Logic]
// ↓
// [Prepare Outgoing Context]
// ↓
// [External System Call]
type (
metadataCurrentKey struct{}
metadataIncomingKey struct{}
metadataOutgoingKey struct{}
rawMetadata struct {
md Metadata
added [][]string
}
)
// NewContext creates a new context with the provided Metadata attached.
// The Metadata must not be modified after calling this function.
func NewContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataCurrentKey{}, rawMetadata{md: md})
}
// NewIncomingContext creates a new context with the provided incoming Metadata attached.
// The Metadata must not be modified after calling this function.
func NewIncomingContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataIncomingKey{}, rawMetadata{md: md})
}
// NewOutgoingContext creates a new context with the provided outgoing Metadata attached.
// The Metadata must not be modified after calling this function.
func NewOutgoingContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataOutgoingKey{}, rawMetadata{md: md})
}
// AppendContext returns a new context with the provided key-value pairs (kv)
// merged with any existing metadata in the context. For a description of kv,
// please refer to the Pairs documentation.
func AppendContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataCurrentKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataCurrentKey{}, rawMetadata{md: md.md, added: added})
}
// AppendOutgoingContext returns a new context with the provided key-value pairs (kv)
// merged with any existing metadata in the context. For a description of kv,
// please refer to the Pairs documentation.
func AppendOutgoingContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendOutgoingContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataOutgoingKey{}, rawMetadata{md: md.md, added: added})
}
// FromContext retrieves a deep copy of the metadata from the context and returns it
// with a boolean indicating if it was found.
func FromContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataCurrentKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, true
}
// MustContext retrieves a deep copy of the metadata from the context and panics
// if the metadata is not found.
func MustContext(ctx context.Context) Metadata {
md, ok := FromContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// FromIncomingContext retrieves a deep copy of the metadata from the context and returns it
// with a boolean indicating if it was found.
func FromIncomingContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataIncomingKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromIncomingContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, true
}
// MustIncomingContext retrieves a deep copy of the metadata from the context and panics
// if the metadata is not found.
func MustIncomingContext(ctx context.Context) Metadata {
md, ok := FromIncomingContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// FromOutgoingContext retrieves a deep copy of the metadata from the context and returns it
// with a boolean indicating if it was found.
func FromOutgoingContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromOutgoingContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, ok
}
// MustOutgoingContext retrieves a deep copy of the metadata from the context and panics
// if the metadata is not found.
func MustOutgoingContext(ctx context.Context) Metadata {
md, ok := FromOutgoingContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// ValueFromCurrentContext retrieves a deep copy of the metadata for the given key
// from the context, performing a case-insensitive search if needed. Returns nil if not found.
func ValueFromCurrentContext(ctx context.Context, key string) []string {
md, ok := ctx.Value(metadataCurrentKey{}).(rawMetadata)
if !ok {
return nil
}
if v, ok := md.md[key]; ok {
return copyOf(v)
}
for k, v := range md.md {
// Case-insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}
// ValueFromIncomingContext retrieves a deep copy of the metadata for the given key
// from the context, performing a case-insensitive search if needed. Returns nil if not found.
func ValueFromIncomingContext(ctx context.Context, key string) []string {
raw, ok := ctx.Value(metadataIncomingKey{}).(rawMetadata)
if !ok {
return nil
}
if v, ok := raw.md[key]; ok {
return copyOf(v)
}
for k, v := range raw.md {
// Case-insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}
// ValueFromOutgoingContext retrieves a deep copy of the metadata for the given key
// from the context, performing a case-insensitive search if needed. Returns nil if not found.
func ValueFromOutgoingContext(ctx context.Context, key string) []string {
md, ok := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
if !ok {
return nil
}
if v, ok := md.md[key]; ok {
return copyOf(v)
}
for k, v := range md.md {
// Case-insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}

View File

@@ -2,18 +2,18 @@
package metadata package metadata
var ( var (
// HeaderTopic is the header name that contains topic name // HeaderTopic is the header name that contains topic name.
HeaderTopic = "Micro-Topic" HeaderTopic = "Micro-Topic"
// HeaderContentType specifies content type of message // HeaderContentType specifies content type of message.
HeaderContentType = "Content-Type" HeaderContentType = "Content-Type"
// HeaderEndpoint specifies endpoint in service // HeaderEndpoint specifies endpoint in service.
HeaderEndpoint = "Micro-Endpoint" HeaderEndpoint = "Micro-Endpoint"
// HeaderService specifies service // HeaderService specifies service.
HeaderService = "Micro-Service" HeaderService = "Micro-Service"
// HeaderTimeout specifies timeout of operation // HeaderTimeout specifies timeout of operation.
HeaderTimeout = "Micro-Timeout" HeaderTimeout = "Micro-Timeout"
// HeaderAuthorization specifies Authorization header // HeaderAuthorization specifies Authorization header.
HeaderAuthorization = "Authorization" HeaderAuthorization = "Authorization"
// HeaderXRequestID specifies request id // HeaderXRequestID specifies request id.
HeaderXRequestID = "X-Request-Id" HeaderXRequestID = "X-Request-Id"
) )

7
metadata/helpers.go Normal file
View File

@@ -0,0 +1,7 @@
package metadata
func copyOf(v []string) []string {
vals := make([]string, len(v))
copy(vals, v)
return vals
}

37
metadata/iterator.go Normal file
View File

@@ -0,0 +1,37 @@
package metadata
import "sort"
type Iterator struct {
md Metadata
keys []string
cur int
cnt int
}
// Next advances the iterator to the next element.
func (iter *Iterator) Next(k *string, v *[]string) bool {
if iter.cur+1 > iter.cnt {
return false
}
if k != nil && v != nil {
*k = iter.keys[iter.cur]
vv := iter.md[*k]
*v = make([]string, len(vv))
copy(*v, vv)
iter.cur++
}
return true
}
// Iterator returns an iterator for iterating over metadata in sorted order.
func (md Metadata) Iterator() *Iterator {
iter := &Iterator{md: md, cnt: len(md)}
iter.keys = make([]string, 0, iter.cnt)
for k := range md {
iter.keys = append(iter.keys, k)
}
sort.Strings(iter.keys)
return iter
}

View File

@@ -1,21 +1,18 @@
package metadata package metadata
import ( import (
"context"
"fmt" "fmt"
"net/textproto" "net/textproto"
"sort"
"strings" "strings"
) )
// defaultMetadataSize used when need to init new Metadata // defaultMetadataSize is used when initializing new Metadata.
var defaultMetadataSize = 2 var defaultMetadataSize = 2
// Metadata is a mapping from metadata keys to values. Users should use the following // Metadata maps keys to values. Use the New, NewWithMetadata and Pairs functions to create it.
// two convenience functions New and Pairs to generate Metadata.
type Metadata map[string][]string type Metadata map[string][]string
// New creates an zero Metadata. // New creates a zero-value Metadata with the specified size.
func New(l int) Metadata { func New(l int) Metadata {
if l == 0 { if l == 0 {
l = defaultMetadataSize l = defaultMetadataSize
@@ -24,7 +21,7 @@ func New(l int) Metadata {
return md return md
} }
// NewWithMetadata creates an Metadata from a given key-value map. // NewWithMetadata creates a Metadata from the provided key-value map.
func NewWithMetadata(m map[string]string) Metadata { func NewWithMetadata(m map[string]string) Metadata {
md := make(Metadata, len(m)) md := make(Metadata, len(m))
for key, val := range m { for key, val := range m {
@@ -33,8 +30,7 @@ func NewWithMetadata(m map[string]string) Metadata {
return md return md
} }
// Pairs returns an Metadata formed by the mapping of key, value ... // Pairs returns a Metadata formed from the key-value mapping. It panics if the length of kv is odd.
// Pairs panics if len(kv) is odd.
func Pairs(kv ...string) Metadata { func Pairs(kv ...string) Metadata {
if len(kv)%2 == 1 { if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: Pairs got the odd number of input pairs for metadata: %d", len(kv))) panic(fmt.Sprintf("metadata: Pairs got the odd number of input pairs for metadata: %d", len(kv)))
@@ -46,12 +42,19 @@ func Pairs(kv ...string) Metadata {
return md return md
} }
// Len returns the number of items in Metadata. // Join combines multiple Metadatas into a single Metadata.
func (md Metadata) Len() int { // The order of values for each key is determined by the order in which the Metadatas are provided to Join.
return len(md) func Join(mds ...Metadata) Metadata {
out := Metadata{}
for _, md := range mds {
for k, v := range md {
out[k] = append(out[k], v...)
}
}
return out
} }
// Copy returns a copy of Metadata. // Copy returns a deep copy of Metadata.
func Copy(src Metadata) Metadata { func Copy(src Metadata) Metadata {
out := make(Metadata, len(src)) out := make(Metadata, len(src))
for k, v := range src { for k, v := range src {
@@ -60,7 +63,7 @@ func Copy(src Metadata) Metadata {
return out return out
} }
// Copy returns a copy of Metadata. // Copy returns a deep copy of Metadata.
func (md Metadata) Copy() Metadata { func (md Metadata) Copy() Metadata {
out := make(Metadata, len(md)) out := make(Metadata, len(md))
for k, v := range md { for k, v := range md {
@@ -69,8 +72,28 @@ func (md Metadata) Copy() Metadata {
return out return out
} }
// AsHTTP1 returns a copy of Metadata // CopyTo performs a deep copy of Metadata to the out.
// with CanonicalMIMEHeaderKey. func (md Metadata) CopyTo(out Metadata) {
for k, v := range md {
out[k] = copyOf(v)
}
}
// Len returns the number of items in Metadata.
func (md Metadata) Len() int {
return len(md)
}
// AsMap returns a deep copy of Metadata as a map[string]string
func (md Metadata) AsMap() map[string]string {
out := make(map[string]string, len(md))
for k, v := range md {
out[k] = strings.Join(v, ",")
}
return out
}
// AsHTTP1 returns a deep copy of Metadata with keys converted to canonical MIME header key format.
func (md Metadata) AsHTTP1() map[string][]string { func (md Metadata) AsHTTP1() map[string][]string {
out := make(map[string][]string, len(md)) out := make(map[string][]string, len(md))
for k, v := range md { for k, v := range md {
@@ -79,8 +102,7 @@ func (md Metadata) AsHTTP1() map[string][]string {
return out return out
} }
// AsHTTP1 returns a copy of Metadata // AsHTTP2 returns a deep copy of Metadata with keys converted to lowercase.
// with strings.ToLower.
func (md Metadata) AsHTTP2() map[string][]string { func (md Metadata) AsHTTP2() map[string][]string {
out := make(map[string][]string, len(md)) out := make(map[string][]string, len(md))
for k, v := range md { for k, v := range md {
@@ -89,75 +111,35 @@ func (md Metadata) AsHTTP2() map[string][]string {
return out return out
} }
// CopyTo copies Metadata to out. // Get retrieves the values for a given key, checking the key in three formats:
func (md Metadata) CopyTo(out Metadata) { // - exact case,
for k, v := range md { // - lower case,
out[k] = copyOf(v) // - canonical MIME header key format.
} func (md Metadata) Get(k string) []string {
}
// Get obtains the values for a given key.
func (md Metadata) MustGet(k string) []string {
v, ok := md.Get(k)
if !ok {
panic("missing metadata key")
}
return v
}
// Get obtains the values for a given key.
func (md Metadata) Get(k string) ([]string, bool) {
v, ok := md[k] v, ok := md[k]
if !ok { if !ok {
v, ok = md[strings.ToLower(k)] v, ok = md[strings.ToLower(k)]
} }
if !ok { if !ok {
v, ok = md[textproto.CanonicalMIMEHeaderKey(k)] v = md[textproto.CanonicalMIMEHeaderKey(k)]
}
return v, ok
}
// MustGetJoined obtains the values for a given key
// with joined values with "," symbol
func (md Metadata) MustGetJoined(k string) string {
v, ok := md.GetJoined(k)
if !ok {
panic("missing metadata key")
} }
return v return v
} }
// GetJoined obtains the values for a given key // GetJoined retrieves the values for a given key and joins them into a single string, separated by commas.
// with joined values with "," symbol func (md Metadata) GetJoined(k string) string {
func (md Metadata) GetJoined(k string) (string, bool) { return strings.Join(md.Get(k), ",")
v, ok := md.Get(k)
if !ok {
return "", ok
}
return strings.Join(v, ","), true
} }
// Set sets the value of a given key with a slice of values. // Set assigns the values to the given key.
func (md Metadata) Add(key string, vals ...string) { func (md Metadata) Set(key string, vals ...string) {
if len(vals) == 0 { if len(vals) == 0 {
return return
} }
md[key] = vals md[key] = vals
} }
// Set sets the value of a given key with a slice of values. // Append adds values to the existing values for the given key.
func (md Metadata) Set(kvs ...string) {
if len(kvs)%2 == 1 {
panic(fmt.Sprintf("metadata: Set got an odd number of input pairs for metadata: %d", len(kvs)))
}
for i := 0; i < len(kvs); i += 2 {
md[kvs[i]] = append(md[kvs[i]], kvs[i+1])
}
}
// Append adds the values to key k, not overwriting what was already stored at
// that key.
func (md Metadata) Append(key string, vals ...string) { func (md Metadata) Append(key string, vals ...string) {
if len(vals) == 0 { if len(vals) == 0 {
return return
@@ -165,7 +147,10 @@ func (md Metadata) Append(key string, vals ...string) {
md[key] = append(md[key], vals...) md[key] = append(md[key], vals...)
} }
// Del removes the values for a given keys k. // Del removes the values for the given keys k. It checks and removes the keys in the following formats:
// - exact case,
// - lower case,
// - canonical MIME header key format.
func (md Metadata) Del(k ...string) { func (md Metadata) Del(k ...string) {
for i := range k { for i := range k {
delete(md, k[i]) delete(md, k[i])
@@ -173,321 +158,3 @@ func (md Metadata) Del(k ...string) {
delete(md, textproto.CanonicalMIMEHeaderKey(k[i])) delete(md, textproto.CanonicalMIMEHeaderKey(k[i]))
} }
} }
// Join joins any number of Metadatas into a single Metadata.
//
// The order of values for each key is determined by the order in which the Metadatas
// containing those values are presented to Join.
func Join(mds ...Metadata) Metadata {
out := Metadata{}
for _, Metadata := range mds {
for k, v := range Metadata {
out[k] = append(out[k], v...)
}
}
return out
}
type (
metadataIncomingKey struct{}
metadataOutgoingKey struct{}
metadataCurrentKey struct{}
)
// NewContext creates a new context with Metadata attached. Metadata must
// not be modified after calling this function.
func NewContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataCurrentKey{}, rawMetadata{md: md})
}
// NewIncomingContext creates a new context with incoming Metadata attached. Metadata must
// not be modified after calling this function.
func NewIncomingContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataIncomingKey{}, rawMetadata{md: md})
}
// NewOutgoingContext creates a new context with outgoing Metadata attached. If used
// in conjunction with AppendOutgoingContext, NewOutgoingContext will
// overwrite any previously-appended metadata. Metadata must not be modified after
// calling this function.
func NewOutgoingContext(ctx context.Context, md Metadata) context.Context {
return context.WithValue(ctx, metadataOutgoingKey{}, rawMetadata{md: md})
}
// AppendContext returns a new context with the provided kv merged
// with any existing metadata in the context. Please refer to the documentation
// of Pairs for a description of kv.
func AppendContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataCurrentKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataCurrentKey{}, rawMetadata{md: md.md, added: added})
}
// AppendIncomingContext returns a new context with the provided kv merged
// with any existing metadata in the context. Please refer to the documentation
// of Pairs for a description of kv.
func AppendIncomingContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendIncomingContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataIncomingKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataIncomingKey{}, rawMetadata{md: md.md, added: added})
}
// AppendOutgoingContext returns a new context with the provided kv merged
// with any existing metadata in the context. Please refer to the documentation
// of Pairs for a description of kv.
func AppendOutgoingContext(ctx context.Context, kv ...string) context.Context {
if len(kv)%2 == 1 {
panic(fmt.Sprintf("metadata: AppendOutgoingContext got an odd number of input pairs for metadata: %d", len(kv)))
}
md, _ := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
added := make([][]string, len(md.added)+1)
copy(added, md.added)
kvCopy := make([]string, 0, len(kv))
for i := 0; i < len(kv); i += 2 {
kvCopy = append(kvCopy, strings.ToLower(kv[i]), kv[i+1])
}
added[len(added)-1] = kvCopy
return context.WithValue(ctx, metadataOutgoingKey{}, rawMetadata{md: md.md, added: added})
}
// FromContext returns the metadata in ctx if it exists.
func FromContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataCurrentKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, true
}
// MustContext returns the metadata in ctx.
func MustContext(ctx context.Context) Metadata {
md, ok := FromContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// FromIncomingContext returns the incoming metadata in ctx if it exists.
func FromIncomingContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataIncomingKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromIncomingContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, true
}
// MustIncomingContext returns the incoming metadata in ctx.
func MustIncomingContext(ctx context.Context) Metadata {
md, ok := FromIncomingContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// ValueFromIncomingContext returns the metadata value corresponding to the metadata
// key from the incoming metadata if it exists. Keys are matched in a case insensitive
// manner.
func ValueFromIncomingContext(ctx context.Context, key string) []string {
raw, ok := ctx.Value(metadataIncomingKey{}).(rawMetadata)
if !ok {
return nil
}
if v, ok := raw.md[key]; ok {
return copyOf(v)
}
for k, v := range raw.md {
// Case insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}
// ValueFromCurrentContext returns the metadata value corresponding to the metadata
// key from the incoming metadata if it exists. Keys are matched in a case insensitive
// manner.
func ValueFromCurrentContext(ctx context.Context, key string) []string {
md, ok := ctx.Value(metadataCurrentKey{}).(rawMetadata)
if !ok {
return nil
}
if v, ok := md.md[key]; ok {
return copyOf(v)
}
for k, v := range md.md {
// Case insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}
// MustOutgoingContext returns the outgoing metadata in ctx.
func MustOutgoingContext(ctx context.Context) Metadata {
md, ok := FromOutgoingContext(ctx)
if !ok {
panic("missing metadata")
}
return md
}
// ValueFromOutgoingContext returns the metadata value corresponding to the metadata
// key from the incoming metadata if it exists. Keys are matched in a case insensitive
// manner.
func ValueFromOutgoingContext(ctx context.Context, key string) []string {
md, ok := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
if !ok {
return nil
}
if v, ok := md.md[key]; ok {
return copyOf(v)
}
for k, v := range md.md {
// Case insensitive comparison: Metadata is a map, and there's no guarantee
// that the Metadata attached to the context is created using our helper
// functions.
if strings.EqualFold(k, key) {
return copyOf(v)
}
}
return nil
}
func copyOf(v []string) []string {
vals := make([]string, len(v))
copy(vals, v)
return vals
}
// FromOutgoingContext returns the outgoing metadata in ctx if it exists.
func FromOutgoingContext(ctx context.Context) (Metadata, bool) {
raw, ok := ctx.Value(metadataOutgoingKey{}).(rawMetadata)
if !ok {
return nil, false
}
metadataSize := len(raw.md)
for i := range raw.added {
metadataSize += len(raw.added[i]) / 2
}
out := make(Metadata, metadataSize)
for k, v := range raw.md {
out[k] = copyOf(v)
}
for _, added := range raw.added {
if len(added)%2 == 1 {
panic(fmt.Sprintf("metadata: FromOutgoingContext got an odd number of input pairs for metadata: %d", len(added)))
}
for i := 0; i < len(added); i += 2 {
out[added[i]] = append(out[added[i]], added[i+1])
}
}
return out, ok
}
type rawMetadata struct {
md Metadata
added [][]string
}
// Iterator used to iterate over metadata with order
type Iterator struct {
md Metadata
keys []string
cur int
cnt int
}
// Next advance iterator to next element
func (iter *Iterator) Next(k *string, v *[]string) bool {
if iter.cur+1 > iter.cnt {
return false
}
if k != nil && v != nil {
*k = iter.keys[iter.cur]
vv := iter.md[*k]
*v = make([]string, len(vv))
copy(*v, vv)
iter.cur++
}
return true
}
// Iterator returns the itarator for metadata in sorted order
func (md Metadata) Iterator() *Iterator {
iter := &Iterator{md: md, cnt: len(md)}
iter.keys = make([]string, 0, iter.cnt)
for k := range md {
iter.keys = append(iter.keys, k)
}
sort.Strings(iter.keys)
return iter
}

View File

@@ -5,6 +5,15 @@ import (
"testing" "testing"
) )
func TesSet(t *testing.T) {
md := Pairs("key1", "val1", "key2", "val2")
md.Set("key1", "val2", "val3")
v := md.GetJoined("X-Request-Id")
if v != "val2, val3" {
t.Fatal("set not works")
}
}
/* /*
func TestAppendOutgoingContextModify(t *testing.T) { func TestAppendOutgoingContextModify(t *testing.T) {
md := Pairs("key1", "val1") md := Pairs("key1", "val1")
@@ -19,8 +28,8 @@ func TestAppendOutgoingContextModify(t *testing.T) {
func TestLowercase(t *testing.T) { func TestLowercase(t *testing.T) {
md := New(1) md := New(1)
md["x-request-id"] = []string{"12345"} md["x-request-id"] = []string{"12345"}
v, ok := md.GetJoined("X-Request-Id") v := md.GetJoined("X-Request-Id")
if !ok || v == "" { if v == "" {
t.Fatalf("metadata invalid %#+v", md) t.Fatalf("metadata invalid %#+v", md)
} }
} }
@@ -47,33 +56,9 @@ func TestMultipleUsage(t *testing.T) {
_ = omd _ = omd
} }
func TestMetadataSetMultiple(t *testing.T) {
md := New(4)
md.Set("key1", "val1", "key2", "val2")
if v, ok := md.GetJoined("key1"); !ok || v != "val1" {
t.Fatalf("invalid kv %#+v", md)
}
if v, ok := md.GetJoined("key2"); !ok || v != "val2" {
t.Fatalf("invalid kv %#+v", md)
}
}
func TestAppend(t *testing.T) {
ctx := context.Background()
ctx = AppendIncomingContext(ctx, "key1", "val1", "key2", "val2")
md, ok := FromIncomingContext(ctx)
if !ok {
t.Fatal("metadata empty")
}
if _, ok := md.Get("key1"); !ok {
t.Fatal("key1 not found")
}
}
func TestPairs(t *testing.T) { func TestPairs(t *testing.T) {
md := Pairs("key1", "val1", "key2", "val2") md := Pairs("key1", "val1", "key2", "val2")
if _, ok := md.Get("key1"); !ok { if v := md.Get("key1"); v == nil {
t.Fatal("key1 not found") t.Fatal("key1 not found")
} }
} }
@@ -97,7 +82,7 @@ func TestPassing(t *testing.T) {
if !ok { if !ok {
t.Fatalf("missing metadata from outgoing context") t.Fatalf("missing metadata from outgoing context")
} }
if v, ok := md.Get("Key1"); !ok || v[0] != "Val1" { if v := md.Get("Key1"); v == nil || v[0] != "Val1" {
t.Fatalf("invalid metadata value %#+v", md) t.Fatalf("invalid metadata value %#+v", md)
} }
} }
@@ -127,21 +112,21 @@ func TestIterator(t *testing.T) {
func TestMedataCanonicalKey(t *testing.T) { func TestMedataCanonicalKey(t *testing.T) {
md := New(1) md := New(1)
md.Set("x-request-id", "12345") md.Set("x-request-id", "12345")
v, ok := md.GetJoined("x-request-id") v := md.GetJoined("x-request-id")
if !ok { if v == "" {
t.Fatalf("failed to get x-request-id") t.Fatalf("failed to get x-request-id")
} else if v != "12345" { } else if v != "12345" {
t.Fatalf("invalid metadata value: %s != %s", "12345", v) t.Fatalf("invalid metadata value: %s != %s", "12345", v)
} }
v, ok = md.GetJoined("X-Request-Id") v = md.GetJoined("X-Request-Id")
if !ok { if v == "" {
t.Fatalf("failed to get x-request-id") t.Fatalf("failed to get x-request-id")
} else if v != "12345" { } else if v != "12345" {
t.Fatalf("invalid metadata value: %s != %s", "12345", v) t.Fatalf("invalid metadata value: %s != %s", "12345", v)
} }
v, ok = md.GetJoined("X-Request-ID") v = md.GetJoined("X-Request-ID")
if !ok { if v == "" {
t.Fatalf("failed to get x-request-id") t.Fatalf("failed to get x-request-id")
} else if v != "12345" { } else if v != "12345" {
t.Fatalf("invalid metadata value: %s != %s", "12345", v) t.Fatalf("invalid metadata value: %s != %s", "12345", v)
@@ -153,8 +138,8 @@ func TestMetadataSet(t *testing.T) {
md.Set("Key", "val") md.Set("Key", "val")
val, ok := md.GetJoined("Key") val := md.GetJoined("Key")
if !ok { if val == "" {
t.Fatal("key Key not found") t.Fatal("key Key not found")
} }
if val != "val" { if val != "val" {
@@ -169,8 +154,8 @@ func TestMetadataDelete(t *testing.T) {
} }
md.Del("Baz") md.Del("Baz")
_, ok := md.Get("Baz") v := md.Get("Baz")
if ok { if v != nil {
t.Fatal("key Baz not deleted") t.Fatal("key Baz not deleted")
} }
} }
@@ -269,20 +254,6 @@ func TestNewOutgoingContext(t *testing.T) {
} }
} }
func TestAppendIncomingContext(t *testing.T) {
md := New(1)
md.Set("key1", "val1")
ctx := AppendIncomingContext(context.TODO(), "key2", "val2")
nmd, ok := FromIncomingContext(ctx)
if nmd == nil || !ok {
t.Fatal("AppendIncomingContext not works")
}
if v, ok := nmd.GetJoined("key2"); !ok || v != "val2" {
t.Fatal("AppendIncomingContext not works")
}
}
func TestAppendOutgoingContext(t *testing.T) { func TestAppendOutgoingContext(t *testing.T) {
md := New(1) md := New(1)
md.Set("key1", "val1") md.Set("key1", "val1")
@@ -292,7 +263,7 @@ func TestAppendOutgoingContext(t *testing.T) {
if nmd == nil || !ok { if nmd == nil || !ok {
t.Fatal("AppendOutgoingContext not works") t.Fatal("AppendOutgoingContext not works")
} }
if v, ok := nmd.GetJoined("key2"); !ok || v != "val2" { if v := nmd.GetJoined("key2"); v != "val2" {
t.Fatal("AppendOutgoingContext not works") t.Fatal("AppendOutgoingContext not works")
} }
} }

View File

@@ -4,8 +4,8 @@ package meter
import ( import (
"io" "io"
"sort" "sort"
"strconv"
"strings" "strings"
"sync"
"time" "time"
) )
@@ -49,9 +49,11 @@ type Meter interface {
Set(opts ...Option) Meter Set(opts ...Option) Meter
// Histogram get or create histogram // Histogram get or create histogram
Histogram(name string, labels ...string) Histogram Histogram(name string, labels ...string) Histogram
// HistogramExt get or create histogram with specified quantiles
HistogramExt(name string, quantiles []float64, labels ...string) Histogram
// Summary get or create summary // Summary get or create summary
Summary(name string, labels ...string) Summary Summary(name string, labels ...string) Summary
// SummaryExt get or create summary with spcified quantiles and window time // SummaryExt get or create summary with specified quantiles and window time
SummaryExt(name string, window time.Duration, quantiles []float64, labels ...string) Summary SummaryExt(name string, window time.Duration, quantiles []float64, labels ...string) Summary
// Write writes metrics to io.Writer // Write writes metrics to io.Writer
Write(w io.Writer, opts ...Option) error Write(w io.Writer, opts ...Option) error
@@ -59,6 +61,8 @@ type Meter interface {
Options() Options Options() Options
// String return meter type // String return meter type
String() string String() string
// Unregister metric name and drop all data
Unregister(name string, labels ...string) bool
} }
// Counter is a counter // Counter is a counter
@@ -80,7 +84,11 @@ type FloatCounter interface {
// Gauge is a float64 gauge // Gauge is a float64 gauge
type Gauge interface { type Gauge interface {
Add(float64)
Get() float64 Get() float64
Set(float64)
Dec()
Inc()
} }
// Histogram is a histogram for non-negative values with automatically created buckets // Histogram is a histogram for non-negative values with automatically created buckets
@@ -117,6 +125,39 @@ func BuildLabels(labels ...string) []string {
return labels return labels
} }
var spool = newStringsPool(500)
type stringsPool struct {
p *sync.Pool
c int
}
func newStringsPool(size int) *stringsPool {
p := &stringsPool{c: size}
p.p = &sync.Pool{
New: func() interface{} {
return &strings.Builder{}
},
}
return p
}
func (p *stringsPool) Cap() int {
return p.c
}
func (p *stringsPool) Get() *strings.Builder {
return p.p.Get().(*strings.Builder)
}
func (p *stringsPool) Put(b *strings.Builder) {
if b.Cap() > p.c {
return
}
b.Reset()
p.p.Put(b)
}
// BuildName used to combine metric with labels. // BuildName used to combine metric with labels.
// If labels count is odd, drop last element // If labels count is odd, drop last element
func BuildName(name string, labels ...string) string { func BuildName(name string, labels ...string) string {
@@ -125,8 +166,6 @@ func BuildName(name string, labels ...string) string {
} }
if len(labels) > 2 { if len(labels) > 2 {
sort.Sort(byKey(labels))
idx := 0 idx := 0
for { for {
if labels[idx] == labels[idx+2] { if labels[idx] == labels[idx+2] {
@@ -141,7 +180,9 @@ func BuildName(name string, labels ...string) string {
} }
} }
var b strings.Builder b := spool.Get()
defer spool.Put(b)
_, _ = b.WriteString(name) _, _ = b.WriteString(name)
_, _ = b.WriteRune('{') _, _ = b.WriteRune('{')
for idx := 0; idx < len(labels); idx += 2 { for idx := 0; idx < len(labels); idx += 2 {
@@ -149,8 +190,9 @@ func BuildName(name string, labels ...string) string {
_, _ = b.WriteRune(',') _, _ = b.WriteRune(',')
} }
_, _ = b.WriteString(labels[idx]) _, _ = b.WriteString(labels[idx])
_, _ = b.WriteString(`=`) _, _ = b.WriteString(`="`)
_, _ = b.WriteString(strconv.Quote(labels[idx+1])) _, _ = b.WriteString(labels[idx+1])
_, _ = b.WriteRune('"')
} }
_, _ = b.WriteRune('}') _, _ = b.WriteRune('}')

View File

@@ -50,11 +50,12 @@ func TestBuildName(t *testing.T) {
data := map[string][]string{ data := map[string][]string{
`my_metric{firstlabel="value2",zerolabel="value3"}`: { `my_metric{firstlabel="value2",zerolabel="value3"}`: {
"my_metric", "my_metric",
"zerolabel", "value3", "firstlabel", "value2", "firstlabel", "value2",
"zerolabel", "value3",
}, },
`my_metric{broker="broker2",register="mdns",server="tcp"}`: { `my_metric{broker="broker2",register="mdns",server="tcp"}`: {
"my_metric", "my_metric",
"broker", "broker1", "broker", "broker2", "server", "http", "server", "tcp", "register", "mdns", "broker", "broker1", "broker", "broker2", "register", "mdns", "server", "http", "server", "tcp",
}, },
`my_metric{aaa="aaa"}`: { `my_metric{aaa="aaa"}`: {
"my_metric", "my_metric",

View File

@@ -28,6 +28,10 @@ func (r *noopMeter) Name() string {
return r.opts.Name return r.opts.Name
} }
func (r *noopMeter) Unregister(name string, labels ...string) bool {
return true
}
// Init initialize options // Init initialize options
func (r *noopMeter) Init(opts ...Option) error { func (r *noopMeter) Init(opts ...Option) error {
for _, o := range opts { for _, o := range opts {
@@ -66,6 +70,11 @@ func (r *noopMeter) Histogram(_ string, labels ...string) Histogram {
return &noopHistogram{labels: labels} return &noopHistogram{labels: labels}
} }
// HistogramExt implements the Meter interface
func (r *noopMeter) HistogramExt(_ string, quantiles []float64, labels ...string) Histogram {
return &noopHistogram{labels: labels}
}
// Set implements the Meter interface // Set implements the Meter interface
func (r *noopMeter) Set(opts ...Option) Meter { func (r *noopMeter) Set(opts ...Option) Meter {
m := &noopMeter{opts: r.opts} m := &noopMeter{opts: r.opts}
@@ -132,6 +141,18 @@ type noopGauge struct {
labels []string labels []string
} }
func (r *noopGauge) Add(float64) {
}
func (r *noopGauge) Set(float64) {
}
func (r *noopGauge) Inc() {
}
func (r *noopGauge) Dec() {
}
func (r *noopGauge) Get() float64 { func (r *noopGauge) Get() float64 {
return 0 return 0
} }

View File

@@ -4,6 +4,8 @@ import (
"context" "context"
) )
var DefaultQuantiles = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}
// Option powers the configuration for metrics implementations: // Option powers the configuration for metrics implementations:
type Option func(*Options) type Option func(*Options)
@@ -23,6 +25,8 @@ type Options struct {
WriteProcessMetrics bool WriteProcessMetrics bool
// WriteFDMetrics flag to write fd metrics // WriteFDMetrics flag to write fd metrics
WriteFDMetrics bool WriteFDMetrics bool
// Quantiles specifies buckets for histogram
Quantiles []float64
} }
// NewOptions prepares a set of options: // NewOptions prepares a set of options:
@@ -61,14 +65,12 @@ func Address(value string) Option {
} }
} }
/* // Quantiles defines the desired spread of statistics for histogram metrics:
// TimingObjectives defines the desired spread of statistics for histogram / timing metrics: func Quantiles(quantiles []float64) Option {
func TimingObjectives(value map[float64]float64) Option {
return func(o *Options) { return func(o *Options) {
o.TimingObjectives = value o.Quantiles = quantiles
} }
} }
*/
// Labels add the meter labels // Labels add the meter labels
func Labels(ls ...string) Option { func Labels(ls ...string) Option {

View File

@@ -91,7 +91,7 @@ func (p *bro) Connect(_ context.Context) error { return nil }
func (p *bro) Disconnect(_ context.Context) error { return nil } func (p *bro) Disconnect(_ context.Context) error { return nil }
// NewMessage creates new message // NewMessage creates new message
func (p *bro) NewMessage(_ context.Context, _ metadata.Metadata, _ interface{}, _ ...broker.PublishOption) (broker.Message, error) { func (p *bro) NewMessage(_ context.Context, _ metadata.Metadata, _ interface{}, _ ...broker.MessageOption) (broker.Message, error) {
return nil, nil return nil, nil
} }

View File

@@ -11,8 +11,8 @@ import (
) )
type httpProfile struct { type httpProfile struct {
server *http.Server server *http.Server
sync.Mutex mu sync.Mutex
running bool running bool
} }
@@ -21,8 +21,8 @@ var DefaultAddress = ":6060"
// Start the profiler // Start the profiler
func (h *httpProfile) Start() error { func (h *httpProfile) Start() error {
h.Lock() h.mu.Lock()
defer h.Unlock() defer h.mu.Unlock()
if h.running { if h.running {
return nil return nil
@@ -30,9 +30,9 @@ func (h *httpProfile) Start() error {
go func() { go func() {
if err := h.server.ListenAndServe(); err != nil { if err := h.server.ListenAndServe(); err != nil {
h.Lock() h.mu.Lock()
h.running = false h.running = false
h.Unlock() h.mu.Unlock()
} }
}() }()
@@ -43,8 +43,8 @@ func (h *httpProfile) Start() error {
// Stop the profiler // Stop the profiler
func (h *httpProfile) Stop() error { func (h *httpProfile) Stop() error {
h.Lock() h.mu.Lock()
defer h.Unlock() defer h.mu.Unlock()
if !h.running { if !h.running {
return nil return nil

View File

@@ -17,7 +17,7 @@ type profiler struct {
cpuFile *os.File cpuFile *os.File
memFile *os.File memFile *os.File
opts profile.Options opts profile.Options
sync.Mutex mu sync.Mutex
running bool running bool
} }
@@ -39,8 +39,8 @@ func (p *profiler) writeHeap(f *os.File) {
} }
func (p *profiler) Start() error { func (p *profiler) Start() error {
p.Lock() p.mu.Lock()
defer p.Unlock() defer p.mu.Unlock()
if p.running { if p.running {
return nil return nil
@@ -86,8 +86,8 @@ func (p *profiler) Start() error {
} }
func (p *profiler) Stop() error { func (p *profiler) Stop() error {
p.Lock() p.mu.Lock()
defer p.Unlock() defer p.mu.Unlock()
select { select {
case <-p.exit: case <-p.exit:

View File

@@ -33,7 +33,7 @@ type memory struct {
records map[string]services records map[string]services
watchers map[string]*watcher watchers map[string]*watcher
opts register.Options opts register.Options
sync.RWMutex mu sync.RWMutex
} }
// services is a KV map with service name as the key and a map of records as the value // services is a KV map with service name as the key and a map of records as the value
@@ -57,7 +57,7 @@ func (m *memory) ttlPrune() {
defer prune.Stop() defer prune.Stop()
for range prune.C { for range prune.C {
m.Lock() m.mu.Lock()
for namespace, services := range m.records { for namespace, services := range m.records {
for service, versions := range services { for service, versions := range services {
for version, record := range versions { for version, record := range versions {
@@ -72,24 +72,24 @@ func (m *memory) ttlPrune() {
} }
} }
} }
m.Unlock() m.mu.Unlock()
} }
} }
func (m *memory) sendEvent(r *register.Result) { func (m *memory) sendEvent(r *register.Result) {
m.RLock() m.mu.RLock()
watchers := make([]*watcher, 0, len(m.watchers)) watchers := make([]*watcher, 0, len(m.watchers))
for _, w := range m.watchers { for _, w := range m.watchers {
watchers = append(watchers, w) watchers = append(watchers, w)
} }
m.RUnlock() m.mu.RUnlock()
for _, w := range watchers { for _, w := range watchers {
select { select {
case <-w.exit: case <-w.exit:
m.Lock() m.mu.Lock()
delete(m.watchers, w.id) delete(m.watchers, w.id)
m.Unlock() m.mu.Unlock()
default: default:
select { select {
case w.res <- r: case w.res <- r:
@@ -113,8 +113,8 @@ func (m *memory) Init(opts ...register.Option) error {
} }
// add services // add services
m.Lock() m.mu.Lock()
defer m.Unlock() defer m.mu.Unlock()
return nil return nil
} }
@@ -124,8 +124,8 @@ func (m *memory) Options() register.Options {
} }
func (m *memory) Register(_ context.Context, s *register.Service, opts ...register.RegisterOption) error { func (m *memory) Register(_ context.Context, s *register.Service, opts ...register.RegisterOption) error {
m.Lock() m.mu.Lock()
defer m.Unlock() defer m.mu.Unlock()
options := register.NewRegisterOptions(opts...) options := register.NewRegisterOptions(opts...)
@@ -197,8 +197,8 @@ func (m *memory) Register(_ context.Context, s *register.Service, opts ...regist
} }
func (m *memory) Deregister(ctx context.Context, s *register.Service, opts ...register.DeregisterOption) error { func (m *memory) Deregister(ctx context.Context, s *register.Service, opts ...register.DeregisterOption) error {
m.Lock() m.mu.Lock()
defer m.Unlock() defer m.mu.Unlock()
options := register.NewDeregisterOptions(opts...) options := register.NewDeregisterOptions(opts...)
@@ -264,9 +264,9 @@ func (m *memory) LookupService(ctx context.Context, name string, opts ...registe
// if it's a wildcard domain, return from all domains // if it's a wildcard domain, return from all domains
if options.Namespace == register.WildcardNamespace { if options.Namespace == register.WildcardNamespace {
m.RLock() m.mu.RLock()
recs := m.records recs := m.records
m.RUnlock() m.mu.RUnlock()
var services []*register.Service var services []*register.Service
@@ -286,8 +286,8 @@ func (m *memory) LookupService(ctx context.Context, name string, opts ...registe
return services, nil return services, nil
} }
m.RLock() m.mu.RLock()
defer m.RUnlock() defer m.mu.RUnlock()
// check the domain exists // check the domain exists
services, ok := m.records[options.Namespace] services, ok := m.records[options.Namespace]
@@ -319,9 +319,9 @@ func (m *memory) ListServices(ctx context.Context, opts ...register.ListOption)
// if it's a wildcard domain, list from all domains // if it's a wildcard domain, list from all domains
if options.Namespace == register.WildcardNamespace { if options.Namespace == register.WildcardNamespace {
m.RLock() m.mu.RLock()
recs := m.records recs := m.records
m.RUnlock() m.mu.RUnlock()
var services []*register.Service var services []*register.Service
@@ -336,8 +336,8 @@ func (m *memory) ListServices(ctx context.Context, opts ...register.ListOption)
return services, nil return services, nil
} }
m.RLock() m.mu.RLock()
defer m.RUnlock() defer m.mu.RUnlock()
// ensure the domain exists // ensure the domain exists
services, ok := m.records[options.Namespace] services, ok := m.records[options.Namespace]
@@ -371,9 +371,9 @@ func (m *memory) Watch(ctx context.Context, opts ...register.WatchOption) (regis
wo: wo, wo: wo,
} }
m.Lock() m.mu.Lock()
m.watchers[w.id] = w m.watchers[w.id] = w
m.Unlock() m.mu.Unlock()
return w, nil return w, nil
} }

View File

@@ -69,7 +69,8 @@ type Service struct {
type Node struct { type Node struct {
Metadata metadata.Metadata `json:"metadata,omitempty"` Metadata metadata.Metadata `json:"metadata,omitempty"`
ID string `json:"id,omitempty"` ID string `json:"id,omitempty"`
Address string `json:"address,omitempty"` // Address also prefixed with scheme like grpc://xx.xx.xx.xx:1234
Address string `json:"address,omitempty"`
} }
// Option func signature // Option func signature

View File

@@ -6,7 +6,6 @@ import (
"sync" "sync"
"time" "time"
"go.unistack.org/micro/v4/codec"
"go.unistack.org/micro/v4/logger" "go.unistack.org/micro/v4/logger"
"go.unistack.org/micro/v4/register" "go.unistack.org/micro/v4/register"
maddr "go.unistack.org/micro/v4/util/addr" maddr "go.unistack.org/micro/v4/util/addr"
@@ -14,11 +13,6 @@ import (
"go.unistack.org/micro/v4/util/rand" "go.unistack.org/micro/v4/util/rand"
) )
// DefaultCodecs will be used to encode/decode
var DefaultCodecs = map[string]codec.Codec{
"application/octet-stream": codec.NewCodec(),
}
type rpcHandler struct { type rpcHandler struct {
opts HandlerOptions opts HandlerOptions
handler interface{} handler interface{}
@@ -51,13 +45,13 @@ func (r *rpcHandler) Options() HandlerOptions {
} }
type noopServer struct { type noopServer struct {
h Handler h Handler
wg *sync.WaitGroup wg *sync.WaitGroup
rsvc *register.Service rsvc *register.Service
handlers map[string]Handler handlers map[string]Handler
exit chan chan error exit chan chan error
opts Options opts Options
sync.RWMutex mu sync.RWMutex
registered bool registered bool
started bool started bool
} }
@@ -125,10 +119,10 @@ func (n *noopServer) String() string {
//nolint:gocyclo //nolint:gocyclo
func (n *noopServer) Register() error { func (n *noopServer) Register() error {
n.RLock() n.mu.RLock()
rsvc := n.rsvc rsvc := n.rsvc
config := n.opts config := n.opts
n.RUnlock() n.mu.RUnlock()
// if service already filled, reuse it and return early // if service already filled, reuse it and return early
if rsvc != nil { if rsvc != nil {
@@ -144,9 +138,9 @@ func (n *noopServer) Register() error {
return err return err
} }
n.RLock() n.mu.RLock()
registered := n.registered registered := n.registered
n.RUnlock() n.mu.RUnlock()
if !registered { if !registered {
if config.Logger.V(logger.InfoLevel) { if config.Logger.V(logger.InfoLevel) {
@@ -164,8 +158,8 @@ func (n *noopServer) Register() error {
return nil return nil
} }
n.Lock() n.mu.Lock()
defer n.Unlock() defer n.mu.Unlock()
n.registered = true n.registered = true
if cacheService { if cacheService {
@@ -178,9 +172,9 @@ func (n *noopServer) Register() error {
func (n *noopServer) Deregister() error { func (n *noopServer) Deregister() error {
var err error var err error
n.RLock() n.mu.RLock()
config := n.opts config := n.opts
n.RUnlock() n.mu.RUnlock()
service, err := NewRegisterService(n) service, err := NewRegisterService(n)
if err != nil { if err != nil {
@@ -195,29 +189,29 @@ func (n *noopServer) Deregister() error {
return err return err
} }
n.Lock() n.mu.Lock()
n.rsvc = nil n.rsvc = nil
if !n.registered { if !n.registered {
n.Unlock() n.mu.Unlock()
return nil return nil
} }
n.registered = false n.registered = false
n.Unlock() n.mu.Unlock()
return nil return nil
} }
//nolint:gocyclo //nolint:gocyclo
func (n *noopServer) Start() error { func (n *noopServer) Start() error {
n.RLock() n.mu.RLock()
if n.started { if n.started {
n.RUnlock() n.mu.RUnlock()
return nil return nil
} }
config := n.Options() config := n.Options()
n.RUnlock() n.mu.RUnlock()
// use 127.0.0.1 to avoid scan of all network interfaces // use 127.0.0.1 to avoid scan of all network interfaces
addr, err := maddr.Extract("127.0.0.1") addr, err := maddr.Extract("127.0.0.1")
@@ -235,11 +229,11 @@ func (n *noopServer) Start() error {
config.Logger.Info(n.opts.Context, "server [noop] Listening on "+config.Address) config.Logger.Info(n.opts.Context, "server [noop] Listening on "+config.Address)
} }
n.Lock() n.mu.Lock()
if len(config.Advertise) == 0 { if len(config.Advertise) == 0 {
config.Advertise = config.Address config.Advertise = config.Address
} }
n.Unlock() n.mu.Unlock()
// use RegisterCheck func before register // use RegisterCheck func before register
// nolint: nestif // nolint: nestif
@@ -273,9 +267,9 @@ func (n *noopServer) Start() error {
select { select {
// register self on interval // register self on interval
case <-t.C: case <-t.C:
n.RLock() n.mu.RLock()
registered := n.registered registered := n.registered
n.RUnlock() n.mu.RUnlock()
rerr := config.RegisterCheck(config.Context) rerr := config.RegisterCheck(config.Context)
// nolint: nestif // nolint: nestif
if rerr != nil && registered { if rerr != nil && registered {
@@ -332,29 +326,29 @@ func (n *noopServer) Start() error {
}() }()
// mark the server as started // mark the server as started
n.Lock() n.mu.Lock()
n.started = true n.started = true
n.Unlock() n.mu.Unlock()
return nil return nil
} }
func (n *noopServer) Stop() error { func (n *noopServer) Stop() error {
n.RLock() n.mu.RLock()
if !n.started { if !n.started {
n.RUnlock() n.mu.RUnlock()
return nil return nil
} }
n.RUnlock() n.mu.RUnlock()
ch := make(chan error) ch := make(chan error)
n.exit <- ch n.exit <- ch
err := <-ch err := <-ch
n.Lock() n.mu.Lock()
n.rsvc = nil n.rsvc = nil
n.started = false n.started = false
n.Unlock() n.mu.Unlock()
return err return err
} }

View File

@@ -8,7 +8,6 @@ import (
"time" "time"
"github.com/KimMachineGun/automemlimit/memlimit" "github.com/KimMachineGun/automemlimit/memlimit"
"go.uber.org/automaxprocs/maxprocs"
"go.unistack.org/micro/v4/broker" "go.unistack.org/micro/v4/broker"
"go.unistack.org/micro/v4/client" "go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/config" "go.unistack.org/micro/v4/config"
@@ -23,8 +22,8 @@ import (
) )
func init() { func init() {
_, _ = maxprocs.Set()
_, _ = memlimit.SetGoMemLimitWithOpts( _, _ = memlimit.SetGoMemLimitWithOpts(
memlimit.WithRefreshInterval(1*time.Minute),
memlimit.WithRatio(0.9), memlimit.WithRatio(0.9),
memlimit.WithProvider( memlimit.WithProvider(
memlimit.ApplyFallback( memlimit.ApplyFallback(
@@ -96,9 +95,10 @@ func RegisterHandler(s server.Server, h interface{}, opts ...server.HandlerOptio
} }
type service struct { type service struct {
done chan struct{} done chan struct{}
opts Options opts Options
sync.RWMutex mu sync.RWMutex
stopped bool
} }
// NewService creates and returns a new Service based on the packages within. // NewService creates and returns a new Service based on the packages within.
@@ -320,9 +320,9 @@ func (s *service) Health() bool {
func (s *service) Start() error { func (s *service) Start() error {
var err error var err error
s.RLock() s.mu.RLock()
config := s.opts config := s.opts
s.RUnlock() s.mu.RUnlock()
for _, cfg := range s.opts.Configs { for _, cfg := range s.opts.Configs {
if cfg.Options().Struct == nil { if cfg.Options().Struct == nil {
@@ -379,9 +379,9 @@ func (s *service) Start() error {
} }
func (s *service) Stop() error { func (s *service) Stop() error {
s.RLock() s.mu.RLock()
config := s.opts config := s.opts
s.RUnlock() s.mu.RUnlock()
if config.Loggers[0].V(logger.InfoLevel) { if config.Loggers[0].V(logger.InfoLevel) {
config.Loggers[0].Info(s.opts.Context, fmt.Sprintf("stoppping [service] %s", s.Name())) config.Loggers[0].Info(s.opts.Context, fmt.Sprintf("stoppping [service] %s", s.Name()))
@@ -424,7 +424,7 @@ func (s *service) Stop() error {
} }
} }
close(s.done) s.notifyShutdown()
return nil return nil
} }
@@ -448,10 +448,23 @@ func (s *service) Run() error {
return err return err
} }
// wait on context cancel
<-s.done <-s.done
return s.Stop() return nil
}
// notifyShutdown marks the service as stopped and closes the done channel.
// It ensures the channel is closed only once, preventing multiple closures.
func (s *service) notifyShutdown() {
s.mu.Lock()
if s.stopped {
s.mu.Unlock()
return
}
s.stopped = true
s.mu.Unlock()
close(s.done)
} }
type Namer interface { type Namer interface {

View File

@@ -3,7 +3,9 @@ package micro
import ( import (
"reflect" "reflect"
"testing" "testing"
"time"
"github.com/stretchr/testify/require"
"go.unistack.org/micro/v4/broker" "go.unistack.org/micro/v4/broker"
"go.unistack.org/micro/v4/client" "go.unistack.org/micro/v4/client"
"go.unistack.org/micro/v4/config" "go.unistack.org/micro/v4/config"
@@ -737,3 +739,41 @@ func Test_getNameIndex(t *testing.T) {
} }
} }
*/ */
func TestServiceShutdown(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("service shutdown failed: %v", r)
}
}()
s, ok := NewService().(*service)
require.NotNil(t, s)
require.True(t, ok)
require.NoError(t, s.Start())
require.False(t, s.stopped)
require.NoError(t, s.Stop())
require.True(t, s.stopped)
}
func TestServiceMultipleShutdowns(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("service shutdown failed: %v", r)
}
}()
s := NewService()
go func() {
time.Sleep(10 * time.Millisecond)
// first call
require.NoError(t, s.Stop())
// duplicate call
require.NoError(t, s.Stop())
}()
require.NoError(t, s.Run())
}

View File

@@ -139,7 +139,7 @@ func (n *noopStore) fnExists(ctx context.Context, _ string, _ ...ExistsOption) e
return ctx.Err() return ctx.Err()
default: default:
} }
return nil return ErrNotFound
} }
func (n *noopStore) Write(ctx context.Context, key string, val interface{}, opts ...WriteOption) error { func (n *noopStore) Write(ctx context.Context, key string, val interface{}, opts ...WriteOption) error {

View File

@@ -2,6 +2,7 @@ package store
import ( import (
"context" "context"
"errors"
"testing" "testing"
) )
@@ -25,7 +26,8 @@ func TestHook(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
if err := s.Exists(context.TODO(), "test"); err != nil { err := s.Exists(context.TODO(), "test")
if !errors.Is(err, ErrNotFound) {
t.Fatal(err) t.Fatal(err)
} }

View File

@@ -9,7 +9,7 @@ type memorySync struct {
locks map[string]*memoryLock locks map[string]*memoryLock
options Options options Options
mtx gosync.RWMutex mu gosync.RWMutex
} }
type memoryLock struct { type memoryLock struct {
@@ -74,7 +74,7 @@ func (m *memorySync) Options() Options {
func (m *memorySync) Lock(id string, opts ...LockOption) error { func (m *memorySync) Lock(id string, opts ...LockOption) error {
// lock our access // lock our access
m.mtx.Lock() m.mu.Lock()
var options LockOptions var options LockOptions
for _, o := range opts { for _, o := range opts {
@@ -90,11 +90,11 @@ func (m *memorySync) Lock(id string, opts ...LockOption) error {
release: make(chan bool), release: make(chan bool),
} }
// unlock // unlock
m.mtx.Unlock() m.mu.Unlock()
return nil return nil
} }
m.mtx.Unlock() m.mu.Unlock()
// set wait time // set wait time
var wait <-chan time.Time var wait <-chan time.Time
@@ -124,12 +124,12 @@ lockLoop:
// wait for the lock to be released // wait for the lock to be released
select { select {
case <-lk.release: case <-lk.release:
m.mtx.Lock() m.mu.Lock()
// someone locked before us // someone locked before us
lk, ok = m.locks[id] lk, ok = m.locks[id]
if ok { if ok {
m.mtx.Unlock() m.mu.Unlock()
continue continue
} }
@@ -141,7 +141,7 @@ lockLoop:
release: make(chan bool), release: make(chan bool),
} }
m.mtx.Unlock() m.mu.Unlock()
break lockLoop break lockLoop
case <-ttl: case <-ttl:
@@ -160,8 +160,8 @@ lockLoop:
} }
func (m *memorySync) Unlock(id string) error { func (m *memorySync) Unlock(id string) error {
m.mtx.Lock() m.mu.Lock()
defer m.mtx.Unlock() defer m.mu.Unlock()
lk, ok := m.locks[id] lk, ok := m.locks[id]
// no lock exists // no lock exists

View File

@@ -46,6 +46,10 @@ func (s memoryStringer) String() string {
return s.s return s.s
} }
func (t *Tracer) Enabled() bool {
return t.opts.Enabled
}
func (t *Tracer) Flush(_ context.Context) error { func (t *Tracer) Flush(_ context.Context) error {
return nil return nil
} }
@@ -89,6 +93,10 @@ func (s *Span) Tracer() tracer.Tracer {
return s.tracer return s.tracer
} }
func (s *Span) IsRecording() bool {
return true
}
type Event struct { type Event struct {
name string name string
labels []interface{} labels []interface{}

View File

@@ -4,7 +4,7 @@ import (
"context" "context"
"time" "time"
"go.unistack.org/micro/v4/util/id" "github.com/google/uuid"
) )
var _ Tracer = (*noopTracer)(nil) var _ Tracer = (*noopTracer)(nil)
@@ -18,6 +18,12 @@ func (t *noopTracer) Spans() []Span {
return t.spans return t.spans
} }
var uuidNil = uuid.Nil.String()
func (t *noopTracer) Enabled() bool {
return t.opts.Enabled
}
func (t *noopTracer) Start(ctx context.Context, name string, opts ...SpanOption) (context.Context, Span) { func (t *noopTracer) Start(ctx context.Context, name string, opts ...SpanOption) (context.Context, Span) {
options := NewSpanOptions(opts...) options := NewSpanOptions(opts...)
span := &noopSpan{ span := &noopSpan{
@@ -28,8 +34,8 @@ func (t *noopTracer) Start(ctx context.Context, name string, opts ...SpanOption)
labels: options.Labels, labels: options.Labels,
kind: options.Kind, kind: options.Kind,
} }
span.spanID.s, _ = id.New() span.spanID.s = uuidNil
span.traceID.s, _ = id.New() span.traceID.s = uuidNil
if span.ctx == nil { if span.ctx == nil {
span.ctx = context.Background() span.ctx = context.Background()
} }
@@ -120,6 +126,10 @@ func (s *noopSpan) SetStatus(st SpanStatus, msg string) {
s.statusMsg = msg s.statusMsg = msg
} }
func (s *noopSpan) IsRecording() bool {
return false
}
// NewTracer returns new memory tracer // NewTracer returns new memory tracer
func NewTracer(opts ...Option) Tracer { func NewTracer(opts ...Option) Tracer {
return &noopTracer{ return &noopTracer{

View File

@@ -142,6 +142,8 @@ type Options struct {
Name string Name string
// ContextAttrFuncs contains funcs that provides tracing // ContextAttrFuncs contains funcs that provides tracing
ContextAttrFuncs []ContextAttrFunc ContextAttrFuncs []ContextAttrFunc
// Enabled specify trace status
Enabled bool
} }
// Option func signature // Option func signature
@@ -181,6 +183,7 @@ func NewOptions(opts ...Option) Options {
Logger: logger.DefaultLogger, Logger: logger.DefaultLogger,
Context: context.Background(), Context: context.Background(),
ContextAttrFuncs: DefaultContextAttrFuncs, ContextAttrFuncs: DefaultContextAttrFuncs,
Enabled: true,
} }
for _, o := range opts { for _, o := range opts {
o(&options) o(&options)
@@ -194,3 +197,10 @@ func Name(n string) Option {
o.Name = n o.Name = n
} }
} }
// Disabled disable tracer
func Disabled(b bool) Option {
return func(o *Options) {
o.Enabled = !b
}
}

View File

@@ -29,10 +29,10 @@ type ContextAttrFunc func(ctx context.Context) []interface{}
func init() { func init() {
logger.DefaultContextAttrFuncs = append(logger.DefaultContextAttrFuncs, logger.DefaultContextAttrFuncs = append(logger.DefaultContextAttrFuncs,
func(ctx context.Context) []interface{} { func(ctx context.Context) []interface{} {
if span, ok := SpanFromContext(ctx); ok { if sp, ok := SpanFromContext(ctx); ok && sp != nil && sp.IsRecording() {
return []interface{}{ return []interface{}{
TraceIDKey, span.TraceID(), TraceIDKey, sp.TraceID(),
SpanIDKey, span.SpanID(), SpanIDKey, sp.SpanID(),
} }
} }
return nil return nil
@@ -51,6 +51,8 @@ type Tracer interface {
// Extract(ctx context.Context) // Extract(ctx context.Context)
// Flush flushes spans // Flush flushes spans
Flush(ctx context.Context) error Flush(ctx context.Context) error
// Enabled returns tracer status
Enabled() bool
} }
type Span interface { type Span interface {
@@ -78,4 +80,6 @@ type Span interface {
TraceID() string TraceID() string
// SpanID returns span id // SpanID returns span id
SpanID() string SpanID() string
// IsRecording returns the recording state of the Span.
IsRecording() bool
} }

View File

@@ -1,13 +1,16 @@
package buffer package buffer
import "io" import (
"fmt"
"io"
)
var _ interface { var _ interface {
io.ReadCloser io.ReadCloser
io.ReadSeeker io.ReadSeeker
} = (*SeekerBuffer)(nil) } = (*SeekerBuffer)(nil)
// Buffer is a ReadWriteCloser that supports seeking. It's intended to // SeekerBuffer is a ReadWriteCloser that supports seeking. It's intended to
// replicate the functionality of bytes.Buffer that I use in my projects. // replicate the functionality of bytes.Buffer that I use in my projects.
// //
// Note that the seeking is limited to the read marker; all writes are // Note that the seeking is limited to the read marker; all writes are
@@ -23,6 +26,7 @@ func NewSeekerBuffer(data []byte) *SeekerBuffer {
} }
} }
// Read reads up to len(p) bytes into p from the current read position.
func (b *SeekerBuffer) Read(p []byte) (int, error) { func (b *SeekerBuffer) Read(p []byte) (int, error) {
if b.pos >= int64(len(b.data)) { if b.pos >= int64(len(b.data)) {
return 0, io.EOF return 0, io.EOF
@@ -30,29 +34,51 @@ func (b *SeekerBuffer) Read(p []byte) (int, error) {
n := copy(p, b.data[b.pos:]) n := copy(p, b.data[b.pos:])
b.pos += int64(n) b.pos += int64(n)
return n, nil return n, nil
} }
// Write appends the contents of p to the end of the buffer. It does not affect the read position.
func (b *SeekerBuffer) Write(p []byte) (int, error) { func (b *SeekerBuffer) Write(p []byte) (int, error) {
if len(p) == 0 {
return 0, nil
}
b.data = append(b.data, p...) b.data = append(b.data, p...)
return len(p), nil return len(p), nil
} }
// Seek sets the read pointer to pos. // Seek sets the offset for the next Read operation.
// The offset is interpreted according to whence:
// - io.SeekStart: relative to the beginning of the buffer
// - io.SeekCurrent: relative to the current position
// - io.SeekEnd: relative to the end of the buffer
//
// Returns an error if the resulting position is negative or if whence is invalid.
func (b *SeekerBuffer) Seek(offset int64, whence int) (int64, error) { func (b *SeekerBuffer) Seek(offset int64, whence int) (int64, error) {
var newPos int64
switch whence { switch whence {
case io.SeekStart: case io.SeekStart:
b.pos = offset newPos = offset
case io.SeekEnd: case io.SeekEnd:
b.pos = int64(len(b.data)) + offset newPos = int64(len(b.data)) + offset
case io.SeekCurrent: case io.SeekCurrent:
b.pos += offset newPos = b.pos + offset
default:
return 0, fmt.Errorf("invalid whence: %d", whence)
} }
if newPos < 0 {
return 0, fmt.Errorf("invalid seek: resulting position %d is negative", newPos)
}
b.pos = newPos
return b.pos, nil return b.pos, nil
} }
// Rewind resets the read pointer to 0. // Rewind resets the read position to 0.
func (b *SeekerBuffer) Rewind() error { func (b *SeekerBuffer) Rewind() error {
if _, err := b.Seek(0, io.SeekStart); err != nil { if _, err := b.Seek(0, io.SeekStart); err != nil {
return err return err
@@ -67,12 +93,24 @@ func (b *SeekerBuffer) Close() error {
return nil return nil
} }
// Reset clears all the data out of the buffer and sets the read position to 0.
func (b *SeekerBuffer) Reset() {
b.data = nil
b.pos = 0
}
// Len returns the length of data remaining to be read. // Len returns the length of data remaining to be read.
func (b *SeekerBuffer) Len() int { func (b *SeekerBuffer) Len() int {
if b.pos >= int64(len(b.data)) {
return 0
}
return len(b.data[b.pos:]) return len(b.data[b.pos:])
} }
// Bytes returns the underlying bytes from the current position. // Bytes returns the underlying bytes from the current position.
func (b *SeekerBuffer) Bytes() []byte { func (b *SeekerBuffer) Bytes() []byte {
if b.pos >= int64(len(b.data)) {
return []byte{}
}
return b.data[b.pos:] return b.data[b.pos:]
} }

View File

@@ -2,54 +2,384 @@ package buffer
import ( import (
"fmt" "fmt"
"strings" "io"
"testing" "testing"
"github.com/stretchr/testify/require"
) )
func noErrorT(t *testing.T, err error) { func TestNewSeekerBuffer(t *testing.T) {
if nil != err { input := []byte{'a', 'b', 'c', 'd', 'e'}
t.Fatalf("%s", err) expected := &SeekerBuffer{data: []byte{'a', 'b', 'c', 'd', 'e'}, pos: 0}
require.Equal(t, expected, NewSeekerBuffer(input))
}
func TestSeekerBuffer_Read(t *testing.T) {
tests := []struct {
name string
data []byte
initPos int64
readBuf []byte
expectedN int
expectedData []byte
expectedErr error
expectedPos int64
}{
{
name: "read with empty buffer",
data: []byte("hello"),
initPos: 0,
readBuf: []byte{},
expectedN: 0,
expectedData: []byte{},
expectedErr: nil,
expectedPos: 0,
},
{
name: "read with nil buffer",
data: []byte("hello"),
initPos: 0,
readBuf: nil,
expectedN: 0,
expectedData: nil,
expectedErr: nil,
expectedPos: 0,
},
{
name: "read full buffer",
data: []byte("hello"),
initPos: 0,
readBuf: make([]byte, 5),
expectedN: 5,
expectedData: []byte("hello"),
expectedErr: nil,
expectedPos: 5,
},
{
name: "read partial buffer",
data: []byte("hello"),
initPos: 2,
readBuf: make([]byte, 2),
expectedN: 2,
expectedData: []byte("ll"),
expectedErr: nil,
expectedPos: 4,
},
{
name: "read after end",
data: []byte("hello"),
initPos: 5,
readBuf: make([]byte, 5),
expectedN: 0,
expectedData: make([]byte, 5),
expectedErr: io.EOF,
expectedPos: 5,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sb := NewSeekerBuffer(tt.data)
sb.pos = tt.initPos
n, err := sb.Read(tt.readBuf)
if tt.expectedErr != nil {
require.Equal(t, err, tt.expectedErr)
} else {
require.NoError(t, err)
}
require.Equal(t, tt.expectedN, n)
require.Equal(t, tt.expectedData, tt.readBuf)
require.Equal(t, tt.expectedPos, sb.pos)
})
} }
} }
func boolT(t *testing.T, cond bool, s ...string) { func TestSeekerBuffer_Write(t *testing.T) {
if !cond { tests := []struct {
what := strings.Join(s, ", ") name string
if len(what) > 0 { initialData []byte
what = ": " + what initialPos int64
} writeData []byte
t.Fatalf("assert.Bool failed%s", what) expectedData []byte
expectedN int
}{
{
name: "write empty slice",
initialData: []byte("data"),
initialPos: 0,
writeData: []byte{},
expectedData: []byte("data"),
expectedN: 0,
},
{
name: "write nil slice",
initialData: []byte("data"),
initialPos: 0,
writeData: nil,
expectedData: []byte("data"),
expectedN: 0,
},
{
name: "write to empty buffer",
initialData: nil,
initialPos: 0,
writeData: []byte("abc"),
expectedData: []byte("abc"),
expectedN: 3,
},
{
name: "write to existing buffer",
initialData: []byte("hello"),
initialPos: 0,
writeData: []byte(" world"),
expectedData: []byte("hello world"),
expectedN: 6,
},
{
name: "write after read",
initialData: []byte("abc"),
initialPos: 2,
writeData: []byte("XYZ"),
expectedData: []byte("abcXYZ"),
expectedN: 3,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sb := NewSeekerBuffer(tt.initialData)
sb.pos = tt.initialPos
n, err := sb.Write(tt.writeData)
require.NoError(t, err)
require.Equal(t, tt.expectedN, n)
require.Equal(t, tt.expectedData, sb.data)
require.Equal(t, tt.initialPos, sb.pos)
})
} }
} }
func TestSeeking(t *testing.T) { func TestSeekerBuffer_Seek(t *testing.T) {
partA := []byte("hello, ") tests := []struct {
partB := []byte("world!") name string
initialData []byte
initialPos int64
offset int64
whence int
expectedPos int64
expectedErr error
}{
{
name: "seek with invalid whence",
initialData: []byte("abcdef"),
initialPos: 0,
offset: 1,
whence: 12345,
expectedPos: 0,
expectedErr: fmt.Errorf("invalid whence: %d", 12345),
},
{
name: "seek negative from start",
initialData: []byte("abcdef"),
initialPos: 0,
offset: -1,
whence: io.SeekStart,
expectedPos: 0,
expectedErr: fmt.Errorf("invalid seek: resulting position %d is negative", -1),
},
{
name: "seek from start to 0",
initialData: []byte("abcdef"),
initialPos: 0,
offset: 0,
whence: io.SeekStart,
expectedPos: 0,
expectedErr: nil,
},
{
name: "seek from start to 3",
initialData: []byte("abcdef"),
initialPos: 0,
offset: 3,
whence: io.SeekStart,
expectedPos: 3,
expectedErr: nil,
},
{
name: "seek from end to -1 (last byte)",
initialData: []byte("abcdef"),
initialPos: 0,
offset: -1,
whence: io.SeekEnd,
expectedPos: 5,
expectedErr: nil,
},
{
name: "seek from current forward",
initialData: []byte("abcdef"),
initialPos: 2,
offset: 2,
whence: io.SeekCurrent,
expectedPos: 4,
expectedErr: nil,
},
{
name: "seek from current backward",
initialData: []byte("abcdef"),
initialPos: 4,
offset: -2,
whence: io.SeekCurrent,
expectedPos: 2,
expectedErr: nil,
},
{
name: "seek to end exactly",
initialData: []byte("abcdef"),
initialPos: 0,
offset: 0,
whence: io.SeekEnd,
expectedPos: 6,
expectedErr: nil,
},
{
name: "seek to out of range",
initialData: []byte("abcdef"),
initialPos: 0,
offset: 2,
whence: io.SeekEnd,
expectedPos: 8,
expectedErr: nil,
},
}
buf := NewSeekerBuffer(partA) for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sb := NewSeekerBuffer(tt.initialData)
sb.pos = tt.initialPos
boolT(t, buf.Len() == len(partA), fmt.Sprintf("on init: have length %d, want length %d", buf.Len(), len(partA))) newPos, err := sb.Seek(tt.offset, tt.whence)
b := make([]byte, 32) if tt.expectedErr != nil {
require.Equal(t, tt.expectedErr, err)
n, err := buf.Read(b) } else {
noErrorT(t, err) require.NoError(t, err)
boolT(t, buf.Len() == 0, fmt.Sprintf("after reading 1: have length %d, want length 0", buf.Len())) require.Equal(t, tt.expectedPos, newPos)
boolT(t, n == len(partA), fmt.Sprintf("after reading 2: have length %d, want length %d", n, len(partA))) require.Equal(t, tt.expectedPos, sb.pos)
}
n, err = buf.Write(partB) })
noErrorT(t, err) }
boolT(t, n == len(partB), fmt.Sprintf("after writing: have length %d, want length %d", n, len(partB))) }
n, err = buf.Read(b) func TestSeekerBuffer_Rewind(t *testing.T) {
noErrorT(t, err) buf := NewSeekerBuffer([]byte("hello world"))
boolT(t, buf.Len() == 0, fmt.Sprintf("after rereading 1: have length %d, want length 0", buf.Len())) buf.pos = 4
boolT(t, n == len(partB), fmt.Sprintf("after rereading 2: have length %d, want length %d", n, len(partB)))
require.NoError(t, buf.Rewind())
partsLen := len(partA) + len(partB) require.Equal(t, []byte("hello world"), buf.data)
_ = buf.Rewind() require.Equal(t, int64(0), buf.pos)
boolT(t, buf.Len() == partsLen, fmt.Sprintf("after rewinding: have length %d, want length %d", buf.Len(), partsLen)) }
buf.Close() func TestSeekerBuffer_Close(t *testing.T) {
boolT(t, buf.Len() == 0, fmt.Sprintf("after closing, have length %d, want length 0", buf.Len())) buf := NewSeekerBuffer([]byte("hello world"))
buf.pos = 2
require.NoError(t, buf.Close())
require.Nil(t, buf.data)
require.Equal(t, int64(0), buf.pos)
}
func TestSeekerBuffer_Reset(t *testing.T) {
buf := NewSeekerBuffer([]byte("hello world"))
buf.pos = 2
buf.Reset()
require.Nil(t, buf.data)
require.Equal(t, int64(0), buf.pos)
}
func TestSeekerBuffer_Len(t *testing.T) {
tests := []struct {
name string
data []byte
pos int64
expected int
}{
{
name: "full buffer",
data: []byte("abcde"),
pos: 0,
expected: 5,
},
{
name: "partial read",
data: []byte("abcde"),
pos: 2,
expected: 3,
},
{
name: "fully read",
data: []byte("abcde"),
pos: 5,
expected: 0,
},
{
name: "pos > len",
data: []byte("abcde"),
pos: 10,
expected: 0,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
buf := NewSeekerBuffer(tt.data)
buf.pos = tt.pos
require.Equal(t, tt.expected, buf.Len())
})
}
}
func TestSeekerBuffer_Bytes(t *testing.T) {
tests := []struct {
name string
data []byte
pos int64
expected []byte
}{
{
name: "start of buffer",
data: []byte("abcde"),
pos: 0,
expected: []byte("abcde"),
},
{
name: "middle of buffer",
data: []byte("abcde"),
pos: 2,
expected: []byte("cde"),
},
{
name: "end of buffer",
data: []byte("abcde"),
pos: 5,
expected: []byte{},
},
{
name: "pos beyond end",
data: []byte("abcde"),
pos: 10,
expected: []byte{},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
buf := NewSeekerBuffer(tt.data)
buf.pos = tt.pos
require.Equal(t, tt.expected, buf.Bytes())
})
}
} }

View File

@@ -137,7 +137,7 @@ type cache struct {
opts Options opts Options
sync.RWMutex mu sync.RWMutex
} }
type cacheEntry struct { type cacheEntry struct {
@@ -171,7 +171,7 @@ func (c *cache) put(req string, res string) {
ttl = c.opts.MaxCacheTTL ttl = c.opts.MaxCacheTTL
} }
c.Lock() c.mu.Lock()
if c.entries == nil { if c.entries == nil {
c.entries = make(map[string]cacheEntry) c.entries = make(map[string]cacheEntry)
} }
@@ -207,7 +207,7 @@ func (c *cache) put(req string, res string) {
} }
c.opts.Meter.Counter(semconv.CacheItemsTotal, "type", "dns").Inc() c.opts.Meter.Counter(semconv.CacheItemsTotal, "type", "dns").Inc()
c.Unlock() c.mu.Unlock()
} }
func (c *cache) get(req string) (res string) { func (c *cache) get(req string) (res string) {
@@ -219,8 +219,8 @@ func (c *cache) get(req string) (res string) {
return "" return ""
} }
c.RLock() c.mu.RLock()
defer c.RUnlock() defer c.mu.RUnlock()
if c.entries == nil { if c.entries == nil {
return "" return ""

View File

@@ -20,7 +20,7 @@ type dnsConn struct {
ibuf bytes.Buffer ibuf bytes.Buffer
obuf bytes.Buffer obuf bytes.Buffer
sync.Mutex mu sync.Mutex
} }
type roundTripper func(ctx context.Context, req string) (res string, err error) type roundTripper func(ctx context.Context, req string) (res string, err error)
@@ -42,15 +42,15 @@ func (c *dnsConn) Read(b []byte) (n int, err error) {
} }
func (c *dnsConn) Write(b []byte) (n int, err error) { func (c *dnsConn) Write(b []byte) (n int, err error) {
c.Lock() c.mu.Lock()
defer c.Unlock() defer c.mu.Unlock()
return c.ibuf.Write(b) return c.ibuf.Write(b)
} }
func (c *dnsConn) Close() error { func (c *dnsConn) Close() error {
c.Lock() c.mu.Lock()
cancel := c.cancel cancel := c.cancel
c.Unlock() c.mu.Unlock()
if cancel != nil { if cancel != nil {
cancel() cancel()
@@ -78,9 +78,9 @@ func (c *dnsConn) SetDeadline(t time.Time) error {
} }
func (c *dnsConn) SetReadDeadline(t time.Time) error { func (c *dnsConn) SetReadDeadline(t time.Time) error {
c.Lock() c.mu.Lock()
c.deadline = t c.deadline = t
c.Unlock() c.mu.Unlock()
return nil return nil
} }
@@ -90,8 +90,8 @@ func (c *dnsConn) SetWriteDeadline(_ time.Time) error {
} }
func (c *dnsConn) drainBuffers(b []byte) (string, int, error) { func (c *dnsConn) drainBuffers(b []byte) (string, int, error) {
c.Lock() c.mu.Lock()
defer c.Unlock() defer c.mu.Unlock()
// drain the output buffer // drain the output buffer
if c.obuf.Len() > 0 { if c.obuf.Len() > 0 {
@@ -119,8 +119,8 @@ func (c *dnsConn) drainBuffers(b []byte) (string, int, error) {
} }
func (c *dnsConn) fillBuffer(b []byte, str string) (int, error) { func (c *dnsConn) fillBuffer(b []byte, str string) (int, error) {
c.Lock() c.mu.Lock()
defer c.Unlock() defer c.mu.Unlock()
c.obuf.WriteByte(byte(len(str) >> 8)) c.obuf.WriteByte(byte(len(str) >> 8))
c.obuf.WriteByte(byte(len(str))) c.obuf.WriteByte(byte(len(str)))
c.obuf.WriteString(str) c.obuf.WriteString(str)
@@ -128,8 +128,8 @@ func (c *dnsConn) fillBuffer(b []byte, str string) (int, error) {
} }
func (c *dnsConn) childContext() (context.Context, context.CancelFunc) { func (c *dnsConn) childContext() (context.Context, context.CancelFunc) {
c.Lock() c.mu.Lock()
defer c.Unlock() defer c.mu.Unlock()
if c.ctx == nil { if c.ctx == nil {
c.ctx, c.cancel = context.WithCancel(context.Background()) c.ctx, c.cancel = context.WithCancel(context.Background())
} }

View File

@@ -52,7 +52,7 @@ type clientTracer struct {
tr tracer.Tracer tr tracer.Tracer
activeHooks map[string]context.Context activeHooks map[string]context.Context
root tracer.Span root tracer.Span
mtx sync.Mutex mu sync.Mutex
} }
func NewClientTrace(ctx context.Context, tr tracer.Tracer) *httptrace.ClientTrace { func NewClientTrace(ctx context.Context, tr tracer.Tracer) *httptrace.ClientTrace {
@@ -83,8 +83,8 @@ func NewClientTrace(ctx context.Context, tr tracer.Tracer) *httptrace.ClientTrac
} }
func (ct *clientTracer) start(hook, spanName string, attrs ...interface{}) { func (ct *clientTracer) start(hook, spanName string, attrs ...interface{}) {
ct.mtx.Lock() ct.mu.Lock()
defer ct.mtx.Unlock() defer ct.mu.Unlock()
if hookCtx, found := ct.activeHooks[hook]; !found { if hookCtx, found := ct.activeHooks[hook]; !found {
var sp tracer.Span var sp tracer.Span
@@ -104,8 +104,8 @@ func (ct *clientTracer) start(hook, spanName string, attrs ...interface{}) {
} }
func (ct *clientTracer) end(hook string, err error, attrs ...interface{}) { func (ct *clientTracer) end(hook string, err error, attrs ...interface{}) {
ct.mtx.Lock() ct.mu.Lock()
defer ct.mtx.Unlock() defer ct.mu.Unlock()
if ctx, ok := ct.activeHooks[hook]; ok { // nolint:nestif if ctx, ok := ct.activeHooks[hook]; ok { // nolint:nestif
if span, ok := tracer.SpanFromContext(ctx); ok { if span, ok := tracer.SpanFromContext(ctx); ok {
if err != nil { if err != nil {
@@ -136,8 +136,8 @@ func (ct *clientTracer) getParentContext(hook string) context.Context {
} }
func (ct *clientTracer) span(hook string) (tracer.Span, bool) { func (ct *clientTracer) span(hook string) (tracer.Span, bool) {
ct.mtx.Lock() ct.mu.Lock()
defer ct.mtx.Unlock() defer ct.mu.Unlock()
if ctx, ok := ct.activeHooks[hook]; ok { if ctx, ok := ct.activeHooks[hook]; ok {
return tracer.SpanFromContext(ctx) return tracer.SpanFromContext(ctx)
} }

View File

@@ -2,12 +2,8 @@ package id
import ( import (
"crypto/rand" "crypto/rand"
"encoding/binary"
"errors" "errors"
"fmt"
"time"
uuidv8 "github.com/ash3in/uuidv8"
"github.com/google/uuid" "github.com/google/uuid"
nanoid "github.com/matoous/go-nanoid" nanoid "github.com/matoous/go-nanoid"
) )
@@ -25,6 +21,7 @@ type Type int
const ( const (
TypeUnspecified Type = iota TypeUnspecified Type = iota
TypeNanoid TypeNanoid
TypeUUIDv7
TypeUUIDv8 TypeUUIDv8
) )
@@ -58,14 +55,14 @@ func (g *Generator) New() (string, error) {
} }
return nanoid.Generate(g.opts.NanoidAlphabet, g.opts.NanoidSize) return nanoid.Generate(g.opts.NanoidAlphabet, g.opts.NanoidSize)
case TypeUUIDv8: case TypeUUIDv7:
timestamp := uint64(time.Now().UnixNano()) uid, err := uuid.NewV7()
clockSeq := make([]byte, 2) if err != nil {
if _, err := rand.Read(clockSeq); err != nil { return "", err
return "", fmt.Errorf("failed to generate random clock sequence: %w", err)
} }
clockSeqValue := binary.BigEndian.Uint16(clockSeq) & 0x0FFF // Mask to 12 bits return uid.String(), nil
return uuidv8.NewWithParams(timestamp, clockSeqValue, g.opts.UUIDNode[:], uuidv8.TimestampBits48) case TypeUUIDv8:
return "", errors.New("unsupported uuid version v8")
} }
return "", errors.New("invalid option, Type unspecified") return "", errors.New("invalid option, Type unspecified")
} }
@@ -82,16 +79,15 @@ func New(opts ...Option) (string, error) {
if options.NanoidSize <= 0 { if options.NanoidSize <= 0 {
return "", errors.New("invalid option, NanoidSize must be positive integer") return "", errors.New("invalid option, NanoidSize must be positive integer")
} }
return nanoid.Generate(options.NanoidAlphabet, options.NanoidSize) return nanoid.Generate(options.NanoidAlphabet, options.NanoidSize)
case TypeUUIDv8: case TypeUUIDv7:
timestamp := uint64(time.Now().UnixNano()) uid, err := uuid.NewV7()
clockSeq := make([]byte, 2) if err != nil {
if _, err := rand.Read(clockSeq); err != nil { return "", err
return "", fmt.Errorf("failed to generate random clock sequence: %w", err)
} }
clockSeqValue := binary.BigEndian.Uint16(clockSeq) & 0x0FFF // Mask to 12 bits return uid.String(), nil
return uuidv8.NewWithParams(timestamp, clockSeqValue, options.UUIDNode[:], uuidv8.TimestampBits48) case TypeUUIDv8:
return "", errors.New("unsupported uuid version v8")
} }
return "", errors.New("invalid option, Type unspecified") return "", errors.New("invalid option, Type unspecified")
@@ -145,7 +141,7 @@ func WithUUIDNode(node [6]byte) Option {
// NewOptions returns new Options struct filled by opts // NewOptions returns new Options struct filled by opts
func NewOptions(opts ...Option) Options { func NewOptions(opts ...Option) Options {
options := Options{ options := Options{
Type: TypeUUIDv8, Type: TypeUUIDv7,
NanoidAlphabet: DefaultNanoidAlphabet, NanoidAlphabet: DefaultNanoidAlphabet,
NanoidSize: DefaultNanoidSize, NanoidSize: DefaultNanoidSize,
UUIDNode: generatedNode, UUIDNode: generatedNode,

View File

@@ -489,35 +489,74 @@ func URLMap(query string) (map[string]interface{}, error) {
return mp.(map[string]interface{}), nil return mp.(map[string]interface{}), nil
} }
// FlattenMap expand key.subkey to nested map // FlattenMap flattens a nested map into a single-level map using dot notation for nested keys.
func FlattenMap(a map[string]interface{}) map[string]interface{} { // In case of key conflicts, all nested levels will be discarded in favor of the first-level key.
// preprocess map //
nb := make(map[string]interface{}, len(a)) // Example #1:
for k, v := range a { //
ps := strings.Split(k, ".") // Input:
if len(ps) == 1 { // {
nb[k] = v // "user.name": "alex",
// "user.document.id": "document_id"
// "user.document.number": "document_number"
// }
// Output:
// {
// "user": {
// "name": "alex",
// "document": {
// "id": "document_id"
// "number": "document_number"
// }
// }
// }
//
// Example #2 (with conflicts):
//
// Input:
// {
// "user": "alex",
// "user.document.id": "document_id"
// "user.document.number": "document_number"
// }
// Output:
// {
// "user": "alex"
// }
func FlattenMap(input map[string]interface{}) map[string]interface{} {
result := make(map[string]interface{})
for k, v := range input {
parts := strings.Split(k, ".")
if len(parts) == 1 {
result[k] = v
continue continue
} }
em := make(map[string]interface{})
em[ps[len(ps)-1]] = v current := result
for i := len(ps) - 2; i > 0; i-- {
nm := make(map[string]interface{}) for i, part := range parts {
nm[ps[i]] = em // last element in the path
em = nm if i == len(parts)-1 {
} current[part] = v
if vm, ok := nb[ps[0]]; ok { break
// nested map }
nm := vm.(map[string]interface{})
for vk, vv := range em { // initialize map for current level if not exist
nm[vk] = vv if _, ok := current[part]; !ok {
current[part] = make(map[string]interface{})
}
if nested, ok := current[part].(map[string]interface{}); ok {
current = nested // continue to the nested map
} else {
break // if current element is not a map, ignore it
} }
nb[ps[0]] = nm
} else {
nb[ps[0]] = em
} }
} }
return nb
return result
} }
/* /*

View File

@@ -6,6 +6,7 @@ import (
"testing" "testing"
"time" "time"
"github.com/stretchr/testify/require"
rutil "go.unistack.org/micro/v4/util/reflect" rutil "go.unistack.org/micro/v4/util/reflect"
) )
@@ -319,3 +320,140 @@ func TestIsZero(t *testing.T) {
// t.Logf("XX %#+v\n", ok) // t.Logf("XX %#+v\n", ok)
} }
func TestFlattenMap(t *testing.T) {
tests := []struct {
name string
input map[string]interface{}
expected map[string]interface{}
}{
{
name: "empty map",
input: map[string]interface{}{},
expected: map[string]interface{}{},
},
{
name: "nil map",
input: nil,
expected: map[string]interface{}{},
},
{
name: "single level",
input: map[string]interface{}{
"username": "username",
"password": "password",
},
expected: map[string]interface{}{
"username": "username",
"password": "password",
},
},
{
name: "two level",
input: map[string]interface{}{
"order_id": "order_id",
"user.name": "username",
"user.password": "password",
},
expected: map[string]interface{}{
"order_id": "order_id",
"user": map[string]interface{}{
"name": "username",
"password": "password",
},
},
},
{
name: "three level",
input: map[string]interface{}{
"order_id": "order_id",
"user.name": "username",
"user.password": "password",
"user.document.id": "document_id",
"user.document.number": "document_number",
},
expected: map[string]interface{}{
"order_id": "order_id",
"user": map[string]interface{}{
"name": "username",
"password": "password",
"document": map[string]interface{}{
"id": "document_id",
"number": "document_number",
},
},
},
},
{
name: "four level",
input: map[string]interface{}{
"order_id": "order_id",
"user.name": "username",
"user.password": "password",
"user.document.id": "document_id",
"user.document.number": "document_number",
"user.info.permissions.read": "available",
"user.info.permissions.write": "available",
},
expected: map[string]interface{}{
"order_id": "order_id",
"user": map[string]interface{}{
"name": "username",
"password": "password",
"document": map[string]interface{}{
"id": "document_id",
"number": "document_number",
},
"info": map[string]interface{}{
"permissions": map[string]interface{}{
"read": "available",
"write": "available",
},
},
},
},
},
{
name: "key conflicts",
input: map[string]interface{}{
"user": "user",
"user.name": "username",
"user.password": "password",
},
expected: map[string]interface{}{
"user": "user",
},
},
{
name: "overwriting conflicts",
input: map[string]interface{}{
"order_id": "order_id",
"user.document.id": "document_id",
"user.document.number": "document_number",
"user.info.address": "address",
"user.info.phone": "phone",
},
expected: map[string]interface{}{
"order_id": "order_id",
"user": map[string]interface{}{
"document": map[string]interface{}{
"id": "document_id",
"number": "document_number",
},
"info": map[string]interface{}{
"address": "address",
"phone": "phone",
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for range 100 { // need to exclude the impact of key order in the map on the test.
require.Equal(t, tt.expected, rutil.FlattenMap(tt.input))
}
})
}
}

View File

@@ -14,7 +14,7 @@ type Buffer struct {
vals []*Entry vals []*Entry
size int size int
sync.RWMutex mu sync.RWMutex
} }
// Entry is ring buffer data entry // Entry is ring buffer data entry
@@ -35,8 +35,8 @@ type Stream struct {
// Put adds a new value to ring buffer // Put adds a new value to ring buffer
func (b *Buffer) Put(v interface{}) { func (b *Buffer) Put(v interface{}) {
b.Lock() b.mu.Lock()
defer b.Unlock() defer b.mu.Unlock()
// append to values // append to values
entry := &Entry{ entry := &Entry{
@@ -63,8 +63,8 @@ func (b *Buffer) Put(v interface{}) {
// Get returns the last n entries // Get returns the last n entries
func (b *Buffer) Get(n int) []*Entry { func (b *Buffer) Get(n int) []*Entry {
b.RLock() b.mu.RLock()
defer b.RUnlock() defer b.mu.RUnlock()
// reset any invalid values // reset any invalid values
if n > len(b.vals) || n < 0 { if n > len(b.vals) || n < 0 {
@@ -80,8 +80,8 @@ func (b *Buffer) Get(n int) []*Entry {
// Since returns the entries since a specific time // Since returns the entries since a specific time
func (b *Buffer) Since(t time.Time) []*Entry { func (b *Buffer) Since(t time.Time) []*Entry {
b.RLock() b.mu.RLock()
defer b.RUnlock() defer b.mu.RUnlock()
// return all the values // return all the values
if t.IsZero() { if t.IsZero() {
@@ -109,8 +109,8 @@ func (b *Buffer) Since(t time.Time) []*Entry {
// Stream logs from the buffer // Stream logs from the buffer
// Close the channel when you want to stop // Close the channel when you want to stop
func (b *Buffer) Stream() (<-chan *Entry, chan bool) { func (b *Buffer) Stream() (<-chan *Entry, chan bool) {
b.Lock() b.mu.Lock()
defer b.Unlock() defer b.mu.Unlock()
entries := make(chan *Entry, 128) entries := make(chan *Entry, 128)
id := id.MustNew() id := id.MustNew()

View File

@@ -24,7 +24,7 @@ type stream struct {
err error err error
request *request request *request
sync.RWMutex mu sync.RWMutex
} }
type request struct { type request struct {
@@ -57,9 +57,9 @@ func (s *stream) Request() server.Request {
func (s *stream) Send(v interface{}) error { func (s *stream) Send(v interface{}) error {
err := s.Stream.SendMsg(v) err := s.Stream.SendMsg(v)
if err != nil { if err != nil {
s.Lock() s.mu.Lock()
s.err = err s.err = err
s.Unlock() s.mu.Unlock()
} }
return err return err
} }
@@ -68,17 +68,17 @@ func (s *stream) Send(v interface{}) error {
func (s *stream) Recv(v interface{}) error { func (s *stream) Recv(v interface{}) error {
err := s.Stream.RecvMsg(v) err := s.Stream.RecvMsg(v)
if err != nil { if err != nil {
s.Lock() s.mu.Lock()
s.err = err s.err = err
s.Unlock() s.mu.Unlock()
} }
return err return err
} }
// Error returns error that stream holds // Error returns error that stream holds
func (s *stream) Error() error { func (s *stream) Error() error {
s.RLock() s.mu.RLock()
defer s.RUnlock() defer s.mu.RUnlock()
return s.err return s.err
} }

View File

@@ -6,7 +6,7 @@ import (
"strconv" "strconv"
"time" "time"
"gopkg.in/yaml.v3" "github.com/goccy/go-yaml"
) )
type Duration int64 type Duration int64
@@ -58,9 +58,9 @@ func (d Duration) MarshalYAML() (interface{}, error) {
return time.Duration(d).String(), nil return time.Duration(d).String(), nil
} }
func (d *Duration) UnmarshalYAML(n *yaml.Node) error { func (d *Duration) UnmarshalYAML(data []byte) error {
var v interface{} var v interface{}
if err := yaml.Unmarshal([]byte(n.Value), &v); err != nil { if err := yaml.Unmarshal(data, &v); err != nil {
return err return err
} }
switch value := v.(type) { switch value := v.(type) {

View File

@@ -6,7 +6,7 @@ import (
"testing" "testing"
"time" "time"
"gopkg.in/yaml.v3" "github.com/goccy/go-yaml"
) )
func TestMarshalYAML(t *testing.T) { func TestMarshalYAML(t *testing.T) {

View File

@@ -6,18 +6,18 @@ import (
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time"
"go.unistack.org/micro/v4/meter" "go.unistack.org/micro/v4/meter"
"go.unistack.org/micro/v4/semconv" "go.unistack.org/micro/v4/semconv"
) )
var ( func unregisterMetrics(size int) {
pools = make([]Statser, 0) meter.DefaultMeter.Unregister(semconv.PoolGetTotal, "capacity", strconv.Itoa(size))
poolsMu sync.Mutex meter.DefaultMeter.Unregister(semconv.PoolPutTotal, "capacity", strconv.Itoa(size))
) meter.DefaultMeter.Unregister(semconv.PoolMisTotal, "capacity", strconv.Itoa(size))
meter.DefaultMeter.Unregister(semconv.PoolRetTotal, "capacity", strconv.Itoa(size))
}
// Stats struct
type Stats struct { type Stats struct {
Get uint64 Get uint64
Put uint64 Put uint64
@@ -25,41 +25,13 @@ type Stats struct {
Ret uint64 Ret uint64
} }
// Statser provides buffer pool stats
type Statser interface {
Stats() Stats
Cap() int
}
func init() {
go newStatsMeter()
}
func newStatsMeter() {
ticker := time.NewTicker(meter.DefaultMeterStatsInterval)
defer ticker.Stop()
for range ticker.C {
poolsMu.Lock()
for _, st := range pools {
stats := st.Stats()
meter.DefaultMeter.Counter(semconv.PoolGetTotal, "capacity", strconv.Itoa(st.Cap())).Set(stats.Get)
meter.DefaultMeter.Counter(semconv.PoolPutTotal, "capacity", strconv.Itoa(st.Cap())).Set(stats.Put)
meter.DefaultMeter.Counter(semconv.PoolMisTotal, "capacity", strconv.Itoa(st.Cap())).Set(stats.Mis)
meter.DefaultMeter.Counter(semconv.PoolRetTotal, "capacity", strconv.Itoa(st.Cap())).Set(stats.Ret)
}
poolsMu.Unlock()
}
}
var (
_ Statser = (*BytePool)(nil)
_ Statser = (*BytesPool)(nil)
_ Statser = (*StringsPool)(nil)
)
type Pool[T any] struct { type Pool[T any] struct {
p *sync.Pool p *sync.Pool
get *atomic.Uint64
put *atomic.Uint64
mis *atomic.Uint64
ret *atomic.Uint64
c int
} }
func (p Pool[T]) Put(t T) { func (p Pool[T]) Put(t T) {
@@ -70,37 +42,82 @@ func (p Pool[T]) Get() T {
return p.p.Get().(T) return p.p.Get().(T)
} }
func NewPool[T any](fn func() T) Pool[T] { func NewPool[T any](fn func() T, size int) Pool[T] {
return Pool[T]{ p := Pool[T]{
p: &sync.Pool{ c: size,
New: func() interface{} { get: &atomic.Uint64{},
return fn() put: &atomic.Uint64{},
}, mis: &atomic.Uint64{},
ret: &atomic.Uint64{},
}
p.p = &sync.Pool{
New: func() interface{} {
p.mis.Add(1)
return fn()
}, },
} }
meter.DefaultMeter.Gauge(semconv.PoolGetTotal, func() float64 {
return float64(p.get.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolPutTotal, func() float64 {
return float64(p.put.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolMisTotal, func() float64 {
return float64(p.mis.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolRetTotal, func() float64 {
return float64(p.ret.Load())
}, "capacity", strconv.Itoa(p.c))
return p
} }
type BytePool struct { type BytePool struct {
p *sync.Pool p *sync.Pool
get uint64 get *atomic.Uint64
put uint64 put *atomic.Uint64
mis uint64 mis *atomic.Uint64
ret uint64 ret *atomic.Uint64
c int c int
} }
func NewBytePool(size int) *BytePool { func NewBytePool(size int) *BytePool {
p := &BytePool{c: size} p := &BytePool{
c: size,
get: &atomic.Uint64{},
put: &atomic.Uint64{},
mis: &atomic.Uint64{},
ret: &atomic.Uint64{},
}
p.p = &sync.Pool{ p.p = &sync.Pool{
New: func() interface{} { New: func() interface{} {
atomic.AddUint64(&p.mis, 1) p.mis.Add(1)
b := make([]byte, 0, size) b := make([]byte, 0, size)
return &b return &b
}, },
} }
poolsMu.Lock()
pools = append(pools, p) meter.DefaultMeter.Gauge(semconv.PoolGetTotal, func() float64 {
poolsMu.Unlock() return float64(p.get.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolPutTotal, func() float64 {
return float64(p.put.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolMisTotal, func() float64 {
return float64(p.mis.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolRetTotal, func() float64 {
return float64(p.ret.Load())
}, "capacity", strconv.Itoa(p.c))
return p return p
} }
@@ -110,49 +127,73 @@ func (p *BytePool) Cap() int {
func (p *BytePool) Stats() Stats { func (p *BytePool) Stats() Stats {
return Stats{ return Stats{
Put: atomic.LoadUint64(&p.put), Put: p.put.Load(),
Get: atomic.LoadUint64(&p.get), Get: p.get.Load(),
Mis: atomic.LoadUint64(&p.mis), Mis: p.mis.Load(),
Ret: atomic.LoadUint64(&p.ret), Ret: p.ret.Load(),
} }
} }
func (p *BytePool) Get() *[]byte { func (p *BytePool) Get() *[]byte {
atomic.AddUint64(&p.get, 1) p.get.Add(1)
return p.p.Get().(*[]byte) return p.p.Get().(*[]byte)
} }
func (p *BytePool) Put(b *[]byte) { func (p *BytePool) Put(b *[]byte) {
atomic.AddUint64(&p.put, 1) p.put.Add(1)
if cap(*b) > p.c { if cap(*b) > p.c {
atomic.AddUint64(&p.ret, 1) p.ret.Add(1)
return return
} }
*b = (*b)[:0] *b = (*b)[:0]
p.p.Put(b) p.p.Put(b)
} }
func (p *BytePool) Close() {
unregisterMetrics(p.c)
}
type BytesPool struct { type BytesPool struct {
p *sync.Pool p *sync.Pool
get uint64 get *atomic.Uint64
put uint64 put *atomic.Uint64
mis uint64 mis *atomic.Uint64
ret uint64 ret *atomic.Uint64
c int c int
} }
func NewBytesPool(size int) *BytesPool { func NewBytesPool(size int) *BytesPool {
p := &BytesPool{c: size} p := &BytesPool{
c: size,
get: &atomic.Uint64{},
put: &atomic.Uint64{},
mis: &atomic.Uint64{},
ret: &atomic.Uint64{},
}
p.p = &sync.Pool{ p.p = &sync.Pool{
New: func() interface{} { New: func() interface{} {
atomic.AddUint64(&p.mis, 1) p.mis.Add(1)
b := bytes.NewBuffer(make([]byte, 0, size)) b := bytes.NewBuffer(make([]byte, 0, size))
return b return b
}, },
} }
poolsMu.Lock()
pools = append(pools, p) meter.DefaultMeter.Gauge(semconv.PoolGetTotal, func() float64 {
poolsMu.Unlock() return float64(p.get.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolPutTotal, func() float64 {
return float64(p.put.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolMisTotal, func() float64 {
return float64(p.mis.Load())
}, "capacity", strconv.Itoa(p.c))
meter.DefaultMeter.Gauge(semconv.PoolRetTotal, func() float64 {
return float64(p.ret.Load())
}, "capacity", strconv.Itoa(p.c))
return p return p
} }
@@ -162,10 +203,10 @@ func (p *BytesPool) Cap() int {
func (p *BytesPool) Stats() Stats { func (p *BytesPool) Stats() Stats {
return Stats{ return Stats{
Put: atomic.LoadUint64(&p.put), Put: p.put.Load(),
Get: atomic.LoadUint64(&p.get), Get: p.get.Load(),
Mis: atomic.LoadUint64(&p.mis), Mis: p.mis.Load(),
Ret: atomic.LoadUint64(&p.ret), Ret: p.ret.Load(),
} }
} }
@@ -174,34 +215,43 @@ func (p *BytesPool) Get() *bytes.Buffer {
} }
func (p *BytesPool) Put(b *bytes.Buffer) { func (p *BytesPool) Put(b *bytes.Buffer) {
p.put.Add(1)
if (*b).Cap() > p.c { if (*b).Cap() > p.c {
atomic.AddUint64(&p.ret, 1) p.ret.Add(1)
return return
} }
b.Reset() b.Reset()
p.p.Put(b) p.p.Put(b)
} }
func (p *BytesPool) Close() {
unregisterMetrics(p.c)
}
type StringsPool struct { type StringsPool struct {
p *sync.Pool p *sync.Pool
get uint64 get *atomic.Uint64
put uint64 put *atomic.Uint64
mis uint64 mis *atomic.Uint64
ret uint64 ret *atomic.Uint64
c int c int
} }
func NewStringsPool(size int) *StringsPool { func NewStringsPool(size int) *StringsPool {
p := &StringsPool{c: size} p := &StringsPool{
c: size,
get: &atomic.Uint64{},
put: &atomic.Uint64{},
mis: &atomic.Uint64{},
ret: &atomic.Uint64{},
}
p.p = &sync.Pool{ p.p = &sync.Pool{
New: func() interface{} { New: func() interface{} {
atomic.AddUint64(&p.mis, 1) p.mis.Add(1)
return &strings.Builder{} return &strings.Builder{}
}, },
} }
poolsMu.Lock()
pools = append(pools, p)
poolsMu.Unlock()
return p return p
} }
@@ -211,24 +261,28 @@ func (p *StringsPool) Cap() int {
func (p *StringsPool) Stats() Stats { func (p *StringsPool) Stats() Stats {
return Stats{ return Stats{
Put: atomic.LoadUint64(&p.put), Put: p.put.Load(),
Get: atomic.LoadUint64(&p.get), Get: p.get.Load(),
Mis: atomic.LoadUint64(&p.mis), Mis: p.mis.Load(),
Ret: atomic.LoadUint64(&p.ret), Ret: p.ret.Load(),
} }
} }
func (p *StringsPool) Get() *strings.Builder { func (p *StringsPool) Get() *strings.Builder {
atomic.AddUint64(&p.get, 1) p.get.Add(1)
return p.p.Get().(*strings.Builder) return p.p.Get().(*strings.Builder)
} }
func (p *StringsPool) Put(b *strings.Builder) { func (p *StringsPool) Put(b *strings.Builder) {
atomic.AddUint64(&p.put, 1) p.put.Add(1)
if b.Cap() > p.c { if b.Cap() > p.c {
atomic.AddUint64(&p.ret, 1) p.ret.Add(1)
return return
} }
b.Reset() b.Reset()
p.p.Put(b) p.p.Put(b)
} }
func (p *StringsPool) Close() {
unregisterMetrics(p.c)
}