Commit Graph

115 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
2d7f9a140e
allow passing metric names with labels into Write{Counter,Gauge}{Uint64,Float64} functions 2023-12-19 02:49:57 +02:00
Roman Khavronenko
9dc7358869
allow exposing meta information for registered metrics (#61)
* allow exposing meta information for registered metrics

New public method `ExposeMetadata` allows enabling exposition
of dummy meta-info for all exposed metrics across all Sets.

This feature is needed to improve compatibility
with 3rd-party scrapers that require meta information to be present.

This commit doesn't update exposition of default system/process
metrics to keep the list of changes small. This change should
be added in a follow-up commit.

https://github.com/VictoriaMetrics/metrics/issues/48

* cleanup

* wip

* wip

* wip

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-12-19 02:36:54 +02:00
Aliaksandr Valialkin
fd25889711
push.go: added PushMetrics() function for force pushing of existing metrics to the specified url
Thanks to @aliaaaaaaaaa for the initial implementation at https://github.com/VictoriaMetrics/metrics/pull/37
2023-12-19 00:20:25 +02:00
Aliaksandr Valialkin
bee9e4faf1
push.go: initialize nil opts with the zero PushOptions inside InitPushExtWithOptions
This makes the InitPushExtWithOptions code more maintainable because of removed checks for nil opts.
2023-12-18 12:51:28 +02:00
Aliaksandr Valialkin
b96951708c
push_test.go: properly close the created httptest.Server after every test call 2023-12-18 12:35:07 +02:00
Aliaksandr Valialkin
de719538d1
push.go: expose InitPushExtWithOptions() function, since this it is neede by VictoriaMetrics 2023-12-17 19:39:36 +02:00
Aliaksandr Valialkin
bd3cd7b6ff
push.go: add ability to cancel periodic push via passed context 2023-12-17 18:23:37 +02:00
Aliaksandr Valialkin
a9e3faa53c
push.go: add InitPushWithOptions() function, which allows extending push options without breaking backwards compatibility and without the need to introduce new functions
Overrides https://github.com/VictoriaMetrics/metrics/pull/53
Overrides https://github.com/VictoriaMetrics/metrics/pull/55

Allows specifying authorization headers via PushOptions.Headers for https://github.com/VictoriaMetrics/metrics/issues/36
Allows disabling request body compression via PushOption.DisableCompression for https://github.com/VictoriaMetrics/metrics/pull/41
2023-12-17 17:42:39 +02:00
Aliaksandr Valialkin
447d235cbb
Do not panic on unsupported Go runtime metrics
Log the unsupported Go runtime metrics on startup instead, so the user is aware of unsupported metrics.
The solution for removing the log lines is to upgrade Go builder.

Reduce the minimum supported Go version at go.mod from Go1.20 to Go1.16, where the runtime/metrics package has been added.
See https://tip.golang.org/doc/go1.16#runtime

Updates https://github.com/VictoriaMetrics/metrics/issues/59
Updates https://github.com/VictoriaMetrics/metrics/pull/60
2023-12-17 16:30:30 +02:00
Aliaksandr Valialkin
cd448dd4c1
Revert "support push config for InitPush (#53)"
This reverts commit 42c28a8486.

Reason for revert: the provided additional public API looks not very good for future support.
See https://github.com/VictoriaMetrics/metrics/pull/53#pullrequestreview-1785442537 for details.
2023-12-17 15:39:32 +02:00
Dmytro Kozlov
42c28a8486
support push config for InitPush (#53)
PushConfig represents a struct package uses for pushing
metrics to remote destination. Having a structure
helps to extend functionality in the future, without
touching the signature of existing functions.

For example, `PushConfig` supports custom HTTP headers
via `Headers` param.

Updates https://github.com/VictoriaMetrics/metrics/issues/52
Updates https://github.com/VictoriaMetrics/metrics/issues/36 

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-12-15 13:08:55 +01:00
Aliaksandr Valialkin
2ec14979a8
go_metrics.go: use histogram buckets instead of summary for Go runtime histogram
It is unclear how and when to reset summary - it is smoothed during the Go app uptime.
On the other hand, histogram buckets can be wrapped into increase() or rate() in order
to calculate the histogram distribution on arbitrary time range.

Limit the number of buckets per Go runtime histogram to 30 in order to prevent from high cardinality issues.
2023-11-30 01:50:21 +02:00
Aliaksandr Valialkin
5b58446f57
go_metrics.go: follow-up for 8870cd36e7
- Expose Go runtime histograms as summaries, since they have too many buckets,
  which may lead to high cardinality issues at scraper side.

- Expose go_gc_pauses_seconds summary

- Document exposed go_* metrics
2023-11-30 00:58:08 +02:00
Aliaksandr Valialkin
5ad4f70016
go.mod: run `go get -u ./... && go mod tidy && go mod vendor 2023-11-29 23:49:15 +02:00
Nikolay
8870cd36e7
go_metrics: adds new runtime metrics (#58)
* go_metrics: adds new runtime metrics
go_sched_latency_seconds - Distribution of the time goroutines have spent in the scheduler in a runnable state before actually running
go_mutex_wait_total_seconds - Approximate cumulative time goroutines have spent blocked on a sync.Mutex or sync.RWMutex
https://github.com/VictoriaMetrics/metrics/issues/54

* fixes data race

* add tests

* wip

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-11-29 23:46:41 +02:00
Nikolay
ae1e9d8058
process_metrics: adds metrics for windows OS (#47)
* process_metrics: adds metrics for windows OS

* windows: fixes build constraints

* bumps github ci go to 1.20 version

* bumps go version at go.mod

* go mod vendor
2023-05-16 09:59:00 -07:00
Aliaksandr Valialkin
fd0a3a0deb
process_metrics_linux.go: follow-up after 7921ac9c64
It is better to just log the error once if /proc/self/io cannot be opened,
without exposing additional counters.
The error message should contain directions on how to fix the error.

Updates https://github.com/VictoriaMetrics/metrics/issues/42
2023-01-15 14:01:46 -08:00
Zakhar Bessarab
7921ac9c64
linux process metrics: add error handling for metrics from /proc/self/io (#43)
* linux process metrics: add error handling for metrics from `/proc/self/io`

* linux process metrics: address review feedback

* linux process metrics: restore error handling for self/stat metrics

* linux process metrics: implement self/io error check with `init` function
2023-01-15 13:51:57 -08:00
Aliaksandr Valialkin
b2edcdcdd1
.github/workflows/main.yml: bump Go version from 1.13 to 1.15 in order to support url.Redacted() 2022-10-29 05:20:10 +03:00
Aliaksandr Valialkin
17839691f2
Add UnregisterAllMetrics() and GetDefaultSet() functions
While at it, make sure that ListMetricNames() returns sorted list of the registered metrics
2022-10-25 10:59:37 +03:00
Aliaksandr Valialkin
cdba092549
go.mod: bump minimum supported Go version from 1.12 to 1.15
This is needed in order to support `net/url.URL.Redacted()` method used in push.go
This method appeared in Go1.15 - see https://tip.golang.org/doc/go1.15
2022-08-17 21:09:18 +03:00
Aliaksandr Valialkin
4ca082233a
all: go fmt from Go1.19 2022-08-08 17:15:06 +03:00
Aliaksandr Valialkin
c6c6640e5b
add RegisterSet() and UnregisterSet() functions for registering the set in metrics export via global WritePrometheus() call 2022-08-08 17:13:46 +03:00
Aliaksandr Valialkin
e9b4bb1534
push.go: expose metrics_push_* metrics related to metrics push process
Updates https://github.com/VictoriaMetrics/metrics/issues/35
2022-08-04 18:12:16 +03:00
Aliaksandr Valialkin
fe970f4df0
push.go: properly flush compressed metrics to pushURL 2022-07-25 10:37:04 +03:00
Aliaksandr Valialkin
b305bc011e
push.go: push metrics to remote storage in gzip-compressed form
This should reduce the needed network bandwidth by ~10x for reading the metrics at remote storage side
2022-07-25 10:19:30 +03:00
Aliaksandr Valialkin
f790ba580c
return error instead of panicing in InitPush* functions 2022-07-21 19:45:07 +03:00
Aliaksandr Valialkin
c75f3497fe
process_metrics_linux.go: consistently mention the metrics package in error logs 2022-07-21 19:33:33 +03:00
Aliaksandr Valialkin
195ac63dbf
Improve error logging when metrics cannot be pushed to pushURL passed to InitPush*() 2022-07-21 19:32:48 +03:00
Aliaksandr Valialkin
5710165c4b
Add a benchmark for addExtraLabels function 2022-07-21 18:42:53 +03:00
Aliaksandr Valialkin
3515499561
Export InitPushExt function for pushing metrics provided via caller callback 2022-07-21 18:35:46 +03:00
Aliaksandr Valialkin
aab1d62de8
push.go: limit the maximum timeout for pushing metrics to the provided interval between pushes
This should guarantee that metrics are pushed regularly with the provided interval.
If the remote storage cannot keep up with push frequency, then timeout errors will be logged.
2022-07-21 18:21:11 +03:00
sequix
2767350561
Add function to list all metric names from default set (#29) 2022-07-21 18:14:01 +03:00
Aliaksandr Valialkin
6cf96d45a0
add functionality for periodic pushing metrics to remote storage via InitPush* 2022-07-20 16:53:12 +03:00
Aliaksandr Valialkin
7fa54d40f0
README.md: rename "time series" to more clear "metrics" in the example 2022-07-20 14:56:10 +03:00
Aliaksandr Valialkin
96a67ce905
README.md: add a link to Prometheus metric naming convention 2022-07-20 12:50:30 +03:00
Aliaksandr Valialkin
f05b7d77a2
go fmt 2021-11-01 14:53:23 +02:00
Aliaksandr Valialkin
0ee99fd214
set.go: fix data race in Set.ListMetricNames when accessing Set.m map
The Set.m map can be updated by concurrent goroutines when ListMetricNames is called
2021-11-01 14:52:22 +02:00
Aliaksandr Valialkin
2ec4485cad vendor: update github.com/valyala/histogram from v1.1.2 to v1.2.0 in order to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612 2021-09-15 09:20:11 +03:00
1aa57265d4
allow dot in metric name (#26) 2021-07-07 15:36:08 +03:00
Aliaksandr Valialkin
bfa3fbc1a7 Document go_memstats_* metrics 2021-03-22 14:36:33 +02:00
Aliaksandr Valialkin
264e71c98a document exported process_* metrics 2021-03-18 18:49:19 +02:00
Aliaksandr Valialkin
096d705076 cleanup after 5a49bb8e88 2021-03-17 23:17:01 +02:00
Aliaksandr Valialkin
53fe52ac0e Revert "export process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics"
This reverts commit 55d5027c97.

Reason for revert: the process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics have been substituted
by more complete set of metrics in the commit 5a49bb8e88:

* process_resident_memory_anon_bytes
* process_resident_memory_file_bytes
* process_resident_memory_shared_bytes
2021-03-17 23:10:35 +02:00
Nikolay
5a49bb8e88
adds extended memory stats (#21) 2021-03-17 23:07:19 +02:00
Aliaksandr Valialkin
55d5027c97 export process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics
- process_resident_memory_anonymous_bytes shows RSS share for memory allocated by the process.
- process_resident_memory_pagecache_bytes shows RSS share for page cache (aka memory-mapped files).
2021-03-17 17:48:31 +02:00
Aliaksandr Valialkin
98a1807528 Inline Histogram.resetLocked function 2021-03-16 12:14:36 +02:00
Aliaksandr Valialkin
cea3ca311b Reset histogram sum on Histogram.Reset call 2021-03-16 12:13:43 +02:00
Aliaksandr Valialkin
57c9db1bc3 properly handle Histogram.Update(1e-9)
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1096
2021-03-01 16:47:08 +02:00
Aliaksandr Valialkin
16e4558792 Move bucket index calculations to the place where they are used
This can help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1096
2021-02-28 17:39:14 +02:00