138 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
c75f3497fe
process_metrics_linux.go: consistently mention the metrics package in error logs 2022-07-21 19:33:33 +03:00
Aliaksandr Valialkin
195ac63dbf
Improve error logging when metrics cannot be pushed to pushURL passed to InitPush*() 2022-07-21 19:32:48 +03:00
Aliaksandr Valialkin
5710165c4b
Add a benchmark for addExtraLabels function 2022-07-21 18:42:53 +03:00
Aliaksandr Valialkin
3515499561
Export InitPushExt function for pushing metrics provided via caller callback 2022-07-21 18:35:46 +03:00
Aliaksandr Valialkin
aab1d62de8
push.go: limit the maximum timeout for pushing metrics to the provided interval between pushes
This should guarantee that metrics are pushed regularly with the provided interval.
If the remote storage cannot keep up with push frequency, then timeout errors will be logged.
2022-07-21 18:21:11 +03:00
sequix
2767350561
Add function to list all metric names from default set (#29) 2022-07-21 18:14:01 +03:00
Aliaksandr Valialkin
6cf96d45a0
add functionality for periodic pushing metrics to remote storage via InitPush* 2022-07-20 16:53:12 +03:00
Aliaksandr Valialkin
7fa54d40f0
README.md: rename "time series" to more clear "metrics" in the example 2022-07-20 14:56:10 +03:00
Aliaksandr Valialkin
96a67ce905
README.md: add a link to Prometheus metric naming convention 2022-07-20 12:50:30 +03:00
Aliaksandr Valialkin
f05b7d77a2
go fmt 2021-11-01 14:53:23 +02:00
Aliaksandr Valialkin
0ee99fd214
set.go: fix data race in Set.ListMetricNames when accessing Set.m map
The Set.m map can be updated by concurrent goroutines when ListMetricNames is called
2021-11-01 14:52:22 +02:00
Aliaksandr Valialkin
2ec4485cad vendor: update github.com/valyala/histogram from v1.1.2 to v1.2.0 in order to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612 2021-09-15 09:20:11 +03:00
1aa57265d4
allow dot in metric name (#26) 2021-07-07 15:36:08 +03:00
Aliaksandr Valialkin
bfa3fbc1a7 Document go_memstats_* metrics 2021-03-22 14:36:33 +02:00
Aliaksandr Valialkin
264e71c98a document exported process_* metrics 2021-03-18 18:49:19 +02:00
Aliaksandr Valialkin
096d705076 cleanup after 5a49bb8e88e070e43cbffaa68776259f11f6c053 2021-03-17 23:17:01 +02:00
Aliaksandr Valialkin
53fe52ac0e Revert "export process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics"
This reverts commit 55d5027c97e2484d3e2b63ec6eb54845a8f69227.

Reason for revert: the process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics have been substituted
by more complete set of metrics in the commit 5a49bb8e88e070e43cbffaa68776259f11f6c053:

* process_resident_memory_anon_bytes
* process_resident_memory_file_bytes
* process_resident_memory_shared_bytes
2021-03-17 23:10:35 +02:00
Nikolay
5a49bb8e88
adds extended memory stats (#21) 2021-03-17 23:07:19 +02:00
Aliaksandr Valialkin
55d5027c97 export process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics
- process_resident_memory_anonymous_bytes shows RSS share for memory allocated by the process.
- process_resident_memory_pagecache_bytes shows RSS share for page cache (aka memory-mapped files).
2021-03-17 17:48:31 +02:00
Aliaksandr Valialkin
98a1807528 Inline Histogram.resetLocked function 2021-03-16 12:14:36 +02:00
Aliaksandr Valialkin
cea3ca311b Reset histogram sum on Histogram.Reset call 2021-03-16 12:13:43 +02:00
Aliaksandr Valialkin
57c9db1bc3 properly handle Histogram.Update(1e-9)
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1096
2021-03-01 16:47:08 +02:00
Aliaksandr Valialkin
16e4558792 Move bucket index calculations to the place where they are used
This can help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1096
2021-02-28 17:39:14 +02:00
Aliaksandr Valialkin
5523b934ae add process_io_* metrics read from /proc/self/io`
Metrics are:

process_io_read_bytes_total - the number of bytes read via io syscalls such as read and pread
process_io_written_bytes_total - the number of bytes written via io syscalls such as write and pwrite
process_io_read_syscalls_total - the number of read syscalls such as read and pread
process_io_write_syscalls_total - the number of write syscalls such as write and pwrite
process_io_storage_read_bytes_total - the number of bytes read from storage layer
process_io_storage_written_bytes_total - the number of bytes written to storage layer
2021-02-21 22:48:19 +02:00
Harmen
b862701a8d
fix example link (#20)
The link redirects to https://pkg.go.dev/github.com/VictoriaMetrics/metrics#example-Counter--Vec , which doesn't exist (note the double `--`: godoc has two slashes, pkg.go.dev only one).

It's maybe a bit weird to link to pkg.go.dev for only one link. I can switch them all over if you prefer. Or they could all go to https://godocs.io instead, which still uses the double `--`: https://godocs.io/github.com/VictoriaMetrics/metrics#example-Counter--Vec
2021-02-17 01:17:17 +02:00
Aliaksandr Valialkin
e7f78fa63c Switch from log-linear histograms to log-based histograms
Log-based histograms provide lower estimation error for the same number of buckets compared to log-linear histograms.

For example, the current Histogram implementation splits each decimal range (10^n .. 10^(n+1)] into 18 buckets.
These buckets have the following bounds:

- for log-linear histogram: (1 .. 1.5], (1.5 .. 2], (2 .. 2.5], ... (9.5 .. 10]
- for log-based histogram: (1 .. 1.136], (1.136 .. 1.292], ... (8.799 ... 10]

The maximum estimated error for log-linear histogram is reached in the first bucket per each decimal range and equals to 1.5-1=0.5 or 50%.
The maximum estimated error for log-based histogram is constant across buckets and equals to 1.136-1=0.136 or 13.6%.

This means that log-based histogram improves histogram accuracy by up to 50%/13.6% = 3.6 times when using the same number of buckets.

Further reading - https://linuxczar.net/blog/2020/08/13/histogram-error/
2021-02-15 14:26:03 +02:00
Aliaksandr Valialkin
43691b65a0 Read /proc/* file contents with ioutil.ReadFile() instead of opening it with os.Open() and scanning with bufio.Scanner
This simplifies the code a bit.
2021-02-08 14:23:40 +02:00
Aliaksandr Valialkin
c070763356 follow-up for 2a12f948a97173ec8bf4c900e6725712021ef737 2021-02-04 16:28:55 +02:00
Nikolay
2a12f948a9
adds file descriptors metrics, (#19)
* adds file descriptors metrics,
for linux based systems,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1037

* small fix
2021-02-04 16:10:10 +02:00
Roman Khavronenko
be819551e3
summary: fix Unregister behaviour for summary metric type (#16)
The change covers two things:
1. Cleanup of Set.a metrics list from per-quantile metrics
for summary.
2. Register summary metric and per-quantile metrics in one take.
This prevents registry corruption when Unregister called in the
middle of metric register process.
2020-08-07 12:59:44 +03:00
Roman Khavronenko
dd0c59c0d6
Remove timeseries with quantile labels when Unregister is called for summary #14 (#15) 2020-07-28 21:58:28 +03:00
Roman Khavronenko
6a144d3619
Add function to unregister metrics from default set #12 (#13) 2020-07-27 22:05:14 +03:00
Aliaksandr Valialkin
5edf5eaace vendor: update github.com/valyala/histogram from v1.1.1 to v1.1.2 2020-07-20 16:54:34 +03:00
Aliaksandr Valialkin
7ec95de8e8 vendor: update github.com/valyala/histogram from v1.0.1 to v1.1.1
This should reduce memory usage when big number of Summary metrics are in use
while small number of these metrics are updated
2020-07-20 16:50:58 +03:00
Aliaksandr Valialkin
fd1ac75aa0 gauge.go: mention FloatCounter in Gauge docs
Updates https://github.com/VictoriaMetrics/metrics/issues/11
2020-07-05 17:49:52 +03:00
Aliaksandr Valialkin
d81dd5ec9c Increase value precision in histograms from 5e-3 to 1e-12
Previously values could go into lower bucket because of too coarse precision.

Updates https://github.com/VictoriaMetrics/metrics/issues/8
2020-05-20 02:49:55 +03:00
Aliaksandr Valialkin
f6ff0f32a2 fix data race when accessing h.sum during histogram marshaling 2020-03-13 12:36:02 +02:00
Artem Navoiev
93f7956e2a
bump version of codecov-action to v1.0.6 2020-03-05 23:26:04 +02:00
Aliaksandr Valialkin
a494df3462 Properly unregister Summary metric in Set.UnregisterMetric 2020-02-26 20:37:47 +02:00
Roland Lammel
900c892625
Allow unregistration of metrics (#6)
* Allow unregistration of metrics

* Provide metrics.WritePrometheusSet to write using a dedicated metric set

* Cleanup unregister metrics patch, add generic WriteProcessMetrics
2020-02-26 19:52:38 +02:00
Aliaksandr Valialkin
0ee6a1fa32 Update url to MetricsQL in docs to Histogram 2020-02-10 23:05:09 +02:00
Ivan G
e6d6f46b5d Proposal: Add new type of counter: FloatCounter (#5)
* Add new type of counter: FloatCounter
* sometimes you need to count things with more precision than uint64
* FloatCounter also usefull if you need setable Gauge w/o callback func

* Fix PR review:
* sync.RWMutex -> sync.Mutex
* more idiomatic add/sub
2020-01-23 12:48:00 +02:00
Aliaksandr Valialkin
21c3ffd10e histogram.go: rename Extended PromQL to PromQL extensions to be more clear 2019-12-12 19:24:01 +02:00
Aliaksandr Valialkin
fcb89496ec Prevent from possible deadlock in Set.WritePrometheus when Gague callback could call Set.mu.Lock 2019-12-02 22:26:56 +02:00
Aliaksandr Valialkin
ddbf13fd17 histogram.go: add a link to https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL in Histogram docs 2019-11-27 14:06:17 +02:00
Aliaksandr Valialkin
4d9d3a095f Add examples for Histogram 2019-11-27 14:04:12 +02:00
Aliaksandr Valialkin
007e78a6c7 Add links to https://medium.com/@valyala/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350 from Histogram docs 2019-11-27 14:01:15 +02:00
Aliaksandr Valialkin
674ca26d5f Reduced the number of steps per decimal exponent from 5 to 2, since 5 is too big for the majority of cases 2019-11-25 15:19:31 +02:00
Aliaksandr Valialkin
3306660b50 Reduce histogram buckets from 0.1 to 0.02 of the size for the current decimal exponent
Also increase upper bound for histogram range from 10^12 to 10^18
2019-11-25 13:08:44 +02:00
Aliaksandr Valialkin
eb780f4caa Buffer all the output in memory before writing it to the provided io.Writer in WritePrometheus
This should prevent from indefinite locking of internal structures in the case of laggy io.Writer
2019-11-25 13:01:26 +02:00