Commit Graph

98 Commits

Author SHA1 Message Date
Zakhar Bessarab
7921ac9c64
linux process metrics: add error handling for metrics from /proc/self/io (#43)
* linux process metrics: add error handling for metrics from `/proc/self/io`

* linux process metrics: address review feedback

* linux process metrics: restore error handling for self/stat metrics

* linux process metrics: implement self/io error check with `init` function
2023-01-15 13:51:57 -08:00
Aliaksandr Valialkin
b2edcdcdd1
.github/workflows/main.yml: bump Go version from 1.13 to 1.15 in order to support url.Redacted() 2022-10-29 05:20:10 +03:00
Aliaksandr Valialkin
17839691f2
Add UnregisterAllMetrics() and GetDefaultSet() functions
While at it, make sure that ListMetricNames() returns sorted list of the registered metrics
2022-10-25 10:59:37 +03:00
Aliaksandr Valialkin
cdba092549
go.mod: bump minimum supported Go version from 1.12 to 1.15
This is needed in order to support `net/url.URL.Redacted()` method used in push.go
This method appeared in Go1.15 - see https://tip.golang.org/doc/go1.15
2022-08-17 21:09:18 +03:00
Aliaksandr Valialkin
4ca082233a
all: go fmt from Go1.19 2022-08-08 17:15:06 +03:00
Aliaksandr Valialkin
c6c6640e5b
add RegisterSet() and UnregisterSet() functions for registering the set in metrics export via global WritePrometheus() call 2022-08-08 17:13:46 +03:00
Aliaksandr Valialkin
e9b4bb1534
push.go: expose metrics_push_* metrics related to metrics push process
Updates https://github.com/VictoriaMetrics/metrics/issues/35
2022-08-04 18:12:16 +03:00
Aliaksandr Valialkin
fe970f4df0
push.go: properly flush compressed metrics to pushURL 2022-07-25 10:37:04 +03:00
Aliaksandr Valialkin
b305bc011e
push.go: push metrics to remote storage in gzip-compressed form
This should reduce the needed network bandwidth by ~10x for reading the metrics at remote storage side
2022-07-25 10:19:30 +03:00
Aliaksandr Valialkin
f790ba580c
return error instead of panicing in InitPush* functions 2022-07-21 19:45:07 +03:00
Aliaksandr Valialkin
c75f3497fe
process_metrics_linux.go: consistently mention the metrics package in error logs 2022-07-21 19:33:33 +03:00
Aliaksandr Valialkin
195ac63dbf
Improve error logging when metrics cannot be pushed to pushURL passed to InitPush*() 2022-07-21 19:32:48 +03:00
Aliaksandr Valialkin
5710165c4b
Add a benchmark for addExtraLabels function 2022-07-21 18:42:53 +03:00
Aliaksandr Valialkin
3515499561
Export InitPushExt function for pushing metrics provided via caller callback 2022-07-21 18:35:46 +03:00
Aliaksandr Valialkin
aab1d62de8
push.go: limit the maximum timeout for pushing metrics to the provided interval between pushes
This should guarantee that metrics are pushed regularly with the provided interval.
If the remote storage cannot keep up with push frequency, then timeout errors will be logged.
2022-07-21 18:21:11 +03:00
sequix
2767350561
Add function to list all metric names from default set (#29) 2022-07-21 18:14:01 +03:00
Aliaksandr Valialkin
6cf96d45a0
add functionality for periodic pushing metrics to remote storage via InitPush* 2022-07-20 16:53:12 +03:00
Aliaksandr Valialkin
7fa54d40f0
README.md: rename "time series" to more clear "metrics" in the example 2022-07-20 14:56:10 +03:00
Aliaksandr Valialkin
96a67ce905
README.md: add a link to Prometheus metric naming convention 2022-07-20 12:50:30 +03:00
Aliaksandr Valialkin
f05b7d77a2
go fmt 2021-11-01 14:53:23 +02:00
Aliaksandr Valialkin
0ee99fd214
set.go: fix data race in Set.ListMetricNames when accessing Set.m map
The Set.m map can be updated by concurrent goroutines when ListMetricNames is called
2021-11-01 14:52:22 +02:00
Aliaksandr Valialkin
2ec4485cad vendor: update github.com/valyala/histogram from v1.1.2 to v1.2.0 in order to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612 2021-09-15 09:20:11 +03:00
1aa57265d4
allow dot in metric name (#26) 2021-07-07 15:36:08 +03:00
Aliaksandr Valialkin
bfa3fbc1a7 Document go_memstats_* metrics 2021-03-22 14:36:33 +02:00
Aliaksandr Valialkin
264e71c98a document exported process_* metrics 2021-03-18 18:49:19 +02:00
Aliaksandr Valialkin
096d705076 cleanup after 5a49bb8e88 2021-03-17 23:17:01 +02:00
Aliaksandr Valialkin
53fe52ac0e Revert "export process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics"
This reverts commit 55d5027c97.

Reason for revert: the process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics have been substituted
by more complete set of metrics in the commit 5a49bb8e88:

* process_resident_memory_anon_bytes
* process_resident_memory_file_bytes
* process_resident_memory_shared_bytes
2021-03-17 23:10:35 +02:00
Nikolay
5a49bb8e88
adds extended memory stats (#21) 2021-03-17 23:07:19 +02:00
Aliaksandr Valialkin
55d5027c97 export process_resident_memory_anonymous_bytes and process_resident_memory_pagecache_bytes metrics
- process_resident_memory_anonymous_bytes shows RSS share for memory allocated by the process.
- process_resident_memory_pagecache_bytes shows RSS share for page cache (aka memory-mapped files).
2021-03-17 17:48:31 +02:00
Aliaksandr Valialkin
98a1807528 Inline Histogram.resetLocked function 2021-03-16 12:14:36 +02:00
Aliaksandr Valialkin
cea3ca311b Reset histogram sum on Histogram.Reset call 2021-03-16 12:13:43 +02:00
Aliaksandr Valialkin
57c9db1bc3 properly handle Histogram.Update(1e-9)
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1096
2021-03-01 16:47:08 +02:00
Aliaksandr Valialkin
16e4558792 Move bucket index calculations to the place where they are used
This can help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1096
2021-02-28 17:39:14 +02:00
Aliaksandr Valialkin
5523b934ae add process_io_* metrics read from /proc/self/io`
Metrics are:

process_io_read_bytes_total - the number of bytes read via io syscalls such as read and pread
process_io_written_bytes_total - the number of bytes written via io syscalls such as write and pwrite
process_io_read_syscalls_total - the number of read syscalls such as read and pread
process_io_write_syscalls_total - the number of write syscalls such as write and pwrite
process_io_storage_read_bytes_total - the number of bytes read from storage layer
process_io_storage_written_bytes_total - the number of bytes written to storage layer
2021-02-21 22:48:19 +02:00
Harmen
b862701a8d
fix example link (#20)
The link redirects to https://pkg.go.dev/github.com/VictoriaMetrics/metrics#example-Counter--Vec , which doesn't exist (note the double `--`: godoc has two slashes, pkg.go.dev only one).

It's maybe a bit weird to link to pkg.go.dev for only one link. I can switch them all over if you prefer. Or they could all go to https://godocs.io instead, which still uses the double `--`: https://godocs.io/github.com/VictoriaMetrics/metrics#example-Counter--Vec
2021-02-17 01:17:17 +02:00
Aliaksandr Valialkin
e7f78fa63c Switch from log-linear histograms to log-based histograms
Log-based histograms provide lower estimation error for the same number of buckets compared to log-linear histograms.

For example, the current Histogram implementation splits each decimal range (10^n .. 10^(n+1)] into 18 buckets.
These buckets have the following bounds:

- for log-linear histogram: (1 .. 1.5], (1.5 .. 2], (2 .. 2.5], ... (9.5 .. 10]
- for log-based histogram: (1 .. 1.136], (1.136 .. 1.292], ... (8.799 ... 10]

The maximum estimated error for log-linear histogram is reached in the first bucket per each decimal range and equals to 1.5-1=0.5 or 50%.
The maximum estimated error for log-based histogram is constant across buckets and equals to 1.136-1=0.136 or 13.6%.

This means that log-based histogram improves histogram accuracy by up to 50%/13.6% = 3.6 times when using the same number of buckets.

Further reading - https://linuxczar.net/blog/2020/08/13/histogram-error/
2021-02-15 14:26:03 +02:00
Aliaksandr Valialkin
43691b65a0 Read /proc/* file contents with ioutil.ReadFile() instead of opening it with os.Open() and scanning with bufio.Scanner
This simplifies the code a bit.
2021-02-08 14:23:40 +02:00
Aliaksandr Valialkin
c070763356 follow-up for 2a12f948a9 2021-02-04 16:28:55 +02:00
Nikolay
2a12f948a9
adds file descriptors metrics, (#19)
* adds file descriptors metrics,
for linux based systems,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1037

* small fix
2021-02-04 16:10:10 +02:00
Roman Khavronenko
be819551e3
summary: fix Unregister behaviour for summary metric type (#16)
The change covers two things:
1. Cleanup of Set.a metrics list from per-quantile metrics
for summary.
2. Register summary metric and per-quantile metrics in one take.
This prevents registry corruption when Unregister called in the
middle of metric register process.
2020-08-07 12:59:44 +03:00
Roman Khavronenko
dd0c59c0d6
Remove timeseries with quantile labels when Unregister is called for summary #14 (#15) 2020-07-28 21:58:28 +03:00
Roman Khavronenko
6a144d3619
Add function to unregister metrics from default set #12 (#13) 2020-07-27 22:05:14 +03:00
Aliaksandr Valialkin
5edf5eaace vendor: update github.com/valyala/histogram from v1.1.1 to v1.1.2 2020-07-20 16:54:34 +03:00
Aliaksandr Valialkin
7ec95de8e8 vendor: update github.com/valyala/histogram from v1.0.1 to v1.1.1
This should reduce memory usage when big number of Summary metrics are in use
while small number of these metrics are updated
2020-07-20 16:50:58 +03:00
Aliaksandr Valialkin
fd1ac75aa0 gauge.go: mention FloatCounter in Gauge docs
Updates https://github.com/VictoriaMetrics/metrics/issues/11
2020-07-05 17:49:52 +03:00
Aliaksandr Valialkin
d81dd5ec9c Increase value precision in histograms from 5e-3 to 1e-12
Previously values could go into lower bucket because of too coarse precision.

Updates https://github.com/VictoriaMetrics/metrics/issues/8
2020-05-20 02:49:55 +03:00
Aliaksandr Valialkin
f6ff0f32a2 fix data race when accessing h.sum during histogram marshaling 2020-03-13 12:36:02 +02:00
Artem Navoiev
93f7956e2a
bump version of codecov-action to v1.0.6 2020-03-05 23:26:04 +02:00
Aliaksandr Valialkin
a494df3462 Properly unregister Summary metric in Set.UnregisterMetric 2020-02-26 20:37:47 +02:00
Roland Lammel
900c892625
Allow unregistration of metrics (#6)
* Allow unregistration of metrics

* Provide metrics.WritePrometheusSet to write using a dedicated metric set

* Cleanup unregister metrics patch, add generic WriteProcessMetrics
2020-02-26 19:52:38 +02:00