Monitoring

Prometheus CLI

Prometheus commands and PromQL for metrics monitoring.

33 commands
Windows MacOS Linux
#metrics #alerting

Service Control

Start Prometheus with config file
prometheus --config.file=prometheus.yml
prometheus --config.file=prometheus.yml --storage.tsdb.path=/data # Start with custom data path
prometheus --config.file=prometheus.yml --web.listen-address=:9090 # Start on custom port
Validate configuration file
promtool check config prometheus.yml
Validate alerting rules file
promtool check rules rules.yml
Analyze TSDB block statistics
promtool tsdb analyze /data

PromQL Basics

Show all monitored targets status
up
Query raw CPU time metric
node_cpu_seconds_total
Per-second rate over 5 minutes
rate(http_requests_total[5m])
Instant rate over 5 minutes
irate(http_requests_total[5m])
Total increase over 1 hour
increase(http_requests_total[1h])
http_requests_total{method="GET",status="200"} # Filter by label values
Filter with regex matching
http_requests_total{status=~"5.."}

Aggregation

sum(rate(http_requests_total[5m])) by (service) # Sum request rate grouped by service
avg(node_memory_MemAvailable_bytes) by (instance) # Average available memory per instance
Count of healthy targets
count(up == 1)
Top 5 highest request rates
topk(5, rate(http_requests_total[5m]))
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m])) # 95th percentile latency
max(node_filesystem_avail_bytes) by (mountpoint) # Max available disk per mount

Configuration

Validate Prometheus config
promtool check config prometheus.yml
Validate alerting rules
promtool check rules alerts.yml
curl -X POST http://localhost:9090/-/reload # Reload configuration via API
curl http://localhost:9090/api/v1/targets # Query targets via HTTP API
curl http://localhost:9090/api/v1/alerts # Query active alerts via API

TSDB Management

List TSDB blocks
promtool tsdb list /data
Dump TSDB samples
promtool tsdb dump /data
promtool tsdb create-blocks-from openmetrics metrics.txt /data # Import metrics from file
curl -X POST http://localhost:9090/api/v1/admin/tsdb/snapshot # Create TSDB snapshot
curl -X POST http://localhost:9090/api/v1/admin/tsdb/clean_tombstones # Clean tombstones
curl -X PUT http://localhost:9090/api/v1/admin/tsdb/delete_series -d 'match[]=up' # Delete series

Quick Commands

Validate Prometheus configuration file
promtool check config prometheus.yml
Calculate per-second request rate over 5 minutes
rate(http_requests_total[5m])
Calculate 95th percentile latency from histogram
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))