
:guide-id: metrics-for-troubleshooting
:guide-title: Troubleshooting using metrics
:guide-summary: Use metrics for troubleshooting errors and performance issues.
:guide-priority: 999
:guide-tile-visible: true
:version: 26.3.4

include::../attributes.adoc[]

[[metrics-for-troubleshooting]]
= Troubleshooting using metrics

ifeval::["" == "true"]
WARNING: This {section} is describing a feature which is currently in preview.
ifeval::["" == ""]
Please provide your feedback while we’re continuing to work on this.
endif::[]
ifeval::["" != ""]
Please provide your feedback by link:[joining this discussion] while we’re continuing to work on this.
endif::[]
endif::[]


For a running {project_name} deployment it is important to understand how the system performs and whether it meets your service level objectives (SLOs).
For more details on SLOs, proceed to the link:{links_observability_keycloak-service-level-indicators_url}[{links_observability_keycloak-service-level-indicators_name}] {section}.

This guide will provide directions to answer the question: "`What can I do when my SLOs are not met?`"

{project_name} consists of several components where an issue or misconfiguration of one of them can move your service level indicators to undesirable numbers.

A guidance provided by this guide is illustrated in the following example:

====

*Observation:* Latency service level objective is not met.

*Metrics that indicate a problem*:

. {project_name}'s database connection pool is often exhausted, and there are threads queuing for a connection to be retrieved from the pool.
. {project_name}'s `users` cache hit ratio is at a low percentage, around 5%. This means only 1 out of 20 user searches is able to obtain user data from the cache and the rest needs to load it from the database.

*Possible mitigations suggested:*

* Increasing the `users` cache size to a higher number which would decrease the number of reads from the database.
* Increasing the number of connections in the connection pool. This would need to be checked with metrics for your database and tuning it for a higher load, for example, by increasing the number of available processors.

====

[NOTE]
====
* This guide focuses on {project_name} metrics.
Troubleshooting the database itself is out of scope.
* This guide provides general guidance.
You should always confirm the configuration change by conducting a performance test comparing the metrics in question for the old and the new configuration.
====

[NOTE]
====
Grafana dashboards for the metrics below can be found in link:{links_observability_grafana-dashboards_url}[{links_observability_grafana-dashboards_name}] {section}.
====

== List of {project_name} key metrics

* link:{links_observability_metrics-for-troubleshooting-keycloak_url}[{links_observability_metrics-for-troubleshooting-keycloak_name}]
* link:{links_observability_metrics-for-troubleshooting-jvm_url}[{links_observability_metrics-for-troubleshooting-jvm_name}]
* link:{links_observability_metrics-for-troubleshooting-database_url}[{links_observability_metrics-for-troubleshooting-database_name}]
* link:{links_observability_metrics-for-troubleshooting-http_url}[{links_observability_metrics-for-troubleshooting-http_name}]
* Single site metrics (without external {jdgserver_name})
** link:{links_observability_metrics-for-troubleshooting-clustering-and-network_url}[{links_observability_metrics-for-troubleshooting-clustering-and-network_name}]
** link:{links_observability_metrics-for-troubleshooting-embedded-caches_url}[{links_observability_metrics-for-troubleshooting-embedded-caches_name}]
* Multiple sites metrics (as described in link:{links_high-availability_introduction_url}[{links_high-availability_introduction_name}])
** link:{links_observability_metrics-for-troubleshooting-embedded-caches-multi-site_url}[{links_observability_metrics-for-troubleshooting-embedded-caches-multi-site_name}]
** link:{links_observability_metrics-for-troubleshooting-external-infinispan-multi-site_url}[{links_observability_metrics-for-troubleshooting-external-infinispan-multi-site_name}]


