Monitoring Weblogic, SOA, OSB using Prometheus and Grafana

In the previous blog post, we discussed about how to monitor weblogic based applications including SOA and OSB using Vmware Wavefront. In this blog post, let us explore open source alternative with Prometheus and Grafana. We will use Oracle Weblogic Monitoring Exporter in place of jolokia agent to export metric to Prometheus and Grafana for visualizing metrics. High level steps are:
1. Download and install Weblogic Monitoring Exporter.
2. Install Prometheus & Grafana
3. Configure Prometheus to scrape metrics from wls-exporter
4. Configure Grafana dashboards using Prometheus datasource.

Below are detailed steps:

  1. Install Weblogic Monitoring Exporter: Go to Weblogic Monitoring Exporter github releases page and download latest get_v<version>.sh script. Copy the exporter configuration file located here. and pass that as parameter (e.g., ./get_v2.1.2.sh exporter_config.yaml) to get_v<version>.sh. This script downloads war file and copies configuration into war file. There is another approach, instead of packaging configuration into war file, using web interface to change the configuration. We will follow the first approach in this post. Deploy updated war file on Admin and managed servers.
  2. Install Prometheus & Grafana: It is simple and straight forward and there are lot of resources on the web. So, leaving that to you.
  3. Configure Prometheus: Prometheus uses prometheus.yml file (usually located at: /etc/prometheus/) to get metrics source details. Sample file can be found here. Modify this file according to your environment (Need to mention admin server and manager server hostname and port. ) and copy to prometheus.yml. Restart Prometheus to pick latest configuration. You should be able to view metrics in Prometheus, usually at http://<hostname>:9090/graph. Metrics can also be checked from exporter web interface using url: http://serverhost:port/wls-exporter/metrics
  4. Configure Grafana: There are two steps here. Configuring Prometheus datasource in Grafana. This steps helps in pulling metrics data into Grafana. Please refer to official documentation for the detailed steps. Second step is to configure dashboards using Prometheus metric data. Copy the dashboard configuration from here. This configuration is taken from official Weblogic Monitoring Exporter page and made minor tweaks to it. Setps to import dashboards can be found here.

Sample Screenshots:

Overview
CPU and Heap Usage
JMS

Next Steps: We have covered gathering basic metrics and visualizing them in dashboards. Persistence store, JTA metrics etc can be added to cover complete monitoring. If you have SOA/OSB application, please use custom scripts mentioned in previous post to send time series metrics to Prometheus. These scripts need to be modified to suit to Prometheus. Grafana also supports alerting. So, setup alerts based on your application requirement.

One disadvantage that I noticed with Weblogic Monitoring Exporter is that we can export only runtime JMX metrics and can’t export config details. For example, one good metric to monitor datasource is connection utilization. For that, we need to know max allowed connections from connection pool which is not available with exporter (at least to my knowledge). Another approach is using telegraf Prometheus exporter as discussed in this post. To export using telegraf refer to previous post. Instead of sending metrics to Wavefront, we need to use Prometheus and Grafana.

Related Post