Step 6: Monitor Statistics¶

In this step, you monitor the CDC and file statistics of the SweetFactoryApp Siddhi application you deployed and ran in the previous steps. WSO2 Integrator: SI ships pre-configured Grafana dashboards for this purpose. You host the dashboards in Grafana, with Prometheus scraping metrics directly from the SI server. For more information about these dashboards, see Monitoring ETL Statistics with Grafana.

Prerequisites¶

The WSO2 Integrator: SI server is running, with SweetFactoryApp deployed (see Step 3: Deploy the Siddhi Application and Step 4: Run the Siddhi Application).

Download the pre-configured dashboards¶

WSO2 Integrator: SI provides pre-configured dashboards in JSON format. For this scenario, download the following:

Configure SI to expose metrics¶

Open <SI_HOME>/conf/server/deployment.yaml. In the wso2.metrics section, set enabled: true:

wso2.metrics:
  enabled: true
  reporting:
    console:
      - name: Console
        enabled: false
        pollingPeriod: 2

Add a metrics.prometheus section that exposes metrics on port 9005:

metrics.prometheus:
  reporting:
    prometheus:
      - name: prometheus
        enabled: true
        serverURL: "http://localhost:9005"

Save the file. Restart the SI runtime so the configuration takes effect — stop any running Siddhi applications and click Run again, or restart the SI server in its terminal (see Starting the WSO2 Integrator: SI server).

Install and configure Prometheus¶

Download Prometheus from the Prometheus download page.
Extract the downloaded archive. The extracted directory is referred to as <PROMETHEUS_HOME> from here on.

Open <PROMETHEUS_HOME>/prometheus.yml and add the SI metrics endpoint as a scrape target under scrape_configs:

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9005']

From the <PROMETHEUS_HOME> directory, start Prometheus on port 9091 (the default port 9090 conflicts with SI's HTTP interface, so specify an alternate port):
```
./prometheus --web.listen-address=:9091
```
Verify that Prometheus is scraping SI. Open http://localhost:9091/targets in a browser. The prometheus job should appear with its localhost:9005 endpoint listed as UP.

Install and configure Grafana¶

Download Grafana from Grafana Labs - Download Grafana, then start it according to your installation method. For example, on macOS with Homebrew, run brew services start grafana.
Open http://localhost:3000 in a browser. Sign in with the default credentials (admin / admin) and set a new password when prompted.
Add Prometheus as a data source:
1. In the left sidebar, open Connections → Data sources.
2. Click + Add new data source and select Prometheus.
3. In the URL field under Connection, enter http://localhost:9091 (the port where you started Prometheus in the previous section).
4. (Optional) Enable Default to make Prometheus the default data source.
5. Click Save & test. A success message confirms the data source is reachable.
Import each dashboard JSON file you downloaded earlier:
1. In the left sidebar, open Dashboards → New → Import.
2. Click Upload dashboard JSON file and select one of the dashboard files.
3. (Optional) Change the Unique Identifier (uid) if prompted.
4. Click Import.
5. Repeat for each of the seven dashboard files.

Enable the Siddhi application to publish statistics¶

By default, SI emits only server-level metrics (JVM, uptime, HTTP transports). To also emit per-Siddhi-application metrics — which the pre-configured SI app, file, and CDC dashboards depend on — add the @App:statistics annotation to the Siddhi application immediately below the @App:name annotation:

@App:name('SweetFactoryApp')
@App:statistics(reporter = 'prometheus')

Redeploy the application (see Step 3: Deploy the Siddhi Application) for the annotation to take effect.

Note

Without the @App:statistics annotation, the dashboards render but show no per-app data. This is the most common reason for empty SI app, file, or CDC dashboards.

View the statistics¶

Insert records into the production database table to generate CDC events:
```
insert into SweetProductionTable values('eclairs', 100.0);
insert into SweetProductionTable values('toffee', 40.0);
insert into SweetProductionTable values('gingerbread', 200.0);
```
(You can also test the file-source → Kafka path directly by appending rows to <YOUR_HOME>/productioninserts.csv, which the FilterStream @source(type='file') reads. See Step 5: Update the Siddhi Application for the full pipeline.)
Open Grafana, then navigate to Dashboards in the left sidebar.

Open the WSO2 Integrator: SI - Overall Statistics dashboard:

Info

The statistics displayed depend on the number of events generated in the configured time window. Adjust the time-range selector in the top-right if your recent events do not appear.
Under Overview Statistics, click SweetFactoryApp to open the WSO2 Integrator: SI App Statistics dashboard.
Scroll down to the Sources section to see the file and cdc sources used by SweetFactoryApp:
Scroll down to the Destinations section to see the file sink:
Under Sources, click the link to the productioninserts.csv file. The WSO2 Integrator: SI - File Statistics dashboard opens. Because the file is the output of one query and the input of another, statistics appear under both Source and Sink:

Source statistics

Sink statistics
On the File Statistics dashboard, under Sources, click the file link to open the File Source Statistics dashboard — detailed statistics for the file functioning as a source:
On the File Statistics dashboard, under Destinations, click the file link to open the File Sink Statistics dashboard — detailed statistics for the file functioning as a sink:
On the App Statistics dashboard, in the CDC section, click the SweetProductionTable link. The CDC Statistics dashboard opens with statistics generated for the cdc source:

Under Streaming, click SweetProductionTable. The CDC Streaming Statistics dashboard opens:

What's Next?

To package the SweetFactoryApp Siddhi application as a Docker artifact for containerized deployment, proceed to Step 7: Export the Siddhi Application as a Docker Artifact.
To learn more about the key concepts of WSO2 Integrator: SI, see Key Concepts.
For more hands-on experience, try the Tutorials.
For guidance on common Streaming Integration use cases, see Use Cases.
To run WSO2 Integrator: SI in containerized environments, see Running SI with Docker and Kubernetes.

Edit this page

Last update: May 14, 2026