Monitoring SI Artifacts and Logs¶

The WSO2 Integration Control Plane (ICP) monitors WSO2 Streaming Integrator (SI) instances in a deployment. This can be a single SI instance or multiple instances in a group. It provides a graphical view of the artifacts deployed in the SI runtime. You can also perform monitoring and limited management tasks using the ICP server. This feature is available from the Streaming Integrator version 4.3.0 or higher.

The ICP server communicates with the SI instances via the ICP reporter module to collect runtime artifact and system data.

Capabilities of the ICP Server¶

You can use the ICP server to perform the following tasks related to your Streaming Integrator deployment:

Tip

You can search for certain artifacts using the specific name of the artifact if there are many such items in the list.

View the SI servers in the deployment

View basic information of each SI server node.
View Siddhi applications deployed in a group

View a list of deployed Siddhi applications and inspect each application.
Enable/Disable Siddhi applications

Control the state of Siddhi applications. Enabling or disabling a Siddhi application affects associated sources and sinks.
View Siddhi application details

View the source code and metadata of each application.
View runtime components inside Siddhi applications

Inspect individual Siddhi components including:
- Sources
- Sinks
- Queries
- Stores
- Windows
- Aggregations
View logs

View and download log files generated by each SI instance.

Using the Integration Control Plane¶

Follow the steps given below to configure and use the WSO2 Integration Control Plane with WSO2 Streaming Integrator.

Step 1 - Download the Integration Control Plane¶

Download the binary distribution of the product, and then follow the instructions to start the Streaming Integrator and the ICP server.

Note

Make sure get the latest update the of the Streaming Integrator 4.3.0 or higher. After updating check the JAR file org.wso2.carbon.si.management.icp_<version>.jar located in the <SI_HOME>/wso2/lib/plugins/ directory is version 3.0.75 or above. If an earlier version is present, follow the steps below:

Download the required version from this link.
Place the downloaded JAR file in the <SI_HOME>/wso2/lib/plugins/ directory.
Open the <SI_HOME>/wso2/server/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info file.
Update the existing entry for the org.wso2.carbon.si.management.icp_<VERSION>.jar JAR as follows, replacing the version and file name as necessary: org.wso2.carbon.si.management.icp,<version>,../lib/plugins/org.wso2.carbon.si.management.icp_<version>.jar,4,true

Step 2 - Configure the SI servers¶

Follow the steps given below to configure the SI servers to publish data to the ICP server.

To connect the SI servers with the ICP server, add the following configuration to the deployment.yaml file (stored in the <SI_HOME>conf/server/ folder) of each server instance. You can make the nodeID here unique. Also enable the metrics under the wso2.metrics section in the deployment.yaml file.

ICP configuration for SI node.

management.icp:
  reporting:
    icp:
    - name: icp
      enabled: true
      groupID: "si-001"
      nodeID: "node-00"

Enabling metrics

wso2.metrics:
    # Enable Metrics
    enabled: true

If the Streaming Integrator server is deployed in a Kubernetes environment, add the following configuration to the deployment.yaml file.

If communicating via Ingress, use the following configuration:

dashboardURL: "https://{hostname/ip}:{port}/dashboard/api/"

dashboardURL	Optional. This is the URL to access ICP server. Replace the hostname/IP and port (default - 9743) with relevant values from your environment.
heartbeatInterval	Optional. The time interval (in seconds) between two heartbeats sent from the Streaming Integrator to the ICP server. By default, the heartbeatInterval is set to `5`.
groupID	Optional. In a clustered deployment, the group ID should be the same in all Streaming Integrator Instances. The ICP server displays information from one group at a time. By default, the groupID is set to `si_dev`.
nodeID	Optional A unique ID that can be set for each node to identify a node in an multi instance environment.

Regardless of the user who logs in, the ICP server uses the user configured in its deployment.toml to fetch the data to the ICP server. Then the ICP server renders these data in the UI according to logged-in user. Hence, configure the super admin user credentials in the user store as mentioned below in the deployment.toml file (stored in the <ICP_HOME>/conf/ folder).
```
[si_service_account]
username = "admin"
password = "admin"
```

Step 3 - Start the ICP server¶

Follow the steps given below.

Open a terminal and navigate to the <ICP_HOME>/bin folder.
Execute one of the commands given below.
On MacOS/LinuxOn Windows
./dashboard.sh
dashboard.bat

Step 4 - Start the SI servers¶

Follow the steps given below.

Open a terminal and navigate to the <SI_HOME>/bin folder.
Execute one of the commands given below.
On MacOS/LinuxOn Windows
./server.sh
server.bat

Once you have set up and started the ICP server, you can access the ICP server URL.

Before you begin

Be sure to have at least one Streaming Integrator server connected to the ICP server before attempting to sign in to it. This can be verified by checking the presence of the following log.

New node <node_id> in group : <group_id> is registered. Inserting heartbeat information

Copy the following ICP server URL to your browser:
```
https://localhost:9743/login
```

Enter the following details to sign in.

login form for integration control plane

Username	The user name to sign in. Note: This should be a valid username that is saved in the Streaming Integrator server's user store. By default, the 'admin' user name is configured in the default user store.
Password	The password of the user name. By default, 'admin' is the user name and password.

Click Sign In.

You are redirected to the home page of the Integration Control Plane.

Step 6 - Monitor SI artifacts and logs¶

Follow the steps given below.

Select the group ID that you want to view from the upper left menu.

You can see the list of server nodes in each group, as shown in the above diagram.

Now you can view details of artifacts, update artifacts, and perform various other administration tasks. Select the required option from the left-hand navigator.

Monitoring health of ICP server¶

The Integration Control Plane provides a health check endpoint that can be used to monitor the status of the ICP server. The following API format can be used to invoke the health API.

https://localhost:9743/dashboard/api/healthz

If the ICP server is up and running properly, the following response will appear.

{"status":"ready"}

Monitoring SI Artifacts and Logs¶

Capabilities of the ICP Server¶

Using the Integration Control Plane¶

Step 1 - Download the Integration Control Plane¶

Step 2 - Configure the SI servers¶

Step 3 - Start the ICP server¶

Step 4 - Start the SI servers¶

Step 5 - Sign in to the ICP server¶

Step 6 - Monitor SI artifacts and logs¶

Monitoring health of ICP server¶