Optimize And Tuning WebLogic Server Performance: Tuning JDBC Data Source Connection Pool

High performance WebLogicapplication and system influenced by many factors. One of important factor is good and properly JDBC data sources setting in WebLogicServer domain. Bellow check list can be considered while tuning a JDBC data source:

Setting WebLogic Connection Pool Size

Maximum Capacity

Maximum capacity is number of allocated session in database side for WebLogic Server divide by number of WebLogic server. Example, DBA allocated 500 session for WebLogic server which has 5 server, so this config value is 500/5 = 100

Initial Capacity

Initial capacity mean number of initial session created when data source started. Recommended value for Initial Capacity is equal to Maximum Capacity size database connection pools. In this case WebLogicnever increase the number of connections when it needed by application. Negative impact for this case is increase DB load although only few session use by WebLogic, but if impact in database side is not significant its work well.

Capacity Increment

There is no rule about how many session added when need more connection from application to database, but need to make sure that database can be handle this increment session login. This value should be less than 10 to increase speed of session increment.

Store the Prepared Statements in the Statement Cache

Every statement from an application or EJB, there is a processing cost at application server, database server and network communication between both. This cost can be minimizing using Statement Cacheapproach by cache statements used in application. In the next time WebLogic will be reuses the statement stored in the cache. Using cache will be improve application and system performance by reduce CPU usage in application server side and database server side.

Statement Cache Type

Two algorithm options available for maintaining the prepared statements stored in the statement cache

• LRU (Least Recently Used)

LRU is default option for Statement Cache Type. In this algorithm, WebLogic Server caches statements used on the connection until the statement cache size is reached. When an application calls prepare Statement, WebLogic Server checks to see if the statement is stored in the statement cache. If cache is available (currently not being used), WebLogic Server returns the cached statement, else (statement is not in the cache, and the cache is full), WebLogic Server determines which existing statement in the cache was the least recently used and replaces that statement in the cache with the new statement.

• Fixed

In Fixed algorithm, WebLogic Server caches statements used on the connection until the statement cache size is reached. When other statements are used, they are not cached, even though the statement is frequently used. Weakness of this algorithm is WebLogic will cache infrequently used statements.

In most of WebLogic implementation LRU algorithm is preferred because caches only save frequently used statements, so goal of Statement caching is reach.

Statement Cache Size

Statement Cache Size shows the total number of statements to cache for each connection in each instance of the data source. WebLogic Server can reuse statements in the cache without reloading the statements, which can increase server performance. Each connection in the connection pool has its own cache of statements.

Setting the statement cache size = 0 mean that disable statement caching. For example, if a data source with 10 connections deployed on 5servers, and setting Statement Cache Size = 10, Database maximum will be open 500 cursors (10 Conn x 2 server x size 10) on database server for the cached statements.

WebLogic statement caches related to database cursor. The statement may open a cursor in the database. If cache contains too many statements, database connection may reach maximum limit of open cursors for. Avoiding this case can be done in two approaches; increase maximum open cursors allowed for a connection or reduce Statement Cache Size setting.

The conclusion for ideal number of Statement Cache Size is depends on bellow factor:

• Number of servers which data source deployed
• Maximum connection for data source
• Maximum cursor open allowed on database side

Connection Testing Options for a Data Source

The purpose of connection testing is to make sure that the database connections in a data source in a health status healthy. Connection testing should execute periodically in a specific interval time.

There are two methods to testing a data source connection in WebLogic Server:

• Manual testing

Testing by manually execute connection testing. Usually do while trouble-shoot a data source for a problem. To do manual testing, go to Tab Monitoring, and then tab Testing. Choose data source which want to test and click “Test Data Source”. If there is no problem will be shown a notification Test of SUGI-DB on server MedRecServer was successful.

• Automatic testing

Configure options on the data source so that WebLogic Server will be periodically test to makes sure that database connections remain healthy.

Test Connections on Reserve

This option will be enables WebLogic Server to test a connection before giving it to a client application. Test connection done by WebLogic using SQL statement based on value of Test Table Name. This connection test adds a small delay in serving the client’s request for a connection from the pool, but ensures that the client receives a reliable connection.

Test Frequency

Test Frequency is time interval in seconds WebLogic Server tests unused connections. Connections that fail the test are closed, re-establish a valid physical connection to database, and test the new connection before returning it to the pool. If the test fails again, the connection is closed. If value set to 0, periodic testing is disabled.

Test Table Name

Test table name is config for SQL Statement which used when testing physical database connections. This value is required when Test Frequency value more than 0 and Test Reserved Connections is enable. The SQL code used to test a connection is depends on database provider. Best practice for this value is fastest SQL statement to the database. Example in ORACLE database, SQL SELECT 1 FROM DUAL is super-fast SQL statement which can be used in this option. Test Table Name must begin with string “SQL “, and then followed by SQL statement.

Maintain Connection to Data Source

Seconds to Trust an Idle Pool Connection

This option can minimize the impact of testing reserved connections. The appropriate value depends on system environment. Too low setting will be poor because during heavy traffic connection testing is give more loads to database, but if value is set too high will be reduce effectiveness of connection testing itself. Default value for this config is 10.

Shrink Frequency

The number of seconds to wait before shrinking a connection pool that has incrementally increased to meet demand. Shrinking means that WebLogic drop some connections from the data source when a peak usage period has ended, freeing up WebLogic Server and DBMS resources. When set to 0, shrinking is disabled.

Init SQL

Init SQL statement will be execute while WebLogic initialize newly created physical database connections in a data source. Default value is blank, but for specific purposes, this value can be set based on Database Provider Script. For example in Oracle, you can set Init SQL with alter session. Init SQL must begin with string “SQL “, and then followed by SQL statement.

Connection Creation Retry Frequency

This value represent interval in seconds between attempts to establish connections to the database. If the database is unavailable when the data source is created, WebLogic Server will attempt to create connections in the pool again after the number of seconds you specify, and will continue to attempt to create the connections until it succeeds. When set to 0, connection retry is disabled. Recommended to set this value to a desired seconds. The exact value need suggest from Database Administrator because related to database environment behavior.

Login Delay

Login delay means interval in seconds before creating each physical database connection. Recommended to set the value with 0 (no delay), but if database cannot handle multiple connection requests it means that need delay during creating database connection.

Inactive Connection Timeout

Inactive connection in WebLogic Server will be releases and back into the connection pool if this value set more than 0. Recommendation for this config is specific value in seconds, so if found leaked connections that were not correctly closed by the application will be handle via this feature.

Maximum Waiting for Connection

The maximum number of connection requests that can concurrently block threads while waiting to reserve a connection from the data source’s connection pool. Leave this setting with default value (Maximum value).

Connection Reserve Timeout

When an application requests a connection from a data source, if all connections in the data source are in use and if the data source has expanded to its maximum capacity, the application will get a Connection Unavailable SQL Exception. Configure the Connection Reserve Timeout value (in seconds) will be affected that connection requests will wait for a connection to become available. After the Connection Reserve Timeout has expired, if no connection becomes available, the request will fail and the application will get a PoolLimitSQLException exception.  To set connection request will timeout immediately, set value = -1. To set a connection request will wait until connection available, set value = 0. Fair value is 10 seconds by default.

Statement Timeout

Statement currently being executed will time out if consume execution time more than this setting. Set timeout via this setting will impacted to every statement in this data source. It recommended that setting timeout in application adapter depends on every statement. But also recommended to set this value also with upper limit of timeout define by business process to anticipate application adapter with no timeout.

Ignore In-Use Connections

This option enables the data source to be shutdown even if connections obtained from the pool are still in use. Enable or disable this option depends on policy in the system, but default value is enable.

Pinned-To-Thread

PinnedToThread is an option that can improve performance by enabling execute threads to keep a pooled database connection even after the application closes the logical connection. Default value is disable. This option can be enable and will be useful if implemented with correct calculation between WebLogicresource and database resource because related to reserved thread.

Remove Infected Connections Enabled

Default and safe value is enable which means the physical connection is not returned to the connection pool after the application closes the logical connection. Instead, the physical connection is closed and recreated.

WebLogic – Database Connection Pinned-to-Thread

Using Pinned-To-Thread Property to Increase Performance

To minimize the time it takes for an application to reserve a database connection from a data source and to eliminate contention between threads for a database connection, you can add the Pinned-To-Thread property in the connection Properties list for the data source, and set its value to true.

When Pinned-To-Thread is enabled, WebLogic Server pins a database connection from the data source to an execution thread the first time an application uses the thread to reserve a connection. When the application finishes using the connection and calls connection.close(), which otherwise returns the connection to the data source, WebLogic Server keeps the connection with the execute thread and does not return it to the data source. When an application subsequently requests a connection using the same execute thread, WebLogic Server provides the connection already reserved by the thread. There is no locking contention on the data source that occurs when multiple threads attempt to reserve a connection at the same time and there is no contention for threads that attempt to reserve the same connection from a limited number of database connections.

 

 

Note:

In this release, the Pinned-To-Thread feature does not work with multi data sources, Oracle RAC, and IdentityPool. These features rely on the ability to return a connection to the connection pool and reacquire it if there is a connection failure or connection identity does not match.

 

 

Changes to Connection Pool Administration Operations When PinnedToThread is Enabled

Because the nature of connection pooling behavior is changed when PinnedToThread is enabled, some connection pool attributes or features behave differently or are disabled to suit the behavior change:

• Maximum Capacity is ignored. The number of connections in a connection pool equals the greater of either the initial capacity or the number of connections reserved from the connection pool.
• Shrinking does not apply to connection pools with PinnedToThread enabled because connections are never returned to the connection pool. Effectively, they are always reserved.
• When you Reset a connection pool, the reset connections from the connection pool are marked as Test Needed. The next time each connection is reserved, WebLogic Server tests the connection and recreates it if necessary. Connections are not tested synchronously when you reset the connection pool. This feature requires that Test Connections on Reserve is enabled and a Test Table Name or query is specified.

Additional Database Resource Costs When PinnedToThread is Enabled

When PinnedToThread is enabled, the maximum capacity of the connection pool (maximum number of database connections created in the connection pool) becomes the number of execute threads used to request a connection multiplied by the number of concurrent connections each thread reserves. This may exceed the Maximum Capacity specified for the connection pool. You may need to consider this larger number of connections in your system design and ensure that your database allows for additional associated resources, such as open cursors.

Also note that connections are never returned to the connection pool, which means that the connection pool can never shrink to reduce the number of connections and associated resources in use. You can minimize this cost by setting an additional driver parameter onePinnedConnectionOnly. WhenonePinnedConnectionOnly=true, only the first connection requested is pinned to the thread. Any additional connections required by the thread are taken from and returned to the connection pool as needed. Set onePinnedConnectionOnly using the Properties attribute, for example:

Properties="PinnedToThread=true;onePinnedConnectionOnly=true;user=examples"

If your system can handle the additional resource requirements, Oracle recommends that you use the PinnedToThread option to increase performance.

If your system cannot handle the additional resource requirements or if you see database resource errors after enabling PinnedToThread, Oracle recommends not using PinnedToThread.

Start Weblogic without user and password

When you want to launch your weblogic server at production, you need to use the startWeblogic.sh in your bin folder.

But each time it’s launched it’s asking you to authenticate yourself by typing a user and password. It’s a problem when you want to automate the execution of your server…

To avoid this problem all you need to do is creating a file named “boot.properties“ and insert into the two following lines :

username=<yourUserName>
password=<yourPassword>

this file must be placed in each server security folder :$WLS_HOME/user_projects/domains/<domainName/servers/<serverName>/security

Then, start server, it shouldn’t ask you for anythin, then reopen your boot.properties file, password and username should be automatically encrypted !

Avoid java out of memory with Weblogic

This is common exception always exists if you install weblogic and doesn’t extend memory arguments in server.

The file “setDomainEnv.sh” in $WLS_HOME/user_projects/domains/<domainName>/bin/ has configuration of domain.

If you edit this file it will have the following default values.

MEM_ARGS=”-Xms256m -Xmx512m”

export MEM_ARGS

MEM_PERM_SIZE=”-XX:PermSize=48m”

export MEM_PERM_SIZE

MEM_MAX_PERM_SIZE=”-XX:MaxPermSize=128m”

export MEM_MAX_PERM_SIZE

you should modify the MEM_ARGS java memory value depending of your server, here is the suggested to increase.

MEM_ARGS=”-Xms2024m -Xmx3036m”

export MEM_ARGS

MEM_PERM_SIZE=”-XX:PermSize=128m”

export MEM_PERM_SIZE

MEM_MAX_PERM_SIZE=”-XX:MaxPermSize=512m”

export MEM_MAX_PERM_SIZE

Note : choosing the values depend on you server hardware.

Redirecting JVM Output to Server Logging Destinations

The JVM in which a WebLogic Server instance runs, sends messages to standard error and standard out. Server as well as application code write directly to these streams instead of using the logging mechanism. Through a configuration option, you can redirect the JVM output to all the registered log destinations, like the server terminal console and log file. When enabled, a log entry appears as a message of NOTICE severity. Redirecting the JVM output does not capture output from native code, for example thread dumps from the JVM.

1)To redirect the JVM standard out messages:

When you start the Administration Server, include the following Java option in the weblogic.Server command:

-Dweblogic.log.RedirectStdoutToServerLogEnabled=true

2)Redirecting Stdout to Server Logging Destinations

C:\>java weblogic.WLST
wls:/offline> connect(‘username’,'password’)
wls:/mydomain/serverConfig> edit()
wls:/mydomain/edit> startEdit()
wls:/mydomain/edit !> cd(“Servers/myserver/Log/myserver”)
wls:/mydomain/edit/Servers/myserver/Log/myserver !> cmo.setRedirectStdoutToServerLogEnabled(true)
wls:/mydomain/edit/Servers/myserver/Log/myserver !> save()
wls:/mydomain/edit/Servers/myserver/Log/myserver !> activate()

Thread Dump in WebLogic Server

Thread dumps are very useful to analyze and troubleshoot performance related issues such as server hang, deadlocks, slow running, idle or stuck applications,  etc.

Different ways to take thread dumps in WebLogic Server

Always prefer Operating system(OS) commands rather instead of Admin Console or Java Classes, because if the console is hanging, users won’t be able to connect to it to issue thread dumps.

1. OS Commands for Thread Dumps

i) On Windows,

<ctrl>+<break> — the thread dumps are generated in the server stdout

ii) On Solaris / Linux

first identify the process ID (pid) using   ps -ef | grep java, then run   kill -3 <pid> td_filename 2>&1

2. Using weblogic.WLST ( work only from WLS 9.x onwards)

First set CLASSPATH using setDomain.cmd or setDomain.sh (wlst.sh /wlst.cmd will also do in path C:\Oracle\Middleware\wlserver_12.1\common\bin ). Then run below command

java weblogic.WLST ThreadDumps.py

save below code in ThreadDumps.py file:
connect(“<username>”,”<password>”,”t3://<url>:<port>”)
cd(‘Servers’)
cd(‘AdminServer’)
threadDump()
disconnect()
exit()
threadDump()

The thread dumps get stored in the location from where you run it.

3. From Weblogic Administration Console

navigating to Server -> <server_name> -> Monitoring -> Dump threads stack.

4. From the JRockit Command line

jrcmd <pid> print_threads

How to enable the Heapdump in Weblogic Server

In order to get Heap-dump to be generated in Weblogic server  we have to set the following parameters in server start-up argument
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=C:\Heapdump

Thread Related Definitions

• CompletedRequestCount. Number of completed requests in the priority queue.
• ExecuteThreadIdleCount. Number of idle threads in the pool. This count does not include standby threads and stuck threads. The count indicates threads that are ready to pick up new work when it arrives.
• ExecuteThreadTotalCount. Total number of threads in the pool.
• HoggingThreadCount. Number of threads that are being hogged by a request right now. These threads will either be declared as stuck after the configured timeout or will return to the pool before that. The self-tuning mechanism will backfill if necessary.
• MinThreadsConstraintsCompleted. Number of requests with min threads constraint picked up out of order for execution immediately since their min threads requirement was not met. This does not include the case where threads are idle during schedule.
• MinThreadsConstraintsPending. Number of requests that should be executed now to satisfy the min threads requirement.
• PendingUserRequestCount. Number of pending user requests in the priority queue. The priority queue contains requests from internal subsystems and users. This is just the count of all user requests.
• QueueLength. Number of pending requests in the priority queue. This is the total of internal system requests and user requests.
• StandbyThreadCount. Number of threads in the standby pool. Surplus threads that are not needed to handle the present work load are designated as standby and added to the standby pool. These threads are activated when more threads are needed.
• Throughput. Mean number of requests completed per second

Hogging Threads in Weblogic Server

Hogging threads are candidates for stuck threads.

Threads that “might” get stuck. These threads will be declared “stuck” after StuckThreadMaxTimeout seconds which usually is 600secs.

If the thread gets released by the request before this timeout, it wont be called hogging thread anymore and will be released to the thread pool.

Hogging Threads that have taken the too much time and we can assume that they are never going to come back.
Hogging threads help us take some decisions, lets say many threads are hogging, we may take a decision to create new threads for next cycle.

Understanding about Thread States in WebLogic Server:
ACTIVE
STUCK
STANDBY

A live thread which is ready to process the request, which is known as ACTIVE state. That is indicated when thread newly created. WebLogic Server start the server instance with 1 ACTIVE thread and the thread count grows as per the min size if specified other wise it will do self-tune as per the request.

Threads might wait for other thread to release resource. This might happen due to application variables. The variables are 2 types thread-safe other is risk for thread. All local variables in the methods are thread-safe. The variable defined in class level are unsafe. which causes memory leak, this state of threads are known as hogging. WebLogic identify a thread as hog by the time interval. If thread is waiting more than 600 sec will be treated as hog. STUCK thread interval we can tune as per the project need.
If the number of HoggingThreadCount increases then the server health is in dangerous. That time you can take the ThreadDump
After Threads increase to a max utilization then the thread will be in STANDBY state.

WebLogic Server Health Status can be one of the following:

HEALTH_OK
HEALTH_WARN
HEALTH_FAILED
HEALTH_CRITICAL
LOW_MEMORY_REASON
HEALTH_OVERLOADED

OK is indicates everything fine, no worries!!

WARN raised when there is few stuck threads in the server instance.

LOW_MEMORY_REASON is going to tell you about JVM crash expected. You can configure to ‘Exit’ the managed server on low memory conditions with the help of NodeManager and WorkManager.

CRITICAL when multiple number of stuck threads happening and the threadpool count reaching unsual number. This case you need to suspect Network, JDBC or back-end connectivity has trouble.

FAILED happen when the new deployments fails. The NodeManager should not restart this managed server.

OVERLOADED Change the server health state to OVERLOADED on overload. The Nodemanager need to work at this state and bounce such WebLogic instance. This is a new feature of WebLogic 9.x and later versions, for detecting, avoiding and recovering from an overload condition of a WebLogic managed server. Overload protection can be used to throttle Work Managers and thread pools for performance. You can configure Shutdown the Work Manager or application on stuck threads when it crosss more than 5 or you can set threshold.

Thread Stuck Reasons

A stuck thread means there is a thread that is blocking for some conditions and it cannot be returned to the original thread pool (execute queue). The modern design of many application servers utilizes the concept about thread pool to maximize the whole performance. When an execute thread is blocking for something, it will mean it cannot be quickly released and repeatedly reused one after another.In most of production situations, the root cause of these stuck threads is also the root cause of the system performance degradation.

- Network connecting timeout/Read timeout
- Long SQL query execution time
- Resource contention
- Connection leak
- User threads blooming
- Excessive Full GC pause time in the application code using System.gc() RMI daemon threads or memory leaks
- Internal transaction process by a slow resource manager.
- A huge value object RMI serialization/de-serialization in the network or ‘call by value’ semantic in applications.
- Unnecessary synchronized modifier in the code.
- Remote invocation hang or a slow response of remote call of another server.
- A huge write to a very slow client.
- A loop condition in the application code even server code.
- Threads with high CPU usage exhaust the whole CPU power.
- A premature version of JVM during bytecode optimization or garbage collection.

Weblogic Monitoring Available statistics

Server. Allows monitoring the open sockets for every server.

• SocketsOpenedTotalCount. Total number of opened sockets for selected server.
• OpenSocketsCurrentCount. Number of currently opened sockets for selected server.

Session. Allows monitoring session counters for every EAR and WAR deployed on a server.

• OpenSessionsCurrentCount. Total number of currently opened sessions in a selected module.
• OpenSessionsHighCount. Highest number of opened sessions in a selected module. The counter starts from zero every time the module is activated.
• SessionsOpenedTotalCount. Total number of sessions opened in a selected module.

JDBC Connection Pool. Allows monitoring of JDBC connection pool counters. The counters are available at connection pool level.

• ActiveConnectionsCurrentCount. Number of physical database connections in a selected connection pool.
• WaitingForConnectionCurrentCount. Number of application requests currently waiting for a connection.
• NumUnavailable. Number of connections currently unavailable because they are being tested or refreshed.
• ActiveConnectionsAverageCount. Average number of active connections in the selected connection pool.
• ConnectionDelayTime. Average time taken, in milliseconds, needed to create a physicaldatabase connection. The time is calculated as the sum of the time taken to create all physical database connections in the connection pool, divided by the total number of connections created.
• ActiveConnectionsHighCount. Highest number of active database connections in the selected connection pool since it was instantiated. Active connections are connections in use by an application.
• ConnectionsTotalCount. Total number of database connections created in the selected connection pool since it was instantiated.
• CurrCapacity. Current number of database connections the selected connection pool (includes available connections, connections in use and connections unavailable).
• FailuresToReconnectCount. Number of times the selected connection pool attempted to refresh a connection to a database and failed. This can occur when the database is down or due to network problems.
• HighestNumUnavailable. Highest number of unavailable connections in the selected connection pool since it was instantiated.
• WaitSecondsHighCount. Highest number of seconds an application waited for a connection from the selected connection pool since it was instantiated.
• WaitingForConnectionHighCount. Highest number of application requests concurrently waiting for a connection from the selected connection pool.

JRockit. Allows monitoring the runtime counters of each JRockit Virtual Machine instance.

• AllProcessorsAverageLoad. Average load of all host machine processors.
• JvmProcessorLoad. The load of Java Virtual Machine on all host machine processors. When there’s more then one processor exists on the host machine, this counter will give t he average value.
• HeapFreeCurrent. Current available memory in Virtual Machine (in bytes).
• HeapSizeCurrent. Current memory size of the Virtual Machine (in bytes).
• FreeHeap. Amount of currently free memory of Java heap in Virtual Machine (in bytes).
• UsedHeap. Amount of currently used memory of Java heap by the Virtual Machine (in bytes).
• TotalHeap. Amount of heap memory, in bytes, currently allocated to the Virtual Machine.
• FreePhysicalMemory. Amount of currently free physical memory on the host machine (in bytes).
• UsedPhysicalMemory. Amount of currently used physical memory on the host machine (in bytes). The value represents the memory used by all processes on the machine, not just the JVM.
• TotalPhysicalMemory. Amount of physical memory on the host machine. Swap or other types of virtual memory are not included in this value.
• NumberOfDaemonThreads. Total number of daemon Java threads currently running in Virtual Machine in all processors.
• NumberOfProcessors. Number of processors on computer where Java Virtual Machine is running. For systems that are not Symmetric Multi Processor (SMP), value will always be 1.
• TotalGarbageCollectionCount. Number of garbage collection runs since the Virtual Machine was started.
• TotalGarbageCollectionTime. Number of milliseconds spent on all garbage collection runs since the Virtual Machine was started.
• TotalNumberOfThreads. Total number of Java deamon and non-deaemon threads currently running in the Virtual Machine. This is summary value for all processors.
• TotalNurserySize. Amount of memory currently allocated to the nurseryn (in bytes). When non-generational garbage collector is in use thsi value is 0.

JVM. Allows monitoring JVM machine counters.

• HeapFreeCurrent. Available memory in bytes for this Virtual Machine.
• HeapSizeCurrent. Total memory size in bytes for this Virtual Machine.

Connector.

• ConnectionPoolsTotalCount. Total size of the connection pool.
• ConnectionPoolCurrentCount. Current size of the connection pool.

JMS Total. Allows monitoring JMS counters for all JMS Servers of the selected WebLogic server.

• ConnectionsCurrentCount. Current number of connections to the selected JMS server.
• ConnectionsHighCount. Highest number of connections to the selected JMS server since the last reset.
• ConnectionsTotalCount. Total number of connections made to the selected JMS server since the last reset.
• JMSServersCurrentCount. Current number of JMS servers deployed on this WebLogic Server instance.
• JMSServersHighCount. Highest number of JMS servers deployed to the selected WebLogic Server instance since it has started.
• JMSServersTotalCount. Total number of JMS servers deployed to the selected WebLogic Server instance since it has started.

JMS Servers. Allow monitoring each JMS Server on the selected WebLogic server.

• DestinationsCurrentCount. Number of current destinations for the selected JMS server.
• DestinationsHighCount. Highest number of destinations on the selected JMS server since the last reset.
• DestinationsTotalCount. Number of destinations instantiated on the selected JMS server since the last reset.
• MessagesCurrentCount. Number of messages currently stored on the selected JMS server. This number does not include the pending messages.
• MessagesHighCount. Highest number of messages stored in the selected JMS server since the last reset.
• MessagesPendingCount. Number of unacknowledged or uncommitted messages currently pending stored on the selected JMS server. Pending messages are over and above the current number of messages.
• MessagesReceivedCount. Number of messages received on this destination since the last reset.
• SessionPoolsCurrentCount. Number of session pools currently instantiated on the selected JMS server.
• SessionPoolsHighCount. Highest number of session pools instantiated on the selected JMS server since the last reset.
• SessionPoolsTotalCount. Number of session pools instantiated on the selected JMS server since the last reset.

JTA. Allows monitoring the transaction counters for the selected WebLogic server. All values are calculated per interval.

• TransactionTotalCount. Total number of transactions processed by the selected server.
• TransactionCommittedTotalCount. Total number of transactions committed by the selected server.
• TransactionRolledBackTotalCount. Total number of transactions rolled back by the selected server.
• TransactionRolledBackTimeoutTotalCount. Number of transactions rolled back by the selected server due to a timeout expiration.
• TransactionRolledBackResourceTotalCount. Number of transactions rolled back by the selected server due to a resource error.
• TransactionRolledBackAppTotalCount. Number of transactions rolled back by the selected server due to an application error.
• TransactionRolledBackSystemTotalCount. Number of transactions rolled back by the selected server due to an internal system error.
• TransactionHeuristicsTotalCount. Total number of transactions completed with a heuristic status.
• TransactionAbandonedTotalCount. Total number of transactions abandoned by the selected server.
• ActiveTransactionsTotalCount. Number of active transactions on the selected server.
• SecondsActiveTotalCount. Average amount of time (in milliseconds) that transactions coordinated by the selected server have taken to commit.

Queue. Allows monitoring counters for each WebLogic Queue of the selected WebLogic Server.

• PendingRequestCurrentCount. Number of waiting requests in the queue.
• ExecuteThreadTotalCount. Total number of execute threads assigned to the queue.
• ServicedRequestTotalCount. Number of requests that have been processed by the queue.
• ExecuteThreadCurrentIdleCount. Number of idle threads assigned to the queue.

EJB Pool. Monitoring EJB instances for Entity Beans, MessageDriven Beans and Stateless Beans.

• AccessTotalCount. Total number of times an attempt was made to get an instance from the free pool.
• BeansInUseCurrentCount. Number of bean instances currently being used from the free pool.
• DestroyedTotalCount. Total number of times a bean instance from this pool was destroyed due to a non-application exception being thrown from it.
• MissTotalCount. Total number of times a failed attempt was made to get an instance from the free pool. An Attempt to get a bean from the pool will fail if there are no available instances in the pool.
• PooledBeansCurrentCount. Number of currently available bean instances in the free pool.
• TimeoutTotalCount. Total number of threads that have timed out waiting for an available bean instance from the free pool.
• WaiterCurrentCount. Number of threads currently waiting for an available bean instance from the free pool.

EJB Transaction. Allows monitoring transaction counters for Entity Beans, MessageDriven Beans, Stateless Beans and Stateful Beans.

• TransactionsCommittedTotalCount. Total number of transactions that have been committed for the selectedEJB.
• TransactionsRolledBackTotalCount. Total number of transactions that have been rolled back for the selected EJB.
• TransactionsTimedOutTotalCount. Total number of transactions that have timed out for the selected EJB.

EJB Cache. Allows monitoring cache counters for Entity Beans and Stateful Beans.

• ActivationCount. Total number of beans from the selected EJB that have been activated (beans that have been deserialized by the application server).
• CacheAccessCount. Total number of attempts to access a bean from the cache.
• CachedBeansCurrentCount. Total number of beans from the selected EJB currently in the EJB cache.
• CacheMissCount. Total number of times an attempt to access a bean from the cache failed.
• PassivationCount. Total number of beans from the selected EJB that have been passivated (beans that have been serialized by the application server).

Thread. Monitors thread pool information for the WebLogic Server.

• CompletedRequestCount. Number of completed requests in the priority queue.
• ExecuteThreadIdleCount. Number of idle threads in the pool. This count does not include standby threads and stuck threads. The count indicates threads that are ready to pick up new work when it arrives.
• ExecuteThreadTotalCount. Total number of threads in the pool.
• HoggingThreadCount. Number of threads that are being hogged by a request right now. These threads will either be declared as stuck after the configured timeout or will return to the pool before that. The self-tuning mechanism will backfill if necessary.
• MinThreadsConstraintsCompleted. Number of requests with min threads constraint picked up out of order for execution immediately since their min threads requirement was not met. This does not include the case where threads are idle during schedule.
• MinThreadsConstraintsPending. Number of requests that should be executed now to satisfy the min threads requirement.
• PendingUserRequestCount. Number of pending user requests in the priority queue. The priority queue contains requests from internal subsystems and users. This is just the count of all user requests.
• QueueLength. Number of pending requests in the priority queue. This is the total of internal system requests and user requests.
• StandbyThreadCount. Number of threads in the standby pool. Surplus threads that are not needed to handle the present work load are designated as standby and added to the standby pool. These threads are activated when more threads are needed.
• Throughput. Mean number of requests completed per second

Channel. This section allows the user to monitor the channels (network access points from the WebLogic Server). The counters are available for each channel.

• AcceptCount. Number of sockets that have been accepted on this channel. This includes sockets both past and present so gives a good idea of the connection rate to the server.
• ConnectionsCount. Number of active connections and sockets associated with this channel.
• MessagesReceivedCount. Number of messages received on this channel.
• MessagesSentCount. Number of messages sent on this channel.