Looking at SMF data for problem determination

Audience level

Some knowledge of MQ or z/OS

Skillset

MQ Administration, z/OS systems programming

Background

MP1B is a utility provided by IBM to analyze your IBM MQ environment’s performance. MP1B shows you your SMF performance data and allows you to roll it off platform to CSV files for further analysis.

MP1B is installable at Link

Out of the box, it contains:

MQCMD – a program to display queue statistics and channel status over time

MQSMF – a program for interpreting your own accounting and statistics data

OEMPUT - a program to put/get messages in high quantities, useful for testing throughput

Overview of exercise

I. Set up the local queue MP1B.TESTER

II. Make sure settings are in place to record SMF data

III. Run JCL to record our SMF data

IV. Navigate the SMF data output to find performance problems in our queue

V. Interpret the performance problem

Video tutorial of the following exercise: Link

Exercise

MP1B has been installed on this environment, and you can find it by searching for the directory ZQS1.MP1B.JCL in the =3.4 data set search bar.
Now, outside of z/OS, open up MQ Explorer on your Windows Desktop. The icon should look like this:
Once you’ve opened MQ Explorer, you should see a left-hand menu bar like below. Right click on the ZQS1 queue manager and hit ‘Connect’.
By clicking on the arrow to the left of ZQS1, a dropdown list of MQ objects will appear. Right click on the ‘Queues’ folder and construct a new local queue called MP1B.TESTER.
Create a queue on your queue manager using MQ Explorer. The queue should have the following properties:

Why make the queue shareable? Great question! Shareable queues tend to come in handy in a test environment, so that developers can browse the queues.

Now that we have our queue defined, head back to z/OS.
Now, we will enter a series of MVS commands to adjust the settings of the queue manager to prepare it for the collection of SMF data. To do this, navigate to the ISPF main menu
Once in the ISPF main menu, enter ‘d’ in the command line and hit enter
Once in SDSF, place a / in the command input line and hit enter
A MVS command prompt like this should pop up:
Enter the following commands here, one at a time. Each command will take you out of the System Command Extension window, so you will have to use the / command to return to the correct window for executing commands.
```
ZQS1 SET SYSTEM STATIME(1.00)
```
To change the statistics time interval to 1 minute
```
ZQS1 SET SYSTEM ACCTIME(-1)
```
To change the accounting time interval to match the statistics time interval
```
ZQS1 SET SYSTEM LOGLOAD(200)
```
To change the log load attribute to the minimum.

We want to modify our queue manager’s log load attribute to be super low in order to manufacture a lot of checkpointing so we see something interesting in the SMF records for the purpose of the lab
```
DISPLAY SMF
```
This tells us where our SMF data will be stored
```
ZQS1 ALTER QMGR STATCHL(MEDIUM)
```
This tells z/OS we want to enable channel statistics to be collected at a moderate ratio of data collection
```
ZQS1 ALTER QMGR MONQ(MEDIUM)
```
This tells z/OS to turn on monitoring for the queue manager’s queues at a moderate ratio of data collection
```
ZQS1 ALTER QMGR MONCHL(MEDIUM)
```
This tells z/OS to turn on monitoring for the queue manager’s channels at a moderate ratio of data collection
```
ZQS1 START TRACE(STAT) CLASS(1,2,4,5)

ZQS1 START TRACE(ACCTG) CLASS(3,4)
```
Now all the settings should be in place for our queue manager. Head back to ZQS1.MP1B.JCL using 3.4 from the main ISPF menu.
We will use OEMPUT to load messages into MP1B.TESTER. In the directory ZQS1.MP1B.JCL, place an ‘e’ to the left of the OEMPUT member.
Make sure that your queue manager and queue names are correct in lines 46 and 47.

Once in OEMPUT, type ‘submit’ on the command line and hit enter to load persistent messages into the queue manager.

I won’t summarize the whole JCL, but pay attention to this particular line:

PARM=('-M&QM -tm3 -Q&Q -crlf -fileDD:MSGIN -P')

Lets break it down:

Parameter	Meaning
'-M&QM	Queue manager name
-tm3	Send messages for 3 minutes
-Q&Q	The queue name
-crlf	Each line in the input message file is used in sequence as message data
-fileDD:MSGIN	Use the MSGIN file as input
-P	Use persistent messages

If you look at your MQ Explorer, you should now see that your queue is populated with lots of messages!
Back in ZQS1.MP1B.JCL, navigate to the SMFDUMP member. Once inside, enter ‘submit’ on the command line to execute SMFDUMP JCL. The SMFDUMP JCL starts with deleting old tasks, then outputs it in a specified location, in our case, ZQS1.QUEUE.MQSMF.SHRSTRM2.
You can check that the SMFDUMP is processing by navigating to your job using SDSF. Access SDSF using =D from the ISPF menu.
Once in SDSF, select ST from the menu and hit ‘enter’
Type in ‘prefix ZQS1*’. This will show you a list of all jobs submitted that start with ZQS1. Remember, we define our job names at the top left of each JCL file.
Here, you put a ‘?’ mark besides the jobname. Hit enter, then a screen with a SYSPRINT menu option should pop up. Next to SYSPRINT, put a ‘s’ and hit enter.
Enter ‘bottom’ on the command line and you should see a screen like below, indicating that records are being written. You can also confirm this by looking in the output for the SUMMARY ACTIVITY REPORT.
After submitting, you will have to submit another job MQSMFP in ZQS1.MP1B.JCL. This job will give us some formatted information about the SMF data. Make one change before submitting: ensure that the julian date is correct. For labs taking place on 2/24/2025, the julian date is 25055.
Type ‘submit’ and hit enter.
Now, navigate to the SDSF output for the submitted job. We will be able to see the SMF output in useful categories that can also be exported as CSV files.
Navigate to the LOG statistics by putting a ‘s’ next to it and hitting enter. Scroll down until you see a screen similar to the one below.
Here you can see LLCheckpoints has a value of 1564. Within our interval, we would expect this value to be 0’s or single-digits. 1564 is way too high. This indicates we should adjust our LOGLOAD attribute to have it write more log records between checkpoints.

Summary

Picture of logging schematic

The LOGLOAD parameter specifies the number of log records that are written between checkpoints. In the figure above, you can see the LOGLOAD indicated by the blue brackets. For the above image’s example, the LOGLOAD looks to be 6 here (6 would be impossibly small in a real environment). We set our queue manager’s LOGLOAD attribute to the lowest possible value of 200 then flood our environment with messages. We saw see this cause high checkpointing in our recorded SMF window, resulting in unnecessary consumption of processor time and additional I/O.