|
|
The sar(1M) command can be used either to gather system activity data itself or to extract what has been collected in the daily activity files created by sa1 and sa2.
The sar command has the following formats:
sar [-P processor_id[, . . . ] | ALL] [-ubdycwaqvtmpgrkAR]In the first format, sar samples cumulative activity counters in the operating system at intervals specified by n for a time (in seconds) specified by t (t should be 5 seconds or greater). The default value of n is 1. If the -o option is specified, samples are saved in file in binary format.
[-o file] t [n]sar [-P processor_id[, . . . ] | ALL] [-ubdycwaqvtmpgrkAR]
[-s time] [-e time] [-i sec] [-f file]
In the second format, with no sampling interval specified, sar extracts data from a previously recorded file, either the one specified by the -f option or, by default, the standard daily activity file, /var/adm/sa/sardd. The -s and -e options define the starting and ending times for the report. The starting and ending times are of the form hh[:mm[:ss]]. The -i option specifies, in seconds, the intervals to select records. If the -i option is not included, all intervals found in the daily activity file are reported.
You can use the -P option to specify collection of data for particular processors rather than for system-wide activity.
The following options are available with sar:
option | report | to determine: |
---|---|---|
-a | file access operations | amount of inode and directory activity |
-b | buffer activity | the effectiveness of system buffering and the appropriateness of the value of the BUFHWM system parameter |
-c | system call activity | the percentage of read and write system calls |
-d | disk activity | disk slowdowns |
-g | page-out and memory freeing | whether you have a memory shortage |
-k | kernel memory allocation | if you make special use of KMA, if it is too large a portion of total memory |
-m | interprocess communication | the number of messages and semaphore calls |
-p | page-in and fault activity | whether you have a memory shortage |
-q | queue activity | whether you have a memory shortage |
-r | unused memory | whether you have a memory shortage |
-t | inode or DNLC activity by filesystem type | if inode limits should be changed so that more inodes and pages can be kept in memory and reused |
-u | CPU utilization | if there are tuning problems or memory constraints |
-v | system table status | if system tables need to be made larger |
-w | swapping and switching volume | whether you have a memory shortage |
-y | asynchronous terminal activity | determine if there are any bad lines |
-A | report overall system performance, same as entering all options | view overall system performance |
-P | report system activity for specified processors only | view performance of a specific processor or distribution among processors |
-R | report raw data values | view the values of the counters from which metrics are computed, rather than computed values |
When displaying data, sar fits numbers within their allocated field widths by changing output formats.
When absolute numbers are mentioned in the following descriptions of the sar command options, they are correct for Intel486/33 processors. Faster processors can support more activity per second.
The sar -a option reports on the use of file access operations. The following system routines are reported:
iget/s
namei/s
dirbk/s
%dnlc
If -R is specified then ``%dnlc'' is replaced by ``dnlc-hits'' and ``dnlc-misses'', the counts of cache hits and misses.
The following is an example of sar -a output.
UnixWare holly 5 7.1.0 i386 04/22/99The larger the values reported, the more time the system kernel is spending to access files. The amount of time reflects how heavily programs and applications are using the filesystem(s). The -a option is helpful for understanding how disk-dependent an application system is. It might be useful in tuning your directory name lookup cache (dnlc) size; change the value of the filesystem system parameter DNLCSIZE if %dnlc is often low. (That is, when %dnlc is low other than when a large number of files are accessed for the first time in a while.)00:00:00 iget/s namei/s dirbk/s %dnlc 01:00:00 0 1 0 97 02:00:00 0 1 0 97 03:00:00 0 1 0 97 04:00:00 0 1 0 97 05:00:00 0 1 0 97 06:00:00 0 1 0 97 07:00:00 17 49 24 62 08:00:00 0 1 0 97 . . . 14:00:00 0 1 0 97 14:20:00 0 3 0 93 14:40:00 0 3 0 90 Average 1 5 1 73
The buffer caches filesystem meta-data about inodes, indirect blocks, super-blocks and so on. The -b option reports on the following buffer activities:
bread/s
lread/s
%rcache
bwrit/s
lwrit/s
%wcache
pread/s
pwrit/s
%rcache
falls below 90, or
if %wcache
falls below 60, it might be possible to
improve performance by increasing the buffer space
by adjusting the buffer cache system parameter BUFHWM.
Check again when running the new system to be sure they are still
acceptable values.
If the -R option is specified, the %rcache
and %wcache
columns are not displayed.
The following is an example of sar -b output:
UnixWare holly 5 7.1.0 i386 04/22/9900:00:00 bread/s lread/s %rcache bwrit/s lwrit/s %wcache pread/s pwrite/s 01:00:00 0 1 100 0 0 2 0 0 02:00:00 0 1 100 0 0 2 0 0 03:00:01 0 0 100 0 0 3 0 0 04:00:00 0 1 100 0 0 2 0 0 05:00:00 0 1 100 0 0 2 0 0 06:00:00 0 1 100 0 0 2 0 0 07:00:01 1 41 97 1 1 32 0 0 08:00:00 0 1 100 0 0 2 0 0 . . . 14:00:00 0 1 100 0 0 1 0 0 14:20:00 0 1 100 1 1 18 0 0 14:40:00 0 2 96 1 1 11 0 0 Average 0 3 97 0 1 9 0 0
The -c option reports on system calls in the following categories:
scall/s
sread/s
swrit/s
fork/s
lwpcr/s
exec/s
rchar/s
wchar/s
This is an example of sar -c output:
UnixWare holly 5 7.1.0 i386 04/22/9900:00:00 scall/s sread/s swrit/s fork/s lwpcr/s exec/s rchar/s wchar/s 01:00:00 14 1 0 0.04 0.00 0.01 777 47 02:00:00 14 1 0 0.04 0.00 0.01 771 46 03:00:01 14 1 0 0.04 0.00 0.01 778 47 04:00:00 14 1 0 0.04 0.00 0.01 777 47 05:00:00 14 1 0 0.04 0.00 0.01 796 47 06:00:00 14 1 0 0.04 0.00 0.01 780 47 07:00:01 237 13 6 2.51 0.00 2.80 4858 471 08:00:00 14 1 0 0.04 0.00 0.01 779 47 . . . 14:00:00 14 1 0 0.04 0.00 0.01 771 52 14:20:00 31 2 1 0.09 0.00 0.06 1081 389 14:40:00 23 2 2 0.10 0.00 0.08 4550 418 Average 31 2 1 0.22 0.00 0.21 1215 109
The sar -d option reports the activities of disk devices.
device
%busy
avque
r+w/s
blks/s
avwait
avserv
When the -R option is specified, the columns avque
,
avwait
, and avserv
are not displayed.
An additional column, busy
,
the total time the disk was active, is displayed.
The following is an example of sar -d output:
UnixWare holly 5 7.1.0 i386 04/22/99Note that queue lengths and wait and service times are measured when there is something in the queue. If00:00:00 device %busy avque r+w/s blks/s avwait avserv 01:00:00 sd011 0 1.3 0 7 3.7 12.9 02:00:00 sd011 0 1.3 0 6 3.6 12.6 03:00:01 sd011 0 1.3 0 6 3.2 12.6 04:00:00 sd011 0 1.3 0 7 3.7 13.1 05:00:00 sd011 0 1.4 0 6 4.5 12.5 06:00:00 sd011 0 1.3 0 7 3.6 13.1 07:00:01 sd011 3 1.7 3 41 6.9 10.5 08:00:00 sd011 0 1.6 0 6 6.2 11.1 08:00:00 sd011 0 1.6 0 6 6.2 11.1 . . . 14:00:00 sd011 0 1.5 0 7 6.0 11.4 14:20:00 sd011 1 1.6 1 15 7.0 11.6 14:40:00 sd011 1 1.8 1 21 8.7 10.9 Average sd011 0 1.5 0 10 5.8 11.5
%busy
is small, large queues and service times probably represent
the periodic sync efforts by the system
to ensure altered blocks are written to the disk in a timely fashion.
The sar -g option reports page-out and memory freeing activities as follows:
pgout/s
ppgout/s
vfree/s
pfree/s
vscan/s
UnixWare holly 5 7.1.0 i386 04/22/99The sar -g report is a good indicator of whether more memory will improve the performance of your system. If the values of vfree/s, pfree/s and vscan/s are high, or the attach rate and00:00:00 pgout/s ppgout/s vfree/s pfree/s vscan/s 01:00:00 0.00 0.00 0.01 0.01 0.94 02:00:00 0.00 0.00 0.01 0.01 1.12 03:00:01 0.00 0.00 0.01 0.01 0.95 04:00:00 0.00 0.00 0.01 0.01 0.80 05:00:00 0.00 0.00 0.01 0.01 1.23 06:00:00 0.00 0.00 0.02 0.02 1.14 07:00:01 0.15 0.15 1.52 0.92 12.72 08:00:00 0.00 0.00 0.00 0.00 0.00 . . . 14:00:00 0.00 0.00 0.01 0.01 1.03 14:20:00 0.37 0.37 0.77 0.52 11.91 14:40:00 0.14 0.14 0.52 0.22 10.38 15:00:00 0.00 0.00 0.25 0.06 2.74 Average 0.03 0.03 0.20 0.10 2.58
vflt
rates from
sar -p are high, then you probably have a memory shortage.
sar -p, sar -u, sar -r, and sar -w
are also good memory shortage indicators.
The -k option reports on the following activities of the Kernel Memory Allocator (KMA).
The following information is displayed for each memory pool:
size
Ovsz
for the oversize pool.
mem
alloc
succ
alloc
column
since the buffers are predetermined sizes.
fail
If your system is being used to write drivers
or STREAMS that use KMA resources,
then sar -k will likely prove useful;
otherwise, you will probably not need the information it provides.
However, if the total KMA represents a large portion of total memory,
it might be an indication that the system is not properly tuned
or that a locally written driver or STREAMS module has a memory leak.
Any driver or module that uses KMA resources
but does not specifically return the resources
before it exits can create a memory leak.
A memory leak will cause the amount of memory allocated by KMA
to increase over time.
Thus, if the alloc
fields of sar -k
increase steadily over time, there might be a memory leak.
Another indication of a memory leak is failed requests.
If many requests fail, then it is likely that a memory leak has caused
KMA to be unable to reserve and allocate memory.
If it appears that a memory leak has occurred, check any locally written drivers or STREAMS that may have requested memory from KMA and have not returned it.
The sar -m option reports interprocess communication activities. Message and semaphore calls are reported as follows:
UnixWare holly 5 7.1.0 i386 04/22/99These figures will usually be zero (0.00) unless you are running applications that use messages or semaphores.00:00:00 msg sema 01:00:00 0.00 0.00 02:00:00 0.00 0.00 03:00:01 0.00 0.00 04:00:00 0.00 0.00 05:00:00 0.00 0.00 06:00:00 0.00 0.00 07:00:01 0.00 0.00 08:00:00 0.00 0.00 . . . 14:00:00 0.00 0.00 14:20:00 0.00 0.35 14:40:00 0.00 0.06 15:00:00 0.00 0.00 Average 0.00 0.02
The sar -p option reports page-in activity, which includes protection and validation faults.
atch/s
atfree/s
atmiss/s
pgin/s
ppgin/s
pflt/s
vflt/s
slock/s
UnixWare holly 5 7.1.0 i386 04/22/99If00:00:00 atch/s atfree/s atmiss/s pgin/s ppgin/s pflt/s vflt/s slock/s 01:00:00 5.85 1.43 0.05 0.01 0.01 1.02 3.28 0.00 02:00:00 5.81 1.44 0.05 0.00 0.00 1.01 3.27 0.00 03:00:01 5.84 1.49 0.05 0.00 0.00 1.03 3.26 0.00 04:00:00 5.88 1.52 0.05 0.00 0.00 1.03 3.30 0.00 05:00:00 5.92 1.48 0.05 0.00 0.01 1.04 3.32 0.00 06:00:00 5.89 1.49 0.05 0.00 0.00 1.04 3.29 0.00 07:00:01 85.65 2.92 3.91 0.70 1.10 66.11 28.16 0.00 08:00:00 5.85 1.50 0.05 0.00 0.00 1.02 3.29 0.00 . . . 14:00:00 5.83 1.44 0.05 0.00 0.00 1.00 3.28 0.00 14:20:00 18.87 2.01 0.20 0.09 0.23 5.11 4.68 0.00 14:40:00 8.34 2.41 0.27 0.17 1.09 1.80 4.89 0.00 15:00:00 7.52 2.10 0.12 0.06 0.11 1.59 3.80 0.00 Average 12.18 1.70 0.33 0.07 0.14 5.70 5.17 0.00
vflt/s
becomes much higher than 50,
look at sar -g to determine if there is a memory shortage
or if the inode free list is page bound.
(See sar -t for more details.)
In addition, sar -u, sar -w,
and sar -r can help verify whether memory is a bottleneck.
In particular, a high attach rate and a high vflt
rate indicates that memory is scarce.
The sar -q option reports the average queue length while the queue is occupied and the percentage of time the queue is occupied.
prunq
%prunocc
runq
%runocc
swpq
%swpocc
If the -P option has not been entered to specify one or more
processors, then the prunq
and %prunocc
columns are blank.
The following is an example of sar -q output:
UnixWare holly 5 7.1.0 i386 04/22/99If00:00:00 prunq %prunocc runq %runocc swpq %swpocc 01:00:00 1.5 0 2.0 100 02:00:00 2.8 0 2.0 100 03:00:01 3.0 0 2.0 100 04:00:00 1.2 0 2.0 100 05:00:00 1.5 0 2.1 100 06:00:00 1.5 0 2.1 100 07:00:01 1.4 3 2.1 100 1.0 0 08:00:00 2.5 0 2.0 100 . . . 14:00:00 2.0 100 14:20:00 1.5 0 2.0 100 14:40:00 6.0 0 2.2 100 15:00:00 2.0 0 2.7 100 Average 1.7 0 2.0 100 1.0 0
%runocc
is greater than 90 percent and
runq-sz
is greater than 3, the CPU might be
heavily loaded and response may be degraded.
(See sar -u.)
In this case, additional CPU capacity may be required to
obtain acceptable system response.
If sar -p shows a large number of validity faults and
sar -g shows high page-out activity,
then more memory may be required.
swpq-sz
and %swpocc
are blank unless there are processes
swapped out that are now ready to run.
A swpq-sz
greater than ``2'' usually indicates a memory
shortage.
sar -w reports the number of processes swapped in and out.
The swpq-sz
reports how many of those were ready to run
as opposed to those waiting for an event.
The -r option records the number of memory pages and swap file disk blocks that are currently unused.
freemem
freeswap
UnixWare holly 5 7.1.0 i386 04/22/9900:00:00 freemem freeswap 01:00:00 3703 22865 02:00:00 3704 22865 03:00:01 3703 22866 04:00:00 3702 22859 05:00:00 3703 22865 06:00:00 3705 22863 07:00:01 3700 22610 08:00:00 3835 22709 . . . 14:00:00 3704 22570 14:20:00 3754 22520 14:40:00 3803 22463 15:00:00 3819 22412 Average 3755 22725
The -t option reports the inode usage by filesystem type. This report shows when inode limits need to be changed. The -t is a companion to the -v option, which reports the sum of all inodes in the system, as well as other system table usage.
filesystem
inodes inuse
alloc
limit
fail
%ipf
iget
.
These pages are flushed and cannot be reclaimed by processes.
Thus, this is the percentage of iget
s with page flushes.
If this value is greater than 20 percent,
then the free list of inodes is considered
page-bound and the number of inodes may need to be increased.
If the -R option is specified, the %ipf
column
is not displayed.
Instead, the columns ipage
and inopage
,
(equal to the counts of inodes
with and without reusable pages, respectively) are displayed.
The following is an example of sar -t output:
UnixWare holly 5 7.1.0 i386 04/22/99Because many filesystems allocate inodes dynamically, the number of inodes alloced will often be less than the limit. Then, no inodes are recycled during a sample; therefore, making the00:00:00 file system inodes inuse alloc limit fail %ipf 01:00:00 s5 0 0 0 0 100 sfs/ufs 0 0 0 0 100 vxfs 1009 1010 5125 0 0 other 0 0 0 0 100
02:00:00 s5 0 0 0 0 100 sfs/ufs 0 0 0 0 100 vxfs 1009 1010 5125 0 0 other 0 0 0 0 100 . . . 14:40:00 s5 0 0 0 0 100 sfs/ufs 0 0 0 0 100 vxfs 890 911 5125 0 30 other 0 0 0 0 100
15:00:00 s5 0 0 0 0 100 sfs/ufs 0 0 0 0 100 vxfs 879 880 5125 0 10 other 0 0 0 0 100
Average s5 0 0 0 0 100 sfs/ufs 0 0 0 0 100 vxfs 426 429 5125 0 3 other 0 0 0 0 100
%ipf
value meaningless, because of
division by zero.
If the inodes in use
number is consistently at the
limit and the value of %ipf
is higher than 20, then raising the limit may enhance system performance.
As long as memory is available (see the -p,
-g, -w, -q, and -u
options), increasing inode limits will help keep inodes and pages in the
system so they can be used again.
The sar -u option lists CPU use. At any given moment the processor is either busy or idle. When busy, the processor is in either user or system mode, or is handling interrupts. When idle, the processor is either waiting for input or output completion or ``sitting still'' with no work to do. sar -u lists the percentage of time that the processor is in system mode, in user mode, waiting for input or output completion, interrupt handling or idle.
%usr
%sys
%wio
%idle
%intr
The following is an example of sar -u output:
UnixWare holly 5 7.1.0 i386 04/22/9900:00:00 %usr %sys %wio %idle %intr 01:00:00 0 0 0 100 0 02:00:00 0 0 0 100 0 03:00:01 0 0 0 100 0 04:00:00 0 0 0 100 0 05:00:00 0 0 0 100 0 06:00:00 0 0 0 100 0 07:00:01 3 5 2 90 0 08:00:00 0 0 0 100 0 . . . 14:00:00 0 0 0 100 0 14:20:00 0 1 0 99 0 14:40:00 2 0 0 97 0 15:00:00 0 0 0 99 0 Average 0 1 0 99 0
In typical timesharing use, %sys
is
usually a little higher than %usr
.
If %sys is more that double %usr
,
this may indicate tuning problems or that the system is being asked to do
an unusually large amount of work.
ps and kernel profiling can help identify the problems.
In special applications, either of these may be larger
than the other without anything being abnormal.
A high %wio
generally means a disk slowdown has occurred.
A high %idle,
with degraded response time,
may mean memory constraints are present;
time spent waiting for memory is attributed to %idle.
The -v option reports the status of the process, lightweight processes (LWPs), inode, file, and shared memory record table. From this report, you can determine when the system tables need to be updated.
proc-sz
inod-sz
file-sz
fail
lock-sz
The following is an example of sar -v output:
UnixWare holly 5 7.1.0 i386 04/22/99This example shows that all tables are large enough to have no overflows. If the values in these tables never exceed those shown here, you can reduce the size of the tables as a way of saving space in main memory. The00:00:00 proc-sz fail lwp fail inod-sz fail file fail lock 01:00:00 70/400 0 131 0 1009/5125 0 453 0 11 02:00:00 70/400 0 131 0 1009/5125 0 453 0 11 03:00:01 70/400 0 131 0 0/5125 0 458 0 11 04:00:00 70/400 0 131 0 1011/5125 0 453 0 11 05:00:00 70/400 0 131 0 998/5125 0 453 0 11 06:00:00 74/400 0 135 0 1002/5125 0 453 0 11 07:00:01 70/400 0 131 0 0/5125 0 460 0 11 08:00:00 70/400 0 131 0 0/5125 0 454 0 11 . . . 14:00:00 74/400 0 135 0 804/5125 0 461 0 11 14:20:00 73/400 0 134 0 840/5125 0 461 0 11 14:40:00 74/400 0 135 0 890/5125 0 461 0 11 15:00:00 74/400 0 135 0 879/5125 0 462 0 11 15:20:01 74/400 0 135 0 0/5125 0 468 0 11 Average 71/400 0 132 0 412/5125 0 457 0 11
inod-sz
values are the total of
s5, ufs, sfs, and vxfs
inodes and their maximums.
sar -t breaks down the inode usage by filesystem type.
The -w option reports swapping and switching activity.
The following are some target values and observations:
swpin/s
pswin/s
swpot/s
pswot/s
vpswout/s
pswot/s
if some physical pages
were being used by multiple processes.
pswch/s
UnixWare holly 5 7.1.0 i386 04/22/99This example shows that because no swapping is occurring, there is no memory shortage for the currently active users.00:00:00 swpin/s pswin/s swpot/s pswot/s vpswout/s pswch/s 01:00:00 0.00 0.0 0.00 0.0 0.0 12 02:00:00 0.00 0.0 0.00 0.0 0.0 12 03:00:01 0.00 0.0 0.00 0.0 0.0 12 04:00:00 0.00 0.0 0.00 0.0 0.0 12 05:00:00 0.00 0.0 0.00 0.0 0.0 12 06:00:00 0.00 0.0 0.00 0.0 0.0 12 07:00:01 0.00 0.0 0.00 0.1 0.2 41 08:00:00 0.00 0.0 0.00 0.0 0.0 12 . . . 14:00:00 0.00 0.0 0.00 0.0 0.0 11 14:20:00 0.00 0.0 0.00 0.0 0.0 15 14:40:00 0.00 0.0 0.00 0.0 0.0 13 15:00:00 0.00 0.0 0.00 0.0 0.0 12 15:20:01 0.00 0.0 0.00 0.0 0.0 14 Average 0.00 0.0 0.00 0.0 0.0 14
The -y option monitors asynchronous terminal device activities. If there is a lot of terminal I/O, you can use this report to determine if there are any bad lines. If you are not using serial lines the report will show zeros for all activities. The activities recorded are defined as follows:
rawch/s
canch/s
outch/s
rcvin/s
xmtin/s
mdmin/s
An example of sar -y output follows:
UnixWare holly 5 7.1.0 i386 04/22/99The number of modem interrupts per second (mdmin/s) should be close to00:00:00 rawch/s canch/s outch/s rcvin/s xmtin/s mdmin/s 01:00:00 0 0 0 0 0 0 02:00:00 0 0 0 0 0 0 03:00:01 0 0 0 0 0 0 04:00:00 0 0 0 0 0 0 05:00:00 0 0 0 0 0 0 06:00:00 0 0 0 0 0 0 07:00:01 0 0 0 0 0 0 08:00:00 0 0 0 0 0 0 . . . 14:00:00 0 0 0 0 0 0 14:20:00 1 0 18 0 0 0 14:40:00 1 1 75 0 0 0 15:00:00 0 0 27 0 0 0 15:20:01 0 0 43 0 0 0 Average 0 0 7 0 0 0
0
, and the receive and transmit interrupts per second
(xmtin/s and rcvin/s) should be less than or
equal to the number of incoming or outgoing characters, respectively.
If this is not the case, check for bad lines.
The sar -A option provides a view of overall system performance. Use it to get a more global perspective. If data from more than one time slice is shown, the report includes averages.
The sar -R option provides a report of raw data values. Using this option, sar displays the values of the counters from which metrics are computed, rather than computed values. For example, when used with -u, the number of clock ticks are displayed rather than a percentage.
When using the -R option, %
prefixes and
/s suffixes are not displayed in column headings.
For example, instead of %name
or name/s,
the heading becomes name
.
Some tables are displayed differently;
the layout of tables reporting the ratio of two counters is changed.
Columns that are computed from more than one counter are not displayed.
If one, or both, of these counters is not already output in another
column, then a new column is added for it.
This changes the tables for the options -b,
-d, -a, -t.
At the end of the report averages are not computed;
instead, the totals from which the averages are derived are displayed.
An example of sar -R output follows:
UnixWare holly 5 7.1.0 i386 04/22/9900:00:00 usr sys wio idle intr 01:00:00 332 684 460 358421 88 02:00:00 330 662 490 358598 85 03:00:01 299 690 343 358652 81 04:00:00 299 707 423 358464 80 05:00:00 287 722 496 358588 89 06:00:00 278 713 392 358504 81 07:00:01 11767 16541 7938 323096 937 08:00:00 261 723 342 358566 75 . . . 14:00:00 91 255 141 119499 34 14:20:00 535 696 420 118264 120 14:40:00 1950 495 489 116910 103 15:00:00 123 370 292 119167 71 15:20:01 523 759 415 118232 108 15:40:00 1119 1209 573 117074 170 16:00:00 395 538 296 118737 60 Average 20994 31295 17230 5.69e6 2881