Skip to content

iptrace at will

To record communication information on a specific port between two hosts, execute the following command:

# iptrace -a -i enX -s SourceHost -b -d DestinationHost -p PortNumber trace.out

Where enX -is the network interface use for the data transfer. The trace.out file is the bucket storing the received data. When enough data has been captured kill iptrace and translate its output so it can be read executing the ipreport command.

# ipreport trace.out >

To intercept all packets from a SourceHost, modify the last command like this:

 iptrace -a -i enX -s SourceHost trace.out

You can ask for the “bi-directional” traffic including -b and port using -p PortNumber.

Posted in Real life AIX.

ldapsearch exceeds search limits

When you configure the ITDS Directory server for the first time there are a
couple of default limits that are set on non-admin users. These settings are
designed to prevent non-admin users from running very large and expensive
searches on the entire directory, as that can have a very negative impact on
performance and resources. These settings are:

ibm-slapdSizeLimit: 500


ibm-slapdTimeLimit: 900 

In this example I have the following user defined in my directory:

The dc=mycompany,dc=com subtree currently has over 1 million entries in it.

The objective is to give the user "cn=searchuser,dc=mycompany,dc=com" access to
search the entire subtree, but the default ibm-slapdSizeLimit will not allow this.

For example:

ldapsearch -D cn=searchuser,dc=mycompany,dc=com -w passw0rd -b dc=mycompany,dc=com objectclass=* 

results in 500 results then:

ldap_search: Sizelimit exceeded

In order to allow the directory user a higher limit in the number of entries
that can be returned while:
1. Not adding the user to an administrative group
2. Not effecting the default size and time limits in the ibmslapd.conf
We will add a group with an objectclass of ibm-searchLimits, define our limits
in that group and add the user “cn=searchuser,dc=mycompany,dc=com” to that

—- Adding the search group via the command line: —-

Lets begin by defining the following objectives for this example:
1. We want the user: cn=searchuser,dc=mycompany,dc=com
2. To be a member of cn=searchgroup,cn=ibmpolicies
3. To have unlimited search and time limits
4. We assume in this example the user cn=searchuser,dc=mycompany,dc=com already exists

To add and define the search group via the command line we run:

# ldapadd -D cn=root -w *****
dn: cn=searchgroup,cn=ibmpolicies
objectclass: groupOfNames
objectclass: top
objectclass: ibm-searchLimits
cn: searchgroup
ibm-searchSizelimit: 0
ibm-searchTimelimit: 0
member: cn=searchuser,dc=mycompany,dc=com 

(hit enter 2 times)

will return:

adding new entry cn=searchgroup,cn=ibmpolicies 

(ctrl + c to regain the command line)

To verify the group is there:

ldapsearch -D cn=root -w ? -b cn=searchgroup,cn=ibmpolicies objectclass=* 

— Adding the search group via the webadmin —
In this example we are using the same 4 objectives stated under the command line method.

Code levels used in this example:

*** ITDS webadmin
/opt/IBM/ldap/V6.0/idstools# ./ -v
SCRIPT_DATE: 1/14/07

Mon 08/20/2007

*** ibmslapd -v
GLPSRV033I The Directory server version is IBM Tivoli Directory (SSL), 6.0 (Sep
22 2007 09:55:45)

1. Log in to the ITDS Webadmin tool as the administrative DN (cn=root typically)

2. Directory Management -> Manage entries -> select “cn=ibmpolicies” and
select the Add button

In the “Add an entry” screen we select the objectclass “groupOfNames”:
and click Next
In the “Auxiliary object classes” scroll under the “Available” section, select
“ibm-searchLimits”, select Add and then select Next:

Finally we define the group by giving it a name and adding
“cn=searchuser,c=mycompany,dc=com” to that group:
By setting the values to 0 the limits are set to “unlimited”:
ibm-searchSizeLimit: 0
ibm-searchTimeLimit: 0

To complete click “Next”
you will see the screen:
Click “Finish”

and you will see:
Select Yes to add other groups, or No to complete the process.

The user “cn=searchuser,dc=mycompany,dc=com” will now be able to search the
entire subtree without the ldap_search: Sizelimit exceeded message

Posted in Real life AIX.

Tagged with .

you are only as good as the last few weeks

Today is Friday! I am busy migrating a two node cluster from XIV to HDS SAN storage. Nothing could be easier, right? After HDS disks are zoned to the hosts WWPNs and cfgmgr takes them in and I establish what new disk comes from what HDS controller I do extend the volume group and execute the following command to get the third mirror (on the set of new disk delivered by the first HDS controller).

# mirrorvg -S -c 3 lawap_vg hdisk10 hdisk11 hdisk12 hdisk13
0516-404 allocp: This system cannot fulfill the allocation request.
There are not enough free partitions or not enough physical volumes
to keep strictness and satisfy allocation requests.  The command
should be retried with different allocation characteristics.
0516-1517 mklvcopy: Failed to create a valid partition allocation.
0516-842 mklvcopy: Unable to make logical partition copies for
logical volume.
0516-1199 mirrorvg: Failed to create logical partition copies
for logical volume u10_lv.
0516-1200 mirrorvg: Failed to mirror the volume group.

What? There is space for God’s sake. Do you see it?

# lsvg -p lawap_vg
hdisk3    active      799         400         80..00..00..160..160
hdisk5    active      15          0           00..00..00..00..00
hdisk4    active      15          0           00..00..00..00..00
hdisk7    active      799         400         65..00..15..160..160
hdisk10   active      249         249         50..50..49..50..50
hdisk11   active      249         249         50..50..49..50..50
hdisk12   active      249         249         50..50..49..50..50
hdisk13   active      249         249         50..50..49..50..50

The lslv shows that logical volumes are set to the minimum number of disks, and that they are relocatable. The chlv command comes to mind and I execute the following cludge just to be sure.

for lv in `lsvg -l lawap_vg | grep -v grep \
                    | grep jfs2 | awk '{print $1}'`
chlv -s n -e x  $lv

Next, the mirrorvg command is recalled from the shell history and it dies promptly generating as many error messages as before…..

“What is going on?” – I ask surprised. I look and read the man pages and I still cannot get it running. After a while when I am looking at the output of the lslv command while describing my predicament to my wife it suddenly hits me what I have forgotten! It is the UPPER BOUND parameter of each logical volume of this volume group! Yes Sir, this parameter value is currently smaller than the present number of disks this vg has!

I modify the previous cludge to change it to 8 to match the number of disks in the volume group.

# for lv in `lsvg -l lawap_vg | grep -v grep \
                       | grep jfs2 | awk '{print $1}'`
chlv -u 8 $lv

I am ready to do it again. This time mirroring works like it should and I smile for a while until I recognize that I have not adjusted the queue to the values HDS recommends for AIX….. Well, I will wait till mirroring is done, reduce the third mirror, remove the disks form the volume group, change their queue depths and back track back to the mirrorvg that now, I remember again how to make to execute.

Have a nice weekend!

Posted in Real life AIX.

concurrent volume groups - issues with their replications

If you do any replication of the “enhanced” concurrent volume groups using FlashCopy or ShadowImage you may have problems…. The recreatevg command may fail to bring the volume group copy into a different host (backup host) with this message:

0516-1972 varyonvg: The volume group is varied on in other node in concurrent
mode; you cannot vary on the volume group in non-concurrent mode.
Use -O flag to force varyon the volume group if needed.
0516-1320 recreatevg: Unable to recreate volume group.

If you search the FixCentral for a solution, you may find it! IBM has these APARs accordingly to

6100-08 – use AIX APAR IV41209
7100-02 – use AIX APAR IV40515

My cluster runs on 7.1.3 and TSM server on 6.1.9 with these APARs present but executing of the recreatevg still does not work…. Apparently these APARs not always work as IBM offers this “local fix” in the same page:

If recreatevg is being run on a different server than the source VG, and there are no name conflicts, then the following steps can be used:

# importvg -y vgname hdiskX
# varyonvg -O vgname
# varyoffvg vgname
# exportvg vgname

Then run recreatevg.

If you are familiar with the recreatevg command, you know that it automatically varies ON the “recreated” volume group. Look at the message produced by the failing recreatevg command again. Does is really applies to this or to the varyonvg command?

# recreatevg -f -y epcchePRD_vg -L /SImg -Y SImg $epcchePRD_vg_disks
0516-1972 varyonvg: The volume group is varied on in other node in concurrent
mode; you cannot vary on the volume group in non-concurrent mode.
Use -O flag to force varyon the volume group if needed.
0516-1320 recreatevg: Unable to recreate volume group.

I think that this message is poorly written. For me it suggest to use -O as the argument to the varyonvg command?
What this message really means is to use “-O” with the recreatevg like that:

# recreatevg -f -y epcchePRD_vg \
              -L /SImg -Y SImg -O $epcchePRD_vg_disks

It works! By the way, the variable epcchePRD_vg_disks contains a space separated list of appropriate disks.

Posted in Real life AIX.

Tagged with , , .

cannot create or extend a volume group, shadowimage fails too.....

somewhere, someone gave too much power to nmon ……. If you noticed that suddenly you cannot do what the topic of this post says check if nmon is running.
Next, kill it and repeat the command(s) that previously failed. Do they work now? I think so.

In the last few months I noticed this on few of our hosts but since I have not patched them I missed to make the connection. Yesterday, my colleague patched our DSMC backup servers to AIX and tonight’s ShadowImage backups failed……

The following output explains everything:

# recreatevg -f -y epcdbm_vg $epcdbm_vg_disk
Method error (/usr/lib/methods/chgdisk):
0514-062 Cannot perform the requested function because the
specified device is busy.
0516-1320 recreatevg: Unable to recreate volume group.

The command fails. I kill nmon

# ps -ef | grep nmon
    root  4587738 11731134   0 11:13:15  pts/2  0:00 grep nmon
    root 10748198        1   0 00:00:01      -  0:02 /usr/bin/topas_nmon  -f -T -d -A -m /var/nmon -s 180 -c 480 -youtput_dir=/var/nmon/tsmdbrpu001 -ystart_time=00:00:00,Jul25,2014
# kill -9 10748198

Let’s try to bring the vg back.

# recreatevg -f -y epcdbm_vg $epcdbm_vg_disk

It worked! Like I said, the same will make extendvg work as well.

To resolve this new issue permanently, I modified my backup scripts as follows. Before I bring the volume groups in, my scripts executes now the following line.

kill -9 `ps -ef | grep nmon | grep -v grep | awk '{print $2}'`

At the end, just before the exist, I start nmon as follows (using the entry from root's own crontab.


Please let me know if you know about any other fix.

Have a good weekend!!!

Posted in Real life AIX.

Tagged with , , .

checksums …..

After downloading upgrade media for an HMC, I wanted to check consistency of the packages. During the download, I had to expand their file system and this gave me the “verify it” idea.

These are the contents I got from IBM server:

# ls -ltr
total 6219440
-rw-r--r--    1 root     system      2730176 Jul 08 06:59 bzImage
-rw-r--r--    1 root     system    817065984 Jul 08 07:20 disk1.img
-rw-r--r--    1 root     system   1456427008 Jul 08 07:58 disk2.img
-rw-r--r--    1 root     system           78 Jul 08 08:21 hmcnetworkfiles.sum
-rw-r--r--    1 root     system    873922560 Jul 08 08:21 disk3.img
-rw-r--r--    1 root     system     34185788 Jul 08 08:22 initrd.gz

The hmcnetworkfiles.sum list each file and its checksum value. See for yourself:

# cat hmcnetworkfiles.sum

To identify checksum of these files, you will use the sum command maybe even in a loop like this one:

# for f in `cat hmcnetworkfiles.sum | awk -F ':' '{print $2}'`
sum $f | awk '{print $1":"$3}'

Which in this case generated the following output:


Comparing the latest with the cat hmcnetworkfiles.sum command proves that the data transferred successfully.

Posted in Real life AIX.

Tagged with , , .

map FC devices on aix box

A host with a large number of FC adapters, disks, tape drives, and so forth occasionally may put you in a difficult situation, especially when during a meeting someone asks you for “what is where”…..
The following few lines of shell can help you respond.


# W.M.Duszyk 2/12
# map FC devices to their adapters and interfaces

for f in $(lsdev|awk '/fcs/{print $1}'); do
        printf "${f} - "
        j=$(lsdev -p $f|awk '/fscsi/{print $1}')
        echo "${j}:"
        z=$(lspath -p $j)
        if [ -n "$z" ]
                 echo "$z"|awk '/rmt|smc|disk|d1/{printf "%s ", $2}END{print "\n"}'
                 lsdev -p $j|awk '/rmt|smc|disk/{printf "%s ", $1}END{print "\n"}'

For example:

# ./MapDev.ksh
fcs0 - fscsi0:

fcs1 - fscsi1:

fcs2 - fscsi2:

fcs3 - fscsi3: hdisk30 hdisk31 hdisk32 hdisk33 hdisk34 hdisk35 hdisk36 hdisk37 hdisk38 hdisk39 hdisk40 hdisk41 hdisk42 hdisk43 hdisk44 hdisk45 hdisk46 hdisk47 hdisk48 hdisk49 hdisk50 hdisk51 hdisk52

fcs4 - fscsi4: hdisk30 hdisk31 hdisk32 hdisk33 hdisk34 hdisk35 hdisk36 hdisk37 hdisk38 hdisk39 hdisk40 hdisk41 hdisk42 hdisk43 hdisk44 hdisk45 hdisk46 hdisk47 hdisk48 hdisk49 hdisk50 hdisk51 hdisk52

fcs5 - fscsi5:

fcs6 - fscsi6: rmt12 rmt13 rmt14 rmt15 rmt54 rmt55 rmt82 rmt83 rmt100 rmt101 rmt102 rmt103 rmt132 rmt133 rmt154 rmt155 rmt168 rmt169 rmt170 rmt171 rmt196 rmt197 rmt198 rmt199 rmt226 rmt227 rmt248 rmt249 rmt250 rmt251 rmt252 rmt253 rmt254 rmt255 rmt256 rmt257 rmt258 rmt259 rmt286 rmt287 rmt288 rmt289 smc1 smc11 smc17

fcs7 - fscsi7: rmt16 rmt17 rmt18 rmt19 rmt44 rmt45 rmt46 rmt47 rmt56 rmt57 rmt64 rmt65 rmt66 rmt104 rmt105 rmt106 rmt107 rmt134 rmt135 rmt156 rmt157 rmt172 rmt173 rmt174 rmt175 rmt200 rmt201 rmt202 rmt203 rmt228 rmt229 rmt296 rmt297 rmt298 rmt299 rmt300 rmt301 rmt302 rmt303 rmt304 rmt305 rmt306 rmt307 smc2 smc9 smc19

fcs8 - fscsi8: rmt20 rmt21 rmt22 rmt23 rmt58 rmt59 rmt67 rmt68 rmt69 rmt108 rmt109 rmt110 rmt111 rmt116 rmt117 rmt118 rmt158 rmt159 rmt176 rmt177 rmt178 rmt179 rmt204 rmt205 rmt206 rmt207 rmt230 rmt231 rmt260 rmt261 rmt262 rmt263 rmt264 rmt265 rmt266 rmt267 rmt268 rmt269 rmt270 rmt271 rmt336 rmt337 rmt338 rmt339 rmt340 rmt341 smc3 smc12 smc23

fcs9 - fscsi9: rmt24 rmt25 rmt26 rmt27 rmt60 rmt61 rmt70 rmt71 rmt72 rmt112 rmt113 rmt114 rmt115 rmt119 rmt120 rmt121 rmt140 rmt141 rmt142 rmt180 rmt181 rmt182 rmt183 rmt208 rmt209 rmt210 rmt211 rmt232 rmt233 rmt272 rmt273 rmt274 rmt275 rmt276 rmt277 rmt278 rmt279 rmt280 rmt281 rmt282 rmt283 smc4 smc13

fcs10 - fscsi10:rmt28 rmt29 rmt30 rmt31 rmt62 rmt63 rmt73 rmt74 rmt75 rmt84 rmt85 rmt86 rmt87 rmt122 rmt123 rmt124 rmt143 rmt144 rmt145 rmt184 rmt185 rmt186 rmt187 rmt212 rmt213 rmt214 rmt215 rmt234 rmt235 rmt236 rmt237 rmt238 rmt239 rmt240 rmt241 rmt242 rmt243 rmt244 rmt245 rmt246 rmt247 rmt284 rmt285 smc5 smc14 smc16

fcs11 - fscsi11: rmt32 rmt33 rmt34 rmt35 rmt48 rmt49 rmt76 rmt77 rmt88 rmt89 rmt90 rmt91 rmt125 rmt126 rmt127 rmt146 rmt147 rmt148 rmt188 rmt189 rmt190 rmt191 rmt216 rmt217 rmt218 rmt219 rmt308 rmt309 rmt310 rmt311 rmt324 rmt325 rmt326 rmt327 rmt328 rmt329 rmt330 rmt331 rmt332 rmt333 rmt334 rmt335 smc6 smc15 smc20 smc22

fcs12 - fscsi12: rmt0 rmt1 rmt2 rmt3 rmt36 rmt37 rmt38 rmt39 rmt50 rmt51 rmt78 rmt79 rmt92 rmt93 rmt94 rmt95 rmt128 rmt129 rmt136 rmt137 rmt138 rmt139 rmt149 rmt150 rmt151 rmt160 rmt161 rmt162 rmt163 rmt220 rmt221 rmt222 rmt223 rmt312 rmt313 rmt314 rmt315 rmt316 rmt317 rmt318 rmt319 rmt320 rmt321 rmt322 rmt323 smc7 smc10 smc21

fcs13 - fscsi13: rmt40 rmt41 rmt42 rmt43 rmt52 rmt53 rmt80 rmt81 rmt96 rmt97 rmt98 rmt99 rmt130 rmt131 rmt152 rmt153 rmt164 rmt165 rmt166 rmt167 rmt192 rmt193 rmt194 rmt195 rmt224 rmt225 rmt290 rmt291 rmt292 rmt293 rmt294 rmt295 smc8 smc18

Posted in Real life AIX.

mounting cifs from WIN2012

WIN2012 is not really supported….. These were the words of IBM engineer working on my PMR which I opened to find out how to mount WIN2012 shares (cifs)….. Well we can mount cifs from the previous releases of WINDOWS OS but not 2012 one. Next, he emailed me a link to Microsoft document explaining how to disable SMB signing – the reason for the failure. I was advised to follow the WIN2003 procedure with still applies to WIN2012.

You can have a look at the document following this link – “Overview of Server Message Block signing

To mount CIFS share in AIX:

# mount -v cifs -n lawisnqw1/lawsona/law1199 \
                 -o wrkgrp=wmd-edu,fmode=755 /tjtest /tjtest


# mount -v cifs -n WINhost/WinUser/WinUserPassword \
                -o wrkgrp=wmd-edu,fmode=755 /ShareName /AIXmountPoint

To mount CIFS in RedHat6:

# mount.cifs //lawisnqw1/tjtest /test \
                -o username=lawsona,password=law1199,domain=wmd-edu

Now, let’s wait for Igor to get a moment of free time to disable SMB singing and check if this will put this request to bed…. :-)

Posted in Linux, Real life AIX.

Kerberos, Active Directory and ftp

It is not surprising that more and more users looks for Active Directory as the way to unify and to simplify the way to authenticate/authorize users and to save some money too.
After all Active Directory and Windows are almost in every office on this planet. Not to mention that if you work in a heavily audited environment the ability to have a single store of users definitions is really a blessing!

For about the last two months now, I am trying to get ftp working for AIX users authenticated with Kerberos services provided by Active Directory (2012).
I am not doing it alone, I have an open PMR and after numerous iptraces, snaps, and so forth my KERBEROS authenticated users still cannot use ftp…. and this sucks!
The locally (admin accounts) defined users can but the “flash & bone” users, the ones “living” in Active Directory cannot.

Is there someone out there who got this working and who is willing to share his/her knowledge, please?


Posted in Real life AIX.

Tagged with .

nim client removal

I have not done any patching for a while and today, when I had to remove a nim client definition I could not remember the second command to use. Now, I do so here it is for the record the process:

First, reset the client

# nim -F -o reset NIM_CLIENT_NAME

Now, remove all associated with the client resources.

# nim -o deallocate -a subclass=all  NIM_CLIENT_NAME

At this stage the client can be removed.

# nim -o remove -F  NIM_CLIENT_NAME

the NIM_CLIENT_NAME is the hostname of the client to be removed.

Posted in Real life AIX.

Tagged with , .

© 2008-2014 - best viewed with your eyes.