Skip to content


Integrating Red Hat with Active Directory

This morning, I found this publication showing everything required for a successful integration with AD – including time synchronization, DNS, Samba setup and so forth.
If you are branching to LINUX or just have to support LINUX in addition to AIX then this document may help you.

Integrating Red Hat with Active Directory

Posted in Linux.

Tagged with , , , , , , .


a nice WMWARE link

My friend Adam is into WMWARE.Today, he sent me an email with this link, which he highly recommends. If you are “into” VMWARE this could be something for you too.

http://vsphere-land.com/

My friend Tony, recommends this link

http://planet.vsphere-land.com/

Posted in Real life AIX.

Tagged with .


any issues upgrading to AIX 6.1.7.5?

Recently, we upgraded two machines to AIX 6.1.7.5 and both experienced incidents of very heavy paging to the point that they had to be rebooted – we are not sure that there is any relation between this upgrade and paging. Has anybody else experienced the same?
If so, please let me know – we have to patch few extremely “important” machines and we do not want to complicate our lives.

Thanks!!!!

Update:

so far it looks like TL7SP5 automatically turns ON support for 64kb memory pages. You can check it executing vmstat –P ALL. To disable this feature, execute vmo -r -o vmm_mpsize_support=0, agree to update multibos and reboot the system.

Posted in Real life AIX.


when ShadowImage pairs do not cooperate

After a while, a long while indeed we had to move our ShadowImage based backup process to another server. When I say server, I mean the host with active horcm processes i.e. the host which owns the P-VOLS.
After a few modifications to our backup scripts we were ready to go. We queried the disks status and were immediately surprised – the pairs were not in the same state!

pairdisplay -g epicdb_SI1 -CLI -fcx -IM0
Group   PairVol L/R   Port# TID  LU-M   Seq# LDEV# P/S Status % P-LDEV# M
epicdb_SI1 000A-0055 L   CL1-A-1  0 2 0  29386 a P-VOL PAIR  99 55 -
epicdb_SI1 000A-0055 R   CL3-B-0  0 0 0  29386 55 S-VOL PAIR  99  a -
epicdb_SI1 000D-0056 L   CL1-A-1  0 3 0  29386 d P-VOL PAIR  99 56 -
.....................................................................
epicdb_SI1 00A5-00AB L   CL1-A-1  0 19 0  29386 a5 P-VOL COPY 94 ab -
epicdb_SI1 00A5-00AB R   CL3-B-2  0 15 0  29386 ab S-VOL COPY 94 a5 -
epicdb_SI1 00A6-00AC L   CL1-A-1  0 20 0  29386 a6 P-VOL COPY 94 ac -
epicdb_SI1 00A6-00AC R   CL3-B-2  0 16 0  29386 ac S-VOL COPY 94 a6 -
epicdb_SI1 00A7-00AD L   CL1-A-1  0 21 0  29386 a7 P-VOL COPY 94 ad -
epicdb_SI1 00A7-00AD R   CL3-B-2  0 17 0  29386 ad S-VOL COPY 94 a7 

The pairs were not in the same state and nobody could say why it is like that. The decision was made to split the whole consistency group.

# pairsplit -g epicdb_SI1 -d 000A-0055 -IM0

# pairdisplay -g epicdb_SI1 -CLI -fcx -IM0
Group   PairVol L/R   Port# TID  LU-M   Seq# LDEV# P/S Status % P-LDEV# M
epicdb_SI1 000A-0055 L   CL1-A-1  0 2 0  29386  a P-VOL PSUS  99 55 W
epicdb_SI1 000A-0055 R   CL3-B-0  0 0 0  29386 55 S-VOL COPY  99  a -
epicdb_SI1 000D-0056 L   CL1-A-1  0 3 0  29386  d P-VOL PAIR  99 56 -
epicdb_SI1 000D-0056 R   CL3-B-0  0 1 0  29386 56 S-VOL PAIR  99  d -
epicdb_SI1 0013-0057 L   CL1-A-1  0 4 0  29386 13 P-VOL PAIR  99 57 -
epicdb_SI1 0013-0057 R   CL3-B-0  0 2 0  29386 57 S-VOL PAIR  99 13 -
.....................................................................
epicdb_SI1 00A5-00AB L   CL1-A-1  0 19 0  29386 a5 P-VOL COPY 94 ab -
epicdb_SI1 00A5-00AB R   CL3-B-2  0 15 0  29386 ab S-VOL COPY 94 a5 -
epicdb_SI1 00A6-00AC L   CL1-A-1  0 20 0  29386 a6 P-VOL COPY 94 ac -
epicdb_SI1 00A6-00AC R   CL3-B-2  0 16 0  29386 ac S-VOL COPY 94 a6 -
epicdb_SI1 00A7-00AD L   CL1-A-1  0 21 0  29386 a7 P-VOL COPY 94 ad -
epicdb_SI1 00A7-00AD R   CL3-B-2  0 17 0  29386 ad S-VOL COPY 94 a7 

As you see, the split worked for some but not all disk pairs. Something is definitely rotten in the state Denmark (do not get me wrong, I love the state of Denmark, for me Copenhagen is the most beautiful city ever).

But the way, trying to split pairs one by one (as shown bellow) did not change the situation neither.

# pairsplit -g epicdb_SI1 -d 000D-0056 -IM0
# pairsplit -g epicdb_SI1 -d 0013-0057 -IM0

In the last two cases HORCM produced the following messages:

pairsplit: [EX_CMDRJE] An order to the control/command device was rejected. Refer to the command log(/HORCM/log0/horcc_marcoPolo.log) for details. It was rejected due to SKEY=0x05, ASC=0x26, SSB=0xB9A1,0x2310 on Serial#(XXXXX)

It was SAN administrator idea to split the pair using the -S – apparently it was time to stop being a nice AIX administrator that I always am. This execution will delete disk pairs, which meant (at least to us) that all observed inconsistencies will be gone too. At the end the disks should show SMPL as their MODE.

pairsplit -g epicdb_SI1 -S -IM0

The following execution of pairdisply command shows all disks in SMPL mode – their basic state at which there are no PAIRs. Now, the PAIRs must be re-created executing the paircrate command.

paircreate -g epicdb_SI1 -m grp -vl -IM0

At this moment we were very happy campers indeed. Little we knew that the next scheduled backup will destined to fail due to recreatevg fussing about one of the disks not being a member of the same volume group as the rest of them ….. It was a very short night indeed, little sleep but a lot of stress – this is the life I have chosen, and I love it!
🙂

Posted in Real life AIX.


to prevent an error from showing in errorlog

Sometimes the contents of the error log are not really errors but still they are interpreted as ones – it is in the error log so it must be an error. Right? Well, for a lot of analyst this is true for a lot of admins not always. Today, one machine error log showed a number of entries defined by

LABEL:          DR_DMA_MIGRATE_FAIL
IDENTIFIER:     4DA8FE60
.................................
Description
Memory related DR operation failed

These messages generated a lot of excitement today …. Verifying firmware, FC adapters settings and a few others “things” not to mention calling for IBM support we determined beyond any doubt that they represent no harm. It is time to turn the off with the errupdate command. What follows is self-explanatory.

# errupdate
=4DA8FE60: 
Report=False 
<Ctrl-D>
<Ctrl-D>

If you change your mind and want to log these messages again do the following:

# errupdate
-4DA8FE60:
<Ctrl-D>
<Ctrl-D>

Posted in Real life AIX.

Tagged with , .



installing WebSphere Server in a wpar

Annwoy, just made the following comment about installing WebSphere in a wpar: “it will save you time if before attempting to do this for the first time, you spend a few minutes following and reading this link” http://www-01.ibm.com/support/docview.wss?uid=swg21293695 . It looks like he was right.

Posted in AIX, wpar.

Tagged with , , .


SAN storage migration for lpar and its wpar

Here we go again! It is time to migrate from SVC to XIV – the decision has been made! This post shows my approach with this aspect of AIX life. For the sake of simplicity I will show interaction between one lpar and one wpar. The same procedure applies to one or many wpars.

Posted in Real life AIX, wpar.

Tagged with , , , , , .


Nowe Sutry

pod Sutrami znajdziesz moje dwa nowe tlumaczenia – Sutra Vimalakitri oraz Sutra Siedmiu Słońc czyli Sutra Końca Świata.

Posted in BuddhaTaoZen, Sutry.

Tagged with , , , , , , .


missing disks, dump devices, mirroring, etc.

Since, we are pretty much always learning …. Some of us on a more elementary, intermediate, or advanced level but regardless of the level we all always learn or re-learn (because what we have mastered we had an ample time to forget – not doing it for a while), here it is a reminder of how to deal with a volume group (in this case it is rootvg which for some reasons lost one of it disks. The loss could be a function of SAN, VIO or other event. It could be a permanent loss – a disk is dead, broken, no longer functioning or the loss was/is temporary in nature; AIX kernel detected a timeout (without any disk errors associated with device failure) long enough for the kernel to mark the disks missing.

If faced with such situation, I always hope for the “temporary” and execute the varyonvg vg_nam command in order to re-establish communication path to the offending disk. If this action fails, than I investigate command errpt -a to hopefully find more details to gain better understanding of what is really going on.

For the last few days, I am dealing with a peculiar issue of one specific user having login issues with a specific AIX host. The password is delivered via LDAP, the users is “kicked out” as soon as he logs in so to deal with this witchcraft I decided that it is time to download and to update the client’s side of LDAP (TDS LDAP filesets on the host) – not that I am 100% convinced that this will save the day. To accommodate the download, I had to increase the capacity of the /root file system (root’s home directory). While doing so, the following message showed on my screen.

# chfs -a size=+1G /root
lquerypv: Warning, physical volume hdisk1 is excluded since it may be either missing or removed.
0516-404 allocp: This system cannot fulfill the allocation request. There are not enough free partitions or not enough physical volumes to keep strictness and satisfy allocation requests.  The command  should be retried with different allocation characteristics.

By the way, where is Tivoli Monitoring when you need it? Well, I do not see any messages in my mailbox and it could be that it is not Tivoli but how it (Tivoli) has been configured ……..

It is the time to follow my usual procedure in such case, let’s vary it on!

# varyonvg rootvg
0516-1774 varyonvg: Cannot varyon volume group with an active dump device on a missing physical volume. Use sysdumpdev to temporarily replace the dump device with /dev/sysdumpnull and try again.

Our AIX hosts always have two dump devices (logical volumes) – one per the rootvg disk. The primary in always located on hdisk0, the secondary is always on hdisk1. If I did not know it, to establish what dump device resides on a missing disk, I would execute the sysdumpdev command followed with lslv -m lv_name (for example).

To deactivate the dump device requires the following re-direction:

# sysdumpdev -s /dev/sysdumpnull
primary              /dev/dump0
secondary            /dev/sysdumpnull
copy directory       /var/adm/ras
forced copy flag     TRUE
always allow dump    TRUE
dump compression     ON
type of dump         traditional

As seen above, the primary dump is still live and kicking. Now, let’s try again to varyon the volume group:

# varyonvg rootvg
# lsvg -p rootvg
rootvg:
PV_NAME  PV STATE    TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk1   active      559         346         87..05..30..112..112
hdisk0   active      559         346         87..05..30..112..112

This time, there are no complaints and the disk is meditatively listed as active. This volume group is always mirrored (in our case), we need to verify that the syncvg has been automatically started.

# ps -ef | grep sync
root   655432 1 0   Jun 26      - 47:47 /usr/sbin/syncd 60
root  3735800 1 0 10:10:35  pts/0  0:00 /bin/ksh /usr/sbin/syncvg -v rootvg
root 16842916  3735800   1 10:10:46  pts/0  0:00 lresynclv -P 24 -l 00f66d0c871

Yes, the volume group is already being synchronized. The last step is to restore the dump environment:

# sysdumpdev -P -s /dev/dump1
primary              /dev/dump0
secondary            /dev/dump1
copy directory       /var/adm/ras
forced copy flag     TRUE
always allow dump    TRUE
dump compression     ON
type of dump         traditional

Well, the previous step was not the last – Now, I will be able to increase to size of /root – which is the “last” step. 🙂

Posted in AIX, Real life AIX.

Tagged with , , , , .




Copyright © 2015 - 2016 Waldemar Mark Duszyk. - best viewed with your eyes.. Created by Blog Copyright.