Due to moving into the new office I had to change several configurations in my infrastructure. Sure, the Linux and Windows systems were changed very quickly, the virtual machines did not cause any problems either. The switch of my RAC cluster then was a little bit more difficult, because it is not all done by changing the entries in the /etc/hosts, there is actually much more required.
To spare you this long lasting and hurtful experience, I stated the important steps for a successful switch in this article. I got great support for it from this article: http://zhefeng.wordpress.com/2009/02/02/change-ip-address-for-oracle-rac-public-and-vip-interfaces
The base is my database RACVM1 on the cluster gallier with the nodes asterix and obelix (see my blog: “Oracle RAC 11.2.0.3 on VMware ESXi”).
The hosts file originally looked like this:
# SCAN 192.168.151.60 gallier.carajandb.com gallier # Public 192.168.151.61 asterix.carajandb.com asterix 192.168.151.62 obelix.carajandb.com obelix # VIP 192.168.151.71 asterix-vip.carajandb.com asterix-vip 192.168.151.72 obelix-vip.carajandb.com obelix-vip # Interconnect 31.10.151.61 asterix-priv.carajandb.com asterix-priv 31.10.151.62 obelix-priv.carajandb.com obelix-priv
Only the public IP addresses shall be changed from 192.168.151.x to 192.168.152.x . The private addresses for the interconnect are kept for now.
Stop the Applications
Before we can start the switch, the database should be shut down first. When working RAC databases you should take care not to use the SQL*Plus for this, but only the command srvctl (even if it is not really telling).
$ srvctl stop database -d RACVM1
Next all other applications (including the VIP addresses) on the nodes are stopped.
$ srvctl stop nodeapps -n asterix $ srvctl stop nodeapps -n obelix
All applications but ASM and listener should be stopped by that. This can be checked by:
$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE asterix ora....ER.lsnr ora....er.type ONLINE OFFLINE ora....N1.lsnr ora....er.type OFFLINE OFFLINE ora.asm ora.asm.type ONLINE ONLINE asterix ora....SM1.asm application ONLINE ONLINE asterix ora....IX.lsnr application ONLINE OFFLINE ora....rix.gsd application OFFLINE OFFLINE ora....rix.ons application OFFLINE OFFLINE ora....rix.vip ora....t1.type OFFLINE OFFLINE ora.cvu ora.cvu.type OFFLINE OFFLINE ora.gsd ora.gsd.type OFFLINE OFFLINE ora....network ora....rk.type OFFLINE OFFLINE ora....SM2.asm application ONLINE ONLINE obelix ora....IX.lsnr application ONLINE OFFLINE ora.obelix.gsd application OFFLINE OFFLINE ora.obelix.ons application OFFLINE OFFLINE ora.obelix.vip ora....t1.type OFFLINE OFFLINE ora.oc4j ora.oc4j.type ONLINE ONLINE asterix ora.ons ora.ons.type OFFLINE OFFLINE ora.racvm1.db ora....se.type OFFLINE OFFLINE ora....rod.svc ora....ce.type OFFLINE OFFLINE ora....est.svc ora....ce.type OFFLINE OFFLINE ora....st1.svc ora....ce.type OFFLINE OFFLINE ora....st2.svc ora....ce.type OFFLINE OFFLINE ora....ry.acfs ora....fs.type ONLINE ONLINE asterix ora.scan1.vip ora....ip.type OFFLINE OFFLINE
Who wants to be completely sure should now make a backup of OCR and voting disk. As there was not yet much to be damaged on my test database I skipped that.
Change Addresses in the Oracle Configuration
Next you should have a look at the current configuration of the adapters (the command vipca does not exist anymore in 11.2 as it is now working with the SCAN address, strangely the file vipca still exists in the $ORACLE_HOME/bin, though it is empty).
$ oifcfg getif eth1 31.10.151.0 global cluster_interconnect eth0 192.168.151.0 global public
As I already mentioned the public IP addresses (i.e. asterix, obelix, asterix-vip and obelix-vip) shall be changed. Even though it cost me some negotiation, it actually works to first delete the entry for eth0 and recreate it then:
$ oifcfg delif -global eth0 $ oifcfg setif -global eth0/192.168.152.0:public
Now the VIP addresses have to be changed. Of course, after an appropriate check:
$ srvctl config nodeapps -a Network exists: 1/192.168.151.0/255.255.255.0/eth0, type static VIP exists: /192.168.151.71/192.168.151.71/192.168.151.0/255.255.255.0/eth0, hosting node asterix VIP exists: /192.168.151.72/192.168.151.72/192.168.151.0/255.255.255.0/eth0, hosting node obelix
… and now the change:
# srvctl modify nodeapps -n asterix -A 192.168.152.0/255.255.255.0/eth0 # srvctl modify nodeapps -n obelix -A 192.168.152.0/255.255.255.0/eth0
… and the check, of course:
# srvctl config nodeapps -a Network exists: 1/192.168.152.0/255.255.255.0/eth0, type static VIP exists: /192.168.152.71/192.168.152.71/192.168.152.0/255.255.255.0/eth0, hosting node asterix VIP exists: /192.168.152.72/192.168.152.72/192.168.152.0/255.255.255.0/eth0, hosting node obelix
Change the configuration of the operating system.
Now the/etc/hostsfile can be modified accordingly, so that they look like this afterwards:
# SCAN 192.168.152.60 gallier.carajandb.com gallier # Public 192.168.152.61 asterix.carajandb.com asterix 192.168.152.62 obelix.carajandb.com obelix # VIP 192.168.152.71 asterix-vip.carajandb.com asterix-vip 192.168.152.72 obelix-vip.carajandb.com obelix-vip # Interconnect 31.10.151.61 asterix-priv.carajandb.com asterix-priv 31.10.151.62 obelix-priv.carajandb.com obelix-priv
The last change affects the Linux network settings. It is your choice to either use the toll network connections or to edit the entries in/etc/sysconfig/network-scriptsmanually.
This is the example of the entry inifcfg-eth0for asterix:
TYPE=Ethernet BOOTPROTO=none IPADDR=192.168.152.61 PREFIX=24 GATEWAY=192.168.152.1 DNS1=192.168.152.1 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME=eth0 UUID=d04279f7-dd5a-40fa-ace8-d50fd62f50d7 ONBOOT=yes HWADDR=00:0C:29:67:66:D5
When adjusting manually you additionally have to change the default gateway to/etc/sysconfig/network.
A good advice from the article also was to adjust the file known_host accordingly for the ssh connections between the systems.
Restart with the new Configuration
Now, if you made the changes on the network manually, the network can be started completely on both systems once.
# service network restart
Last but not least the crs daemon is stopped on both nodes and restarted afterwards.
# crsctl stop crs # crsctl start crs
As a conclusion I would recommend to start completely once with machines to verify that the resources are being started again also after a reboot. As you stopped the database, you may in the end start it again – but please use the command srvctl!
$ srvctl start database -d RACVM1
Besides that, good luck with you migration and I appreciate your comments.