RAC PSU 11.2.0.3.5 to Grid Infrastructure/DB Home

Applying PSU 11.2.0.3.5 to Grid Infrastructure and DB Home

Applying PSU 11.2.0.3.5 is pretty simple and straighforward task. I am documenting this as easy reference for people looking for steps. These steps have been tried on Base 11.2.0.3 software on RHEL5.6 on Linux x86-64.

1) We must use OPatch utility version 11.2.0.3.0 or later to apply this patch. For my 11.2.0.3 base install, opatch version is 11.2.0.1.7

[oracle@devdb-005]~% $ORACLE_HOME/OPatch/opatch version
Invoking OPatch 11.2.0.1.7

OPatch Version: 11.2.0.1.7

OPatch succeeded.

So we need to first download patch 6880880

I used following wget command to directly download the patch to my server.

read username
read -s h_passwd
wget --http-user="$username" --http-password="$h_passwd" --output-document="p6880880_112000_Linux-x86-64.zip" "https://updates.oracle.com/Orion/Services/download/p6880880_112000_Linux-x86-64.zip?aru=15741318&patch_file=p6880880_112000_Linux-x86-64.zip"

The new opatch utility should be updated in all the Oracle RAC database homes and the GI home that are being patched.
unzip the patch and then copy to both nodes

unzip p6880880_112000_Linux-x86-64.zip
[oracle@devdb-005]/oracle/product/11.2.0.3/dbhome_1% mv OPatch OPatch.old
[oracle@devdb-005]/oracle/product/11.2.0.3/dbhome_1% cp -pr /home/oracle/OPatch .

Similarly for Grid home but this time as root

[root@devdb-005 ~]# cd /oragrid/11.2.0.3/grid
[root@devdb-005 grid]# mv OPatch/ OPatch.old
[root@devdb-005 grid]# cp -pr /home/oracle/OPatch .

Repeat for other nodes in cluster. Since this is 2 node RAC, I repeated steps on devdb-006.

Verify Opatch version

[oracle@devdb-005]~% $ORACLE_HOME/OPatch/opatch version    
OPatch Version: 11.2.0.3.3

OPatch succeeded.

[oracle@devdb-005]~% $ORACLE_GRID/OPatch/opatch version
OPatch Version: 11.2.0.3.3

OPatch succeeded.

2)Opatch requires ocm.rsp file. Note 966023.1 describes How To Create An OCM Response File For Opatch Silent Installation
Let's create one for our setup in current directory. I didn't specify my MOS credentials

[oracle@devdb-005]~% $ORACLE_HOME/OPatch/ocm/bin/emocmrsp   -no_banner
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name: 

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (ocm.rsp) was successfully created.

3) Oracle recommends to run following command to verify consistency of all homes being patched.

$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME

In case there is some issue with inventory , it will report error. In that case you should first fix the error before proceeding with patching.
This will also give the list of currently applied patches.

4)In case you have configured DB Console for RAC, it should be stopped before patching

As the Oracle RAC database home owner execute:

$ $ORACLE_HOME/bin/emctl stop dbconsole

5)Let's download the patch 14727347- GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.5 (INCLUDES DB PSU 11.2.0.3.5)

read username
read -s h_passwd
wget --http-user="$username" --http-password="$h_passwd" --output-document="p14727347_112030_Linux-x86-64.zip" "https://updates.oracle.com/Orion/Services/download/p14727347_112030_Linux-x86-64.zip?aru=15680307&patch_file=p14727347_112030_Linux-x86-64.zip"

6) Finally we come to patching step. If you are not using shared homes and ACFS, then patching is straight forward i.e use opatch auto. In case you are using ACFS or Shared home, then refer to patch readme here
You need to run this command as root from one node at a time

#/oragrid/11.2.0.3/grid/OPatch/opatch auto /home/oracle/psupatch/ -ocmrf /home/oracle/ocm.rsp

This will shutdown instance followed by Clusterware shutdown. Finally it will patch both Grid home and database. You would see following output in end

patch /home/oracle/psupatch/15876003 apply successful for home /oragrid/11.2.0.3/grid
patch /home/oracle/psupatch/14727310 apply failed for home /oragrid/11.2.0.3/grid
CRS-4123: Oracle High Availability Services has been started.

7)Run catbundle.sql from each database running from DB home. In RAC this needs to be done only one node

@catbundle.sql psu apply

Check the following log files in $ORACLE_BASE/cfgtoollogs/catbundle for any errors:

catbundle_PSU__APPLY_.log
catbundle_PSU__GENERATE_.log

As you can see patching process is simplified as compared to 10.2 patch bundles when you needed to execute some scripts as root and then as oracle .