PSU 2 node RAC(No DR) - Applying PSU Patch 11.2.0.1.2 to a 2 Node RAC

Applying PSU Patch 11.2.0.1.2 To A Two Node RAC

Posted by Srikrishna Murthy Annam on May 5, 2011
NOTE : This article is aimed at showing the issues and resolution while applying the PSU patch 11.2.0.1.2 (9655006) to a specific environment and not generalized. Please refer the readme of the PSU patch for detailed procedure to apply PSU patch.
Video Demos are also uploaded to show you the errors and successfull installation of the PSU to 2 node RAC.

1. Record Pre Patch Information.

  1. Login to each node in RAC as grid user and execute the following command.
    $GRID_ORACLE_HOME/OPatch/opatch lsinventory
    $GRID_ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed | grep -i ‘GI PSU’
  2. Login to each node in RAC as oracle user and execute the following command.
    $ORACLE_HOME/OPatch/opatch lsinventory
    $ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed | grep -i ‘DATABASE PSU’
  3. Connect to each instance and record registry information.
    SQL> select comp_name,version,status from dba_registry;

2. OPatch Utility Information.

$ORACLE_HOME/OPatch/opatch version -h /u01/home/oracle/product/11.2.0/db_1
$GRID_ORACLE_HOME/OPatch/opatch version -h /u01/home/11.2.0/grid

3. OCM Configuration.

Create ocm response file using the following command and provide appropriate values for the prompts.
$GRID_ORACLE_HOME/OPatch/ocm/bin/emocmrsp
Verify the created file using,
$GRID_ORACLE_HOME/OPatch/ocm/bin/emocmrsp –verbose ocm.rsp
NOTE: The Opatch utility will prompt for your OCM (Oracle Configuration Manager) response file when it is run. Without which we cant proceed further.

4. Validation of Oracle Inventory.

$GRID_ORACLE_HOME/OPatch/opatch lsinventory -detail -oh /u01/home/11.2.0/grid
$ORACLE_HOME/OPatch/opatch lsinventory -detail –oh /u01/home/oracle/product/11.2.0/db_1

5. One-off Patch Conflict Detection and Resolution.

NA

6. Download and Unzip the PSU Patch 9655006

$cd /u01/home/oracle/product/11.2.0/db_1/patches/psupatch
$unzip p9655006_11201_Linux.zip
$chmow -R 777 *

7. Patching GI Home

NOTE: If the GI home is shared, then make sure to shut down the GI stack on all remote nodes. Keep the GI stack up and running on the local node.
NOTE: If the GI home is not shared, then make sure the GI stack is running on all nodes in the cluster.
Our Grid Home is not shared, So don’t shutdown any services.
$su – ( Login to root user )
#/u01/home/11.2.0/grid/OPatch/opatch auto /u01/home/oracle/product/11.2.0/db_1/patches/psupatch -oh /u01/home/11.2.0/grid
Monitor the logfile created in $GRID_ORACLE_HOME/cfgtoollogs/
Execute the above opatch command on each RAC node as root user.
** Please refer the Issue & Resolutions secion in the same document for any issues.

8. Patching RAC Database Homes

All Oracle processes and applications (such as emconsole and emagent) that are running from the database home and that are not managed by clusterware should be stopped manually before you apply the patch using the opatch auto command.
$su – ( Login to root user )
# /u01/home/oracle/product/11.2.0/db_1/OPatch/opatch auto /u01/home/oracle/product/11.2.0/db_1/patches/psupatch -oh /u01/home/oracle/product/11.2.0/db_1
Monitor the logfile created in $ORACLE_HOME/cfgtoollogs/
Execute the above opatch command on each RAC node as root user.
** Please refer the Issue & Resolutions secion in the same document for any issues.

9. Loading Modified SQL Files into the Database.

For each database instance running on the Oracle home being patched, connect to the database using SQL*Plus. Connect as SYSDBA and run the catbundle.sql script as follows:
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @catbundle.sql psu apply
SQL> QUIT
Check the log files in $ORACLE_HOME/cfgtoollogs/catbundle for any errors

10. Patch Successful Verification Steps.

  1. Login to each node in RAC as grid user and execute the following command.
    $GRID_ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed | grep -i ‘GI PSU’
  2. Login to each node in RAC as oracle user and execute the following command.
    $ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed | grep -i ‘DATABASE PSU’
  3. Connect to each instance and record registry information.
    SQL> select comp_name,version,status from dba_registry;
    SQL> select * from dba_registry_history;

11. Issues & Resolutions.

Issue 1:
When applying the patch, you may get the following error
The opatch minimum version check for patch /u01/home/oracle/product/11.2.0/db_1/patches/9655006/custom failed for /u01/home/11.2.0/grid
The opatch minimum version check for patch /u01/home/oracle/product/11.2.0/db_1/patches/9655006/etc failed for /u01/home/11.2.0/grid
The opatch minimum version check for patch /u01/home/oracle/product/11.2.0/db_1/patches/9655006/files failed for /u01/home/11.2.0/grid
Opatch version check failed for oracle home /u01/home/11.2.0/grid
Opatch version check failed
update the opatch version for the failed homes and retry
Solution:
Ref Note : 1308858.1
We need to provide the Patch unzipped base directory, not the directory including patch number
Ex : /u01/home/oracle/product/11.2.0/db_1/patches/psupatch/9655006 ( Wrong )
/u01/home/oracle/product/11.2.0/db_1/patches/psupatch ( correct )
Issue 2
Patch may exit with the following error messages
Unable to determine if /u01/home/11.2.0/grid is shared oracle home
Enter ‘yes’ if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no):yes
You must kill crs processes or reboot the system to properly
cleanup the processes started by Oracle clusterware
The Oracle Clusterware stack failed to stop.
You should stop the stack with ‘crsctl stop crs’ and rerun the command
The opatch Applicable check failed for /u01/home/11.2.0/grid. The patch is not applicable for /u01/home/11.2.0/grid
patch ././9655006 apply failed for home /u01/home/11.2.0/grid
Solution :
This error may be specific to this environment though want to specify it here. This error is due to the reason that there are some cluster resources available referring the 10g database installed earlier. Delete those resources from clusterware.
# crsctl delete resource -f
Issue 3 :
PSU patch for GRID home on node 2 failed with the following error
Unable to determine if /u01/home/11.2.0/grid is shared oracle home
Enter ‘yes’ if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no):yes
Successfully unlock /u01/home/11.2.0/grid
patch ././9655006 apply failed for home /u01/home/11.2.0/grid
Verified the detailed log file locate in /u01/home/11.2.0/grid/cfgtoollogs/ and found the permission issue on some files.
The following actions have failed:
Copy failed from ‘/u01/home/oracle/product/11.2.0/db_1/patches/psupatch/9655006/files/bin/crsctl.bin’ to ‘/u01/home/11.2.0/grid/bin/crsctl.bin’…
Copy failed from ‘/u01/home/oracle/product/11.2.0/db_1/patches/psupatch/9655006/files/bin/oifcfg.bin’ to ‘/u01/home/11.2.0/grid/bin/oifcfg.bin’…
Solution:
Give 777 permission to these files crsctl.bin and oifcfg.bin
Issue 4 :
PSU Patch for RDBMS Home on node 1 failed with the following error
Unable to determine if /u01/home/oracle/product/11.2.0/db_1 is shared oracle home
Enter ‘yes’ if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no):yes
patch ././9655006/custom/server/9655006 apply failed for home /u01/home/oracle/product/11.2.0/db_1
Verified the detailed log file locate in /u01/home/oracle/product/11.2.0/db_1/cfgtoollogs and found the platform issue.
Running prerequisite checks…
Prerequisite check “CheckPatchApplicableOnCurrentPlatform” failed.
The details are:
Patch ( 9655006 ) is not applicable on current platform.
Platform ID needed is : 46
Platform IDs supported by patch are: 226
UtilSession failed: Prerequisite check “CheckPatchApplicableOnCurrentPlatform” failed.
OPatch failed with error code 73
Solution:
export OPATCH_PLATFORM_ID=226 and execute the psu patch command again.

Issue 5:

While applying Grid Infrastructure PSU (patch 9343627) to $GRID_HOME, opatch prerequisite check CheckSystemSpace failed:
Running prerequisite checks…
 Prerequisite check “CheckSystemSpace” failed.
 The details are:
 Required amount of space(4373569440) is not available.
 UtilSession failed: Prerequisite check “CheckSystemSpace” failed.

 OPatch failed with error code 73
Solution:
Follow the note ID 1088455.1 and make sure you have enough space on disk in GRID HOME mount point.
Reference NOTE IDs : 1082394.1 , 1308858.1 , 1169036.1 , 1290354.1, 1088455.1 and 1210964.1

No comments:

Post a Comment

Subscribe to: Posts (Atom)