DataGuard - Switchover

5.3 Switchover

You can switch a database from the primary role to the standby role, as well as from standby to primary. This is known as a database switchover, because the standby database that you specify becomes the primary database, and the original primary database becomes a standby database, with no loss of data.

Whenever possible, you should switch over to a physical standby database:

• If the switchover transitions a physical standby database to the primary role, then:
  • The original primary database will be switched to a physical standby role.
  • The online redo log files will be continuously archived from the new primary database to all standby databases in the configuration.
  • The original primary database will be restarted as a part of the switchover operation. Note that the new primary database does not need to be restarted.
    Standby databases not involved in the switchover (known as bystander standby databases) continue operating in the state they were in before the switchover occurred and will automatically begin applying redo data received from the new primary database.
• If the switchover transitions a logical standby database to the primary role, then:
  • The original primary database will be switched to a logical standby role.
  • Neither the primary database nor the logical standby database needs to be restarted after the switchover completes.
    Other logical standby bystander databases in the broker configuration will remain viable after the switchover. There is no need to restart any databases. All physical and snapshot standby databases will be disabled and must be re-created from a copy of the new primary database after a switchover to a logical standby database.
  Switchover to a logical standby database is disallowed when the configuration is operating in maximum protection mode.
  WARNING:

  Switching over to a logical standby database results in the snapshot and physical standby databases in the broker configuration being disabled by the broker, making these databases no longer viable as standby databases. Section 5.4.3 describes how to restore their viability as standby databases.

  If you intend to switch back to the original primary database relatively soon, you may allow the physical and snapshot standbys to remain disabled. Once you have completed the switchover back to the original primary, you may then reenable the physical and snapshot standby databases since they are still viable standbys for the original primary database.

5.3.1 Before You Perform a Switchover Operation

Consider the following points before you begin a switchover:

• When you start a switchover, the broker verifies that at least one standby database, including the primary database that is about to be transitioned to the standby role, is configured to support the overall protection mode (maximum protection, maximum availability, or maximum performance) after the switchover is completed.
• Prepare the primary database in advance for its possible future role as a standby database in the context of the overall protection mode (see Section 4.6). Such preparation includes:
  • Ensuring that standby redo log files are configured on the primary database.
  • Presetting database properties related to redo transport services, such as LogXptMode, NetTimeout, StandbyArchiveLocation, andAlternateLocation. For more details about managing redo transport services using database properties, see Section 4.4.
  • Presetting database properties related to Redo Apply services, such as DelayMins. For more details about managing Redo Apply services using properties, see Section 4.5.
  • For each temporary table, verifying that temporary files associated with that table on the primary database also exist on the standby database.
  Note that the broker does not use the properties to set up redo transport services and Redo Apply services until you actually switch over the primary database to the standby role. Thus, the validity of the values of these properties is not verified until after the switchover. Once you set these properties, their values persist through role changes during switchover and failover.
• Before performing a switchover to a physical standby database that is in real-time query mode, consider bringing all instances of that standby database to the mounted but not open state to achieve the fastest possible role transition and to cleanly terminate any user sessions connected to the physical standby database prior to the role transition.
• If fast-start failover is enabled in either maximum availability or maximum performance mode, a switchover can be performed only to the pre-specified target standby database and only if the standby database is synchronized with the primary database. For information about enabling fast-start failover, see Section 5.5.2.

After a switchover completes, the broker preserves the overall Data Guard protection mode as part of the switchover process by keeping the protection mode at the same protection level (maximum protection, maximum availability, or maximum performance) it was at before the switchover. Also, the network transmission mode (SYNCor ASYNC) for transporting redo to other bystander standby databases does not change after a switchover. Apply services on all other bystander standby databases automatically begin applying redo data received from the new primary database.

If there are physical or snapshot standby databases in the configuration and the switchover occurs to a logical standby database, you need to re-create those databases from a copy of the new primary database and then reenable those databases, as described in Section 5.4.3.

5.3.2 Starting a Switchover

The act of switching roles should be a well-planned activity. The primary and standby databases involved in the switchover should have as small a redo lag as possible.Oracle Data Guard Concepts and Administration provides information about setting up the databases in preparation of a switchover.

To start a switchover using Enterprise Manager, select the standby database that you want to change to the primary role and click Switchover. When using DGMGRL, you need to issue only one SWITCHOVER command to specify the name of the standby database that you want to change into the primary role.

The broker controls the rest of the switchover, as described in Section 5.3.3.

5.3.3 How the Broker Performs a Switchover

Once you start the switchover, the broker:

1. Verifies that the primary and the target standby databases are in the following states:
   1. The primary database is enabled and is in the TRANSPORT-ON state.
   2. The target standby database is enabled and is in the APPLY-ON state.
   The broker allows the switchover to proceed as long as there are no errors for the primary database and the standby database that you selected to participate in the switchover operation. Errors occurring for any other bystander standby databases will not impede the switchover.
2. Shuts down all instances except one, if required.
   If you are switching over to a physical standby database, the broker shuts down all but one instance on the current primary database. No instances will be shut down on the target physical standby database.
   No instances will be shut down if switching over to a logical standby database. You cannot switch over to a snapshot standby database.
3. Switches roles between the primary and standby databases.
   The broker first converts the original primary database to run in the standby role. Then, the broker transitions the target standby database to the primary role. If any errors occur during either conversion, the broker stops the switchover. See Section 9.3, "Troubleshooting Problems During a Switchover Operation" for more information.
4. Updates the broker configuration file to record the change in roles.
   Because the configuration file profiles all database objects in the configuration, this ensures that each database will run in the correct role and state should it be restarted later for any reason.
5. Restarts the new standby (former primary) database if the switchover occurs to a physical standby database, and Redo Apply begins applying redo data from the new primary database. If this is an Oracle RAC physical standby database, the broker directs Oracle Clusterware to restart the instances that were shut down prior to the switchover. In a configuration operating in maximum protection mode, the new primary database will also be restarted.
6. The new primary database is opened in read/write mode and redo transport services are started.
   If the former physical standby database was running with real-time query enabled, the new physical standby database will run with real-time query enabled.

The broker verifies the state and status of the databases to ensure that the switchover transitioned the databases to their new role correctly. Bystander standby databases that are not disabled by the broker after the switchover will continue operating in the state they were in before the switchover. Redo Apply and SQL Apply on all other bystander standby databases automatically begin applying redo data received from the new primary database.

5.4 Manual Failover

You can convert a standby database to a primary database when the original primary database fails and there is no possibility of recovering the primary database in a timely manner. This is known as a manual failover. There may or may not be data loss depending upon whether your primary and target standby databases were synchronized at the time of the primary database failure. The word manual is used to contrast this type of failover with a fast-start failover (described in Section 5.5).

Note:

You can perform a manual failover even if fast-start failover is enabled. See Section 5.5.2.4 for more information.

The following sections describe how to perform manual failovers:

• Section 5.4.1, "Complete and Immediate Manual Failovers"
• Section 5.4.2, "Performing a Manual Failover Operation"
• Section 5.4.3, "Reenabling Disabled Databases After a Role Change"

5.4.1 Complete and Immediate Manual Failovers

Using Enterprise Manager or DGMGRL, you can perform either a complete (recommended) or an immediate failover:

• A complete failover is the recommended and default failover option. It automatically recovers the maximum amount of redo data for the protection mode the configuration is operating in. A complete failover also attempts to avoid disabling any standby databases that were not the target of the failover, so that they may continue serving as standby databases to the new primary database.
  Whether or not standby databases that were not the target of failover (bystander standby databases) are disabled depends upon how much redo data they have applied relative to the failover target and the standby type of the failover target:
  • If the failover target is a physical or snapshot standby database, the original primary database must be reinstated or re-created in order to be a standby database for the new primary database. In addition, some standby databases may be disabled by the broker during the failover if the broker detects that they have applied redo beyond where the new primary database had applied. Any standby database that was disabled by the broker must be reinstated or re-created, as described in Section 5.4.3, before it can be a standby database for the new primary database.
    Note that if failover was performed on a snapshot standby database, the old primary must be either reinstated or re-created as a physical standby database.
  • If the failover target is a logical standby database, the original primary database and all physical and snapshot standby databases in the configuration will be disabled. The primary database can be reinstated if it had flashback database enabled. The physical and snapshot standby databases will have to be re-created from a copy of the new primary database. See Section 5.4.3 for more information.
  If the primary database can be mounted, it may be possible to flush any unsent redo data from the primary database to the target standby database using theALTER SYSTEM FLUSH REDO SQL statement. If this operation is successful, a zero data loss failover may be possible even if the primary database is not in a zero data loss protection mode. See Oracle Data Guard Concepts and Administration for more information on using the ALTER SYSTEM FLUSH REDO statement.
  During a complete failover, the broker performs the failover steps described in Section 5.4.2.1.
• An immediate failover is the fastest type of failover. However, no additional data is applied on the standby database once you invoke the failover. Another consequence of immediate failover is that all other databases in the configuration are disabled and must be reinstated or re-created before they can serve as standby databases for the new primary database. Section 5.4.3 describes how to do this. During an immediate failover, the broker performs the failover steps described in Section 5.4.2.2.
  Caution:

  Always try to perform a complete failover first unless redo apply has stopped at the failover target due to an ORA-752 or ORA-600 [3020] error. If one of these errors has occurred, follow the guidelines in "Resolving ORA-752 or ORA-600 [3020] During Standby Recovery" in My Oracle Support Note 1265884.1 before proceeding. This support note is available at http://support.oracle.com.
  

  An immediate failover should only be performed when a complete failover is unsuccessful or in the case just noted above. A complete failover can occur without any data loss, depending on the destination attributes of redo transport services, but an immediate failover usually results in some data loss.

5.4.2 Performing a Manual Failover Operation

After determining that there is no possibility of recovering the primary database in a timely manner, ensure that the primary database is shut down and then begin the failover operation.

The steps in this section describe how to perform a manual failover. Depending on the failover and the types of standby databases involved, some of the databases may need to be reinstated or re-created. The instructions guide you through the appropriate steps for each type of situation.

Step 1   Determine which of the available standby databases is the best target for the failover.

Follow the guidelines described in Section 5.2, "Choosing a Target Standby Database".

Step 2   Start the failover.

Using Enterprise Manager or DGMGRL, perform either a complete (recommended) or an immediate failover.

Manual Failover Using Enterprise Manager:

On the Data Guard Overview Page in Enterprise Manager, select the standby database that you want to change to the primary role and click Failover. Then, on the Failover Confirmation Page, click Yes to invoke the default Complete failover option.

Manual Failover Using DGMGRL:

On the target standby database, issue the FAILOVER command to perform a failover, specifying the name of the standby database that you want to become the primary database:

DGMGRL> FAILOVER TO database-name;

Specify the optional IMMEDIATE clause to perform an immediate failover if any of the the following conditions are true:

• An ORA-752 error has occurred at the standby database
• An ORA-600 [3020] error has occurred at the standby database and Oracle support has determined that it was caused by a lost write at the primary database
• A complete failover is not possible

DGMGRL> FAILOVER TO database-name IMMEDIATE;

See Also:

• FAILOVER

If the target is a snapshot standby database, the broker first converts the database to a physical standby database. If performing a complete failover, all accumulated redo data will be applied before changing the role of the database to primary. If performing an immediate failover, the role of the database will be changed to primary without applying any accumulated redo data.

No instances will be shut down if failing over to a physical or logical standby database.

Step 3   Reset the protection mode.

After a manual failover (complete or immediate), the overall Data Guard protection mode is handled as follows:

• If the protection mode was at maximum protection, it is reset to maximum performance. You can upgrade the protection mode later, if necessary, as described in Section 4.6.1.
• If the protection mode was at maximum availability or maximum performance, it remains unchanged.

Note:

If you perform a manual failover when fast-start failover is enabled:

• The failover can only be performed to the pre-selected target standby database.
• The broker preserves the protection mode that was in effect prior to the failover.

Step 4   Re-establish a disaster-recovery configuration.

To maintain a viable disaster-recovery solution in the event of another disaster, you may need to perform the additional steps described in Section 5.4.3 to:

• Reinstate the original primary database to act as a standby database in the new configuration.
  Caution:

  Do not attempt to reinstate the old primary database if an ORA-752 or ORA-600 [3020] error has occurred at the failover target, because doing so may lead to data loss. Instead, the old primary database must be re-created as a standby from a backup of the new primary using the procedure described in Section 5.4.3.2.
• Reinstate or re-create standby databases in the configuration that were disabled by the broker.
  After a complete failover finishes, any bystander standby database that is not viable as a standby for the new primary database will be disabled by the broker. This can happen for either of the following reasons:
  • A bystander standby database has applied more redo data than the new primary database itself had applied when it was a standby database. The standby database must be re-created or reinstated before it can serve as a standby for the new primary database.
  • The failover was to a logical standby database. The broker disables all of the physical and snapshot standby databases in the configuration. They must be re-created before they can serve as standby to the new primary database.

5.4.2.1 How the Broker Performs a Complete Failover Operation

Once you start a complete failover, the broker:

1. Verifies that the target standby database is enabled. If the database is not enabled, you will not be able to perform a failover to this database.
2. Waits for the target standby database to finish applying any unapplied redo data before stopping Redo Apply (if the target is a physical standby database) or SQL Apply (if the target is a logical standby database).
   If the target is a snapshot standby database, the broker first converts the database back to a physical standby and then starts Redo Apply to apply all the accumulated redo before completing the failover and opening the database as a primary database.
3. Transitions the target standby database into the primary database role, as follows:
   1. Changes the role of the database from standby to primary.
   2. Opens the new primary database in read/write mode.
   3. Determines whether or not any standby databases that did not participate in the failover operation have applied redo data beyond the new primary database, and thus need to be disabled.
      If a bystander standby database is not disabled by the broker during this failover, it will remain in the state it was in before the failover. For example, if a physical standby database was in the APPLY-OFF state, it will remain in the APPLY-OFF state.
      By default, the broker always determines whether bystander standby databases will be viable standby databases for the new primary when performing a complete failover. If you want the broker to skip this viability check of bystander standby databases during a complete failover, thus decreasing the overall failover time, set the BystandersFollowRoleChange configuration property to NONE.
      When this property is set to NONE, the broker will disable all bystander standby databases without checking whether they have applied more redo data than the new primary database. You will have to reinstate or re-create (see Section 5.4.3) the standby databases after failover has completed. The SHOW CONFIGURATION command will show you which databases can be reinstated and which databases must be re-created. Use the SHOW CONFIGURATION BystandersFollowRoleChange command to see the value of this property. The default value is ALL.
      This property also affects whether the broker skips viability checks of bystander standby databases when a fast-start failover occurs.
   4. Starts redo transport services to begin transmitting redo data to all bystander standby databases that were not disabled.
      Note:

      Bystander standby databases may be disabled by the broker during the failover, and they must be reinstated or re-created before they can serve as standby databases to the new primary database. Oracle recommends configuring Flashback Database on every database so that if failover occurs to a physical standby database, you can more easily reinstate any disabled standby databases. If failover occurs to a logical standby database, all physical and snapshot standby databases will be disabled by the broker. In this case, Flashback Database cannot be used to reinstate databases. They must be re-created from a copy of the new primary database. Logical standby databases that are disabled during failover can be reinstated.
4. If the failover target database is an Oracle RAC physical or snapshot standby database, the broker directs Oracle Clusterware to restart all instances that may have been shut down prior to the failover.

The broker allows the failover to proceed as long as there are no errors for the standby database that you selected to participate in the failover. Errors occurring for any bystander standby databases will not stop the failover. If you initiated a complete failover and it fails, you might need to use immediate failover.

5.4.2.2 How the Broker Performs an Immediate Failover Operation

Once you start an immediate failover, the broker:

1. Verifies that the target standby database is enabled. If the standby database is not enabled for management by the broker, then the failover cannot occur.
2. Stops Redo Apply or SQL Apply on the standby database immediately, without waiting until all available redo data has been applied. This may result in data loss.
3. Transitions the target standby database into the primary role, opens the new primary database in read/write mode, and starts redo transport services.
   After an immediate failover completes, all the standby databases in the configuration, regardless of their type, are disabled. They may be reinstated if Flashback Database is enabled on those databases. Otherwise, they must be re-created from a copy of the new primary database.

The broker allows a complete failover to proceed as long as there are no errors present on the standby database that you selected to participate in the failover.

The broker allows an immediate failover to proceed even if there are errors present on the standby database that you selected to participate in the failover.

5.4.3 Reenabling Disabled Databases After a Role Change

To restore your original disaster-recovery solution after switchover to a logical standby database or after failover to any standby database, you may need to perform additional steps.

Databases that have been disabled after a role transition are not removed from the broker configuration, but they are no longer managed by the broker.

To reenable broker management of these databases, you must reinstate or re-create the databases using one of the following procedures:

• If a database can be reinstated, the database will show the following status:

  ORA-16661: the standby database needs to be reinstated
  

  Reinstate the database using the DGMGRL REINSTATE DATABASE command or the reinstate option in Enterprise Manager, as described in Section 5.4.3.1, "How to Reinstate a Database". The broker automatically reenables the database as part of reinstating it.
• If a database must be re-created from a copy of the new primary database, it will have the following status:

  ORA-16795: the standby database needs to be re-created
  

  Re-create the standby database from a copy of the primary database and then reenable it. The procedures for creating a standby database are documented inOracle Data Guard Concepts and Administration. See Section 5.4.3.2, "How to Re-create and Reenable a Disabled Database" for more information.

Note:

Any database that was disabled while multiple role changes were performed cannot be reinstated. You must re-create the database manually from a copy of the current primary database and then reenable the database in the broker configuration.

Whether you reinstate or re-create a database depends on if you performed a switchover or failover and on the type of standby database that was the target of the operation. Note that role changes to logical and snapshot standby databases will always result in physical standby database bystanders being disabled. They cannot be reinstated. They must be re-created from a copy of the new primary database.

The following sections describe how to reinstate or reenable a database.

5.4.3.1 How to Reinstate a Database

You can use the broker's reinstate capability to make the failed primary database a viable standby database for the new primary. This can be done regardless of whether the failover was done to a physical, logical, or snapshot standby database.

You can also reinstate bystander standby databases that were disabled during a failover operation.

Databases that can be reinstated will have the following status value:

ORA-16661: the standby database needs to be reinstated

For the REINSTATE command to succeed, Flashback Database must have been enabled on the database prior to the failover and there must be sufficient flashback logs on that database. In addition, the database to be reinstated and the new primary database must have network connectivity.

To reinstate a database:

1. Restart the database to the mounted state
2. Connect to the new primary database
3. Use Enterprise Manager or DGMGRL to reinstate the database

The broker reinstates a failed primary database as a standby database of the same type (physical or logical standby database) as the old standby database. The only exception to this is failovers to snapshot standby databases. In such cases, the failed primary database is reinstated as a physical standby database.

The broker reinstates bystander standby databases that were disabled during a failover as standby databases to the new primary database.

Reinstatement Using Enterprise Manager

On the Data Guard Overview page, click the Database must be reinstated link. This brings up the General Properties page that provides a Reinstate button. After you click the Reinstate button, Enterprise Manager begins reinstating the database.

When the process is complete, the database will be enabled as a standby database to the new primary database, and Enterprise Manager displays the Data Guard Overview page.

Reinstatement Using DGMGRL

Issue the following command while connected to any database in the broker configuration, except the database that is to be reinstated:

DGMGRL> REINSTATE DATABASE db_unique_name;

The newly reinstated standby database will begin serving as a standby database to the new primary database. If reinstatement of a database fails, its status changes toORA-16795: the standby database needs to be re-created. You must then re-create it from a copy of the new primary database and reenable it as described inSection 5.4.3.2.

5.4.3.2 How to Re-create and Reenable a Disabled Database

If you performed a failover or switchover that requires you to re-create the failed primary database or standby databases that were disabled during the role transition, follow the procedures in the Oracle Data Guard Concepts and Administration chapters, "Creating a Physical Standby Database" and "Creating a Logical Standby Database."

Note that if you are re-creating the old primary database, it must be created as the standby type of the old standby database. For example, if the old standby was a physical or snapshot standby, then the old primary must be re-created as a physical standby.

After the database has been re-created, enable broker management of the re-created standby database by using the DGMGRL ENABLE DATABASE command.