GotoDBA Features,Infrastructure Patching Oracle Standard Edition High Availability (SEHA)

Patching Oracle Standard Edition High Availability (SEHA)

SEHA is a new Oracle Database 19c (>=19.7) feature to allow Standard Edition to integrate with Grid Infrastructure (GI) and allow a simple fail-over cluster option (after Oracle has removed the RAC option from Standard Edition). You can read more about what it is in my introduction to SEHA blog post, and more about how it actually works under different circumstances in my testing SEHA post. In this post it’s time to upgrade SEHA.

Introduction

As I explained in my first post about SEHA, for SEHA to work you’ll need Oracle 19.7 or later and I installed 19.8. So in my environment the GI and DB are 19.8 and last week 19.9 was released. What interested me is how the patch works: Will opatchauto patch the Standard Edition home as well? Will it add the OCW patch? How about running the sqlpatch scripts? And in general, what is the best way to patch this?

The Patch Process

In my case, I have 2 servers: SEHA1 and SEHA2, and 2 databases: se1 (SEHA-enabled, currently on SEHA1) and se2 (SEHA-disabled, runnning on SEHA2). This is how I’m going to patch the entire environment:

  1. Patch SEHA2 (se2 will be down during the patch, se1 will be on SEHA1)
  2. Move se1 to SEHA2 and patch it
  3. Patch SEHA1

Patch Node SEHA2

Initial status:

[oracle@seha2 ~]$ srvctl status database -db se1
Instance se1 is running on node seha1
[oracle@seha2 ~]$ srvctl status database -db se2
Instance se2 is running on node seha2

Installing the patch:

[root@seha2 ~]# export ORACLE_HOME=/oracle/grid/19
[root@seha2 ~]# $ORACLE_HOME/OPatch/opatchauto apply /oracle/install/31750108

OPatchauto session is initiated at Wed Oct 28 14:39:46 2020

System initialization log file is /oracle/grid/19/cfgtoollogs/opatchautodb/systemconfig2020-10-28_02-39-52PM.log.

Session log file is /oracle/grid/19/cfgtoollogs/opatchauto/opatchauto2020-10-28_02-40-41PM.log
The id for this session is U8ZB

Executing OPatch prereq operations to verify patch applicability on home /oracle/grid/19

Executing OPatch prereq operations to verify patch applicability on home /oracle/db/19
Patch applicability verified successfully on home /oracle/db/19

Patch applicability verified successfully on home /oracle/grid/19


Verifying SQL patch applicability on home /oracle/db/19
SQL patch applicability verified successfully on home /oracle/db/19


Preparing to bring down database service on home /oracle/db/19
Successfully prepared home /oracle/db/19 to bring down database service


Bringing down database service on home /oracle/db/19
Following database has been stopped and will be restarted later during the session: se2
Database service successfully brought down on home /oracle/db/19


Bringing down CRS service on home /oracle/grid/19
CRS service brought down successfully on home /oracle/grid/19


Start applying binary patch on home /oracle/db/19
Binary patch applied successfully on home /oracle/db/19


Start applying binary patch on home /oracle/grid/19
Binary patch applied successfully on home /oracle/grid/19


Starting CRS service on home /oracle/grid/19
CRS service started successfully on home /oracle/grid/19


Starting database service on home /oracle/db/19
Database service successfully started on home /oracle/db/19


Preparing home /oracle/db/19 after database service restarted
No step execution required.........


Trying to apply SQL patch on home /oracle/db/19
SQL patch applied successfully on home /oracle/db/19

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:seha2
SIDB Home:/oracle/db/19
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /oracle/install/31750108/31773437
Reason: This patch is not applicable to this specified target type - "oracle_database"

Patch: /oracle/install/31750108/31780966
Reason: This patch is not applicable to this specified target type - "oracle_database"


==Following patches were SUCCESSFULLY applied:

Patch: /oracle/install/31750108/31771877
Log: /oracle/db/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_14-45-37PM_1.log

Patch: /oracle/install/31750108/31772784
Log: /oracle/db/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_14-45-37PM_1.log


Host:seha2
CRS Home:/oracle/grid/19
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /oracle/install/31750108/31771877
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_15-01-38PM_1.log

Patch: /oracle/install/31750108/31772784
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_15-01-38PM_1.log

Patch: /oracle/install/31750108/31773437
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_15-01-38PM_1.log

Patch: /oracle/install/31750108/31780966
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_15-01-38PM_1.log



OPatchauto session completed at Wed Oct 28 15:41:00 2020
Time taken to complete the session 61 minutes, 15 seconds

Status After SEHA2 Patch

After we finished patching the node, let’s see what the status of the databases is:

[oracle@seha1 ~]$ srvctl status database -db se1
Instance se1 is running on node seha1
[oracle@seha1 ~]$ srvctl status database -db se2
Instance se2 is running on node seha2

# on SEHA1
[oracle@seha1 ~]$ $ORACLE_HOME/OPatch/opatch lspatches
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)

[oracle@seha1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Oct 28 15:44:09 2020
Version 19.8.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
se1

SQL>  select comp_id,version_full from dba_registry where comp_id='CATALOG';

COMP_ID                        VERSION_FULL
------------------------------ ------------------------------
CATALOG                        19.8.0.0.0

# on SEHA2
[oracle@seha2 ~]$ $ORACLE_HOME/OPatch/opatch lspatches
31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)
31771877;Database Release Update : 19.9.0.0.201020 (31771877)

OPatch succeeded.
[oracle@seha2 ~]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Oct 28 15:47:19 2020
Version 19.9.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production
Version 19.9.0.0.0

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
se2

SQL> select comp_id,version_full from dba_registry where comp_id='CATALOG';

COMP_ID                        VERSION_FULL
------------------------------ ------------------------------
CATALOG                        19.9.0.0.0

We can see here a few things:

  1. The databases are still located on the same nodes
  2. se1 database is 19.8 and wasn’t affected by SEHA2 patch
  3. se2 was upgraded to 19.9
  4. SEHA2 now has 19.9 RU and 19.9 OCW. This is great because now we can use srvctl from the DB home (and if you have no idea what I’m talking about, check my introduction to SEHA post)

Relocate and Patch DB se1

After we patched the SEHA2 node without downtime to the SEHA database se1, it’s time to patch se1. I’ll do this by moving it to the patched node (minimum downtime) and use datapach. Note that datapatch can be executed online, so basically we have a very short downtime during the relocate itself and that’s it.

I will do everything from SEHA2 node:

[oracle@seha2 ~]$ srvctl relocate database -db se1 -node seha2
[oracle@seha2 ~]$ srvctl status database -db se1
Instance se1 is running on node seha2

[oracle@seha2 OPatch]$ ./datapatch -verbose -db se1
SQL Patching tool version 19.9.0.0.0 Production on Wed Oct 28 15:58:58 2020
Copyright (c) 2012, 2020, Oracle.  All rights reserved.

Log file for this invocation: /oracle/db/cfgtoollogs/sqlpatch/sqlpatch_23299_2020_10_28_15_58_58/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
  No interim patches found

Current state of release update SQL patches:
  Binary registry:
    19.9.0.0.0 Release_Update 200930183249: Installed
  PDB CDB$ROOT:
    Applied 19.8.0.0.0 Release_Update 200703031501 successfully on 27-OCT-20 11.14.43.728061 PM
  PDB PDB:
    Applied 19.8.0.0.0 Release_Update 200703031501 successfully on 27-OCT-20 11.14.47.991871 PM
  PDB PDB$SEED:
    Applied 19.8.0.0.0 Release_Update 200703031501 successfully on 27-OCT-20 11.14.47.991871 PM

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  For the following PDBs: CDB$ROOT PDB$SEED PDB
    No interim patches need to be rolled back
    Patch 31771877 (Database Release Update : 19.9.0.0.201020 (31771877)):
      Apply from 19.8.0.0.0 Release_Update 200703031501 to 19.9.0.0.0 Release_Update 200930183249
    No interim patches need to be applied

Installing patches...
Patch installation complete.  Total patches installed: 3

Validating logfiles...done
Patch 31771877 apply (pdb CDB$ROOT): SUCCESS
  logfile: /oracle/db/cfgtoollogs/sqlpatch/31771877/23869227/31771877_apply_SE1_CDBROOT_2020Oct28_16_01_06.log (no errors)
Patch 31771877 apply (pdb PDB$SEED): SUCCESS
  logfile: /oracle/db/cfgtoollogs/sqlpatch/31771877/23869227/31771877_apply_SE1_PDBSEED_2020Oct28_16_02_53.log (no errors)
Patch 31771877 apply (pdb PDB): SUCCESS
  logfile: /oracle/db/cfgtoollogs/sqlpatch/31771877/23869227/31771877_apply_SE1_PDB_2020Oct28_16_02_53.log (no errors)
SQL Patching tool complete on Wed Oct 28 16:05:32 2020
[oracle@seha2 OPatch]$ export ORACLE_SID=se1
[oracle@seha2 OPatch]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Oct 28 16:05:42 2020
Version 19.9.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production
Version 19.9.0.0.0

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
se1

SQL> select comp_id,version_full from dba_registry where comp_id='CATALOG';

COMP_ID                        VERSION_FULL
------------------------------ ------------------------------
CATALOG                        19.9.0.0.0

Patch Node SEHA1

Now both se1 and se2 databases are on SEHA2 node and both are 19.9. All we’ve got to do left is to patch SEHA1 node:

[root@seha1 ~]# export ORACLE_HOME=/oracle/grid/19
[root@seha1 ~]# $ORACLE_HOME/OPatch/opatchauto apply /oracle/install/31750108

OPatchauto session is initiated at Wed Oct 28 16:07:22 2020

System initialization log file is /oracle/grid/19/cfgtoollogs/opatchautodb/systemconfig2020-10-28_04-07-29PM.log.

Session log file is /oracle/grid/19/cfgtoollogs/opatchauto/opatchauto2020-10-28_04-08-13PM.log
The id for this session is MR9L

Executing OPatch prereq operations to verify patch applicability on home /oracle/grid/19
Patch applicability verified successfully on home /oracle/grid/19


Bringing down CRS service on home /oracle/grid/19
CRS service brought down successfully on home /oracle/grid/19


Start applying binary patch on home /oracle/grid/19
Binary patch applied successfully on home /oracle/grid/19


Starting CRS service on home /oracle/grid/19
CRS service started successfully on home /oracle/grid/19

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:seha1
CRS Home:/oracle/grid/19
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /oracle/install/31750108/31771877
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_16-12-28PM_1.log

Patch: /oracle/install/31750108/31772784
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_16-12-28PM_1.log

Patch: /oracle/install/31750108/31773437
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_16-12-28PM_1.log

Patch: /oracle/install/31750108/31780966
Log: /oracle/grid/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_16-12-28PM_1.log



OPatchauto session completed at Wed Oct 28 16:51:13 2020
Time taken to complete the session 43 minutes, 51 seconds

Status After SEHA1 Patch

Let’s check the DB home:

[oracle@seha1 ~]$ $ORACLE_HOME/OPatch/opatch lspatches
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)

OPatch succeeded.

Wait, what? How come it’s not patched?

I don’t know if you’ve noticed, but if you look now at the patch process, you’ll see that only the GI was patched and not the database. But we didn’t have this problem on SEHA2, so why here?

When patching Oracle software, opatchauto will patch the GI and ACTIVE DB homes. This means that only homes that have running databases will be patched. On SEHA1 we don’t have any running databases, so opatchauto will not patch any DB home. If you remember, on SEHA2, because of my specific configuration, we did have the se2 database up, so its home was patched.

If this is the case, we are not done, we still have to patch the DB home on SEHA1:

[root@seha1 ~]# export ORACLE_HOME=/oracle/db/19
[root@seha1 ~]# $ORACLE_HOME/OPatch/opatchauto apply /oracle/install/31750108 -oh /oracle/db/19

OPatchauto session is initiated at Wed Oct 28 17:08:20 2020

System initialization log file is /oracle/db/19/cfgtoollogs/opatchautodb/systemconfig2020-10-28_05-08-30PM.log.

Session log file is /oracle/db/19/cfgtoollogs/opatchauto/opatchauto2020-10-28_05-09-58PM.log
The id for this session is ICNW

Executing OPatch prereq operations to verify patch applicability on home /oracle/db/19
Patch applicability verified successfully on home /oracle/db/19


Verifying SQL patch applicability on home /oracle/db/19
No step execution required.........


Preparing to bring down database service on home /oracle/db/19
No step execution required.........


Performing prepatch operation on home /oracle/db/19
Perpatch operation completed successfully on home /oracle/db/19


Start applying binary patch on home /oracle/db/19
Binary patch applied successfully on home /oracle/db/19


Performing postpatch operation on home /oracle/db/19
Postpatch operation completed successfully on home /oracle/db/19


Preparing home /oracle/db/19 after database service restarted
No step execution required.........


Trying to apply SQL patch on home /oracle/db/19
No step execution required.........

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:seha1
RAC Home:/oracle/db/19
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /oracle/install/31750108/31773437
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /oracle/install/31750108/31780966
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /oracle/install/31750108/31771877
Log: /oracle/db/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_17-12-19PM_1.log

Patch: /oracle/install/31750108/31772784
Log: /oracle/db/19/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-28_17-12-19PM_1.log



OPatchauto session completed at Wed Oct 28 17:34:59 2020
Time taken to complete the session 26 minutes, 40 seconds
[root@seha1 ~]# su - oracle
Last login: Wed Oct 28 17:34:57 PDT 2020
[oracle@seha1 ~]$ $ORACLE_HOME/OPatch/opatch lspatches
31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)
31771877;Database Release Update : 19.9.0.0.201020 (31771877)

OPatch succeeded.

Now the DB home on SEHA1 node is patched (with OCW patch as well). We can leave se1 database on SEHA2 or relocate it back to SEHA1, in any case both servers and both databases are up to date and the patch process is completed.

What Else Can Be Done

One thing I could have done differently is using the Out Of Place (OOP) GI patching. Using this feature we can patch the GI while the database is still up and running, resulting in a shorter downtime (the downtime will be necessary for the DB home patch only). However, in our case, to reduce the downtime we relocated the database, which was done after GI and DB patch, so we actually got an even shorter downtime for se1 (se2 could have benefited from a shorter downtime using the OOP GI patch).

Note: this feature is documented for GI patching using RAC databases. I’m not sure it’s supported for SEHA databases (and officially it’s not supported for Oracle Restart).

For more information about OOP GI patching check MOS note 2662762.1.

Conclusion

Patching SEHA environment is not very different from RAC environment, but there are a few things we need to remember:

  1. SEHA database will only run on one node, so make sure where you would like it to run before patching a specific node
  2. opatchauto will not patch a home without a running database, so if you don’t have active databases on this home, you’ll have to patch this home manually
  3. If you relocated an unpatched database to a patched home, don’t forget to run datapatch manually to patch the database

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Post