All You Need to Know about Oracle Database Patching

This is a topic I wrote about in the past (understanding Oracle patching and new version numbering). But now, after the new patching concept (since 12.2) and numbering change (since 18c) has been here for a while and has been stabilized, I think it’s time to summaries the new patching/versioning in a new post, so here it is.

Oracle Database Versions, Support and Upgrades

Just to sum things up, Oracle has decided to abandon the concept of a few releases for each version (e.g. 12.1.0.1, 12.1.0.2, etc.) and go for a yearly major release with the year number (i.e. 18c, 19c, etc.). However, “under the hood” a few releases will still be part of the same “release group”. So, 12cR2 was introduced with 12.2.0.1, but it continues (even though the naming was changed) with 18c (which is actually 12.2.0.2) and 19c (12.2.0.3).

This is super important for support issues. Since 12.2, 18c and 19c are still under “12cR2”, Oracle will have a short (and not extendable) support for 12.2 and 18c while 19c will be the long term support version. As you can see in MOS note# 742060.1, support for 12.2 will end in November 2020 and will not be extended. Support for 18c will not be extended as well and will end in June 2021. Version 19c will be the long term support and will be supported until March 2023 with extended paid support until March 2026.

According to multiple Oracle resources, it seems that the same concept will stay in future versions. So expect to see Oracle 20c next year and 21c in the following year as a short-term support versions while 22c will be the long term support version.

If you plan to upgrade your database now from an earlier version, consider to go straight to 19c, and plan for 22c in the future.

Patching Oracle Database – RU and RUR

With Oracle 12.2, Oracle changed the patching concept as well. No more PSU and DBBP, now we have RU and RUR. This change was introduced only for *NIX systems, so what I explain here is not relevant for Windows. Windows patching has not changed and is still called Bundle Patch (BP).

Database patch sets are released once a quarter (January, April, July and October). There are 2 patch types available now:

• RU (Release Update) – this is the major patch and it includes a large set of fixes. When installing RU you will get the 2nd digit of your database release changed. For example, RU of July 2019 for 18c is 18.7. The next one (October) is 18.8. The same patch (July) for 19c is 19.4 and the October one is 19.5.
• RUR (Release Update Revision) – this is a smaller patch set and it includes a smaller set of changes (only security and regression fixes since the last RU). Installing RUR will change the 3rd digit of the database release. Note the RUR will be supported for 6 months only for every RU. For example, if I have RU of April 2019 installed on my 18c database (that’s 18.6) and I decide to install RUR of July, I will have Oracle 18.6.1. In October I can decide to install the RU and get 18.8, or the RUR on top of my version and get 18.6.2. But in January 2020 there won’t be another RUR on top of 18.6. My options will be installing the new RU and get 18.9, install 18.7 and the new RUR and get 18.7.2, or install 18.8 and the new RUR and get 18.8.1.

Both RU and RUR are cumulative, so each patch includes all the fixes of the previous patches. If you wish to install Oracle 18.7, you can install it on any 18c version, you don’t have to install 18.6 first.

Patching Oracle Database – List of Patches Available

When you go to the patch availability note on MOS you’ll see quite a lot of options for each version. This is what you should know about each patch on that list (I took 18.7 as an example, and it’s not ordered in the same order as in the note):

• Database Release Update 18.7 – this is the main RU for the 18c database
• Database Release Update Revision 18.6.1 – this is the RUR that can be installed on top 18.6 RU
• Database Release Update Revision 18.5.2 – this is the RUR that can be installed on top of 18.5 RU
• Grid Infrastructure Release Update 18.7 – this is the main RU for GI, it also includes the DB RU, so if you use GI (either for Oracle Restart or for RAC), you can install this patch on both GI and DB homes
• Grid Infrastructure Release Update 18.6.1 – this is the RUR for GI on top of 18.6, it also includes the DB 18.6.1 RUR
• Grid Infrastructure Release Update 18.5.2 – this is the RUR for GI 18.5, it also includes the DB 18.5.2 RUR
• OJVM RU 18.7 – if you have OJVM installed in your database (whether you use it or not) you should install this patch in your database home. If you don’t have OJVM installed in the database, there is no need to install this patch (Oracle recommends installing it in case you will create a new database with OJVM in the future)
• Combo of OJVM + DB RU 18.7 – this is simply a single download for both 18.7 DB RU and OJVM 18.7. You can download them separately and get the exact same patches. This is for convenience purposes only
• Combo of OJVM + GI RU 18.7 – this is simply a single download for both 18.7 GI RU and OJVM 18.7. You can download them separately and get the exact same patches. This is for convenience purposes only
• Windows BP 18.7 – this is the 18.7 patch for database running on Windows platforms

Now choose the relevant patch to download and keep on reading

Additional Patches

Once you decided which patch you wish to install, in the same table in the MOS note there is a column named “known issues” with a link to a note. Go to this note and read the known issues for this specific RU/RUR. In most cases you’ll find some recommended patches for this version. You should follow the note, download all the recommended patches and install them when you install the new RU/RUR.

You might also hit specific bugs or issues in your environment. For these issues you’ll need specific one-off patches. If you already have one off patches installed, first check if they are included in the RU/RUR you are about to install. If not, make sure they don’t conflict with the new RU/RUR or ask for an updated version from Oracle.

Installation Type (for RAC/standby)

When installing patch on a complex environment, there are two types of patches:

• Standby first patches for Data Guard – these patches should be installed on the standby side first, then on the primary side. After both sides are patched, you should run any required scripts on the primary side. If a patch is NOT a standby first patch, it means that you need to take down the entire environment and patch both sides before you start the databases again.
• Rolling patches for RAC – if you’re using RAC, a rolling patch is a patch you can install on one node at a time, while the others are up and running. After patching one node is completed and the node is up again, you can continue to the next node, and so on. This way there is zero down time while patching. If a patch is NOT a rolling patch, this means that nodes with the patch installed and without it can’t coexist in the cluster. The best method here will be to install the patch on the RAC one node after the other, while you don’t start the instance on the nodes after the patch. When you get to the last node you’ll have a down time while patching it, and after that you can start all nodes

RU and RUR patches are always both standby first and RAC rolling patches. With OJVM patch it’s more complicated, you can read about it in MOS note# 2217053.1. One-off patches can be of any type, you should check their readme file.

Installing the Patches

This can be a post of its own, but I’ll just write a short overview of the installation process. For each patch (RU, RUR, one-off) you should review the readme and follow the installation instructions.

• For GI RU/RUR use “opatchauto apply”. You can apply the patch on the GI and DB homes together or separately (by using the -oh flag to opatchauto). This will also run the datapatch script for your database (if it was up before running patchauto)
• For a single instance DB use opatch apply, then run the datapatch script manually
• Again, check the readme and plan installation on RAC or standby environments

Best Practices for Complex Environments

Standby environment (assuming serverA is currently the primary and serverB is currently the standby):

• Standby first patches
  • Patch serverB
  • Switchover to serverB (with a short downtime)
  • Patch serverA
  • Run required scripts (if needed) on serverB (now the primary)
  • You can switchover again if you wish to go back to serverA with another short downtime, but that’s not mandatory, your environment is already patched
• Non standby first patches
  • Patch serverB but do not start the database after the patch
  • Patch serverA (downtime until the patch is installed)
  • Start both databases
  • Run any required scripts on serverA

RAC environment (assuming we have 3 nodes: serverA, serverB and serverC):

• RAC rolling patches (no complete downtime)
  • Patch serverA and start it after the patch
  • Patch serverB and start it after the patch
  • Patch serverC and start it after the patch
• Non RAC rolling patch
  • Patch serverA and don’t start the DB after the patch
  • Patch serverB and don’t start the DB after the patch
  • Shutdown the DB on serverC (a complete downtime)
  • Start serverA and serverB
  • Run any required scripts on serverA/B
  • Patch serverC and start the database

Final Note

One last comment regarding Active Data Guard – if the patch requires “startup upgrade” (like I realized when patching DST), Active Data Guard can’t roll forward the redo log generated (ORA-10485). Before starting up the primary in upgrade mode, the standby has to be mounted instead of open until all changes are applied.