Reducing Downtime While Patching Multi-OMS Environments

Enterprise Manager 12.1.0.4 has now been released for a few weeks, as well as the 12.1.0.4 OMS Bundle patches (also known as System patches). If you plan to apply these bundle patches to your 12.1.0.4 OMS, and you are concerned about the downtime, then, you can reduce the downtime by referring to this whitepaper that contains patching instructions to reduce downtime.

Check this article on Oracle’s blog on EM Reducing Downtime While Patching Multi-OMS Environments

Optimize Oracle WebLogic performance/manageability: Oracle EM12c

Screen Shot 2012-10-01 at 9.34.03 AM

For those of you who are still not aware; the days of EM being just a nice Database monitoring tool are way behind us. Curious to know what managing the complete stack means, might have persuade you to join this session by Nicole Haba (Oracle) and Ishaq Mohammed (Pacific Gas & Electric).

IMG_0098

Nicole starts with pointing out some of the challenges we have when trying to monitor and manage the Middleware Stack.

These include:

  • Monitoring & Diagnostics
    • Multiple Domain Discovery
    • Out-of-Box Monitoring
    • Metric Extensions
    • Administration Groups & Template Collections
    • Incident & Problem Management
    • Business Application Management
    • Middleware Diagnostics Advisor
    • JVM Diagnostics
    • Log Viewer
  • Administration
    • Credential Management
    • Record WLST scripts
    • System MBean Browser
    • Configure Log File Settings
    • Change Center
    • JDBC Data Source Management
    • Process Control
    • Selective Tracking
  • Configuration Management
    • Automated Asset Directory
    • Compare Configurations
    • Search Configurations
    • Compliance Management
    • Custom Configurations
    • Synchronize Configurations
    • Config Change History
  • Lifecycle Management
    • Cloning Domains
    • Patch Automation

Nicole introduces some Hints and Tips… :

Like: How can we discover multiple domains in EM? Simply create a csv file containing the specs of all domains to be discovered and use EMCLI to perform the discovery, using the csv as input parameter file.

IMG_0081

Using the Middleware Diagnostics Advisor to identify and resolve performance issues

IMG_0087

I must admit after conducting several EM12c Bootcamps, I am a MDA fan myself…

Enforcing standards by cloning WebLogic Domains. Exactly as we are used to working with “Golden Images” of Database Homes, the same goes for WebLogic domains.

Uploading binaries to the EM Software Library (a central location in EM to store images like OS, Database, Middleware and Applications) to allow provisioning of these to new servers and by that enforcing standardization.

IMG_0089

Next Ishaq Mohammed gives us an insight of the challenges at Pacific Gas & Electric and the way they implemented EM12c to monitor and manage the middleware stack.

IMG_0092

Several challenges motivated PG&E to start EM12c implementation, including:

  • 20% increase of WebLogic landscape over next 12 months with same number of administrators
  • Unawareness of performance problems until end users complain
  • Manual deployments of software being much too time-consuming, error prone, leading to inconsistent  non-standard configurations

Key Use cases included:

  • Administration Groups and Template Collections – to enforce standard monitoring settings
  • Middleware Diagnostics Advisor – To allow for fast root cause analysis
  • JVM Diagnostics – For application to disk drilldown
  • Provisioning Profiles and Cloning WebLogic Domains – to enforce standard domain deployment

IMG_0103

IMG_0106

Using Provisioning Profiles and Cloning of WebLogic Domains allowed for SSL-Enabled WebLogic Domains with OSB deployed as well as SSL-Enabled Weblogic Domain with WebLogic Portal deployed.

It also supports the differentiation of provisioning profiles in case of dev, test or production situations.

Customized Deployment Procedures were created to provision OSB domains and WebLogic Portal domains.

Initially the EM console has been used for launching deployment procedures, to be later replaced with EMCLI.

Ishaq concludes with some final steps finalise the project including: the creation of custom reports using BI Publisher, the patching of WebLogic Servers across multiple domains, the monitoring to Service Level Agreements and the implementation of Business Transaction Management (BTM).

I must say a very interesting customer story appreciated by many in the audience.

Installing Patch Set Update using EM12c

The integration between EM12c and My Oracle Support provides us with a constant inside on available and relevant patches. Based on this integration we can access all patch information in MOS instantly from Enterprise Manager and create a Patch Plan, Analyse included Patches and start the actual deployment to multiple Targets.

In this paper we will see that the installation of a Patch Set Update using EM12c is as easy as 1,2,3

Installing Patch Set Update using EM12c

Integration between OEM Console and My Oracle Support

First announced at Oracle Open World 2009, the integration between Oracle Enterprise Manager and the My Oracle Support website.

This new features allows you to track down Patch recommendations, create Patch Plans, validate Patch Plans for possible conflicts and many more.

Read the Paper I published on this subject

Integration of Enterprise Manager Grid Control and My Oracle Support

Organizations that are facing strict Security Compliance rules and are dealing with 24/7 demands, will find this feature extremely helpful to allow there data enter staff to prepare for necessary patching with a minimum of outage.

One of the major frustrations this feature is dealing with is the situation where administrators have been investing an enormous amount of time in debugging issues, some other customers already dealt with months ago and found a solution for.

Using the Recommended Patching feature helps you to avoid situations like this.

New Recommended Bundle Patch (APR 2010) – 9405592 for Patch Automation on EM 10.2.0.5

This one I received from Oracle today.

New Recommended Bundle Patch 9405592 is available for download from My Oracle Support now. This patch primarily enhances the Patching functionality offered by Oracle Enterprise Manager Grid Control.

This patch is  cumulative and is a superset of the previously released bundles # 9132461, #8992470, and #8653501 that were released previously, and
therefore, includes all the features that were introduced as part of those Recommended Bundle Patches.

For more information, refer to Comprehensive Overview of Recommended Bundle Patch 9405592 under
support note – OMS and Agent Patches required for setting up Provisioning, Patching and Cloning in 10.2.0.3 to 10.2.0.5 GC [ID 427577.1]


FAQ:

#1 If I had applied the previous recommended patches, do I need to rollback before applying this?
Yes, if you had applied any of the patches (# 9132461, #8992470 and #8653501) you would need to rollback the patch and apply this.
For rollback instructions, refer to the patch README from the support note 427577.1

#2 I recently applied the patch 9132461, do I still need the new patch?
The new patch contains additional bug fixes.  (
For more info see, Comprehensive Overview of Recommended Bundle Patch 9405592)
Augmented Verification and Support for Oracle Database 9i Release 2 (9.2.0.8) and Oracle  Databases on Microsoft Windows Platform
Bug fixes resolving issues with patching CPUs on Databases running on Windows platforms.
– Key Bug fixes identified at the various customers
Oracle strongly recommends you to apply the latest patch to make sure your do not encounter these issues and you are at the current patch levels for faster issue resolution through support

#3 Can I apply this patch on top of PSU3 for Enterprise Manager ?
Yes, this patch does NOT conflict with PSU3 and can be applied over it.

#4 Is there any known conflicts?
If you had applied the patch 8573971, it would conflict with this patch(9405592). You would need to rollback the patch 8573971 and apply this Bundle.
Apply the overlay patch –
9583322 to get the fixes of the rolled back patch 8573971. Note: The overlay patch is currently unavailable, it will be made available in few days.

Installing Agent 10.2.0.5.0 and upgrading to 10.2.0.5.2 using OEM Grid Control

After installing OEM GC 10.2.0.5.2 (see my previous post), I did an installation of a new Agent on a different Host, pushing it from the OMS.

After installing the 10.2.0.5.0 Agent, I upgraded the Agent to 10.2.0.5.2 by using the OEM Procedure “Patch Agent”

I have created a document describing all steps I went through

Installing Agent 10.2.0.5.0 and upgrading to 10.2.0.5.2 using OEM Grid Control