Sunday, October 4, 2015

Adding Node to a Oracle RAC Cluster and Extending DB instance to the Node in 11.1.0.7 cluster

The article deals with Adding node and extending DB to the new node in a 11.1.0.7 cluster. The activity can be broadly divided into
1. Adding Node to the existing cluster
2. Adding instance to the existing DB

Adding Node to Existing Cluster

Adding node can be accomplished by two methods namely Interactive and Silent. In both the cases OUI is used in add node mode to add a new node. While adding the node you have to ensure that all the RAC pre-reqs are in place.

Interactive Method

1. From one of the node of the existing cluster we will launch the OUI and for that ensure ORACLE_HOME is set to the CRS.
2. Go to ORACLE_HOME/oui/bin and run addNode.sh. It will launch the OUI in Add Node mode and displays the welcome screen. Click Next and the Specify Cluster Nodes for Node Addition page displays.
3. Oracle Universal Installer displays the Node Selection Page on which you should enter the node or nodes that you want to add and click Next.
4. Verify the entries that Oracle Universal Installer displays on the Summary Page and click Next. Oracle now runs the usual RAC tests to ensure the new Node is fit to be clubbed into the cluster.
5. The Node Addition Summary page displays the following information showing the products that are installed in the Oracle Clusterware home that you are extending to the new nodes:
  • The source for the add node process, which in this case is the Oracle Clusterware home
  • The private node names that you entered for the new nodes
  • The new nodes that you entered
  • The required and available space on the new nodes
  • The installed products listing the products that are already installed on the existing Oracle Clusterware home
6. Click Next and Oracle Universal Installer displays the Cluster Node Addition Progress page.
7. The Cluster Node Addition Progress page shows the status of the cluster node addition process.
8. From the ORACLE_HOME/install directory on an existing node, run the Oracle Notification Service configuration utility (ONSCONFIG)
./onsconfig add_config node2:remote_port
9. You can check the nodes using ORACLE_HOME/bin/olsnodes -n


Silent Method

1. From one of the node of the existing cluster we will launch the OUI and for that ensure ORACLE_HOME is set to the CRS.
2. Go to ORACLE_HOME/oui/bin and run addNode.sh using following syntax
./addNode.sh –silent "CLUSTER_NEW_NODES={node2}" "CLUSTER_NEW_PRIVATE_NODE_NAMES={node2-priv}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={node2-vip}"
3. From the ORACLE_HOME/install directory on an existing node, run the Oracle Notification Service configuration utility (ONSCONFIG)
./onsconfig add_config node2:remote_port












Posted by at No comments:
Labels: ,

Thursday, October 1, 2015

Oracle TNS Poison Vulnerability Detection and Remediation

Introduction

The Oracle TNS Poison vulnerability was discovered by Joxean Koret, and vulnerability is defined in Oracle’s security note CVE-2012-1675.
Affected Products and Versions

  • Oracle Database 11g Release 2, versions 11.2.0.2, 11.2.0.3, 11.2.0.4
  • Oracle Database 11g Release 1, version 11.1.0.7
  • Oracle Database 10g Release 2, versions 10.2.0.3, 10.2.0.4, 10.2.0.5

The following metalink notes provide the solution and detailed explanation to the vulnerability.

Detection

The idea is to register a service from a machine(rogue) to the listener of the deployment as a result of which the connections can be directed to this remote service and the TNS packet flowing through this remote service can be sniffed. As a result, the confidentiality and integrity of the Database system can be compromised.
The following step by step process is used to detect in a database.
  • Create a DB instance in a local machine (desktop/laptop).
  • Add a entry lie the following in tnsnames.ora
       POISON=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname)(PORT=1521)))
                change the hostname and port of the DB host.
  • Login to the DB and execute
ALTER SYSTEM SET REMOTE_LISTENER=’POISON’;
ALTER SYSTEM REGISTER;
  • The listener running in the DB host will register the instance running from the local machine.  The following can be seen in the listener log.
    Dynamic address is already listened on (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xxxxxx)(PORT=1521)))
    30-SEP-2015 09:14:15 * service_register * orcl * 0
lsnrctl status shows.
Service "orcl" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...

Remediation

There are three ways to remediate the vulnerability.
  • Disable Dynamic registration of instances at listener.
  • Using Class of Secure Transfer(COST) for single and RAC deployments.
  • Enable Valid Node Checking for Registration(VNCR) for 11.2.0.4 and above.

Disable Dynamic Registration

Switch off dynamic registration at the listener in it’s listener.ora by using the following
dynamic_registration_LISTENER_NAME=off

COST for Single and RAC deployments

Technically the steps for RAC and Standalone are same as the principle for instance registration is same in both cases, however in case of RAC there is added complexity to introduce load balancing in SCAN listeners. Further in RAC TCPS will be used with 1523 port to ensure that registration happens over TCPS protocol. The best approach would be to create the wallet in one node and then copy the wallet file to all the nodes of the cluster. Please do note at only the wallet file(ewallet.p12) is copied and the SSO file is created at each node separately.
To introduce COST the following are the steps. Further ensure that the bug fix patch 12880299 or latest PSU is applied to the Oracle Home from where the Listener is running.
  • Create a wallet and delete all the pre-configured certificates
        $ORACLE_HOME/bin/orapki wallet create -wallet <wallet_location>
        $ORACLE_HOME/bin/orapki wallet remove -trusted_cert_all -wallet <wallet_location>
  • Add a self signed wallet entry. The following adds the certificate valid for 10 years (3650)
        $ORACLE_HOME/bin/orapki wallet add -wallet <wallet_location> -self_signed -dn "cn=secure_register" -keysize 1024 -validity 3650
  • Create a SSO file
        $ORACLE_HOME/bin/orapki wallet create -wallet <wallet_location> -auto_login
  • Add the parameter secure_register and wallet location to listener.ora
        SECURE_REGISTER_LISTENER = (IPC,TCP,TCPS)
        WALLET_LOCATION =(SOURCE =(METHOD = FILE)(METHOD_DATA = (DIRECTORY = <wallet_location>)))
  • Add the wallet location to the sqlnet.ora of the instance
        WALLET_LOCATION =(SOURCE =(METHOD = FILE)(METHOD_DATA = (DIRECTORY = <wallet_location>)))
  • Bounce the DB instance and Listener to take effect
After the remediation if an instance is tried to register with the listener without wallet information the following will be seen in the listener log.
Wed Sep 30 10:01:59 2015
30-SEP-2015 10:01:59 * service_register_NSGR * 1194
TNS-01194: The listener command did not arrive in a secure transport

 RAC Specific steps
  • Adding TCPS to the Secure_Register to ensure that registration happens only over IPC and TCPS

       SECURE_REGISTER_LISTENER_SCAN1 = (IPC,TCPS)
       SECURE_REGISTER_LISTENER_SCAN2 = (IPC,TCPS)
       SECURE_REGISTER_LISTENER_SCAN3 = (IPC,TCPS)
  • Adding TCPS and 1523 to SCAN Listener configuration
       $CRS_HOME/bin/srvctl modify scan_listener -p TCP:1521/TCPS:1523
  • Bounce the Scan Listeners
  • Updates the REMOTE_LISTENER in the DB
       alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=IP_1)(PORT=1523))(ADDRESS=(PROTOCOL=TCPS)(HOST=IP_2)(PORT=1523))(ADDRESS=(PROTOCOL=TCPS)(HOST=IP_3)(PORT=1523)))' scope=both sid='*';

Valid Node Checking for Registration (VNCR)

VNCR is the solution provided by Oracle from Database version 11.2.0.4 and above. The solution is build in the listener binaries and can be switched on by enabling the following switches in listener.ora

In case of Standalone
              VALID_NODE_CHECKING_REGISTRATION_LISTENER=ON

 In case of RAC, the SCAN listeners need to have the address of the remote nodes as well, so a parameter REGISTRATION_INVITED_NODES is used
VALID_NODE_CHECKING_REGISTRATION_LISTENER=ON
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN1=ON
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN2=ON
VALID_NODE_CHECKING_REGISTRATION_LISTENER_SCAN3=ON
REGISTRATION_INVITED_NODES_LISTENER_SCAN1=(NODE_1,NODE_2)
REGISTRATION_INVITED_NODES_LISTENER_SCAN2=(NODE_1,NODE_2)
REGISTRATION_INVITED_NODES_LISTENER_SCAN3=(NODE_1,NODE_2)

Posted by at 1 comment:
Labels: , , , , ,

Tuesday, September 29, 2015

Oracle Home missing from Oracle Central Inventory

Due to mysterious reasons the Central Inventory gets corrupted and you get the error following message while performing "opatch lsinventory"


Oracle Interim Patch Installer version 11.2.0.3.12
Copyright (c) 2015, Oracle Corporation.  All rights reserved.
Oracle Home       : /u01/app/db1/oracle/product/11.2.0.3/db
Central Inventory : /u01/app/db1/oraInventory
   from           : /u01/app/db1/oracle/product/11.2.0.3/db/oraInst.loc
OPatch version    : 11.2.0.3.12
OUI version       : 11.2.0.3.0
Log file location : /u01/app/db1/oracle/product/11.2.0.3/db/cfgtoollogs/opatch/opatch2015-09-28_20-56-29PM_1.log
List of Homes on this system:
  Home name= agent12c1, Location= "/u01/app/db1/oracle/product/agent12c/core/12.1.0.3.0"
  Home name= sbin12c1, Location= "/u01/app/db1/oracle/product/agent12c/sbin"
...
Inventory load failed... OPatch cannot load inventory for the given Oracle Home.
Possible causes are:
   Oracle Home dir. path does not exist in Central Inventory
   Oracle Home is a symbolic link
   Oracle Home inventory is corrupted
LsInventorySession failed: Oracle Home inventory cannot be loaded.
OPatch failed with error code 73


Due The issue was due to missing Oracle Home (/u01/app/db1/oracle/product/11.2.0.3/db) in the Central Inventory, so the opatch could not load the home details.
The resolution to this issue is quite simple. Using the runInstaller update the inventory.xml for the missing Oracle Home.


cd $ORACLE_HOME/oui/bin
./runInstaller -silent -ignoreSysPrereqs -attachHome ORACLE_HOME="/u01/app/db1/oracle/product/11.2.0.3/db" ORACLE_HOME_NAME="OraDB11g_home1"
Checking swap space: must be greater than 500 MB.   Actual 8133 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/db1/oraInventory
'AttachHome' was successful.


Note : Do not try to update the inventory.xml manually (vi or notepad) as it may corrupt the Central Inventory.


After the Inventory is repaired try to list inventory using opatch again and VOILA!! it's resolved.
Oracle Interim Patch Installer version 11.2.0.3.12
Copyright (c) 2015, Oracle Corporation.  All rights reserved.
Oracle Home       : /u01/app/db1/oracle/product/11.2.0.3/db
Central Inventory : /u01/app/db1/oraInventory
   from           : /u01/app/db1/oracle/product/11.2.0.3/db/oraInst.loc
OPatch version    : 11.2.0.3.12
OUI version       : 11.2.0.3.0
Log file location : /u01/app/db1/oracle/product/11.2.0.3/db/cfgtoollogs/opatch/opatch2015-09-28_21-07-25PM_1.log
Lsinventory Output file location : /u01/app/db1/oracle/product/11.2.0.3/db/cfgtoollogs/opatch/lsinv/lsinventory2015-09-28_21-07-25PM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: <Removed>
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Database 11g                                                  11.2.0.3.0
There are 1 products installed in this Oracle Home.
Interim patches (27) :
Patch  15862024     : applied on Thu Mar 07 19:39:13 PST 2013
Unique Patch ID:  15795282
   Created on 26 Dec 2012, 07:12:21 hrs PST8PDT
   Patch  13742433     : applied on Thu Mar 07 12:40:29 PST 2013
Unique Patch ID:  15418815
   Created on 9 Mar 2012, 03:31:54 hrs PST8PDT
   Bugs fixed:
     13742433
--------------------------------------------------------------------------------
OPatch succeeded.


Hopefully the small note helps. Happy Patching !!

Posted by at 2 comments:
Labels:
Subscribe to: Posts (Atom)