11G R2 RAC: GPNP PROFILE DEMYSTIFIED

In this post, I will explain what is GPnP profile, what does it contain and how is it used by clusterware.
WHAT IS GPNP PROFILE?
The GPnP profile is a small XML file located in GRID_HOME/gpnp/<hostname>/profiles/peer under the name profile.xml. It is used to establish the correct global personality of a node. Each node maintains a local copy of the GPnP Profile and is maintanied by the GPnP Deamon (GPnPD) .
WHAT DOES GPNP PROFILE CONTAIN?
GPnP Profile  is used to store necessary information required for the startup of Oracle Clusterware like  SPFILE location,ASM DiskString  etc.
It contains various attributes defining node personality.
- Cluster name
- Network classifications (Public/Private)
- Storage to be used for CSS
- Storage to be used for ASM : SPFILE location,ASM DiskString  etc
- Digital signature information : The profile is security sensitive. It might identify the storage to be used as the root partition of a machine.  Hence, it contains digital signature information of the provisioning authority.
Here is the GPnP profile of my RAC setup.
gpnptool can be  used  for reading/editing the gpnp profile.
[root@host01 peer]# gpnptool get
<?xml version=”1.0″ encoding=”UTF-8″?><gpnp:GPnP-Profile Version=”1.0″ xmlns=”http://www.grid-pnp.org/2005/11/gpnp-profile” xmlns:gpnp=”http://www.grid-pnp.org/2005/11/gpnp-profile” xmlns:orcl=”http://www.oracle.com/gpnp/2005/11/gpnp-profile” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd” ProfileSequence=”7″ ClusterUId=”14cddaccc0464f92bfc703ec1004a386″ ClusterName=”cluster01″ PALocation=””><gpnp:Network-Profile><gpnp:HostNetwork id=”gen” HostName=”*”><gpnp:Network id=”net1″ IP=”192.9.201.0″ Adapter=”eth0″ Use=”public”/><gpnp:Network id=”net2″ IP=”10.0.0.0″ Adapter=”eth1″ Use=”cluster_interconnect”/></gpnp:HostNetwork></gpnp:Network-Profile><orcl:CSS-Profile id=”css” DiscoveryString=”+asm” LeaseDuration=”400″/><orcl:ASM-Profile id=”asm” DiscoveryString=”” SPFile=”+DATA/cluster01/asmparameterfile/registry.253.783619911″/><ds:Signature xmlns:ds=”http://www.w3.org/2000/09/xmldsig#”><ds:SignedInfo><ds:CanonicalizationMethod Algorithm=”http://www.w3.org/2001/10/xml-exc-c14n#”/><ds:SignatureMethod Algorithm=”http://www.w3.org/2000/09/xmldsig#rsa-sha1″/><ds:Reference URI=””><ds:Transforms><ds:Transform Algorithm=”http://www.w3.org/2000/09/xmldsig#enveloped-signature”/><ds:Transform Algorithm=”http://www.w3.org/2001/10/xml-exc-c14n#”> <InclusiveNamespaces xmlns=”http://www.w3.org/2001/10/xml-exc-c14n#” PrefixList=”gpnp orcl xsi”/></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm=”http://www.w3.org/2000/09/xmldsig#sha1″/><ds:DigestValue>4VMorzxVNa+FeOx2SCk1unVBpfU=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>bbzV04n2zSGTtUEvqqB+pjw1vH7i8MOEUqkhXAyloX0a41T2FkDEA++ksc0BafndAk7tR+6LGdppE1aOsaJUtYxQqaHJdpVsJF+sj2jN7LPJlT5NBt+K7b08TLjDID92Se6vEiDAeeKlEbpVWKMUIvQvp6LrYK8cDB/YjUnXuGU=</ds:SignatureValue></ds:Signature></gpnp:GPnP-Profile>
WHO UPDATES GPNP PROFILE?
GPnPd daemon replicates changes to the profile during
  • - installation,
  • - system boot or
  • - when updated
Profile is updated Whenever changes are made to a cluster with configuration tools like
  • . oifcfg (Change network),
  • . crsctl (change location of voting disk),
  • . asmcmd (change ASM_DISKSTRING, SPfile location) etc.
HOW IS GPNP PROFILE USED BY CLUSTERWARE?

To start clusterware, voting disk needs to be accessed. If voting disk is on ASM, this information (that voting disk is on ASM) is read from GPnP profile (<orcl:CSS-Profile id=”css” DiscoveryString=”+asm” LeaseDuration=”400″/>).   The voting disk is read using kfed utility  even if ASM is not up.
Next,  the clusterware checks if all the nodes have the updated GPnP profile and the node joins the cluster based on the GPnP configuration . Whenver a node is started/added to the cluster, the clusterware software on the starting node starts a GPnP agent.
  • - If the node is already part of the cluster, the GPnP agent reads the existing profile on that node.
  • - If the node is being added to the cluster, GPnP agent locates agent on another existing node using multicast protocol (provided by mDNS) and gets the profile from that agent.
Next CRSD needs to read OCR to startup various resources on the node and hence update it as status of resources changes. Since OCR is also on ASM, location of ASM SPfile should be known.
The order of searching the ASM SPfile is
  • - GPnP profile
  • - ORACLE_HOME/dbs/spfile<sid.ora>
  • - ORACLE_HOME/dbs/init<sid.ora>
In cluster environment, the location of  SPfile for ASMread from GPnP profile.
[grid@host01 peer]$ gpnptool getpval -asm_spf
Warning: some command line parameters were defaulted. Resulting command line:
         /u01/app/11.2.0/grid/bin/gpnptool.bin getpval -asm_spf -p=profile.xml -o-
+DATA/cluster01/asmparameterfile/registry.253.793721441
The oputput of the query shows that SPfile is on ASM in DATA diskgroup. To find out the
 location of ASM disks, following query is issued :
[root@host01 peer]# gpnptool getpval -asm_dis
ASM-Profile id=”asm” DiscoveryString=””
The  device headers of every device in the disk string returned by the above query are scanned  (if configured by you at ASM initial setup time). Here Discovery String is blank is as ASMDISKSTRINGS parameter has not been set. Hence, headers of all the ASM disks are scanned .
Here, I have shown the output of the query only on the disk which contains SPfile.(spfflg is not null)
[root@host01 ~]#  kfed read /dev/sdb3 | grep -E ‘spf|ausize’
kfdhdb.ausize:                  1048576 ; 0x0bc: 0x00100000
kfdhdb.spfile:                       16 ; 0x0f4: 0x00000010
kfdhdb.spfflg:                        1 ; 0x0f8: 0x00000001
In the output above, we see that
     the device /dev/sdb3 contains a copy of the ASM spfile (spfflg=1).
     The ASM spfile location starts at the disk offset of 16 (spfile=16)
Considering the allocation unit size (kfdhdb.ausize = 1M), let’s dump the ASM spfile from the device:
[root@host01 ~]#  dd if=/dev/sdb3 of=spfileASM_Copy2.ora skip=16  bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.170611 seconds, 6.1 MB/s
[root@host01 ~]# strings spfileASM_Copy2.ora
+ASM1.__oracle_base=’/u01/app/grid’#ORACLE_BASE set from in memory value
+ASM2.__oracle_base=’/u01/app/grid’#ORACLE_BASE set from in memory value
+ASM3.__oracle_base=’/u01/app/grid’#ORACLE_BASE set from in memory value
+ASM3.asm_diskgroups=’FRA’#Manual Mount
+ASM2.asm_diskgroups=’FRA’#Manual Mount
+ASM1.asm_diskgroups=’FRA’#Manual Mount
*.asm_power_limit=1
*.diagnostic_dest=’/u01/app/grid’
*.instance_type=’asm’
*.large_pool_size=12M
*.remote_login_passwordfile=’EXCLUSIVE’
Using the parameters in SPfile, ASM is started.
Once ASM is up, OCR is read by CRSD and various resources on the node are started.
Each node reads network information in GPnP profile and using GNS,  negotiates appropriate network identity for itself . Hence, nodes can be dynamically added/deleted.
What happens if GPnP profile is lost?
To know please click  here.
————————————————————
GPNPTOOL COMMAND REFERENCE:

- How to read the profile
[root@inssc3 bin]# ./gpnptool get
- How to find GPnP Deamons are running on the local node
[root@host01 peer]# gpnptool lfind
Success. Local gpnpd found.
- How to find the location of ASM spfile if the ASM is down
[root@host01 peer]# gpnptool getpval -asm_spf
+DATA/cluster01/asmparameterfile/registry.253.783619911
- How to find all RD-discoverable resources of given type
[root@host01 peer]# gpnptool find
Found 3 instances of service ‘gpnp’.
        mdns:service:gpnp._tcp.local.://host03:18015/agent=gpnpd,cname=cluster01,host=host03,pid=5066/gpnpd h:host03 c:cluster01
        mdns:service:gpnp._tcp.local.://host02:17637/agent=gpnpd,cname=cluster01,host=host02,pid=5236/gpnpd h:host02 c:cluster01
        mdns:service:gpnp._tcp.local.://host01:16633/agent=gpnpd,cname=cluster01,host=host01,pid=5206/gpnpd h:host01 c:cluster01
References:
                     —————
Related links:

 

18 thoughts on “11G R2 RAC: GPNP PROFILE DEMYSTIFIED

  1. [root@host01 ~]# kfed read /dev/sdb3 | grep -E ‘spf|ausize’
    kfdhdb.ausize: 1048576 ; 0x0bc: 0×00100000
    kfdhdb.spfile: 16 ; 0x0f4: 0×00000010
    kfdhdb.spfflg: 1 ; 0x0f8: 0×00000001
    In the output above, we see that
    the device /dev/sdb1 contains a copy of the ASM spfile (spfflg=1).

    Should be /dev/sdb3 not /dev/sdb1

  2. Hi Anju, Your blog is extremely good. I have never gone through a blog which has simplified the concept of RAC to this extent. 11gRAC index is ultimate. I thank you for all your efforts to put up this blog. God bless you for your work and best wishes & regards from my side. Respect to you.

  3. Hi Anju,

    Lets say that we have not configured gns for IP address resolution ( I mean no DHCP).
    Will the Oracle clusterware still read the “profile.xml” for voting disk and ASM spfile location.

    Regards
    Krunal

  4. Hi Anju,

    Lets say that we have not configured gns for IP address resolution ( I mean no DHCP).
    Will the Oracle clusterware still read the “profile.xml” for voting disk and ASM spfile location.

    Regards
    Krunal

  5. This statment I got from oracle doc. – Grid Plug and Play (GPnP) reduces the costs of installing, configuring, and managing server nodes by using a Grid Naming Service (GNS).

    to use GPNP is it mandatory to have GNS ? can’t we use GPNp if we have RAC set up using DNS or host files ?

    1. Yes, GPNP is relevant only in case of cluster with GNS.
      DNS or hosts is needed for node / private IPs even with GNS. GNS is used to configure all the VIPs – Node VIPs and SCAn VIPs.

      Regards
      Anju

  6. Hi Anju,

    The way you show internal details is too good that actually makes you different than others.

    one questions: I tried to understand OLR and GPNP in details somehow I got the point but still there is something that is making me sick and that is exactly where is the major difference between both of them.

    As per my understanding OLR introduced in 11gr2 and before that for everything GPNP was enough for starting up all resources then why OLR.

    What is the one line difference that makes impression above all ??

    Regards,
    Varun

    1. Hi Varun,

      Thanks for your time and feedback.

      OLR is the first file that is accessed when clusterware starts up. Prior to 11g R2, since OCR was not in ASM, it was the first file to be accessed when clusterware is started and contains info about the resources which need to be started on each node. From 11g R2 onwards, since OCR is on ASM and is not available intially, OLR which contains info about resources needed to be started on a node, is employed to provide the same info. Since OLR is stored locally on filesystem on each node, it contains info about resources specific to that node only. Once clusterware knows about the resources to be started , it needs additional info required to start those resources which is provided by gpnp profile. Although Gpnp profile contains info which is global to all the nodes, it cannot be stored on ASM as it is required prior to start of ASM. Hence, it is stored locally on each node and is kept synchronized across all the nodes by gpnpd.

      To put it layman’s language, OLR is the menu which tells the cook what should be cooked and gpnp profile informs it about ingredients of the recipe.

      I hope it answers your question. If you still have any doubt, please feel free to revert.

      Regards
      Anju

      1. Hi Anju,

        Everything is clear now. I also read “How Does 11G R2 Clusterware Start ASM When ASM SPfile Is On ASM Itself?” which helped me understanding whole scenario.

        Thanks once again.

        Varun

  7. Hi Anju,
    In the beginning of article, you state that Gpnp profile is used to establish global node personality of a node. But as far as i know gpnp contents are common across all the nodes. So what is that global node personality when there is nothing specific to node available in gpnp profile. Please clarify.

Your comments and suggestions are welcome!