Fixing 12c ASM and an adventure with "oracle restart"



We've been playing around with ASM with a view to roll it out on all of our development environments. ASM is at a point now (and to be honest has for a while) where you are crazy to not use it. These environments are all single server/single instance (i.e no RAC) but ASM still offers performance benefits and should further simplify our provisioning processes.

We had an install already present on our first server where we were trying out acfs for which we had actually removed all the raw devices used for ASM so the install was slightly 'broken'. So first we had to fix it before re-initialising ASM and divvying up some space.

Lets first see what the cut down clusterware thinks is configured:

crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details     
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               OFFLINE OFFLINE      server STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       server STABLE
ora.asm
               OFFLINE OFFLINE      server Instance Shutdown,ST
                                                             ABLE
ora.ons
               OFFLINE OFFLINE      server STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       server STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.endcon.db
      1        OFFLINE OFFLINE                               Instance Shutdown,ST
                                                             ABLE
ora.evmd
      1        ONLINE  ONLINE       server STABLE
ora.orcl.db
      1        OFFLINE OFFLINE                               Instance Shutdown,ST
                                                             ABLE
--------------------------------------------------------------------------------

Straight away i can see that the database "endcon" whcih we used for our ACFS tests is no longer used and should be removed - so lets do that first

srvctl remove database -d endcon
Remove the database endcon? (y/[n]) y
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

As i know that the raw devices are 'toast' the existing diskgroup can be removed - lets do that

srvctl remove diskgroup -g DATA
PRCA-1002 : Failed to remove CRS resource ora.DATA.dg for ASM Disk Group DATA
PRCR-1028 : Failed to remove resource ora.DATA.dg
PRCR-1072 : Failed to unregister resource ora.DATA.dg
CRS-0222: Resource 'ora.DATA.dg' has dependency error.
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

And we cant as there is some dependency on that - lets see what it is. I though there would be a simple command to show this but couldn;t find one so i ended up writing a (very) simple shell loop to give me those details:

for res in `crsctl stat res -t |grep ora. `; do echo $res ;crsctl status res $res -f |grep DEPENDENCIES; done

This shows us the resource followed by any dependencies it has


ora.DATA.dg
START_DEPENDENCIES=hard(ora.asm) pullup(ora.asm)
STOP_DEPENDENCIES=hard(intermediate:ora.asm)
ora.LISTENER.lsnr
START_DEPENDENCIES=
STOP_DEPENDENCIES=
ora.asm
START_DEPENDENCIES=hard(ora.cssd) weak(ora.LISTENER.lsnr)
STOP_DEPENDENCIES=hard(ora.cssd)
ora.ons
START_DEPENDENCIES=
STOP_DEPENDENCIES=
ora.cssd
START_DEPENDENCIES=weak(concurrent:ora.diskmon)
STOP_DEPENDENCIES=hard(shutdown:ora.diskmon)
ora.diskmon
START_DEPENDENCIES=weak(concurrent:ora.cssd)pullup:always(ora.cssd)
STOP_DEPENDENCIES=
ora.evmd
START_DEPENDENCIES=
STOP_DEPENDENCIES=hard(shutdown:ora.cssd)
ora.orcl.db
START_DEPENDENCIES=hard(ora.DATA.dg) weak(type:ora.listener.type,uniform:ora.ons) pullup(ora.DATA.dg)
STOP_DEPENDENCIES=hard(intermediate:ora.asm,shutdown:ora.DATA.dg)

So we can see that the orcl database (which somehow i overlooked) has files in ASM - i know this database has already gone so lets remove that database too from the config.

srvctl remove database -d orcl
Remove the database orcl? (y/[n]) y
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

Now the diskgroup should go

srvctl remove diskgroup -g DATA
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

Hmm no feedback, lets try again

[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]# srvctl remove diskgroup -g DATA
PRCA-1000 : ASM Disk Group DATA does not exist
PRCR-1001 : Resource ora.DATA.dg does not exist
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

Ok so it had gone it just returned no message
Now we've cleaned up all of that lets try and start the original asm resource that the install created - this should just start the instance without trying to mount any disk groups - and indeed it does.

 crsctl start res ora.asm
CRS-2672: Attempting to start 'ora.asm' on 'server'
CRS-2676: Start of 'ora.asm' on 'server' succeeded

Now lets log on to asm (with sysasm rights) and see if it can see the new disks that have been advertised to it.


  1* select path,state,header_status,OS_MB,TOTAL_MB,FREE_MB,NAME from v$asm_disk
SQL> /
/dev/raw/raw3  NORMAL    CANDIDATE      61441      0        0
/dev/raw/raw2  NORMAL    CANDIDATE      61441      0        0
/dev/raw/raw6  NORMAL    CANDIDATE      61441      0        0
/dev/raw/raw7  NORMAL    CANDIDATE      61441      0        0
/dev/raw/raw4  NORMAL    CANDIDATE      61441      0        0
/dev/raw/raw1  NORMAL    CANDIDATE      61441      0        0
/dev/raw/raw5  NORMAL    CANDIDATE      61441      0        0

7 rows selected.

Ok looks good - lets now create some diskgroups - one for DATA and one for FRA as seems to be the normal convention.

SQL> create diskgroup data external redundancy disk '/dev/raw/raw1','/dev/raw/raw2','/dev/raw/raw3','/dev/raw/raw4','/dev/raw/raw5';

Diskgroup created.

SQL>


SQL> create diskgroup fra external redundancy disk '/dev/raw/raw6','/dev/raw/raw7';

Diskgroup created.

SQL>

Lets see what v$asm_disk looks like now - and we can see the details have updated and the disks now have ASM names.

SQL> select path,state,header_status,OS_MB,TOTAL_MB,FREE_MB,NAME from v$asm_disk;
/dev/raw/raw7  NORMAL    MEMBER          61441      61441    61416 FRA_0001
/dev/raw/raw6  NORMAL    MEMBER          61441      61441    61414 FRA_0000
/dev/raw/raw5  NORMAL    MEMBER          61441      61441    61431 DATA_0004
/dev/raw/raw4  NORMAL    MEMBER          61441      61441    61430 DATA_0003
/dev/raw/raw3  NORMAL    MEMBER          61441      61441    61430 DATA_0002
/dev/raw/raw2  NORMAL    MEMBER          61441      61441    61427 DATA_0001
/dev/raw/raw1  NORMAL    MEMBER          61441      61441    61429 DATA_0000

Lets see what shows in the diskgroups view

SQL> select NAME,TOTAL_MB,FREE_MB,USABLE_FILE_MB,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;
FRA                   122882     122830         122830 10.1.0.0.0                             10.1.0.0.0
DATA                   307205     307147         307147 10.1.0.0.0                             10.1.0.0.0

SQL>

OK all looks good - has the config been updated?

SQL> sho parameter asm
asm_diskgroups                 string     DATA, FRA
asm_diskstring                 string     /dev/raw/*
asm_power_limit              integer     1
asm_preferred_read_failure_groups    string

It seems so - lets restart ASM

SQL> shutdown;
ASM diskgroups dismounted
ASM instance shutdown
SQL>

sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 13:31:17 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ASM instance started

Total System Global Area 1135747072 bytes
Fixed Size            2297344 bytes
Variable Size         1108283904 bytes
ASM Cache           25165824 bytes
ORA-15110: no diskgroups mounted

Hmm - that didn't work and it seems the change to asm_diskgroups was only done in memory (as we are using an old style pfile)

SQL> sho parameter asm

NAME                     TYPE     VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups                 string
asm_diskstring                 string     /dev/raw/*
asm_power_limit              integer     1
asm_preferred_read_failure_groups    string
SQL>

Lets convert to spfile

SQL> create spfile from pfile;
create spfile from pfile
*
ERROR at line 1:
ORA-00349: failure obtaining block size for
'+DATA/ASM/ASMPARAMETERFILE/registry.253.828807127'
ORA-15001: diskgroup "DATA" does not exist or is not mounted

And we can;t as the spfile needs to be stored in the DATA diskgroup which is not mounted yet (chicken and egg situation developing here...)

Lets put the diskgroups in the init file

 vi init+ASM.ora
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]# cat init+ASM.ora
  large_pool_size          = 12M
  remote_login_passwordfile= "EXCLUSIVE"
  local_listener           = "LISTENER_+ASM"
  asm_diskstring           = "/dev/raw/*"
  asm_power_limit          = 1
  asm_diskgroups            = DATA,FRA

Lets start now

sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 13:33:54 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ASM instance started

Total System Global Area 1135747072 bytes
Fixed Size            2297344 bytes
Variable Size         1108283904 bytes
ASM Cache           25165824 bytes
ASM diskgroups mounted
SQL>

Now lets convert to spfile

[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]# sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 13:33:54 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ASM instance started

Total System Global Area 1135747072 bytes
Fixed Size            2297344 bytes
Variable Size         1108283904 bytes
ASM Cache           25165824 bytes
ASM diskgroups mounted
SQL> create spfile from pfile;
create spfile from pfile
*
ERROR at line 1:
ORA-17502: ksfdcre:4 Failed to create file
+DATA/ASM/ASMPARAMETERFILE/registry.253.828807127
ORA-15221: ASM operation requires compatible.asm of 11.2.0.0.0 or higher

And we can't - lets change the compatible


SQL> alter diskgroup data set attribute 'compatible.asm'='11.2.0.0.0';

Diskgroup altered.

SQL> alter diskgroup fra set attribute 'compatible.asm'='11.2.0.0.0';

Diskgroup altered.

SQL> alter diskgroup data set attribute 'compatible.rdbms'='11.2.0.0.0';

Diskgroup altered.

SQL> alter diskgroup fra set attribute 'compatible.rdbms'='11.2.0.0.0';

Diskgroup altered.

Now lets try the spfile

SQL> create spfile from pfile;

File created.

OK - all looks good - lets see what the local clusterware knows about now

crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details     
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       server STABLE
ora.FRA.dg
               ONLINE  ONLINE       server STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       server STABLE
ora.asm
               ONLINE  ONLINE       server Started,STABLE
ora.ons
               OFFLINE OFFLINE      server STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       server STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       server STABLE
--------------------------------------------------------------------------------

So it seems creating the diskgroups automatically populated into the local cluster config.

Now ASM is up and running and should have some usable space - lets try creating a file in it from the instance TEST.

Setting Enviroment for TEST
[ oracle@server]:TEST :[ /oracle/product/12.1.0/grid/dbs ]# sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 13:43:02 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/oracle/12.0.0.1/dbs/dbs1TEST.dbf
/oracle/12.0.0.1/dbs/dbx1TEST.dbf
/oracle/12.0.0.1/dbs/dbu1TEST.dbf

SQL> create tablespace demo datafile '+DATA';

Tablespace created.

SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/oracle/12.0.0.1/dbs/dbs1TEST.dbf
/oracle/12.0.0.1/dbs/dbx1TEST.dbf
/oracle/12.0.0.1/dbs/dbu1TEST.dbf
+DATA/TEST/DATAFILE/demo.256.840548627

All looks good - lets now look at the local config again

crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details     
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       server STABLE
ora.FRA.dg
               ONLINE  ONLINE       server STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       server STABLE
ora.asm
               ONLINE  ONLINE       server Started,STABLE
ora.ons
               OFFLINE OFFLINE      server STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       server STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       server STABLE
--------------------------------------------------------------------------------

And there is no dependency there shown - i guess as the TEST database was not registered with the software so it couldnt build the dependency

Lets switch back to ASM and see that the TEST database shows up as a client now.

sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 15:26:53 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Automatic Storage Management option

SQL> select * from v$asm_client;

GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION                         CON_ID
------------------------------------------------------------ ----------
       1 TEST
TEST     CONNECTED
12.1.0.1.0
12.0.0.0.0                                  0

Yep all looks good - lets drop the tablespace form the TEST database and see if the relationship disappears from v$asm_client

sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 15:44:24 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options

SQL> drop tablespace demo;

Tablespace dropped.

SQL>


SQL> select * from v$asm_client;

GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION                         CON_ID
------------------------------------------------------------ ----------
       1 TEST
TEST     CONNECTED
12.1.0.1.0
12.0.0.0.0                                  0


SQL>

And it's still there - so the link is more persitent than that - lets bounce the database and see if it goes.

SQL> shutdown;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 1169227776 bytes
Fixed Size            2680792 bytes
Variable Size         1107298344 bytes
Database Buffers       50331648 bytes
Redo Buffers            8916992 bytes
Database mounted.
Database opened.
SQL>


 sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Wed Feb 26 15:46:46 2014

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Automatic Storage Management option

SQL> select * from v$asm_client;

no rows selected

SQL>

Yep all gone now. Lets now add the TEST database that we created manually into the oracle restart (or local clusterware whatever you want to call it)

srvctl add database -d TEST -o /oracle/12.0.0.1
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

Does it appear ok?

crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details     
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       server STABLE
ora.FRA.dg
               ONLINE  ONLINE       server STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       server STABLE
ora.asm
               ONLINE  ONLINE       server Started,STABLE
ora.ons
               OFFLINE OFFLINE      server STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       server STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       server STABLE
ora.test.db
      1        OFFLINE OFFLINE                               STABLE
--------------------------------------------------------------------------------
[ oracle@server ]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

It's there - but it shows as offline - lets try and start it via srvctl

srvctl start database -db test
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#

crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details     
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       server STABLE
ora.FRA.dg
               ONLINE  ONLINE       server STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       server STABLE
ora.asm
               ONLINE  ONLINE       server Started,STABLE
ora.ons
               OFFLINE OFFLINE      server STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       server STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       server STABLE
ora.test.db
      1        ONLINE  ONLINE       server Open,STABLE
--------------------------------------------------------------------------------

Ok status set but checking the alert log nothing actually happened - so it must have realised DB was up and just updated the status.

Now lets stop the whole 'thing' and check that everything shuts down

 crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'server'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'server'
CRS-2673: Attempting to stop 'ora.test.db' on 'server'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'server'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'server' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'server' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'server'
CRS-2673: Attempting to stop 'ora.FRA.dg' on 'server'
CRS-2677: Stop of 'ora.FRA.dg' on 'server' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'server'
CRS-2677: Stop of 'ora.test.db' on 'server' succeeded
CRS-2677: Stop of 'ora.asm' on 'server' succeeded

CRS-2675: Stop of 'ora.evmd' on 'server' failed
CRS-2679: Attempting to clean 'ora.evmd' on 'server'
CRS-2681: Clean of 'ora.evmd' on 'server' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'server'
CRS-2677: Stop of 'ora.cssd' on 'server' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'server' has completed
CRS-4133: Oracle High Availability Services has been stopped.
You have new mail in /var/mail/oracle

So that all look good but took a worrying amount of time -  about 5 minutes
the TEST db is down - lets just check the alert log

USER (ospid: 28673): terminating the instance
Instance terminated by USER, pid = 28673

Hmm -don't like the look of that - looks like it's just been killed

Lets bring everything back up

 crsctl start has

CRS-4123: Oracle High Availability Services has been started.
[ oracle@server]:+ASM :[ /oracle/product/12.1.0/grid/dbs ]#


Again took ages - no idea what some of those daemons are doing.

Lets check the shutdown option for the TEST database

crsctl status res ora.test.db -f
NAME=ora.test.db
TYPE=ora.database.type
STATE=ONLINE
TARGET=ONLINE
ACL=owner:oracle:rwx,pgrp:oinstall:r-x,other::r--,group:oinstall:r-x,user:oracle:rwx
ACTIONS=
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTION_TIMEOUT=60
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
ALERT_TEMPLATE=
ALIAS_NAME=
AUTO_START=restore
CARDINALITY=1
CARDINALITY_ID=0
CHECK_INTERVAL=1
CHECK_TIMEOUT=30
CLEAN_TIMEOUT=60
CLUSTER_DATABASE=false
CONFIG_VERSION=1
CREATION_SEED=30
DATABASE_TYPE=SINGLE
DB_UNIQUE_NAME=TEST
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=database) PROPERTY(DB_UNIQUE_NAME= CONCAT(PARSE(%NAME%, ., 2), %USR_ORA_DOMAIN%, .)) ELEMENT(INSTANCE_NAME= %GEN_USR_ORA_INST_NAME%) ELEMENT(DATABASE_TYPE= %DATABASE_TYPE%)
DEGREE=1
DELETE_TIMEOUT=60
DESCRIPTION=Oracle Database resource
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=60
FAILURE_THRESHOLD=1
GEN_AUDIT_FILE_DEST=/oracle/12.0.0.1/rdbms/audit
GEN_START_OPTIONS=
GEN_USR_ORA_INST_NAME=TEST
HOSTING_MEMBERS=
ID=ora.test.db
INSTANCE_COUNT=1
INSTANCE_FAILOVER=1
INTERMEDIATE_TIMEOUT=0
LOAD=1
LOGGING_LEVEL=1
MANAGEMENT_POLICY=AUTOMATIC
MODIFY_TIMEOUT=60
NLS_LANG=
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
ONLINE_RELOCATION_TIMEOUT=0
ORACLE_HOME=/oracle/12.0.0.1
ORACLE_HOME_OLD=
OS_CRASH_THRESHOLD=0
OS_CRASH_UPTIME=0
PLACEMENT=balanced
PROFILE_CHANGE_TEMPLATE=
PWFILE=
RELOCATE_BY_DEPENDENCY=1
RESTART_ATTEMPTS=2
ROLE=PRIMARY
SCRIPT_TIMEOUT=60
SERVER_CATEGORY=
SERVER_POOLS=
SERVER_POOLS_PQ=
SPFILE=
START_CONCURRENCY=0
START_DEPENDENCIES=weak(type:ora.listener.type,uniform:ora.ons)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_CONCURRENCY=0
STOP_DEPENDENCIES=
STOP_TIMEOUT=600
TYPE_VERSION=3.3
UPTIME_THRESHOLD=1h
USER_WORKLOAD=yes
USE_STICKINESS=0
USR_ORA_DB_NAME=
USR_ORA_DOMAIN=
USR_ORA_ENV=
USR_ORA_FLAGS=
USR_ORA_INST_NAME=TEST
USR_ORA_OPEN_MODE=open
USR_ORA_OPI=false
USR_ORA_STOP_MODE=immediate
VERSION=12.1.0.1.0

Which would seem to indicate any stop should be 'immediate'. (as a slight aside here i noticed in the alert log that oracle restart is tagging the startup operations in the grid control by adding a comment as you can see below here

ALTER DATABASE MOUNT /* db agent *//* {0:0:2} */
2014-02-26 16:14:47.633000 +00:00
Using default pga_aggregate_limit of 115120 MB
2014-02-26 16:14:49.044000 +00:00
Successful mount of redo thread 1, with mount id 2138666230
Database mounted in Exclusive Mode
Lost write protection disabled
Ping without log force is disabled.
Completed: ALTER DATABASE MOUNT /* db agent *//* {0:0:2} */
ALTER DATABASE OPEN /* db agent *//* {0:0:2} */

Lets just check what a shutdown immediate looks like in the alert log
shutdown immediate from sqlplus shows this

2014-02-26 16:24:05.096000 +00:00
Shutting down instance (immediate)
Stopping background process SMCO
Stopping background process FBDA
2014-02-26 16:24:07.095000 +00:00
Shutting down instance: further logons disabled
Stopping background process CJQ0
Stopping background process MMNL
Stopping background process MMON
2014-02-26 16:24:09.107000 +00:00
License high water mark = 8
ALTER DATABASE CLOSE NORMAL
SMON: disabling tx recovery
Stopping Emon pool
Stopping Emon pool
SMON: disabling cache recovery
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
Thread 1 closed at log sequence 100
Successful close of redo thread 1
Completed: ALTER DATABASE CLOSE NORMAL
ALTER DATABASE DISMOUNT
Shutting down archive processes
Archiving is disabled
Completed: ALTER DATABASE DISMOUNT
2014-02-26 16:24:15.269000 +00:00
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
2014-02-26 16:24:17.267000 +00:00
ARCH: Archival disabled due to shutdown: 1089
Stopping background process VKTM


Shutting down archive processes
Archiving is disabled
2014-02-26 16:24:19.502000 +00:00
Instance shutdown complete
                                        
And a shutdown abort does this

USER (ospid: 6148): terminating the instance
2014-02-26 16:27:40.243000 +00:00
Instance terminated by USER, pid = 6148
Instance shutdown complete

so it seems restart doesn't even use shutdown abort it just kills the processes. So something to be aware of stopping the local cluster stack just kills everything whereas a srvctl stop database will actually do a shutdown immediate as configured in the resource config.

A useful exercise - i think we'll me trying out more things with restart to make our lives easier and will continue the trial of ASM by beginning to migrate the databases into it.

If you're wondering what the picture has to do with this post then the answer is nothing - i just liked it....




Comments