We're undertaking a storage migration in some of our data centres from one vendor to another so all of the content has to be moved over to allow the old system to be switched off (and probably thrown in the bin).
Other teams are handling the simple cases where it's just 'normal' filesystems and we've just got to pick up the 'difficult' ASM ones. At least that's the perception from people not in the DBA team as actually the ASM migration is pretty much as simple as it gets i think for this kind of thing....
So the steps are:
1. Get the new devices/luns advertised to the server in question (storage team)
2. Create the devices files/links as appropriate at the os level (unix team)
3. Do the DBA bits (see below)
4. Remove the old os stuff (unix team)
5. Deadvertise (if thats the right word) the luns from the old array (storage team)
So for the DBA bits it's amazingly easy
First we just check the new devices have become visible in v$asm_disk
If they are visible we can do the migration a diskgroup at a time using this syntax
SQL> alter diskgroup fra add disk '/dev/mapper/oracleasm-disk1_3PAR','/dev/mapper/oracleasm-disk2_3PAR' drop disk 'FRA_0001','FRA_0000' rebalance power 5;
Diskgroup altered.
Few points on this:
1. We do the add of the new and the drop of the old in one command to only have to rebalance once
2. The old devices we seem to have to specify the ASM name - the full path to the disk does not work
3. The command returns instantly - but that does not mean it's complete
To check for completeness we can use this query - run over and over which is similar to v$session_longops
SQL> select INST_ID, OPERATION, STATE, POWER, SOFAR, EST_WORK, EST_RATE, EST_MINUTES from GV$ASM_OPERATION where GROUP_NUMBER=1;
1 REBAL WAIT 5
1 REBAL WAIT 5
SQL> /
1 REBAL EST 5 0 0 0 0
1 REBAL WAIT 5 0 0 0 0
SQL> /
1 REBAL RUN 5 3 16196 2507 6
1 REBAL WAIT 5 0 0 0 0
SQL> /
1 REBAL RUN 5 914 16196 6744 2
1 REBAL WAIT 5 0 0 0 0
SQL> /
1 REBAL RUN 5 11234 16196 9135 0
1 REBAL WAIT 5 0 0 0 0
SQL> /
1 REBAL RUN 5 13296 16196 9126 0
1 REBAL WAIT 5 0 0 0 0
SQL> /
1 REBAL RUN 5 14458 16196 8714 0
1 REBAL WAIT 5 0 0 0 0
SQL> /
no rows selected
SQL> /
You can see the numbers go up/down until the process is complete.
You can also check in the alert log for all the messages related to the background cleanup that is happening - see excerpt below
SQL> alter diskgroup data add disk '/dev/mapper/oracleasm-disk3_3PAR','/dev/mapper/oracleasm-disk4_3PAR','/dev/mapper/oracleasm-disk5_3PAR','/dev/mapper/oracleasm-disk6_3PAR','/dev/mapper/oracleasm-disk7_3PAR'
drop disk 'DATA_0000','DATA_0001','DATA_0002','DATA_0003','DATA_0004' rebalance power 5
Wed Feb 25 11:49:30 2015
NOTE: Assigning number (1,5) to disk (/dev/mapper/oracleasm-disk3_3PAR)
NOTE: Assigning number (1,6) to disk (/dev/mapper/oracleasm-disk4_3PAR)
NOTE: Assigning number (1,7) to disk (/dev/mapper/oracleasm-disk5_3PAR)
NOTE: Assigning number (1,8) to disk (/dev/mapper/oracleasm-disk6_3PAR)
NOTE: Assigning number (1,9) to disk (/dev/mapper/oracleasm-disk7_3PAR)
NOTE: requesting all-instance membership refresh for group=1
NOTE: Disk 0 in group 1 is assigned fgnum=1
NOTE: Disk 1 in group 1 is assigned fgnum=2
NOTE: Disk 2 in group 1 is assigned fgnum=3
NOTE: Disk 3 in group 1 is assigned fgnum=4
NOTE: Disk 4 in group 1 is assigned fgnum=5
NOTE: Disk 5 in group 1 is assigned fgnum=6
NOTE: Disk 6 in group 1 is assigned fgnum=7
NOTE: Disk 7 in group 1 is assigned fgnum=8
NOTE: Disk 8 in group 1 is assigned fgnum=9
NOTE: Disk 9 in group 1 is assigned fgnum=10
NOTE: initializing header on grp 1 disk DATA_0005
NOTE: initializing header on grp 1 disk DATA_0006
NOTE: initializing header on grp 1 disk DATA_0007
NOTE: initializing header on grp 1 disk DATA_0008
NOTE: initializing header on grp 1 disk DATA_0009
NOTE: requesting all-instance disk validation for group=1
Wed Feb 25 11:49:30 2015
NOTE: skipping rediscovery for group 1/0x6e68e620 (DATA) on local instance.
Wed Feb 25 11:49:30 2015
NOTE: requesting all-instance disk validation for group=1
Wed Feb 25 11:49:30 2015
NOTE: skipping rediscovery for group 1/0x6e68e620 (DATA) on local instance.
Wed Feb 25 11:49:30 2015
GMON updating for reconfiguration, group 1 at 18 for pid 26, osid 11057
Wed Feb 25 11:49:30 2015
NOTE: group 1 PST updated.
Wed Feb 25 11:49:30 2015
NOTE: membership refresh pending for group 1/0x6e68e620 (DATA)
Wed Feb 25 11:49:30 2015
GMON querying group 1 at 19 for pid 15, osid 30181
NOTE: cache opening disk 5 of grp 1: DATA_0005 path:/dev/mapper/oracleasm-disk3_3PAR
NOTE: cache opening disk 6 of grp 1: DATA_0006 path:/dev/mapper/oracleasm-disk4_3PAR
NOTE: cache opening disk 7 of grp 1: DATA_0007 path:/dev/mapper/oracleasm-disk5_3PAR
NOTE: cache opening disk 8 of grp 1: DATA_0008 path:/dev/mapper/oracleasm-disk6_3PAR
NOTE: cache opening disk 9 of grp 1: DATA_0009 path:/dev/mapper/oracleasm-disk7_3PAR
GMON querying group 1 at 20 for pid 15, osid 30181
Wed Feb 25 11:49:30 2015
SUCCESS: refreshed membership for 1/0x6e68e620 (DATA)
Wed Feb 25 11:49:30 2015
NOTE: starting rebalance of group 1/0x6e68e620 (DATA) at power 5
Starting background process ARB0
Wed Feb 25 11:49:30 2015
SUCCESS: alter diskgroup data add disk '/dev/mapper/oracleasm-disk3_3PAR','/dev/mapper/oracleasm-disk4_3PAR','/dev/mapper/oracleasm-disk5_3PAR','/dev/mapper/oracleasm-disk6_3PAR','/dev/mapper/oracleasm-disk7_3PAR'
drop disk 'DATA_0000','DATA_0001','DATA_0002','DATA_0003','DATA_0004' rebalance power 5
Wed Feb 25 11:49:30 2015
ARB0 started with pid=27, OS id=11066
NOTE: assigning ARB0 to group 1/0x6e68e620 (DATA) with 5 parallel I/Os
Wed Feb 25 11:49:30 2015
NOTE: F1X0 on disk 0 (fmt 1) relocated at fcn 0.29403: AU 2 -> AU 0
NOTE: F1X0 on disk 9 (fmt 1) relocated at fcn 0.29403: AU 0 -> AU 2
NOTE: 02/25/15 11:49:30 DATA.F1X0 copy 1 relocating from 0:2 to 9:2 at FCN 0.29403
Wed Feb 25 11:49:34 2015
NOTE: Attempting voting file refresh on diskgroup DATA
Wed Feb 25 11:50:35 2015
NOTE: stopping process ARB0
Wed Feb 25 11:50:35 2015
NOTE: requesting all-instance membership refresh for group=1
Wed Feb 25 11:50:35 2015
GMON updating for reconfiguration, group 1 at 21 for pid 29, osid 26407
Wed Feb 25 11:50:35 2015
NOTE: group 1 PST updated.
SUCCESS: grp 1 disk DATA_0000 emptied
SUCCESS: grp 1 disk DATA_0001 emptied
SUCCESS: grp 1 disk DATA_0002 emptied
SUCCESS: grp 1 disk DATA_0003 emptied
SUCCESS: grp 1 disk DATA_0004 emptied
NOTE: erasing header on grp 1 disk DATA_0000
NOTE: erasing header on grp 1 disk DATA_0001
NOTE: erasing header on grp 1 disk DATA_0002
NOTE: erasing header on grp 1 disk DATA_0003
NOTE: erasing header on grp 1 disk DATA_0004
NOTE: process _x000_+asm (26407) initiating offline of disk 0.2526549720 (DATA_0000) with mask 0x7e in group 1 (DATA) without client assisting
NOTE: process _x000_+asm (26407) initiating offline of disk 1.2526549719 (DATA_0001) with mask 0x7e in group 1 (DATA) without client assisting
NOTE: process _x000_+asm (26407) initiating offline of disk 2.2526549723 (DATA_0002) with mask 0x7e in group 1 (DATA) without client assisting
NOTE: process _x000_+asm (26407) initiating offline of disk 3.2526549721 (DATA_0003) with mask 0x7e in group 1 (DATA) without client assisting
NOTE: process _x000_+asm (26407) initiating offline of disk 4.2526549722 (DATA_0004) with mask 0x7e in group 1 (DATA) without client assisting
NOTE: initiating PST update: grp 1 (DATA), dsk = 0/0x969816d8, mask = 0x6a, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 1/0x969816d7, mask = 0x6a, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 2/0x969816db, mask = 0x6a, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 3/0x969816d9, mask = 0x6a, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 4/0x969816da, mask = 0x6a, op = clear
Wed Feb 25 11:50:35 2015
GMON updating disk modes for group 1 at 22 for pid 29, osid 26407
NOTE: group DATA: updated PST location: disk 0005 (PST copy 0)
Wed Feb 25 11:50:35 2015
NOTE: PST update grp = 1 completed successfully
NOTE: initiating PST update: grp 1 (DATA), dsk = 0/0x969816d8, mask = 0x7e, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 1/0x969816d7, mask = 0x7e, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 2/0x969816db, mask = 0x7e, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 3/0x969816d9, mask = 0x7e, op = clear
NOTE: initiating PST update: grp 1 (DATA), dsk = 4/0x969816da, mask = 0x7e, op = clear
Wed Feb 25 11:50:35 2015
GMON updating disk modes for group 1 at 23 for pid 29, osid 26407
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 0 of grp 1: DATA_0000
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 1 of grp 1: DATA_0001
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 2 of grp 1: DATA_0002
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 3 of grp 1: DATA_0003
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 4 of grp 1: DATA_0004
Wed Feb 25 11:50:35 2015
NOTE: PST update grp = 1 completed successfully
Wed Feb 25 11:50:35 2015
GMON updating for reconfiguration, group 1 at 24 for pid 29, osid 26407
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 0 of grp 1: (not open) DATA_0000
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 1 of grp 1: (not open) DATA_0001
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 2 of grp 1: (not open) DATA_0002
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 3 of grp 1: (not open) DATA_0003
Wed Feb 25 11:50:35 2015
NOTE: cache closing disk 4 of grp 1: (not open) DATA_0004
Wed Feb 25 11:50:35 2015
NOTE: group 1 PST updated.
Wed Feb 25 11:50:36 2015
NOTE: membership refresh pending for group 1/0x6e68e620 (DATA)
Wed Feb 25 11:50:36 2015
SUCCESS: rebalance completed for group 1/0x6e68e620 (DATA)
Wed Feb 25 11:50:36 2015
GMON querying group 1 at 25 for pid 15, osid 30181
GMON querying group 1 at 26 for pid 15, osid 30181
Wed Feb 25 11:50:36 2015
NOTE: Disk DATA_0000 in mode 0x0 marked for de-assignment
NOTE: Disk DATA_0001 in mode 0x0 marked for de-assignment
NOTE: Disk DATA_0002 in mode 0x0 marked for de-assignment
NOTE: Disk DATA_0003 in mode 0x0 marked for de-assignment
NOTE: Disk DATA_0004 in mode 0x0 marked for de-assignment
SUCCESS: refreshed membership for 1/0x6e68e620 (DATA)
NOTE: Attempting voting file refresh on diskgroup DATA
Easy huh?
Nice one. One thing can be added to the following line...
ReplyDelete--First we just check the new devices have become visible in v$asm_disk -> + HEADER_STATUS='CANDIDATE'.