Oracle Automatic storage management Cluster File System - ACFS

Important is that the storage devices operational, mounted and stamped as ASM device to the machine. Using Management Volume (ACFS) the compatibility level must be at least 11.2. on diskgroup level. See below the created statements from the diskgroups inside the ASM. De disksgroups are created with the compatible.* -> 11.2. The diskgroup that will be used for ACFS is the diskgroup “DG_DISK11G_01”

Steps involved to use ACFS

1. Create Diskgroups
2. Load ACFS Drivers
3. Create Filesystem for ACFS Volume Device
4. Mount ACFS Filesystem
5. ASMCA validation of ACFS Filesystem
6. Automatic startup ACFS Mount Device
7. Oracle ACFS Mount Registry

Login in ASM as sysasm with sqlplus.

CREATE DISKGROUP DG_DATA11G_01 NORMAL REDUNDANCY
FAILGROUP FG_DATA11G_DATA_01 DISK
'ORCL:DATAVOL1' NAME DATAVOL1
FAILGROUP FG_DATA11G_DATA_02 DISK
'ORCL:DATAVOL2' NAME DATAVOL2
ATTRIBUTE 'au_size'='1M',
'compatible.asm' = '11.2',
'compatible.rdbms' = '11.2',
'compatible.advm' = '11.2';

CREATE DISKGROUP DG_FRA11G_01 NORMAL REDUNDANCY
FAILGROUP FG_DATA11G_FRA_01 DISK
'ORCL:FRAVOL1' NAME FRAVOL1
FAILGROUP FG_DATA11G_FRA_02 DISK
'ORCL:FRAVOL2' NAME FRAVOL2
ATTRIBUTE 'au_size'='1M',
'compatible.asm' = '11.2', 
'compatible.rdbms' = '11.2',
'compatible.advm' = '11.2';

CREATE DISKGROUP DG_DISK11G_01 EXTERNAL REDUNDANCY
DISK 'ORCL:ACFSVOL1' NAME ACFSVOL1
ATTRIBUTE 'au_size'='1M',
'compatible.asm' = '11.2',
'compatible.rdbms' = '11.2',
'compatible.advm' = '11.2';

Validation ACFS drivers

The validation of the ACFS drivers on the node(s). Without ACFS drivers loaded, it is not possible to use Management Volume. The validation of the status for ACFS and ADVM use command “lsmod|grep oracle” as root.

lsmod|grep oracle
oracleacfs            787460  2
oracleadvm            177792  6
oracleoks             226656  2 oracleacfs,oracleadvm
oracleasm              46100  1

On my machine the ASM, ADVM and ACFS modules are loaded.
If ADVM, ACFS are not loaded; the row with “oracleacfs,oracleadvm” is not visible.

Load ACFS drives

To start the ACFS drives login as root and execute the following command.


$CRS_HOME/bin/acfsload start –s
Where $CRS_HOME is your Oracle Clusterware Home of the Oracle cluster software tree.

/u01/app/grid/product/11.2.0/grid/bin/acfsload start –s

ps -ef|grep acfs
root      3865  3730  1 08:15 ?        00:00:00 /u01/app/11.2.0/grid/perl/bin/perl -w -I /u01/app/11.2.0/grid/perl/lib/5.10.0 -I /u01/app/11.2.0/grid/perl/lib/site_perl/5.10.0 -I /u01/app/11.2.0/grid/lib -I /u01/app/11.2.0/grid/lib /u01/app/11.2.0/grid/lib/acfsload.pl acfsload start

Create the Dynamic volume on ACFS

After this it is now possible to create a dynamic volume on ACFS inside ASM.
Log in on the ASM instance with sqlplus and create 100M dynamic volume on diskgroup DG_DISK11G_01.

sqlplus / as sysasm 
alter diskgroup DG_DISK11G_01 add volume acfsvol_01 size 100M;

select VOLUME_NAME, VOLUME_DEVICE from v$asm_volume;

VOLUME_NAME                    VOLUME_DEVICE
------------------------------ ------------------------------
ACFSVOL_01                     /dev/asm/acfsvol_01-335

Filesystem for ACFS volume device

To create a filesystem on linux for the ACFS volume device, the ACFS volume device has to be identified by Operating System [My operating System = Linux]. Indentification on Linux will be done by using the value in the volume_device column of v$asm_volume.
As root you create a filesystem with de value of VOLUME_DEVICE with the mkfs command. The mkfs has an attribute –t [type] to be used by creating a filesystem on linux.

mkfs -t acfs -b 4k /dev/asm/acfsvol_01-335
mkfs.acfs: version                   = 11.2.0.1.0.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfsvol_01-335
mkfs.acfs: volume size               = 268435456

Now the volume is prepared to be mounted as a filesystem on the Linux node.

Mount ACFS filesystem

Before actually mount it, Create first a directory name.

mkdir -p /acfsdir1

Then mount the filesystem “/acfsdir1” with the dynamic volume that we have created earlier inside ASM.

mount -t acfs /dev/asm/acfsvol_01-335 /acfsdir1

Verifation of the filesystem that it is mounted properly.

mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
oracleasmfs on /dev/oracleasm type oracleasmfs (rw)
/dev/asm/acfsvol_01-335 on /acfsdir1 type acfs (rw)

df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      38471112  14498092  21987280  40% /
/dev/sda1               101086     64067     31800  67% /boot
tmpfs                   517568    157532    360036  31% /dev/shm
/dev/asm/acfsvol_01-335
                        262144     73964    188180  29% /acfsdir1

Directory ownership of the filestucture mounted dynamic volume

Change the directory ownership to oracle and grid user.

mkdir -p /acfsdir1/oracle
mkdir -p /acfsdir1/grid 
chown root:dba /acfsdir1
chown oracle:dba /acfsdir1/oracle
chown grid:dba /acfsdir1/grid

ASMCA validation method for dynamic Volumes

Check if ADVM Drive Status is loaded. If not loaded see <Load ACFS drives>

Automatic startup ACFS mount devices

the acfsutil registry adds or deletes a file system from the Oracle ACFS persistent mount registry. The mount registry is a global registry that is used at Oracle ACFS startup on each node to mount all file systems specified in it. root or asmadmin privileges are required to modify the registry.
Any user is allowed to display the contents of the registry. To mount all the file systems in the Oracle ACFS mount registry, use the platform specific mount command with the alloption. This is done automatically at Oracle ACFS startup on each node.
If no options are specified, the command displays all of the Oracle ACFS mounts in the registry.

acfsutil registry -a /dev/asm/acfsvol_01-335 /acfsdir

And we have created a shared filesystem with ACFS on ASM.
Check if averting is working with a reboot.

Oracle ACFS Mount Registry

The Oracle ACFS mount registry is an operating system independent, multi-platform mount registration facility supporting both supporting both Oracle Restart and Oracle Grid Infrastructure cluster configurations. File systems that are to be mounted persistently (across reboots) can be registered with the Oracle ACFS mount registry. In cluster configurations, registered Oracle ACFS file systems are automatically mounted by the mount registry, similar to a clusterwide mount table. However, automatic mounting of registered Oracle ACFS file systems are not supported for Oracle Restart configurations.

By default, an Oracle ACFS file system that is inserted into the cluster mount registry is automatically mounted on all cluster members, including cluster members that are added after the registry addition. However, the cluster mount registry also accommodates single-node and multi-node (subset of cluster nodes) file system registrations. The mount registry actions for each cluster member mount only registered file systems that have been designated for mounting on that member.

The Oracle ACFS registry resource actions are designed to automatically mount a file system only one time for each Oracle Grid Infrastructure initialization to avoid potential conflicts with administrative actions to dismount a given file system.

For information about registering an Oracle ACFS file system using the acfsutil command

For more information see:

• Oracle Documentation 11gR2 - Introduction to Oracle ACFS