- iRODS admins
- iRODS users
- connected to iRODS, see How to connect to iRODS using iCommands
- iRODS instance with SURF Data Archive coupled as storage resource (you can check with 'ilsresc' command)
We use the offline facilities that the SURF Data Archive service offers. For reference, we keep here a link to the DMF commands, which display, among other information, the status that a file is in: Data Archive: DMF commands#CheckingFileStatus
Data objects stored through iRODS on SURF Data Archive will (in time) also be put offline on tape according to the policy dictated by the SURF Data Archive service. When that has occurred and you try to get/download the file using iCommands or any other iRODS client, the get request gives an error, because iRODS rules will flag that the data is offline and iRODS will cut the connection as it can not be retrieved without staging the file back online.
Do you wish to download offline data?
This command will fail when the file is offline. You can check the status of a file and bring it online in the following sections.
Checking the status of a file
Check the metadata of the object for additional information:
The metadata attribute 'SURF-DMF: OFL' indicates that the data object is offline and 'SURF-BFID' and 'SURF-TIME' are additional metadata used by the iRODS ruleset to process the status of the data object.
In order to stage the data object to ONLINE, so that you can download the file, you can use the DMF tools installed on the iRODS resource server which is connected to the SURF Data Archive.
Note that the metadata of the iRODS object will not be updated until another get request is made. i.e. it will not be updated by default when it is online and the metadata information might not be up to date. See the last section below.
Bringing a file back from offline
With a simple iRODS rule (as below), we can initiate the staging of the file from offline (OFL) to online (DUL):
In the following code, the *dataPath variable actually is pointing to a physical path in the file system (i.e.: not an iRODS logical path)
Note that the argument, `*dataPath`, for iRODS Rule is not the logical iRODS path, but the physical path of the object. This argument can be a space-separated list of objects:
Note that the hostname "your-irods-instance.irods.surfsara.nl" is not necessarily the main endpoint of your iRODS service (typically the iCAT server), it is the hostname of the iRODS resource server where the SURF Data Archive is attached. For example, it could be something like myorg-resc.irods.surfsara.nl or myorg-data.irods.surfsara.nl.
Either you can invoke the rule manually:
or you can use this simple rule as a component in a more complex iRODS rule structure.
Viewing the file's status in the file system
If you want to discover when an object is online you can poll its status, bypassing the iRODS metadata, using this rule:
if you write it in a file called dmattr.r and execute it, you will get an output similar to this:
The acronym "DUL" indicates that it is online. If "UNM" appears, it means that the system is in the process of staging the object online (i.e., the copy is still in progress).
Viewing the file's status in the iRODS metadata
Once a file has been offline and successfully brought back online, the iRODS metadata will not yet reflect the new status.
You can see that a file is successfully brought back online by looking at the file's status in the file system (as explained in the previous section).
In this scenario, a new
iget will succeed, and then the iRODS metadata will also reflect the new status:
You can now check the new status in the iRODS metadata:
See that the new value is DUL.