rirods vs iCommands

This article is meant for iRODS users familiar with iCommands, the command-line interface to iRODS. While rirods functions are based on iCommands, not all iCommands are covered and some functionality may differ. The article also compares the bash calls to the R calls in case the reader is not so familiar with R syntax.

The table below lists the main functions available in rirods and their counterparts in iCommands.

rirods	`iCommands`
`icd()`	`icd`
`ipwd()`	`ipwd`
`ils()`	`ils`
`iget()`, `ireadRDS()`	`iget`
`iput()`, `isaveRDS()`	`iput`
`imeta()`	`imeta`
`iquery()`	`iquest`
`imkdir()`	`imkdir`
`irm()`	`irm`

A general note is that in iCommands the -h argument provides the documentation for a certain command (e.g. icd -h), whereas in R this is achieved by preceding the name of the function with ? (e.g. ?icd). Moreover, the verbose boolean argument in most of rirods commands is used to print information about the HTTP request and response (and is by default FALSE), which is not a behaviour relevant to iCommands.

Working directory and collections

The iCommand ipwd and its rirods counterpart ipwd() work exactly the same, providing the path to the current working directory. The case of icd() is different; unlike icd, which returns the “/zone/home” directory if no argument is provided, icd() needs an argument with the path for a new working directory. icd("/tempZone/home") will take us to "/tempZone/home" like icd might, but icd() with no arguments will throw an error.

When it comes to listing the contents of a collection, both ils and ils() can be given a path, and if they are not, they list the contents of the working directory. Permissions can be shown by ils with the -A argument and by ils() by providing permissions=TRUE, although the latter does not return information about inheritance¹. The time that the item was last modified and the size of data objects is shown by ils -l and by ils(stats=TRUE).

ils() # ils
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                         logical_path
#>  /tempZone/home/rods/some_collection
#>   /tempZone/home/rods/200numbers.rds
ils("some_collection") # ils some_collection
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path
#>       /tempZone/home/rods/some_collection/subcollection
#>  /tempZone/home/rods/some_collection/random_numbers.rds
ils("some_collection", stat = TRUE) # ils -l some_collection
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path inheritance_enabled
#>       /tempZone/home/rods/some_collection/subcollection               FALSE
#>  /tempZone/home/rods/some_collection/random_numbers.rds                <NA>
#>  status_code modified_at permissions.name permissions.perm permissions.type
#>            0  1710617057             rods              own        rodsadmin
#>            0  1710617057             rods              own        rodsadmin
#>  permissions.zone registered        type checksum size
#>          tempZone       TRUE  collection     <NA> <NA>
#>          tempZone       TRUE data_object           832
ils("some_collection", permissions = TRUE) # ils -A some_collection
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path
#>       /tempZone/home/rods/some_collection/subcollection
#>  /tempZone/home/rods/some_collection/random_numbers.rds
ils(permissions=TRUE, stat = TRUE) # ils -Al
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                         logical_path inheritance_enabled status_code
#>  /tempZone/home/rods/some_collection               FALSE           0
#>   /tempZone/home/rods/200numbers.rds                <NA>           0
#>  modified_at permissions.name permissions.perm permissions.type
#>   1710617057             rods              own        rodsadmin
#>   1710617058             rods              own        rodsadmin
#>  permissions.zone registered        type checksum size
#>          tempZone       TRUE  collection     <NA> <NA>
#>          tempZone       TRUE data_object          1632

A crucial difference between ils and ils() is that ils(metadata=TRUE) is used to print the metadata information of each element listed, whereas iCommands only provides metadata information of one element at a time by calling imeta ls on it. At the moment, rirods does not offer any fast way of restricting the metadata to a specific item, although the custom function filter_ils() shown below could be used to achieve this result.

filter_ils <- function(pattern, ils_output = ils()) {
  stopifnot(inherits(ils_output, "irods_df"))
  ils_df <- as.data.frame(ils_output)
  if (length(pattern) == 1) {
    filtered <- ils_df[grepl(pattern, ils_df$logical_path), ]
  } else {
    filtered <- ils_df[basename(ils_df$logical_path) %in% pattern, ]
  }
  rirods:::new_irods_df(filtered)
}

my_files <- ils("some_collection", metadata = TRUE)
my_files
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path    attribute  value
#>  /tempZone/home/rods/some_collection/random_numbers.rds distribution normal
#>                                                               length    100
#>       /tempZone/home/rods/some_collection/subcollection         <NA>   <NA>
#>  units
#>       
#>       
#>   <NA>
filter_ils("random", my_files) # imeta ls -d some_collection/random_numbers.rds
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path    attribute  value
#>  /tempZone/home/rods/some_collection/random_numbers.rds distribution normal
#>                                                               length    100
#>  units
#>       
#>

Creating and deleting collections or data objects

With iCommands we can create new collections and data objects with imkdir and itouch respectively. The former is matched in rirods by imkdir(), but the latter is not covered by the R package yet.

Both in iCommands and in rirods, imkdir() requires a path and has an argument to request creating the parent collections:

imkdir("new_collection") # imkdir new_collection
ils()
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                         logical_path
#>   /tempZone/home/rods/new_collection
#>  /tempZone/home/rods/some_collection
#>   /tempZone/home/rods/200numbers.rds

# imkdir -p another_collection/subcollection
imkdir("another_collection/subcollection", create_parent_collections = TRUE)

ils()
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                            logical_path
#>  /tempZone/home/rods/another_collection
#>      /tempZone/home/rods/new_collection
#>     /tempZone/home/rods/some_collection
#>      /tempZone/home/rods/200numbers.rds
ils("another_collection")
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                          logical_path
#>  /tempZone/home/rods/another_collection/subcollection

Removing data objects and collections can be achieved by irm in iCommands and irm() in rirods, and both functions have a force and recursive arguments:

irm("200numbers.rds", force = FALSE) # irm 200numbers.rds
ils("/tempZone/trash/home/rods")
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                              logical_path
#>  /tempZone/trash/home/rods/200numbers.rds

irm("another_collection", force = TRUE, recursive = TRUE) # irm -rf another_collection
iquery("SELECT COLL_NAME WHERE COLL_NAME LIKE '%_collection'")
#>                                           COLL_NAME
#> 1                /tempZone/home/rods/new_collection
#> 2               /tempZone/home/rods/some_collection
#> 3 /tempZone/home/rods/some_collection/subcollection

Getting and putting data

In order to transfer data between a local system and iRODS, the main iCommands are iput and iget, which have iput() and iget() as rirods counterparts. The rirods functions are much more restricted in that they don’t really offer the variety of arguments that the iCommands do, but only options related to the HTTP requests. Moreover, they require the user to provide both the local and logical paths explicitly, whereas the iCommands reuse the source path by default. In other words, while iput myfile.txt will take a local file “myfile.txt” and store it in the iRODS working directory as “myfile.txt”, iput("myfile.txt") will throw an error; iput("myfile.txt", "myfile.txt") is the correct syntax to achieve the desired effect:

rirods	`iCommands`	Result
`iget("myfile.txt", "myfile.txt")`	`iget myfile.txt`	“myfile.txt” is copied from iRODS to local as “myfile.txt”
`iget("myfile.txt", "another_path/filename.txt")`	`iget myfile.txt another_path/filename.txt`	“myfile.txt” is copied from iRODS to local as “filename.txt” in “another_path”
`iput("data.csv", "data.csv")`	`iput data.csv`	“data.csv” is copied from local to iRODS as “data.csv”
`iput("data.csv", "data2.csv")`	`iput data.csv data2.csv`	“data.csv” is copied from local to iRODS as “data2.csv”

Next to these functions, rirods provides two functions to store R objects directly into iRODS or read RDS files directly from iRODS: isaveRDS() and ireadRDS(). There is no counterpart in iCommands. To achieve the same effect as isaveRDS(), a user would first have to save their object with saveRDS(), then send it to iRODS with iput and finally remove it with irm -f; in the same vein, the effect of ireadRDS() would be achieved by retrieving the file with iget, reading it with readRDS() and then removing it from the local space. Read more about this in vignette("local-irods").

Metadata and querying

The rirods imeta() function covers two of the main functionalities of the imeta iCommand: adding and removing metadata from a data object or collection. This function has three main arguments: the logical path of the data object or collection on which to add the metadata, the entity type (e.g. “data_object”, the default, or “collection”) and a list of operations, in which it is specified whether a certain AVU should be added or removed.

icd("some_collection")
ils(metadata=TRUE)
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path    attribute  value
#>  /tempZone/home/rods/some_collection/random_numbers.rds distribution normal
#>                                                               length    100
#>       /tempZone/home/rods/some_collection/subcollection         <NA>   <NA>
#>  units
#>       
#>       
#>   <NA>

# imeta add -C subcollection foo bar baz
imeta("subcollection",
      operations = list(
        list(operation="add", attribute="foo", value="bar", units="baz")
        )
      )

# imeta rm -d random_numbers.rds distribution normal
imeta("random_numbers.rds",
      operations = list(
        list(operation="remove", attribute="distribution", value="normal")
      ))

ils(metadata=TRUE)
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path attribute value units
#>  /tempZone/home/rods/some_collection/random_numbers.rds       foo   bar   baz
#>       /tempZone/home/rods/some_collection/subcollection       foo   bar   baz

In order to “modify” an AVU with rirods, we would have to remove it and add its replacement. This also illustrates how we can add/remove several AVUs in one imeta() call, which would not be possible with imeta².

# imeta mod -d random_numbers.rds length 100 items u:elements
imeta("random_numbers.rds",
      operations = list(
        list(operation="remove", attribute="length", value="100", units="items"),
        list(operation="add", attribute="length", value="100", units="elements")
      ))
ils(metadata=TRUE)
#> 
#> ==========
#> iRODS Zone
#> ==========
#>                                            logical_path attribute value units
#>  /tempZone/home/rods/some_collection/random_numbers.rds       foo   bar   baz
#>       /tempZone/home/rods/some_collection/subcollection       foo   bar   baz

As mentioned before, whereas the iCommand imeta ls is used to list the metadata of an item, ils(metadata=TRUE) is used in rirods instead.

Finally, the rirods equivalent of iCommand iquest is iquery(), with the same GenQuery expression as main argument and a few compatible arguments. Given a query variable such as “SELECT DATA_NAME, DATA_CHECKSUM WHERE COLL_NAME LIKE ‘/tempZone/home/rods%’”:

rirods	`iCommands`
`iquery(query)`	`iquest "$query"`
`iquery(query, case_sensitive = FALSE)`	`iquest uppercase "$query"`
`iquery(query, distinct = FALSE)`	`iquest no-distinct "$query"`

To learn more about imeta() and iquery(), see vignette("metadata").

Mariana Montes

2024-03-16

Working directory and collections

Creating and deleting collections or data objects

Getting and putting data

Metadata and querying