Hello, can data be transferred directly between two directories within Isilon (now PowerScale) without using an intermediary system? Our system administrator typically sets up a jump host, mounts both shares, and uses rsync to migrate the data, but that process seems unnecessarily complex. With administrative privileges, is there a more efficient way to handle this transfer within the system? Can this be done directly from the console? If so, are there any reasons to avoid this approach?" Thanks!
I do believe this can be done with admin privs on the system. We do this within our oneFS deployment on the isilon, but I don’t think we have very complicated permissions structures in place. Have you had any luck trying this out? I’m curious to know if you run into issues or if the transfer takes very long.
Thanks, Roy. No, we didn’t try that because our IT team said they weren’t able to copy directly—they could only use rsync through a separate host with both directories mounted.
Breaking this into Copy vs Move
For a Copy:
Assuming that the Isilon cluster is licensed for SyncIQ they could create a job to do the copy directly from all the Isilon nodes to all the Isilon nodes (or a subset of nodes based on if they choose to limit the SyncIQ job to a particular network pool) What is important to note is the copy brings along all the permissions so there may be a need afterwards to adjust them for the target location.
While the Isilon file system can be seen from the Isilon Nodes themselves they tend to be under powered to run a normal cli copy command directly on them and even if you did run it on one it wouldn’t scale out to multiple Isilon nodes automatically like SyncIQ does.
For a Move:
If it is within a single quota then using an nfs client will easily allow you to run mv between the two locations assuming the account doing it has rights to both locations. If you need to move between two different shares and/or quotas it gets a bit more complicated. Firstly to do a “free” move you need it to exist within the same export/share so you may need to mount higher up the tree. Typically we would have a data management node for the sysadmins that mounted /ifs/ rather than individual sub directories. That fixes the everything appearing within the same “share” to the client. The next bit is you can’t move between active quotas. To address this you have to remove the quotas that would apply to the data you want to move and the target location you want to send it to then put them back after the move. Our storage admin had a script that would use the Isilon API to pull the current quota on both the source and target, remove the quotas, perform the move command, then put back the same quotas. This was possible because we only used directory based quotas and there were only ever one quota applying to a directory tree. But the catch here is when the quota is put back on a new quota scan has to be performed so depending on how many files and if your Isilon Cluster has filesystem meta data on SSD it could take a while to finish. Same permissions updates may be needed if they need to be different in the new location.
Afterthought: The move might be able to be run on one of the Isilon Nodes directly but as of the last time we had Isilon on the floor a mv couldn’t be done through the API.
So depending on if they have the feature license and how complicated the quota structure is it may be technically possible but functionally impossible without degrading the system for normal operations.