We have a large (~ 10 PB) GPFS File Storage on our system. Some projects have hundreds of terabytes of files stored in their projects. We would like to make it easier for the project Principal Investigators to better understand and manage their space. It is possible that some data (possibly created by a former member of the team or a student) might occupy a large amount of space but have not been accessed for many years. This information would allow PI to archive or (possibly) delete these directories. We explored some products (like Starfish) and did not find them suitable for our system. Has anyone explored this problem and found a good solution?
Have you explored IRODS or GUFI? This article might be helpful for you: User-Friendly Data Management for Scientific Computing Users.