Robinhood Policy Engine is a multi-purpose tool for managing the content of large filesystems. It can audit filesystem content, perform accounting, remove old unused files according to admin-defined policies, show customizable alerts based on file properties, backup data to external storage, and more. It has advanced capabilities for Lustre filesystems. It leverages OST usage, and lists or purges files per OST, with policy criteria based on pools and OST index. It can also process MDT changelogs with Lustre v2. Originally developped for HPC, it has been designed to perform all of its tasks in parallel, so it is particularly adapted for running on large filesystems with millions of entries and petabytes of data. But you can nonetheless take advantage of all of its features for managing smaller filesystems.
ClusterShell is a set of tools and an event-based Python library to execute commands on local or remote cluster nodes in parallel. The framework also provides advanced methods for handling node sets and node groups to ease and improve administration of large compute clusters or server farms. Three convenient command line utilities, clush, clubak, and nodeset, allow traditional shell scripts to benefit some useful features offered by the library.