Pawsey Supercomputing Research Centre Status

Investigating: Currently, Data Portal and pshell users may be experiencing unusually long wait times downloading files from banksia. This will sometimes end in errors (eg timeout or content not available) however the banksia scheduler will still eventually bring the files back online and you will be able to download when trying again a day or so after the initial attempt. We are currently investigating this with the vendor as a potential scheduler issue.

Investigating: We have resolved most of the instability between Mediaflux and the tape storage system. Some minor issues remain which we are continuing to monitor and work on.

Investigating: Work continues with two Vendors to get this resolved. We will be making and testing some recommended steps today to see if this can be completely resolved.

Investigating: A setting was adjusted on the Mediaflux server and this has improved success rate for obtaining files however some issues still remain which we are still investigating with the vendor.

Investigating: We are aware of an issue that may be affecting some users attempting file transfers with Mediaflux (data portal and pshell.) It is currently being investigated.

Resolved: Writes have been re-enabled for the 4 OSTs that were previously 100% full. /scratch is now back to it's normal configuration.

Monitoring: Bulk deletion of older files from /scratch (see the policy at https://pawsey.atlassian.net/wiki/display/US//Filesystem+Policies) has increased the free capacity of the flash pool to over 100 TB. Individual OSTs within this pool are still between 90 and 95% full and researchers are requested to delete any unneeded files from /scratch to prevent jobs failing when they cannot write output.

Investigating: The /scratch filesystem used by setonix is composed of two pools - a high performance flash component, and a much larger but slower disk component. The flash pool is presently 98% full with some individual OSTs having only a few tens of GB free. While there is still plenty of capacity (over 3PB) free on the disk pool, users may see jobs fail with write errors, especially if they are overriding the default striping applied by Pawsey.