When working with huge data files, the question of where to store your input/output files becomes an issue. Here is a list of what is available.
|/tmp||local+||Fast local storage on the cluster nodes||Very Fast||Only local; very temporary|
|/tmpdata/< username >*||local||Fast local storage only on spinoza and schroedinger||Very Fast||Only local; temporary, see policy|
|/scratch/< username >||shared||Fast shared storage on all servers||Available on all servers||Temporary, see policy|
|tier2 e.g. /g/tier2/< groupname >/||shared||Cheap permanent storage||Cheap and safe (no backup though!)||Not accessible on the cluster; Slow|
|tier1 e.g. /g/< groupname >/||shared||Backed-up shared storage||Available on all servers; Backed-up||Expensive; Not as fast as /scratch or /tmpdata|
* if /tmpdata/< username > does not exists, try creating it. Remember to use your unix account or ask GBCS.
+Local meaning that the /tmp on schroedinger is not the same as on spinoza or any other node.
On the GB servers, Spinoza and Schroedinger, all of the above are available. Tier2 is however not accessible from the cluster nodes.
- /tmp is exclusively reserved for real temporary data and is cleaned up automatically.
- /tmpdata on spinoza and schroedinger is used to store a copy of your data locally with the underlying goal to speed up computing and reduce the I/O load due to NFS access.
- Please use /tmpdata/< username > . Files and directories not matching this format are deleted on a regular basis.
- Cleanup your data regularly.
- /tmpdata/< alumni > directories are deleted without notice after a 6 month grace period from the time the user left EMBL.
- /scratch is cleaned every day automatically by the IT. If the overall usage is above 90% then everything older than 45 days (approx. 6 weeks) is deleted.
See also New at GB