Storage

Options

When working with huge data files, the question of where to store your input/output files becomes an issue. Here is a list of what is available.

Name/Location	Local/Shared	Use	Advantage	Disadvantage
/tmp	local⁺	Fast local storage on the cluster nodes	Very Fast	Only local; very temporary
/tmpdata/< username >*	local	Fast local storage only on spinoza and schroedinger	Very Fast	Only local; temporary, see policy
/scratch/< username >	shared	Fast shared storage on all servers	Available on all servers	Temporary, see policy
tier2 e.g. /g/tier2/< groupname >/	shared	Cheap permanent storage	Cheap and safe (no backup though!)	Not accessible on the cluster; Slow
tier1 e.g. /g/< groupname >/	shared	Backed-up shared storage	Available on all servers; Backed-up	Expensive; Not as fast as /scratch or /tmpdata

* if /tmpdata/< username > does not exists, try creating it. Remember to use your unix account or ask GBCS.

⁺Local meaning that the /tmp on schroedinger is not the same as on spinoza or any other node.

On the GB servers, Spinoza and Schroedinger, all of the above are available. Tier2 is however not accessible from the cluster nodes.

/tmp is exclusively reserved for real temporary data and is cleaned up automatically.
/tmpdata on spinoza and schroedinger is used to store a copy of your data locally with the underlying goal to speed up computing and reduce the I/O load due to NFS access.
- Please use /tmpdata/< username > . Files and directories not matching this format are deleted on a regular basis.
- Cleanup your data regularly.
- /tmpdata/< alumni > directories are deleted without notice after a 6 month grace period from the time the user left EMBL.
/scratch is cleaned every day automatically by the IT. If the overall usage is above 90% then everything older than 45 days (approx. 6 weeks) is deleted.