No edit summary
No edit summary
 
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page discusses what happens when you use more disk space than has been allotted to your account (your quota) and how to reduce your disk usage. You may have been pointed here through an automatic email notification you are running over your quote, or through our support. This page is also useful if you want to reduce your disk usage even if you're not using more than your quota. You can check your current disk usage as well as your quota on https://ucc.ulyssis.org/quota.  
__TOC__
This page discusses what happens when you use more disk space than has been allotted to your account (your quota) and how to reduce your disk usage. You may have been pointed here through an automatic email notification you are running over your quota, or through our support. This page is also useful if you want to reduce your disk usage even if you're not using more than your quota. You can check your current disk usage as well as your quota on https://ucc.ulyssis.org/quota.  


This page first discusses what happens when you use more disk space than is available to your account type (so you therefore go over your quota). This is followed by a detailed manual on how to use the command line tool ''ncdu'' to look for the files and folders that may be causing trouble within your account. Finally, there's a list of examples of common causes for large disk space usage.  
This page first discusses what happens when you use more disk space than is available to your account type (so you therefore go over your quota). This is followed by a detailed manual on how to use the command line tool ''ncdu'' to look for the files and folders that may be causing trouble within your account. Finally, there's a list of examples of common causes for large disk space usage.  
Line 5: Line 6:
== What happens when I go over my quota? ==
== What happens when I go over my quota? ==


If you are using more disk space than the allowed quota for your account type for more than seven consecutive days (the so called grace period), you will no longer be able to write any additional files or extend existing files. You will probably no longer be able to edit websites on your account or it might even stop working outright. If you haven't configured a forwarder for your <code>''username''@ulyssis.org</code> email address, you will lose emails sent to this address. The grace period of seven days is intended for you to have time to reduce your disk space. You will receive an email every day reminding you that you are using too much disk space. In order to prevent impact on other users in cases where disk usage within an account suddenly and substantially surges, an additional quota slightly higher than your normal quota is imposed, independent of any grace period.
If you are using more disk space than the allowed quota for your account type for more than seven consecutive days (the so-called grace period), you will no longer be able to write any additional files or extend existing files. You will probably no longer be able to edit websites on your account or it might even stop working outright. If you haven't configured a forwarder for your <code>''username''@ulyssis.org</code> email address, you will lose emails sent to this address. The grace period of seven days is intended for you to have time to reduce your disk space. You will receive an email every day reminding you that you are using too much disk space. In order to prevent impact on other users in cases where disk usage within an account suddenly and substantially surges, an additional quota slightly higher than your normal quota is imposed, independent of any grace period.


== Identifying which files & folders use the most disk space  ==
== Identifying which files & folders use the most disk space  ==
Line 18: Line 19:
=== Common sources of unnecessary disk usage ===
=== Common sources of unnecessary disk usage ===


* Deleted emails (or emails in general): if you have deleted a substantial amount of emails (or some emails with large attachments) and they still exist in the ''Trash'' folder of your ''@ulyssis.org'' email account, they might take op a large amount of disk space. In order to actually delete them you will gave to empty this ''Trash'' folder. You can do this on https://webmail.ulyssis.org or by deleting the folder ''Maildir/.Trash'' using ncdu (see previous section).
* Your ULYSSIS mailbox: Depending on your account settings, your ''@ulyssis.org'' email account may be used as a forwarder or an inbox. When set to ''Inbox'' on https://ucc.ulyssis.org/mail, email will be saved to the ''Maildir'' folder within your account. A substantial amount of email, especially with large attachments, may grow to quite a significant size. You can verify the contents of your mailbox easily by logging in on https://webmail.ulyssis.org using your ULYSSIS account credentials. You can then remove those emails that are no longer useful, and don't forget to empty the trash. To remove the entire mailbox, you can simply remove the Maildir folder from your account (using ncdu, Cyberduck, or another tool). To empty the trash manually, you can remove the ''Maildir/.Trash'' folder. If you no longer wish to use you mailbox, don't forget to change the configuration on https://ucc.ulyssis.org/mail to forward your messages, since you may receive relevant information about your account through this address.
* Old websites: if you have older websites on your account that you are no longer using, it might be worth removing them in their entirety. If the deleted website uses a database (this will always be the case if your using WordPress, MediaWiki or another popular CMS), then you can delete that as well through our control panel: https://ucc.ulyssis.org. If this website had an associated (sub)domain, please send us an email to have it removed.
* Old websites: if you have older websites on your account that you are no longer using, it might be worth removing them in their entirety. If the deleted website uses a database (this will always be the case if your using WordPress, MediaWiki or another popular CMS), then you should also delete those through our control panel: https://ucc.ulyssis.org (either the MySQL or PostgreSQL sections). If this website had an associated (sub)domain, please send us an email to have it removed. Also keep in mind that old websites may pose a security risk, so it's never good to leave them lying about.
* Unused images: in the case of many CMS's (WordPress, Drupal, ...), removing an image from a post will not delete it. For example, in the case of WordPress it will still be available through the [https://wordpress.com/support/media/deleting-files/ Media Library]. In order to reclaim the disk space used by the image, you will have to delete it there as well.
* Unused images: in the case of many CMSes (WordPress, Drupal, ...), removing an image from a post will not delete it. For example, in the case of WordPress it will still be available through the Media Library. You can easily find these [https://wordpress.com/support/media/2/#unattached-files unattached files]. In order to reclaim the disk space used by the image, you will have to [https://wordpress.com/support/trash/#permanently-deleting-an-upload delete] it there as well. Similarly, MediaWiki offers a list of ''unused files'' as a special page available in the list of ''Special Pages''. Keep in mind that WordPress or MediaWiki might not always be aware an image or other file is used when it has not been added through its interfaces or through a third party plugin.
* Large images: images are often several MBs in size and often have a large resolution. While this is great for editing, printing, ..., such large sizes are not all that useful for inclusion in a webpage. They can make your side load slower and use more disk space without a significant increase in quality. You should therefore consider resizing images before uploading them or installing a plugin that reduces their size on upload. However, do watch out for plugins that keep the original image around as this will not reduce but increase your disk usage. For a tutorial on how to resize images, you can take a look at https://wordpress.com/support/media/image-optimization/. This tutorial is geared towards WordPress, but is also applicable in general. If you are using MediaWiki, we have written a plugin that reduces the size of various type of files. You can find more details on [[Setting up MediaWiki#Extensions|Setting up MediaWiki - Extensions]].
* Large images: images and especially pictures are often several MBs in size and often have a large resolution. While this is great for editing and printing, such large sizes are not all that useful for inclusion in a webpage. They can make your site load slower and use more disk space without a significant increase in quality. You should therefore consider resizing images before uploading them or installing a plugin, module or extension that reduces their size on upload. However, make sure to watch out for plugins that keep the original image around as this will not reduce but increase your disk usage. For a tutorial on how to resize images, you can take a look at https://wordpress.com/support/media/image-optimization/. This tutorial is geared towards WordPress, but is also applicable in general. If you are using MediaWiki, we have written a plugin that reduces the size of various types of files. You can find more details on [[Setting up MediaWiki#Extensions|Setting up MediaWiki - Extensions]].
* Backups: ULYSSIS makes daily backups of all account data, including databases, and keeps them for at least a month. It is therefore not necessary to make automated backups of your own, for example with plugins like [https://nl.wordpress.org/plugins/updraftplus/ Updraft] in the case of WordPress. Although it is a good practice to make backups whenever you make changes to your site (for example, before performing updates), it is not necessary to keep them after you have verified that your site still works.
* Backups: ULYSSIS makes daily backups of all account data, including databases, and keeps them for at least a month. It is therefore not necessary to make automated backups of your own, for example with plugins such as [https://nl.wordpress.org/plugins/updraftplus/ Updraft] for WordPress. Although it is a good practice to make backups whenever you are making large changes to your site, it is not necessary to keep them after you have verified that your site works correctly.
* Anything missing on this list? Feel free to contact us on <code>ulyssis@ulyssis.org</code> with your suggestions.
[[Category:Files]]
[[Category:Files]]
__FORCETOC__
[[Category:Account]]
[[Category:Account]]

Latest revision as of 02:03, 4 August 2021

This page discusses what happens when you use more disk space than has been allotted to your account (your quota) and how to reduce your disk usage. You may have been pointed here through an automatic email notification you are running over your quota, or through our support. This page is also useful if you want to reduce your disk usage even if you're not using more than your quota. You can check your current disk usage as well as your quota on https://ucc.ulyssis.org/quota.

This page first discusses what happens when you use more disk space than is available to your account type (so you therefore go over your quota). This is followed by a detailed manual on how to use the command line tool ncdu to look for the files and folders that may be causing trouble within your account. Finally, there's a list of examples of common causes for large disk space usage.

What happens when I go over my quota?

If you are using more disk space than the allowed quota for your account type for more than seven consecutive days (the so-called grace period), you will no longer be able to write any additional files or extend existing files. You will probably no longer be able to edit websites on your account or it might even stop working outright. If you haven't configured a forwarder for your username@ulyssis.org email address, you will lose emails sent to this address. The grace period of seven days is intended for you to have time to reduce your disk space. You will receive an email every day reminding you that you are using too much disk space. In order to prevent impact on other users in cases where disk usage within an account suddenly and substantially surges, an additional quota slightly higher than your normal quota is imposed, independent of any grace period.

Identifying which files & folders use the most disk space

It is not always easy to immediately identify what is taking up space on your account. Sadly, there are no dependable cross-platform tools to help with this. Instead, we will be using the command line tool ncdu, which runs directly on one of our shell servers. You do not need to be proficient with command line interfaces in order to use this tool, the step-by-step manual below will guide you through the process. However, should you get stuck, do not hesitate to contact us at ulyssis@ulyssis.org.

  • Connect through SSH to one of the shell servers: https://docs.ulyssis.org/Accessing_shell_servers_over_SSH
  • Type in ncdu and press enter. You will then see that ncdu is scanning your account.
  • Once ncdu has finished scanning, it will display which files or folders located directly in your home directory are using the most disk space. When the filename stars with / it is a folder. The results are ordered by size, so it makes most sense to start with the top items.
  • You can select different files/folders using the up and down arrow keys, navigate the directory structure using either the enter and backspace keys or the left and right arrows to move in and out of folders, quit ncdu by pressing q and delete a file or folder with d. Be careful to not delete files you still need as you will need to contact us in order to restore them from a backup.
  • You can check your progress on https://ucc.ulyssis.org/quota. The percentage and values displayed are always current and accurate, there is no delay.
    Disk usage ncdu.png

Common sources of unnecessary disk usage

  • Your ULYSSIS mailbox: Depending on your account settings, your @ulyssis.org email account may be used as a forwarder or an inbox. When set to Inbox on https://ucc.ulyssis.org/mail, email will be saved to the Maildir folder within your account. A substantial amount of email, especially with large attachments, may grow to quite a significant size. You can verify the contents of your mailbox easily by logging in on https://webmail.ulyssis.org using your ULYSSIS account credentials. You can then remove those emails that are no longer useful, and don't forget to empty the trash. To remove the entire mailbox, you can simply remove the Maildir folder from your account (using ncdu, Cyberduck, or another tool). To empty the trash manually, you can remove the Maildir/.Trash folder. If you no longer wish to use you mailbox, don't forget to change the configuration on https://ucc.ulyssis.org/mail to forward your messages, since you may receive relevant information about your account through this address.
  • Old websites: if you have older websites on your account that you are no longer using, it might be worth removing them in their entirety. If the deleted website uses a database (this will always be the case if your using WordPress, MediaWiki or another popular CMS), then you should also delete those through our control panel: https://ucc.ulyssis.org (either the MySQL or PostgreSQL sections). If this website had an associated (sub)domain, please send us an email to have it removed. Also keep in mind that old websites may pose a security risk, so it's never good to leave them lying about.
  • Unused images: in the case of many CMSes (WordPress, Drupal, ...), removing an image from a post will not delete it. For example, in the case of WordPress it will still be available through the Media Library. You can easily find these unattached files. In order to reclaim the disk space used by the image, you will have to delete it there as well. Similarly, MediaWiki offers a list of unused files as a special page available in the list of Special Pages. Keep in mind that WordPress or MediaWiki might not always be aware an image or other file is used when it has not been added through its interfaces or through a third party plugin.
  • Large images: images and especially pictures are often several MBs in size and often have a large resolution. While this is great for editing and printing, such large sizes are not all that useful for inclusion in a webpage. They can make your site load slower and use more disk space without a significant increase in quality. You should therefore consider resizing images before uploading them or installing a plugin, module or extension that reduces their size on upload. However, make sure to watch out for plugins that keep the original image around as this will not reduce but increase your disk usage. For a tutorial on how to resize images, you can take a look at https://wordpress.com/support/media/image-optimization/. This tutorial is geared towards WordPress, but is also applicable in general. If you are using MediaWiki, we have written a plugin that reduces the size of various types of files. You can find more details on Setting up MediaWiki - Extensions.
  • Backups: ULYSSIS makes daily backups of all account data, including databases, and keeps them for at least a month. It is therefore not necessary to make automated backups of your own, for example with plugins such as Updraft for WordPress. Although it is a good practice to make backups whenever you are making large changes to your site, it is not necessary to keep them after you have verified that your site works correctly.
  • Anything missing on this list? Feel free to contact us on ulyssis@ulyssis.org with your suggestions.