ULYSSIS documentation - User contributions [en]

Using (Fast)CGI for non-PHP websites

2023-09-11T19:13:17Z

Thomasd: /* Python and Django */

You're not stuck with PHP if you want to make a dynamic website. You can use all sorts of frameworks and programming languages with ULYSSIS, as long as it has CGI or FastCGI support.

If you want to use a certain programming language or framework, and you can't get it to work, don't hesitate to contact us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org]. We will of course not write any code for you, but we can give you some pointers or directions.

==CGI==

CGI is a simple way to create a dynamic website. We use Apache's [http://httpd.apache.org/docs/2.4/mod/mod_cgid.html mod_cgid] to provide support for CGI. Note, however, that CGI is inefficient and
in general we recommend you to use a framework together with FastCGI. There are many frameworks, ranging from very simple and lightweight, to more complex and feature rich.

If, for example, you want Apache to interpret all files with the extension <code>.cgi</code> as CGI scripts, use the following <tt>.htaccess</tt> file (to be placed in your <code>www</code> folder):
<pre>Options +ExecCGI
AddHandler cgi-script .cgi</pre>

Here's an example of a very simple Python CGI script (called <code>hello.cgi</code>):
<pre>#!/usr/bin/env python

print 'Content-Type: text/plain\n'
print 'Hello world!'</pre>

CGI scripts need to be executable, otherwise they won't work:
<pre>chmod +x hello.cgi</pre>

==FastCGI==

We use Apache's [http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html mod_fcgid] to provide support for FastCGI.

In order to use FastCGI, you will generally need a starter script specific to your programming languae, framework or application. Just do a web search for "''keyword'' mod_fcgid", where ''keyword'' is the programming language, framework or application you're planning to use, and you will often find good instructions. We've included some examples below

You'll want to treat this starter script as a <code>mod_fcgid</code> script, and usually, you'll want to redirect everything to this script, so you can use an <code>.htaccess</code> file that looks like this:
<syntaxhighlight lang="apache">Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ starter.fcgi/$1 [QSA,L]</syntaxhighlight>
Replace ''starter.fcgi'' with the name of your starter script. Don't forget to make your starter script executable:
<pre>chmod +x starter.fcgi</pre>
The starter script must be located in the same folder as the <code>.htaccess</code> file.

{{warning|Warning!|Do not place sensitive files inside the web accessible folder. Make sure they are located in a folder outside the document root, or they will be downloadable through the webserver.}}
It is adviced to only place the fcgi starter script, <code>.htaccess</code> and static files (images, stylesheets) inside the document root.

=== Startup time and persistency ===

It is important to keep in mind that our Apache webworkers will only run your start script when a request is made. As soon as that first request has been received, it will try and keep a few instances of your application around for future use. This of course affects how you restart your application after changes (see below), but also means that applications with a longer start up time (for example a language such as Python combined with a larger framework) will experience an initial slow request. This slow startup is obviously not a factor for faster languages such as Haskell, C++ or Go.

Sadly, Apache is unable to have FastCGI processes persist beyond a reload or restart. This means in practice that every night around 6h25 when the logs are rotated, which requires Apache to be reloaded, all running fcgi processes will be gracefully terminated. The same thing happens when we make changes to the Apache configuration when a new website or user is added.

For those who have a very high startup time and don't generate enough traffic to not experience issues with this behaviour, it's possible to use a [[Managing_Cron_jobs | cronjob]] to visit the website every night at 6h30 or 7h, or even hourly.

===Restarting your application===

If you've changed your application, you will need to restart it for these changes to take effect. However, because you're editing the application on a different server than where it is running, you do not have access to the running process. You'll have to create some sort of mechanism to restart the process.

In the Django and Go examples below, the starter script has been written so that when it is changed, the application is automatically restarted within 1 second. The same mechanism can be applied to other Python sites. You can change the modification date without actually changing the contents of the starter script by using <code>touch</code>:
<pre>touch starter.fcgi</pre>
Replace <code>starter.fcgi</code> with the name of your starter script.

=== Examples ===
==== Python and Django ====

If you want to make a website using Python and [https://www.djangoproject.com/ Django], we recommend that you use a virtual environment. You can use the following steps to get up and running with Django on your ULYSSIS hosting account.

<ol>
<li>[[Accessing shell servers over SSH|Log in to a shell server]]</li>
<li>Create a new virtualenv, you can create one in <code>~/.venv</code>, for example:
<pre>python3 -m venv ~/.venv</pre>
</li>
<li>Activate this virtualenv:
<pre>. ~/.venv/bin/activate</pre>
</li>
<li>
Install Django and flup for FastCGI inside of this virtualenv:
<pre>pip install django flup6</pre>
</li>
<li>
Create a new Django project:
<pre>django-admin startproject mysite</pre>
Replace ''mysite'' with your project's desired name.
</li>
<li>
Edit the allowed hosts in your Django project's settings, in <code>mysite/mysite/settings.py</code>. Look for the line <code>ALLOWED_HOSTS = []</code> and replace it with:
<pre>
ALLOWED_HOSTS = [ 'username.ulyssis.be', 'username.studentenweb.org' ]
</pre>
Use your ULYSSIS username in place of ''username''.
</li>
<li>
Put an <code>.htaccess</code> file in your site's directory, e.g. the <code>www</code> folder:
<pre>Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]</pre>
</li>
<li>
Create the starter script (replace <code>user</code> with org if you are an organization, and <code>username</code> with your ULYSSIS username):
<syntaxhighlight lang="python">#!/home/user/username/.venv/bin/python

import sys, os, os.path
from threading import Thread

this_file = os.path.realpath(__file__)

site_dir = '/home/user/username/mysite'
sys.path.insert(0, site_dir)
os.chdir(site_dir)

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

def stat_thread():
import time, os, signal
start_mtime = os.stat(this_file).st_mtime
while True:
cur_mtime = os.stat(this_file).st_mtime
if cur_mtime != start_mtime:
os.kill(os.getpid(), signal.SIGTERM)
time.sleep(1)

Thread(target=stat_thread).start()

from django.core.servers.basehttp import get_internal_wsgi_application
from flup.server.fcgi import WSGIServer
WSGIServer(get_internal_wsgi_application(), debug=False).run()
</syntaxhighlight>
Call this script <code>mysite.fcgi</code>.
</li>
<li>
Make the starter script executable:
<pre>chmod +x mysite.fcgi</pre>
</li>
<li>
Go to ''username''.ulyssis.be (or whatever URL you chose to use), and it should show you the default "Welcome to Django" page.
</li>
</ol>

This guide was based on, and you can find more information in the official Django documentation:
* [https://docs.djangoproject.com/en/stable/intro/tutorial01/ The Django tutorial]
* [https://flask.palletsprojects.com/en/1.1.x/deploying/fastcgi/ Flask documentation for FastCGI]

==== Go ====
To make a website or web application in Go and host it on your ULYSSIS hosting account, you can use the standard library ''fcgi'' package.

<ol>
<li> [[Accessing shell servers over SSH|Log in to a shell server]]</li>
<li> Put an <code>.htaccess</code> file in your site's directory (e.g. the <code>www</code> folder) with a reference to your executable. Here we've chosen <code>mysite.fcgi</code>, but you can adapt this to your liking.
<pre>Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]</pre>
</li>
<li> Write your code, and compile the binary to <code>mysite.fcgi</code>. Below you can find a basic example of how to register two handlers (of which one is the fallback), then start the FastCGI client, and finally have a function (which is called as a goroutine) to check if the binary has been replaced so old processes don't stick around.
<syntaxhighlight lang="go">package main

import (
"fmt"
"net/http"
"net/http/fcgi"
"os"
"path/filepath"
"time"
)

func uri_interesting(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
fmt.Fprintln(w, "<h1>Oh, seems like this link isn't that interesting after all</h1>")
fmt.Fprintln(w, "<p>Back to the <a href=\"/\">Homepage</a></p>")
}

func uri_default(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
w.WriteHeader(http.StatusNotFound)
fmt.Fprintln(w, "<h1>This page doesn't exist</h1>")
fmt.Fprintln(w, "<p>Back to the <a href=\"/\">Homepage</a></p>")
} else {
w.WriteHeader(http.StatusOK)
fmt.Fprintln(w, "<h1>Welcome to this testwebsite!</h1>")
fmt.Fprintln(w, "<p>This website is written in Go but doesn't really do much.")
fmt.Fprintln(w, "Feel free to visit this <a href=\"/interesting/link\">Interesting link</a> or this <a href=\"/non-existent/link\">non-existent link</a>.</p>")
}
}

func main() {
go check_selfreplacement()
http.HandleFunc("/interesting/link", uri_interesting)
http.HandleFunc("/", uri_default)
if err := fcgi.Serve(nil, nil); err != nil {
panic(err)
}
}

func check_selfreplacement() {
fcgi_location, _ := os.Executable()
fcgi_location, _ = filepath.EvalSymlinks(fcgi_location)
start_stat, _ := os.Stat(fcgi_location)
for {
current_stat, _ := os.Stat(fcgi_location)
if start_stat.ModTime() != current_stat.ModTime() {
os.Exit(0)
}
time.Sleep(1 * time.Second)
}
}
</syntaxhighlight>
If you want to give this example a try, save it as <code>mysite.fcgi.go</code> and then use <code>go build mysite.fcgi.go</code> to compile it.
</li>
<li>Visit your website to verify everything works</li>
</ol>

[[Category:Webserver]]

Using (Fast)CGI for non-PHP websites

2023-09-11T19:11:52Z

Thomasd: /* Python and Django */

You're not stuck with PHP if you want to make a dynamic website. You can use all sorts of frameworks and programming languages with ULYSSIS, as long as it has CGI or FastCGI support.

If you want to use a certain programming language or framework, and you can't get it to work, don't hesitate to contact us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org]. We will of course not write any code for you, but we can give you some pointers or directions.

==CGI==

CGI is a simple way to create a dynamic website. We use Apache's [http://httpd.apache.org/docs/2.4/mod/mod_cgid.html mod_cgid] to provide support for CGI. Note, however, that CGI is inefficient and
in general we recommend you to use a framework together with FastCGI. There are many frameworks, ranging from very simple and lightweight, to more complex and feature rich.

If, for example, you want Apache to interpret all files with the extension <code>.cgi</code> as CGI scripts, use the following <tt>.htaccess</tt> file (to be placed in your <code>www</code> folder):
<pre>Options +ExecCGI
AddHandler cgi-script .cgi</pre>

Here's an example of a very simple Python CGI script (called <code>hello.cgi</code>):
<pre>#!/usr/bin/env python

print 'Content-Type: text/plain\n'
print 'Hello world!'</pre>

CGI scripts need to be executable, otherwise they won't work:
<pre>chmod +x hello.cgi</pre>

==FastCGI==

We use Apache's [http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html mod_fcgid] to provide support for FastCGI.

In order to use FastCGI, you will generally need a starter script specific to your programming languae, framework or application. Just do a web search for "''keyword'' mod_fcgid", where ''keyword'' is the programming language, framework or application you're planning to use, and you will often find good instructions. We've included some examples below

You'll want to treat this starter script as a <code>mod_fcgid</code> script, and usually, you'll want to redirect everything to this script, so you can use an <code>.htaccess</code> file that looks like this:
<syntaxhighlight lang="apache">Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ starter.fcgi/$1 [QSA,L]</syntaxhighlight>
Replace ''starter.fcgi'' with the name of your starter script. Don't forget to make your starter script executable:
<pre>chmod +x starter.fcgi</pre>
The starter script must be located in the same folder as the <code>.htaccess</code> file.

{{warning|Warning!|Do not place sensitive files inside the web accessible folder. Make sure they are located in a folder outside the document root, or they will be downloadable through the webserver.}}
It is adviced to only place the fcgi starter script, <code>.htaccess</code> and static files (images, stylesheets) inside the document root.

=== Startup time and persistency ===

It is important to keep in mind that our Apache webworkers will only run your start script when a request is made. As soon as that first request has been received, it will try and keep a few instances of your application around for future use. This of course affects how you restart your application after changes (see below), but also means that applications with a longer start up time (for example a language such as Python combined with a larger framework) will experience an initial slow request. This slow startup is obviously not a factor for faster languages such as Haskell, C++ or Go.

Sadly, Apache is unable to have FastCGI processes persist beyond a reload or restart. This means in practice that every night around 6h25 when the logs are rotated, which requires Apache to be reloaded, all running fcgi processes will be gracefully terminated. The same thing happens when we make changes to the Apache configuration when a new website or user is added.

For those who have a very high startup time and don't generate enough traffic to not experience issues with this behaviour, it's possible to use a [[Managing_Cron_jobs | cronjob]] to visit the website every night at 6h30 or 7h, or even hourly.

===Restarting your application===

If you've changed your application, you will need to restart it for these changes to take effect. However, because you're editing the application on a different server than where it is running, you do not have access to the running process. You'll have to create some sort of mechanism to restart the process.

In the Django and Go examples below, the starter script has been written so that when it is changed, the application is automatically restarted within 1 second. The same mechanism can be applied to other Python sites. You can change the modification date without actually changing the contents of the starter script by using <code>touch</code>:
<pre>touch starter.fcgi</pre>
Replace <code>starter.fcgi</code> with the name of your starter script.

=== Examples ===
==== Python and Django ====

If you want to make a website using Python and [https://www.djangoproject.com/ Django], we recommend that you use a virtual environment. You can use the following steps to get up and running with Django on your ULYSSIS hosting account.

<ol>
<li>[[Accessing shell servers over SSH|Log in to a shell server]]</li>
<li>Create a new virtualenv, you can create one in <code>~/.venv</code>, for example:
<pre>python3 -m venv ~/.venv</pre>
</li>
<li>Activate this virtualenv:
<pre>. ~/.venv/bin/activate</pre>
</li>
<li>
Install Django and flup for FastCGI inside of this virtualenv:
<pre>pip install django flup6</pre>
</li>
<li>
Create a new Django project:
<pre>django-admin startproject mysite</pre>
Replace ''mysite'' with your project's desired name.
</li>
<li>
Edit the allowed hosts in your Django project's settings, in <code>mysite/mysite/settings.py</code>. Look for the line <code>ALLOWED_HOSTS = []</code> and replace it with:
<pre>
ALLOWED_HOSTS = [ 'username.ulyssis.be', 'username.studentenweb.org' ]
</pre>
</li>
<li>
Put an <code>.htaccess</code> file in your site's directory, e.g. the <code>www</code> folder:
<pre>Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]</pre>
</li>
<li>
Create the starter script (replace <code>user</code> with org if you are an organization, and <code>username</code> with your ULYSSIS username):
<syntaxhighlight lang="python">#!/home/user/username/.venv/bin/python

import sys, os, os.path
from threading import Thread

this_file = os.path.realpath(__file__)

site_dir = '/home/user/username/mysite'
sys.path.insert(0, site_dir)
os.chdir(site_dir)

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

def stat_thread():
import time, os, signal
start_mtime = os.stat(this_file).st_mtime
while True:
cur_mtime = os.stat(this_file).st_mtime
if cur_mtime != start_mtime:
os.kill(os.getpid(), signal.SIGTERM)
time.sleep(1)

Thread(target=stat_thread).start()

from django.core.servers.basehttp import get_internal_wsgi_application
from flup.server.fcgi import WSGIServer
WSGIServer(get_internal_wsgi_application(), debug=False).run()
</syntaxhighlight>
Call this script <code>mysite.fcgi</code>.
</li>
<li>
Make the starter script executable:
<pre>chmod +x mysite.fcgi</pre>
</li>
<li>
Go to ''username''.ulyssis.be (or whatever URL you chose to use), and it should show you the default "Welcome to Django" page.
</li>
</ol>

This guide was based on, and you can find more information in the official Django documentation:
* [https://docs.djangoproject.com/en/stable/intro/tutorial01/ The Django tutorial]
* [https://flask.palletsprojects.com/en/1.1.x/deploying/fastcgi/ Flask documentation for FastCGI]

==== Go ====
To make a website or web application in Go and host it on your ULYSSIS hosting account, you can use the standard library ''fcgi'' package.

<ol>
<li> [[Accessing shell servers over SSH|Log in to a shell server]]</li>
<li> Put an <code>.htaccess</code> file in your site's directory (e.g. the <code>www</code> folder) with a reference to your executable. Here we've chosen <code>mysite.fcgi</code>, but you can adapt this to your liking.
<pre>Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]</pre>
</li>
<li> Write your code, and compile the binary to <code>mysite.fcgi</code>. Below you can find a basic example of how to register two handlers (of which one is the fallback), then start the FastCGI client, and finally have a function (which is called as a goroutine) to check if the binary has been replaced so old processes don't stick around.
<syntaxhighlight lang="go">package main

import (
"fmt"
"net/http"
"net/http/fcgi"
"os"
"path/filepath"
"time"
)

func uri_interesting(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
fmt.Fprintln(w, "<h1>Oh, seems like this link isn't that interesting after all</h1>")
fmt.Fprintln(w, "<p>Back to the <a href=\"/\">Homepage</a></p>")
}

func uri_default(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
w.WriteHeader(http.StatusNotFound)
fmt.Fprintln(w, "<h1>This page doesn't exist</h1>")
fmt.Fprintln(w, "<p>Back to the <a href=\"/\">Homepage</a></p>")
} else {
w.WriteHeader(http.StatusOK)
fmt.Fprintln(w, "<h1>Welcome to this testwebsite!</h1>")
fmt.Fprintln(w, "<p>This website is written in Go but doesn't really do much.")
fmt.Fprintln(w, "Feel free to visit this <a href=\"/interesting/link\">Interesting link</a> or this <a href=\"/non-existent/link\">non-existent link</a>.</p>")
}
}

func main() {
go check_selfreplacement()
http.HandleFunc("/interesting/link", uri_interesting)
http.HandleFunc("/", uri_default)
if err := fcgi.Serve(nil, nil); err != nil {
panic(err)
}
}

func check_selfreplacement() {
fcgi_location, _ := os.Executable()
fcgi_location, _ = filepath.EvalSymlinks(fcgi_location)
start_stat, _ := os.Stat(fcgi_location)
for {
current_stat, _ := os.Stat(fcgi_location)
if start_stat.ModTime() != current_stat.ModTime() {
os.Exit(0)
}
time.Sleep(1 * time.Second)
}
}
</syntaxhighlight>
If you want to give this example a try, save it as <code>mysite.fcgi.go</code> and then use <code>go build mysite.fcgi.go</code> to compile it.
</li>
<li>Visit your website to verify everything works</li>
</ol>

[[Category:Webserver]]

Webserver changes summer 2022

2022-09-05T23:35:29Z

Thomasd:

This page lists the changes to the webservers of ULYSSIS in the summer of 2022, and how you can prepare for it. If any of these
instructions are not clear to you, or if you have some more questions about the changes, '''don't hesitate to e-mail us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org]'''.

== When are the changes planned? ==

These changes are planned in the weekend of 17 and 18 September.

== What will change? ==

We will upgrade the Ubuntu release on our servers from 20.04 LTS to 22.04 LTS. This will include newer version of many services. An overview of the most important version changes can be found below. For other services/packages, you can always use the [https://packages.ubuntu.com/ Ubuntu Package Search]. The distribution name for Ubuntu 22.04 LTS is "jammy".

* PHP will be updated from 7.4 to 8.1
* PostgreSQL will be updated from 12 to 14
* MariaDB (MySQL) will be updated from 10.3.22 to 10.6.7
* Python will be updated from 3.8 to 3.10

If you host your own PHP website, you might want to check out these guides:
* [https://www.php.net/manual/en/migration80.php Migrating from PHP 7.4.x to PHP 8.0.x]
* [https://www.php.net/manual/en/migration81.php Migrating from PHP 8.0.x to PHP 8.1.x]

Note that any existing Python virtual environments in use will need to be recreated in order to function properly.

Webserver changes summer 2022

2022-09-05T23:34:31Z

Thomasd:

This page lists the changes to the webservers of ULYSSIS in the summer of 2022, and how you can prepare for it. If any of these
instructions are not clear to you, or if you have some more questions about the changes, '''don't hesitate to e-mail us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org]'''.

== When are the changes planned? ==

These changes are planned in the weekend of 17 and 18 September.

== What will change? ==

We will upgrade the Ubuntu release on our servers from 20.04 LTS to 22.04 LTS. This will include newer version of many services. An overview of the most important version changes can be found below. For other services/packages, you can always use the [https://packages.ubuntu.com/ Ubuntu Package Search]. The distribution name for Ubuntu 22.04 LTS is "jammy".

* PHP will be updated from 7.4 to 8.1. If you host your own PHP website, you might want to check out these guides:
** [https://www.php.net/manual/en/migration80.php Migrating from PHP 7.4.x to PHP 8.0.x]
** [https://www.php.net/manual/en/migration81.php Migrating from PHP 8.0.x to PHP 8.1.x]
* PostgreSQL will be updated from 12 to 14
* MariaDB (MySQL) will be updated from 10.3.22 to 10.6.7
* Python will be updated from 3.8 to 3.10

Note that any existing Python virtual environments in use will need to be recreated in order to function properly.

Webserver changes summer 2022

2022-08-17T15:36:05Z

Thomasd: Created page with "This page lists the changes to the webservers of ULYSSIS in the summer of 2022, and how you can prepare for it. If any of these instructions are not clear to you, or if you ha..."

This page lists the changes to the webservers of ULYSSIS in the summer of 2022, and how you can prepare for it. If any of these
instructions are not clear to you, or if you have some more questions about the changes, '''don't hesitate to e-mail us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org]'''.

== When are the changes planned? ==

todo

== What will change? ==

We will upgrade the Ubuntu release on our servers from 20.04 LTS to 22.04 LTS. This will include newer version of many services. An overview of the most important version changes can be found below. For other services/packages, you can always use the [https://packages.ubuntu.com/ Ubuntu Package Search]. The distribution name for Ubuntu 22.04 LTS is "jammy".

* PHP will be updated from 7.4 to 8.1
* PostgreSQL will be updated from 12 to 14
* MariaDB (MySQL) will be updated from 10.3.22 to 10.6.7
* Python will be updated from 3.8 to 3.10

Note that any existing Python virtual environments in use will need to be recreated in order to function properly.

Getting Apache logs

2021-10-09T13:40:18Z

Thomasd:

You can find all your Apache logs (like <code>access.log</code> and <code>error.log</code>) on all of our shell servers in the directory <code>/var/log/apache_user/''username''</code>. For more information on how to access your files, please visit [[Accessing your files]].

==Using Cyberduck (graphical interface)==

You can access the log files using an SFTP client like Cyberduck. After logging in to one of our shell servers as per [[Accessing your files]], click on "Go" on the top bar and then click "Go to Folder...":

[[File:Getting Apache Logs - Cyberduck 1.png]]

Then enter <code>/var/log/apache_user/''username''</code> as path name (replace ''username'' with your own username):

[[File:Getting Apache Logs - Cyberduck 2.png]]

After pressing "Go", you will see a directory for each of your websites, containing their Apache logs.

[[File:Getting Apache Logs - Cyberduck 3.png]]

You can view a log file by selecting it and pressing the "Edit" button.

[[File:Getting Apache Logs - Cyberduck Edit button.png]]

After a few days, logs will be compressed into a <code>bz2</code> file. You can download such a file and open it with [https://www.7-zip.org/ 7-Zip].

==Using the command line==

You can also access your logs by [[Accessing shell servers over SSH|logging in to one of our shell servers over SSH]] and navigating to the correct directory:
username@ssh1:~$ cd /var/log/apache_user/username

username@ssh1:/var/log/apache_user/username$ ls
username.ulyssis.be

username@ssh1:/var/log/apache_user/username$ cd username.ulyssis.be

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ ls
access-2014-05-07.log error-2014-05-07.log

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ tail error-2014-05-07.log
[Wed May 07 01:27:14 2014] [error] [client 10.0.0.1] File does not exist: /home/user/username/www/favicon.ico

If you can't find your username inside of <code>/var/log/apache_user</code>, don't worry. If you enter it with <code>cd username</code>, it will automatically appear.

[[Category:Webserver]]

File:Getting Apache Logs - Cyberduck Edit button.png

2021-10-09T13:20:17Z

Thomasd:

Log file overview with edit button

Overview

2021-09-07T21:21:09Z

Thomasd: /* CMSs */

<div style="background:#f9f9f9;border:1px solid #ddd;font-size:95%;padding:1.5em;">
Welcome to our documentation website. Many of the common procedures and issues our users experience are described here. If you still encounter problems we haven't covered or you just can't seem to get things to work, even though you followed one of our manuals, feel free to contact us on ulyssis@ulyssis.org for personal support.</div>

<div>
<div style="width:45%;float:left;padding-right:10px;">
== Account ==
* [[Registering a new account]]
* [[Renewing your account]]
* [[Transferring your account]]
* [[Resetting your password]]
* [[Reducing disk usage|Help, my account uses too much disk space]]
* [[Why do I have to pay 0.00 euro and how?]]
</div><div style="width:45%;float:left">
== Files ==
* [[Accessing your files]]
* [[Transferring files over SFTP]]
* [[Making Backups]]
* [[Secure file permissions]]
* [[Reducing disk usage]]
</div>
<div style="clear:both;"></div>
</div>

<div>
<div style="width:45%;float:left;padding-right:10px;">

== Webserver ==
* [[Webserver changes summer 2020]]
* [[Using your webspace]]
* [[Using (Fast)CGI for non-PHP websites]]
* [[Getting Apache logs]]
* [[Managing PHP errors]]
* [[Setting PHP options]]
* [[Sending email from websites]]
* [[Adding domain names|Adding domain names (.be, .com, .org, ...)]]
* [[Getting SSL/TLS|Getting SSL/TLS (HTTPS security)]]
* [[Shibboleth|Shibboleth (Centrale KU Leuven Login)]]
* [[Claiming a port]]
* [[ULYSSIS security measures]]
</div><div style="width:45%;float:left">

== Mail ==
* [[Introduction to ULYSSIS email]]
* [[Mailbox]]
* [[Forwarders]]
* [[Using a forwarder as an alias]]
* [[Add an alias in Gmail]]
* [[Add an alias in Hotmail/Outlook]]
* [[Add an alias in Thunderbird]]
* [[Fetch your KU Leuven email into another email address]]
* [[Spam]]
* [[ULYSSIS security measures]]
</div>
<div style="clear:both;"></div>
</div>

<div>
<div style="width:45%;float:left;padding-right:10px;">

== CMSs ==
* [[Software Version Checker]]
* [[Setting up WordPress]]
* [[Setting up Drupal]]
* [[Setting up Joomla]]
* [[Setting up MediaWiki]]
* [[Updating WordPress]]
* [[Updating MediaWiki]]
</div><div style="width:45%;float:left">

== Security & anti-spam ==
* [[Secure file permissions]]
* [[Software Version Checker]]
* [[Preventing spam on Wordpress]]
* [[Preventing spam on Joomla]]
* [[Preventing spam on Drupal]]
* [[Preventing spam on MediaWiki]]
* [[Securing MediaWiki using Centrale KU Leuven Login]]
* [[ULYSSIS security measures]]
</div>
<div style="clear:both;"></div>
</div>

<div>
<div style="width:45%;float:left;padding-right:10px;">

== Databases ==
* [[Using PostgreSQL]]
* [[Using MySQL]]
* [[Using PHPMyAdmin]]
* [[Using PHPPgAdmin]]
* [[Using Adminer]]
* [[Making Backups]]

</div><div style="width:45%;float:left">

== Shell ==
* [[Accessing your files]]
* [[Transferring files over SFTP]]
* [[Accessing shell servers over SSH]]
* [[ULYSSIS public IRC]]
* [[Useful Linux Commands]]
* [[Managing Cron jobs]]
* [[Claiming a port]]
* [[Installing packages]]
</div>
<div style="clear:both;"></div>
</div>

<div>
<div style="width:45%;float:left;padding-right:10px;">

== Versioning ==
* [[GitLab]]
</div><div style="width:45%;float:left">

== ULYSSIS Public Services ==
* [[ULYSSIS public IRC]]
</div>
<div style="clear:both;"></div>

<div>
<div style="width:45%;float:left;padding-right:10px;">
== Tutorials ==
* [[Using the kulemt package]]
* [[Exporting your KU Leuven class schedule and calendar]]
</div>
<div style="clear:both;"></div>
</div>

__NOTOC__

Updating MediaWiki

2021-09-07T21:09:50Z

Thomasd: /* Finalizing the update */

The MediaWiki project provides an official [https://www.mediawiki.org/wiki/Manual:Upgrading wiki page] which explains the procedure to update a wiki. Unfortunately, their instructions are complex, and the page contains a lot of information that is outdated or irrelevant for our users. To make it easier for our users to update, this documentation page aims to be more accessible and easier to understand. However, this page is not an authoritative source on the subject. If the instructions on this page are unclear at any point, you should always refer back to the official instructions.

{{info|In this guide, we assume you are familiar with the files on your ULYSSIS account. If you don't know how to access these files, please read [[Accessing your files]] first.}}

== Downloading the right version ==
To start updating MediaWiki, you will need to download the archive file of the version you want to update to. If you did not receive an email from our [[Software Version Checker]], follow the instructions in the next subsection. Otherwise, you can skip to the subsection [[#Downloading the right version using the Software Version Checker | Downloading the right version using the Software Version Checker]].

=== Downloading the right version based on your wiki ===
You can find your current MediaWiki version by going to the <code>Special:Version</code> page on your wiki (simply paste this in the search box) and looking for the "MediaWiki" version under "Installed software". For example, for this wiki, you can find the current version at [[Special:Version]]. At the time of writing, it looks like this:

[[File:Installed software.png|center|frame]]

A table of all recent MediaWiki versions can be found on [https://en.wikipedia.org/wiki/MediaWiki_version_history wikipedia]. Currently supported versions are indicated by a green or yellow color in the first column. To determine which version you want to download, follow these steps:
* If the ''branch'' of your version ('''the first two numbers''', like <code>1.xx</code>) is currently supported (green or yellow), you should choose the latest version for this branch. For example, for version 1.35.3, the branch is 1.35.
* If this branch is not supported anymore, you should choose the most recent LTS branch. This is indicated by '''(LTS)''' in the second column. '''Make sure this LTS branch is not older than your current branch!'''
* If the most recent LTS branch is older than your current branch, you should choose the newest supported branch (green).

Now, click on the link of the branch you want to download. This will redirect you to a page with information about this version. The first paragraph on this page contains a link to <code>mediawiki-1.xx.yy.tar.gz</code>. Download this archive and save it somewhere on your computer.

=== Downloading the right version using the Software Version Checker ===
The email you received will contain a link to the correct MediaWiki version. This link looks something like:
https://releases.wikimedia.org/mediawiki/1.xx/mediawiki-core-1.xx.yy.tar.gz

Simply click this link to download the archive and save it somewhere on your computer.

== Renaming the old installation ==
An important step in the update process is to rename the old installation directory. This way, you can simply copy your data from the old installation directory to the new installation directory during the update. Additionally, you can easily restore your old wiki if something goes wrong.

The easiest way to do this, is to '''rename''' your wiki location (the directory containing your <code>LocalSettings.php</code> configuration file) to a new directory. If you don't know how to access your files on your ULYSSIS account, refer to [[Accessing your files]] for easy instructions. For example, if <code>LocalSettings.php</code> is stored in <code>www/wiki</code>, you should '''rename''' the wiki folder to <code>wiki_old</code>.

== Installing the new files ==
Now, you will need to upload the <code>mediawiki-1.xx.yy.tar.gz</code> file you downloaded in step 1 next to the old installation directory. For example, if your old installation directory is located in <code>www/wiki_old</code>, upload <code>mediawiki-1.xx.yy.tar.gz</code> to <code>www/</code>.

After uploading, you can extract the archive file on the server by using the Cyberduck "Expand Archive" feature. Simply right click the archive and click "Expand Archive".

[[File:Cyberduck expand.png|thumb|center]]

This will create a directory named <code>mediawiki-1.xx.yy</code>. Rename this directory to the original name of your wiki directory on your ULYSSIS account. For example, if the original name was <code>wiki</code>, rename this directory from <code>mediawiki-1.xx.yy</code> to <code>wiki</code>. This can be done using Cyberduck by right clicking the directory and clicking "Rename".

[[File:Cyberduck rename.png|thumb|center]]

Finally, you need to move certain other other files and directories from the '''old''' directory to the '''new''' directory. The easiest way to do this using Cyberduck is by going up a directory (so if your wiki is in <code>www/wiki</code>, go to <code>www</code>), and drag-and-dropping the necessary files and directories. For example, moving <code>LocalSettings.php</code> by drag-and-dropping it from <code>wiki_old</code> into <code>wiki</code>:

[[File:Cyberduck wiki move 1.png|500px]]

[[File:Cyberduck wiki move 2.png|500px]]

Although the exact files you need to move are different depending on your wiki, here are some suggestions from the [https://www.mediawiki.org/wiki/Manual:Upgrading#Other_files official instructions]:
* <code>LocalSettings.php</code>, which contains your old configuration settings.
* The <code>images/</code> directory, containing the uploaded files to the wiki. Because the new wiki directory already contains an (empty) <code>images</code> directory, you need to delete that one first by right-clicking on it and pressing "Delete". Make sure to only delete the <code>images</code> directory in the '''new''' wiki directory. Then you can move the old <code>images</code> directory into the new wiki directory.
* Custom extensions from within the <code>extensions/</code> directory. Be careful not to overwrite the default extensions that are bundled with the new MediaWiki version.
* Custom skins from within the <code>skins/</code> directory. Be careful not to overwrite the default skins that are bundled with the new MediaWiki version.
* Any <code>.htaccess</code> file, if present. '''Make sure you can view hidden files'''; to enable this for Cyberduck, you can look at [[Accessing_your_files#Viewing_hidden_files|Accessing your files]].

== Updating extensions ==
If you use any extensions that are not bundled with MediaWiki by default, you should update them too. A list of bundled extensions can be found at https://www.mediawiki.org/wiki/Bundled_extensions_and_skins, these extensions will be updated automatically. However, for example, you might have the ULYSSIS extensions [[Securing MediaWiki using Centrale KU Leuven Login|MediaWikiShibboleth]] or [https://github.com/ULYSSIS-KUL/CompressUploads CompressUploads] installed. As with MediaWiki itself, you can find the versions of your installed extensions on the <code>Special:Version</code> page, under "Installed extensions". Extensions will often link to a website where you can download the latest versions. For example, for this wiki, you can find the installed extensions at [[Special:Version]]. At the time of writing, it looks like this:

[[File:Installed extensions.png|center|frame]]

Because the update instructions are different for each extension, you will have to refer to the update instructions of the extension you want to update. However, most extensions follow the same template as the MediaWiki upgrade itself:
* Downloading the latest version: make sure to download the correct update for your new MediaWiki version.
* Making a backup of the extension: you should already have a backup in the directory you created in step 2.
* Extracting the archive files: similar to the MediaWiki tar.gz archive files, you might have to copy the new extension files to a directory in the new <code>extension/</code> directory.

== Finalizing the update ==
The final step in the update process is to update the database structure of your wiki. Your MediaWiki installation contains a script, called <code>update.php</code>, to perform the necessary database upgrades. This script can be executed using the Cyberduck "Send Command" feature.

[[File:Cyberduck send command.png|thumb|center]]

Enter the following command in the pop-up box (of course, replace <code><wiki installation location></code> with the location of your new installation):

php <wiki installation location>/maintenance/update.php

For example, if your wiki is located at <code>www/wiki</code>, the command should be as follows:

[[File:Cyberduck update.png|center|750px]]

After pressing "Send", the command will be executed on the server. If everything went well, you should see a lot of output, ending with something like:

[[File:Cyberduck update output.png|center|750px]]

If you get the error <code>Could not open input file</code>, you might have misspelled the installation location. Double check the path to make sure everything is correct.

If you encounter any other errors while executing the script, you could try taking a look at the [https://www.mediawiki.org/wiki/Manual:Upgrading#Command_line_2 official instructions]. If this does not resolve your problems, feel free to send us an email. We will try to assist you in completing the update.

Congratulations! You successfully updated MediaWiki. Still, there are two more important steps you must perform:
* Test your new MediaWiki installation: make sure all basic functionality (viewing, editing pages, file upload) works and all your extensions function properly.
* Delete the old installation: you should remove the previously created <code>www/wiki_old</code> directory, using the command line or through a GUI. A simple GUI explanation can be found on [[Accessing_your_files#Creating_and_Deleting_files_and_folders|Accessing your files]].

== If something goes wrong ==

If something goes wrong during the update so your wiki doesn't work anymore, you can revert the process to go back to your old wiki. If you already moved files or directories such as <code>LocalSettings.php</code> or <code>images</code> from <code>wiki_old</code> to <code>wiki</code>, move them back into <code>wiki_old</code>. Then delete <code>wiki</code> and rename <code>wiki_old</code> back to <code>wiki</code>. Your old wiki is then restored so you can retry the update process. If you still run into any trouble, feel free to email us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org].

Updating MediaWiki

2021-08-31T23:18:32Z

Thomasd: /* Installing the new files */

Updating MediaWiki

2021-08-31T23:16:31Z

Thomasd:

The MediaWiki project provides an official [https://www.mediawiki.org/wiki/Manual:Upgrading wiki page] which explains the procedure to update a wiki. Unfortunately, their instructions are complex, and the page contains a lot of information that is outdated or irrelevant for our users. To make it easier for our users to update, this documentation page aims to be more accessible and easier to understand. However, this page is not an authoritative source on the subject. If the instructions on this page are unclear at any point, you should always refer back to the official instructions.

{{info|In this guide, we assume you are familiar with the files on your ULYSSIS account. If you don't know how to access these files, please read [[Accessing your files]] first.}}

== Downloading the right version ==
To start updating MediaWiki, you will need to download the archive file of the version you want to update to. If you did not receive an email from our [[Software Version Checker]], follow the instructions in the next subsection. Otherwise, you can skip to the subsection [[#Downloading the right version using the Software Version Checker | Downloading the right version using the Software Version Checker]].

=== Downloading the right version based on your wiki ===
You can find your current MediaWiki version by going to the <code>Special:Version</code> page on your wiki (simply paste this in the search box) and looking for the "MediaWiki" version under "Installed software". For example, for this wiki, you can find the current version at [[Special:Version]]. At the time of writing, it looks like this:

[[File:Installed software.png|center|frame]]

A table of all recent MediaWiki versions can be found on [https://en.wikipedia.org/wiki/MediaWiki_version_history wikipedia]. Currently supported versions are indicated by a green or yellow color in the first column. To determine which version you want to download, follow these steps:
* If the ''branch'' of your version ('''the first two numbers''', like <code>1.xx</code>) is currently supported (green or yellow), you should choose the latest version for this branch. For example, for version 1.35.3, the branch is 1.35.
* If this branch is not supported anymore, you should choose the most recent LTS branch. This is indicated by '''(LTS)''' in the second column. '''Make sure this LTS branch is not older than your current branch!'''
* If the most recent LTS branch is older than your current branch, you should choose the newest supported branch (green).

Now, click on the link of the branch you want to download. This will redirect you to a page with information about this version. The first paragraph on this page contains a link to <code>mediawiki-1.xx.yy.tar.gz</code>. Download this archive and save it somewhere on your computer.

=== Downloading the right version using the Software Version Checker ===
The email you received will contain a link to the correct MediaWiki version. This link looks something like:
https://releases.wikimedia.org/mediawiki/1.xx/mediawiki-core-1.xx.yy.tar.gz

Simply click this link to download the archive and save it somewhere on your computer.

== Renaming the old installation ==
An important step in the update process is to rename the old installation directory. This way, you can simply copy your data from the old installation directory to the new installation directory during the update. Additionally, you can easily restore your old wiki if something goes wrong.

The easiest way to do this, is to '''rename''' your wiki location (the directory containing your <code>LocalSettings.php</code> configuration file) to a new directory. If you don't know how to access your files on your ULYSSIS account, refer to [[Accessing your files]] for easy instructions. For example, if <code>LocalSettings.php</code> is stored in <code>www/wiki</code>, you should '''rename''' the wiki folder to <code>wiki_old</code>.

== Installing the new files ==
Now, you will need to upload the <code>mediawiki-1.xx.yy.tar.gz</code> file you downloaded in step 1 next to the old installation directory. For example, if your old installation directory is located in <code>www/wiki_old</code>, upload <code>mediawiki-1.xx.yy.tar.gz</code> to <code>www/</code>.

After uploading, you can extract the archive file on the server by using the Cyberduck "Expand Archive" feature. Simply right click the archive and click "Expand Archive".

[[File:Cyberduck expand.png|thumb|center]]

This will create a directory named <code>mediawiki-1.xx.yy</code>. Rename this directory to the original name of your wiki directory on your ULYSSIS account. For example, if the original name was <code>wiki</code>, rename this directory from <code>mediawiki-1.xx.yy</code> to <code>wiki</code>. This can be done using Cyberduck by right clicking the directory and clicking "Rename".

[[File:Cyberduck rename.png|thumb|center]]

Finally, you need to move certain other other files and directories from the '''old''' directory to the '''new''' directory. The easiest way to do this using Cyberduck is by going up a directory (so if your wiki is in <code>www/wiki</code>, go to <code>www</code>), and drag-and-dropping the necessary files and directories. For example, moving <code>LocalSettings.php</code> by drag-and-dropping it from <code>wiki_old</code> into <code>wiki</code>:

[[File:Cyberduck wiki move 1.png|thumb|center]]

[[File:Cyberduck wiki move 2.png|thumb|center]]

Although the exact files you need to move are different depending on your wiki, here are some suggestions from the [https://www.mediawiki.org/wiki/Manual:Upgrading#Other_files official instructions]:
* <code>LocalSettings.php</code>, which contains your old configuration settings.
* The <code>images/</code> directory, containing the uploaded files to the wiki. Because the new wiki directory already contains an (empty) <code>images</code> directory, you need to delete that one first by right-clicking on it and pressing "Delete". Make sure to only delete the <code>images</code> directory in the '''new''' wiki directory. Then you can move the old <code>images</code> directory into the new wiki directory.
* Custom extensions from within the <code>extensions/</code> directory. Be careful not to overwrite the default extensions that are bundled with the new MediaWiki version.
* Custom skins from within the <code>skins/</code> directory. Be careful not to overwrite the default skins that are bundled with the new MediaWiki version.
* Any <code>.htaccess</code> file, if present. '''Make sure you can view hidden files'''; to enable this for Cyberduck, you can look at [[Accessing_your_files#Viewing_hidden_files|Accessing your files]].

== Updating extensions ==
If you use any extensions that are not bundled with MediaWiki by default, you should update them too. A list of bundled extensions can be found at https://www.mediawiki.org/wiki/Bundled_extensions_and_skins, these extensions will be updated automatically. However, for example, you might have the ULYSSIS extensions [[Securing MediaWiki using Centrale KU Leuven Login|MediaWikiShibboleth]] or [https://github.com/ULYSSIS-KUL/CompressUploads CompressUploads] installed. As with MediaWiki itself, you can find the versions of your installed extensions on the <code>Special:Version</code> page, under "Installed extensions". Extensions will often link to a website where you can download the latest versions. For example, for this wiki, you can find the installed extensions at [[Special:Version]]. At the time of writing, it looks like this:

[[File:Installed extensions.png|center|frame]]

Because the update instructions are different for each extension, you will have to refer to the update instructions of the extension you want to update. However, most extensions follow the same template as the MediaWiki upgrade itself:
* Downloading the latest version: make sure to download the correct update for your new MediaWiki version.
* Making a backup of the extension: you should already have a backup in the directory you created in step 2.
* Extracting the archive files: similar to the MediaWiki tar.gz archive files, you might have to copy the new extension files to a directory in the new <code>extension/</code> directory.

== Finalizing the update ==
The final step in the update process is to update the database structure of your wiki. Your MediaWiki installation contains a script, called <code>update.php</code>, to perform the necessary database upgrades. This script can be executed using the Cyberduck "Send Command" feature.

[[File:Cyberduck send command.png|thumb|center]]

Enter the following command in the pop-up box (of course, replace <code><wiki installation location></code> with the location of your new installation):

php <wiki installation location>/maintenance/update.php

For example, if your wiki is located at <code>www/wiki</code>, the command should be as follows:

[[File:Cyberduck update.png|center|750px]]

After pressing "Send", the command will be executed on the server. If everything went well, you should see a lot of output, ending with something like:

[[File:Cyberduck update output.png|center|750px]]

If you get the error <code>Could not open input file</code>, you might have misspelled the installation location. Double check the path to make sure everything is correct.

If you encounter any other errors while executing the script, you could try taking a look at the [https://www.mediawiki.org/wiki/Manual:Upgrading#Command_line_2 official instructions]. If this does not resolve your problems, feel free to send us an email. We will try to assist you in completing the update.

Congratulations! You successfully updated MediaWiki. Still, there are two more important steps you must perform:
* Test your new MediaWiki installation: make sure all basic functionality (viewing, editing pages, file upload) works and all your extensions function properly.
* Delete the old installation: you should remove the previously created <code>www/wiki_old</code> directory, using the command line or through a GUI. A simple GUI explanation can be found on [[Accessing_your_files#Creating_and_Deleting_files_and_folders|Accessing your files]].

File:Cyberduck wiki move 2.png

2021-08-31T23:11:02Z

Thomasd:

Second step in moving files/directories between wiki installations in Cyberduck

File:Cyberduck wiki move 1.png

2021-08-31T23:10:04Z

Thomasd:

First step in moving files/directories between wiki installations in Cyberduck

Getting Apache logs

2021-08-15T20:38:13Z

Thomasd: /* Using Cyberduck (graphical interface) */

You can find all your Apache logs (like <code>access.log</code> and <code>error.log</code>) on all of our shell servers in the directory <code>/var/log/apache_user/''username''</code>. For more information on how to access your files, please visit [[Accessing your files]].

==Using Cyberduck (graphical interface)==

You can access the log files using an SFTP client like Cyberduck. After logging in to one of our shell servers as per [[Accessing your files]], click on "Go" on the top bar and then click "Go to Folder...":

[[File:Getting Apache Logs - Cyberduck 1.png]]

Then enter <code>/var/log/apache_user/''username''</code> as path name (replace ''username'' with your own username):

[[File:Getting Apache Logs - Cyberduck 2.png]]

After pressing "Go", you will see a directory for each of your websites, containing their Apache logs.

[[File:Getting Apache Logs - Cyberduck 3.png]]

After a few days, logs will be compressed into a <code>bz2</code> file.

==Using the command line==

You can also access your logs by [[Accessing shell servers over SSH|logging in to one of our shell servers over SSH]] and navigating to the correct directory:
username@ssh1:~$ cd /var/log/apache_user/username

username@ssh1:/var/log/apache_user/username$ ls
username.ulyssis.be

username@ssh1:/var/log/apache_user/username$ cd username.ulyssis.be

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ ls
access-2014-05-07.log error-2014-05-07.log

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ tail error-2014-05-07.log
[Wed May 07 01:27:14 2014] [error] [client 10.0.0.1] File does not exist: /home/user/username/www/favicon.ico

If you can't find your username inside of <code>/var/log/apache_user</code>, don't worry. If you enter it with <code>cd username</code>, it will automatically appear.

[[Category:Webserver]]

Getting Apache logs

2021-08-15T20:37:54Z

Thomasd:

You can find all your Apache logs (like <code>access.log</code> and <code>error.log</code>) on all of our shell servers in the directory <code>/var/log/apache_user/''username''</code>. For more information on how to access your files, please visit [[Accessing your files]].

==Using Cyberduck (graphical interface)==

You can access the log files using an SFTP client like Cyberduck. After logging in to one of our shell servers as per [[Accessing your files]], click on "Go" on the top bar and then click "Go to Folder...":

[[File:Getting Apache Logs - Cyberduck 1.png]]

Then enter <code>/var/log/apache_user/''username''</code> as path name (replace ''username'' with your own username):

[[File:Getting Apache Logs - Cyberduck 2.png]]

After pressing "Go", you will see a directory for each of your websites, containing their Apache logs.

[[File:Getting Apache Logs - Cyberduck 3.png]]

After a few days, logs will be compressed into a `bz2` file.

==Using the command line==

You can also access your logs by [[Accessing shell servers over SSH|logging in to one of our shell servers over SSH]] and navigating to the correct directory:
username@ssh1:~$ cd /var/log/apache_user/username

username@ssh1:/var/log/apache_user/username$ ls
username.ulyssis.be

username@ssh1:/var/log/apache_user/username$ cd username.ulyssis.be

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ ls
access-2014-05-07.log error-2014-05-07.log

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ tail error-2014-05-07.log
[Wed May 07 01:27:14 2014] [error] [client 10.0.0.1] File does not exist: /home/user/username/www/favicon.ico

If you can't find your username inside of <code>/var/log/apache_user</code>, don't worry. If you enter it with <code>cd username</code>, it will automatically appear.

[[Category:Webserver]]

Reducing disk usage

2021-08-04T01:03:38Z

Thomasd:

__TOC__
This page discusses what happens when you use more disk space than has been allotted to your account (your quota) and how to reduce your disk usage. You may have been pointed here through an automatic email notification you are running over your quota, or through our support. This page is also useful if you want to reduce your disk usage even if you're not using more than your quota. You can check your current disk usage as well as your quota on https://ucc.ulyssis.org/quota.

This page first discusses what happens when you use more disk space than is available to your account type (so you therefore go over your quota). This is followed by a detailed manual on how to use the command line tool ''ncdu'' to look for the files and folders that may be causing trouble within your account. Finally, there's a list of examples of common causes for large disk space usage.

== What happens when I go over my quota? ==

If you are using more disk space than the allowed quota for your account type for more than seven consecutive days (the so-called grace period), you will no longer be able to write any additional files or extend existing files. You will probably no longer be able to edit websites on your account or it might even stop working outright. If you haven't configured a forwarder for your <code>''username''@ulyssis.org</code> email address, you will lose emails sent to this address. The grace period of seven days is intended for you to have time to reduce your disk space. You will receive an email every day reminding you that you are using too much disk space. In order to prevent impact on other users in cases where disk usage within an account suddenly and substantially surges, an additional quota slightly higher than your normal quota is imposed, independent of any grace period.

== Identifying which files & folders use the most disk space ==
It is not always easy to immediately identify what is taking up space on your account. Sadly, there are no dependable cross-platform tools to help with this. Instead, we will be using the command line tool ''ncdu'', which runs directly on one of our shell servers. You do not need to be proficient with command line interfaces in order to use this tool, the step-by-step manual below will guide you through the process. However, should you get stuck, do not hesitate to contact us at <code>ulyssis@ulyssis.org</code>.

* Connect through SSH to one of the shell servers: https://docs.ulyssis.org/Accessing_shell_servers_over_SSH
* Type in <code>ncdu</code> and press enter. You will then see that <code>ncdu</code> is scanning your account.
* Once <code>ncdu</code> has finished scanning, it will display which files or folders located directly in your home directory are using the most disk space. When the filename stars with <code>/</code> it is a folder. The results are ordered by size, so it makes most sense to start with the top items.
* You can select different files/folders using the '''up and down arrow keys''', navigate the directory structure using either the '''enter and backspace''' keys or the '''left and right''' arrows to move in and out of folders, quit ncdu by pressing '''q''' and delete a file or folder with '''d'''. Be careful to not delete files you still need as you will need to contact us in order to restore them from a backup.
* You can check your progress on https://ucc.ulyssis.org/quota. The percentage and values displayed are always current and accurate, there is no delay.[[File:Disk usage ncdu.png|thumb|760x760px]]

=== Common sources of unnecessary disk usage ===

* Your ULYSSIS mailbox: Depending on your account settings, your ''@ulyssis.org'' email account may be used as a forwarder or an inbox. When set to ''Inbox'' on https://ucc.ulyssis.org/mail, email will be saved to the ''Maildir'' folder within your account. A substantial amount of email, especially with large attachments, may grow to quite a significant size. You can verify the contents of your mailbox easily by logging in on https://webmail.ulyssis.org using your ULYSSIS account credentials. You can then remove those emails that are no longer useful, and don't forget to empty the trash. To remove the entire mailbox, you can simply remove the Maildir folder from your account (using ncdu, Cyberduck, or another tool). To empty the trash manually, you can remove the ''Maildir/.Trash'' folder. If you no longer wish to use you mailbox, don't forget to change the configuration on https://ucc.ulyssis.org/mail to forward your messages, since you may receive relevant information about your account through this address.
* Old websites: if you have older websites on your account that you are no longer using, it might be worth removing them in their entirety. If the deleted website uses a database (this will always be the case if your using WordPress, MediaWiki or another popular CMS), then you should also delete those through our control panel: https://ucc.ulyssis.org (either the MySQL or PostgreSQL sections). If this website had an associated (sub)domain, please send us an email to have it removed. Also keep in mind that old websites may pose a security risk, so it's never good to leave them lying about.
* Unused images: in the case of many CMSes (WordPress, Drupal, ...), removing an image from a post will not delete it. For example, in the case of WordPress it will still be available through the Media Library. You can easily find these [https://wordpress.com/support/media/2/#unattached-files unattached files]. In order to reclaim the disk space used by the image, you will have to [https://wordpress.com/support/trash/#permanently-deleting-an-upload delete] it there as well. Similarly, MediaWiki offers a list of ''unused files'' as a special page available in the list of ''Special Pages''. Keep in mind that WordPress or MediaWiki might not always be aware an image or other file is used when it has not been added through its interfaces or through a third party plugin.
* Large images: images and especially pictures are often several MBs in size and often have a large resolution. While this is great for editing and printing, such large sizes are not all that useful for inclusion in a webpage. They can make your site load slower and use more disk space without a significant increase in quality. You should therefore consider resizing images before uploading them or installing a plugin, module or extension that reduces their size on upload. However, make sure to watch out for plugins that keep the original image around as this will not reduce but increase your disk usage. For a tutorial on how to resize images, you can take a look at https://wordpress.com/support/media/image-optimization/. This tutorial is geared towards WordPress, but is also applicable in general. If you are using MediaWiki, we have written a plugin that reduces the size of various types of files. You can find more details on [[Setting up MediaWiki#Extensions|Setting up MediaWiki - Extensions]].
* Backups: ULYSSIS makes daily backups of all account data, including databases, and keeps them for at least a month. It is therefore not necessary to make automated backups of your own, for example with plugins such as [https://nl.wordpress.org/plugins/updraftplus/ Updraft] for WordPress. Although it is a good practice to make backups whenever you are making large changes to your site, it is not necessary to keep them after you have verified that your site works correctly.
* Anything missing on this list? Feel free to contact us on <code>ulyssis@ulyssis.org</code> with your suggestions.
[[Category:Files]]
[[Category:Account]]

Getting Apache logs

2021-08-04T00:54:16Z

Thomasd:

You can find all your Apache logs (like access.log and error.log) on all of our shell servers in the directory <code>/var/log/apache_user/''username''</code>. For more information on how to access your files, please visit [[Accessing your files]].

==Using Cyberduck==

You can access the log files using an SFTP client like Cyberduck. After logging in to one of our shell servers as per [[Accessing your files]], click on "Go" on the top bar and then click "Go to Folder...":

[[File:Getting Apache Logs - Cyberduck 1.png]]

Then enter <code>/var/log/apache_user/''username''</code> as path name (replace ''username'' with your own username):

[[File:Getting Apache Logs - Cyberduck 2.png]]

After pressing "Go", you will see a directory for each of your websites, containing their Apache logs.

[[File:Getting Apache Logs - Cyberduck 3.png]]

==Using the command line==

You can also access your logs by [[Accessing shell servers over SSH|logging in to one of our shell servers over SSH]] and navigating to the correct directory:
username@ssh1:~$ cd /var/log/apache_user/username

username@ssh1:/var/log/apache_user/username$ ls
username.ulyssis.be

username@ssh1:/var/log/apache_user/username$ cd username.ulyssis.be

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ ls
access-2014-05-07.log error-2014-05-07.log

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ tail error-2014-05-07.log
[Wed May 07 01:27:14 2014] [error] [client 10.0.0.1] File does not exist: /home/user/username/www/favicon.ico

If you can't find your username inside of <code>/var/log/apache_user</code>, don't worry. If you enter it with <code>cd username</code>, it will automatically appear.

[[Category:Webserver]]

Getting Apache logs

2021-08-04T00:52:03Z

Thomasd: Filezilla -> Cyberduck, gooi volgorde om

You can find all your Apache logs (like access.log and error.log) on all of our shell servers in the directory <code>/var/log/apache_user/''username''</code>. For more general information on how to access your files, please visit [[Accessing your files]].

==Using Cyberduck==

You can access the log files using an SFTP client like Cyberduck. Click on "Go" on the top bar and then click "Go to Folder...":

[[File:Getting Apache Logs - Cyberduck 1.png]]

Then enter <code>/var/log/apache_user/''username''</code> as path name (replace ''username'' with your own username):

[[File:Getting Apache Logs - Cyberduck 2.png]]

After pressing "Go", you will see a directory for each of your websites, containing their Apache logs.

[[File:Getting Apache Logs - Cyberduck 3.png]]

==Using the command line==

You can also access your logs by [[Accessing shell servers over SSH|logging in to one of our shell servers over SSH]] and navigating to the correct directory:
username@ssh1:~$ cd /var/log/apache_user/username

username@ssh1:/var/log/apache_user/username$ ls
username.ulyssis.be

username@ssh1:/var/log/apache_user/username$ cd username.ulyssis.be

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ ls
access-2014-05-07.log error-2014-05-07.log

username@ssh1:/var/log/apache_user/username/username.ulyssis.be$ tail error-2014-05-07.log
[Wed May 07 01:27:14 2014] [error] [client 10.0.0.1] File does not exist: /home/user/username/www/favicon.ico

If you can't find your username inside of <code>/var/log/apache_user</code>, don't worry. If you enter it with <code>cd username</code>, it will automatically appear.

[[Category:Webserver]]

File:Getting Apache Logs - Cyberduck 2.png

2021-08-04T00:48:46Z

Thomasd: Thomasd uploaded a new version of File:Getting Apache Logs - Cyberduck 2.png

Second Cyberduck screenshot for Getting Apache Logs

File:Getting Apache Logs - Cyberduck 3.png

2021-08-04T00:41:02Z

Thomasd:

Third Cyberduck screenshot for Getting Apache Logs

File:Getting Apache Logs - Cyberduck 2.png

2021-08-04T00:39:18Z

Thomasd:

Second Cyberduck screenshot for Getting Apache Logs

File:Getting Apache Logs - Cyberduck 1.png

2021-08-04T00:36:22Z

Thomasd:

First Cyberduck screenshot for Getting Apache Logs

Setting up MediaWiki

2020-11-09T01:41:48Z

Thomasd: /* Setup your wiki */

This page will go through a basic setup of MediaWiki. It is based on [http://www.mediawiki.org/wiki/Manual:Installing_MediaWiki the official MediaWiki installation wikipage], and it gives a few specifics for setting it up on a hosted account over at ULYSSIS.

This tutorial we'll use a ULYSSIS-account with username "me".

== Putting setup-files in place ==

First we'll have to download the setup file which you can find over at [http://www.mediawiki.org/wiki/Download the official MediaWiki website].

It's an archived file, so you need to extract it first and put the files in a subdirectory of your "document root".
The default is just a www directory in your home folder.

Putting your files in document_root/wiki/ will make your wiki be located at usersname.ulyssis.be/wiki/.

If you prefer, you can do this all graphically using the manual [[Accessing_your_files]].

If you prefer the command line;

<pre>
homeuser@home:~$ ssh me@ssh1.ulyssis.org
enter super secret password:
me@zap:~$ wget http://releases.wikimedia.org/mediawiki/1.23/mediawiki-1.23.6.tar.gz
me@zap:~$ tar xvxf mediawiki-1.23.6.tar.gz
me@zap:~$ mv mediawiki-1.23.6 www/wiki
me@zap:~$ rm -r mediawiki-1.23.6.tar.gz
</pre>

==Setting up database==

Like the [http://www.mediawiki.org/wiki/Manual:Installing_MediaWiki the official MediaWiki installation wiki] tells us, we have three different options to choose from; PostgreSQL, MySQL and SQLite.

It's easy to set a PostgreSQL or MySQL database; just go to the UCC. For this tutorial we will continue with a MySQL database (PostgreSQL installation steps are quite similar).

Let us visit https://ucc.ulyssis.org/mysql; if this is your first time using MySQL you'll need to create a new account by putting in a super secret password.
The wizard will create your MySQL account with the same name as your user-account.
We'll also create a new database with an appropriate name by clicking on 'Add database'. In this tutorial we will choose; "wiki", notice that it will be prepended with our username, so we'll end up with a new database named "me_wiki".

==Setup your wiki==

If all went well, you can access your website at http://username.ulyssis.be/the_folder_you_used and start the setup wizard.
In our example, will surf to http://me.ulyssis.be/wiki.

The wizard will guide you through some configuration options. Most of them our pretty straight-forward.
When in doubt, you can always click the question mark wich will show you some sensible and recommended default...
We'll just discuss the page were the wizard asks you for the database.
In our example;

<pre>
Database host: mysql.ulyssis.org
Database name: me_wiki
// The name of our database; i.e. the name we chose, prepended by our username.
Database table prefix:
// We can leave this blank.
Database user: me
// This is the name of our MySQL-user, which was named after our ULYSSIS-username.
Database password: supersecretpassword
// Please note that this is not your ULYSSIS-password, but the password you used in UCC when creating the MySQL user!
</pre>

==Creating the wiki==

After the setup wizard is complete, you can begin the installation procedure.
If all went well, you'll be greeted by a page which congratulates you with your new installation.

==Finishing up installation==

The setup will offer you a configuration file (containing some settings you specified) that you need to download to your server (more specifically in directory where your MediaWiki-files reside in).

You again can do this graphically, or if you prefer a terminal;

<pre>
homeuser@home:~$ scp path_to_file/LocalSettings.php me@ssh1.ulyssis.org:~/www/wiki
enter super secret password:
</pre>

After this is done, you can finally start using your wiki.

Please refer to [http://www.mediawiki.org/wiki/MediaWiki the original MediaWiki documentation] for a more detailed or advanced information.

== Extending MediaWiki ==

By default, every MediaWiki installation looks and works identical. While that is convenient for users, it can be less than ideal if you need specific functionality or wish to adapt your wiki to your style.

=== Extensions ===

Extensions add functionality to a wiki, such as more fine-grained authentication, extra layout elements for articles or text filters. An overview of what kinds of extensions are possible can be found on [https://www.mediawiki.org/wiki/Manual:Extensions the official manual].

Some noteworthy extensions can be used to prevent spam or unauthorised access. You can find more information on [[Preventing spam on MediaWiki]] and [[Securing MediaWiki using Centrale KU Leuven Login]].

We've also found that users sometimes have problems with high amounts of disk space usage. To help those users out, we've developed a MediaWiki extension that can automatically reduce file size of new uploads. Specifically it's possible to compress PDFs, convert inefficient image formats such as BMP and TIFF to PNG, loslessly compress PNG, lossy compress JPG, resize images and strip EXIF data. You can find more details on https://github.com/ULYSSIS-KUL/CompressUploads

=== Skins ===

Skins are similar to themes, templates or styles within other software. By default MediaWiki uses the well known Vector skin. But other skins are available. Sadly, MediaWiki doesn't maintain a complete list of all available skins and whether they work with recent releases, so it may take a bit of perseverance. General information on Skins is available on the [https://www.mediawiki.org/wiki/Manual:Skins Official Manual]. Besides that there's the [https://www.mediawiki.org/wiki/Manual:Gallery_of_user_styles the user style gallery] and [https://www.mediawiki.org/wiki/Category:All_skins the list of skins].

[[Category:CMSs]]

Using (Fast)CGI for non-PHP websites

2019-11-15T18:37:22Z

Thomasd: verwijder verkeerde "you"

You're not stuck with PHP if you want to make a dynamic website. You can use all sorts of frameworks and programming languages with ULYSSIS, as long as it has CGI or FastCGI support.

If you want to use a certain programming language or framework, and you can't get it to work, don't hesitate to contact us at [mailto:ulyssis@ulyssis.org ulyssis@ulyssis.org]. We will of course not write any code for you, but we can give you some pointers or directions.

==CGI==

CGI is a simple way to create a dynamic website. We use Apache's [http://httpd.apache.org/docs/2.4/mod/mod_cgid.html mod_cgid] to provide support for CGI. Note, however, that CGI is inefficient and
in general we recommend you to use a framework together with FastCGI. There are many frameworks, ranging from very simple and lightweight, to more complex and feature rich.

If, for example, you want Apache to interpret all files with the extension <tt>.cgi</tt> as CGI scripts, use the following <tt>.htaccess</tt> file (to be placed in your <tt>www</tt> folder):
<pre>Options +ExecCGI
AddHandler cgi-script .cgi</pre>

Here's an example of a very simple Python CGI script (called <tt>hello.cgi</tt>):
<pre>#!/usr/bin/env python

print 'Content-Type: text/plain\n'
print 'Hello world!'</pre>

CGI scripts need to be executable, otherwise they won't work:
<pre>chmod +x hello.cgi</pre>

==FastCGI==

We use Apache's [http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html mod_fcgid] to provide support for FastCGI.

In order to use FastCGI, you will generally need a starter script specific to your framework and application. Just do a web search for "''framework'' mod_fcgid", where ''framework'' is the framework you're using, and you will find good instructions. There's a Django example below.

You'll want to treat this starter script as a <tt>mod_fcgid</tt> script, and usually, you'll want to redirect everything to this script, so you can use an <tt>.htaccess</tt> file that looks like this:
<syntaxhighlight lang="apache">Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ starter.fcgi/$1 [QSA,L]</syntaxhighlight>
Replace ''starter.fcgi'' with the name of your starter script. Don't forget to make your starter script executable:
<pre>chmod +x starter.fcgi</pre>
The starter script must be located in the same folder as the <tt>.htaccess</tt> file.

{{warning|Warning!|Do not place sensitive files inside the web accessible folder. Make sure they are located in a folder outside the document root, or they will be downloadable through the webserver.}}
It is adviced to only place the fcgi starter script, <tt>.htaccess</tt> and static files (images, stylesheets) inside the document root.

=== Startup time and persistency ===

It is important to keep in mind that our Apache webworkers will only run your start script when a request is made. As soon as that first request has been received, it will try and keep a few instances of your application around for future use. This of course affects how you restart your application after changes (see below), but also means that applications with a longer start up time (for example a language such as Python combined with a larger framework) will experience an initial slow request. This slow startup is obviously not a factor for faster languages such as Haskell, C++ or Go.

Sadly, Apache is unable to have FastCGI processes persist beyond a reload or restart. This means in practice that every night around 6h25 when the logs are rotated, which requires Apache to be reloaded, all running fcgi processes will be gracefully terminated. The same thing happens when we make changes to the Apache configuration when a new website or user is added.

For those who have a very high startup time and don't generate enough traffic to not experience issues with this behaviour, it's possible to use a [[Managing_Cron_jobs | cronjob]] to visit the website every night at 6h30 or 7h, or even hourly.

===Restarting your application===

If you've changed your application, you will need to restart it for these changes to take effect. However, because you're editing the application on a different server than were it is running, you do not have access to the running process. You'll have to create some sort of mechanism to restart the process.

In the Django example below, the starter script has been written so that when it is changed, the application is automatically restarted within 1 second. The same mechanism can be applied to other Python sites. You can change the modification date without actually changing the contents of the starter script by using <tt>touch</tt>:
<pre>touch starter.fcgi</pre>
Replace ''starter.fcgi'' with the name of your starter script.

===Example: Django===

If you want to make a website using [https://www.djangoproject.com/ Django], we recommend that you use a [http://virtualenv.readthedocs.org/ virtualenv]. This tool is already preinstalled on our shell servers.

You can use the following steps to get up and running with Django at ULYSSIS:

<ol>
<li>[[Accessing shell servers over SSH|Log in to a shell server]]</li>
<li>Create a new virtualenv, you can create one in <tt>~/.venv</tt>, for example:
<pre>virtualenv ~/.venv</pre>
Or, for Python 3:
<pre>virtualenv ~/.venv --python=python3</pre>
</li>
<li>Activate this virtualenv:
<pre>. ~/.venv/bin/activate</pre>
</li>
<li>
Install Django inside of this virtualenv:
<pre>pip install django</pre>
</li>
<li>
Create a new Django project:
<pre>django-admin.py startproject mysite</pre>
Replace ''mysite'' with your project's desired name.
</li>
<li>
Install flup for FastCGI:
<pre>pip install flup6</pre>
</li>
<li>
Put an <tt>.htaccess</tt> file in your site's directory, e.g. the <tt>www</tt> folder:
<pre>Options +ExecCGI
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]</pre>
</li>
<li>
Create the starter script (replace ''user'' with org if you are an organization, and ''username'' with your ULYSSIS username):
<syntaxhighlight lang="python">#!/home/user/username/.venv/bin/python

import sys, os, os.path
from threading import Thread

this_file = os.path.realpath(__file__)

site_dir = '/home/user/username/mysite'
sys.path.insert(0, site_dir)
os.chdir(site_dir)

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

def stat_thread():
import time, os, signal
start_mtime = os.stat(this_file).st_mtime
while True:
cur_mtime = os.stat(this_file).st_mtime
if cur_mtime != start_mtime:
os.kill(os.getpid(), signal.SIGTERM)
time.sleep(1)

Thread(target=stat_thread).start()

from django.core.servers.basehttp import get_internal_wsgi_application
from flup.server.fcgi import WSGIServer
WSGIServer(get_internal_wsgi_application(), debug=False).run()
</syntaxhighlight>
Call this script <tt>mysite.fcgi</tt>.
</li>
<li>
Make the starter script executable:
<pre>chmod +x mysite.fcgi</pre>
</li>
<li>
Go to ''username''.ulyssis.be (or whatever URL you chose to use), and it should show you the default "Welcome to Django" page.
</li>
</ol>

This guide was based on, and you can find more information in the official Django documentation:
* [https://docs.djangoproject.com/en/stable/intro/tutorial01/ The Django tutorial]
* [https://flask.palletsprojects.com/en/1.1.x/deploying/fastcgi/ Flask documentation for FastCGI]

[[Category:Webserver]]