There are various types of servers, but they all consist of a physical computer somewhere. This tutorial shows how to set up a server first with the simplest of physical devices you can have in your home for less than $20 and side-by-side with how to do it in its more abstract – and more common – form, a VPS (virtual private server) you can rent from a hosting service.
The two procedures are very analogous. Seeing them side-by-side helps make concrete what you are actually doing even though sometimes you can’t physically see it.
Server 1. Raspberry Pi Zero W Running Raspberry Pi OS (Debian, a flavor of Linux)
Can be purchased many places, such as Microcenter or Amazon.
Server 2. VPS (Virtual Private Server) Running Ubuntu 20.04 (a flavor of Linux)
I purchased a VPS from Hostwinds. There are various operating systems available. I used Ubuntu 20.04.
Connect to Your Server over a Network (or the Internet)
Connect Via Secure Shell (SSH)
I SSH login using the MobaXterm SSH Client. The server is usually a remote computer to which you do not have physical access, so instead of plugging in a keyboard, mouse and screen, you establish a connection that gives you access to the server’s command prompt. SSH stands for secure shell, meaning the data you transfer between your computer and the server are encrypted. SSH is enabled by default in Ubuntu, but not all Linux flavors.
Connect the Raspberry Pi Zero to your LAN. There is a way to do this with command line but the Raspberry Pi OS connects to WiFi on the initial startup, so just do it on initialization. If you want, to view the saved password with the command line to know generally where WiFi passwords are stored, use
sudo grep psk= /etc/wpa_supplicant/*
which searches the appropriate directory for “psk=” and you see the stored WiFi keys.
Enable SSH. SSH is not enabled by default with Raspberry Pi Zero. Many tutorials tell you to add a blank file to the /boot folder called SSH (no extension at all). Use the command touch ssh, then reboot. If you do this, notice once SSH is enabled then the file gets deleted so you will not see any change other than the SSH login should work.
Or enable SSH by: in the raspberry pi terminal window, enter sudo raspi-config, select Interfacing Options, SSH, enable it.
2. Ubuntu
Must install openssh-server. Use sudo apt install openssh-server. Check status after install with sudo systemctl status ssh.
Find the Raspberry Pi’s own IP address in the command line with ip add or ip addr or ifconfig. The IP will show after wlan0: inet __.__.__.__ Within a LAN, it is often something like 192.168.1.* Alternatively, you can type your router’s IP into your browser and view a list of connected devices. Or, use nmap, sudo apt install nmap and the command sudo nmap -sn 192.168.1.0/24 (without sudo you won’t see all the MAC addresses)
2. VPS
On a VPS, SSH is (should be) enabled by your host to allow the purchaser to access it. You will get a default username and password from the hosting service who established the VPS with its default settings.
In MobaXterm, “Remote host” is where you specify the IP address of your server, available in your Hostwinds account.
The username is “root”
The password is whatever you set in Hostwinds
Optional: Update Operating System
1. Raspberry Pi OS
sudo apt update is the Debian update command (Raspberry Pi OS is based on Debian).
2. VPS
Update Ubuntu (only required if there is a new version of Ubuntu).*
*First, be sure you can login as the non-root super-user before updating Ubuntu as the new install will default to not allow root login. This means if you were planning to just use the existing root user with infinite powers you are now infinitely locked out of your own VPS and have to have your host re-initialize it. Your VPS host changed this setting when it set up your VPS so you can login but when you update the Ubuntu OS, the OS returns to the OS default which is to not allow SSH root login! This is a GREAT example of why to do the user basics as the very first step.
sudo do-release-upgrade
Update the advanced package tool:
sudo apt-get update (without the -get is newer, so I use it)
sudo apt update
Make Website Available Outside Your LAN
1. Raspberry Pi Zero W
Once the Pi server is serving a site on its IP within the LAN, making the site available outside the LAN (on the internet) is as simple as directing site requests that arrive to your internet IP to the Pi server. Sounds complicated, but when a browser looks for a website on the internet, it looks on port 80. Most routers have an option to direct all traffic arriving on a specific port to a specific IP within the LAN. Connect to your network’s router to configure it, usually by entering its IP (often 192.168.1.1) into a browser and logging in with a password you set. You should see an option like this under the advanced settings.
To Do: Establish an SSH Connection with SSH Keys for the Non-Root User
Normally, you generate a public and private key on your local computer then copy the public key to the server along with some settings. Hostwinds has an option in server management to generate the key, download the private key, and install the public key on the server. Reboot required.
Astropy 0.3.0 is what Miniconda 3.5.5 installs. This is a much earlier version than I would have expected. Astropy 0.3 was released in Nov 2013.
Astropy 0.4 was released in Jul 2014. Therefore Astropy 0.3 was the latest version when Miniconda 3.5.5 was released and is probably why Miniconda 3.5.5 installs with Astropy 0.3.
Astropy v1.2 requires Python v2.7 or later and Numpy 1.7.0 or later.
Astropy v1.3: “vectors and coordinates can be reshaped like arrays.”
Astropy v2.0.0 started to implement Python 3 but specifically says does not change functionality with Python 2. However, from 2.0.0 to 2.0.18, Python 2 was gradually phased out.
Astropy v3.0 is the first version that supports only Python 3.
I think I want Astropy v1.3 – unless it won’t run for some reason. Why does Miniconda 3.5.5 install Astropy 0.3.0? Was it the current Astropy at the release time? Answer: yes, it was. Do the newer Astropy versions require newer ISA? At some point, I got a “need ARMv7” error so probably.
However, also unexpectedly, it updates Python from 2.7.7 to 2.7.8. Python 2.7.8 was released in Jul 2014.
A lot of this is about Astropy and Astroplan because that’s what I was doing at the time. The main issue I remember is having to install the correct Raspberry Pi build of certain programs (I don’t remember which exactly now but the information is in here somewhere if I do again).
Miniconda 3.5.5 installs Python 2.7.7. Python 2.7.7 was released in Jun 2014 (same month as Miniconda 3.5.5)
The Miniconda installation instructions specifically say you can leave Python installed do not remove it (see link below under Miniconda).
Astroplan uses the Python package Numpy. Use either the Linux command pip3 list, or python -c "import numpy; print(numpy.__version__)" to find the Numpy version installed.
I have Numpy 1.19.5. Is that because the legacy Miniconda installed it or because of the Python 3.9.2 installed on the Raspberry Pi OS? I think it’s the Python 3.9.2 from the OS.
On my first test with Astroplan (and Astropy) I had a Numpy error that sent me to this link. The correction was sudo apt-get install libatlas-base-dev. I also updated Numpy with pip3 install numpy --upgrade and it upgraded to Numpy 1.21.5
Astropy on a Raspberry Pi Zero W
This is step-by-step how to install Astropy and Astroplan on a Raspberry Pi Zero W (1st gen, not the ‘2’).
sudo passwd (change from default as desired for security)
sudo raspi-config
Set up location, time zone, language, keyboard, Wi-Fi, SSH. Turn off Bluetooth? How?
python --version
>> Python 3.9.2
sudo apt install python3-pip – for the Raspberry Pi, it is better to not use pip because it installs software not compiled for the Pi (the Pi Zero ISA is ARMv6l).
sudo apt install python3-numpy
sudo apt install python3-astropy
Note! Astropy installs this one. Not the rc2 or any of the other ones.
sudo apt install python3-pyerfa (installs with Astropy – I think)
sudo apt install python3-pyyaml (installs with Astropy – I think)
sudo apt install python3-packaging (installs with Astropy – I think)
sudo apt install python3-astroplan
sudo apt install python3-pytz (installs with Astroplan)
The Raspberry Pi Zero W is 32-bit and uses ARMv6 instead of the newer ARMv7. The [biggest? only?] difference is in how the processor handles floating point operations.
The Raspberry Pi Zero 2 W is 64-bit.
The Raspberry Pi OS version 11 – the latest version still – is 32-bit. A 64-bit Raspberry Pi OS is in development but not available yet.
To check the version of Raspberry Pi OS installed, cat /etc/os-release.
Python
Python comes already installed with the Raspberry Pi OS. To verify, type python --version. You could install if necessary with sudo apt install python3.8x.
This required a lot of configuration to work with WSGI, but I finally made it work for the tlom project while working at Revi North and it was very stable in the end. However, Flask isn’t so great because you’re writing html with Python, then serving the html, which javascript is much better at.
sudo apt install libapache2-mod-wsgi-py3
sudo a2enmod wsgi
Disable apache2:
sudo update-rc.d apache2 disable
sudo update-rc.d apache2 enable
sudo service apache2 restart
sudo service apache2 start
sudo service apache2 stop
Serve a Flask App from Ubuntu Apache Server with WSGI
What the following does: download source code, unpack it, make directory ~/.localpython to install into, run the configuration file setting install going to install folder, compile, install compilation, create a virtualenv pointing to the install, switch to the virtualenv to use it:
Note the venv folder stores neither the Python installation nor your code for your project. It is only used to store version information about the Python installation used for your project.
virtualenv instead of venv
I ran into a not-so-obscure reason to use virtualenv instead of venv. If you ever want to serve a Flask app using Apache or some other production server, virtualenv creates a file called activate_this, which Apache can use to run the Flask app in the appropriate Python environment.
getent group or getent group | grep searchsomething
To get the status, including rules being enforced, of the uncomplicated firewall:
sudo ufw status
To list all processes currently running;
ps aux | grep searchsomething
To find a particular file or directory from among all files:
sudo find / -name "searchsomething"
or for directory name search only:
sudo find / -type d -name "searchsomething"
To search for specific text within the files within the current directory:
grep -nr 'searchsomething*'
To show all ports being listened to:
ss -tuln
$ is the normal prompt for commands
# is the system administrator prompt
in the C shell, the prompt ends with %
> is used to send the output to a text file. >> is used to append the output to an existing text file without over-writing.
$VARIABLE_NAME is a variable
apt combines apt-get and apt-cache and it is newer. Use apt.
sudo apt update && sudo apt upgrade -y
To install Python packages, use apt only if you want to install directly on the machine, otherwise use pip. Pip installs modules slower, but installs the latest versions, and most importantly works within a virtualenv. In the special case of Raspberry Pi, use apt because it knows the unique processor architecture and installs the correct builds of modules.
script savedcommands.txt
A .sh file is an executable shell script. # to comment within
./ is used to specify the current working directory especially when running something that could be the same as a system command.
nano is the basic user-friendly text editor.
To view $PATH, echo "${PATH//:/$'\n'}" or just echo $PATH
To edit $PATH for the current session, export PATH='/new/directory:$PATH'
> sends command output to a file. >> appends the output to a file without overwriting existing contents.
ls -al shows all files in directory including hidden
whatis is a brief explanation
man gives a manual
less is like cat, but one page at a time
Useful Linux Commands
At this point, you are using Linux. Either Raspberry Pi OS or Ubuntu are “flavors” of Linux. Having a list of basic commands is helpful:
This step appears complex and does not appear necessary especially this early in the process. Strictly speaking, it is indeed not necessary. You could skip this step and do everything as the root or default user that already exists. However, it is best to do this now because:
to do things in the right order
to highlight an important aspect of the Linux OS: that Linux is very user-specific and permissions-based. This makes Linux less intuitive at first but makes it secure enough to be accessed by many different anonymous people as a server without allowing hackers to access sensitive parts of the server.
to avoid inevitable frustration later. Using Linux you will see “permission denied” errors periodically throughout your experience and you are better off expecting and troubleshooting them than believing that Linux is just annoying. Permissions are built-in to Linux from the ground up and it’s best to work with it rather than try to ignore it.
Logging in as a non-root user is safer. For example, some installations of Ubuntu default to external root login disabled, which means for a remote server you would be locked out if this were set and you don’t have another user to login as.
sudo adduser new_username
usermod -aG sudo new_username
Change the password for the current user as desired with:
passwd
Grant all privileges to the user with the command
sudo visudo
and add a line in the /etc/sudoers file below the root user line: new_username ALL=(ALL:ALL)ALL
This only allows the user to give itself privileges. The user does not have all read/write privileges like the root itself. Log in as the new user through SSH.
