Category Archives: Coding

I love to code interesting projects.

Python Flask App

Deploy Flask App with Apache

This required a lot of configuration to work with WSGI, but I finally made it work for the tlom project while working at Revi North and it was very stable in the end. However, Flask isn’t so great because you’re writing html with Python, then serving the html, which javascript is much better at.

sudo apt install libapache2-mod-wsgi-py3

sudo a2enmod wsgi

Disable apache2:

sudo update-rc.d apache2 disable

sudo update-rc.d apache2 enable

sudo service apache2 restart

sudo service apache2 start

sudo service apache2 stop

Serve a Flask App from Ubuntu Apache Server with WSGI
sudo apt-get install libapache2-mod-wsgi-py3

Put the files you have been using to dev in

/var/www/app_name

Python on Linux

Install a Local Python from Source

sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev

To install dependencies.

What the following does: download source code, unpack it, make directory ~/.localpython to install into, run the configuration file setting install going to install folder, compile, install compilation, create a virtualenv pointing to the install, switch to the virtualenv to use it:

mkdir ~/src

cd ~/src

$ wget http://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz

$ tar -zxvf Python-3.11.0.tgz

$ cd Python-3.11.0

$ mkdir ~/localpython3110

$ ./configure --prefix=$HOME/localpython3110 --enable-optimizations

https://realpython.com/installing-python/#how-to-build-python-from-source-code

$ make &&make altinstall

Set Up venv

$ ~/localpython3110/bin/python3.11 -m venv py3110_venv

$ source py3110_venv/bin/activate

$ sudo apt install python3-pip -y

$ pip install --upgrade pip

$ pip install tk pillow numpy astropy astroplan pandas pytz matplotlib scikit-learn

$ pip list

$ pip -V

$ which python

$ pip install --upgrade pip

$ pip freeze --local > requirements.txt

$ deactivate

$ rm -rf somename_env

$ pip install -r requirements.txt

Note the venv folder stores neither the Python installation nor your code for your project. It is only used to store version information about the Python installation used for your project.

virtualenv instead of venv

I ran into a not-so-obscure reason to use virtualenv instead of venv. If you ever want to serve a Flask app using Apache or some other production server, virtualenv creates a file called activate_this, which Apache can use to run the Flask app in the appropriate Python environment.

sudo apt install python3-pip -y

pip install virtualenv

python3 -m virtualenv py31012_virtualenv

source py31012_virtualenv/bin/activate

pip install --upgrade pip

Install Packages from Local Folder

pip download package1 package2 -d'~/src/python-packages'

pip install package1 package2 -f ~/src/python-packages --no-index

Python Path

To see the path to the current python installation:

python -c 'import os, sys; print(os.path.dirname(sys.executable))'

To see the path variable in Windows (readable format):

PS > $env:path -split ';'

Linux, Get Oriented

To login to a linux machine remotely:

ssh username@ipaddress

To see the version of Linux running:

cat /etc/os-release

printenv lists all environment variables

Print working directory:

pwd

Get IP address:

ifconfig or ifconfig | grep inet

See what ports are open:

netstat

List all users:

less /etc/passwd

List groups a user belongs to:

groups username or id username

List groups with their members:

getent group or getent group | grep searchsomething

To get the status, including rules being enforced, of the uncomplicated firewall:

sudo ufw status

To list all processes currently running;

ps aux | grep searchsomething

To find a particular file or directory from among all files:

sudo find / -name "searchsomething"

or for directory name search only:

sudo find / -type d -name "searchsomething"

To search for specific text within the files within the current directory:

grep -nr 'searchsomething*'

  • $ is the normal prompt for commands
  • # is the system administrator prompt
  • in the C shell, the prompt ends with %
  • > is used to send the output to a text file. >> is used to append the output to an existing text file without over-writing.
  • $VARIABLE_NAME is a variable
  • apt combines apt-get and apt-cache and it is newer. Use apt.
  • sudo apt update && sudo apt upgrade -y
    • To install Python packages, use apt only if you want to install directly on the machine, otherwise use pip. Pip installs modules slower, but installs the latest versions, and most importantly works within a virtualenv. In the special case of Raspberry Pi, use apt because it knows the unique processor architecture and installs the correct builds of modules.
  • script savedcommands.txt
  • A .sh file is an executable shell script. # to comment within
  • ./ is used to specify the current working directory especially when running something that could be the same as a system command.
  • nano is the basic user-friendly text editor.
  • To view $PATH, echo "${PATH//:/$'\n'}" or just echo $PATH
  • To edit $PATH for the current session, export PATH='/new/directory:$PATH'
  • > sends command output to a file. >> appends the output to a file without overwriting existing contents.
  • ls -al shows all files in directory including hidden
  • whatis is a brief explanation
  • man gives a manual
  • less is like cat, but one page at a time

Useful Linux Commands

At this point, you are using Linux. Either Raspberry Pi OS or Ubuntu are “flavors” of Linux. Having a list of basic commands is helpful:

Set up Firewall

sudo ufw allow OpenSSH

sudo ufw allow 22

sudo ufw allow 3306

sudo ufw default deny incoming

sudo ufw default allow outgoing

sudo ufw allow ssh

sudo ufw allow http or sudo ufw allow 80

sudo ufw allow https or sudo ufw allow 443

sudo ufw allow 'apache full'

sudo ufw show added

sudo ufw enable

Create Non-Root Super-User

This step appears complex and does not appear necessary especially this early in the process. Strictly speaking, it is indeed not necessary. You could skip this step and do everything as the root or default user that already exists. However, it is best to do this now because:

  • to do things in the right order
  • to highlight an important aspect of the Linux OS: that Linux is very user-specific and permissions-based. This makes Linux less intuitive at first but makes it secure enough to be accessed by many different anonymous people as a server without allowing hackers to access sensitive parts of the server.
  • to avoid inevitable frustration later. Using Linux you will see “permission denied” errors periodically throughout your experience and you are better off expecting and troubleshooting them than believing that Linux is just annoying. Permissions are built-in to Linux from the ground up and it’s best to work with it rather than try to ignore it.
  • Logging in as a non-root user is safer. For example, some installations of Ubuntu default to external root login disabled, which means for a remote server you would be locked out if this were set and you don’t have another user to login as.

sudo adduser new_username

usermod -aG sudo new_username

Change the password for the current user as desired with:

passwd

Grant all privileges to the user with the command

sudo visudo

and add a line in the /etc/sudoers file below the root user line: new_username ALL=(ALL:ALL)ALL

This only allows the user to give itself privileges. The user does not have all read/write privileges like the root itself. Log in as the new user through SSH.

Non-Root Super-User Gives Itself Read/Write Privileges

This sounds convoluted and it is when you are accustomed to dealing with desktop computers designed for convenience. Linux is designed for security.

Make a new group

sudo addgroup servermanager

and add the new user to the group

sudo adduser new_username servermanager

groups new_username

make the new group the owner of the required directories:

sudo chown -vR :servermanager /var/www/

sudo chown -vR :servermanager /etc/apache2/sites-available/

sudo chown -vR :servermanager /etc/apache2/sites-enabled

then modify the directory permission to be written by the owner group:

sudo chmod -vR g+w /var/www/ (add +x if you want to be able to develop in this directory)

sudo chmod -vR g+w /etc/apache2/sites-available

sudo chmod -vR g+w /etc/apache2/sites-enabled

(Useful Users and Permissions Commands)

Linux has a group and user structure to manage permissions and it is very useful to be able to view the current state:

List all users in the system:

cat /etc/passwd

List all groups on the system:

cat etc/group or less etc/group

or

getent group

for all members of a single group:

getent group group_name

check ownership of a directory, for example:

ls -ld /var/www/

check ownership of a file:

ls -l /var/www/

Find all the files owned by a particular user (may take some time):

sudo find / -user username

Change the active group for the session, possibly not:

newgrp servermanager

delete a group:

sudo groupdel group_name

delete a user (-r removes the user’s directory and mail spool):

sudo userdel -r username

search “linux octal permissions” to understand the numbering system.

Show all currently logged in users on a system:

w

Coding Projects

  1. Astronomical Clock
  2. AstroWideImageMapper
  3. Tag audio files / sheet music images to relate the audio to the location on the sheet music. Kinda like a Karaoke lyrics read-out but advanced and customizable for musicians / music students.
    • As a lifelong music learner, I view sheet music while playing as a crutch that probably hurts musicianship in the long-run. However, sheet music while listening I view as a major enhancement that could especially help kids learn music the same way they learn language, by mass exposure. Following sheet music while listening brings meaning to hours of listening and helps a person’s brain assign value to understanding the notes they hear while they listen. It can help a person “develop their ear.”
  4. “Chess Arena” using camera with software board recognition and Stockfish chess engine to analyze the games for an audience on a screen facing away from the players. Enhance in-person chess.
  5. Scrabble remote using camera with software board recognition.
  6. Fully-connected house with
    • Automated windows up and down.
    • Fully-automated lights.
    • Security system that stores footage on the LAN with fully open-source video storage code. No subscription with high-def storage not reliant on SLOW internet upload limits. (This does not exist because companies make money on security subscriptions).
    • Temperature monitors throughout the house.
  7. OstranderNet. A second internet. There shouldn’t be just one.
    • Limited bandwidth. Start by transmitting chess games.
    • Completely wireless using amateur radio bands.
    • ‘All equipment runs from 12V. Car batteries. Apocalypse-ready.
  8. Done. Wordle Scratchpad. Front-end practice. Model / View / Controller architecture. Python with tkinter is one-install set-up to start coding with powerful language.
  9. Ongoing. Model number decoder et cetera (work project)
  10. Done. Buyouts and drawings search tools (work project)
  11. Secure text app like Signal with:
    • Annual / monthly / daily customizable word limits on group chats (with suggestions and holiday-specific text allowances).
    • Maybe a regular text message app that works with SMS / MMS that auto-replies to groups that go over a certain limit. Spam the spammers.

Git Commands

Terminology

origin =

main =

master =

remote =

Head =

Headless mode =

Modified =

Staged =

Committed =

Connecting

Start the ssh agent:

eval $(ssh-agent)

To generate an SSH key:

ssh-keygen

To add the SSH key to the SSH client for use:

ssh-add id_ed25519_github

Informational Commands

Git can be a little intimidating at first because it has the power to change the files you are working on, so it’s good to start with commands that don’t do anything to get a feel for the situation before you actually do something.

git config --list

Shows useful information about the current repository. Shows the remote.origin.url variable for example.

git config --get remote.origin.url

Shows just the url associated with local git repository.

git config --list --show-origin

Shows ___.

git status

Probably the most common git command.

git diff

To see changes since the last commit. Or

git diff > gitdiff.txt

for a text file of the information.

ls -al ~/.ssh

To check for SSH keys (on Linux only I think).

git log --oneline

Is useful to see all the commits in a row with their commit messages. Or

git log --oneline --decorate --graph --all

for more detailed information.

git log -p filename.py > log.txt

Makes a file with the entire change history of filename. The -p is patch text. This is great for resurrecting “lost portions” of code and replaced the file I used to keep: “unused code.txt”

git branch -v -a

Lists all the branches including remotes, with the current branch highlighted in green plus an asterisk.

git branch --show-current

Shows just the current branch.

git tag -n1

Says to list all the tags and show 1 line of annotation per tag. Default is 0 lines of annotation.

git ls-tree -r trunk --name-only

Initial Set-Up Commands and Connecting to GitHub

Notice above command to check for existing keys.

The first step is establishing an SSH key on the machine to be connected to GitHub then copy the public key to GitHub. This is a pretty good description of how to do that. Notice Git bash in Linux is just the command line of a Linux machine with Git installed while on Windows you have to open the Git bash executable for a Git-specific terminal.

& 'C:\Program Files\Git\bin\sh.exe' --login

to open the Git command line.

git clone git@github.com:rr34/Astro.git

Notice the format of the remote url determines SSH versus https transfer. The above format is for SSH, which is required to commit changes because password identification was disabled for committing changes.

To initialize, navigate to the directory where your code is stored, and:

git init

to initialize git tracking of the directory.

git add

to add files. You can add one at a time to be selective or

? how to add all the files in the directory, then ignore?

Create repository on Github, then start committing to it:

git commit -am 'Initial commit.'

To set the location of the remote repository:

git remote add origin git@github.com:rr34/Astroclock.git

To change the branch name. -m would be --move, -M is --move --force

git branch -M trunk

To make your first push to the remote repository. -u is --set-upstream:

git push -u origin trunk

Daily “Save Your Work” is Commit

This is a pretty good list here, but it lacks informational commands.

git commit -a -m 'commit message here'

git push

Are the daily “save your work” commands.

Renaming a File

git mv old-filename.extension new-filename.extension

To rename a file, use git to rename it instead of renaming it manually. This allows git to track the rename instead of appearing to be “delete and add new.”

Deleting a File

git rm filename

To delete a file, use git to allow git to track the deletion. Don’t do it manually or it just shows up missing.

Revert a Single File to an Old Version

git checkout <commit ID> filename

There are various ways to do this (restore and revert maybe?) but I did it once like this and it worked. You have to be OK with losing any work in that file since the last commit. The point is it doesn’t affect the other files.

Branching

git branch <new-branch-name>

To create a new branch.

git switch <new-branch-name>

To switch to a different branch (same as the still-valid git checkout <new-branch-name>)

git checkout -b new-branch-name

To create a new local branch and switch to it in one command.

git push --set-upstream origin new-branch-name

? is that right, not just push?

How to create a branch on GitHub here.

Tagging

git tag -a v1.0.0 ec595fb -m 'message here'

then

git push origin v1.0.0

to tag a past commit. Version naming convention is [Major].[Minor].[Patch]

Merging

To merge, create a pull request on GitHub, then compare and merge. It’s pretty self-explanatory, THEN to update your local repository:

git fetch

because you made changes to the GitHub version, but not your local version, so the changes have to be fetched (opposite of push I think?)

git switch main

to switch to the main branch, and you should see a message saying the local main is behind origin/main, then

git pull

to pull the commits into the local branch. Merge complete.

Remote Repository (GitHub Usually)

git remote -v

lists the remote repositories.

git remote set-url origin git@github.com:rr34/Astro.git

allows you to change the remote url for the origin.

GitHub-Specific Features

The following are features in GitHub only, not classic command line Git:

  • Releases – but tags are command line and also a way of doing releases.
  • Pull requests

Computer Architecture, Terminology, Classification

Computer Architecture = Instruction Set Architecture (ISA) = Processor Architecture

At its lowest level, software must execute code properly on the processor chip itself. Processor architecture is standardized to allow for code to be written and compiled that works across many individual models of chips. Some common architectures with examples are:

IA-32

IA-32 is the 32-bit version of Intel’s x86 architecture.*

*Notice x86** does not by itself specify an architecture, it is a family of Intel architecture that includes everything from 16-bit released in 1978 to modern 64-bit.

**Note, the x86 Wikipedia article has a good explanation of which chips generally use which ISA.

  • The Intel 80386 microprocessor released in 1985 was the first commercial 32-bit processor.
X86-64 = X64 = X86_64 = AMD64 = INTEL 64

Released by Intel in 2003, 64-bit began to replace 32-bit.

  • Intel Pentium 4 F series.
ARMV8-A

ARMv8-A is the first 64-bit version of the ARM architecture. ARMv8-A is AArch64, which is the same as ARM64.

ARM is a double-acronym. RISC stands for Reduced Instruction Set Computing. All together, ARM stands for “Advanced Reduced Instruction Set Computing Machine.” The ARM architecture in general is maintained by ARM Ltd. in Cambridge, UK.

  • The ARM Cortex-A53 in the Raspberry Pi Zero 2 W uses ARMv8-A and is considered 64-bit.

Operating Systems Classified by ISA

Raspberry Pi OS

Currently, Raspberry Pi does not offer a 64-bit OS, but a beta 64-bit OS is in work and already available for download from the official library.

In a Linux command terminal, you can see processor information including architecture with the command:

lscpu

From Windows PowerShell, you can see processor information including architecture with the command:

systeminfo

Get-WmiObject Win32_Processor