Open Source!

What is Open Source? Here's the quick, easy way to think about one aspect of the Open Source philosophy:

The Free, Libre, Open-Source Software movement (FLOSS) is a techno-political philosophy that espouses the idea that the tools we use to compute—hardware and software—work best when they are freely available, free to use and free to modify as desired.

Every moment, your own daily interactions with computers and with the Internet are at least partly based on FLOSS software. Most of the world's websites are operated on servers running a variety of Linux. If you're using some kind of MacBook you are running macOS, which is a variant of FreeBSD. Apple's macOS and the distributions of Linux all trace their ancestry back to UNIX.

Windows 11 users have the ability (in Developer mode) to run Ubuntu Bash for Windows, which gives Windows users the ability to run a Linux-based Terminal.

For a brief introduction to FLOSS and Linux, check out this 5-minute video, Free, Open Source Software [YouTube].

What does Open source software look like?

• youtube-dl is a YouTube video downloader, an open-source project hosted on github.
• FileZilla and its License
• http://meyerweb.com/eric/tools/s5/ and its license
• A lot of "personal" open source software is distributed through github.com.

Computer History

• Mainframes
• Minicomputers
• 1970s - Microcomputers introduced
• 1975 - Heathkit H8
• 1977 - Apple II
• 1981 - Microsoft Disk Operating System (MS-DOS)
• 1984 - Macintosh
• 1985 - Microsoft Windows
• 1990 - Internet
• 1991 - Linux released
• 2002 - Apple OS X
• 2016 - Apple macOS released

GNU/Linux

• gnu.org
• Free Software Foundation
• YouTube video of Richard M Stallman presenting at TEDxGeneva [2014, 13 minutes]

Servers

Many of the world's computers connected to the Internet run Linux. By learning how to use a Terminal on your own Apple or Windows computer, you've gotten a head start in learning how to operate computers on the Internet.

• Virtualization (Virtual Box running Windows 10)
• Containers (eg. Docker) provide a sandboxed, consistent environment
• Virtual Private Servers (Ubuntu running course website)
• Demo: Linode VPS
  1. instance bootup
     initiated from the command line
  2. student login
     

     $ ssh userID@@xx.xx.xxx.xxx

     Use your userID and password given in class. Note: On this machine, you have sudo privileges. What can you do with that?
  3. update the server
     
     $ sudo apt update $ sudo apt upgrade
  4. install software with apt
     

     $ sudo apt install bsdgames

  5. using write to talk to other users
  6. using mesg n to block other users
  7. using sudo to elevate your privileges

Using open source tools on your computer

find files on your computer

Your computer has the ability to search for files and directories. Windows has a Search function built into the task bar, and OS X has the Spotlight feature on the right side of the menu bar. Those functions are GUI front-ends that look pretty, but we can perform customized searches by using the find command in the Terminal.

• To find a file in your home directory:

  $ find ~ -type f -iname filename

  The ~ specifies your home directory, -type f specifies that we're looking for a file, and -iname indicates that we want to look for the given name, case-insensitive
• If you're not exactly sure of the filename, you can use "*" to indicate an unknown string of characters, and "?" to indicate a single unknown character. You need to enclose the string in quotes for these to work:

  $ find ~ -type f -iname "*.m??"

  This would match any file with the extension "m-something-something": .MOV, .mov, .mp4, .m4v, etc.
• To find a recent file in your home directory:

  $ find ~ -type f -mtime -7 -iname filename

  -mtime -7 restricts the search results to files that have been modified in the last 7 days or less.

tar = Tape ARchive

• To compress a directory into a single file:

  # tar -czf directory filename.tgz

• To extract that directory:

  # tar -xzf filename.tgz

openssl

• To encrypt a file:

  # openssl aes-256-ecb -salt -pbkdf2 -in myInfile.pdf -out myInfile.enc

• To decrypt a file:

  # openssl aes-256-ecb -d -in myInfile.enc -out restoredInfile.pdf

• To encrypt a folder, pipe a tar command into an openssl command:

  $ tar -cz myFolder | openssl aes-256-ecb -salt -pbkdf2 -out untitled.enc

  If you're really trying to cover your tracks you should be sure to erase the original directory:
  $ rm -rP myFolder # -P flag overwrites files before deletion on macOS
• To decrypt that folder:

  $ openssl aes-256-ecb -d -in untitled.enc | tar -xz

wget

To download a website directory:

$ wget -r -k -np www.crashwhite.com/apcompsci/materials/lectures/

-r = recursive

-k = convert-links in the document so files will be suitable for local viewing

-np = no-parent links, so that only links below this directory will be recursed

It may be the case that a website has a robots.txt file that disallows automated following of links, or is blocking non-browser viewing of the website. In that case:

$ wget -r -k -np --execute robots=off --user-agent=Mozilla www.crashwhite.com/apcompsci/materials/lectures/

should allow you to download the site.

Automating your workflow with bash, alias, and keys

You've been uploading files to the server for most of the year. Let's figure out how to make that process a whole lot easier. :)

The encryption above used a symmetric key: the same key was used for encoding and decoding. Public key encryption is differet strategy for exchanging data securely. It relies on a pair of keys, one public and one private.

1. Open up a Terminal and make sure you can log on to the server with your userID and password. Place the window for this "remote" Terminal on the right side of your screen.
2. Open up another Terminal. Place the window for this "local" Terminal on the left side of the your screen.
3. On the local terminal, make a folder in your home directory where you'll store your public-private keypair:

   $ mkdir ~/keys

4. Generate a pair of keys in that directory:

   $ ssh-keygen -t rsa -f ~/keys/key

   Do not enter a passphrase for this key. You should see output something like this:
   Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/userID/keys/key. Your public key has been saved in /Users/userID/keys/key.pub. The key fingerprint is: SHA256:ZZkm/Vmp9DSwBiZodEiBrYcbYQlh3Y7FtdFypvZAJXI userID@ComputerName The key's randomart image is: +---[RSA 2048]----+ | ++ O==Eo+ . | | . * O+oB++ o . | | . O .o=B + = | | = o += + * . | | + .So + . | | . . | | | | | | | +----[SHA256]-----+
   Once this process is complete, you'll be able to cat your public key (you can share this with anyone), and your private key (you shouldn't share this with anyone).
5. Confirm that your public and private key exist:

   $ cat ~/keys/key

   $ cat ~/keys/key.pub

6. Securely copy your public key from your local home directory to your home directory on the server:

   $ scp ~/keys/key.pub userID@crashwhite.polytechnic.org:~

7. Confirm on the server Terminal that your public key is now located there in your home directory.

   $ ls ~

8. Now execute a series of commands, carefully, on the Server terminal:
   $ mkdir ~/.ssh # creates a hidden directory $ chmod 755 ~/.ssh # changes permissions $ cat ~/key.pub >> ~/.ssh/authorized_keys # copies key.pub to file $ chmod 600 ~/.ssl/authorized_keys # change permissions on file
   These commands place the public key in a file that is accessible to your computer when you try to log in automatically.
9. On the local Terminal, use the following command to try to log on to the server:

   $ ssh -i ~/keys/key userID@crashwhite.polytechnic.org

   You may get a warning about a possible man-in-the-middle attack. Accept it, by hitting the enter key.
   At this point, you should be automatically logged in to the Poly server from your local Terminal, and you didn't type in your password!

We've just set up a public-private keypair that will allow you to log on to the crashwhite.polytechnic.org server without having to enter your password. But here's the really good part.

1. Using the local Terminal on the left, log off from the Poly server that you just logged onto ( exit or ^D ).
2. In the local terminal, create an empty test file:

   $ touch test.txt

3. In the local terminal carefully type:

   $ scp -i ~/keys/key test.txt userID@crashwhite.polytechnic.org:~/forInstructor

   This should scp the test file to your forInstructor file.
4. On the server terminal, confirm that the test.txt file got uploaded as desired.
5. Once you've confirmed that this works, in the local Terminal, carefully enter:

   $ alias up=""

To be completed...

Other fun in the Terminal: monitoring your computer

• Doing a screencapture from the Terminal:

  $ screencapture -x ~/Desktop/screenmonitor/capture.jpg

• Converting the current date to a string:

  $ date +%Y%m%d%H%M%S

• A one-line bash script to capture your desktop silently every 30 seconds:

  while x=0; do screencapture -x ~/Desktop/screenmonitor/`date +%Y%m%d%H%M%S`.jpg; sleep 30; done;

Other fun in the terminal: using cron to launch a webpage

The cron utility is a daemon, a small program that runs without user intervention. This cron program is launched every minute that your computer is in operation, at which time it checks to see if there are, in turn, any programs that it is supposed to run. As a user, you can write small programs that you wish to run automatically on a given date or time, and create an entry in the cron table, or crontab interface.

Let's see how easy it is to do that.

a. Launching a webpage from the browser

I like to start my day reading the Penny Arcade comic. I can open by browser and type in the URL, and I might even make a bookmark in my browser for that page: https://www.penny-arcade.com/comic

Conveniently, I can also launch that webpage on my Apple OS X machine using the open command in the Terminal:

$ /usr/bin/open https://www.penny-arcade.com/comic

In Ubuntu (Debian Linux), one can launch the default terminal using the x-www-browser command:

$ x-www-browser https://www.penny-arcade.com/comic

Go ahead and try it!

b. Writing a script

Once we've confirmed that the command works, let's write a shell script that will run that line for us. This shell script, when executed, will issue the Terminal instruction that we just tried out above.

We're going to call the script penny_arcade_launcher.sh, and we're going to save it in an OS X directory called /usr/local/bin/, as shown below.

Create the following shell script using a text editor.

Here's what it looked like when I created that shell script using nano:

Confirm that your script works by opening up a Terminal and using the bash shell to run the script:

$ bash /usr/local/bin/penny_arcade_launcher.sh

You should see the page launch from that script!

c. Setting up a cron job

Now that we have a script, we can set up a cron job that will launch that script at a pre-specified time. If I want to read the comic when I start my day at 8am, I'm going to set up cron to run that script at 7:30 in the morning, Monday through Friday.

In the terminal, enter this command with spaces just as indicated.

$ export EDITOR=nano

This command sets up the editor that we'll be using for this activity.

Now:

$ crontab -e

The -e flag launches an editor for the crontab, which we can use to tell the computer when to run a given command or script. Add a line to the crontab similar to the one below, which uses bash to run the script at 7:30 every day of the week, Monday through Friday:

30 07 * * 1-5 bash /usr/local/bin/penny_arcade_launcher.sh

The instruction to start the launcher using bash is the last entry. The five fields in front of that instruction indicate when the command should be run.

From the Wikipedia entry on cron:

The asterisk * indicates that the job should be run during every time period indicated. So, our launcher will run at 7:30 every day of the month, every month of the year, Monday through Friday.

When you save the crontab editor you'll probably get a message like this:

File Name to Write: /tmp/crontab.GHSgEb/crontab

That's okay. Enter "Y" to save the changes.

In OS X, cron is now deprecated in favor of using launchd. Although cron is still supported, you may wish to consider creating a launchd plist instead.

See below for more information on setting that up.

d. Advanced Users: Setting up launchd

cron has been deprecated on many systems in favor of a more powerful and versatile (and more complex) system called launchd for "launch daemon." This launch daemon is responsible for managing the running of files on your system in the same way that cron is, but configuring it takes a bit more work.

We're assuming you already have your penny_arcade_launcher.sh shell script set up and stored in /usr/local/bin/.

1. Create your plist file
   

   Here's an example that you can start from. You should call this file something like com.crashwhite.launchPennyArcade.plist.

   <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.crashwhite.pennyarcadelauncher</string> <!-- The job that should be run --> <key>ProgramArguments</key> <array> <string>bash</string> <string>/usr/local/bin/penny_arcade_launcher.sh</string> <string>sleep</string> <string>10</string> </array> <!-- A job that should run at a certain day/date/time --> <key>StartCalendarInterval</key> <array> <dict> <key>Hour</key> <integer>23</integer> <key>Minute</key> <integer>25</integer> <key>Weekday</key> <integer>1</integer> </dict> <dict> <key>Hour</key> <integer>23</integer> <key>Minute</key> <integer>25</integer> <key>Weekday</key> <integer>4</integer> </dict> </array> <!-- A regularly repeating job --> <key>StartInterval</key> <integer>120</integer> <!-- Run it when it is first loaded? --> <key>RunAtLoad</key> <true/> <key>UserName</key> <string>rwhite</string> </dict> </plist>
2. Move this file to the LaunchAgents directory
   

   For this launchd file to work, it needs to be placed in the appropriate directory. In OS X, user files are placed in ~/Library/LaunchAgents. (There are other places the file may run as well, but we're going to stick with this for now.)

3. Load your file using launchctl
   

   You now need to tell launchd that you'd like this agent to be included with all the other things that launchd runs from time to time.

   In the Terminal:

   $ launchctl load ~/Library/LaunchAgents/com.crashwhite.launchPennyArcade.plist

   If you don't get any error messages, then your agent is being monitored and should be run at the time and day you've indicated.

   If you decide you don't want to run this script anymore, be sure to unload the plist from launchctl:

   $ launchctl unload ~/Library/LaunchAgents/com.crashwhite.launchPennyArcade.plist

Advanced Users: Taking a picture (OS X only)

Just as the script above took a screenshot every 30 seconds, you can write a script that takes a picture using the webcam every 30 seconds as well.

To do this you'll need to install ffmpeg onto your computer.

Then, write and run the following script: