IFI 8410 — Session 6: Unix File System and Command Line

The Blueprint of Modern Computing

One tree, small tools, and text flowing between them

Where we are going

Today in four moves

Running example: what are the most frequent words in Shakespeare? — answered without writing a program.

Which one am I on?

The family is real, and you are in it

• The commands in this session work the same in bash and zsh.
• Small differences exist between GNU (Linux) and BSD (macOS) versions of tools — check man command on your own machine.
• Windows: use WSL, or work on the course server over SSH.

Do one thing well

Each tool is almost insultingly simple

wc -l   # count lines
sort    # put lines in order
uniq -c # collapse repeats, with a count
grep    # keep lines that match
head    # show the first few
tr      # translate or delete characters

The power is not in any one tool. It is in the interface they share.

Reading a command line

Command, options, arguments

The shell reads the line, expands shorthand such as ~ and *, then starts the program. The program never sees the wildcard — only the filenames it matched.

Why bother

When the shell is the right tool

The terminal is not an alternative to programming — it is one of the environments programming happens in.

Knowing where you are

Three commands, most of the time

Building a workspace

Create, copy, move, remove

• Separate inputs from outputs: you can delete out/ and rebuild it without risking data.
• cp and mv overwrite the target without asking.

Looking without opening

A 5 MB file is not for your editor

less file pages through interactively — space scrolls, / searches, q quits. It never loads the whole file into memory, so it opens a gigabyte instantly.

Reading the nine characters

Permissions in practice

• On a directory, x means "may enter" — without it you cannot cd in, even if you can read it.
• chmod 755 is the same as rwxr-xr-x — one digit per class.

Why two output channels

Results and complaints travel separately

import sys

# the answer — the next program's input
print(f"{count} {word}")

# the commentary — for a human
print("warning: 3 rows skipped",
      file=sys.stderr)

Same rule in the shell: echo results, but send progress messages with echo "..." >&2.

Repointing the channels

Redirection, four symbols

Building it one stage at a time

Most frequent words in Shakespeare

tr -d '.,:;?!"()[]' < data/shakespeare.txt \
| tr 'A-Z' 'a-z' \
| tr -s ' \t\r' '\n' \
| grep -v -e '^[[:space:]]*$' \
| sort \
| uniq -c \
| sort -rn \
| head -n 5

From typing to tooling

A pipeline you repeat is a script

#!/bin/bash
# Read text on stdin,
# write "count word" on stdout.
tr -d '.,:;?!"()[]' \
| tr 'A-Z' 'a-z' \
| tr -s ' \t\r' '\n' \
| grep -v -e '^[[:space:]]*$' \
| sort | uniq -c | sort -rn

The shell starts your programs too

Running Python from the command line

import sys
from pathlib import Path

def main():
    if len(sys.argv) < 2:
        print("usage: wordfreq.py FILE [N]",
              file=sys.stderr)
        return 1                  # exit status
    path = Path(sys.argv[1])
    top_n = int(sys.argv[2]) if len(sys.argv) > 2 else 20
    ...

if __name__ == "__main__":
    sys.exit(main())

Same operating system, two dialects

The shell command and its Python twin

Neither is "the real way". The shell is for interactive work and quick composition; Python is for logic that belongs in a program.

Now: hands-on

Open a terminal

Homework 5 — organize a directory tree from the shell, run your scripts from the command line, and show the commands you used. Due Wednesday, October 14.

Take away

Nine things worth remembering

• The file system is one tree rooted at /; your working directory decides how relative paths resolve.
• pwd, ls, cd answer where you are, what is here, how to move — check before you act.
• mkdir, cp, mv, rm organize the tree; rm has no undo.
• Every program has three streams: stdin, stdout, stderr.
• Results on stdout, messages on stderr — that separation is what makes composition safe.
• > and >> send output to a file; < feeds input from one; | connects one program to the next.
• A repeated pipeline belongs in a script: hash-bang plus chmod +x.
• Permissions answer who may do what: read, write, execute for owner, group, others.
• sys.argv, exit status, and pathlib are the same ideas, seen from inside Python.