---
title: "Software Tools, Getting Started"
series: "Software Tools"
number: 1
author: "rsdoiel@gmail.com (R. S. Doiel)"
date: "2020-09-29"
keywords: [ "Oberon", "Pascal", "programming" ]
copyright: "copyright (c) 2020, R. S. Doiel"
license: "https://creativecommons.org/licenses/by-sa/4.0/"
---


# Software Tools, Getting Started

## Overview

This post is the first in a series revisiting the
programs described in the 1981 book by Brian W. Kernighan and
P. J. Plauger's called [Software Tools in Pascal](https://archive.org/details/softwaretoolsinp00kern).
The book is available from the [Open Library](https://openlibrary.org/)
and physical copies are still (2020) commonly available from used book
sellers.  The book was an early text on creating portable command
line programs.  

In this series I present the K & P (i.e. Software Tools in Pascal)
programs re-implemented in Oberon-07. I have testing my implementations
using Karl Landström's [OBNC](http://miasap.se/obnc/)
compiler and his implementation of the Oakwood Guide's modules
for portable Oberon programs. Karl also provides a few additional
modules for working in a POSIX environment (e.g. BSD, macOS, Linux,
Windows 10 with Linux subsystem). I have also tested these
programs with Mike Spivey's [Oxford Oberon Compiler](http://spivey.oriel.ox.ac.uk/corner/Oxford_Oberon-2_compiler) an aside
from the differences file extensions that both compilers use
the source code works the same. 

NOTE: OBNC compiler is the work of Karl Langström, it is portable across many systems where the C tool chain is available.

NOTE: POSIX defines a standard of compatibility inspired by [UNIX](https://en.wikipedia.org/wiki/Unix), see <https://en.wikipedia.org/wiki/POSIX>


## Getting Started.

Chapter one in K & P is the first chapter that presents code. It introduces
some challenges and constraints creating portable Pascal suitable for use
across hardware architectures and operating systems. In 1981 this included
mainframes, minicomputers as well as the recent evolution of the microcomputer.
The programs presented build up from simple to increasingly complex as
you move through the book.  They provide example documentation and discuss
their implementation choices. It is well worth reading the book for those
discussions, while specific to the era, mirror the problems program authors
face today in spite of the wide spread success of the POXIS model, the
consolidation of CPU types and improvements made in development tools in
the intervening decades.

Through out K & P you'll see the bones of many POSIX commands we have today.

Programs from this chapter include:

1. **copyprog**, this is like "cat" in a POSIX system
2. **charcount**, this is like the "wc" POSIX command using the "-c" option
3. **linecount**, this is like the "wc" POSIX command using the "-l" option
4. **wordcount**, this is like the "wc" POSIX command using the "-w" option
5. **detab**, converts tabs to spaces using tab stops every four characters in a line

All programs in this chapter rely solely on standard input and output.
Today's reader will notice an absence to common concepts in today's
command line programs.  First is the lack of interaction with command line
parameters, the second is no example take advantage of environment variables.
These operating system features were not always available across
operating systems of the early 1980s. Finally I'd like to point out a
really nice feature included in the book. It is often left out as a topic
in programming books.  K & P provide example documentation. It's structure
like an early UNIX man page. It very clear and short. This is something
I wish all programming texts at least mentioned. Documentation is important
to the program author because it clarifies scope of the problem being
tackled and to the program user so they understand what they are using.


### [1.1. File Copying](https://archive.org/details/softwaretoolsinp00kern/page/7/mode/1up)

Here's how K & P describe "copyprog.pas" (referred to as "copy" in
the documentation).


~~~

PROGRAM

    copy    copy input to output

USAGE

    copy

FUNCTION

    copy copies its input to its output unchanged. It is useful for copying
    from a terminal to a file, from file to file, or even from terminal to
    terminal. It may be used for displaying the contents of a file, without
    interpretation or formatting, by copying from the file to terminal.

EXAMPLE

    To echo lines type at your terminal.

    copy
    hello there, are you listening?
    **hello there, are you listening?**
    yes, I am.
    **yes, I am.**
    <ENDFILE>

~~~

The source code for "copyprog.pas" is shown on
[page 9](https://archive.org/details/softwaretoolsinp00kern/page/9/mode/1up)
of K & P.  First the authors introduce the __copy__ procedure
then a complete the section introducing it in context of the complete Pascal
program. After this first example K & P leave implementation of the full
program up to the reader.

The body of the Pascal program invokes a procedure called
[copy](https://archive.org/details/softwaretoolsinp00kern/page/8/mode/1up)
which reads from standard input character by character and writes
to standard output character by character without modification.  Two
supporting procedures are introduced, "getc" and "putc". These are shown
in the complete program listing on page 9. They are repeatedly used
through out the book. One of the really good aspects of this simple
program is relying on the idea of standard input and output. This makes
"copyprog.pas" a simple filter and template for writing many of the programs
that follow. K & P provide a good explanation for this simple approach.
Also note K & P's rational for working character by character versus
line by line.

My Oberon-07 version takes a similar approach. The module looks remarkably
similar to the Pascal but is shorter because reading and writing characters are
provided for by Oberon's standard modules "In" and "Out".
I have chosen to use a "REPEAT/UNTIL" loop over the "WHILE"
loop used by K & P is the attempt to read from standard input needs to happen
at least once. Note in my "REPEAT/UNTIL" loop's terminating condition.
The value of `In.Done` is true on successful read and false
otherwise (e.g. you hit an end of the file). That means our loop must
terminate on `In.Done # TRUE` rather than `In.Done = TRUE`. This appears
counter intuitive unless you keep in mind our loop stops when we having
nothing more to read, rather than when we can continue to read.
It `In.Done` means the read was successful and does not
mean "I'm done and can exit now". Likewise before writing out the character
we read, it is good practice to check the `In.Done` value. If `In.Done` is
TRUE, I know can safely display the character using `Out.Char(c);`.

~~~

MODULE CopyProg;
IMPORT In, Out;

PROCEDURE copy;
VAR
  c : CHAR;
BEGIN
  REPEAT
    In.Char(c);
    IF In.Done THEN
        Out.Char(c);
    END;
  UNTIL In.Done # TRUE;
END copy;

BEGIN
  copy();
END CopyProg.

~~~

#### Limitations

This program only works with standard input and output. A more generalized
version would work with named files.

### [1.2 Counting Characters](https://archive.org/details/softwaretoolsinp00kern/page/13/mode/1up)

~~~

PROGRAM

  charcount count characters in input

USAGE

  charcount

FUNCTION

  charcount counts the characters in its input and writes the total
  as a single line of text to the output. Since each line of text is
  internally delimited by a NEWLINE character, the total count is the
  number of lines plus the number of characters within each line.

EXAMPLE

  charcount
  A single line of input.
  <ENDFILE>
  24

~~~

[On page 13](https://archive.org/details/softwaretoolsinp00kern/page/13/mode/1up)
K & P introduces their second program, **charcount**. It is based on a single
procedure that reads from standard input and counts up the number of
characters encountered then writes the total number found to standard out
followed by a newline. In the text only the procedure is shown, it is
assumed you'll write the outer wrapper of the program yourself as
was done with the **copyprog** program. My Oberon-07 version is very similar
to the Pascal. Like in the our first "CopyProg" we will make use of the
"In" and "Out" modules. Since we will
need to write an INTEGER value we'll also use "Out.Int()" procedure which
is very similar to K & P's "putdec()". Aside from the counting this is very
simple  like our first program.

~~~

MODULE CharCount;
IMPORT In, Out;

PROCEDURE CharCount;
VAR
  nc : INTEGER;
  c : CHAR;
BEGIN
  nc := 0;

  REPEAT
    In.Char(c);
    IF In.Done THEN
      nc := nc + 1;
    END;
  UNTIL In.Done # TRUE;
  Out.Int(nc, 1);
  Out.Ln();
END CharCount;

BEGIN
  CharCount();
END CharCount.

~~~

#### Limitations

The primary limitation in counting characters is most readers are
interested in visible character count. In our implementation
even non-printed characters are counted. Like our first program
this only works on standard input and output. Ideally this should
be written so it works on any file including standard input and
output. If the reader implements that it could become part of a
package on statistical analysis of plain text files.

### [1.3 Counting Lines](https://archive.org/details/softwaretoolsinp00kern/page/14/mode/1up)

~~~

PROGRAM

  linecount count lines in input

USAGE

  linecount

FUNCTION

  linecount counts the lines in its input and write the total as a
  line of text to the output.

EXAMPLE

  linecount
  A single line of input.
  <ENDFILE>
  1

~~~

**linecount**, from [page 15](https://archive.org/details/softwaretoolsinp00kern/page/15/mode/1up)
is very similar to **charcount** except adding a
conditional count in the loop for processing the file. In
our Oberon-07 implementation we'll check if the `In.Char(c)`
call was successful but we'll add a second condition to see if the
character read was a NEWLINE. If it was I increment
our counter variable.

~~~

MODULE LineCount;
IMPORT In, Out;

PROCEDURE LineCount;
CONST
  NEWLINE = 10;

VAR
  nl : INTEGER;
  c : CHAR;
BEGIN
  nl := 0;
  REPEAT
    In.Char(c);
    IF In.Done & (ORD(c) = NEWLINE) THEN
      nl := nl + 1;
    END;
  UNTIL In.Done # TRUE;
  Out.Int(nl, 1);
  Out.Ln();
END LineCount;

BEGIN
  LineCount();
END LineCount.

~~~

#### Limitations

This program assumes that NEWLINE is ASCII value 10. Line delimiters
vary between operating systems.  If your OS used carriage returns
without a NEWLINE then this program would not count lines correctly.
The reader could extend the checking to support carriage returns,
new lines, and carriage return with new lines and cover most versions
of line endings.


### [1.4 Counting Words](https://archive.org/details/softwaretoolsinp00kern/page/14/mode/1up)

~~~

PROGRAM

  wordcount count words in input

USAGE

  wordcount

FUNCTION

  wordcount counts the words in its input and write the total as a
  line of text to the output. A "word" is a maximal sequence of characters
  not containing a blank or tab or newline.

EXAMPLE

  wordcount
  A single line of input.
  <ENDFILE>
  5

BUGS

  The definition of "word" is simplistic.

~~~

[Page 17](https://archive.org/details/softwaretoolsinp00kern/page/17/mode/1up)
brings us to the **wordcount** program. Counting words can be
very nuanced but here K & P have chosen a simple definition
which most of the time is "good enough" for languages like English.
A word is defined simply as an run of characters separated by
a space, tab or newline characters.  In practice most documents
will work with this minimal definition. It also makes the code
straight forward.  This is a good example of taking the simple
road if you can. It keeps this program short and sweet.

If you follow along in the K & P book note their rational
and choices in arriving at there solutions. There solutions
will often balance readability and clarity over machine efficiency.
While the code has progressed from "if then" to "if then else if"
logical sequence, the solution's modeled remains
clear. This means the person reading the source code can easily verify
if the approach chosen was too simple to meet their needs or it was
"good enough".

My Oberon-07 implementation is again very simple. Like in previous programs
I still have an outer check to see if the read worked (i.e. "In.Done = TRUE"),
otherwise the conditional logic is the same as the Pascal implementation.

~~~

MODULE WordCount;
IMPORT In, Out;

PROCEDURE WordCount;
CONST
  NEWLINE = 10;
  BLANK = 32;
  TAB = 9;

VAR
  nw : INTEGER;
  c : CHAR;
  inword : BOOLEAN;
BEGIN
  nw := 0;
  inword := FALSE;
  REPEAT
    In.Char(c);
    IF In.Done THEN
      IF ((ORD(c) = BLANK) OR (ORD(c) = NEWLINE) OR (ORD(c) = TAB)) THEN
        inword := FALSE;
      ELSIF (inword = FALSE) THEN
        inword := TRUE;
        nw := nw + 1;
      END;
    END;
  UNTIL In.Done # TRUE;
  Out.Int(nw, 1);
  Out.Ln();
END WordCount;

BEGIN
  WordCount();
END WordCount.

~~~

## [1.5 Removing Tabs](https://archive.org/details/softwaretoolsinp00kern/page/20/mode/1up)

~~~

PROGRAM

  detab convert tabs into blanks

USAGE

  detab

FUNCTION

  detab copies its input to its output, expanding the horizontal
  tabs to blanks along the way, so that the output is visually
  the same as the input, but contains no tab characters. Tab stops
  are assumed to be set every four columns (i.e. 1, 5, 9, ...), so
  each tab character is replaced by from one to four blanks.

EXAMPLE

  Usaing "->" as a visible tab:

  detab
  ->col 1->2->34->rest
      col 1   2   34  rest

BUGS

  detab is naive about backspaces, vertical motions, and
  non-printing characters.

~~~

The source code for "detab" can be found on
[page 24](https://archive.org/details/softwaretoolsinp00kern/page/24/mode/1up)
in the last section of chapter 1. **detab** removes
tabs and replaces them with spaces. Rather than a simple "tab"
replaced with four spaces **detab** preserves a concept found on
typewriters called "tab stops". In 1981 typewrites were still widely
used though word processing software would become common. Supporting the
"tab stop" model means the program works with what office workers would
expect from older tools like the typewriter or even the computer's
teletype machine. I think this shows an important aspect of writing
programs. Write the program for people, support existing common concepts
they will likely know.

K & P implementation includes separate source files
for setting tab stops and checking a tab stop.  The Pascal K & P
wrote for didn't support separate source files or program modules. Recent Pascal
versions did support the concept of modularization (e.g. UCSD Pascal). Since
and significant goal of K & P was portability they needed to come up
with a solution that worked on the "standard" Pascal compilers available on
minicomputers and mainframes and not write their solution to a specific
Pascal system like UCSD Pascal (see Appendix, "IMPLEMENTATION
PRIMITIVES [page 315](https://archive.org/stream/softwaretoolsinp00kern?ref=ol#page/315/mode/1up)).
Modularization facilitates code reuse and like information hiding is an
import software technique. Unfortunately the preprocessor approach doesn't
support information hiding.

To facilitate code reuse the K & P book includes a preprocessor as part
of the Pascal development tools (see [page 71](https://archive.org/stream/softwaretoolsinp00kern?ref=ol#page/71/mode/1up)
for implementation). The preprocessor written
in Pascal was based on the early versions of the "C" preprocessor
they had available in the early UNIX systems. Not terribly Pascal
like but it worked and allowed the two files to be shared between
this program and one in the next chapter.

Oberon-07 of course benefits from all of Wirth's language improvements
that came after Pascal. Oberon-07 supports modules and as such
there is no need for a preprocessor.  Because of Oberon-07's module
support I've implemented the Oberon version using two files
rather than three. My main program file is "Detab.Mod",
the supporting library module is "Tabs.Mod". "Tabs" is where I
define our tab stop data structure as well as the
procedures that operating on that data structure.

Let's look at the first part, "Detab.Mod". This is the module
that forms the program and it features an module level "BEGIN/END" block.
In that block I call "Detab();" which implements the program's functionality.
I import "In", "Out" as before but I also import "Tabs" which I will show next.
Like my previous examples I validate the read was successful before proceeding
with the logic presented in the original Pascal and deciding
what to write to standard output.

~~~

MODULE Detab;
  IMPORT In, Out, Tabs;

CONST
  NEWLINE = 10;
  TAB = 9;
  BLANK = 32;

PROCEDURE Detab;
VAR
  c : CHAR;
  col : INTEGER;
  tabstops : Tabs.TabType;
BEGIN
  Tabs.SetTabs(tabstops); (* set initial tab stops *)
  col := 1;
  REPEAT
    In.Char(c);
    IF In.Done THEN
      IF (ORD(c) = TAB) THEN
        REPEAT
          Out.Char(CHR(BLANK));
          col := col + 1;
        UNTIL Tabs.TabPos(col, tabstops);
      ELSIF (ORD(c) = NEWLINE) THEN
        Out.Char(c);
        col := 1;
      ELSE
        Out.Char(c);
        col := col + 1;
      END;
    END;
  UNTIL In.Done # TRUE;
END Detab;

BEGIN
  Detab();
END Detab.

~~~

Our second module is "Tabs.Mod". It provides the supporting procedures
and definition of the our "TabType" data structure. For us this
is the first time we write a module which "exports" procedures
and type definitions. If you are new to Oberon, expected constants,
variables and procedures names have a trailing "*". Otherwise the
Oberon compiler will assume a local use only. This is a very
powerful information hiding capability and what allows you to
evolve a modules' internal implementation independently of the
programs that rely on it.

~~~

MODULE Tabs;

CONST
  MAXLINE = 1000; (* or whatever *)

TYPE
  TabType* = ARRAY MAXLINE OF BOOLEAN;

(* TabPos -- return TRUE if col is a tab stop *)
PROCEDURE TabPos*(col : INTEGER; VAR tabstops : TabType) : BOOLEAN;
  VAR res : BOOLEAN;
BEGIN
  res := FALSE; (* Initialize our internal default return value *)
  IF (col >= MAXLINE) THEN
    res := TRUE;
  ELSE
    res := tabstops[col];
  END;
  RETURN res
END TabPos;

(* SetTabs -- set initial tab stops *)
PROCEDURE SetTabs*(VAR tabstops: TabType);
CONST
  TABSPACE = 4; (* 4 spaces per tab *)
VAR
  i : INTEGER;
BEGIN
  (* NOTE: Arrays in Oberon start at zero, we want to
     stop at the last cell *)
  FOR i := 0 TO (MAXLINE - 1) DO
    tabstops[i] := ((i MOD TABSPACE) = 0);
  END;
END SetTabs;

END Tabs.

~~~

NOTE: This module is used by "Detab.Mod" and "Entab.Mod"
and provides for common type definitions and code reuse.
We exported `TabType`, `TabPos` and `SetTabs`. Everything else
is private to this module.

## In closing

This post briefly highlighted ports of the programs
presented in Chapter 1 of "Software Tools in Pascal".
Below are links to my source files of the my
implementations inspired by the K & P book. Included
in each Oberon module source after the module definition
is transcribed text of the program documentation as well
as transcribed text of the K & P Pascal implementations.
Each file should compiler without modification using the
OBNC compiler.  By default the OBNC compiler will use the
module's name as the name of the executable version. I
I have used mixed case module names, if you prefer lower
case executable names use the "-o" option with the OBNC
compiler.

~~~

    obnc -o copy CopyProg.Mod
    obnc -o charcount CharCount.Mod
    obnc -o linecount LineCount.Mod
    obnc -o wordcount WordCount.Mod
    obnc -o detab Detab.Mod

~~~

If you happen to be using The [Oxford Oberon Compiler](http://spivey.oriel.ox.ac.uk/corner/Oxford_Oberon-2_compiler)
you need to rename the files ending in ".Mod" to ".m" 
and you can compiler with the following command.

~~~
    obc -07 -o copyprog CopyProg.m
    obc -07 -o charcount CharCount.m
    obc -07 -o linecount LineCount.m
    obc -07 -o wordcount WordCount.m
    obc -07 -o detab Tabs.m Detab.m
~~~

Note the line for compiling "Detab" with **obc**, your
local modules need to become before the module calling them.


+ [CopyProg](CopyProg.Mod)
+ [CharCount](CharCount.Mod)
+ [LineCount](LineCount.Mod)
+ [WordCount](WordCount.Mod)
+ [Detab](Detab.Mod)
    + [Tabs](Tabs.Mod), this one we'll revisit in next installment.


# Next

+ [Filters](../../10/31/Filters.html)