Nim Day 3: IO ============================================================ At the end of Nim Day 2, I said I'd be talkin' about Nim's type system next. But I did a little thinking since then and I've decided to cover Nim's IO (input/output) facilities instead. The change is for one simple reason: reading and writing tends to be more fun! When in doubt, I'd rather err on the side of having too much fun with this visit to Nim-land than too little. And when I say "fun", I mean getting into the meat of making things work rather than worrying overly much about syntax. To be sure, I take syntax very seriously and learning the fine details of a language is important! But I've also learned that you can learn a *lot* of minutia about a language's details and not ever learn *how to use it*. That's where I was years ago when read Stroustrup's _The C++ Programming Language_ cover-to-cover, did all of the exercises, bought the STL reference book (some big Addison-Wesley hardcover) and read through that, and then daydreamed about all of the things I would build...and somehow never got around to actually using it to build anything! What a waste! The point is, I try not to repeat that experience. I still forget as much as I learn, but I try to make sure that I can apply most of what I'm learning. File IO in the `system` module ------------------------------------------------------------ Let's jump right in and see what Nim provides in the core standard library. The 'system' module contains basic IO facilities. It's already imported into every Nim project, so there's no need to include it. You can see the entire list of contents of the 'system' module here: https://nim-lang.org/docs/system.html The very first IO thing we run into in the list of methods is `open()`. I also see a `writeLine()` and `close()` method further down. Let's see if we write a file and read it: # write.nim var myfile: File discard open(myfile, "hotdata", fmWrite) myfile.writeLine("wigglers=3") myfile.writeLine("danglers=6") close(myfile) A quick explanatory list: - Create variable of type File to hold our file reference - Call open() with the file referece, file name to open, and the mode - The file mode fmWrite means...oh, I think you can guess - Nim requires that we do something with all return values - `discard` is a non-action we can perform if we don't want the value - Call the writeLine() method on myfile and send it a string - Do it again - Close the file - Don't bother with error checking because that's for wimps Now we compile and run: $ nim compile write $ ./write $ Exciting? Maybe, let's see if our file was written: $ ls -l hotdata -rw-r--r-- 1 dave users 22 Aug 27 18:35 hotdata Yes. $ file hotdata hotdata: ASCII text Yes! $ wc hotdata 2 2 22 hotdata YES! $ cat hotdata wigglers=3 danglers=6 Yahooo! Yessss, yessss, YES! Okay, okay, calm down. I'm excited too. But can we *read* the file? # read.nim var myfile: File discard open(myfile, "hotdata", fmRead) echo myfile.readLine() echo myfile.readLine() close(myfile) By the way, there's a shortcut for compiling and running a Nim program all at once. $ nim compile --run read Or shorter: $ nim c --run read Or my favorite: $ nim c -r read ... wigglers=3 danglers=6 Woo! There's our data. Also, I find the Nim compiler to be a bit noisy by default, so we can quiet it down like so: $ nim c -r --verbosity:0 read Now how about reading from STDIN and writing to STDOUT? The `system` module defines those as File objects, so let's see if we can play with those: # stdio.nim var foo = stdin.readLine() stdout.writeLine(foo) Boy, since these objects are already defined and opened for us, this is sure a small program! Now we compile and run: nim c -r stdio ... Hey there, Gophersphere!!! Hey there, Gophersphere!!! Sweet, it echoes the line we typed and then exits. It's like a terrible `cat`. Redirecting stdin from a file works great, too, so this is the real deal: $ ./stdio < hotdata wigglers=3 Neat! Command line params in the `os` module ------------------------------------------------------------ If we import the `os` module, we can do some other handy things to interface with the outside world. I guess these aren't what we might think of as traditional "I/O" per se, but they do let us "input" and "output". For example, we can get command-line arguments ("arguably" a very important type of input) : # args.nim import os echo "Argh! You gave me:" echo paramStr(1) We can pass arguments to programs through the Nim compiler's arguments: $ nim c -r args Foo ... Argh! You gave me: Foo If we don't give our program, `args`, an argument, it will crash (gracefully!) with an invalid index error: $ ./args Argh! You gave me: Traceback (most recent call last) args.nim(4) args os.nim(1466) paramStr Error: unhandled exception: invalid index [IndexError] The `os` module also lets us examine files and directories, copy, move, and delete files, execute external commands and that sort of thing. Streams ------------------------------------------------------------ For more advanced file IO, there is the `streams` module, which has a ton of methods for reading and writing very specific amounts and types of data to File 'streams'. Let's just see a quick example of writing some binary data, just a couple numbers: # writebin.nim import streams var a: uint8 = 4 var b: int16 = 4095 let fs = newFileStream("hotdata2", fmWrite) fs.write(a) fs.write(b) fs.close Running this should create a binary file. Let's see: $ nim c -r writebin.nim ... $ hexdump hotdata2 0000000 ff04 000f 0000 0000005 Okay, that *looks* roughly right. At least there's a 4 in there and 4095 is 'fff' in hexadecimal, so that's probably right, but it's hard to tell when you factor in the default byte grouping, endianess, etc. So let's just see if Nim can read this back in for us! # readbin.nim import streams let fs = newFileStream("hotdata2") let a = fs.readUint8() let b = fs.readInt16() echo a echo b fs.close And here's the results... $ nim c -r readbin ... 4 4095 Hey, there are our numbers! Strings ------------------------------------------------------------ This doesn't have anything to do with IO, but I want to use this feature later and we need a break after reading and writing binary data, don't we? Nim's string handling is really excellent. I'm not going to attempt any comparisons with, say, Perl, Tcl, or Ruby. But I think it holds its own. Certainly, in the world of compiled languages, Nim's string support is very good. I think that's extremely important. I don't know about you, but almost everything I do involves working with strings in some form or another. Here are three important features: 1) Concatenation is done with the `&` operator: # stringcat.nim let who = "Gophers" echo "Hello " & who & "!" Does what you'd expect: $ nim -r c strings ... Hello Gophers! 2) Conversion to string is done with the `$` operator: # tostring.nim let foo = 5 let bar = ['a', 'b'] echo "My Stuff! foo=" & $foo & ", bar=" & $bar Quite nice output including a textual representation of the array: $ nim c -r tostring.nim ... My Stuff! foo=5, bar=['a', 'b'] 3) Interpolation with `%` requires the `strutils` library: # interpolation.nim import strutils let animal = "cow" let beans = 4342088 echo "Oh no! The $1 ate $2 beans!" % [animal, $beans] Prints: $ nim c -r interpolation ... Oh no! The cow ate 4342088 beans! By the way, when you include the `strutils` library, it brings with it a number of other libraries: Hint: system [Processing] Hint: interpolation [Processing] Hint: strutils [Processing] Hint: parseutils [Processing] Hint: math [Processing] Hint: algorithm [Processing] But even though it all gets compiled in, it barely makes a difference in the resulting executable and your program remains fast, compact, and dependency-free. Parsers ------------------------------------------------------------ On the subject of IO (particularly Input), I think it's highly relavent to make note of the large selection of parsers that Nim includes in the standard library. Here's a couple that might pique your interest: - `parseopt` - parser for short (POSIX-style) and long (GNU-style) command line options - `parsecfg` - a superset of ".ini"-style config files with multi-line strings, etc. - `parsexml` - simple XML parser - lenient enough to parse typical HTML - `parsecsv` - CSV file parser - `json` - JSON parser - `rst` - reStructuredText parser - `sexp` - S-expression parser There is also a whole other section devoted to multiple libraries for parsing and generating XML and HTML. The other thing I love about the Nim standard library is that it includes lower-level tools you can use to implement other parsers such a Perl-compatible Regular Expressions (PCRE) in the `re` library, Parsing Expression Grammars (PEGs) in `pegs` and lexers/parsers in `lexbase`, `parseutils`, and `strscans`. Those are rich enough to warrant a post (or two or three) of their own. But I doubt I want to be that thorough in this rambling phlog. Rather than pick one of the parser libraries at random, I'm going to demonstrate the one I used in real life just today to wrangle some data. The problem: I needed to extract a particular column from a CSV file, perform a number of alterations to it, and then write it back out (along with some other changes which are of no concern). Let's pretend I just need to make one of the columns UPPER CASE. First, our data file: (I've aligned the columns here so it's easier to read.) Category, Acronym, Definition chat, lol, Laughing Out Loud chat, brb, Be Right Back sf, tardis, Time and Relative Dimensions In Space nasa, sfc, Specific Fuel Consumption nasa, acrv, Assured Crew Return Vehicle Now our program: # acro.nim import os, parsecsv, strutils var csv: CsvParser csv.open(paramStr(1)) csv.readHeaderRow() while csv.readRow(): echo "$1,$2,$3" % [ csv.rowEntry("Category"), csv.rowEntry("Acronym").toUpper, csv.rowEntry("Definition")] csv.close Now we cross our fingers and run to see if we were able to write a good CSV: (Again, I've aligned the columns for our viewing pleasure, but normally CSV data is packed,like,this.) $ nim c -r acro.nim acronyms.csv ... Category, Acronym, Definition chat, LOL, Laughing Out Loud chat, BRB, Be Right Back sf, TARDIS, Time and Relative Dimensions In Space nasa, SFC, Specific Fuel Consumption nasa, ACRV, Assured Crew Return Vehicle Awesome! The Acronym column has been converted to uppercase. As I said, I actually did something very similar to this today. I started off with AWK, which certainly made easy work of parsing the data. But before I knew it, I was off in the weeds with an AWK script that was getting towards twenty lines and I was *fighting it*. This is where you turn to your favorite "scripting" language such as Python, Perl, or Ruby (I like all three). But with Nim, you can write the program just as quickly *and* end up with a fast, portable executable! I think it's just fantastic. Okay, this post is in danger of getting crazy long, but there is just one other "IO" related thing I want to mention... Networking ------------------------------------------------------------ Nim's standard library includes libraries for CGI, an HTTP server, an HTTP client, an SMTP client, an FTP client, and two levels of socket support. So it's ready to create all sorts of interesting applications right out of the box. (Sorry, no Gopher libraries in the Nim standard library.) Let's just see a really simple HTTP server example and call it a day. # trollserver.nim import asynchttpserver, asyncdispatch var troll = newAsyncHttpServer() proc handler(req: Request) {.async.} = await req.respond(Http200, "Go use Gopher instead!\n") waitFor troll.serve(Port(8787), handler) Now we compile and run: $ nim c -r trollserver And our server is waiting for us. In another terminal (or full-fledged browser, for that matter) we can make a request: $ curl localhost:8787 Go use Gopher instead! $ curl localhost:8787 Go use Gopher instead! What a ridiculous troll! But you can see how easy it would be to make a little web interface to serve up some data with Nim. I'll try to make the next installment shorter and perhaps a bit more focused. Until the next one, happy Nim hacking!