Going to Xtreams

Perhaps a blog named "WriteStreams of Consciousness" has an implied obligation to cover I/O streams. And since streams are among the most essential programming building blocks, I'm always interested in better mousetraps there.

Smalltalk is home to perhaps the first truly elegant streams implementation, particularly when compared to other approaches developed around that time, such as those in C and C++. You can't get much simpler than 'myfile.txt' asFilename readStream contents to open and read a file, and yet there's significant power in the classes behind that. But new designs for stream libraries since followed, including pluggable/chainable I/O stream architectures, such as those found in Java, C#, and even advanced parallel stream processing frameworks.

Not to be outdone, Michael Lucas-Smith and Martin Kobetic have recently developed a new pluggable stream framework for Smalltalk called Xtreams, and I couldn't resist giving it a try. In the authors' own words,

Xtreams is an abstract producer/consumer pipeline over arbitrary source and destination types... you get a unified API for accessing files, sockets, pipes, strings, collections and many many other kinds of things.

It's a Google Code project, but the code is in the Cincom Public Repository: just three mouse clicks away after downloading and starting VisualWorks. It consists of several packages as described on the project page, along with SUnit test cases. I loaded the base set of packages and walked through the examples in Michael's basic primer and in the various package comments, while reading the wiki documentation.

Immediately, I found the basic API improvements (over conventional streams) refreshing. But the real beauty lies in "stacking" streams, collections, and even block closures. For example, this stacks an encoding stream atop a write stream atop a byte array to convert your string to UTF-8 encoded bytes:

(ByteArray new writing encoding: #utf8) write: 'Hello'; conclusion

Or, to do Base-64 encoding (for sending binary data as text):

String new writing encodingBase64 write: myBytes

For arbitrary transformations, you can provide your own transforming: block, as in this example:

"Multiply each pair of input elements together and return the result" ((Array withAll: (1 to: 20)) reading transforming: [ :in : out | out put: in get * in get ]) rest

And why limit ourselves to strings and arrays on the inside (as terminals)?  Here's streaming over a collection:

(1 to: 10000) reading ++ 1000; read: 5

... and a block closure:

| a b | a := 0. b := 1. "Fibonacci" [ | x | x := a. a := b. b := x + a. x ] reading read: 20

Stacking sources, terminals, and transforms: it's nearly as much fun as the Mousetrap or Incredible Machine games, but without the Rube Goldberg chunkiness.

I only scratched the surface in playing around, but it's a nice package, which is being further refined and extended. If you're a VisualWorks Smalltalker, give it a try.

While Smalltalk is an important incubator for significant technology innovation, it's certainly not a "by the masses" language. So if you're hesitant to jump into a Smalltalk image and code away, just read the summary of what Xtreams does and ask, "can my streams framework do that?"  Hopefully we'll soon see this kind of power and design elegance in other languages.