Don't mux parallel shell output! Introducing Hydra-print.

In this age of multicore-everything (your laptop, your phone, your toaster) there’s no excuse for anything not to run in parallel, right? So what about text-based shell programs? On the one hand, it has always been easy to launch things in parallel, say with bash’s support for backgrounding jobs, or using make -j.


But, on the other hand, I’ve never been happy with the output of make -j. Interleaving lines of output from parallel subprocesses isn’t very user friendly. (Indeed, cabal -j used to do the same thing, and recently it has switched to simply hiding the output instead.) Thus I’ve written a little utility than dynamically splits the terminal window when the compute job enters a parallel region (video below).

---

It’s called “hydra-print” and is both a Haskell library and a pair of command-line programs (hydra-view and hydra-head). You can find it at its Github page here and the released package is on Hackage here.

After running cabal install hydra-print, the simplest way to use it is to bring up a server like this:

hydra-view

And then arrange for a script to pipe the output of multiple commands to it in parallel:

(command1 | hydra-head) &
(command2 | hydra-head) &

It’s possible to compose command’s output sequentially as well, by asking hydra-head to return a named pipe, and deleting it to signal that the stream is finished.

p=`hydra-head -p`
command1 >> $p
command2 >> $p
rm -f $p

Further, its often convenient to use hydra-view to launch another command, using it to seed the view with output:

hydra-view -- make -j

The repository contains an example Makefile here, which shows how to adapt a Makefile to use hydra-print without any direct support builtin to Make. Of course, built-in support is better, and I hope to come up with a patch for cabal-install to use hydra-print soon.

Using the Haskell Library

The library is based on the recent io-streams package, and makes it very easy to send several kinds of Haskell data streams into hydra-print.

import qualified System.IO.Streams as S
import UI.HydraPrint

The two main ways to use the library are to (1) bring up a static interface with a given number of windows, or (2) bring up a dynamic view monitoring a changing collectionn of streams.

The former can be called with hydraPrintStatic defaultHydraConf (zip names strms). It’s type signature is:

hydraPrintStatic :: HydraConf -> [(String, InputStream ByteString)] -> IO ()

As you can see, it takes a fixed number of streams (in a list). A dynamic set of streams is represented as – what else – a stream of streams!

hydraPrint :: HydraConf -> InputStream (String, InputStream ByteString) -> IO ()

In a parallel program one might use a Control.Concurrent.Chan to aggregate streams and send them to hydraPrint:

strmSrc <- span=""> chanToInput strmChan
hydraPrint defaultHydraConf strmSrc

Finally, I think the most obvious improvement to make to this little utility would be to provide the option of a tmux interface instead. This would provide much more functionality and should be pretty straightforward, because tmux itself is quite scriptable. Emacs [client] would be another option.

Enjoy!