The Journalist's Cage

Simpy and Powershell

I have spent a lot of time lately working with Powershell, Microsoft's next generation command-line environment. Built on the .NET architecture, Powershell is extremely versatile and robust. It provides excellent support for dynamic manipulation of XML content, a feature that is particularly useful when used in conjunction with .NET's Internet functionality.

Earlier this week, I did a bit of experimentation and figured out how to use Powershell to interact with the Simpy social bookmark service. When I developed Ruby bindings for the Simpy REST API, the most frustrating impediment was Simpy's mildly awkward redirect system. Ruby's Net::HTTP class doesn't handle redirects automatically, so I had to manually implement support for handling a 302 response. I also had to manually create the authentication string for the header using Ruby's Base64 encoding library. Powershell handles both of those things automatically. The .NET Net.NetworkCredential object automatically generates the proper header string when provided with login information, and the Net.WebClient object's downloadString method does proper redirect handling on its own without necessitating further intervention.

Powershell handles XML very well, which also made the task much easier. Certain aspects of the XML API seem needlessly verbose or a little bit eccentric. For instance, when you use the select-object Cmdlet to access an element node, it returns the node itself rather than the value of the node in some cases. Based on my own experiments, I think that it fails to show the actual text value when the contents of the node are wrapped in a CDATA block. In order to extract the text value, one must use the get_innerText method.

The following example demonstrates how to use Simpy's REST API from the command-line to display the nicknames and tags of recent bookmarks:

> $connect = new-object Net.WebClient
> $connect.credentials = new-object Net.NetworkCredential("segphault", "XXXXXX")
> $links = [xml]$connect.downloadString("http://simpy.com/simpy/api/rest/GetLinks.do")
> $links.links.link | select {$_.nickname.get_innertext()}, {$_.tags.tag | % {$_.get_innnertext()}}

$_.nickname.get_innertext()             $_.tags.tag | % {$_.get_innertext()}
---------------------------             ------------------------------------
GNOME Journal article about Tinymail    {development, software, email, GNOME...
Comparison of various open source gr... {development, software, graphics, Li...
Agile programming considered (mostly... development
Metaprogramming with Ruby               {Ruby, development, metaprogramming}
Linus talks like a pirate               {LinuxArs, Linus, Linux Kernal, soft...
Mark Shuttleworth talks about Ubuntu... {LinuxArs, Ubuntu, Debian}
Optimal use of fonts on Linux           {LinuxArs, X11, fonts}
IBM to provide commercial support fo... {LinuxArs, Eclipse, IBM}
McDonalds changes McFlurry cups to s... strange
GNOME orphaned thumbnail remover        {LinuxArs, GNOME, Glade, Python...}

Note that the inability to consistently extract the text value of a node with select makes it necessary to use a loop in order to display the tags as a list. If CDATA blocks weren't so troublesome, the select invocation would be a lot more concise. Despite the minor deficiencies, Powershell is well suited for this sort of work.

I also experimented with listing tag information. The following demonstrates how to display a list of tags and the number of bookmarks associated with each tag:

> ([xml]$connect.downloadString("http://simpy.com/simpy/api/rest/GetTags.do")).tags.tag

name        count
----        -----
LinuxArs    42
software    22
GNOME       8
strange     6
Mono        5
science     4
Qt          4
...

If I can find some time for further experimentation, I might throw together a few Powershell Cmdlets to automate link and tag manipulation.

Tags: shell, powershell, simpy, programming, dotnet

Posted on 2006-09-300 comments

Fun With Readline

Every now and then I dig up a bash tutorial and learn a few new tricks. The fc built-in caught my eye this time. fc allows you to edit the last command in a traditional text editor. When you quit the editor, the contents of the buffer are evaluated by the shell.

fc is really useful for debugging long pipelines or long ruby operations performed from the command line. The default editor can be customized by changing the FCEDIT variable. I tried making vim the default, but with all my plugins, it loads too slowly. I ended up doing the following:

export FCEDIT="vim --noplugin"

I also figured out how to yank back in deleted strings. You can insert the most recently deleted string by hitting C-y. I learned a couple of neat history tricks as well. If you type !* in a command line sequence, it will replace it with all but the first word of the last command entered. !$ will be replaced with just the last word of the last command, making it the equivelent of the C-. shortcut.

While I was poking through the documentation, I took the time to learn the readline config file format. I configured M-h to perform the history-search-backward operation, which is like cycling back through the history, but only using lines that start the same way the current line does. Check out some of the other spiffy readline commands you can bind to shortcuts.

Tags: shell, bash, vim

Posted on 2005-04-040 comments

Reverie of a Shell Fiend

Yesterday, I sent a message to the Vim development mailing list. I've been using GVim to manually edit the contents of a stream mid-pipeline:

cat testfile | sed '/[0-9]:\|=/d' | awk '{print $2 ", " $1}' |\
 sort | cat -n | gvim -V0 - | grep -v "Vim:" | sed 's/1/*/' > test

Unfortunately, Vim emits a message when it reads from stdin, so I have to use grep -v to pull the notification out of the output stream after closing GVim. Bram responded with an explanation and some suggestions. I'll finally be able to fix the problem by making the message go to stderr. Piping through Vim will be less of a hassle now.

Tags: shell, vim

Posted on 2005-03-230 comments

Vim Shell Patch

I've been following the Vim development mailing list since early January in order to help me keep up with progress on Vim 7. On Friday, somebody mentioned the Vimshell patch in a message about integrated debugging support.

I'd never heard of Vimshell before, but I decided to investigate. One of Vim's major failings is that it can't run an ansi capable shell inside of a buffer. There are some relatively kludgy interactive shell buffer implementations that utilize scripting languages, but none of them are really very useful. I used to have one that uses Python. It doesnt do Ansi, it doesnt do vt100, it doesnt support tab completion, and it has a lot of other issues. Vimshell is different. Implemented in C, it provides complete support for almost all standard shell features. With Vimshell, it's possible to run an entire Vim session inside of a single Vim buffer. It's really a very impressive patch. Unfortunately, it too has some limitations. At the present time, it does not work with Gvim, and it doesnt allow you to use Vim normal mode key shortcuts inside of shell buffers.

My experiments with Vimshell were all quite succesful. I hade some compilation issues at first, because AAP doesn't like patches that add new files. When I dumped the AAP build and grabbed a normal tarball, I was able to patch and build without any errors. All of my tests were successful and impressive: I was able to run ssh, ftp, Vim, top, bash, and even emacs flawlessly in individual vim buffers.

Tags: vim, programming, shell

Posted on 2005-02-271 comments