For many years, whenever I needed specific contents from a file or command output, I used to combine sed, grep, tr, and usually I would get it right after 20 attempts or so.

More recently (as in, the last three years) I have committed myself to not do this anymore. After all, it’s just so very ugly to have a command consisting of five or six commands separated by | pipelines. And I frequently think back to that particular buddy I met on IRC in the ’90s and especially his fierce annoyance with people who suggested the “just connect pipes until you have it right”-type approach. His default ititial reply always was the same:

Why don’t you just use awk?

The answer others gave was usually similar to what I used to answer: “because I don’t understand one bit of awk.” For more than a decade Awk felt like being a dark stranger, wearing a trench coat,standing in a dark shallow alley. But suddenly I wasn’t intimidated anymore. I decided to get to know Awk by just not using grep, sed, tr and similar tools anymore, except for the very simple stuff like ‘delete line 13” or ‘just return foo’.

So, background aside, I’ll just post code snippets that are useful. I am not sure about others but for me the quickest way to learn new stuff is by example.

Find pids of one or more processes

awk '/ruby19: (unicorn.*master|resque)/ && !/awk/ { print $1 }' < <(ps xa -o pid,command)

If you have suggestions for awk snippets and receipes please post them.

Oops. Some update broke most of the links. This has been restored.

I was reinstalling my laptop and found myself wondering about the same thing I always wonder about: which NTP client should I use? I decided to duck around the Internet again, trying to find a recent comparison of ntp versus openntp. I think I searched for the same thing quite some years ago and didn’t really find an answer. I wasn’t disappointed this time because I found a good explanation on the importance of having a synchronised clock that does not set the time backwards. The following comment was made made by sweh, on the Linode forums.

Read More >>

I attempted to answer a question I see asked so often: How can I adequately configure Xen’s networking? Well here’s a suggestion I originally sent to the openvswitch mailinglist. Perhaps it might help one or two of the lone pirates searching the seas of the Internet for clues.

From: me <my_email.com>
Subject: Re: [ovs-discuss] bridge renaming issue with Xen
Date: Sat, 10 Mar 2012

> > [root@hc_2c519_1_1_7 scripts]# ./network-bridge start netdev=eth0
> > Nothing to flush.
> > Waiting for peth0 to negotiate link.
> > can’t add peth0 to bridge eth0: Invalid argument
>
>
> What is generating this last line?

I’d just like to jump in and explain the path I am currently exploring with Xen.

In network-bridge you add one line at line number 2:

exit 0

There, perfect.

Now setup your preferred network without Xen. Setup bridges (wan0, lan0, whatever) first, using openvswitch. Then use your distribution’s standard networking options if they are flexible enough. Or just write your own network init script. This was my own preferred method for a Xen dom0. I chose Arch Linux, not Debian, as dom0 because it proved to be much easier. It has less frustrating conventions and more room for adequate creativity.

So, with network-bridge being disabled and your network back in orderthe next step is your system’s init. Configure it to first start openvswitch and ovs-brcompatd. After those, add your networking script. This shouldn’t be  hard – runlevel scripts are usually far less hard to understand than this ‘network-bridge’ monstrosity. The latter does weird things no one asked it to do: it renames eth0 to peth0 and creates eth0 as a bridge and then adds  peth0 to eth0. Xen doesn’t need to handle that initial system initialisation. Your network script should handle that.

So the alternative I suggest gives you the option of making a bridge with openvswitch, you can call it wan0 and then let Xen know this is the name of the default bridge. Again: in your own network script – not in Xen’s network-bridge. Then add eth0 and don’t forget to set the link state of the devices to up… that one bit me one day.

You can then go even further, by modifying vif-bridge. I copied it to vif-ovs and rewrote most of it. It allowed me to replace the brctl commands with all kinds of (better) ovs commands. I rewrote the xm part of Xen a bit on one box, they’re python scripts, and they now pass a’vtag’ variable to vif-ovs where vtag can either be a VLAN number or the string “trunk.”

So now one host config for Xen might contain a line like this:

vif  = [ 'script=vif-ovs,vifname=vm0,mac=00:16:3e:03:00:03,bridge=vmnet0,vtag=30' ]

So with a bit of hackery here and there you can improve Xen dramatically. Especially when you want it to work perfectly with openvswitch. All of Xen’s scripts (those in /etc/xen/scripts) are hard to understand imho. I am spending some of my free time at (trying to) rewrite them. Unfortunately I don’t know how to handle this with xl rather than with xm. But lest I digress too far let me just leave it at this :)

I’m currently playing with FreeBSD on a VPS. It seems smart to not really touch the base system — everything can be done from within jails that are installed for a certain purpose. From what I understand it’s comparable to virtualisation techniques.

It seems that the newer kid on the block is VIMAGE. As my FreeBSD host is running in a Xen virtual environment I have built the XENHVM image with a few additions to enable NAT and firewall stuff and so forth.

I’ve tried to use VIMAGE before and with sucess… although there seemed to be a problem with natd. When I was running natd the speed of the network link had terrible performance.

In between searching for NAT clues for jails and VIMAGE I could remember that there was a method to install new jails without using the immutable / schg file flag. But I kept forgetting the make option. After grepping around it turns out that the flag is -DNO_FSCHG. So when you install a new jail, you might try something like the following:

make buildworld installworld distribution DESTDIR=/usr/jails/base -DNO_FSCHG

I’ll update this post when I find a fitting NAT solution. The FreeBSD handbook is handy and contains lots of examples but I wonder whether it’s possible to NAT traffic without using a NAT daemon.

I like Terminus, it’s a cool font, small and very readable. And did you know that it’s very easy to switch between fonts with key strokes?
I have this in my ~/.Xdefaults file – after editing, don’t forget to run xrdb -merge ~/.Xdefaults.

For this to work you need a couple of variants: Terminus, Terminusmodx and the TrueType version of Terminus.

These entries are the default:

URxvt.font:       -*-terminus-medium-*-*-*-12-*-*-*-*-*-*
URxvt.boldFont:   -*-terminusmodx-bold-*-*-*-12-*-*-*-*-*-*-*
URxvt.italicFont: xft:Terminus (TTF):style=MediumItalic,pixelsize=12

These entries allow you to zap between a big font and small font with Alt-Shift-+ and Alt-Shift-_:

URxvt.keysym.M-S-plus:         command:\033]710;-*-terminus-medium-*-*-*-32-*-*-*-*-*-*-*\007\033]711;-*-terminus-bold-*-*-*-32-*-*-*-*-*-*-*\007\033]712;xft:Terminus (TTF):style=MediumItalic,pixelsize=32\007
URxvt.keysym.M-S-underscore:   command:\033]710;-*-terminus-*-*-*-*-12-*-*-*-*-*-*-*\007\033]711;-*-terminusmodx-bold-*-*-*-12-*-*-*-*-*-*-*\007\033]712;xft:Terminus (TTF):style=MediumItalic,pixelsize=12\007

Enjoy :)

I didn’t notice that the comments form was broken for quite some time. It should be fixed now. :)

From the fortune cookies:

Need to print a manpage? Use
	man name_of_manpage | col -bx | lpr
		-- Dru 

A little change in code can repair a lot. I updated a FreeBSD amd64 system and replaced the GENERIC kernel with the XENHVM kernel. Then I noticed that it would boot…. up until it loaded its net device xn0 – this is where it halted.

Back in January 2011 someone had exactly the same problem and posted it to the FreeBSD-Xen mailing list. Fortunately it received a reply that fixed the issue, at least for me.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

--- sys/dev/xen/netfront/netfront.c.orig	2010-12-21 17:09:25.000000000 +0000
+++ sys/dev/xen/netfront/netfront.c	2011-01-17 10:11:06.000000000 +0000
@@ -401,13 +401,14 @@
 xen_net_read_mac(device_t dev, uint8_t mac[])
 {
 	int error, i;
 	char *s, *e, *macstr;
 
-	error = xs_read(XST_NIL, xenbus_get_node(dev), "mac", NULL,
-	    (void **) &macstr);
-	if (error)
+	if ((error = xs_read(XST_NIL, xenbus_get_node(dev), "mac", NULL,
+	    (void **) &macstr)) != 0 &&
+	    (error = xs_read(XST_NIL, xenbus_get_otherend_path(dev), 
+	    "mac", NULL, (void **) &macstr)) != 0)
 		return (error);
 
 	s = macstr;
 	for (i = 0; i < ETHER_ADDR_LEN; i++) {
 		mac[i] = strtoul(s, &e, 16);

Thanks for the patch, Janne Snabb, hat tip to you.

Some boxes just have to complain about everything. I use rxvt-unicode (urxvt) as my terminal. This terminal sets the TERM environment variable to rxvt-unicode-256color.

Sometimes I login to a box that will complain about a missing termcap entry. I always ignored it or changed the TERM variable to rxvt before logging into the box, e.g. with TERM=rxvt ssh somebox. But finally, I found an easier fix.
On a box that does have the termcap entry and henceforth does not complain, you can write the following command:

infocmp -C rxvt-unicode-256color

Put the output of this into a file ~/.termcap of the remote box. Doing this will be enough for the box to not complain anymore and render your terminal 100% useful!

Also, did you know that there is a command that can automatically put the output of a local terminal command onto your clipboard? Simply pipe the output to xclip: