5 layer NAT

in a hotel I'm staying in, looks like NAT 5 layers deep... (yes they could be just routed with a final single NAT layer...)

 1. 192.168.99.1                                                      0.0%     6    1.2  15.4   1.1  84.9  34.1
 2. 192.168.20.1                                                      0.0%     6    2.4  36.4   2.3 206.1  83.1
 3. 192.168.1.1                                                       0.0%     6    3.8  28.7   2.9 154.5  61.6
 4. 192.168.10.254                                                    0.0%     6   86.7 107.1  77.9 195.2  43.9
 5. 192.168.219.250                                                   0.0%     6   28.2  56.0  27.2 141.7  46.8
 6. 195.14.158.85                                                    16.7%     6   26.6  55.1  26.6 115.7  35.0
 7. 195.14.158.145                                                   25.0%     5   31.1  33.7  30.9  39.1   4.7

Nominet .uk proposals again...

Nominet are proposing a second consulation on their wacky plan to open up second level domains in .uk - so I could have hawaga.org.uk or cqx.uk (assuming someone didn't get there first).

They talk now about owners of existing *.*.uk domains being given first dibs.

One of my concerns is that it fragments the UK namespace - there is already a "default zone" of co.uk and so that makes me wonder if (in the long term) co.uk should be abandoned entirely, and in the mean time, the co.uk and second level uk should be the same: one registration fee gives the same name registered in both zones. If you have the .co.uk, you get the .uk. If you don't have the .co.uk, you don't get the .uk.

That at least gives a bit of an easier user explanation "a .uk is the same as .co.uk used to be"; though its still loads of technical work on the site side for people who want such domains.

And it still destroys the history of having decent information about what kind of entity you are interacting with.

game theory

We played a new wide game with our Scouts.
Each kid in a team got a number written on their hand. To take someone prisoner, you needed to have a bigger number than them, either yourself or in total as a group of captors.
One group, C, cheated and increased several of their members numbers to 8, the maximum.
However, that backfired when we made the final score for each team be the number of points of their prisoners - team C had made its members so valuable compared to everyone else that another team, B, with just three team C prisoners scored more than everyone else.
It was satisfying that punishment for cheating was built into the rules.

mis-XML

There is a style of XML which I find quite frustrating. Here's a fragment:

    <mbeanNode>
      <name>java.lang:type=Memory
      <attributes>
        <attribute>
          <name>HeapMemoryUsage</name>
          <formattedValue>{committed=442433536, init=257990592, max=3670540288, used=103328176}</formattedValue>
         </attribute>
         <attribute>
           <name>NonHeapMemoryUsage</name>
           <formattedValue>{committed=57802752, init=24313856, max=224395264, used=57610720}</formattedValue></attribute>

This particular example came from XML output of javamelody, although I've encountered this misuse on a number of other occasions.

Most annoying is the guerilla formatting inside the <formattedValue> tag. XML already provides plenty of ways to pair up names and values, so why make a guerilla format that I now need to parse myself?

A smaller annoyance is the use of <attribute><key>X</key><value>V</value></attribute> style. XML already provides a mechanism for tagging a named key onto a value: <key>value</key>. The only defence of this I've heard is that it makes it easier to comply with a schema; but alas you miss the point and are also wrong. Wrong, because I can make an XML schema that accepts arbitrary XML tags; and missing the point, because the schema doesn't help much if all its doing is enforcing a skeleton.

I think all of this comes from fairly naive serialization of improperly defined application-internal data structures. suck.

Previously I wired a radio clock via a gertboard into my raspberry pi.

Since then, I figured out that I didn't need the gertboard there, and so now the clock is connected to my other "production" Pi directly onto the GPIO pins.

I've also put the C code I use to interface the radio into ntpd onto github. I needed to copy it from one Pi to the other, and doing it via github means you can download it too now. (yes, I know there are other implementations of MSF decoding around, but I wanted to write the code myself - this is supposed to be fun!)

radio clock NTP server

I've got a couple of Raspberry Pi(e?)s. One I'm using for real server stuff like DNS. The other is for hacking around on. I got a gertboard for it, but hadn't wired anything else up. Then I saw this MSF radio time receiver (for 9 GBP) which is intended to receive 60kHz time broadcasts from the UK National Physical Laboratory.

When I was a teenager, a radio clock was a cool thing that I never had. Then later I got an internet connection, and with it NTP, and so I got geek-time a different way. So I never got an MSF clock until now.

The receiver plugs into the Gertboard (actually it can maybe plug directly in to the Pi's GPIO pins without a Gertboard in the way - I'm not sure) and presents a 1-bit serial signal that is the demodulated 60kHz signal. This is a slightly strange looking signal mostly consisting of 100ms of no carrier, two bits at 100ms each then 700ms of carrier, with 60 seconds of data being enough to transmit the current time.

There was some fiddling soldering to do to get the pins connected - turns out 10y old crocodile clips don't cut it. My father, being better than me at soldering, put those on for me. I cut two of the Gertboard's jumper cables in half so that I'd have wires which connect straight onto Gertboard pins.

The layer above that needs to run in software, at least in my setup.

The first program was a scope program that output * for 1 and . for 0 to the console 80 times a second or so, with a CR every second, to give a line-by-line view of the time. That showed the signal pretty clearly, like this:

.*******************............................................................
.**********.....................................................................
.*********************..........................................................
.**********.....................................................................
.*********......................................................................
.******************.............................................................
.******************.............................................................
.************...................................................................
.***********....................................................................

The short lines are when there's a sync pulse only, and the long lines are when there's a 1-bit immediately following the sync pulse.

The second program used a minute-long circular buffer as a shift register (thanks to Dave_H from the #a&a for that suggestion), and tries to decode time when one end of the buffer looks like the start-of-minute sync pattern. This was able to pretty accurately decode the time. I didn't implement parity checking or other error checking (for example, checking the sync pattern is exactly right) because of that accuracy, which gave trouble later on when I moved the antenna around. Thats something I could implement, and give out some accuracy metric. This took a few hours to code in C (yes, I wrote something in a language that wasn't Haskell). The code is pretty inefficient because it hard polls the GPIO pin. There is some edge triggering available in the Pi's GPIO implementation so I could investigate that. So that ended up as a program which outputs the current time every minute, to the console.

Next, I wanted to connect this with ntpd, so that I could compare the time with other internet time sources, and share the MSF signal with other hosts on the same LAN. ntpd has a facility for connecting "reference clocks" which are sources of time that are not other ntp servers. One of those, the SHMEM driver, communicates with a separate process using a shared memory buffer. So I got to learn about Linux shared memory (indexed using integer keys, I discover, rather than as nodes the file system). For that protocol, basically you stuff the current system clock time and the current received radio time into two fields in that shared memory, every time you get a clock reading. That was a straightforward addition to my second program.

So here's what the hardware setup looks like. The big board is the gertboard. Under it with the big wires is the Pi. The tiny tiny board held in the air by the jumpers is the MSF receiver board, and the big metal rod at the front is the antenna.

And here's what I see in ntpd:

 $ ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+2001:8b0:0:53:: 195.66.241.3     2 u   38   64  177   27.872   -4.596   8.680
-2001:8b0:7c:1:2 61.69.233.69     3 u   39   64  177    0.597   -2.285   8.710
+2001:8b0:7c:1:b 90.155.53.93     3 u   37   64  177    1.000   -3.868   8.395
*127.127.28.2    .MSF.            0 l   44   64   17    0.000  -23.730   6.885

where NTP has recently synced to the MSF driver (with fake IP address 127.127.28.2). Its about 20ms different from the network-connected NTP servers (see the offest column). Its difficult to know how much of that is from NTP over-the-network inaccuracies and how much is from my code, but I suspect the bulk of it is from my MSF code - my polling just isn't accurate enough at the moment, and my parsing has some weird time alignment stuff in it.

All in all, good fun and blinkenlights!

p.s. you're welcome to point your own ntp servers at this for fun. Add server tyne.cqx.ltd.uk to your /etc/ntpd.conf

p.p.s. is it sad that I can decode those octal reachability fields in my head.

scripting languages

another half-serious entry to the scripting languages vs programming languages debate: a scripting language is one which has decent string handling, rather than telling you to man-up.

brief facebook commentary in response to I'm supposed to give a talk on software stuff to college students Wednesday. Kind of a "things that would have helped to know before my first job." What would you put in there?:

possibly they already read "the mythical man month". now they just have to learn to actually believe that most of what he says in that book is relevant; most of what you do in colege is one person doing a project. almost nothing in paid software is that, even when you're the only programmer; use version control goddamnit; if 5 different people/groups have written some thing, and you don't like any of them and are going to write your own thing but "do it right this time", well there's 5 examples of people who tried to do it right and failed so you're feeling pretty cocky, aren't you? when your boss asks you "can we do X" and your answer is "yes (given 10 years and 50 developers)", your boss heards "yes (i can have it tomorrow)".

also, accept that you will not write bugfree code. its not a sign of being a pussy. its how code is. write tests. or, if you think you can write bugfree code, bet someone your next paycheck that its bugfree, if you're so confident. use the machine as your helper for that , and ge tin the habit of 1000 unit tests run while you go take a shit. i mean, before you commit your code.

(run the unit test son the other pussy's commits at least, and find the bugs they committed and make it seem like you are amazing)

london traffic pirate announcement

In-car radios in the UK often have a feature called Traffic Announcement, transmitted as part of RDS, which lets the current channel (or another non-radio input such as CD/mp3 player) be interrupted with a traffic programme being broadcast on another channel.

Driving on the M25 and listening to BBC Radio 1 the other week, the traffic flag clicked on and switched me to receiving some dodgy pirate radio station!

I wondered if this was accidental. Maybe not so accidental though - it turns out many of the pirate radio stations round London transmit RDS.

nagios and andrews and arnold SMS notification

Andrews and Arnold sells me a VOIP service which also lets me send SMSes over HTTP.

I wanted to make Nagios send out notifications this way, in addition to the existing email-based notification.

First I need a Nagios command definition to send out the notifications:

define command {
  command_name cqx-notify-service-by-sms
  command_line /var/cqxmon/secret-aa-curl 0781234567 "$NOTIFICATIONTYPE$: $HOSTALIAS$/$SERVICEDISPLAYNAME$ is $SERVICESTATE$"
}

That will invoke my sender script to send a message to phone number 0781234567.

Then we need the sender script. It has my password hardcoded in it (ick! but that could be replaced with a < redirect).

#!/bin/bash

# DO NOT COMMIT THIS TO VERSION CONTROl!

TARGET=$1
shift

TMPFILE=$(mktemp)

echo $@ > $TMPFILE
curl --get --form-string username=0300300300 \
              --form-string password=XYZ123 --form-string destination=$TARGET \
              --form message=\<$TMPFILE \
              http://sms.aa.net.uk/sms.cgi

rm $TMPFILE

The redirect via TMPFILE is because I was having trouble with spaces on the commandline that I couldn't seem to escape around.

Then I need to modify the Nagios contact definition for myself to call the cqx-notify-service-by-sms command in addition to the regular email notifier. In the contact definition for myself:

service_notification_commands cqx-notify-service-by-email, cqx-notify-service-by-sms

That will work for services. For hosts, you'll have to create a cqx-notify-host-by-sms command which does something similar, and add that to the contact notification_commands.

Nagios has a number of other contact fields that could be used to allow the target number to be configured per contact. I'll get round to that at some point; without making those changes, the above will only let you notify a single SMS number.