Previously I wired a radio clock via a gertboard into my raspberry pi.

Since then, I figured out that I didn't need the gertboard there, and so now the clock is connected to my other "production" Pi directly onto the GPIO pins.

I've also put the C code I use to interface the radio into ntpd onto github. I needed to copy it from one Pi to the other, and doing it via github means you can download it too now. (yes, I know there are other implementations of MSF decoding around, but I wanted to write the code myself - this is supposed to be fun!)

radio clock NTP server

I've got a couple of Raspberry Pi(e?)s. One I'm using for real server stuff like DNS. The other is for hacking around on. I got a gertboard for it, but hadn't wired anything else up. Then I saw this MSF radio time receiver (for 9 GBP) which is intended to receive 60kHz time broadcasts from the UK National Physical Laboratory.

When I was a teenager, a radio clock was a cool thing that I never had. Then later I got an internet connection, and with it NTP, and so I got geek-time a different way. So I never got an MSF clock until now.

The receiver plugs into the Gertboard (actually it can maybe plug directly in to the Pi's GPIO pins without a Gertboard in the way - I'm not sure) and presents a 1-bit serial signal that is the demodulated 60kHz signal. This is a slightly strange looking signal mostly consisting of 100ms of no carrier, two bits at 100ms each then 700ms of carrier, with 60 seconds of data being enough to transmit the current time.

There was some fiddling soldering to do to get the pins connected - turns out 10y old crocodile clips don't cut it. My father, being better than me at soldering, put those on for me. I cut two of the Gertboard's jumper cables in half so that I'd have wires which connect straight onto Gertboard pins.

The layer above that needs to run in software, at least in my setup.

The first program was a scope program that output * for 1 and . for 0 to the console 80 times a second or so, with a CR every second, to give a line-by-line view of the time. That showed the signal pretty clearly, like this:

.*******************............................................................
.**********.....................................................................
.*********************..........................................................
.**********.....................................................................
.*********......................................................................
.******************.............................................................
.******************.............................................................
.************...................................................................
.***********....................................................................

The short lines are when there's a sync pulse only, and the long lines are when there's a 1-bit immediately following the sync pulse.

The second program used a minute-long circular buffer as a shift register (thanks to Dave_H from the #a&a for that suggestion), and tries to decode time when one end of the buffer looks like the start-of-minute sync pattern. This was able to pretty accurately decode the time. I didn't implement parity checking or other error checking (for example, checking the sync pattern is exactly right) because of that accuracy, which gave trouble later on when I moved the antenna around. Thats something I could implement, and give out some accuracy metric. This took a few hours to code in C (yes, I wrote something in a language that wasn't Haskell). The code is pretty inefficient because it hard polls the GPIO pin. There is some edge triggering available in the Pi's GPIO implementation so I could investigate that. So that ended up as a program which outputs the current time every minute, to the console.

Next, I wanted to connect this with ntpd, so that I could compare the time with other internet time sources, and share the MSF signal with other hosts on the same LAN. ntpd has a facility for connecting "reference clocks" which are sources of time that are not other ntp servers. One of those, the SHMEM driver, communicates with a separate process using a shared memory buffer. So I got to learn about Linux shared memory (indexed using integer keys, I discover, rather than as nodes the file system). For that protocol, basically you stuff the current system clock time and the current received radio time into two fields in that shared memory, every time you get a clock reading. That was a straightforward addition to my second program.

So here's what the hardware setup looks like. The big board is the gertboard. Under it with the big wires is the Pi. The tiny tiny board held in the air by the jumpers is the MSF receiver board, and the big metal rod at the front is the antenna.

And here's what I see in ntpd:

 $ ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+2001:8b0:0:53:: 195.66.241.3     2 u   38   64  177   27.872   -4.596   8.680
-2001:8b0:7c:1:2 61.69.233.69     3 u   39   64  177    0.597   -2.285   8.710
+2001:8b0:7c:1:b 90.155.53.93     3 u   37   64  177    1.000   -3.868   8.395
*127.127.28.2    .MSF.            0 l   44   64   17    0.000  -23.730   6.885

where NTP has recently synced to the MSF driver (with fake IP address 127.127.28.2). Its about 20ms different from the network-connected NTP servers (see the offest column). Its difficult to know how much of that is from NTP over-the-network inaccuracies and how much is from my code, but I suspect the bulk of it is from my MSF code - my polling just isn't accurate enough at the moment, and my parsing has some weird time alignment stuff in it.

All in all, good fun and blinkenlights!

p.s. you're welcome to point your own ntp servers at this for fun. Add server tyne.cqx.ltd.uk to your /etc/ntpd.conf

p.p.s. is it sad that I can decode those octal reachability fields in my head.

scripting languages

another half-serious entry to the scripting languages vs programming languages debate: a scripting language is one which has decent string handling, rather than telling you to man-up.

brief facebook commentary in response to I'm supposed to give a talk on software stuff to college students Wednesday. Kind of a "things that would have helped to know before my first job." What would you put in there?:

possibly they already read "the mythical man month". now they just have to learn to actually believe that most of what he says in that book is relevant; most of what you do in colege is one person doing a project. almost nothing in paid software is that, even when you're the only programmer; use version control goddamnit; if 5 different people/groups have written some thing, and you don't like any of them and are going to write your own thing but "do it right this time", well there's 5 examples of people who tried to do it right and failed so you're feeling pretty cocky, aren't you? when your boss asks you "can we do X" and your answer is "yes (given 10 years and 50 developers)", your boss heards "yes (i can have it tomorrow)".

also, accept that you will not write bugfree code. its not a sign of being a pussy. its how code is. write tests. or, if you think you can write bugfree code, bet someone your next paycheck that its bugfree, if you're so confident. use the machine as your helper for that , and ge tin the habit of 1000 unit tests run while you go take a shit. i mean, before you commit your code.

(run the unit test son the other pussy's commits at least, and find the bugs they committed and make it seem like you are amazing)

london traffic pirate announcement

In-car radios in the UK often have a feature called Traffic Announcement, transmitted as part of RDS, which lets the current channel (or another non-radio input such as CD/mp3 player) be interrupted with a traffic programme being broadcast on another channel.

Driving on the M25 and listening to BBC Radio 1 the other week, the traffic flag clicked on and switched me to receiving some dodgy pirate radio station!

I wondered if this was accidental. Maybe not so accidental though - it turns out many of the pirate radio stations round London transmit RDS.

nagios and andrews and arnold SMS notification

Andrews and Arnold sells me a VOIP service which also lets me send SMSes over HTTP.

I wanted to make Nagios send out notifications this way, in addition to the existing email-based notification.

First I need a Nagios command definition to send out the notifications:

define command {
  command_name cqx-notify-service-by-sms
  command_line /var/cqxmon/secret-aa-curl 0781234567 "$NOTIFICATIONTYPE$: $HOSTALIAS$/$SERVICEDISPLAYNAME$ is $SERVICESTATE$"
}

That will invoke my sender script to send a message to phone number 0781234567.

Then we need the sender script. It has my password hardcoded in it (ick! but that could be replaced with a < redirect).

#!/bin/bash

# DO NOT COMMIT THIS TO VERSION CONTROl!

TARGET=$1
shift

TMPFILE=$(mktemp)

echo $@ > $TMPFILE
curl --get --form-string username=0300300300 \
              --form-string password=XYZ123 --form-string destination=$TARGET \
              --form message=\<$TMPFILE \
              http://sms.aa.net.uk/sms.cgi

rm $TMPFILE

The redirect via TMPFILE is because I was having trouble with spaces on the commandline that I couldn't seem to escape around.

Then I need to modify the Nagios contact definition for myself to call the cqx-notify-service-by-sms command in addition to the regular email notifier. In the contact definition for myself:

service_notification_commands cqx-notify-service-by-email, cqx-notify-service-by-sms

That will work for services. For hosts, you'll have to create a cqx-notify-host-by-sms command which does something similar, and add that to the contact notification_commands.

Nagios has a number of other contact fields that could be used to allow the target number to be configured per contact. I'll get round to that at some point; without making those changes, the above will only let you notify a single SMS number.

pronunciation of Haskell applicative operators.

I gave a talk on parsers in Haskell yesterday, and asked the audience how they thought I should pronounce some operators.

They came up with:

<* left looking sparrow
*> right looking sparrow
<*> goatse

If you don't know whatis goatse, you don't want to know. Really. But it made me laugh.

parser combinators talk at London HUG

I'm doing a talk on parser combinators in Haskell, in London on 23rd Jan 2013. Come along if you like.

columns

(Sorry I'm making up the code in this post rather than actually distilling down the real implementations - it probably doesn't run but you can get the idea)

A few times recently I've wanted to output column-like data from haskell: HTML tables in two cases, and CSV in another.

In these two cases, I wanted column headings (<th> tags in the HTML case; and a CSV heading line in the CSV case.

Previously I've written code that looks roughly like:

mapM_ putHeading ["heading1","heading2","heading3"]
forM_ rows $ \(entry1, entry2, entry3) -> do
  putCell entry1
  putCell entry2
  putCell entry3

The annoyance here was that nothing ties together the headings and the data values: although in the case of two or three columns, it is relatively simple to see the correspondence, it was getting hard in some wider cases.

Vaguely inspired by lenses, I rewrote some of this code to look like this:

cols = [("heading 1", \(entry1, _, _) -> entry1),
        ("heading 2", \(_, entry2, _) -> entry2),
        ("heading 3", \(_, _, entry3) -> entry3)
       ]
mapM_ putHeading (map fst cols)
forM_ rows $ \row -> forM_ cols \col -> (putCell ((snd col) row))

What this does is package up the column heading and the code to generate (for each row) the appropriate content. This makes it easier (I hope) to keep the headings and the data aligned. Also, all the boilerplate that you don't see here (putHeading and putCell disguise it) can be shared, with only a new cols defined for each different table.

functor

One of the first cool things you encounter in functional programming is map.

Say you have a function length :: String -> Int which gives the length of a string:

> length "hello"
5

(in real haskell, thats not actually the type of length but its close enough for now)

Now you can apply that to a list of strings, like this:

> map length ["hello","goodbye"]
[5,7]

For most of I've thought of this as meaning "apply the function length to each element of the list ["hello", "goodbye"].

But theres a slightly different interpretation thats a bit more "functional" feeling, that I've come across recently.

Consider only applying the first argument to map (you can do that in haskell...):

map length

Whats the type of this expression? It is [String] -> [Int]. So what its done is converted a function from string to int, into a new function from lists-of-strings to lists-of-ints.
And now we have that function that converts a list of strings to lists of ints, we can apply it to a list of strings:

> (map length) ["hello","goodbye"]
[5,7]

So the different reading that I see now is "lift this function to work on lists", first, followed by application to a list.

The same new intuition applies to functors in general and fmap, and its from thinking more about category theory that this view of things starts to appeal to me.