best ipv6 address seen so far

2600::

It even works, serving http for sprint.net

June 8th is world IPv6 day!

June 8th is ISOC's "world ipv6 day". According to the organisers:

On 8 June, 2011, Google, Facebook, Yahoo!, Akamai and Limelight Networks will be amongst some of the major organisations that will offer their content over IPv6 for a 24-hour “test flight”. The goal of the Test Flight Day is to motivate organizations across the industry – Internet service providers, hardware makers, operating system vendors and web companies – to prepare their services for IPv6 to ensure a successful transition as IPv4 addresses run out.

Some of us have been trying to do that for years ;)

Anyway, some things you can do:

• Read all the ipv6 related posts on this blog
• Make sure you are using IPv6 certified cable
• Play loops of zen, an ipv6-only browser tile-puzzle
• If your ISP does not provide IPv6 and you want easy IPv6 access for a single workstation, I recommend the teredo protocol (its built into windows, and you can apt-get miredo on debian. For a server or network installation, take a look at Hurricane Electric's tunnel broker which will give you a /48 prefix routed over your normal internet connection.
• If you're on ipv6 already and you're feeling hardcore you can use NAT64 and DNS64 to get rid of all your local ipv4 traffic, instead routing it all through a NAT gateway at (for example) Andrews and Arnold.

Panickers guide to world ipv6 day

8th June 2011 is World IPv6 day. Maybe you haven't done anything about getting your website ipv6 enabled. Its taken the world 15 years to develop IPv6, so sure it seems *totally* reasonable that you can get it deployed in 10 days.

What I'm going to tell you about here will get basic IPv6 access to your site. It won't do it in a particularly pretty way, and its probably not the long term way to do it. Also I just hammered this out this afternoon (though its based on years of IPv6 use). (I hope) it should work.

First some DOs and DONTs:

* DON'T deploy IPv6 on your production servers. If you don't know much about IPv6, then blindly sticking it on your real production resources is probably a good way to put even your IPv4 (read: the real IP that everyone actually uses) connectivity at risk (for various reasons that you'll understand when you understand...)

* DO deploy a http proxy server on its own machine, and have that proxy your IPv6 traffic. You shouldn't need to modify *anything* on your production machines.

* DON'T put the IPv6 address record (AAAA, as opposed to the IPv4 A record) in your normal DNS. If you do, then users who have both IPv4 and IPv6 will usually try to connect over IPv6. You don't know how well this is going to scale, or how good your IPv6 connectivity is going to be (or even how good your users ipv6 connectivity is going to be, if everyone is going to be fucking around that day)

* DO put a DNS name (eg www.6.example.com, if your main site is on www.example.com) with the AAAA record. That way, users can choose to try using IPv6, and if its broken can easily get back to your main site. You'll need to publicise this, though, because its not going to get users connecting via IPv6 automatically, and at the same time you should provide some feedback: for example, an email address or a forum.


So what do you need to do:

* Get a dedicated server (either a physical hardware server, or a VPS) running a recent version of Linux. (Ubuntu 10.x would be enough)

* Connect that server to the ipv6 internet. If its on a network with native IPv6, then your host will probably give you connection details. If not, then use Hurricane Electric's free tunnel broker which will connect you over a regular internet connection.

* However you connect, you'll end up with an IPv6 address for your machine. It will be a string something like this 2001:470:1f09:1288::2 that you can get out of ifconfig(specifically, if you have a choice, choose the one that begins with a 2, not the one that begins with an f). Put that IPv6 address into an AAAA record in DNS (better hope your DNS hosting provider does AAAA records - the good ones do...) under a new DNS name. Don't put IPv4 addresses in there too. In my example, I'm going to configure:
blog.6.hawaga.org.uk AAAA 2001:470:1f09:1288::2

* Put apache httpd on your server, apt-get install apache2

* Now you'll need a client machine with IPv6 to try connecting to your new server. If you have a windows PC, you can probably turn on Teredo in the network configuration - it comes built in. On OS X, Linux or BSD, you can install miredo which is a Teredo client. Or you can set up another Hurricane Electric tunnel for your client machine. You can use test-ipv6.com to get a score for how well your new client machine is connected to the ipv6 internet.

* You should now be able to use a web browser to reach the hostname you configured in DNS back there - you should see apache's welcome/default page.

* Now, configure apache to forward all requests it receives onwards to your production website over IPv4. In the following example, my production IPv4 website is the one you are reading right now, benctechnicalblog.blogspot.com. Enable mod_proxy and mod_proxy_http, and then set up a virtual host directive like this (or put it in the base of your server config, seeing as this a host dedicated to forwarding ipv6 traffic):

<virtualHost *:80>
  ServerName blog.6.hawaga.org.uk
  ProxyPass / http://benctechnicalblog.blogspot.com/
  <Proxy http://benctechnicalblog.blogspot.com/>
    Allow from All
  </Proxy>
</virtualhost>

Once you've done that, visiting your ipv6 hostname (eg blog.6.hawaga.org.uk) should serve you the content of your real production website website.

Now publish the new hostname in a news item and make it seem like you know what you're doing...

Some stuff will not work, for sure: if you have anything that does things based on IPv4 address of client, that's all going to be based on the address of the proxy machine, not the real client IPv6 address. Things that might affect are localisation (eg language), and rate/load limiting based on ip address.

So, please ask questions in the comments and I'll see about answering them...

printing a message to stdout in fortran

write (*,'(''I wonder how someone kept a straight face when they invented this syntax for writing a string to the console'')')

ruby cloudwatch -> mrtg interface

I use mrtg to gather historical data on some of my servers. One of those servers lives in Amazon's Elastic Compute Cloud (EC2) and so is also monitored by Amazon CloudWatch.

Can I get cloudwatch data into mrtg?

mrtg has a fairly straightforward interface for plugging in arbitrary unix executables to collect data, so my first attempt was to use the main Java-based cloudwatch client to get data. that attempt started up one jvm for each metric collected, which massively overloaded my ec2 microinstance, keeping the load average around 3. pretty lame.

Amazon also provides a ruby interface. I had never programmed in ruby before, but its often interesting to learn a new language.

Here's what I ended up with.

First the config block for mrtg, which calls out to the ruby-mrtg-cloudwatch3 program that I wrote:

Target[cloudwatch_network]: `/home/mrtg/ruby-mrtg-cloudwatch/ruby-mrtg-cloudwatch3 NetworkIn NetworkOut AWS
/EC2 InstanceId=i-26bcaf51`
Title[cloudwatch_network]: Network traffic according to cloudwatch
options[cloudwatch_network]: growright,absolute,logscaleMaxBytes[cloudwatch_network]: 100000000

This gives a graph of network traffic according to cloudwatch. I can compare that alongside the network traffic graph for eth0 gathered from the local interface statistics. They should roughly match up, and they do (well hopefully they still do by the time you read this - these are live images):

According to the on-host network interface:


According to cloudwatch:


Now the actual ruby code:

#!/usr/bin/ruby1.8

require 'rubygems'
require 'AWS'

The two cloudwatch metric names, one that measures output data, one that measures input data, are give on the command line:

metrico=ARGV[0]
metrici=ARGV[1]

My code has hardcoded access keys at the moment which is a bit shitty:

ACCESS_KEY_ID='foo'
SECRET_ACCESS_KEY='bar'

Using the above credentials, a new cloudwatch object is made, @cw.

@cw = AWS::Cloudwatch::Base.new(:access_key_id => ACCESS_KEY_ID, :secret_access_key => SECRET_ACCESS_KEY, :server => "eu-west-1.monitoring.amazonaws.com" )

Each of the two metrics will be probed with the probe function. This uses a state file based on the metric name to get only readings which have not already been seen by this script. The two metrics use separate state files because cloudwatch doesn't give an atomic read for multiple metrics at once. The state file stores the time of the last seen reading. If there is no state file, we have to invent a time. There is a subtlety here: data does not appear in cloudwatch until around 5 minutes after its time stamp, so using the current time as an initial value results in not seeing any results. Instead, I go back about 15 minutes the first time, which will seems to be far enough back to get something.

def probe(metric)

  et = Time.now()

  statusfn="cloudwatch-"+ARGV[3]+"-"+metric+".status"
  if FileTest.exist?(statusfn) then

    f = File.new(statusfn, "r")
    tstring = f.gets
    ts = Time.parse(tstring)
    f.close
  else
    ts = et - 900 # needs to be more than 5 mins because otherwise we never get any data.
end

  res = @cw.get_metric_statistics(:measure_name => metric,  :statistics => 'Average,Sum', :namespace => ARGV[2], :period => 300, :start_time => ts, :end_time => et, :dimensions=> ARGV[3])

Now we're going to look at the rows that come back. Usually only one row will come back, if we're running this at about the same rate that cloudwatch is adding readings, but sometheres there will be more, or fewer.

In the case of network traffic, I want to return the sum of all readings for this metric. In other cases, such as disk usage, I would want to return the mean. This distinction is the same as default vs gauge measurements in MRTG.

samples = 0
  sum = 0
  avgsum = 0

  datapoints = res["GetMetricStatisticsResult"]["Datapoints"]

  lt = ts
  if datapoints.nil? then
   # nop
  else
    rows = datapoints["member"]

    rows.each { |r|
      nlt = Time.parse(r["Timestamp"])
      if(nlt < ts) then
        # nop - time was before requested start
      else
        samples += Float(r["Samples"])
        avgsum += Float(r["Average"])
        sum += Float(r["Sum"])
        nlt += 1
        if(nlt > lt) then
          lt = nlt
        end
      end
    } 

Now we can write out the new state file:

f=File.new(statusfn, "w")
    f.puts(lt)
    f.close
  end
  return sum
end

and finally output the MRTG format information:

sumo=probe(metrico)
sumi=probe(metrici)

# output mrtg format
puts sumo
puts sumi
puts 0
puts "cloudwatch: "+metrico+" and "+metrici

The end.

UK electric sockets vs chopsticks

My father taught me this trick when I was far too young to be taught this trick - how to connect a Europlug to a UK (actually HK in this case) electrical socket.

PAM python module for out-of-band one-time tokens

I previous wrote about integrating openid with unix shell logins for my public shell server barwen.ch. This post talks about the out-of-band PAM token module that I wrote as part of that - where I'm generating the tokens when the user is authenticated by OpenID. But I guess it could also have use when there's some other out of band mechanism (such as sending something by SMS?)

Subject to some other non-PAM web-based authentication (openid in this case, but it could be anything really), I want to issue a token value to the user in-band with respect to that other authentication (i.e. in a web page shown to the user), which is out-of-band with respect to PAM. That token should then be usable for a short period of time to make a single PAM authorisation to sshd.

That is, if you can log into the web-based authentication, you should then be able to log into the ssh system.

So on one side I need a PAM module (which I will write in Python) to check the tokens, and on the other side I need something (a command line tool) to issue the tokens. To complete the loop, I need some database (the filesystem) to store the tokens on the server side.

So here's the code. The PAM module comes complete with documented security vulnerability which allows anyone to delete certain files on your file system. ho ho.


First the token creator:

#!/usr/bin/python

import base64
import os
import pickle
import sys
import time

VALIDTIME = 60

tokenbits = os.urandom(8)
token = base64.b64encode(tokenbits, "+-")

print token

fn = token + ".token"
fh = open(fn, 'w')

obj = (sys.argv[1], time.time() + VALIDTIME)

pickle.dump(obj, fh)

and secondly the PAM module, which is based on the PAM module in my previous post:

import os
import syslog
import pickle
import time

def pam_sm_authenticate(pamh, flags, argv):
  syslog.syslog("start benc")
  pamh.authtok
  if pamh.authtok == None:
    syslog.syslog("got no password in authtok - trying through conversation")
    passmsg = pamh.Message(pamh.PAM_PROMPT_ECHO_OFF, "Monkeyballs?")
    rsp = pamh.conversation(passmsg)
    syslog.syslog("response is "+rsp.resp)
    pamh.authtok = rsp.resp
  # so we should at this point have the password either through the
  # prompt or from previous module
  syslog.syslog("got password: "+pamh.authtok)

  # now look for token
  # SECURITY BUG TODO: we're using this to make a path so we need to make sure
  # we're not being directed out into some other directory. We know the
  # range of characters that can be used in a token so we can reject if
  # anything other than those exists.
  # Especially as we delete the token file on use - otherwise could delete
  # arbitrary files on the system...
  tfn = "/root/" + pamh.authtok + ".token"

  if os.path.exists(tfn):
    fh = open(tfn)
    tokendata = pickle.load(fh)
    tokenuser = tokendata[0]
    tokentime = tokendata[1]
    fh.close()
    os.remove(tfn);

    # will remove the token even if it was for the wrong user
    # not sure if there's any security different wrt leaving it there if
    # its the wrong user?

    if tokentime < time.time():
      syslog.syslog("token time expired")
      return pamh.PAM_AUTH_ERR

    if tokenuser != pamh.user:
      syslog.syslog("token user "+tokenuser+" does not match requested user "+pamh.user)
      return pamh.PAM_AUTH_ERR
    

    return pamh.PAM_SUCCESS

  return pamh.PAM_AUTH_ERR

def pam_sm_setcred(pamh, flags, argv):
  return pamh.PAM_SUCCESS

ssh-like login with openid

I rigged together shellinabox and mod_auth_openid with some custom PAM glue so people can log into my hobby server s0.barwen.ch with an in-browser terminal window and openid.

shellinabox is not ssh (although web-based ssh is a good approximation). Instead it seems to be AJAX-over-https (which is kinda wtf for terminal access, but hey it seems to work).

The way I've glued it together is: First you visit the login page. That is an openid protected CGI script. The script runs with your openid in $REMOTE_USER, and does three things: it maps your openid to a local username; it generates (via sudo) an authentication token for you; and it HTTP-meta-redirects you to a hacked version of shellinabox.

shellinabox gives you a login prompt, asking for username and password. My hacked version stuffs the username and authentication token from the previous step into the keyboard.

The token ends up at a custom PAM module which verifies that the token is valid (for that user, and within a small time window after issue) and lets you in.

Then you get your shell prompt.

This seems like an interesting addition to barwen.ch's collection of login methods.

If you want a play, you can sign up at s0.barwen.ch

Also, if you break this, let me know rather than deleting / ...

pam_python

I came across pam_python, a PAM module that lets you write PAM modules in Python. I've come across things scripted by python a couple of times in the last few weeks at work so it seems interesting to play in this direction.

The first module I got sort-of working is this, which lets anyone log in with the password poop53.

import syslog

def pam_sm_authenticate(pamh, flags, argv):
  syslog.syslog("start benc")
  at = pamh.authtok
  syslog.syslog("got password: "+at)
  if at == "poop53" : 
    return pamh.PAM_SUCCESS
  else:
    return pamh.PAM_AUTH_ERR

def pam_sm_setcred(pamh, flags, argv):
  return pamh.PAM_SUCCESS

Now this cheats a bit - it assumes that some other module has read in the password from the user - I used pam_unix to do that, configured as below, so that first a check against the unix password happens and then if that fails, check against poop53.

auth sufficient pam_unix.so
auth sufficient pam_python.so /root/auth.py

The specific use I am thinking of, I don't want unix passwords to work. So in that case, I need to read in the password myself if it isn't already set.

Here's how I made that work:

def pam_sm_authenticate(pamh, flags, argv):
  syslog.syslog("start benc")
  pamh.authtok
  if pamh.authtok == None:
    syslog.syslog("got no password in authtok - trying through conversation")
    passmsg = pamh.Message(pamh.PAM_PROMPT_ECHO_OFF, "Monkeyballs?")
    rsp = pamh.conversation(passmsg)
    syslog.syslog("response is "+rsp.resp)
    pamh.authtok = rsp.resp
  # so we should at this point have the password either through the
  # prompt or from previous module
  syslog.syslog("got password: "+pamh.authtok)
  if pamh.authtok == "poop53" : 
    return pamh.PAM_SUCCESS
  else:
    return pamh.PAM_AUTH_ERR

def pam_sm_setcred(pamh, flags, argv):
  return pamh.PAM_SUCCESS

To use this with sshd, I need to enable sshd options UsePAM and ChallengeResponseAuthentication, and now I get this:

$ ssh root@192.168.141.128
Monkeyballs?poop53
Linux alcf3 2.6.35-28-generic #49-Ubuntu SMP Tue Mar 1 14:40:58 UTC 2011 i686 GNU/Linux
#

So I'm happy that I can grab some string from the remote user now, and process it to get an authentication decision.

Thought its pretty weird to have a regular ssh client giving me a Monkeyballs? prompt at auth time...

Modified: 2011-05-08: Later I used pam_python to write an out of band token module

spam and fidonet

I got a spam the other day, addressed to an address that isn't mine. That's not unusual. What was unusual was the address they used. It isn't mine, but it was mine when I was a teenager - benc@donor2.demon.co.uk was my internet address on DoNoR/2, an OS/2 focused BBS near Woking (in those days, young whippersnappers, you cared down to the town level where you were connecting to - DoNoR/2 was in dialing code 01483, the same as Guildford where I grew up). DoNoR/2 was primarily a fidonet system - 2:252/156, and then after the BS of geonetting, 2:440/4, and I was point 2 off that - Ben Clifford at 2:252/156.2

So although I'm usually annoyed when spam gets through my filters, this one made me think: aaah natsukashi.

Date: Thu, 27 Jan 2011 09:44:27 -0800 (PST)                                     
From: Jim Vivona                                            
To: benc@donor2.demon.co.uk                                                     
Subject: re                                                                     
                                        

sup! if you hadn't heard i lost my job at lawncare company about 5              
weeks ago, then i found this news article and made 379 in a few                 
hours!! I guess it was for the best! I learned from - News channel 4            
talk to you later!  

or headers in full:

Return-Path:                                                
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on                      
dildano.hawaga.org.uk                                                       
X-Spam-Level:                                                                   
X-Spam-Status: No, score=-2.1 required=2.5 tests=BAYES_00,DKIM_SIGNED,          
DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,     
URIBL_BLACK autolearn=ham version=3.3.1                                     
X-Spam-ASN: AS36646 98.138.0.0/17                                               
X-Spam-tokens-ham: 0.000-57--100h-0s--0d--hadnt,                                
0.000-55--96h-0s--0d--hadn't,                                               
0.000-28--48h-0s--0d--lisa, 0.000-25--44h-0s--0d--Lisa,                     
0.001-19--32h-0s--0d--madison, 0.001-19--32h-0s--0d--Madison,               
0.001-18--31h-0s--0d--Wisconsin, 0.001-18--31h-0s--0d--wisconsin,           
0.001-9--15h-0s--0d--entrepreneurs, 0.002-124--220h-1s--0d--Fwd             
X-Spam-tokens-spam: 0.902-15425--22770h-653297s--0d--H*c:alternative,           
0.861-28--214h-4131s--0d--president,                                        
0.860-6821--59523h-1134787s--0d--H*Ad:D*uk                                  
X-Spam-relays-trusted:                             

X-Spam-relays-untrusted: [ ip=98.138.85.229                                     
rdns=web120502.mail.ne1.yahoo.com                                           
helo=web120502.mail.ne1.yahoo.com by=dildano.hawaga.org.uk ident=           
envfrom=                                                                    
intl=0 id=p0RHpCGr007860 auth= msa=0 ] [ ip=194.54.47.229 rdns= helo=       
by=web120502.mail.ne1.yahoo.com ident= envfrom= intl=0 id= auth= msa=0 ]    
X-Spam-dkim-identity: @yahoo.com vivonajj@yahoo.com                             
X-Spam-dkim-domain: yahoo.com                                                   
X-Spam-dccb: dcc1.aftenposten.no                                                
X-Spam-dccr: dildano.hawaga.org.uk 1215; Body=1 Fuz1=1 Fuz2=1                   
X-Spam-token-summary: Tokens: new, 1; hammy, 109; neutral, 84; spammy, 3.       
X-Spam-languages: en                                                            
X-Spam-autolearn: ham                                                           
Received: from web120502.mail.ne1.yahoo.com (web120502.mail.ne1.yahoo.com       
[98.138.85.229])                                                            
by dildano.hawaga.org.uk (8.13.8/8.13.8/Debian-3) with SMTP id              
p0RHpCGr007860                                                              
for ; Thu, 27 Jan 2011 17:51:14 GMT                     
Authentication-Results: dildano.hawaga.org.uk; dkim=pass (1024-bit key)         
header.i=@yahoo.com; dkim-adsp=none                                         
Received: (qmail 94875 invoked by uid 60001); 27 Jan 2011 17:44:27 -0000        
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024;     
t=1296150267; bh=niep53FNFb78lkhNFkO8MX4NIpOCMZRTAUTHm1+GJzQ=;              
h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Ver    
sion:Content-Type;                                                          
b=OWa2WgmhLxGRa+pJep7Or929UysoVIk/SCB7BnFfurvysB63Nr6Odfb4b3gm2hFZqK+xOo    
Q6aZmUXn27EQbeA/8av52fU1KV33uhA9Th6rI0uKEIPg5LikGLLXUoaXrLzGdL2qyJ10UTMy    
2TwEkqU6bZQBCpMxY0fWANKPiIK+M=                                              
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;                                
s=s1024; d=yahoo.com;                                                         
h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Versi    
on:Content-Type;                                                            
b=eIuzjdEYirBi9RTFb1voGAKkH4bQUYyRkN6NT6mDBMj2GJcnf8bKz7R7NrQuWd8j9tjoviMF    
bXv/t3D2DtcoNbvQuDSPMa6ycXDMUkNFpW3dMkyrq6ZBSuw+Ye7TZXH7ect5MJcErjTAyu38    
+Dx4kXmdFIlAhs3Q0CBs4L7t+EI=;                                               
Message-ID: <164580.94308.qm@web120502.mail.ne1.yahoo.com>                      
X-YMail-OSG: vT0rPzMVM1kHrbVNywVWrHu0pysHCRnEnLjmjnEWPDU8KWm                    
9AZTRLoDJBmvlFuAhJxD2.uKxh0LPBsJi.WmUQZBW_OOq9UKFRzUPfKAOemv                 
qni6KAcfSaxd8Y6p2Cf6w5PGXAILIpD_0UuPIQ3LtnYoffxsz6w1ytx9R3cG                   
BiqUC4MNLdG0NlSV2mlQCFGOEBHXYNTzl4ejOnjciku0Z1Y5SGW4aUz_4gmA                   
oWXR.IhJ4dfWwgEVx2H9GvVpuPLizb_vwjmqP1e02TP9qrN0cGC1tFWTbyW9                   
ooJoemIjAwkiYhhY-                                                              
Received: from [194.54.47.229] by web120502.mail.ne1.yahoo.com via HTTP; Thu    
, 27 Jan 2011 09:44:27 PST                                                  
X-Mailer: YahooMailRC/553 YahooMailWebService/0.8.107.285259                    
Date: Thu, 27 Jan 2011 09:44:27 -0800 (PST)                                     
From: Jim Vivona                                            
Subject: re                                                                     
To: benc@donor2.demon.co.uk                                                     
MIME-Version: 1.0                                                               
Content-Type: multipart/alternative;                                            
boundary="0-588940240-1296150267=:94308"                                    
X-Greylist: Delayed for 00:06:40 by milter-greylist-3.0                         
(dildano.hawaga.org.uk [81.187.211.37]); Thu,                               
27 Jan 2011 17:51:15 +0000 (GMT)