ssh over CONNECT over port 80

I run an ssh server on port 443 (the https port) of one of my machines. That's good when I want to get ssh from a network which bans "everything but web-browsing" - once I have ssh I can tunnel pretty much what I want.

But the other day I ran across a network (at an airport) which allowed only port 80 - the regular http port. Even port 443 was banned (so not only could I not connect to my ssh server, but no facebook, gmail, etc...)

I already have stuff on port 80 - my webserver - so what could I do? I know ssh can be tunnelled through the http CONNECT method. Could I set that up so that my web server would serve web traffic as usual but allow me to CONNECT to the ssh server?

Turns out yes.

In the http server config - enable mod_proxy and mod_proxy_connect. Then this config in proxy.conf (or the main server config, I guess):

<IfModule mod_proxy.c>
         ProxyRequests On
        <Proxy *>
                AddDefaultCharset off
                Order deny,allow
                Deny from all
        </Proxy>
        <Proxy localhost>
          Allow from all
        </Proxy>
        AllowCONNECT 22
        ProxyVia On
</IfModule>

(If you're doing this yourself - be careful about what you allow - if you are too liberal you turn your server into an open spamming and hacking engine for the world to exploit. And it will be found. Automatically, and within days or maybe hours.)

On the client side, use netcat as a proxy command in SSH: (you can also put the proxy command in ~/.ssh/ssh_config)

ssh -o 'ProxyCommand nc -X connect -x myhost.example.com:80 localhost 22' myhost.example.com

Tada.

Note that the ssh server now sees you connecting from localhost, and the http server is the one who logs the real IP address. A caveat here is if you have IP-based security policies that make localhost special, then anyone connecting through this proxy is also special.

Lots of the config for this came from http://mariobrandt.de/archives/technik/ssh-tunnel-bypassing-transparent-proxy-using-apache-170/

So that gets me a CONNECT proxy.

But does it work at the airport? No.

Turns out the airport is also running everything through a transparent proxy: when I connect to s0.barwen.ch port 80, I'm actually connected to the airport's transparent proxy. So an http CONNECT command to localhost:22 both points to the wrong place and is administratively prohibits.

It turns out I can http CONNECT to s0.barwen.ch port 80, though, and there issue another CONNECT command to get to localhost:22:

$ nc -4 -X connect -x s0.barwen.ch:80 s0.barwen.ch 80 
CONNECT localhost:22
HTTP/1.0 200 Connection Established
Proxy-agent: Apache/2.2.14 (Ubuntu)

SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu5

So I can get to the server this way.

How can I get the ssh commandline to talk to the remote server, with the extra proxy step added?

Like this:

$ cat t.sh 
( echo "CONNECT localhost:22" ; cat ) | nc -X connect -x s0.barwen.ch:80 s0.barwen.ch 80 | ( read ; read ; read; cat)

$ ssh -o 'ProxyCommand ./t.sh' s0.barwen.ch

Now once that's done, I can run a SOCKS proxy over the same ssh connection, and route my normal web browsing through that - giving me access to https sites, and anything else that can use SOCKS.

IPv6 in Amazon EC2

Amazon declares that IPv6 is unsupported on EC2 (the Elastic Compute Cloud), but I wanted it anyway.

I tried two ways, one which worked well, and one which did not.
The first way I tried, which did not work well, was using 6to4; the way which did work was a tunnel from Hurricane Electric.

I did everything below on one of the free Linux micro-instances supplied by Amazon, with an elastic IP address attached so that I'd have a permanent address.

The first approach I tried was using 6to4. This is a protocol which automatically gives a large range of IPv6 addresses to anyone with a single static IPv4, through a decentralised network of protocol gateways.

In another blog post, I described how to get 6to4 running on Linux in 5 command lines. I ran those commands on my EC2 instance and end up with my own IPv6 address configured.

But having an IPv6 address configured is not the same as having an IPv6 address reachable from the internet. There were severe reachability problems. The problems boil down to two causes:

• The decentralised management of 6to4 leads to lots of gateways being broken in a way that makes them act of blackholes; and the protocol makes it hard to discover which those gateways are to fix them. This is a problem for any host, not just for EC2 instances.
• Even with the EC2 firewall turned off as much as possible (i.e. no firewall rules at all), EC2 doesn't cleanly deliver IP traffic to instances. For most traffic, this is not a problem; but it interacts with 6to4 in a terrible way:
  your instance can only receive traffic from any particular 6to4 gateway if it has recently sent at least one packet to that gateway; there is a global network of 6to4 gateways, any of which can send traffic to you; when you send 6to4 traffic, you send it only to your closest 6to4 gateway. As a result, basically no 6to4 gateways can ever send traffic to you.

That second bullet point was especially hard to debug before I figured out what the EC2 firewall was doing - because certain pings and probes to test reachability caused the firewall to open up, at which point IPv6 traffic would flow between some sets of hosts until a few minutes later when it would mysteriously stop working again.

6to4 is basically useless on ec2.

---

The second method I tried, with much greater success [to the extent that, a year later, I regard this as production quality] is a manually configured tunnel via Hurricane Electric. HE have been around a long time; have a good reputation; and I used them before years ago.

The configuration at their end is a set of fairly straightforward web forms. I was allocated a /64 prefix, but they have options for more.

That web form also gives example configuration instructions for a variety of platforms. The Linux-net-tools instructions are the ones I used. I pasted the 4 given commands literally into a root prompt on my EC2 machine, and those configured the interface correctly (at least until reboot).

I get similar "stateful firewall" behaviour from EC2 as I mentioned above, but the difference here is that that the connection is only between me and the HE tunnel endpoint, rather than to arbitrary 6to4 gateways around the network. As long as *anything* goes over the tunnel every few minutes, then connectivity with the entire IPv6 world stays up. Compared to 6to4, when that tunnel is up, the connectivity from machines D and P seem *much* better. I can ping both ways without any mysterious losses. I need a ping to the tunnel endpoint (or anywhere on the IPv6 internet, really) every minute or so. That's no big deal - I have MRTG set up to measure some ipv6 latencies anyway, and that is generating this traffic as a side-effect.

So HE is a little bit (a few web forms) more effort to set up. But the connectivity is much much better. I recommend HE over 6to4 for this.

Other links: aco wrote about getting IPv6 on EC2 using sixxs, and if you're interested in getting a shell account on this machine (barwen.ch) to try for yourself: www.barwen.ch
. I found this this online ping tool useful during testing.

Modified: 2011-04-19 Rephrasing a bit based on ongoing experience, and some more hyperlinks
Modified: 2012-05-05 Rephrasing some more.