Ben Clifford Technical Blog: spam

Showing posts with label spam. Show all posts

05 March, 2017

toad.com open mail server

So there was some controversy decades ago the past about John Gilmore's public open SMTP relay server. I wondered if it still existed.

It does!

benc@dogger:~$ telnet new.toad.com 25
Trying 209.237.225.253...
Connected to new.toad.com.
Escape character is '^]'.
220 new.toad.com ESMTP Sendmail 8.12.9/8.12.9; Sun, 5 Mar 2017 08:12:44 -0800
EHLO dogger.cqx.ltd.uk
250-new.toad.com Hello dynamic-91.hawaga.org.uk [90.155.94.91] (may be forged), pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE 89000000
250-DSN
250-ETRN
250-AUTH GSSAPI
250-STARTTLS
250-DELIVERBY
250 HELP
MAIL FROM:benc@hawaga.org.uk
250 2.1.0 benc@hawaga.org.uk... Sender ok
RCPT TO:benc@hawaga.org.uk
250 2.1.5 benc@hawaga.org.uk... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Subject: test 1

test
.
250 2.0.0 v25GCiXw019546 Message accepted for delivery
221 2.0.0 new.toad.com closing connection

Date: Sun, 5 Mar 2017 07:09:06 -0800
From: benc@hawaga.org.uk
Message-Id: <201703051509.v25F96Xw016104@new.toad.com>
Subject: test 1

17 February, 2012

dmarc

A bunch of email providers announced DMARC which builds on top of SPF and DKIM to allow domains to specify more policy when SPF and/or DKIM fail.

I already have SPF and DKIM set up on my personal domain, hawaga.org.uk, which has been round for over a decade. I run mail servers for various other domains, but those are much younger and much less widely used.

Its been hard to quantify how much this has helped/not helped. I don't get complaints about spam originating from my address. I used to get a lost of postmaster backscatter but not any more - not sure why, though I can invent various possible reasons.

One of the interesting things with DMARC is that it claims to provide feedback about what filtering is happening, from receiving/filtering parties - that makes it especially interesting, I think.

So, given that I already have DKIM and SPF, what extra do I need to do to get something useful from DMARC?

I need to publish a policy in DNS, under my sending domain. (this is also how SPF and DKIM do things)

So I've put in this policy on the 4th of Feb:
_dmarc.hawaga.org.uk. 3583 IN TXT "v=DMARC1\;p=none\;rua=mailto:benc@hawaga.org.uk\;ruf=mailto:benc@hawaga.org.uk\;ri=3600"
That says to not enforce any policy, but to email benc@hawaga.org.uk with reports every 3600 seconds (= 1 hour).

I set this up at about 5pm on a Saturday and about 11am on Sunday morning my first report arrived, with a timestamp range of a day, which must extend back before I turned this on...

In there, three messages from my main outbound mail server, and no others.

Lets see what else I get...

A couple of weeks later...

I got daily reports most days from Google (I think maybe the day I didn't get a report was because I hadn't sent any mail into google all(?)).

A few days after the above I added in two other domains: my company domain with only occasionally sends mail, and my girlfriend's vanity domain. Neither of those have SPF or DKIM on them, even though they come from the same mail servers as hawaga.org.uk.

There was a noticeable lack of reports from anyone other than Google. I asked around (on Google+) to see if anyone had reports from elsewhere (eg AOL or Yahoo, because those were also listed) but no one said yes.

So what about the reports?

Well, there were surprisingly more mail servers than I expected: along with my own two outbound servers, there were about 10 other servers, being the outbound mail servers of a handful of research institutes that I work with. Those reports were tagged by google as being via a mailing list. Its not clear to me what defines a message as being via a mailing list, but I guess it would mean that they'll put less weight on my SPF records? It also highlights how a naive interaction between mailing lists and SPF can result in your message being treated as spam.

I also got some DKIM fails reported from my own legitimate mail server. The best I have been able to diagnose there is that I had sendmail set to deliver mail without a DKIM signature if the dkim milter timed out; but if that's going to contribute negatively to spam treatment, then I think a better configuration is to have the milter set to retry later, resulting in more delayed mail, but more DKIM-signed mail.

The extra domains I added had no DKIM on them, but those weren't treated as DKIM-fails. Instead they were reported as DKIM 'none'. I'm not sure what causes none rather than fail, but my guess is its something to do with the fact that hawaga.org.uk has DKIM records in its DNS, and thats being treated as an indication that there should be DKIM signatures on messages. I think that's extra meaning that I hadn't understood DKIM DNS records to mean.

I have a similar confusion with the interaction between SPF and DMARC: SPF has multiple output states, not just pass or fail, and its not clear to me how those are treated by DMARC.

Processingwise: the reports come as zipped XML documents. It was relatively straightforward to munge these like any other XML (though I made it harder for myself by learning a new Haskell XML library rather than using ones I already knew).

Its unclear to me how I know that a report really is from a particular sender, and what the threat model is for people injecting false DKIM reports - perhaps injecting them to suggest that people's use of DKIM and SPF is causing their mail to be dropped, and thus encouraging them to turn off SPF and DKIM?

So for now, I'll keep this switched on, in monitoring-only mode. I don't feel I understand it well enough to turn it on in enforcement mode (especially as I'm not the only user sending mail under hawaga.org.uk. I think its very interesting and probably useful to be able to specify policy this way; but the policy language at the moment feels either vaguely defined, or at the least not concisely described, in a way that makes me comfortable.

11 February, 2012

DKIM - domainkeys identified mail

Looks like I never wrote a blog posting on setting up DKIM. I just realised one of my servers wasn't set up after a re-install, so I'm having to remember how to do it again.

I'm using sendmail. (yes, shut up) and DKIM hooks in using its milter (mail filter) mechanism.

# apt-get install dkim-filter

Now wire it into sendmail.mc:
INPUT_MAIL_FILTER(`dkim', `S=/var/run/dkim-filter/dkim-filter.sock')

Now when mail comes in, you should see it gets headers like this added by your mail server (dildano.hawaga.org.uk in this case) when DKIM verification happens (eg in mail from gmail).:

Authentication-Results: dildano.hawaga.org.uk; dkim=pass (1024-bit key)
 header.i=@hawaga.org.uk; dkim-adsp=none

The other half of the equation is DKIM signing my outbound mail, so that other people who do checks like this can verify/not-verify my email.

DKIM needs a public/private keypaid

# dkim-genkey -b 1024 -d hawaga.org.uk -s hampshire

-s specifies a selector name. This is a fairly arbitrary identifier used to identify this keypair, because a domain can have multiple keypairs (for example, one per mail server). In the hawaga.org.uk domain, I seem to use names of English counties.

# ls
hampshire.private  hampshire.txt
# cat hampshire.txt
hampshire._domainkey IN TXT "v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDUP+5f0nEWyYICxr8rLN8xannlteBg4WF2Fat/MS8CiAa1lE2wgvhKYJJD/ydJ//5B9fBZAwSXTAq2ZCQYIfRf985Yip0BK80ECTlOunaSnMY/4/RzmkXGpndJaHIFqmSWDhML1yBP6W6owJDXIPDCAbV80kd5Z5aAkv8518lk+wIDAQAB" ; ----- DKIM hampshire for hawaga.org.uk

That .txt file is a DNS record to install under hawaga.org.uk. When you've installed it, you can check with:
dig -t txt hampshire._domainkey.hawaga.org.uk @localhost

That's the public key installed. Now the private key.

In /etc/dkim-filter.conf:

Domain hawaga.org.uk
KeyFile /etc/mail/hampshire.private
Selector hampshire

# /etc/init.d/dkim-filter restart

Now send out some mail to some other account. It should have a DKIM signature header added like this:

DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=hawaga.org.uk;
 s=hampshire; t=1328349841;
 bh=hGo8Oadbgx3cVNwLr3hGDRfMX5LwWwXuz2PzqEowx0I=;
 h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type;
 b=oBeSDSzxz7/awSnxuos6jyJuBoYH2MbiB3HDpbZfLQnTTdEJdx2WD0ubSVAaKAJmV
  ma5xuSaNGeS7X3Xg49obL6nWA89tiOeVAq9FO+7NP+v2DmUPFxEYkLeQJUANYKzAw/
  r8ag9XnbRkxvY+J/rrmeaAjJdnfgUQlKSHlV5CWE=

... and if that other account happens to do DKIM verification, you should see its version of:

Authentication-Results: paella.hawaga.org.uk; dkim=pass (1024-bit key) header.i=@hawaga.org.uk

16 April, 2011

spam and fidonet

I got a spam the other day, addressed to an address that isn't mine. That's not unusual. What was unusual was the address they used. It isn't mine, but it was mine when I was a teenager - benc@donor2.demon.co.uk was my internet address on DoNoR/2, an OS/2 focused BBS near Woking (in those days, young whippersnappers, you cared down to the town level where you were connecting to - DoNoR/2 was in dialing code 01483, the same as Guildford where I grew up). DoNoR/2 was primarily a fidonet system - 2:252/156, and then after the BS of geonetting, 2:440/4, and I was point 2 off that - Ben Clifford at 2:252/156.2

So although I'm usually annoyed when spam gets through my filters, this one made me think: aaah natsukashi.

Date: Thu, 27 Jan 2011 09:44:27 -0800 (PST)                                     
From: Jim Vivona                                            
To: benc@donor2.demon.co.uk                                                     
Subject: re                                                                     

sup! if you hadn't heard i lost my job at lawncare company about 5              
weeks ago, then i found this news article and made 379 in a few                 
hours!! I guess it was for the best! I learned from - News channel 4            
talk to you later!

or headers in full:

Return-Path:                                                
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on                      
dildano.hawaga.org.uk                                                       
X-Spam-Level:                                                                   
X-Spam-Status: No, score=-2.1 required=2.5 tests=BAYES_00,DKIM_SIGNED,          
DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,     
URIBL_BLACK autolearn=ham version=3.3.1                                     
X-Spam-ASN: AS36646 98.138.0.0/17                                               
X-Spam-tokens-ham: 0.000-57--100h-0s--0d--hadnt,                                
0.000-55--96h-0s--0d--hadn't,                                               
0.000-28--48h-0s--0d--lisa, 0.000-25--44h-0s--0d--Lisa,                     
0.001-19--32h-0s--0d--madison, 0.001-19--32h-0s--0d--Madison,               
0.001-18--31h-0s--0d--Wisconsin, 0.001-18--31h-0s--0d--wisconsin,           
0.001-9--15h-0s--0d--entrepreneurs, 0.002-124--220h-1s--0d--Fwd             
X-Spam-tokens-spam: 0.902-15425--22770h-653297s--0d--H*c:alternative,           
0.861-28--214h-4131s--0d--president,                                        
0.860-6821--59523h-1134787s--0d--H*Ad:D*uk                                  
X-Spam-relays-trusted:                             

X-Spam-relays-untrusted: [ ip=98.138.85.229                                     
rdns=web120502.mail.ne1.yahoo.com                                           
helo=web120502.mail.ne1.yahoo.com by=dildano.hawaga.org.uk ident=           
envfrom=                                                                    
intl=0 id=p0RHpCGr007860 auth= msa=0 ] [ ip=194.54.47.229 rdns= helo=       
by=web120502.mail.ne1.yahoo.com ident= envfrom= intl=0 id= auth= msa=0 ]    
X-Spam-dkim-identity: @yahoo.com vivonajj@yahoo.com                             
X-Spam-dkim-domain: yahoo.com                                                   
X-Spam-dccb: dcc1.aftenposten.no                                                
X-Spam-dccr: dildano.hawaga.org.uk 1215; Body=1 Fuz1=1 Fuz2=1                   
X-Spam-token-summary: Tokens: new, 1; hammy, 109; neutral, 84; spammy, 3.       
X-Spam-languages: en                                                            
X-Spam-autolearn: ham                                                           
Received: from web120502.mail.ne1.yahoo.com (web120502.mail.ne1.yahoo.com       
[98.138.85.229])                                                            
by dildano.hawaga.org.uk (8.13.8/8.13.8/Debian-3) with SMTP id              
p0RHpCGr007860                                                              
for ; Thu, 27 Jan 2011 17:51:14 GMT                     
Authentication-Results: dildano.hawaga.org.uk; dkim=pass (1024-bit key)         
header.i=@yahoo.com; dkim-adsp=none                                         
Received: (qmail 94875 invoked by uid 60001); 27 Jan 2011 17:44:27 -0000        
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024;     
t=1296150267; bh=niep53FNFb78lkhNFkO8MX4NIpOCMZRTAUTHm1+GJzQ=;              
h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Ver    
sion:Content-Type;                                                          
b=OWa2WgmhLxGRa+pJep7Or929UysoVIk/SCB7BnFfurvysB63Nr6Odfb4b3gm2hFZqK+xOo    
Q6aZmUXn27EQbeA/8av52fU1KV33uhA9Th6rI0uKEIPg5LikGLLXUoaXrLzGdL2qyJ10UTMy    
2TwEkqU6bZQBCpMxY0fWANKPiIK+M=                                              
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;                                
s=s1024; d=yahoo.com;                                                         
h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Versi    
on:Content-Type;                                                            
b=eIuzjdEYirBi9RTFb1voGAKkH4bQUYyRkN6NT6mDBMj2GJcnf8bKz7R7NrQuWd8j9tjoviMF    
bXv/t3D2DtcoNbvQuDSPMa6ycXDMUkNFpW3dMkyrq6ZBSuw+Ye7TZXH7ect5MJcErjTAyu38    
+Dx4kXmdFIlAhs3Q0CBs4L7t+EI=;                                               
Message-ID: <164580.94308.qm@web120502.mail.ne1.yahoo.com>                      
X-YMail-OSG: vT0rPzMVM1kHrbVNywVWrHu0pysHCRnEnLjmjnEWPDU8KWm                    
9AZTRLoDJBmvlFuAhJxD2.uKxh0LPBsJi.WmUQZBW_OOq9UKFRzUPfKAOemv                 
qni6KAcfSaxd8Y6p2Cf6w5PGXAILIpD_0UuPIQ3LtnYoffxsz6w1ytx9R3cG                   
BiqUC4MNLdG0NlSV2mlQCFGOEBHXYNTzl4ejOnjciku0Z1Y5SGW4aUz_4gmA                   
oWXR.IhJ4dfWwgEVx2H9GvVpuPLizb_vwjmqP1e02TP9qrN0cGC1tFWTbyW9                   
ooJoemIjAwkiYhhY-                                                              
Received: from [194.54.47.229] by web120502.mail.ne1.yahoo.com via HTTP; Thu    
, 27 Jan 2011 09:44:27 PST                                                  
X-Mailer: YahooMailRC/553 YahooMailWebService/0.8.107.285259                    
Date: Thu, 27 Jan 2011 09:44:27 -0800 (PST)                                     
From: Jim Vivona                                            
Subject: re                                                                     
To: benc@donor2.demon.co.uk                                                     
MIME-Version: 1.0                                                               
Content-Type: multipart/alternative;                                            
boundary="0-588940240-1296150267=:94308"                                    
X-Greylist: Delayed for 00:06:40 by milter-greylist-3.0                         
(dildano.hawaga.org.uk [81.187.211.37]); Thu,                               
27 Jan 2011 17:51:15 +0000 (GMT)

Ben Clifford Technical Blog