Golang : Get SPF and DMARC from email headers to fight spam
Problem:
You are building an email server with spam filter from scratch with Golang and you want to query a domain's SPF(Sender Policy Framework) and DMARC(Domain-based Message Authentication, Reporting and Conformance) record. How to get a domain's SPF and DMARC records?
NOTES:
An SPF record is to prevent spammers from sending messages with forged From addresses at your domain. Recipients or spam filter program can refer to the SPF record to determine whether a message purporting to be from your domain comes from an authorized mail server.
DMARC is an email validation system designed to detect and prevent email spoofing. It is built on top of Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM)
Solution:
A proper email has header information that you can use to extract the SPF and DMARC records of the domain. In the email headers, look for the From: and extract the domain name. For example, for spammer@hello.com, we only want hello.com.
Query the domain name with the code example below.
An SPF record is a type of DNS record that shows which mail servers are allowed to send email on behalf of a domain. For example, customers of sengrid.com usually have:
include:sendgrid.net
in their SPF records. It simply means that sendgrid.net is authorized to send email on behalf of the customer. For example, email from testingmom.com will have sendgrid.net in the From: part in the headers.
Not all domains have DMARC records in their DNS TXT. However, it is good to query for the domain's DMARC as well. For example, Foursquare.com is a customer of sendgrid.com and it has DMARC configured in their DNS.
v=DMARC1; p=none; rua=mailto:dmarc@foursquare.com
The code that follows will show you how to query a domain for their DNS records with net.Lookup() function, extract the SPF and DMARC records and convert the DMARC record into a map(hash table) for fast lookup.
With the SPF and DMARC information, you can configure the spam filter to allow an email through, reject it or mark it as spam and route to a spam box.
Here you go!
emailheader.go
package main
import (
"fmt"
"net"
"os"
"strings"
)
func ExtractDMARC(records []string) string {
for _, value := range records {
if strings.Contains(value, "v=DMARC1") {
return value
}
}
// default
return "no DMARC found"
}
func ExtractSPF(records []string) string {
for _, value := range records {
if strings.Contains(value, "v=spf1") {
return value
}
}
// default
return "no SPF found"
}
func main() {
if len(os.Args) != 2 {
fmt.Printf("Usage : %s <domain to query> \n", os.Args[0])
os.Exit(0)
}
domain := os.Args[1]
fmt.Printf("Getting %s DNS records...\n", domain)
// extract DNS records for the target domain
dnsTXT, err := net.LookupTXT(domain)
if err != nil {
fmt.Println("unable to query domain. Error : " + err.Error())
os.Exit(-1)
}
// sanity check
for index, record := range dnsTXT {
fmt.Printf("[%d] : %s\n", index, record)
}
spf := ExtractSPF(dnsTXT)
fmt.Printf("\n\nSPF record : %s\n", spf)
// extract DMARC records by adding "_dmarc" in the LookupTXT query
// DMARC records are published in DNS with a subdomain label _dmarc
dmarcTXT, err := net.LookupTXT("_dmarc." + domain)
if err != nil {
fmt.Println("unable to query DMARC. Error : " + err.Error())
os.Exit(-1)
}
dmarc := ExtractDMARC(dmarcTXT)
fmt.Printf("\n\nDMARC record : %s\n", dmarc)
// break down the DMARC name value tags to slice first and append to a map(hash table)
dmarcSlice := strings.Fields(dmarc)
//fmt.Println(dmarcSlice)
// from https://en.wikipedia.org/wiki/DMARC
// v is the version, p is the policy, sp the subdomain policy,
// pct is the percent of "bad" emails on which to apply the policy,
// and rua is the URI to send aggregate reports to.
dmarcMap := make(map[string]interface{})
for _, value := range dmarcSlice {
// break value by = sign
tmp := strings.Split(value, "=")
dmarcMap[tmp[0]] = tmp[1]
}
// it is up to you on what to do with these data
// store them into database
// use the information to configure spam filter
fmt.Println("DMARC version : ", dmarcMap["v"])
fmt.Println("DMARC policy : ", dmarcMap["p"])
fmt.Println("DMARC subdomain policy : ", dmarcMap["sp"])
fmt.Println("DMARC bad percentage : ", dmarcMap["pct"])
fmt.Println("DMARC send reports to : ", dmarcMap["rua"])
}
Build and run the code.
Sample output for ./emailheader sendgrid.com :
Getting sendgrid.com DNS records...
[0] : loaderio=dc99536e3eb902ffc82885541e224e7c
[1] : MS=ms77173225
[2] : v=spf1 ip4:167.89.32.5 ip4:167.89.32.50 ip4:50.31.36.199 ip4:50.31.36.205 ip4:50.31.36.208 ip4:50.31.36.213 ip4:50.31.36.197 ip4:167.89.25.84 include:_spf.google.com include:partners.sendgrid.com include:_labs.sendgrid.com -all
[3] : google-site-verification=ou-V2e37_e-apeuJQ4IZh8VPHGkwMHCeehxZiVJUQ_s
SPF record : v=spf1 ip4:167.89.32.5 ip4:167.89.32.50 ip4:50.31.36.199 ip4:50.31.36.205 ip4:50.31.36.208 ip4:50.31.36.213 ip4:50.31.36.197 ip4:167.89.25.84 include:_spf.google.com include:partners.sendgrid.com include:_labs.sendgrid.com -all
DMARC record : v=DMARC1; p=quarantine; sp=none; adkim=r; aspf=r; fo=1; pct=100; rf=afrf; ri=86400; rua=mailto:sendgrid@rua.agari.com,mailto:dmarc@sendgrid.com; ruf=mailto:sendgrid@ruf.agari.com,mailto:dmarc@sendgrid.com
[v=DMARC1; p=quarantine; sp=none; adkim=r; aspf=r; fo=1; pct=100; rf=afrf; ri=86400; rua=mailto:sendgrid@rua.agari.com,mailto:dmarc@sendgrid.com; ruf=mailto:sendgrid@ruf.agari.com,mailto:dmarc@sendgrid.com]
DMARC version : DMARC1;
DMARC policy : quarantine;
DMARC subdomain policy : none;
DMARC bad percentage : 100;
DMARC send reports to : mailto:sendgrid@rua.agari.com,mailto:dmarc@sendgrid.com;
and sample output for ./emailheader digitalocean.com :
Getting digitalocean.com DNS records...
[0] : v=spf1 ip4:167.89.16.30/32 ip4:167.89.16.245/32 ip4:167.89.16.183/32 include:servers.mcsv.net include:mailgun.org include:_spf.google.com -all
[1] : mailru-verification: 93fb13cddf97475f
[2] : google-site-verification=l_eZFhD-V_qPB1OYMoA4-Qg1QJT0oQm6j1Z1Dl_f65k
[3] : globalsign-domain-verification=QCdmvx2FmvCpeqYXQsTAMJMq2YjyK0VzLosSdN6aMP
SPF record : v=spf1 ip4:167.89.16.30/32 ip4:167.89.16.245/32 ip4:167.89.16.183/32 include:servers.mcsv.net include:mailgun.org include:_spf.google.com -all
DMARC record : v=DMARC1; p=reject; pct=100; rua=mailto:fdpfb1lo@ag.dmarcian.com; ruf=mailto:fdpfb1lo@fr.dmarcian.com
[v=DMARC1; p=reject; pct=100; rua=mailto:fdpfb1lo@ag.dmarcian.com; ruf=mailto:fdpfb1lo@fr.dmarcian.com]
DMARC version : DMARC1;
DMARC policy : reject;
DMARC subdomain policy : <nil>
DMARC bad percentage : 100;
DMARC send reports to : mailto:fdpfb1lo@ag.dmarcian.com;
Hope this helps and happy coding!
References:
https://www.socketloop.com/references/golang-net-lookuptxt-function-example
https://en.wikipedia.org/wiki/DMARC
https://sendgrid.com/resources/cases/
https://support.google.com/a/answer/33786?hl=en
https://www.codegists.com/snippet/go/emailheadergoamlwwalkergo
See also : Golang : Auto-generate reply email with text/template package
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+5.2k Golang *File points to a file or directory ?
+11.1k Use systeminfo to find out installed Windows Hotfix(s) or updates
+5.8k Golang : Generate multiplication table from an integer example
+15.4k Golang : Validate hostname
+20.9k Golang : How to force compile or remove object files first before rebuild?
+21.6k Golang : Use TLS version 1.2 and enforce server security configuration over client
+7.1k Golang : How to detect if a sentence ends with a punctuation?
+46.1k Golang : Encode image to base64 example
+21.9k Golang : Match strings by wildcard patterns with filepath.Match() function
+13.8k Golang : Fix cannot use buffer (type bytes.Buffer) as type io.Writer(Write method has pointer receiver) error
+14k Golang : syscall.Socket example
+5.9k PageSpeed : Clear or flush cache on web server