Sender Policy Framework
Online Advertising
Sender Policy Framework
In
computing, Sender Policy Framework (SPF) is an
extension to the
Simple Mail
Transfer Protocol (SMTP). SPF allows to identify and reject forged
addresses in the SMTP MAIL FROM (Return-Path), a typical nuisance in
e-mail spam. SPF is defined in the
SPF Classic specification.
Principles of Operation
Normal SMTP allows any computer to send e-mail claiming to be from anyone.
Thus, it's easy for
spammers
to send
e-mail from forged addresses. This makes it difficult to trace back to where the
spam truly comes from, and easy for spammers to appear to be senders the
receiver would ordinarily trust. Many believe that the ability for anyone to
forge Return-Path addresses is a security flaw in modern SMTP, caused by an undesirable side-effect of the deprecation of source
routes.
SPF allows the owner of an Internet domain to use special
DNS records to specify which machines are authorized to transmit e-mail for
that domain. For example, the owner of the example.org domain can designate
which machines are authorized to send e-mail whose e-mail address in the
Return-Path ends with "@example.org". Receivers checking SPF can then reject any
e-mail that claims to come from that domain, but fails in a check against the
IPs listed in the sender policy of this domain.
SPF protects the address in the Return-Path, that is the address to which
bounces would be sent if the mail is not delivered. While the address in the
Return-Path often matches other originator addresses in the mail header like
"From:" or "Sender:" this is not necessarily the case, and SPF does not prevent
forgeries of these other addresses.
Spammers can send e-mail with an SPF PASS result if they have an account in a
domain with a sender policy, or abuse a compromised system in this domain.
However, doing so makes the spam easier to trace and prosecute. An SPF PASS
result from unknown strangers still guarantees that auto-replies like error
messages (bounces) cannot hit innocent bystanders.
The main benefit of SPF is to people whose e-mail addresses are forged in the
Return-Paths. They receive a large mass of unsolicited error messages and other
auto-replies, making it difficult to use e-mail normally. If such people use SPF
to specify their legitimate sending IPs with a FAIL result for all other IPs,
then receivers checking SPF can reject forgeries, already reducing the amount of
back-scatter. More important spammers knowing their trade will avoid to forge
SPF FAIL protected addresses, because they want to reach as many of their
primary victims as possible - there are more than enough unprotected addresses
for this abuse.
SPF has potential advantages beyond helping identify unwanted e-mail. In
particular, if a sender provides SPF information, then receivers can use SPF
PASS results in combination with a white list to identify known reliable
senders. Scenarios like compromised systems and shared sending mailers limit
this use, but at least auto-replies cannot hit innocent bystanders.
FAIL and forwarding
If a domain publishes an SPF FAIL policy, then legit mails sent to receivers
forwarding their mail to third parties can be rejected and bounced if (1) the
forwarder doesn't rewrite the Return-Path unlike mailing lists, (2) the next hop
doesn't white list the forwarder, and (3) this hop checks SPF. This is an
intentional and obvious feature of SPF - checks behind the "border"
MTA ( MX ) of the receiver can't work directly.
Publishers of SPF FAIL policies must accept this potential problem, they
cannot expect that all receivers change their forwarding arrangements, i.e.
clear at least one of the three critical conditions.
A technique called
Sender Rewriting Scheme (SRS) is a way for mail forwarding services to avoid
this problem.
HELO tests
For an empty Return-Path as used in
error messages and other auto-replies SPF checks the HELO-identity. Actually
it checks an artificial postmaster@mail.example.org address for a HELO
or EHLO mail.example.org.
With a bogus HELO identity the result NONE won't help, but for valid host
names SPF also protects the HELO identity. This SPF feature was always supported
as option for receivers, and later SPF drafts including the final specification
recommend to check the HELO always.
This allows to white list sending mailers based on a HELO PASS, or to reject
all mails after a HELO FAIL. It can be also used in reputation systems (any
white or black list is a simple case of a reputation system).
Implementation
Implementing SPF has two parts:
- Domains identify the machines authorized to send e-mail on their behalf.
Domains do this by adding an additional record to their existing DNS
information.
- Receivers can request and use SPF information. They use ordinary DNS
queries, which are typically cached to enhance performance. Receivers then
interpret the SPF information as specified and act upon the result.
Thus, the key issue in SPF is the specification for the new DNS information
that domains set and receivers use. The records are laid out like this (in
typical DNS-syntax):
example.org. IN TXT "v=spf1 a mx -all"
"v=" defines the version of SPF used, the following words provide
mechanisms to use to determine if a domain is eligible to send mail. The "a"
and "mx" specify the systems permitted to send messages for the given domain.
The "-all" at the end specifies that, if the previous mechanisms did not
match, the message should be rejected.
Mechanisms
Eight mechanisms are defined:
ALL |
Matches always, used for a default result like -all for all
IPs not matched by prior mechanisms. |
A |
If the domain name has an A (or AAAA for IPv6 )
record corresponding to the sender's address, it will match. (That is,
the mail comes directly from the domain name.) |
IP4 |
If the sender is in a given
IPv4 range,
match. |
IP6 |
If the sender is in a given
IPv6 range,
match. |
MX |
If the domain name has an
MX
record resolving to the sender's address, it will match. (That is,
the mail comes from one of the domain's mail servers) |
PTR |
If the sender
reverse-resolves to a name ending in the domain, and that name
resolves to the sender's IP address, match. |
EXISTS |
If the given domain resolves, match (no matter the address it
resolves to). This is rarely used, along with the SPF macro language it
offers more complex matches like
DNSBL-queries. |
INCLUDE |
If the included (a misnomer) policy passes the test this
mechanism matches. This is typically used to include policies of
more than one
ISP. |
Qualifiers
Each mechanism can be combined with one of four qualifiers:
- + for a PASS result, this can be omitted, +mx
is the same as mx.
- ? for a NEUTRAL result interpreted like NONE (no
policy).
- ~ for SOFTFAIL, a debugging aid between NEUTRAL and
FAIL.
- - for FAIL, the mail should be rejected (see below).
Modifiers
The modifiers allow for future extensions of the framework. So far
only the two modifiers defined in the
RFC are widely
deployed.
exp=some.example.com gives the name of a domain with a
DNS TXT record, which is interpreted using SPF's macro language to get an
explanation for FAIL results, typically an URL, added to the SMTP error code. This baroque feature is rarely used.
redirect=some.example.com can be used instead of the ALL-mechanism
to link to the policy record of another domain. This modifier is easier
to understand than the somewhat similar INCLUDE-mechanism.
Error handling
As soon as SPF implementations detect syntax errors in a sender policy they
must abort the evaluation with result PERMERROR. Skipping erroneous
mechanisms cannot work as expected, therefore include:bad.example
and redirect=bad.example also cause a PERMERROR.
Another safety guard is the maximum of ten mechanisms querying
DNS, i.e. any mechanism except from IP4, IP6, and ALL. Implementations
can abort the evaluation with result SOFTERROR when it takes too long or a DNS
query times out, but they must return PERMERROR if the policy directly or
indirectly needs more than ten queries for mechanisms, any
redirect= also counts towards this processing limit.
A typical SPF HELO policy v=spf1 a -all needs three DNS queries: (1)
TXT, (2) SPF, and (3) A or AAAA. This last query counts as the first
mechanism towards the limit (10), in this example it's also the last,
because ALL needs no DNS lookup.
Caveats
SPF normally only validates the domain of the envelope sender (in the
Return-Path). Domains that share mail senders (e.g. with virtual hosting) can
forge each others' domain. SPF does not validate that a given e-mail actually
comes from the claimed user, because it operates at the network level.
SPF FAIL rejection
SPF FAIL policies can be an effective but dangerous tool. Some publishers of
SPF policies try to avoid the dangers by using SOFTFAIL (designed for limited
testing periods) instead of FAIL. But SOFTFAIL can be even more dangerous than
FAIL with receivers rejecting FAIL and accepting SOFTFAIL tagged as potential
junk.
A reject in a forwarding scenario is a clean decision, the forwarder will
send an error message to the address in the Return-Path. Typically the
error message (bounce) contains an explanation of the SMTP error and the
failing (forwarded to) address. The original sender can then send his mail again
directly to this address bypassing the forwarder, a crude emulation of the
normal SMTP error code 551 user not local.
However an accepted SOFTFAIL tagged as potential junk could be deleted
by the final recipient. This is a user who has arranged his forwarding in a way
that cannot work with SPF, this user could be also careless with checking his
potential junk and simply delete it.
The same line of arguments also suggests that receivers should take the SPF
recommendation to reject SPF FAIL seriously where possible. Accepting SPF FAIL
results as potential junk can be more dangerous than simply rejecting
FAILed mails. While senders with an SPF FAIL can be expected to know what it
means, the same is obviously not the case for a receiver arrranging his
forwarding in a way that cannot work with SPF.
Checkpoints
Checking SPF behind the "border"
MTA ( MX
) is not impossible, the relevant info is usually noted in a Received
timestamp line added by one of the MXs of the receiver. But at this time it's
too late to reject SPF FAIL, the checking entity can essentially only delete
FAILing mail.
Experts should be able to get this right, but it's no plug and play
situation, therefore the SPF specification recommends to check SPF only at the
"border" (MX) in the SMTP session, not later. Later SPF checks can also have
unexpected results if the publishers of sender policies don't plan modifications
of their policy carefully with regard to
DNS cache expiration.
History
The SPF concept was presented at
YAPC and OSCON (O'Reilly Open Source Convention) in 2003, in a short paper
titled "Repudiating Mail-From" written by Paul Vixie in 2002. Other predecessors
were "Reverse MX" by Hadmut Danisch, and "Designated Mailer Protocol" by Gordon Fecyk.
In June 2003,
Meng Weng Wong merged the RMX and DMP specifications and solicited
suggestions from other programmers. Over the next six months, a large number of
changes were made and a large community had started working on SPF.
Originally SPF stood for Sender Permitted From and was sometimes also
called SMTP+SPF, but it was changed to Sender Policy Framework in February 2004.
In early 2004, the IETF created the MARID working group and tried to use SPF
and Microsoft's CallerID proposal as the basis for what is now known as Sender
ID.
After the collapse of
MARID the SPF community returned to the original "classic" version of SPF. In
July 2005 this version of the specification was approved by the IESG as an IETF
experiment. It is expected to get its RFC number in early 2006.
Controversy
Steven M. Bellovin has written a long e-mail dated January 5, 2004,
discussing some of his concerns with SPF. Some of these include:
- SPF uses TXT records in DNS, which are supposed to be free-form text
with no semantics attached. SPF proponents readily acknowledge that it would
be better to have records specifically designated for SPF, but this choice
was made to enable rapid implementation of SPF. In July 2005,
IANA assigned the Resource Record type 99 to SPF. SPF publishers may
publish both record types and SPF checkers may check for either types. It
will likely take many years before all DNS software fully supports this new
record.
- As of the time he wrote his message, there was no consensus that this is
the right way to go. Some major e-mail service providers have not bought
into this scheme. Unless and until they do, it doesn't help much, either for
their customers (who make up a substantial proportion of the user
population) or for everyone else (since their addresses could be forged).
It's worth noting that since this concern was raised, among others AOL has
embraced SPF.
- Bellovin's strongest concerns involve the underlying assumptions of SPF
(SPF's "semantic model"). When using SPF, the SPF DNS records determine how
a sender is allowed to send. That means that the owner of the domain will
control how senders are allowed to send. People who use "portable" e-mail
addresses (such as e-mail addresses created by professional organizations)
will be required to use the domain owner's SMTP sender, which may not
currently even exist. Organizations providing these "portable" addresses
could, however, create their own Mail Submission Agents (MSAs) (RFC
2476 bis) or offer VPNs. Besides SPF only ties the SMTP Return-Path to permitted MSAs, users are still free to use their
RFC 2822 addresses elsewhere.
Deployment
In spite of its limitations, many people have decided that the pros of
SPF outweigh its cons and have begun implementing SPF. Anti-spam products such
as SpamAssassin version 3.0.0 implement SPF. Many mail transfer agents (MTAs)
support SPF directly such as Wildcat, or have patches/plug-ins available that
support SPF, including Postfix, Sendmail, Exim, and Qmail. Many prominent
domains decided to post SPF data for their domains as of mid-2004, including
Amazon, AOL, EBay, Google, GMX, Hotmail, and W3C.
See also
External links
Home | Up | e-Mail spammers | Spam bait | Word salad | Spamvertising | DNSBL | The Abusive Hosts Blocking List | e-Mail authentication | Sender Policy Framework | Open mail relay | Boulder Pledge
Online Advertising, made by MultiMedia | Free content and software
This guide is licensed under the GNU
Free Documentation License. It uses material from the Wikipedia.
|