Peer-to-Peer Communication Across Network Address Translators

Bryan Ford – M.I.T.

Pyda Srisuresh – Caymas Systems

Dan Kegel – Ixia Communications

J'fais des trous, des petits trous...
toujours des petits trous

– S. Gainsbourg

USENIX – April 14, 2005

Home

NAT Router

Network Address Translation (NAT)

Home

Network

Public Internet

Home

NAT Router

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Home

NAT Router

Private Endpoint

(BIP:Bport)

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Home

NAT Router

Private Endpoint

(BIP:Bport)

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Temporary

Public Endpoint

(TBIP:TBport)

Home

NAT Router

Private Endpoint

(BIP:Bport)

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Address

Translation

Home

NAT Router

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Home

NAT Router

Private Endpoint

(BIP:Bport)

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Home

NAT Router

Private Endpoint

(BIP:Bport)

Public Endpoint

(AIP:Aport)

Public Endpoint

(SIP:Sport)

Network Address Translation (NAT)

Home

Network

Public Internet

Client A

Server S

Client A

Client B

Home

NAT Router

Network Address Translation (NAT)

ISP-Private Network

Home

Network

ISP-deployed

NAT

Public Internet

Public IP Addresses

Private IP Addresses

Home

NAT Router

Network Address Translation (NAT)

NAT

NAT

Home

Network

ISP-Private Network

Home

Network

ISP-deployed

NAT

Public Internet

Public IP Addresses

Private IP Addresses

Home

Network

Home

NAT Router

Network Address Translation (NAT)

NAT

NAT

Home

Network

ISP-Private Network

Home

Network

ISP-deployed

NAT

Public Internet

Public IP Addresses

Private IP Addresses

Home

Network

?

?

Demand for P2P Communication

Many compelling apps need P2P communication, not just “P2P apps”:

Teleconferencing, Voice over IP (VoIP)

Multiplayer on-line games

Remote access/administration (e.g., ssh)

Outline

The NAT Traversal Problem

UDP Hole Punching (not new)

TCP Hole Punching (quite new)

Multi-Level NAT Scenarios

NAT Compatibility with Hole Punching

Related Work

UDP Hole Punching

Usage model assumptions:

Clients register with public “rendezvous server”
to become accessible to other clients

Application implements notion of “identity”

Username, public key [HIP], etc.

Rendezvous server facilitates P2P session setup,
but does not participate in resulting P2P sessions

UDP Hole Punching

NAT

Home

Network

Public Internet

Rendezvous Server S

NAT

Home

Network

Client A

Client B

UDP Hole Punching

(SIP:Sport)

NAT

Home

Network

Public Internet

Rendezvous Server S

NAT

Home

Network

Client A

Client B

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Home

Network

Public Internet

Rendezvous Server S

(BIP:Bport)

NAT

Home

Network

Client A

Client B

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session B-S

(TBIP:TBport) Û (SIP:Sport)

Session A-S

(TAIP:TAport) Û (SIP:Sport)

Session A-S

(AIP:Aport) Û (SIP:Sport)

Session B-S

(BIP:Bport) Û (SIP:Sport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

“Help me reach B”

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

“B is at (TBIP:TBport)”

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

“B is at (TBIP:TBport)”

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

“B is at (TBIP:TBport)”

“A is at (TAIP:TAport)”

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(BIP:Bport) Û (TAIP:TAport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session B-A

(TBIP:TBport) Û (TAIP:TAport)

Session A-B

(BIP:Bport) Û (TAIP:TAport)

Session A-B

(TAIP:TAport) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-B

(BIP:Bport) Û (TAIP:TAport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching Gone Wrong

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-S

(TAIP:TAport) Û (SIP:Sport)

Session A-S

(AIP:Aport) Û (SIP:Sport)

Session A-B

(TA2IP:TA2port) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching Gone Wrong

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-S

(TAIP:TAport) Û (SIP:Sport)

Session A-S

(AIP:Aport) Û (SIP:Sport)

Session A-B

(TA2IP:TA2port) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching Gone Wrong

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-S

(TAIP:TAport) Û (SIP:Sport)

Session A-S

(AIP:Aport) Û (SIP:Sport)

Session A-B

(TA2IP:TA2port) Û (TBIP:TBport)

(TBIP:TBport)

(TAIP:TAport)

UDP Hole Punching Gone Wrong

(AIP:Aport)

(SIP:Sport)

NAT

Rendezvous Server S

(BIP:Bport)

NAT

Client A

Client B

Session A-B

(AIP:Aport) Û (TBIP:TBport)

Session A-S

(TAIP:TAport) Û (SIP:Sport)

Session A-S

(AIP:Aport) Û (SIP:Sport)

TCP Hole Punching

TCP has always supported crucial feature

“Simultaneous TCP Open” [RFC 793]

Difficulties:

More ways for NATs to behave poorly

TCP sockets API oriented toward client/server

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(SIP:Sport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

“Help me reach B”

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

“B is at (TBIP:TBport)”

“A is at (TAIP:TAport)”

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

Connect Socket to B

SYN

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

SYN

Connect Socket to A

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

SYN

Connect Socket to A

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

SYN

Connect Socket to A

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

SYN

Connect Socket to A

The Magic Socket Option:

SO_REUSEADDR

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

Connect Socket to A

(BIP:Bport)

(SIP:Sport)

ACK

“Simultaneous TCP Open”

SYN

SYN

ACK

ACK

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

Connect Socket to A

(BIP:Bport)

(SIP:Sport)

ACK

ACK

“Simultaneous TCP Open”

SYN

SYN

ACK

ACK

(TBIP:TBport)

(TAIP:TAport)

TCP Hole Punching

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

Connect Socket to A

(BIP:Bport)

(SIP:Sport)

“Simultaneous TCP Open”

SYN

SYN

ACK

ACK

(TBIP:TBport)

(TAIP:TAport)

Timing Caveat

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

(TBIP:TBport)

(TAIP:TAport)

Timing Caveat

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

(TBIP:TBport)

(TAIP:TAport)

Timing Caveat

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to B

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

SYN

RST

(TBIP:TBport)

(TAIP:TAport)

Timing Solution

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

Listen Socket

Listen Socket

(TBIP:TBport)

(TAIP:TAport)

Timing Solution

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

Listen Socket

Listen Socket

SYN

Connect Socket to B

(TBIP:TBport)

(TAIP:TAport)

Timing Solution

NAT

Rendezvous Server S

NAT

Client A

Client B

(AIP:Aport)

Connect Socket to S

Connect Socket to S

(BIP:Bport)

(SIP:Sport)

Listen Socket

Connect Socket to B

Listen Socket

“Normal TCP Open”

SYN

SYN-ACK

ACK

TCP Hole Punching Gone Wrong

Potential problems:

Inconsistent endpoint translation

Same as for UDP

NAT could reject “unsolicited” incoming SYNs
with RSTs or ICMP errs instead of just dropping

Connection failures, retry oscillation

Buggy TCP state machine in host OS

Windows before XP SP2

Home

NAT

Home

NAT

Multi-Level NAT

Home

Network

ISP-Private Network

ISP-deployed

NAT

Public Internet

Home

Network

Home

NAT

Home

NAT

Multi-Level NAT

Home

Network

ISP-Private Network

ISP-deployed

NAT

Public Internet

Home

Network

Home

NAT

Home

NAT

Multi-Level NAT

Home

Network

ISP-Private Network

ISP-deployed

NAT

Public Internet

Home

Network

Multi-Level NAT

Home

NAT

Home

NAT

Home

Network

ISP-Private Network

ISP-deployed

NAT

Public Internet

Home

Network

Hairpin

Translation

NAT Check

Tests hole punching “end-to-end” from user's host

Results reflect composition of all NAT(s) in path

No isolation of contention-related “bad” behaviors

No tests for “bad but semi-predictable” behaviors

More detailed tests of specific NATs elsewhere
[Jennings–STUN, Guha–STUNT]

http://midcom-p2p.sourceforge.net/

Data Collection

Results submitted over Web by (self-selecting) community of volunteers

UDP: 380 data points

TCP: 286 data points

Covers

NAT router hardware from 68 vendors

NAT support in 8 popular operating systems

(Breakdown by vendor in paper)

Testing Results

UDP Hole Punching

82% of NATs support

Most common NATs:

Linksys	98% (45/46)

Netgear	84% (31/37)

Windows	94% (31/33)

Linux	81% (26/32)

Hairpin: 24%

TCP Hole Punching

64% of NATs support

Most common NATs:

Linksys	87% (33/38)

Windows	52% (16/31)

Netgear	63% (19/30)

Linux	67% (16/24)

Hairpin: 13%

Related Work

UDP hole punching: [Kegel 1999]

Voice over IP: SIP/ICE [Rosenberg 2003]

Asymmetric TCP hole punching

NUTSS, NATBLASTER, NatTrav

Sometimes compensate for bad NAT behaviors,
but more complex, timing-sensitive

Proxy protocols

SOCKS, RSIP, MIDCOM, UpnP
require explicit NAT support, user setup

Conclusion

NAT is evil, but is here to stay

Hole punching enables practical, automatic traversal of majority of existing NATs

Compatibility good for UDP, tolerable for TCP, increasing with NAT vendor awareness
(hint, hint)

2005-04-08T14:08:13