Sunday, October 26, 2008
Sunday, October 12, 2008
SIP Strict versus Loose Routing
The way how record routing works has evolved. Record routing according to RFC2543 rewrote the Request-URIii. That means the Request-URI always
contained URI of the next hop (which can be either next proxy server which inserted Record-Route header field or destination user agent). Because of that it was necessary to save the original Request-URI as the last Route header field. This approach is called strict routing.
Loose routing, as specified in RFC3261, works in a little bit different way. The Request-URI is no more overwritten, it always contains URI of the destination user agent. If there are any Route header field in a message, than the message is sent to the URI from the topmost Route header field. This is significant change--Request-URI doesn't necessarily contain URI to which the request will be sent. In fact, loose routing is very similar to IP source routing.
Because transit from strict routing to loose routing would break backwards compatibility and older user agents wouldn't work, it is necessary to make loose routing backwards compatible. The backwards compatibility unfortunately adds a lot of overhead and is often source of major problems.
contained URI of the next hop (which can be either next proxy server which inserted Record-Route header field or destination user agent). Because of that it was necessary to save the original Request-URI as the last Route header field. This approach is called strict routing.
Loose routing, as specified in RFC3261, works in a little bit different way. The Request-URI is no more overwritten, it always contains URI of the destination user agent. If there are any Route header field in a message, than the message is sent to the URI from the topmost Route header field. This is significant change--Request-URI doesn't necessarily contain URI to which the request will be sent. In fact, loose routing is very similar to IP source routing.
Because transit from strict routing to loose routing would break backwards compatibility and older user agents wouldn't work, it is necessary to make loose routing backwards compatible. The backwards compatibility unfortunately adds a lot of overhead and is often source of major problems.
Saturday, October 11, 2008
SIP layered protocol
(1)The lowest layer of SIP is its syntax and encoding. Its encoding is specified using an augmented Backus-Naur Form grammar (BNF).
(2)The second layer is the transport layer.
(3)The third layer is the transaction layer.
(4)The layer above the transaction layer is called the transaction user(TU). Each of the SIP entities, except the stateless proxy, is a transaction user.
(2)The second layer is the transport layer.
(3)The third layer is the transaction layer.
(4)The layer above the transaction layer is called the transaction user(TU). Each of the SIP entities, except the stateless proxy, is a transaction user.
Friday, October 10, 2008
SIP Record-Route
In some cases, it may be useful for proxies in the SIP signaling path to see all the messaging between the endpoints for the duration of the session.
For example, if the biloxi.com proxy server wished to remain in the SIP messaging path beyond the initial INVITE, it would add to the INVITE a required routing header field known as Record-Route that contained a URI resolving to the hostname or IP address of the proxy.
This capability is frequently used for proxies that are providing mid-call features.
A route set is a collection of ordered SIP or SIPS URI which represent a list of proxies that must be traversed when sending a particular request. A route set can be learned, through headers like Record-Route, or it can be configured.
For example, if the biloxi.com proxy server wished to remain in the SIP messaging path beyond the initial INVITE, it would add to the INVITE a required routing header field known as Record-Route that contained a URI resolving to the hostname or IP address of the proxy.
This capability is frequently used for proxies that are providing mid-call features.
A route set is a collection of ordered SIP or SIPS URI which represent a list of proxies that must be traversed when sending a particular request. A route set can be learned, through headers like Record-Route, or it can be configured.
SIP Via vs Contact
While the Via header field tells other elements where to send the response, the Contact header field tells other elements where to send future requests.
SIP dialog
http://tools.ietf.org/rfc/rfc3261.txt
A dialog is a peer-to-peer SIP relationship between two UAs that persists for some time. A dialog is established by SIP messages, such as a 2xx response to an INVITE request. A dialog is identified by a call identifier, local tag, and a remote tag. A dialog was formerly known as a call leg in RFC 2543.
The combination of the To tag, From tag, and Call-ID completely defines a peer-to-peer SIP relationship between caller and callee and is referred to as a dialog.
A dialog is a peer-to-peer SIP relationship between two UAs that persists for some time. A dialog is established by SIP messages, such as a 2xx response to an INVITE request. A dialog is identified by a call identifier, local tag, and a remote tag. A dialog was formerly known as a call leg in RFC 2543.
The combination of the To tag, From tag, and Call-ID completely defines a peer-to-peer SIP relationship between caller and callee and is referred to as a dialog.
Friday, June 6, 2008
netbeans java.lang.AbstractMethodError: org. customer .xerces.dom.DeferredDocumentImpl.setDocumentURI(Ljava/lang/String;)V
Context:
when running Netbeans under JDK 1.5.X or later, when one jsp file is opened and being saved, following exception is thrown:
java.lang.AbstractMethodError: org. customer.xerces.dom.DeferredDocumentImpl.setDocumentURI(Ljava/lang/String;)V
o make a long story short:
- Method
public void setDocumentURI(String documentURI);
was added to the org.w3c.dom.Document interface
in J2SE 1.5 (it was not present in J2SE 1.4).
- The new method is implemented by
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl
(in J2SE 1.5), but not by
org.apache.xerces.dom.DeferredDocumentImpl
in xercesImpl.jar
- javax.xml.parsers.DocumentBuilderFactory in J2SE 1.5 will instantiate
factories of type "org.apache.xerces.jaxp.DocumentBuilderFactoryImpl"
for webapps bundling xercesImpl.jar, due to the presence of the
META-INF/services/javax.xml.parsers.DocumentBuilderFactory resource
in that JAR, which specifies
"org.apache.xerces.jaxp.DocumentBuilderFactoryImpl" as the factory
type to be instantiated.
Instructing the WebappClassLoader to avoid loading any symbols from
"org.apache.xerces" (and hoping it would fallback to
"com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl")
won't solve the problem.
I've arrived at a fix, which is to define a system property with name
javax.xml.parsers.DocumentBuilderFactory
and specify
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
as its value.
when running Netbeans under JDK 1.5.X or later, when one jsp file is opened and being saved, following exception is thrown:
java.lang.AbstractMethodError: org. customer.xerces.dom.DeferredDocumentImpl.setDocumentURI(Ljava/lang/String;)V
o make a long story short:
- Method
public void setDocumentURI(String documentURI);
was added to the org.w3c.dom.Document interface
in J2SE 1.5 (it was not present in J2SE 1.4).
- The new method is implemented by
com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl
(in J2SE 1.5), but not by
org.apache.xerces.dom.DeferredDocumentImpl
in xercesImpl.jar
- javax.xml.parsers.DocumentBuilderFactory in J2SE 1.5 will instantiate
factories of type "org.apache.xerces.jaxp.DocumentBuilderFactoryImpl"
for webapps bundling xercesImpl.jar, due to the presence of the
META-INF/services/javax.xml.parsers.DocumentBuilderFactory resource
in that JAR, which specifies
"org.apache.xerces.jaxp.DocumentBuilderFactoryImpl" as the factory
type to be instantiated.
Instructing the WebappClassLoader to avoid loading any symbols from
"org.apache.xerces" (and hoping it would fallback to
"com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl")
won't solve the problem.
I've arrived at a fix, which is to define a system property with name
javax.xml.parsers.DocumentBuilderFactory
and specify
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
as its value.
Sunday, May 25, 2008
How to use STUN in applications
How to use STUN in applications
This document has been reviewed for maemo 3.x.
This tutorial shows how to use the libjingle API to create peer-to-peer connections between parties that are behind NAT routers.
Network Address Translation (NAT)
NAT routers were born because of the imminent IP address exhaustion. The currently most used Internet address, called IPv4, is 32 bits wide, which means around 4 billion address available.
4 billion addresses seem to be an abundant resource, but not all addresses can be assigned to hosts, as not all telephone number permutations can be assigned to users, since IP addresses also carry routing information, analog to the prefix of a telephone number that identifies country, region, city...
The definitive solution for the problem is to increase the address length, that is what IPv6 does, by offering 128-bit addresses. But IPv6 would take (and it is taking) time to be deployed, so an intermediate solution was found: NAT - Network Address Translation.
A NAT router acts like a switchboard between the public Internet and a private network. The NAT router needs to have at least one valid Internet address in order to be "seen" by the rest of the Internet. The computers in the private network have private address in the ranges 10.0.0.0/8, 192.168.0.0/16 or 172.16.0.0/12, that are not routable in the Internet.
When a private computer e.g. address 10.0.0.1 tries to connect a public server e.g. 200.215.89.79, the network packet must pass through the NAT router. The NAT router knows that the source address 10.0.0.1 is not routable. So the NAT router replaces 10.0.0.1 with its own valid address (e.g. 64.1.2.3) and sends the packet forward.
The remote server 200.215.89.79 will "see" a connection coming from the 64.1.2.3 host, and will reply to that host.
When the response packet comes to NAT router, it must know somehow that this packet is not for itself, it is for the computer in the private network. Once the NAT router relates the packet coming from the blue with the active NAT'ed connections, it can replace the destination address from 64.1.2.3 to 10.0.0.1 and deliver the packet in the private network.
In more technical depth, NAT is possible because absolutely every network connection in the Internet has a unique tuple built from the following values:
* Client address (the host that initiates the connection)
* Server address (the passive side that receives the initiation packet)
* Client port number
* Server port number
* The transport protocol e.g. TCP, UDP, SCTP...
If the NAT router replaces the client address by its own, but also replaces the client port number when necessary to avoid clash with another active connection, the uniqueness of each connection is not broken. NAT allows for a virtually unlimited number of computers in a private network to access the Internet via only one NAT router with only one public IP address.
NAT problems
NAT is not without some disvantages. First, the NAT router needs to keep connection states in memory, which partially denies a cornerstone Internet philosophy ("dumb routers, smart hosts"). If a NAT router is reset, all connections will be terminated, while a regular router could be reset without harming any connection.
Second, the hosts in private network cannot easily offer a service to the public Internet, that is, these hosts cannot be the "passive" side in connections, since the initiation packet will come to the NAT router and it will have no related connection in its memory.
A partial solution for that problem is to open a "hole" in the NAT for specific ports, for example any connection to the port 8000 of the NAT router should be redirected to the machine 10.0.0.2 port 80. That works, but it does not scale (the number of ports is limited, and some protocols work only on a very specific port number) and demands configuring the NAT router.
Since the bulk of the Internet traffic is HTTP and initiated from the private network, NATs are fine for most users.
NAT peer-to-peer circumvention techniques
The most problematic services to deploy in presence of NATs are of peer-to-peer (P2P) style, when two clients of the service make direct connections to each other, without sending data through an intermediate server. This is the case of SIP-based VoIP, most P2P file sharing networks etc.
If one of the P2P parties has a routable IP address, the problem is solved: the party behind the NAT router must initate the connection, and that's it. The remaining problematic case (unfortunately the most common one) is when both P2P parties are behind NATs.
Several techniques have been proposed to solve this case. The most elegant solutions require both software updates in the NAT router itself and explicit support in the client software. It is the case of UPnP IGD (Internet Gateway Device) protocol, where a host can explicitely request a "hole" to be open in the NAT router, so a service is acessible from the Internet.
IGD is nice and has been enjoying good support from NAT router vendors, but it is still not ubiquitous and has a major flaw: if there are two or more NAT routers in the packet traffic way (a rather common situation), IGD ceases to be effective.
It is opportune to remember that none of the NAT circumvention techniques, not even the most elegant, solve completely the problem. There still must be a signaling protocol for the parties to exchange the connection parameters. In other words, It does not suffice to be able to open a "hole" in the NAT router; there must be a way to communicate the address and port of the hole to the remote party. A pure P2P service can only be achieved when all parties possess public addresses, which is expected to happen when IPv6 is widely deployed.
Another way is simply to open a NAT hole via router configuration. Most home/SOHO routers allow this configuration to be easily done. It is a very popular technique among gamers (some routers even label the NAT holes as "game ports"). The main advantage of this technique is no need of explicit NAT-piercing support in the client software itself. The software must only be "well-behaved", not trying to guess the local IP address by itself (which would return a private address instead of the router public address).
The main disvantage of manual configuration of NAT holes is that they are, well, manual: one explicit administrative intervention is needed for each new protocol. It it not good enough if the user has no access to NAT configuration (the most common case in collective and corporate networks).
STUN, TURN and ICE protocols
There have been proposed NAT circumvention techniques that do not demand router software updates. STUN (Simple Transversal of UDP through NATs) is the main one. STUN exploits the connectionless nature of UDP, as well as a security weakness of some NAT implementations. The technique is bound to UDP features, so STUN can not be used for e.g. TCP connections.
The STUN protocol demands a STUN server with a well-known public IP address in the Internet. STUN stores the private address/port in an UDP payload and sends the packet to the STUN server. If packet goes through a NAT router, the address/port will be changed in the IP header -- but not in the payload.
By comparing the payload with the IP header, and sending back the results to the client, STUN can deduce if there are any NAT routers in the way. If there are none, the host knows that it can directly receive connections from parties behind NATs.
Some NATs have poor implementations that, in conjunction with STUN, allow for incoming UDP flow. The "full cone" NAT always translates the same source address/source port tuple to the same NAT router port; this relieves the NAT from storing full connection state. Any packet coming from Internet to that router port will be delivered to the host in the private network, no matter it came from.
This allows for the private host to easily "punch a hole" in the NAT router. The first packet that opens the hole goes to STUN server. In turn, the private host learns from the STUN server which is the IP address and port number of the hole. These parameters are sent to the remote peer via signaling protocol. The remote peer can then send UDP data directly.
Unfortunately, most NAT routers are not that naive; they are "symmetric", that is, they store full connection state and do not allow incoming packets from anyone but the party that was first contacted (in our case, the STUN server). In this scenario, STUN is not enough to allow P2P communication in presence of NAT.
TURN (Traversal Using Relay NAT) comes for the rescue. TURN employs the same protocol as STUN, but the TURN server acts as a relayer in which both parties behind NATs can connect to. Since the relay server adds latency and needs to be maintained (and its bandwidth costs money), TURN should be used only as a last resort.
ICE (Interactive Connectivity Establishment), a draft specification that is employed by Google Talk, is not a protocol by itself; it is a collection of techniques like STUN and TURN that finds all possible ways to establish a P2P connection and selects the best one.
ICE works by finding all possible P2P connection candidates and sending this data to the remote peer via a signaling protocol (which is not specified by ICE, so the mechanism can be employed by any existing signaling protocol). Both parties do that simultaneously, so there are candidate messages flowing in both directions. When the parties agree on the best way to make the P2P connection, they can begin to exchange data directly.
The biggest advantage of STUN, TURN and ICE is that they do not depend on specific support by the NAT routers, and ICE will work even if there are several routers in the network path. The downside is the need of public STUN and TURN (relay) servers. This disvantage is dilluted to some extent by the fact that every P2P service demands a signaling server anyway, so the same entity that offers the signaling will certainly offer the auxiliary STUN/TURN services.
Another minor disvantage of ICE is that the signaling protocol will have to accomodate the "connection candidate" message, either by a explicit provision in the protocol (e.g. XMPP) or by some kludge.
NAT transversal API in maemo
In maemo platform, the developer does not need to worry about protocols or anything. maemo includes the libjingle library (the same as Google Talk) which offers an API for P2P connections.
Nothing better than living examples to show how the maemo developer can take advantage on this API. Our example P2P client is very simple: it requests a service at random times and also processes requests from other P2P clients. The "service" is nothing more than adding 2 byte values.
As already stated, every P2P server must have a signaling protocol over which the parties can exchange initial P2P connection parameters, so we need to build a very simple server as well as the signaling protocol. Since it is just an example, our server architecture does not attribute IDs for the clients, and therefore can handle only two simultaneous clients.
The signaling protocol is very simple and has only one message: the connection candidate that one peer sends to the other. The message is simply forwarded to the other peer. Apart from encoding/decoding, these messages are completely handled by libjingle, so we don't need to understand them in depth.
Once the connection parameters have been exchanged via the signaling server, the P2P connection happens and the parties communicate directly without any further signaling. Albeit very simple, this protocol simulates all the basic steps of every real-world P2P service.
If you are interested in using XMPP/Google Talk as the signaling service, Libjingle source also contains examples of P2P communication that employ XMPP/Google Talk accounts and servers.
Example: P2P client
Follows a C++ example of P2P client based on Libjingle APIs. It is the smallest possible demonstration of NAT-piercing capability, so it does not handle network errors and overflows very well. A production implementation must improve in these directions.
First, there is some boilerplate code: includes and prototypes.
#define POSIX
#define SIGSLOT_USE_POSIX_THREADS
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
void signaling_init(const char* ip, unsigned int port);
int signaling_wait(unsigned int timeout);
void signaling_sendall(const char* buffer, unsigned int len);
SocketClient* socketclient_init(const char *stun_ip, unsigned int stun_port,
const char *turn_ip, unsigned int turn_port);
char* socketclient_add_remote_candidates(SocketClient *sc, char* buffer);
void socketclient_add_remote_candidate(SocketClient *sc, const char *candidate);
bool socketclient_is_writable (SocketClient *sc);
void socketclient_send(SocketClient *sc, const char *data, unsigned int len);
void randomize();
For the sake of simplicity, we keep some data in global variables. A production implementation would probably move those data into objects.
The p2p_state shows whether the P2P connection is up or down. signaling_socket is a TCP socket that allows data exchange via the signaling server before the P2P connection is up. main_thread contains a libjingle's Thread object; libjingle is itself multithreaded and employs one thread per P2P connection.
bool p2p_state = false;
int signaling_socket = -1;
cricket::Thread *main_thread = 0;
This is the main program loop. It sets up the signaling connection, forwards signaling data to LibJingle until the P2P connection is up. The P2P connection is simply used to send bytes at random intervals. If the P2P connection breaks, the loop returns to signaling phase. The program only stops when killed or when some unexpected error occurs.
Note that we call main_thread->Loop(10) from time to time. In a "real" application, we would have this method called on idle time (e.g. via GLib's g_idle_add().
We have some IP addresses hardcoded: signaling server, STUN server (if any) and TURN server (if any). Please update those addresses for your particular environment.
int main(int argc, char* argv[])
{
signal(SIGPIPE, SIG_IGN);
randomize();
// P2P signaling server
const char* signaling_ip = "200.184.118.140";
int signaling_port = 14141;
// STUN server if any, set to NULL if there are none
const char* stun_ip = "200.184.118.140";
// const char* stun_ip = 0;
unsigned int stun_port = 7000;
// TURN server if any, set to NULL if there are none
const char* turn_ip = "200.184.118.140";
// const char* turn_ip = 0;
unsigned int turn_port = 5000;
signaling_init(signaling_ip, signaling_port);
SocketClient* sc = socketclient_init(stun_ip, stun_port, turn_ip, turn_port);
sc->getSocketManager()->StartProcessingCandidates();
while (1) {
char buffer[10000];
char *buffer_p = buffer;
char *buffer_interpreted = buffer;
while (! p2p_state) {
main_thread->Loop(10);
if (! signaling_wait(1)) {
printf("-- tick --\n");
continue;
}
int n = recv(signaling_socket, buffer_p, sizeof(buffer) - (buffer_p - buffer), 0);
if (n < 0) {
printf("Signaling socket closed with error\n");
exit(1);
} else if (n == 0) {
printf("Signaling socket closed\n");
exit(1);
}
buffer_p += n;
buffer_interpreted = socketclient_add_remote_candidates(sc, buffer_interpreted);
}
// P2P connection is up by now.
while (p2p_state) {
// sends a byte via P2P connection
unsigned char data = random() % 256;
socketclient_send(sc, (char*) &data, 1);
sleep(random() % 15 + 1);
main_thread->Loop(10);
}
// P2P connection is broken, restart handling connection candidates
}
}
Seeds the random() with some "random" value, otherwise the two peers may end up with exactly the same bytes and time intervals during P2P data exchange.
void randomize()
{
struct timeval tv;
struct timezone tz;
gettimeofday(&tv, &tz);
srandom(tv.tv_usec);
}
This function creates the signaling socket -- just a boring and ordinary TCP connection to the P2P signaling server.
void signaling_init(const char* ip, unsigned int port)
{
struct sockaddr_in sa;
signaling_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
bzero(&sa, sizeof(sa));
sa.sin_family = AF_INET;
sa.sin_port = htons(port);
inet_aton(ip, &(sa.sin_addr));
if (connect(signaling_socket, (struct sockaddr*) &sa, sizeof(sa)) < 0) {
printf("Error in signaling connect()\n");
exit(1);
}
}
This function waits for something to happen with the singaling socket, until the specified timeout. It is important that the timeout is not too large, since the P2P connection may have been brought up meanwhile.
int signaling_wait(unsigned int timeout) {
fd_set rfds;
struct timeval tv;
int retval;
FD_ZERO(&rfds);
FD_SET(signaling_socket, &rfds);
tv.tv_sec = timeout;
tv.tv_usec = 0;
retval = select(signaling_socket+1, &rfds, NULL, NULL, &tv);
if (retval == -1)
printf("error in select()");
return (retval > 0);
}
Libjingle is C++ and employs a signal architecture to notify the client application about changes in P2P connection status. These two classes accomodate the methods that will be called back when something happens.
Since we are not using XMPP as the signaling protocol, we need to provide the signaling protocol handling by ourselves, so all signaling handling is carried out separatedly from those signals.
class SignalListener1 : public sigslot::has_slots<>
{
private:
SocketClient *sc;
public:
SignalListener1(SocketClient *psc);
void OnCandidatesReady(const std::vector& candidates);
void OnNetworkError();
void OnSocketState(bool state);
};
class SignalListener2 : public sigslot::has_slots<>
{
private:
SocketClient *sc;
public:
SignalListener2(SocketClient *psc);
void OnSocketRead(P2PSocket *socket, const char *data, size_t len);
};
SignalListener1::SignalListener1(SocketClient* psc)
{
sc = psc;
}
SignalListener2::SignalListener2(SocketClient* psc)
{
sc = psc;
}
void SignalListener1::OnNetworkError()
{
printf ("Network error encountered at SocketManager");
exit(1);
}
The first signal callback method. It is called when the P2P socket changes state. We update the p2p_state global variable with the reported state, and this will drive our main loop behavior.
void SignalListener1::OnSocketState(bool state)
{
printf("Socket state changed to %d\n", state);
p2p_state = state;
if (state) {
printf("Writable from %s:%d to %s:%d\n",
sc->getSocket()->best_connection()->local_candidate().address().IPAsString().c_str(),
sc->getSocket()->best_connection()->local_candidate().address().port(),
sc->getSocket()->best_connection()->remote_candidate().address().IPAsString().c_str(),
sc->getSocket()->best_connection()->remote_candidate().address().port());
}
}
This function packages all P2P socket creation bureaucracy. It creates the socket object, the socket listeners (whose classes have been defined above by us) and connects the signal callbacks.
SocketClient* socketclient_init(const char *stun_ip, unsigned int stun_port,
const char *turn_ip, unsigned int turn_port)
{
cricket::SocketAddress *stun_addr = NULL;
if (stun_ip) {
stun_addr = new cricket::SocketAddress(std::string(stun_ip), stun_port);
}
cricket::SocketAddress *turn_addr = NULL;
if (turn_ip) {
turn_addr = new cricket::SocketAddress(std::string(turn_ip), turn_port);
}
cricket::PhysicalSocketServer *ss = new PhysicalSocketServer();
main_thread = new Thread(ss);
cricket::ThreadManager::SetCurrent(main_thread);
SocketClient *sc = new SocketClient (stun_addr, turn_addr);
// Note that signal connections pass the SignalListener1 object as well as the
// method. Since a new SocketListener1 is created for every new SocketClient,
// we have the guarantee that each SocketListener1 will be called back only
// in behalf of its related SocketClient.
sc->sigl1 = new SignalListener1(sc);
sc->sigl2 = new SignalListener2(sc);
sc->getSocketManager()->SignalNetworkError.connect(sc->sigl1, &SignalListener1::OnNetworkError);
sc->getSocketManager()->SignalState_s.connect(sc->sigl1, &SignalListener1::OnSocketState);
sc->getSocketManager()->SignalCandidatesReady.connect(sc->sigl1, &SignalListener1::OnCandidatesReady);
sc->CreateSocket(std::string("foobar"));
sc->getSocket()->SignalReadPacket.connect(sc->sigl2, &SignalListener2::OnSocketRead);
return sc;
}
This method is called back when LibJingle has some local candidates for connection, that should be sent to the remote site via the signaling protocol.
The beauty of ICE protocol is that parties will be able to agree on a P2P connection without having to exchange request/response messages. They just send connection candidates to each other. Each side selects the "best" way to send data based on received candidades; being both sides able to send data directly to the remote party, we have a P2P bidirectional channel.
void SignalListener1::OnCandidatesReady(const std::vector& candidates)
{
printf("OnCandidatesReady called with %d candidates in list\n", candidates.size());
for(std::vector::const_iterator it = candidates.begin(); it != candidates.end(); ++it) {
char *marshaled_candidate;
asprintf(&marshaled_candidate, "%s %d %s %f %s %s %s\n",
(*it).address().IPAsString().c_str(), (*it).address().port(), (*it).protocol().c_str(),
(*it).preference(), (*it).type().c_str(), (*it).username().c_str(),
(*it).password().c_str() );
printf("Candidate being sent: %s", marshaled_candidate);
signaling_sendall(marshaled_candidate, strlen(marshaled_candidate));
free(marshaled_candidate);
}
}
An auxiliary function to send a data buffer through the signaling TCP channel. It does not return until all data has been sent.
void signaling_sendall(const char* buffer, unsigned int len)
{
unsigned int sent = 0;
while (sent < len) {
int just_sent;
just_sent = send(signaling_socket, buffer+sent, len-sent, 0);
if (just_sent < 0) {
printf("Signaling socket closed with error.\n");
exit(1);
} else if (just_sent == 0) {
printf("Signaling socket closed.\n");
exit(1);
}
sent += just_sent;
}
}
This function is called by the main loop when some signaling data arrives. It will find out if there is a complete P2P connection candidate in the buffer. If there is one, it is decoded.
// extracts remote candidates from a buffer, returns a pointer to the rest of the buffer
char* socketclient_add_remote_candidates(SocketClient *sc, char* buffer)
{
char *n;
char candidate[1024];
while (1) {
n = strchr(buffer, '\n');
if (! n) {
return buffer;
}
strncpy(candidate, buffer, n-buffer+1);
socketclient_add_remote_candidate(sc, candidate);
buffer = n+1;
}
}
Here, the P2P connection candidate is decoded and made known to LibJingle. Since LibJingle has its own thread and sockets, all further processing of P2P candidates is fortunately outside the scope of our code.
// Inform candidates received from the signaling network to LibJingle
void socketclient_add_remote_candidate(SocketClient *sc, const char* remote_candidate)
{
std::vector candidates;
char ip[100];
unsigned int port;
char protocol[100];
float preference;
char type[100];
char username[100];
char password[100];
// WARNING: using fixed-size buffers and sscanf is utterly unsafe.
// Real implementations must be more robust about data coming from the network!
sscanf(remote_candidate, "%s %d %s %f %s %s %s\n", ip, &port, protocol, &preference, type, username, password);
printf("Received new candidate: %s:%d pref %f\n", ip, port, preference);
Candidate candidate;
candidate.set_name("rtp");
candidate.set_address(SocketAddress(std::string(ip), port));
candidate.set_username(std::string(username));
candidate.set_password(std::string(password));
candidate.set_preference(preference);
candidate.set_protocol(protocol);
candidate.set_type(type);
candidate.set_generation(0);
candidates.push_back(candidate);
sc->getSocketManager()->AddRemoteCandidates(candidates);
}
Simple helper function that shows whether a P2P socket is writable (which means that the P2P connection is up).
bool socketclient_is_writable(SocketClient *sc)
{
return sc->getSocketManager()->writable();
}
Method that is called back when data arrives from the P2P connection. In our example, it is just a byte of data.
void SignalListener2::OnSocketRead(P2PSocket *socket, const char *data, size_t len)
{
printf("Received byte %d from remote P2P\n", data[0]);
}
Auxiliary function that sends data via P2P connection. Not really difficult.
void socketclient_send(SocketClient* sc, const char *data, unsigned int len)
{
sc->getSocket()->Send(data, len);
printf("Sent byte %d to remote P2P\n", data[0]);
}
The signaling server
As already said, our signaling server is incredibly simple and works more like a network pipe, forwarding data from one side to another. The P2P parties ogree on a P2P connection through this channel.
from select import select
import socket
import time
read_socks = []
port = 14141
server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.bind(("", port))
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.listen(5)
read_socks.append(server_sock)
# We accept two parties only
buffer = ""
while True:
rd, wr_dummy, ex_dummy = select(read_socks, [], [], 10)
if not rd:
print "-- tick --"
continue
rd = rd[0]
if rd is server_sock:
# incoming new connection
newsock, address = rd.accept()
print "New connection from %s" % str(address)
if len(read_socks) > 3:
# we only accept two parties at the most
newsock.close()
continue
read_socks.append(newsock)
if buffer:
# we already have data to be sent to the new party
newsock.sendall(buffer)
print " sent buffered data"
buffer = ""
continue
data = rd.recv(999999)
if not data:
# socket closed, remove from list
print "Connection closed"
del read_socks[read_socks.index(rd)]
buffer = ""
continue
if len(read_socks) < 3:
print "Buffering data"
# the other party has not connected; bufferize
buffer += data
continue
print "Forwarding data"
for wr in read_socks:
if wr is not rd and wr is not server_sock:
wr.sendall(data)
STUN and relay servers
The maemo libjingle-utils package includes both a STUN server and a relay server for testing purposes. To run a test server, do the following:
$ stunserver &
$ relayserver &
The relay server will print console messages when a P2P connection is flowing through it. If you need more detailed feedback (e.g. if you are debugging a P2P application), use a network sniffing tool like tcpdump or Ethereal to monitor UDP packets.
This document has been reviewed for maemo 3.x.
This tutorial shows how to use the libjingle API to create peer-to-peer connections between parties that are behind NAT routers.
Network Address Translation (NAT)
NAT routers were born because of the imminent IP address exhaustion. The currently most used Internet address, called IPv4, is 32 bits wide, which means around 4 billion address available.
4 billion addresses seem to be an abundant resource, but not all addresses can be assigned to hosts, as not all telephone number permutations can be assigned to users, since IP addresses also carry routing information, analog to the prefix of a telephone number that identifies country, region, city...
The definitive solution for the problem is to increase the address length, that is what IPv6 does, by offering 128-bit addresses. But IPv6 would take (and it is taking) time to be deployed, so an intermediate solution was found: NAT - Network Address Translation.
A NAT router acts like a switchboard between the public Internet and a private network. The NAT router needs to have at least one valid Internet address in order to be "seen" by the rest of the Internet. The computers in the private network have private address in the ranges 10.0.0.0/8, 192.168.0.0/16 or 172.16.0.0/12, that are not routable in the Internet.
When a private computer e.g. address 10.0.0.1 tries to connect a public server e.g. 200.215.89.79, the network packet must pass through the NAT router. The NAT router knows that the source address 10.0.0.1 is not routable. So the NAT router replaces 10.0.0.1 with its own valid address (e.g. 64.1.2.3) and sends the packet forward.
The remote server 200.215.89.79 will "see" a connection coming from the 64.1.2.3 host, and will reply to that host.
When the response packet comes to NAT router, it must know somehow that this packet is not for itself, it is for the computer in the private network. Once the NAT router relates the packet coming from the blue with the active NAT'ed connections, it can replace the destination address from 64.1.2.3 to 10.0.0.1 and deliver the packet in the private network.
In more technical depth, NAT is possible because absolutely every network connection in the Internet has a unique tuple built from the following values:
* Client address (the host that initiates the connection)
* Server address (the passive side that receives the initiation packet)
* Client port number
* Server port number
* The transport protocol e.g. TCP, UDP, SCTP...
If the NAT router replaces the client address by its own, but also replaces the client port number when necessary to avoid clash with another active connection, the uniqueness of each connection is not broken. NAT allows for a virtually unlimited number of computers in a private network to access the Internet via only one NAT router with only one public IP address.
NAT problems
NAT is not without some disvantages. First, the NAT router needs to keep connection states in memory, which partially denies a cornerstone Internet philosophy ("dumb routers, smart hosts"). If a NAT router is reset, all connections will be terminated, while a regular router could be reset without harming any connection.
Second, the hosts in private network cannot easily offer a service to the public Internet, that is, these hosts cannot be the "passive" side in connections, since the initiation packet will come to the NAT router and it will have no related connection in its memory.
A partial solution for that problem is to open a "hole" in the NAT for specific ports, for example any connection to the port 8000 of the NAT router should be redirected to the machine 10.0.0.2 port 80. That works, but it does not scale (the number of ports is limited, and some protocols work only on a very specific port number) and demands configuring the NAT router.
Since the bulk of the Internet traffic is HTTP and initiated from the private network, NATs are fine for most users.
NAT peer-to-peer circumvention techniques
The most problematic services to deploy in presence of NATs are of peer-to-peer (P2P) style, when two clients of the service make direct connections to each other, without sending data through an intermediate server. This is the case of SIP-based VoIP, most P2P file sharing networks etc.
If one of the P2P parties has a routable IP address, the problem is solved: the party behind the NAT router must initate the connection, and that's it. The remaining problematic case (unfortunately the most common one) is when both P2P parties are behind NATs.
Several techniques have been proposed to solve this case. The most elegant solutions require both software updates in the NAT router itself and explicit support in the client software. It is the case of UPnP IGD (Internet Gateway Device) protocol, where a host can explicitely request a "hole" to be open in the NAT router, so a service is acessible from the Internet.
IGD is nice and has been enjoying good support from NAT router vendors, but it is still not ubiquitous and has a major flaw: if there are two or more NAT routers in the packet traffic way (a rather common situation), IGD ceases to be effective.
It is opportune to remember that none of the NAT circumvention techniques, not even the most elegant, solve completely the problem. There still must be a signaling protocol for the parties to exchange the connection parameters. In other words, It does not suffice to be able to open a "hole" in the NAT router; there must be a way to communicate the address and port of the hole to the remote party. A pure P2P service can only be achieved when all parties possess public addresses, which is expected to happen when IPv6 is widely deployed.
Another way is simply to open a NAT hole via router configuration. Most home/SOHO routers allow this configuration to be easily done. It is a very popular technique among gamers (some routers even label the NAT holes as "game ports"). The main advantage of this technique is no need of explicit NAT-piercing support in the client software itself. The software must only be "well-behaved", not trying to guess the local IP address by itself (which would return a private address instead of the router public address).
The main disvantage of manual configuration of NAT holes is that they are, well, manual: one explicit administrative intervention is needed for each new protocol. It it not good enough if the user has no access to NAT configuration (the most common case in collective and corporate networks).
STUN, TURN and ICE protocols
There have been proposed NAT circumvention techniques that do not demand router software updates. STUN (Simple Transversal of UDP through NATs) is the main one. STUN exploits the connectionless nature of UDP, as well as a security weakness of some NAT implementations. The technique is bound to UDP features, so STUN can not be used for e.g. TCP connections.
The STUN protocol demands a STUN server with a well-known public IP address in the Internet. STUN stores the private address/port in an UDP payload and sends the packet to the STUN server. If packet goes through a NAT router, the address/port will be changed in the IP header -- but not in the payload.
By comparing the payload with the IP header, and sending back the results to the client, STUN can deduce if there are any NAT routers in the way. If there are none, the host knows that it can directly receive connections from parties behind NATs.
Some NATs have poor implementations that, in conjunction with STUN, allow for incoming UDP flow. The "full cone" NAT always translates the same source address/source port tuple to the same NAT router port; this relieves the NAT from storing full connection state. Any packet coming from Internet to that router port will be delivered to the host in the private network, no matter it came from.
This allows for the private host to easily "punch a hole" in the NAT router. The first packet that opens the hole goes to STUN server. In turn, the private host learns from the STUN server which is the IP address and port number of the hole. These parameters are sent to the remote peer via signaling protocol. The remote peer can then send UDP data directly.
Unfortunately, most NAT routers are not that naive; they are "symmetric", that is, they store full connection state and do not allow incoming packets from anyone but the party that was first contacted (in our case, the STUN server). In this scenario, STUN is not enough to allow P2P communication in presence of NAT.
TURN (Traversal Using Relay NAT) comes for the rescue. TURN employs the same protocol as STUN, but the TURN server acts as a relayer in which both parties behind NATs can connect to. Since the relay server adds latency and needs to be maintained (and its bandwidth costs money), TURN should be used only as a last resort.
ICE (Interactive Connectivity Establishment), a draft specification that is employed by Google Talk, is not a protocol by itself; it is a collection of techniques like STUN and TURN that finds all possible ways to establish a P2P connection and selects the best one.
ICE works by finding all possible P2P connection candidates and sending this data to the remote peer via a signaling protocol (which is not specified by ICE, so the mechanism can be employed by any existing signaling protocol). Both parties do that simultaneously, so there are candidate messages flowing in both directions. When the parties agree on the best way to make the P2P connection, they can begin to exchange data directly.
The biggest advantage of STUN, TURN and ICE is that they do not depend on specific support by the NAT routers, and ICE will work even if there are several routers in the network path. The downside is the need of public STUN and TURN (relay) servers. This disvantage is dilluted to some extent by the fact that every P2P service demands a signaling server anyway, so the same entity that offers the signaling will certainly offer the auxiliary STUN/TURN services.
Another minor disvantage of ICE is that the signaling protocol will have to accomodate the "connection candidate" message, either by a explicit provision in the protocol (e.g. XMPP) or by some kludge.
NAT transversal API in maemo
In maemo platform, the developer does not need to worry about protocols or anything. maemo includes the libjingle library (the same as Google Talk) which offers an API for P2P connections.
Nothing better than living examples to show how the maemo developer can take advantage on this API. Our example P2P client is very simple: it requests a service at random times and also processes requests from other P2P clients. The "service" is nothing more than adding 2 byte values.
As already stated, every P2P server must have a signaling protocol over which the parties can exchange initial P2P connection parameters, so we need to build a very simple server as well as the signaling protocol. Since it is just an example, our server architecture does not attribute IDs for the clients, and therefore can handle only two simultaneous clients.
The signaling protocol is very simple and has only one message: the connection candidate that one peer sends to the other. The message is simply forwarded to the other peer. Apart from encoding/decoding, these messages are completely handled by libjingle, so we don't need to understand them in depth.
Once the connection parameters have been exchanged via the signaling server, the P2P connection happens and the parties communicate directly without any further signaling. Albeit very simple, this protocol simulates all the basic steps of every real-world P2P service.
If you are interested in using XMPP/Google Talk as the signaling service, Libjingle source also contains examples of P2P communication that employ XMPP/Google Talk accounts and servers.
Example: P2P client
Follows a C++ example of P2P client based on Libjingle APIs. It is the smallest possible demonstration of NAT-piercing capability, so it does not handle network errors and overflows very well. A production implementation must improve in these directions.
First, there is some boilerplate code: includes and prototypes.
#define POSIX
#define SIGSLOT_USE_POSIX_THREADS
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
void signaling_init(const char* ip, unsigned int port);
int signaling_wait(unsigned int timeout);
void signaling_sendall(const char* buffer, unsigned int len);
SocketClient* socketclient_init(const char *stun_ip, unsigned int stun_port,
const char *turn_ip, unsigned int turn_port);
char* socketclient_add_remote_candidates(SocketClient *sc, char* buffer);
void socketclient_add_remote_candidate(SocketClient *sc, const char *candidate);
bool socketclient_is_writable (SocketClient *sc);
void socketclient_send(SocketClient *sc, const char *data, unsigned int len);
void randomize();
For the sake of simplicity, we keep some data in global variables. A production implementation would probably move those data into objects.
The p2p_state shows whether the P2P connection is up or down. signaling_socket is a TCP socket that allows data exchange via the signaling server before the P2P connection is up. main_thread contains a libjingle's Thread object; libjingle is itself multithreaded and employs one thread per P2P connection.
bool p2p_state = false;
int signaling_socket = -1;
cricket::Thread *main_thread = 0;
This is the main program loop. It sets up the signaling connection, forwards signaling data to LibJingle until the P2P connection is up. The P2P connection is simply used to send bytes at random intervals. If the P2P connection breaks, the loop returns to signaling phase. The program only stops when killed or when some unexpected error occurs.
Note that we call main_thread->Loop(10) from time to time. In a "real" application, we would have this method called on idle time (e.g. via GLib's g_idle_add().
We have some IP addresses hardcoded: signaling server, STUN server (if any) and TURN server (if any). Please update those addresses for your particular environment.
int main(int argc, char* argv[])
{
signal(SIGPIPE, SIG_IGN);
randomize();
// P2P signaling server
const char* signaling_ip = "200.184.118.140";
int signaling_port = 14141;
// STUN server if any, set to NULL if there are none
const char* stun_ip = "200.184.118.140";
// const char* stun_ip = 0;
unsigned int stun_port = 7000;
// TURN server if any, set to NULL if there are none
const char* turn_ip = "200.184.118.140";
// const char* turn_ip = 0;
unsigned int turn_port = 5000;
signaling_init(signaling_ip, signaling_port);
SocketClient* sc = socketclient_init(stun_ip, stun_port, turn_ip, turn_port);
sc->getSocketManager()->StartProcessingCandidates();
while (1) {
char buffer[10000];
char *buffer_p = buffer;
char *buffer_interpreted = buffer;
while (! p2p_state) {
main_thread->Loop(10);
if (! signaling_wait(1)) {
printf("-- tick --\n");
continue;
}
int n = recv(signaling_socket, buffer_p, sizeof(buffer) - (buffer_p - buffer), 0);
if (n < 0) {
printf("Signaling socket closed with error\n");
exit(1);
} else if (n == 0) {
printf("Signaling socket closed\n");
exit(1);
}
buffer_p += n;
buffer_interpreted = socketclient_add_remote_candidates(sc, buffer_interpreted);
}
// P2P connection is up by now.
while (p2p_state) {
// sends a byte via P2P connection
unsigned char data = random() % 256;
socketclient_send(sc, (char*) &data, 1);
sleep(random() % 15 + 1);
main_thread->Loop(10);
}
// P2P connection is broken, restart handling connection candidates
}
}
Seeds the random() with some "random" value, otherwise the two peers may end up with exactly the same bytes and time intervals during P2P data exchange.
void randomize()
{
struct timeval tv;
struct timezone tz;
gettimeofday(&tv, &tz);
srandom(tv.tv_usec);
}
This function creates the signaling socket -- just a boring and ordinary TCP connection to the P2P signaling server.
void signaling_init(const char* ip, unsigned int port)
{
struct sockaddr_in sa;
signaling_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
bzero(&sa, sizeof(sa));
sa.sin_family = AF_INET;
sa.sin_port = htons(port);
inet_aton(ip, &(sa.sin_addr));
if (connect(signaling_socket, (struct sockaddr*) &sa, sizeof(sa)) < 0) {
printf("Error in signaling connect()\n");
exit(1);
}
}
This function waits for something to happen with the singaling socket, until the specified timeout. It is important that the timeout is not too large, since the P2P connection may have been brought up meanwhile.
int signaling_wait(unsigned int timeout) {
fd_set rfds;
struct timeval tv;
int retval;
FD_ZERO(&rfds);
FD_SET(signaling_socket, &rfds);
tv.tv_sec = timeout;
tv.tv_usec = 0;
retval = select(signaling_socket+1, &rfds, NULL, NULL, &tv);
if (retval == -1)
printf("error in select()");
return (retval > 0);
}
Libjingle is C++ and employs a signal architecture to notify the client application about changes in P2P connection status. These two classes accomodate the methods that will be called back when something happens.
Since we are not using XMPP as the signaling protocol, we need to provide the signaling protocol handling by ourselves, so all signaling handling is carried out separatedly from those signals.
class SignalListener1 : public sigslot::has_slots<>
{
private:
SocketClient *sc;
public:
SignalListener1(SocketClient *psc);
void OnCandidatesReady(const std::vector
void OnNetworkError();
void OnSocketState(bool state);
};
class SignalListener2 : public sigslot::has_slots<>
{
private:
SocketClient *sc;
public:
SignalListener2(SocketClient *psc);
void OnSocketRead(P2PSocket *socket, const char *data, size_t len);
};
SignalListener1::SignalListener1(SocketClient* psc)
{
sc = psc;
}
SignalListener2::SignalListener2(SocketClient* psc)
{
sc = psc;
}
void SignalListener1::OnNetworkError()
{
printf ("Network error encountered at SocketManager");
exit(1);
}
The first signal callback method. It is called when the P2P socket changes state. We update the p2p_state global variable with the reported state, and this will drive our main loop behavior.
void SignalListener1::OnSocketState(bool state)
{
printf("Socket state changed to %d\n", state);
p2p_state = state;
if (state) {
printf("Writable from %s:%d to %s:%d\n",
sc->getSocket()->best_connection()->local_candidate().address().IPAsString().c_str(),
sc->getSocket()->best_connection()->local_candidate().address().port(),
sc->getSocket()->best_connection()->remote_candidate().address().IPAsString().c_str(),
sc->getSocket()->best_connection()->remote_candidate().address().port());
}
}
This function packages all P2P socket creation bureaucracy. It creates the socket object, the socket listeners (whose classes have been defined above by us) and connects the signal callbacks.
SocketClient* socketclient_init(const char *stun_ip, unsigned int stun_port,
const char *turn_ip, unsigned int turn_port)
{
cricket::SocketAddress *stun_addr = NULL;
if (stun_ip) {
stun_addr = new cricket::SocketAddress(std::string(stun_ip), stun_port);
}
cricket::SocketAddress *turn_addr = NULL;
if (turn_ip) {
turn_addr = new cricket::SocketAddress(std::string(turn_ip), turn_port);
}
cricket::PhysicalSocketServer *ss = new PhysicalSocketServer();
main_thread = new Thread(ss);
cricket::ThreadManager::SetCurrent(main_thread);
SocketClient *sc = new SocketClient (stun_addr, turn_addr);
// Note that signal connections pass the SignalListener1 object as well as the
// method. Since a new SocketListener1 is created for every new SocketClient,
// we have the guarantee that each SocketListener1 will be called back only
// in behalf of its related SocketClient.
sc->sigl1 = new SignalListener1(sc);
sc->sigl2 = new SignalListener2(sc);
sc->getSocketManager()->SignalNetworkError.connect(sc->sigl1, &SignalListener1::OnNetworkError);
sc->getSocketManager()->SignalState_s.connect(sc->sigl1, &SignalListener1::OnSocketState);
sc->getSocketManager()->SignalCandidatesReady.connect(sc->sigl1, &SignalListener1::OnCandidatesReady);
sc->CreateSocket(std::string("foobar"));
sc->getSocket()->SignalReadPacket.connect(sc->sigl2, &SignalListener2::OnSocketRead);
return sc;
}
This method is called back when LibJingle has some local candidates for connection, that should be sent to the remote site via the signaling protocol.
The beauty of ICE protocol is that parties will be able to agree on a P2P connection without having to exchange request/response messages. They just send connection candidates to each other. Each side selects the "best" way to send data based on received candidades; being both sides able to send data directly to the remote party, we have a P2P bidirectional channel.
void SignalListener1::OnCandidatesReady(const std::vector
{
printf("OnCandidatesReady called with %d candidates in list\n", candidates.size());
for(std::vector
char *marshaled_candidate;
asprintf(&marshaled_candidate, "%s %d %s %f %s %s %s\n",
(*it).address().IPAsString().c_str(), (*it).address().port(), (*it).protocol().c_str(),
(*it).preference(), (*it).type().c_str(), (*it).username().c_str(),
(*it).password().c_str() );
printf("Candidate being sent: %s", marshaled_candidate);
signaling_sendall(marshaled_candidate, strlen(marshaled_candidate));
free(marshaled_candidate);
}
}
An auxiliary function to send a data buffer through the signaling TCP channel. It does not return until all data has been sent.
void signaling_sendall(const char* buffer, unsigned int len)
{
unsigned int sent = 0;
while (sent < len) {
int just_sent;
just_sent = send(signaling_socket, buffer+sent, len-sent, 0);
if (just_sent < 0) {
printf("Signaling socket closed with error.\n");
exit(1);
} else if (just_sent == 0) {
printf("Signaling socket closed.\n");
exit(1);
}
sent += just_sent;
}
}
This function is called by the main loop when some signaling data arrives. It will find out if there is a complete P2P connection candidate in the buffer. If there is one, it is decoded.
// extracts remote candidates from a buffer, returns a pointer to the rest of the buffer
char* socketclient_add_remote_candidates(SocketClient *sc, char* buffer)
{
char *n;
char candidate[1024];
while (1) {
n = strchr(buffer, '\n');
if (! n) {
return buffer;
}
strncpy(candidate, buffer, n-buffer+1);
socketclient_add_remote_candidate(sc, candidate);
buffer = n+1;
}
}
Here, the P2P connection candidate is decoded and made known to LibJingle. Since LibJingle has its own thread and sockets, all further processing of P2P candidates is fortunately outside the scope of our code.
// Inform candidates received from the signaling network to LibJingle
void socketclient_add_remote_candidate(SocketClient *sc, const char* remote_candidate)
{
std::vector
char ip[100];
unsigned int port;
char protocol[100];
float preference;
char type[100];
char username[100];
char password[100];
// WARNING: using fixed-size buffers and sscanf is utterly unsafe.
// Real implementations must be more robust about data coming from the network!
sscanf(remote_candidate, "%s %d %s %f %s %s %s\n", ip, &port, protocol, &preference, type, username, password);
printf("Received new candidate: %s:%d pref %f\n", ip, port, preference);
Candidate candidate;
candidate.set_name("rtp");
candidate.set_address(SocketAddress(std::string(ip), port));
candidate.set_username(std::string(username));
candidate.set_password(std::string(password));
candidate.set_preference(preference);
candidate.set_protocol(protocol);
candidate.set_type(type);
candidate.set_generation(0);
candidates.push_back(candidate);
sc->getSocketManager()->AddRemoteCandidates(candidates);
}
Simple helper function that shows whether a P2P socket is writable (which means that the P2P connection is up).
bool socketclient_is_writable(SocketClient *sc)
{
return sc->getSocketManager()->writable();
}
Method that is called back when data arrives from the P2P connection. In our example, it is just a byte of data.
void SignalListener2::OnSocketRead(P2PSocket *socket, const char *data, size_t len)
{
printf("Received byte %d from remote P2P\n", data[0]);
}
Auxiliary function that sends data via P2P connection. Not really difficult.
void socketclient_send(SocketClient* sc, const char *data, unsigned int len)
{
sc->getSocket()->Send(data, len);
printf("Sent byte %d to remote P2P\n", data[0]);
}
The signaling server
As already said, our signaling server is incredibly simple and works more like a network pipe, forwarding data from one side to another. The P2P parties ogree on a P2P connection through this channel.
from select import select
import socket
import time
read_socks = []
port = 14141
server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.bind(("", port))
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.listen(5)
read_socks.append(server_sock)
# We accept two parties only
buffer = ""
while True:
rd, wr_dummy, ex_dummy = select(read_socks, [], [], 10)
if not rd:
print "-- tick --"
continue
rd = rd[0]
if rd is server_sock:
# incoming new connection
newsock, address = rd.accept()
print "New connection from %s" % str(address)
if len(read_socks) > 3:
# we only accept two parties at the most
newsock.close()
continue
read_socks.append(newsock)
if buffer:
# we already have data to be sent to the new party
newsock.sendall(buffer)
print " sent buffered data"
buffer = ""
continue
data = rd.recv(999999)
if not data:
# socket closed, remove from list
print "Connection closed"
del read_socks[read_socks.index(rd)]
buffer = ""
continue
if len(read_socks) < 3:
print "Buffering data"
# the other party has not connected; bufferize
buffer += data
continue
print "Forwarding data"
for wr in read_socks:
if wr is not rd and wr is not server_sock:
wr.sendall(data)
STUN and relay servers
The maemo libjingle-utils package includes both a STUN server and a relay server for testing purposes. To run a test server, do the following:
$ stunserver &
$ relayserver &
The relay server will print console messages when a P2P connection is flowing through it. If you need more detailed feedback (e.g. if you are debugging a P2P application), use a network sniffing tool like tcpdump or Ethereal to monitor UDP packets.
Wednesday, May 21, 2008
reboot linux
Context: CentOS 5. Postgresql data directory is locked and cannot reboot the server by $reboot
Solution:
$echo 1 > /proc/sys/kernel/sysrq
$echo b > /proc/sysrq-trigger
Above two commands will reboot the server.
Solution:
$echo 1 > /proc/sys/kernel/sysrq
$echo b > /proc/sysrq-trigger
Above two commands will reboot the server.
Friday, April 4, 2008
Using NowSMS as an MMSC in CDMA or CDMA2000 Environments
Delivering MMSC functionality in an CDMA or CDMA2000 environment can be challenging because the original WAP specifications for CDMA have technical requirements that require additional SMSC functionality.
(Technical note: CDMA2000 is different from WCDMA. CDMA2000 is an evolutionary upgrade path from CDMA. By contrast, WCDMA is an evolutionary upgrade from GSM/GPRS. CDMA2000 builds upon technical standards already deployed for CDMA, while WCDMA/UMTS builds upon technical standards already deployed for GSM/GPRS.)
MMS Message Notifications are generally sent via WAP Push over WDP (Wireless Datagram Protocol) over SMS. (It is also possible to deliver WAP Push over WDP over IP, but this is generally not practical unless there is an accessible database lookup for phone number to IP address translation. NowSMS currently can facilitate this IP only scenario, but in a way that is only suitable for test lab environments. This capability is described in a separate technical bulletin, and is outside of the scope of this bulletin.)
When a WAP Push message is larger than the network size limit for SMS (140 bytes in a GSM environment, and somewhat variable in CDMA environments), the WAP Push message must be segmented to enable it to be sent over multiple SMS messages.
In GSM environments, User Data Header (UDH) fields have been standardised to support the necessary segmentation, and the UDH parameters for segmentation are well understood and widely supported.
In CDMA environments, no standard for segmentation exists. Therefore, the WAP Forum defined its own standard for the sending of segmented WDP messages in a CDMA environment.
A key requirement of this standard is that each segment of the message must use the same CDMA SMS message id. However, the CDMA SMS message id field is a value that is generated by the SMSC, and cannot be set by an application that is submitting messages to the SMSC, such as an MMSC or WAP Push Proxy Gateway.
The authors of the WAP specification anticipated this limitation and defined a standard for WDP Adaptation over SMPP (WAP-159-WDPWCMPAdapt).
By default, NowSMS generates WAP Push messages (and MMS Notification messages which are based upon WAP Push) using a format that is specific to GSM (and WCDMA/UMTS) environments. However, there is also a configuration option defined on the "Advanced Settings" page of the SMPP configuration properties titled "Use WDP Adaptation for WAP Push and MMS Notifications (required for CDMA)". This setting is primarily used when connecting to a CDMA based SMSC. When this configuration option is enabled, NowSMS uses a protocol independent format known as "WDP Adaptation". This is usually the only practical option for delivering WAP Push messages in a CDMA environment. In this case, NowSMS will submit messages to the SMPP server using the "WAP" teleservice (SMPP service type), consistent with the WDP and WCMP Adaptation specification.
By default, even if a WAP push message is longer than 140 bytes, when this option is selected, NowSMS will deliver the complete message in a single submission to the SMSC. NowSMS then expects the SMSC to perform any necessary segmentation for delivering the message over SMS. (This expectation exists because the SMSC must use the same CDMA SMS message id for each segment of the message, and there is no place in the SMPP protocol for NowSMS to specify this message id.)
It is also possible to configure NowSMS to use segmentation for longer messages, even if WDP Adaptation is enabled by checking the "Use TLV parameters for port numbers and segmentation" option. In this case, NowSMS segments on a 140 byte boundary, the same as in GSM environments.
When configuring NowSMS in CDMA environments, segmentation issues are usually the biggest problem, because of the WAP protocol requirement that the SMSC use the same CDMA SMS message id for each segment of the message.
If you experience problems sending WAP Push and/or MMS notifications in a CDMA environment with WDP Adaptation, here are some suggestions:
WDP Adaptation (WAP-159-WDPWCMPAdapt). If you are connecting via an independent SMS provider, it is likely that they are not aware of this protocol option, but it is likely to be supported by the CDMA operator's SMSC.
Instead of troubleshooting by sending MMS messages, try sending simple WAP push messages first. Send a simple WAP push message via the NowSMS web interface (page 82), keeping the URL and text relatively short to ensure that the message can be delivered in a single segment. If that works, next try increasing the amount of text in the message. If the message delivery fails with longer WAP push messages, then you will want to take steps to limit the size of MMS notification messages that are generated by the NowSMS MMSC. This can be accomplished with the following suggestions:
1. Use the shortest possible host name (in number of characters) for the "Local Host Name or IP Address" setting of the MMSC. This host name must be included in the every MMS Notification that is generated by the MMSC. Every byte saved in the host name saves a byte in the size of the resulting notification messages.
2. If possible, use Port 80 as the "HTTP Port" for the MMSC. If a port other than port 80 is used, it must be appended to the host name when generating the URL.
3. Edit MMSC.INI, and under the [MMSC] header, add CompactMMSURL=Yes. This setting shortens the length of the dynamic path that is generated when sending MMS messages and will save a few bytes in every MMS notification with no other effect.
4. Edit MMSC.INI, and under the [MMSC] header, add MMSNotificationNoSubject=Yes. By default, NowSMS will include the message subject in the MMS Notification message. Having the subject present can help someone who has a mobile phone configured for manual message download to determine whether or not they want to download the message. However, the specification does not require the subject to be present in the notification, and omitting it can prevent long MMS notifications.
5. Edit MMSC.INI, and under the [MMSC] header, add Nokia3510Compatible=No. This setting reduces the size of the MMS notification by approximately 30 bytes, at the expense of compatibility with early model GSM MMS phones (specifically the original Nokia 3510 and Panasonic GD87). These phones did not understand the short form of MMS content type encoding in the MMS header, and required the long form. The MMS specifications do require that the handsets support the short form of this encoding, and technically these early model handsets were non-conformant. While GSM operators may want to maintain compatibility with these early model handsets, this should not be a consideration in CDMA environments.
NowSMS 2006 has also been extended to provide extensions to the HTTP SMSC interface to facilitate MMS and WAP Push in CDMA environments. This functionality is intended primarily for test lab environments (specifically with Agilent's CDMA test kit), but might also be useful for mobile operators whose SMSCs do not support WDP Adaptation.
To enable this support, define an HTTP SMSC connection in NowSMS as normal.
Next, manually edit the SMSGW.INI, and locate the [HTTP – server:port] section for this HTTP SMSC connection. Under this header, add CDMATemplateWAP=, where this is a URL template to be used for submitting WAP Push messages (MMS notifications are a special type of WAP Push). This URL template is similar to the existing text and binary templates, however you should use the @@CDMAWAPPDU@@ replacement parameter for NowSMS to insert the WAP PDU in CDMA format. (Do not use the @@TEXT@@, @@DATA@@, @@DATABIN@@ or @@UDH@@ parameters!)
By default, NowSMS will segment the CDMA WAP PDU to a size limit of 112 bytes, and issue multiple HTTP requests to submit the message in multiple segments. To change this size limit, specify CDMASizeLimit=### in the same section of the SMSGW.INI file.
An additional replacement parameter, @@CDMAWAPMMTS@@ is available. This parameter will be set to "1" for all segments except the final segment in a multiple segment WAP Push submission. This parameter will be set to "0" in a single segment WAP Push submission, and also in the final segment of a multiple segment WAP Push submission. ("MMTS" stands for "more messages to send", and is used to indicate that an additional segment will follow. This is necessary because in CDMA, the SMSC must apply the same MESSAGE_ID value to all segments of a multiple segment message.)
If the HTTP SMSC does not support the MMTS flag, then you should follow the advice that was given earlier in this bulletin regarding how to limit the size of the MMS Notification message.
Additionally, ensure that the URL template sets whatever flag is necessary to indicate that the CDMA WAP teleservice should be used.
(Technical note: CDMA2000 is different from WCDMA. CDMA2000 is an evolutionary upgrade path from CDMA. By contrast, WCDMA is an evolutionary upgrade from GSM/GPRS. CDMA2000 builds upon technical standards already deployed for CDMA, while WCDMA/UMTS builds upon technical standards already deployed for GSM/GPRS.)
MMS Message Notifications are generally sent via WAP Push over WDP (Wireless Datagram Protocol) over SMS. (It is also possible to deliver WAP Push over WDP over IP, but this is generally not practical unless there is an accessible database lookup for phone number to IP address translation. NowSMS currently can facilitate this IP only scenario, but in a way that is only suitable for test lab environments. This capability is described in a separate technical bulletin, and is outside of the scope of this bulletin.)
When a WAP Push message is larger than the network size limit for SMS (140 bytes in a GSM environment, and somewhat variable in CDMA environments), the WAP Push message must be segmented to enable it to be sent over multiple SMS messages.
In GSM environments, User Data Header (UDH) fields have been standardised to support the necessary segmentation, and the UDH parameters for segmentation are well understood and widely supported.
In CDMA environments, no standard for segmentation exists. Therefore, the WAP Forum defined its own standard for the sending of segmented WDP messages in a CDMA environment.
A key requirement of this standard is that each segment of the message must use the same CDMA SMS message id. However, the CDMA SMS message id field is a value that is generated by the SMSC, and cannot be set by an application that is submitting messages to the SMSC, such as an MMSC or WAP Push Proxy Gateway.
The authors of the WAP specification anticipated this limitation and defined a standard for WDP Adaptation over SMPP (WAP-159-WDPWCMPAdapt).
By default, NowSMS generates WAP Push messages (and MMS Notification messages which are based upon WAP Push) using a format that is specific to GSM (and WCDMA/UMTS) environments. However, there is also a configuration option defined on the "Advanced Settings" page of the SMPP configuration properties titled "Use WDP Adaptation for WAP Push and MMS Notifications (required for CDMA)". This setting is primarily used when connecting to a CDMA based SMSC. When this configuration option is enabled, NowSMS uses a protocol independent format known as "WDP Adaptation". This is usually the only practical option for delivering WAP Push messages in a CDMA environment. In this case, NowSMS will submit messages to the SMPP server using the "WAP" teleservice (SMPP service type), consistent with the WDP and WCMP Adaptation specification.
By default, even if a WAP push message is longer than 140 bytes, when this option is selected, NowSMS will deliver the complete message in a single submission to the SMSC. NowSMS then expects the SMSC to perform any necessary segmentation for delivering the message over SMS. (This expectation exists because the SMSC must use the same CDMA SMS message id for each segment of the message, and there is no place in the SMPP protocol for NowSMS to specify this message id.)
It is also possible to configure NowSMS to use segmentation for longer messages, even if WDP Adaptation is enabled by checking the "Use TLV parameters for port numbers and segmentation" option. In this case, NowSMS segments on a 140 byte boundary, the same as in GSM environments.
When configuring NowSMS in CDMA environments, segmentation issues are usually the biggest problem, because of the WAP protocol requirement that the SMSC use the same CDMA SMS message id for each segment of the message.
If you experience problems sending WAP Push and/or MMS notifications in a CDMA environment with WDP Adaptation, here are some suggestions:
WDP Adaptation (WAP-159-WDPWCMPAdapt). If you are connecting via an independent SMS provider, it is likely that they are not aware of this protocol option, but it is likely to be supported by the CDMA operator's SMSC.
Instead of troubleshooting by sending MMS messages, try sending simple WAP push messages first. Send a simple WAP push message via the NowSMS web interface (page 82), keeping the URL and text relatively short to ensure that the message can be delivered in a single segment. If that works, next try increasing the amount of text in the message. If the message delivery fails with longer WAP push messages, then you will want to take steps to limit the size of MMS notification messages that are generated by the NowSMS MMSC. This can be accomplished with the following suggestions:
1. Use the shortest possible host name (in number of characters) for the "Local Host Name or IP Address" setting of the MMSC. This host name must be included in the every MMS Notification that is generated by the MMSC. Every byte saved in the host name saves a byte in the size of the resulting notification messages.
2. If possible, use Port 80 as the "HTTP Port" for the MMSC. If a port other than port 80 is used, it must be appended to the host name when generating the URL.
3. Edit MMSC.INI, and under the [MMSC] header, add CompactMMSURL=Yes. This setting shortens the length of the dynamic path that is generated when sending MMS messages and will save a few bytes in every MMS notification with no other effect.
4. Edit MMSC.INI, and under the [MMSC] header, add MMSNotificationNoSubject=Yes. By default, NowSMS will include the message subject in the MMS Notification message. Having the subject present can help someone who has a mobile phone configured for manual message download to determine whether or not they want to download the message. However, the specification does not require the subject to be present in the notification, and omitting it can prevent long MMS notifications.
5. Edit MMSC.INI, and under the [MMSC] header, add Nokia3510Compatible=No. This setting reduces the size of the MMS notification by approximately 30 bytes, at the expense of compatibility with early model GSM MMS phones (specifically the original Nokia 3510 and Panasonic GD87). These phones did not understand the short form of MMS content type encoding in the MMS header, and required the long form. The MMS specifications do require that the handsets support the short form of this encoding, and technically these early model handsets were non-conformant. While GSM operators may want to maintain compatibility with these early model handsets, this should not be a consideration in CDMA environments.
NowSMS 2006 has also been extended to provide extensions to the HTTP SMSC interface to facilitate MMS and WAP Push in CDMA environments. This functionality is intended primarily for test lab environments (specifically with Agilent's CDMA test kit), but might also be useful for mobile operators whose SMSCs do not support WDP Adaptation.
To enable this support, define an HTTP SMSC connection in NowSMS as normal.
Next, manually edit the SMSGW.INI, and locate the [HTTP – server:port] section for this HTTP SMSC connection. Under this header, add CDMATemplateWAP=, where this is a URL template to be used for submitting WAP Push messages (MMS notifications are a special type of WAP Push). This URL template is similar to the existing text and binary templates, however you should use the @@CDMAWAPPDU@@ replacement parameter for NowSMS to insert the WAP PDU in CDMA format. (Do not use the @@TEXT@@, @@DATA@@, @@DATABIN@@ or @@UDH@@ parameters!)
By default, NowSMS will segment the CDMA WAP PDU to a size limit of 112 bytes, and issue multiple HTTP requests to submit the message in multiple segments. To change this size limit, specify CDMASizeLimit=### in the same section of the SMSGW.INI file.
An additional replacement parameter, @@CDMAWAPMMTS@@ is available. This parameter will be set to "1" for all segments except the final segment in a multiple segment WAP Push submission. This parameter will be set to "0" in a single segment WAP Push submission, and also in the final segment of a multiple segment WAP Push submission. ("MMTS" stands for "more messages to send", and is used to indicate that an additional segment will follow. This is necessary because in CDMA, the SMSC must apply the same MESSAGE_ID value to all segments of a multiple segment message.)
If the HTTP SMSC does not support the MMTS flag, then you should follow the advice that was given earlier in this bulletin regarding how to limit the size of the MMS Notification message.
Additionally, ensure that the URL template sets whatever flag is necessary to indicate that the CDMA WAP teleservice should be used.
Tuesday, April 1, 2008
make Nokia phone E65 as modem for SMS
(1)Mac/E65
(2)Bluetooth connectivity
(3)minicom installed on Mac
(4)bluetooth standard provides modem interface. So, Making starting bluetooth makes modem available. The serial port is under /dev/tty....(Computer name)
(5)$minicom -s
setup the modem and save as "e65". Then minirc.e65 will be saved.
(6)$minicom e65, which will connect to E65 phone.
(7)Type AT commands, the phone will response properly.
(2)Bluetooth connectivity
(3)minicom installed on Mac
(4)bluetooth standard provides modem interface. So, Making starting bluetooth makes modem available. The serial port is under /dev/tty....(Computer name)
(5)$minicom -s
setup the modem and save as "e65". Then minirc.e65 will be saved.
(6)$minicom e65, which will connect to E65 phone.
(7)Type AT commands, the phone will response properly.
pys60 TCP/IP console
(1)Mac/E65/Wireless router
(2)Mac IP : 192.168.1.64
(3)on Mac run: $stty raw -echo ; nc -l -p 1025; stty sane
(4)vi btc.py file with following codes:
import btconsole
from socket import *
sock=socket(AF_INET, SOCK_STREAM)
sock.connect(("192.168.1.64",1025))
btconsole.run_with_redirected_io(sock,btconsole.interact,None,None,locals())
sock.close()
(5)bluetooth btc.py to E:\\Python folder on E65
(6)Run btc.py through pys60 shell. It will search Mac and connect to Mac console.
(2)Mac IP : 192.168.1.64
(3)on Mac run: $stty raw -echo ; nc -l -p 1025; stty sane
(4)vi btc.py file with following codes:
import btconsole
from socket import *
sock=socket(AF_INET, SOCK_STREAM)
sock.connect(("192.168.1.64",1025))
btconsole.run_with_redirected_io(sock,btconsole.interact,None,None,locals())
sock.close()
(5)bluetooth btc.py to E:\\Python folder on E65
(6)Run btc.py through pys60 shell. It will search Mac and connect to Mac console.
Subscribe to:
Posts (Atom)