Python sockets/port forwarding

You have to change the socket.gethostname() in the server script to the empty string (or just directly call socket.bind(('', port))).

Your problem is not in Python but in the usage of sockets generally. When you create a socket, you just prepare your process to receive/send some data from/to another process.

Server

The first step for creating a socket is to specify what kind of protocol will be used for communication between those processes. In your case it is the socket.AF_INET which is constant for use of IP protocol and the socket.SOCK_STREAM is specify reliable stream-oriented service. The reliable stream-oriented service means that you want to be sure that every single sent byte will be delivered to the other side and nothing can be lost during the communication (the underlying OS will use the TCP protocol for that). From this point we are using the IPv4 protocol (because we set the socket.AF_INET).

The second step is bind it to an address. The bind process assigns an address where you expect a client will join (with your socket’s settings it’s a IP address and the TCP port). Your PC has multiple IP address (well, at least two). It always has 127.0.0.1, which is called "callback" and works only when your applications communicate on the same PC (that is you Linux – Linux scenario in the question), and then you have your external IP address, for communication with other computers (let’s pretend it is 10.0.0.1).

When you call socket.bind(('127.0.0.1', 5555)), you’re setting the socket to listen only for communication from the same PC. If you call socket.bind(('10.0.0.1', 5555)), then the socket setting is ready to receive data targeted to the 10.0.0.1 address.

But what if you have 10 IPs or more and you want to receive everything (with the right TCP port)? For those scenarios you can leave the IP address in bind() empty, and it does exactly what you want.

With Python’s version of bind(), you can also enter a "computer name" instead of the concrete IP. The socket.gethostname() call returns your computer’s name. The problem is in the translation of "computer name" to the IP which Python performs behind your back. The translation has some rules but generally your "computer name" can be translated into any IP address which you have set on your computer. In your case, the your computer’s name is converted into 127.0.0.1, and that’s why communication works only between processes on the same computer.

After socket.bind(), you have the socket ready to use but it is still "inactive". The call to socket.listen() activates the socket and causes it to wait until it receives an attempted connection. When a socket receives a new connection request, it will put it into a queue and wait for processing.

That’s what socket.accept() does. It pulls the connection request from the queue, accepts it, and establishes the stream (remember the socket.SOCK_STREAM when you set up the socket) between the server and the client. The new stream is actually a new socket, but is ready to communicate with other side.

What happened with the old socket? Well, it’s still alive, and you can call socket.listen() again to get another stream (connection).

How is it possible to have multiple sockets on the same port?

Every connection within computer’s network is defined by flow which is 5-item tuple of:

• L4 protocol (usually TCP or UDP)
• Source IP address
• Source L4 port
• Destination IP address
• Destination L4 port

When you create a new connection with a client, the flow can look like this: (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555). Just for clarification, the server’s response flow is (TCP, 10.0.0.1, 55555, 192.168.0.1, 12345), but it isn’t important for us. If you create another connection with a client, that it will differ at source TCP port (if you do it from another computer that it will differ also at the source IP). Only from this information you can distinguish every connection created to your computer.

When you create a server socket in your code and call socket.listen(), it listens for any flow with this pattern (TCP, *, *, *, 55555) (the * means "match everything"). So when you get a connection with (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555), then socket.accept() creates another socket which works only with this one concrete flow while the old socket carries on accepting new connections which haven’t yet been established.

When the operating system receives a packet, it looks in the packet and checks the flow. At this point, several scenarios can take place:

• The packet’s flow matches all 5 items exactly (without usage of *). Then the packet’s content is delivered to the queue associated with that socket (you’re reading the queue when you call socket.recv()).
• The packet’s flow matched socket with associated flow contains * then it is considered as new connection and you can call scoket.accept().
• The operating system doesn’t contain open socket which would match the flow. In that case the OS refuse connection (or just ignore the packet it depends on firewall settings).

Probably an example can clarify these scenarios. The operating system has something like a table where it maps flows to sockets. When you call socket.bind(), it will assign a flow to the socket. After the call, the table can look like this:

+=====================================+========+
|                Flow                 | Socket |
+=====================================+========+
| (TCP, *, *, *, 55555)               |      1 |
+-------------------------------------+--------+

When it receive a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 10) then it won’t match any flow (last port won’t match). So, the connection is refused. If it receives a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 55555), the packet is delivered to the socket 1 (because there is a match). The socket.accept() call creates a new socket and record in the table.

+=====================================+========+
|                Flow                 | Socket |
+=====================================+========+
| (TCP, 1.1.1.1, 10, 10.0.0.1, 55555) |      2 |
+-------------------------------------+--------+
| (TCP, *, *, *, 55555)               |      1 |
+-------------------------------------+--------+

Now you have 2 sockets for 1 port. Every received packet which matches the flow associated with the socket 2 also matches the flow associated with socket 1 (on the contrary, it does not apply). It’s not a problem because the socket 2 has a preciser match (is doesn’t use the *), so any data with that flow will be delivered to socket 2.

How to serve multiple connections

If you want to implement a "real" server, your application should be able to process multiple connections without restarting. There are 2 basic approaches:

1. Sequential processing

   try:
       l = prepare_socket()
       while True:
           l.listen()
           s, a = socket.accept()
           process_connection(s) # before return you should call s.close()
   except KeyboardInterrupt:
       l.close()
   

   In this case, you can process only one client while others clients have to wait for accept. If the process_connection() takes too long, then others clients will timeout.

2. Parallel processing

   import threading
   threads = []
   
   try:
       l = prepare_socket()
       while True:
           l.listen()
           s, a = socket.accept()
           t = threading.Thread(target=process_connection, s)
           threads.append(t)
           t.start()
   except KeyboardInterrupt:
       for t in threads:
           t.join()
       l.close()
   

   Now when you receive a new connection, it will create a new thread so that every connection is processed in parallel. The main disadvantage of this solution is that you have to solve common troubles with threading (like access to shared memory, deadlocks etc.).

Beware that the above snippets are only examples, and are not complete! For example, they don’t contain code for graceful exit on unexpected exceptions.

Servers in Python

Python also contains a module called socketserver, which contains shortcuts for creating servers in Python. You can find examples of how to use it here.

Client

With the client, it’s much more simpler than with the server. You just have to create a socket with some settings (same as server side), and then tell it where the server is (what its IP and TCP port are). This is accomplished through the socket.connect() call. As a bonus, it also establishes the stream between your client and server, so from this point you can communicate.

---

You can find more information about sockets at the Beej’s Guide to Network Programming. It’s written for usage with C, but the concepts are the same.