How does the Listener Work?

My previous post about bypassing the listener was a big success. It actually broke my daily record (the previous one was the Oracle Challenge).
Anyway, because of this success, I’ve decided to write another post on the same topic. Actually, a more basic post that explains how the listener works during a connection (which I mentioned shortly in the previous post). It is a bit long and technical, I hope you’ll enjoy it, and I would appreciate any comment.

A Little Bit about TCP/IP

In order to understand the listener connections, we first need to know a little bit about TCP/IP. This is not the only protocol Oracle’s listener supports, but it is without a doubt the most common one. TCP/IP is a networking protocol with 4 layers (I don’t even want to mention the full 7 layers protocol, if you ever heard about it). I won’t bother you with too many details (and there are lots of them), so I’ll just stick to the basics and what’s important. In order to understand the listener’s networking, we mainly need to understand the 2 middle layers of the protocol (as the lowest one is the infrastructure and hardware, and the highest one is the application, which in our case is Oracle’s networking protocol, SQL*Net).
In order to give you some idea about the TCP/IP protocol, I will use the mailing system (the good old post, not the new email). In TCP/IP we have 4 layers, so I guess that in the mail system the first layer will be the post office itself, the postmen, etc. The second layer will be the addressing system (streets, cities, postal codes), the third will be the envelopes and papers we write on, and the last one will be the letters and language we use.
The second layer in TCP/IP is called internet and it is responsible for addressing (as I explained above). The protocol used for this is called IP (Internet Protocol) and this is how we find and access computers on the network. There are 2 different IP versions today, IPv4 and IPv6, the former is still the more common one, and the latter was created because there are not enough addresses for every connected device in IPv4. So every computer (or device) that is connected to a network has an IP address, and this is how we can reach it. But there are many things we can do with a device we connect to, and that is the third level called transport. The protocol we use in this layer is TCP (Transmission Control Protocol). We call the entire protocol TCP/IP but actually it doesn’t have to use TCP at all. TCP is only one of several available protocols for this layer, but it is very common. It is responsible for transmitting the data and making sure the other side will receive it.
When we transfer data to another computer, IP is responsible for getting to the correct computer and allowing it to know which address to return the answer to. TCP is responsible for cutting our data into pieces, delivering it to the other computer and making sure every piece will arrive and in the correct order.
Another important part of TCP is the port. Port is actually a virtual thing, it doesn’t really say anything, but it was created to separate different connections to the computer. Let’s say we open several browser windows and in every window we use Google search to find something. Browsing uses TCP, so the computer has several connections to Google’s web servers. The connections all look the same – all are browsers, all are TCP from the same computer to the same web server. So when my computer gets data from Google, it needs to know which window this data belong to. And in order to do this we use ports. Every time we send data to Google, the data will contain the IP addresses and the TCP ports (which don’t change during the connection) and this is the way to identify the connection.
There is a lot more than that in TCP/IP. The only thing I’ll add right now is that in order to establish a TCP connection to another computer we need to perform a “handshake” between the computers, so they can know each other, send some details and only after the connection is established they can start sending data.

Listening on a TCP port

In my explanation about TCP/IP so far, one basic thing is missing. When we try to connect to a remote computer, how can we reach a specific program? When a program is willing to get TCP connections from the network, it allocates a TCP port from the operating system. From now on, as long as it is up and running, every connection request for this specific port will get to that piece of software. And as you probably understand, this is exactly what our listener does. When we set up the listener, we configure what host or IP it will listen on, and what port. Using this information it will allocate a port from the OS on that IP/hostname to listen on the network. If we have several IP addresses, we can configure the listener to listen on a specific IP address or several IP addresses, on one port or several. However, for each combination of IP and port, only one program can listen. So if we configure one listener to listen on IP A port X, a different listener (or any other software that listens on the network) can listen on IP A and a different port, or port X but different IP, or different IP and different port.
Once the listener is up, it waits for connection requests from clients. The clients need to know the IP and port and they will start the TCP handshake. After the connection is established, the data from the client will reach the listener to create the connection at Oracle’s level.

Dynamic Ports

When a program needs to connect to other server on the network, it must allocate a TCP port as well (so the other side of the connection can also reach it), but it doesn’t make any sense to configure a port in advance for that. We don’t know how many connections will be established and by which application. The OS takes care of the ports in these cases. When a program asks to connect through the network to some remote server, the OS allocates a dynamic port for it. This port is taken from a pool of unused ports and will return to the pool once the connection is dropped. The pool range depends on the OS and version, but in any case, the port range 1-1024 is reserved for specific purposes and will not be used for dynamic allocation.

Connecting to Oracle

After we’ve covered the basics, let’s talk about connecting to Oracle. When a client tries to connect to Oracle it needs three pieces of information about the database server: the host (IP or hostname), the port and the SID or service_name to connect to. The first two are the network properties of the listener, the last one is used by the listener later. The client establishes a TCP connection with the listener (the handshake I mentioned) and then asks it to connect to the instance. The listener itself is only a connection mechanism, it doesn’t check username and password and doesn’t perform any operation at Oracle’s level besides connecting the client to the correct instance. In order to do this, the listener needs to create a server process (assuming we use dedicated server) and direct the client to it. In UNIX/Linux, the listener uses the fork system call (an OS command) to create the server process and when doing this it can hand-off the connection to the new server process without any interference to the existing TCP connection. This is something that UNIX/Linux allow. Windows, on the other hand, doesn’t allow this. So in Windows, the listener needs to create a server process (which is a thread of oracle.exe process), and because the connection cannot be transferred, the server process itself needs to start listening on a port (it allocates a dynamic port for this). The listener knows which port the server process is listening on, and sends the client a redirect command to this port. At this stage, the client will establish a new connection (including the handshake) with the newly created server process, and then will continue with the connection to Oracle. This is the reason that firewall rules for Oracle connections are a bit trickier when the server is Windows.
If we are not using a dedicated server architecture, but rather a shared server, then the connection needs to be redirected to a dispatcher. The listener does not create the dispatchers, because they start with the database (and allocate dynamic ports to listen on as they start), so it cannot hand-off connections to them. In this case it has to act in a similar way as a dedicated server on Windows. The listener sends a redirect request to the client, who reconnects to the dispatcher.
Now it is very simple to understand a load balance in RAC environment as well. If the listener we are connected to decides to connect us to the local instance, it creates a new server process to which it hands-off or redirects the connection, and everything is exactly as in a single instance. If it decides to connect us to a remote instance, it will send a redirect request to the client to ask it to reconnect to a different node.

Summary

Most of this post was not about Oracle, but on TCP/IP and networking. I hope I didn’t go into too many details and didn’t confuse anyone. Again, comments are very welcome, hope you enjoyed.