TCP Header Anaylsis - Section 4: TCP Flag Options

As we have seen in the previous pages, some TCP segments carry data while others are simple acknowledgements for previously received data. The popular 3-way handshake utilises the SYNs and ACKs available in the TCP to help complete the connection before data is transferred.

Our conclusion is that each TCP segment has a purpose, and this is determined with the help of the TCP flag options, allowing the sender or receiver to specify which flags should be used so the segment is handled correctly by the other end.

Let's take a look at the TCP flags field to begin our analysis:

You can see the 2 flags that are used during the 3-way handshake (SYN, ACK) and data transfers.

As with all flags, a value of 1 means that a particular flag is set or, if you like, is on. In this example, only the SYN flag is set, indicating that this is the first segment of a new TCP connection.

In addition to this, each flag is one bit long, and since there are 6 flags, this makes the Flags section 6 bits in total.

 You would have to agree that the most popular flags are the SYN, ACK and FIN, used to establish connections, acknowledge successful segment transfers and, lastly, terminate connections. While the rest of the flags are not as well known, their role and purpose makes them, in some cases, equally important.

We will begin our analysis by examining all six flags, starting from the top, that is, the Urgent Pointer:

1st Flag - Urgent Pointer

The first flag is the Urgent Pointer flag, as shown in the above screen shot. This flag is used to identify incoming data as urgent, therefore incoming segments do not have to wait until the previous segments are processed by the receiving end but are sent directly and processed immediately.

An a packet with the Urgent Pointer enabled could be used during a stream of data transfer where a host is sending data to an application running on a remote machine. If a problem appears, the host machine needs to abort the data transfer and stop the data processing on the other end. Under normal circumstances, the abort signal will be sent and queued at the remote machine until all previously sent data is processed, however, in this case, we need the abort signal to be processed immediately.

By setting the abort signal's segment Urgent Pointer flag to 1, the remote machine will not wait till all queued data is processed and then execute the abort. Instead, it will give that specific segment priority, processing it immediately and stopping all further data processing.

If you're finding it hard to understand, consider this real-life example:

At your local post office, hundreds of trucks are unloading bags of letters from all over the world. Because the amount of trucks entering the post office building are abundant, they line up one behind the other, waiting for their turn to unload their bags.

As a result, the queue ends up being quite long. However, a truck with a big red flag suddenly joins the queue and the security officer, whose job it is to make sure no truck skips the queue, sees the red flag and knows it's carrying very important letters that need to get to their destination urgently. By following the normal procedures, the security officer signals to the truck to skip the queue and go all the way up to the front, giving it priority over the other the trucks.

In this example, the trucks represent the segments that arrive at their destination and are queued in the buffer waiting to be processed, while the truck with the red flag is the segment with the Urgent Pointer flag set.

A further point to note is the existence of theUrgent Pointer field. This field is covered in section 5, but we can briefly mention that when the Urgent Pointer flag is set to '1' (that's the one we are analysing here), then the Urgent Pointer field specifies the position in the segment where urgent data ends.

2nd Flag - ACKnowledgement (ACK)

The ACKnowledgement flag is used to acknowledge the successful receipt of packets.

If you run a packet sniffer while transferring data using the TCP, you will notice that, in most cases, for every packet you send or receive, an ACK follows. So if you received a packet from a remote host, then your workstation will most probably send one back with the ACK field set to 1.

In some cases where the sender requires one ACK for every 3 packets sent, the receiving end will send the ACK expected once (the 3rd sequential packet is received). This is also called Windowing and is covered extensively in the pages that follow.

3rd Flag - PUSH

The Push flag, like the Urgent flag, exists to ensure that the data is given the priority (that it deserves) and is processed at the sending or receiving end. This particular flag is used quite frequently at the beginning and end of a data transfer, affecting the way the data is handled at both ends.

When developers create new applications, they must make sure they follow specific guidelines given by the RFC's to ensure that their applications work properly and manage the flow of data in and out of the application layer of the OSI model flawlessly. When used, the Push bit makes sure the data segment is handled correctly and given the appropriate priority at both ends of a virtual connection.

When a host sends its data, it is temporarily queued in the TCP buffer, a special area in the memory, until the segment has reached a certain size and is then sent to the receiver. This design guarantees that the data transfer is as efficient as possible, without waisting time and bandwidth by creating multiple segments, but combining them into one or more larger ones.

When the segment arrives at the receiving end, it is placed in the TCP incoming buffer before it is passed onto the application layer. The data queued in the incoming buffer will remain there until the other segments arrive and, once this is complete, the data is passed to the application layer that's waiting for it.

While this procedure works well in most cases, there are a lot of instances where this 'queueing' of data is undesirable because any delay during queuing can cause problems to the waiting application. A simple example would be a TCP stream, e.g real player, where data must be sent and processed (by the receiver) immediately to ensure a smooth stream without any cut offs.

A final point to mention here is that the Push flag is usually set on the last segment of a file to prevent buffer deadlocks. It is also seen when used to send HTTP or other types of requests through a proxy - ensuring the request is handled appropriately and effectively.

4th Flag - Reset (RST) Flag

The reset flag is used when a segment arrives that is not intended for the current connection. In other words, if you were to send a packet to a host in order to establish a connection, and there was no such service waiting to answer at the remote host, then the host would automatically reject your request and then send you a reply with the RST flag set. This indicates that the remote host has reset the connection.

While this might prove very simple and logical, the truth is that in most cases this 'feature' is used by most hackers in order to scan hosts for 'open' ports. All modern port scanners are able to detect 'open' or 'listening' ports thanks to the 'reset' function.

The method used to detect these ports is very simple: When attempting to scan a remote host, a valid TCP segment is constructed with the SYN flag set (1) and sent to the target host. If there is no service listening for incoming connections on the specific port, then the remote host will reply with ACK and RST flag set (1). If, on the other hand, there is a service listening on the port, the remote host will construct a TCP segment with the ACK flag set (1). This is, of course, part of the standard 3-way handshake we have covered.

Once the host scanning for open ports receives this segment, it will complete the 3-way handshake and then terminate it using the FIN (see below) flag, and mark the specific port as "active".

5th Flag - SYNchronisation (SYN) Flag

The fifth flag contained in the TCP Flag options is perhaps the most well know flag used in TCP communications. As you might be aware, the SYN flag is initialy sent when establishing the classical 3-way handshake between two hosts:

In the above diagram, Host A needs to download data from Host B using TCP as its transport protocol. The protocol requires the 3-way handshake to take place so a virtual connection can be established by both ends in order to exchange data.

During the 3-way handshake we are able to count a total of 2 SYN flags transmitted, one by each host. As files are exchanged and new connections created, we will see more SYN flags being sent and received.

6th Flag - FINished (FIN) Flag

The final flag available is the FIN flag, standing for the word FINished. This flag is used to tear down the virtual connections created using the previous flag (SYN), so because of this reason, the FIN flag always appears when the last packets are exchanged between a connection.

It is important to note that when a host sends a FIN flag to close a connection, it may continue to receive data until the remote host has also closed the connection, although this occurs only under certain circumstances. Once the connection is teared down by both sides, the buffers set aside on each end for the connection are released.

A normal teardown procedure is depicted below:

The above diagram represents an existing connection betwen Host A and B, where the two hosts are exchanging data. Once the data transfer is complete, Host A sends a packet with the FIN, ACK flags set (Step 1).

With this packet, Host A is ACKnowledging the previous stream while at the same time initiating the TCP close procedure to kill this connection. At this point, Host A's application will stop receiving any data and will close the connection from this side.

In response to Host A's request to close the connection, Host B will send an ACKnowledgement (STEP 2) back, and also notify its application that the connection is no longer available. Once this is complete, the host (B) will send its own FIN, ACK flags (STEP 3) to close their part of the connection.

If you're wondering why this procedure is required, then you may need to recall that TCP is a Full Duplex connection, meaning that there are two directions of data flow. In our example this is the connection flow from Host A to Host B and vice versa. In addition, it requires both hosts to close the connection from their side, hence the reason behind the fact that both hosts must send a FIN flag and the other host must ACKnowledge it.

Lastly, at Step 4, Host A willl acknowledge the request Host B sent at STEP 3 and the closedown procedure for both sides is now complete!

Summary

This page dealt with the TCP Flag Options available to make life either more difficult, or easy, depending on how you look at the picture :)

Perhaps the most important information given on this page that is beneficial to remember is the TCP handshake procedure and the fact that TCP is a Full Duplex connection.

The next section will examine the TCP Window size, Checksum and Urgent Pointer fields, all of which are relevant and very important. For this reason we strongly suggest you read through these topics, rather than skip over them.

Next: TCP Header Anaylsis - Section 5: TCP Window size, Checksum & Urgent Pointer