Often the first tool used to do port scan - whether ethical or malicious - is nmap, because of its power and flexibility in network discovery. It provides various scanning techniques we will discuss to gather information about target hosts, their services running, and their operating systems. Nmap remains a go-to tool used by the best hackers to learn more about their targets to discover what sorts of vulnerabilities may be found on a system.

Because of this, Trapster now provides portscan detection alerts for SYN scans, XMAS scans, NULL scans, OS detection, and FIN scans.

In this article we will discuss the differences between these scans and some characteristics that make them recognizable.

SYN scan

The SYN scan is the default and most common scan type in Nmap. This scan sends a SYN packet to the target, and if the port is open, the target will respond with a SYN-ACK packet however Nmap doesnt complete the handshake and instead sends a RST packet to close the connection.

This can harder to detect because sending a SYN packet is the first step in the TCP three-way handshake so a SYN packet itself isn’t unusual. However, when we see one IP address sending SYN packets to many different ports in a short period of time, it becomes suspicious and this is what we keep track of and detect in Trapster.

XMAS scan

This scan sets the FIN, PSH, and URG flags in the tcp packet however this is an incorrectly formatted packet and therefore expects receive back a reset packet from an open port . This is easy to detect because we dont typically see these flags set together.

The Xmas scan stands out because these flags don’t logically go together. For example, the FIN flag indicates the end of a connection, but the URG flag marks the packet data as urgent. These flags sending conflicting signals make this scan relatively easy to spot.

NULL Scan & FIN Scan

Both the NULL scan and the FIN scan are simple but effective techniques used to probe open ports. The NULL scan sends a packet with no TCP flags set, while the FIN scan sends a packet with only the FIN flag, which is normally used to terminate a connection. Like the Xmas scan, these scans are intended to elicit a response from closed ports (RST packets) to identify open ones.

OS detection scan

OS detection is a critical phase in the reconnaissance process. Hackers use it to identify the operating system running on a target machine, which can reveal specific vulnerabilities. It is also less sneaky than the other scans because it requires a larger number of packets of different types.

It first runs an initial syn scan but then the behavior will change depending on what sort of responses nmap is receiving. With Trapster, we monitor for suspicious behavior where an attacker first sends a SYN scan and follows it up with a series of unusual packets, such as those with URG, PSH, FIN, or SYN/FIN combinations. This signature pattern makes OS detection scans easier to spot.

Conclusion

Using Trapster, your traffic is being monitored for these abnormal patterns and alerts you the moment a potential hacker begins reconnaissance on your network. This early detection gives you the opportunity to take steps before an attack even takes place.