Linux Router for small ISP

What was discussed on numerous occasions on our forum speedtest.pl, was the subject of the so-called long 'pings' which appear when the network is heavily loaded. In other words, I mean a situation when a few people start the peer-to-peer application. When there is no proper queuing implemented in the router, then the aggressive p2p pushes the remaining network movement out, by extension also the one from ICMP protocol (ping). In order to be successful, you need properly balanced and priority services in the script that manages the band on the router. Such an operation is expected not only to shorten time of the answer to the ping from the 'outside world'. Thanks to solutions described in the below article, we can shape the movement going through our router at will. I hope that this article will become helpful for many computer network administrators. Let's get to the point..

Everything takes place in Linux environment (distribution practically does not matter but personally I recommend Slackware). I presume that people reading this article are equipped with basic knowledge on compilations, patching and software installation in Linux systems and that they are familiar with basic principles of working with packet filter iptables and a tool for queuing tc. The following elements influence the system so that it manages the band properly:

- linux kernel in version 2.6.15.x

- patch to linux kernel -2.6.14-imq6.diff

- patch to kernel esfq-kernel.patch

- latest iptables (1.3.5 currently)

- patch for the iptables iptables-1.3.0-imq1.diff

- ipp2p module for iptables

- packet iproute2

- patch for iproute2 esfq-iproute2.patch

I suggest installation of the software in the above-stated order. Kernel 2.6.x is much more stable than series 2.4.x. The above configuration was tested by me on a big group of routers (10+) and it has been working perfectly well. Needles to say, in Linux, you can only limit the output movement from a given interface with tc (Traffic Control). What we need then is a virtual mediation interface (the so-called IMQ) to which the Input movement should be redirected (the so-called Input). Thanks to such an operation, the input movement could be limited as output movement from a virtual IMQ. Obviously, queuing of solutions can be done without IMQ but I do not recommend this method because it is based up feigning packets, which makes later classification and prioritization impossible. I need to say a few more words on the interfaces. No network card based on Realtek's chipset should be used (especially RTL 8319), as they are devices designed for working posts and they lose work stability when there are more connections, which is manifested in an increase of the 'pings' and in many other awkward phenomena. Instead of them I recommend cards based upon Intel Pro or 3com.

Another aspect is the ESFQ mechanism (Enhanced Stochastic Fair Queuing) for the WAN router. In comparison to the traditionally used SFQ mechanism, it enables just division of links with a view to connections from the source and target address (hash src, hash dst - upload, download to WAN). Thanks to this, we are able to make WAN interfaces totally independent from LAN interfaces. Briefly speaking - no queuing is necessary for LAN, the whole movement will be justly separated between them. All of the above solutions refer obviously to HTB mechanism (Hierarchic Token Bucket).

In order to steer our movement on the router correctly, we need to create a few htb classes in WAN interface. What is a good solution is creation of four classes with rising priorities (prio 1,2,3,4). The first class includes the so-called interactive movement, that is movement on the port sr 22 (ssh), 53 (dns), ping (protocol 1), and all types of movement connected with network games (these are UDP < 512b packets, they have to be properly feigned with the iptables, preferably with connmark method).The second class is the standard services, such as post mailing (ports: 25, 110, 995), www (ports: 80, 443) and VoIP (connmark to port 5060). The third class is a default class which must be pointed out in the main queue of the interface. Each unclassified movement arrives at it. The last class with the highest priority are p2p services. The question of how to make p2p movement stand out from the others arrives in our minds immediately. After all, p2p programs can work on various ports. What comes in helpful is ipp2p module for the iptables. It can distinguish packets initiatiin p2p connections. Thanks to it those packets can be feigned and placed in a proper class. Unfortunately, further packets can be only of transmitting character link in case of ftp or www. Ipp2p does not mark them properly. This notion can be solved by proper marking of the whole connection, which begins in a given packet (connmark method).

Having properly created classes or ones with given priorities, we need to assign the proper band capacity to them. In the parameter ceil of tc filter (maximal value of the transfer) the rate of the main queue value (guaranteed value of the transfer) can be entered into each class. However, I advise you to write 10% less, whereas the rate parameter for every class, determining the guaranteed transfer in maximal saturation of the link, must be determined according to your own individual needs. In my case the following division worked really well 25/60/10/5 % subsequently for particular classes. As it is known, indicator of 'functioning' of the Internet for a common user are www, mail, text communicators and VoIP. This is where 60% of the band is destined for these services. Additionally, these services work usually occasionally (the exceptions include downloading files of considerable size via www), so at the moment they push the p2p services out when it comes to transfer. It must be also noticed that the htp mechanism itself is characterized by inertia of about 10%. In order to avoid a situation in which the link is saturated over the maximum, the main WAN queue must have a 10% link capacity set. Queuing services in this way and their prioritization guarantee comfort when working on the Internet with maximal network saturation. It is known that more than one local (and not only...) operator tries to connect the greatest possible number of users to his network. This solution is therefore a certain compromise between operators offering cheap Access-to-the-Internet services and the always unsatisfied with hardly efficient work of the client network.