Test bed firewall performance testing

Informally, the purpose of this experiment was to determine how many PCs the firewall could cope with being plugged into the hub. The experiment was designed to test if the firewall would become a performance bottleneck when multiple machines were placed behind it. Specifically, we examined the data transfer rate achieved by a given PC with varying amounts of other traffic simultaneously passing through the firewall.

The test conditions:

One PC, called the test PC, was set up to FTP four files from the host zeus, which resides on the same subnet as the PC, but is on the opposite side of the firewall. Selecting a host on the same subnet as the PC should minimise the affect of other network traffic on the time trial. The physical layout is shown here:

Figure 6: architecture used for performance testing

The four files that were FTP'd and their sizes are as follows:

   termcap   183935 bytes 
   bash      295930 bytes 
   image     469988 bytes 
   tcsh      209924 bytes
             ------ 
             115977 bytes 

The firewall rule set was altered only to permit FTP through. The firewall rule set was not changed in any other way.

Series one

In the first test, there was one PC behind the firewall, the test PC. The time taken by this PC to FTP the selected files from zeus was recorded. No other traffic was directed through the firewall.

This test was run three times.

Series two

In the second test, there were two PCs behind the firewall: the test PC and one "traffic-generating" PC. The test PC was set up to FTP the selected files from zeus, as normal. The traffic generating PC was set up to FTP another set of files from the host keppel. keppel was chosen so as not to place extra load on zeus, it was important that zeus did not become part of the bottleneck. The files FTP'd by the traffic generating PC were:

   mail.exe       151552 bytes 
   maillite.exe    30208 bytes 
   mailshow.exe    45056 bytes 
   alml300F.zip  1949333 bytes 
   comctl32.dlx   371984 bytes 
   ctl3d32.dlx     37888 bytes 
                 ------- 
                 2586021 bytes 

I initiated the FTP process on the second PC, and then immediately initiated the FTP process on the test PC. The timing data from each PC was recorded.

This test was run three times.

Series three

In this test, there were three PCs behind the firewall, the test PC, and two traffic-generating PCs. The test PC was set up to FTP its files from zeus as normal, and the two traffic generating PCs FTP'd the selected files from keppel. The test was carried out in a similar fashion to series two.

This test was run three times.

Series four

Series four was essentially the same as series two and three, except that there were four traffic-generating PCs behind the firewall.

This test was run three times.

The analysis

The empirical data is shown in tabular form in Appendix O, and in graphical form in Appendices P and Q.

Unfortunately, WS_FTP failed to record the final run in any of the series. Therefore, the experiment consisted of two runs in each series.

The analysis of the final data was simplified by making the following assumption:

The time taken to transfer a file is linearly dependent on the size of the file.

For each of the test series, the average transfer rate and connection overhead was determined. The transfer rates and overheads were then plotted as a function of the number of machines connected. The resulting graphs are shown in Appendices P and Q. The graphs indicate the transfer rate and connection overhead are not linearly dependent on the number of machines. The form of the graph suggests either a logarithmic or reciprocal relationship between transfer time and the number of machines.

The upper threshold on the number of computers behind the firewall depends on what is considered the lowest acceptable transfer rate. However, it would not be imprudent to suggest that with the given 386 firewall, 2 is the maximum tolerable number of machines that can tolerably operate behind the firewall under heavy load. Note that the firewall machine constitutes a "lower-end" 386.