Transmission of Medical Images over Wide Area Networks

L. Rodney Long

Lister Hill National Center for Biomedical Communications

National Library of Medicine

Abstract: Rapid transmission of medical image data across wide area networks remains a technical challenge, due to the high data volume of the images and the transmission rates typically achievable. We are investigating two methods for potentially achieving better transmission rates when using the TCP protocol: (1) multisocket transmission, an application-level alternative to conventional single- channel TCP transmission; and (2) implementations of Internet RFC 1323, which specifies TCP extensions for high performance. This paper discusses transmission rate limitations in current TCP and presents results to date in using these alternative methods to send large image files across the Internet and across dedicated high-speed links with long signal path delays.

Keywords: medical images, wide area networks, transmission rate, TCP/IP, multisocket, RFC 1323, bandwidth, delay

1.0 Introduction

Wide area network delivery of large amounts of data is a topic of increasing interest, particularly for the transmission of biomedical images, such as the Visible Human digital color slices, because of the large sizes of the image files and the need to disseminate them on a national or international scale. Network protocols have a direct impact on the transmission rate achievable. At NLM's Communications Engineering Branch (CEB), we are focusing on the transport protocol and its relationship to link performance. We are investigating performance on the critical communications links of today (Internet links) and also on emerging high-speed optical fiber and satellite links. While much attention has been given to the emplacement of links with high physical capacity in building the National Information Infrastructure, the role of the transport protocol is also critical, since this protocol incorporates data flow control. The most widely-used transport protocol for Internet communication is TCP.

2.0 TCP characteristics affecting transmission rate performance

TCP windows and data flow. To control data flow, TCP uses windows on the sequence number spaces corresponding to the byte streams being sent and received. The TCP send window is illustrated in Figure 1. The send window slides to the right in the stream of bytes to be transmitted (send sequence space) as acknowledgments are received for sequence numbers within the window, allowing new packets to be sent to the network. Note that TCP sends data in units of packets, but acknowledges data by using the sequence numbers of bytes in the data stream being sent. For example, in Figure 1, the receipt of the packet containing bytes 512-1023 would be acknowledged by transmitting a "1024" (next expected byte) back to the sender. On the receive end, the receive window incorporates a similar sliding window scheme. The receive window size is a measure of the amount of buffer area that the receiver has currently available to receive data. At this end of the communication, there is a conceptual byte stream corresponding to all of the bytes to be received. The receive window slides to the right in this conceptual stream (receive window space) as data is received into and emptied from its available buffer area.

The TCP receiver transmits its current window size to the sender in the TCP header. The sender uses this receive window size as one of its key parameters in controlling data flow.

The sending TCP maintains a limit called the congestion window which, absent any packet loss, is the same size as the receive window. TCP maintains a dynamically-computed estimate of the round-trip time (RTT) for data transfer; when the RTT decreases, it assumes less congestion on the network, and will increase the congestion window size; similarly, when the RTT increases, TCP decreases the congestion window size.

At any time, the send window behaves as if its size S is given by [1]:

S = minimum(receive window, congestion window).

The larger the send window, the more data can be sent to the network, hence the greater throughput potential.

We are analyzing TCP performance in two network environments:

Case 1: Bandwidth-delay limited environment. If the network environment is contention-free (such as a dedicated point-to-point satellite link), the congestion window will be sized the same as the receive window. Hence, the limiting factor for S becomes only the receive window size. In order to maintain full use of a TCP connection at a specified bandwidth, the following relationship must hold:

receive window >= bandwidth * delay

where delay is the round-trip time mentioned above. If a network's throughput is determined by the above relationship, we refer to the network as bandwidth-delay limited. For example, in order to achieve a throughput of 50,000 bytes/second over a link with a 200 millisecond delay, the receive window must be at least 10,000 bytes in size. In current TCP, the receive window size is limited to 2**16-1. This limitation, commonly called the "64K window limit", results from the fact that the receive window size is passed to the sender in the TCP header, where the data field allocated to hold the window size is only two bytes in length. This has serious implications for network transmissions where the transmission distance, and hence the delay is great , and the desired throughput is high. These so-called "long, fat networks" (LFNs) are directly impacted by the receive window limitation in current TCP [2]. The achievable bandwidth in current TCP may be calculated by

bandwidth <= (2**16-1)/ delay.

For example a transmission with a network delay of 50 milliseconds will have a throughput limited by about 1.3 million bytes/second, regardless of the underlying physical capacity of the link. Institutions such as NLM desire to transmit many megabytes of data rapidly across long distances; these are precisely the class of users suited to LFNs, and also those for which the TCP window limitation becomes a potential problem. Figure 2 illustrates the bandwidth-delay limits under current TCP for a range of network delays, by graphing the relationship

bandwidth-delay limit = ( 2**16-1) / ( network delay) .

The significance of this relationship for emerging T-3 networks is illustrated in Figure 3, where published network delays [3] were used to graph the bandwidth-delay limits for transmissions between College Park, Maryland and various sites within the United States over ANSNET. As the figure shows, the limitations are severe for all except the Boston connection.

Case 2: Internet environment. Bandwidth-delay limitations are not yet a major factor on Internet communications within the continental United States. The reason: most Internet users are connected to the Internet with at most a T-1 rate connection. Current TCP can tolerate a delay of (Max window size)/(T-1 rate) = 64 Kbytes/192,000 bytes/sec = about 340 ms without bandwidth- delay affecting the achievable throughput. In other words, unless the Internet delay is at least 340 ms, the transmission will not be bandwidth-delay limited under current TCP. On the Internet, delays within the continental United States typically do not approach the 340 ms figure. However, Internet transmission rates, while not typically bandwidth-delay limited, frequently fall very short of the physical link capacity. Figure 4 shows the results of a three-week Internet data collection for which 5 megabyte digitized x-ray images were transmitted from the University of Arizona (Tucson) to NLM, using the standard File Transfer Protocol (FTP) tool, which uses one TCP connection for data transfer. The files were sent at 15-minute intervals. The graph shows the mean transmission time observed at each hour of the day. The horizontal line shows the transmission time expected based on the physical throughput capacity (192,000 bytes/sec) of the link. If the transmission achieved full link capacity, the file-to-file transmission time should have been 27 seconds for link transmission, plus approximately 6 seconds for file I/O time, giving a total of 33 seconds, as compared to FTP overall mean file-to-file transmission time of 150 seconds.

Accounting for the lost performance, both in the dedicated link case and in the Internet case, and developing technical methods to regain it is the crux of our research. Obviously, we may not expect to regain performance losses due to real congestion on the Internet; but, surprisingly, as we shall explain, it appears that much of the degraded performance in our Internet tests is not due to actual congestion, and is in fact recoverable using transmission alternatives to FTP.

3.0 Methods and results

Bandwidth-delay limited environment. The bandwidth-delay limit has been known for years in the Internet research community and was addressed in a set of proposals which were incorporated into RFC 1323, "TCP Extensions for High Performance" [2]. This RFC includes specifications for allowing TCP to operate with a receive window up to size 230. The RFC 1323 specifications have been incorporated into an experimental TCP which was tested on a DS-3 (44.7 Mbps) wide area network with delay comparable to a transmission across the continental United States; researchers reported achieving 89% of link capacity, using a window size of 512 kilobytes [4]. In addition, Skibo [5] reported achieving a 31 Mbit/sec rate on a DS-3 25 ms link between Urbana, Illinois and Murray Hill, New Jersey; the bandwidth-delay limit for this link using conventional TCP is 21 Mbit/sec. The RFC 1323 specifications have also been incorporated into commercially-available TCP software, for example, a Sun Microsystems product called TCPLFN [6]. Using this software, one researcher has reported obtaining 90% of theoretical link throughput using a 100 kilobyte receive window when transmitting 20 megabyte digital mammography images over a T-1 geosynchronous satellite link [7]. TCP modifications for performance other than those specified in RFC 1323 have been proposed and experimentally tested, one of the most notable being selective acknowledgment [8] of packets as opposed to the cumulative acknowledgment scheme incorporated into current TCP. We have focused on RFC 1323 because it addresses the receive window problem and because commercial implementations are readily available.

At NLM we are testing the performance of this software for transmitting Visible Human and other large medical images. We have also developed an alternative, application-level method called multisocket transmission which has proven effective in tests run over a bandwidth-delay limited link. (See [9] for a description of the multisocket algorithm as implemented by NLM.) We have conducted experiments on a T-1 satellite link using this method. Visible Human digital color images (7 megabyte file size) have been transmitted between NLM and the Laboratory for Radiological Informatics at the University of California at San Francisco. A 100-transmission run using Visible Human images yielded multisocket performance several times faster than FTP. The results of this run, which used 12 TCP connections is shown is Figure 5. In addition, multisocket transmissions across the link have been measured to perform faster than the (single TCP channel) limit predicted by the bandwidth-delay product. In this particular test, illustrated in Figure 6, a 5 megabyte file was transmitted repeatedly using an increasing number of TCP connections; as shown in the figure, when the number of connections exceeded 9, the multisocket rate exceeded the limit predicted by the bandwidth-delay product. (The bandwidth-delay limit is a limiting factor for a single TCP connection, not for the aggregate rate achievable by using multiple connections.) For all data collected to date on the geosynchronous link, the NLM tests indicate achieving approximately 80% of the T-1 link capacity.

A multisocket approach has been independently implemented by researchers at Ohio University, who have modified SunOS FTP source code to build multisocket capability directly into FTP. They have reported achieving 91% of T-1 link capacity over a geosynchronous satellite link [10].

Internet environment. Transmission limiting factors in the complex Internet environment are less well understood, but tests conducted at NLM have demonstrated the existence of available, unutilized bandwidth when TCP transmissions were made on three different Internet links. Multisocket transmission achieved significantly better performance by factors of 1.6 to 3.5 on these links, as compared to conventional TCP transmission.

Our empirical data collections on the Internet show that (1) throughput falls well below the barrier imposed by the TCP window limit; and (2) there is available bandwidth not being used; this bandwidth may be exploited by methods such as multisocket transmission. The question of whether this degraded throughput is correlated to TCP congestion control algorithms remains unanswered: this would require observing TCP dynamically-changing internal parameter values, such as the setting of the congestion window.

What we have ascertained is that the our multisocket transmission method outperforms conventional TCP transfer methods, in particular FTP, on several Internet links, in three series of tests. This data was taken on transmissions between NLM and the University of Arizona (Tucson), Texas Tech University (Lubbock), and the NASA Lewis Research Center (Cleveland). Data collections of durations ranging from 24 hours to 29 days were made. For Test Series 1-2 a 5 megabyte digitized cervical x-ray was transmitted; for Test Series 3, a 7 megabyte Visible Human digital color slice image was transmitted. For each collection, the image file was repeatedly transmitted from the remote site to NLM at small (10-15 minute) time intervals, using the multisocket method. Also, at the same time intervals, but offset from the multisocket transmission times, the same image file was transmitted using FTP or a similar 1-socket-pair transfer. (Conventional TCP transmission uses one pair of communication end-points--"sockets"--to establish one TCP channel.) Transmission times were logged, and statistics of the results were computed, including mean, median, and standard deviation. In every case, the average transmission time using the multisocket method was significantly shorter than for FTP. The speed-up ratios of multisocket transmission as compared to FTP for the first test series ranged from 1.7 to 3.5, depending on the particular link and data collection interval [9]. For the second test series, the speed-ups ranged from 1.6 to 2.7 [11]; and in the single 1996 Internet data collection, the speed-up ratio was 2.1. For Test Series 1, which included the longest-duration collection, confidence intervals for the mean transmission time values were computed, and the multisocket intervals showed clear separation from the FTP intervals at a 95% level of confidence. (For example, the first data collection from Tucson had a multisocket confidence interval of 6 seconds, centered on a mean of 107; for comparison, the single-socket confidence interval was 12 seconds, centered on a mean of 280.) Figure 7 shows the performance of the multisocket method for the same test setup as Figure 4; we concluded that in this case the degraded FTP througput was not due to actual network congestion, since the multisocket method achieved significantly greater througput over the same link and test period.

4.0 Conclusion

Wide area network transmission of medical image data under TCP protocol continues to be an area where conventional transfer methods, such as FTP using current 64K-receive window TCP, are frequently inefficient. High performance TCP as defined by RFC 1323 has been reported in the technical literature to be an effective method for getting more efficient transmission in the dedicated link, bandwidth-delay limited environment. NLM is researching the use of this method not only for possible use on dedicated links, but for use on the Internet, where its performance is less well understood. In addition, NLM is comparing the performance of an inhouse-developed method, multisocket transmission, to the peformance of the RFC 1323 implementation. Multisocket transmission has been shown to have improved transmission rate performance, as compared to FTP, by factors ranging from 1.7 to 3.5 in a series of NLM tests on the Internet. In addition, multisocket transmission has been shown to surpass the single TCP connection bandwidth-delay limit in one test conducted on a dedicated link, geosynchrous satellite environment.

References

1. Comer DE. Internetworking with TCP/IP, Volume I, Principles, Protocols, and Architecture. Prentice-Hall, Englewood Cliffs, New Jersey, 1991.

2. Jacobson V, Braden R, Borman D. TCP extensions for high performance. Internet RFC 1323, May 1992.

3. ANSNET T3 delay matrix report for April 1995. Available from the Merit Network, Inc., Ann Arbor, Michigan as ftp://ftp.merit.net/statistics/nsfnet/1995/nsf-9504.delay.gz.

4. Villamizar C, Song C. High performance TCP in ANSNET. Computer Communications Review, October 1994; vol. 24, no. 5; 45-60.

5. Skibo T. Experiences with TCP extensions for high-performance networks. Department of Computer Science, University of Illinois at Urbana-Champaign. May 14, 1992.

6. Sun Microsystems. CONSULT-TCPLFN data sheet. Sun Consulting, Mountain View, California, February 1996.

7. Barnett BG, Dudding KE, Abdel-Malek A, Mitchell RJ. Satellite teleradiology testbed for digital mammography. Proc. SPIE Medical Imaging '96, Newport Beach, California, February 10- 15, 1996; vol. 2217; 308-317.

8. Jacobson V, Braden R. TCP extensions for long-delay paths. Internet RFC 1072, October 1988.

9. Long LR, Berman LE, Neve L, Roy G, Thoma GR. An application-level technique for faster transmission of large images on the Internet. Proc. SPIE Multimedia Computing and Networking 1995, San Jose, CA, February 6-8, 1995; vol. 2417; 116-129.

10. Allman M, Ostermann S, Kruse H. Data transfer efficiency over satellite circuits using a multi- socket extension to the File Transfer Protocol (FTP). Proceeedings of ACTS Results Conference; NASA Lewis Research Center, Cleveland, OH, September 11-13, 1995; Network section.

11. Long LR, Berman LE, Thoma GR. Client/Server design for fast retrieval of large images on the Internet. Proc. 8th IEEE Symposium of Computer-Based Medical Systems (CBMS'95), Lubbock TX, June 9-10, 1995; 284-291.


Last Modified: 08:11am EDT, October 10, 1996