e-VLBI and End-to-End PerformanceMasaki Hirabaru Yasuhiro Koyama Tetsuro Kondo

NICT Koganei NICT Kashima NICT Kashima

masaki@nict.go.jp koyama@nict.go.jp kondo@nict.go.jp

VLBI (Very Long Baseline Interferometry)

dela

y

radio signal from a star

correlator

A/Dclock

A/D

　Internet

clock

ASTRONOMYGEODESY

~Gbps~Gbps

Fringe

•e-VLBI geographically distributed observation, interconnecting radio antennas over the world

Large Bandwidth-Delay Product Network issue

•Gigabit / real-time VLBI multi-gigabit rate sampling

Example: From Tokyo to BostonTCP on a fast long path with a bottleneck

bottleneck

overflowqueue

loss

Tokyosenderrate control

Bostonreceiverloss detection

feedback

It takes 150ms to know the loss (buffer overflow).It keeps overflowing during the period…150ms is very long for the high-speed network.150ms at 1Gbps generates ~19MByte on the wire.

Los Angeles

50ms 50ms

100ms

bw 1G

bw 0.8G

Analyzing Advanced TCP Dynamic Behavior in a Real Network(Example: From Tokyo to Indianapolis at 1G bps with HighSpeed TCP)

The data was obtained during e-VLBI demonstration at Internet2 Member Meeting 　 in October 2003.

ReceiverLinux TCP

SenderLinux 2.4

GbE

RTT 200ms(100ms one-way)

GbE

Only 800 Mbps available

Replaying in a Laboratory-Evaluation of Advanced TCPs-

TCP 　 NewReno 　 (Linux)

HighSpeed 　 TCP (Web100)

Example of Advanced TCPs 　 with different bottleneck queue sizes

BIC TCP queue size 100 packets FAST TCP queue size 100 packets

BIC TCP queue size 1000 packetsFAST TCP queue size 1000 packets

RouterSwitch RouterSwitch

1Gbps

100Mbps1Gbps

1Gbps

a) b)

DeviceQueuing

Delay (µs)Capacity (Mbps)

Estimated Queue Size

(1500B)

Switch A 6161 100* 50

Switch B 22168 100* 180

Switch C 20847 100* 169

Switch D 738 1000 60

Switch E 3662 1000 298

Router F 148463 1000 12081

Router G 188627 1000 15350

* set to 100M for measurement

Measuring Bottleneck Queue Sizes

Switch/Router Queue Size Measurement Result

Typical Bottleneck Cases

ReceiverSenderCapacity C

packet trainlost packet

measured packet

Queue Size = C x (Delaymax – Delaymin)

　

KwangjuBusan

2.5G

Fukuoka

Korea

2.5G SONET

KORENTaegu

Daejon

10G

1G (10G)1G

1G

Seoul XP

Genkai XP

Kitakyushu

Kashima

1G (10G)

Fukuoka Japan

250km

1,000km2.5G

JGN II

9,000km

4,000km

Los Angeles

Chicago

Washington DC

MIT Haystack

10G

2.4G

APII/JGNII

Abilene

Koganei

1G(10G)

Indianapolis

100kmbwctl server

Experiment for High-Performance Scientific Data Transfer

10G

Tokyo XP /JGN II I-NOC

*Performance Measurement Point Directoryhttp://e2epi.internet2.edu/pipes/pmp/pmp-dir.html

perf server

e-vlbi server

JGNII

10G

GEANT

SWITCH

7,000km

TransPAC

Pittsburgh

U of Tokyo

A BWCTL account available for CMMincluding Korean researchers

International collaboration to support science applications

VLBI Antenna Locations in North-East Asia

Shintotsukawa 3.8m

Tomakomai 11m, FTTH (100M)70km from Sapporo

Mizusawa 10m 20m118km from Sendai

Tsukuba 32m, OC48/ATMx2 SuperSINET

Kashima 34m, 1Gx2 JGN, OC48/ATM Galaxy

Yamaguchi 32m1G, 75M SINET

Gifu 11m 3m, OC48/ATMx2 SuperSINET

Usuda 64m, OC48/ATM Galaxy

Nanshan (Urumqi) 25m70km from Urumqi

Koganei 34m, 1Gx2 JGN, OC48/ATM Galaxy

Miyun (Beijing) 50m50km from Beijing

2Mbps

2Mbps

Sheshan (Shanghai) 25m30km from Shanghai

Observatory is on CSTNET at 100M Jeju 20m

Tamna U

Seoul 20mYonsei U

Ulsan 20mU Ulsan

Daejon 14mTaeduk

Ishigaki 20m

Ogasawara 20mChichijima 10m

Iriki 20mKagoshima 6mAira 10m

Legendconnectednot yet connectedantenna under construction

Summary and Future Work

•High-performance scientific data transfer faces on network issues we need to work out.•Big science applications like e-VLBI and High-Energy Physics need cooperation with network researchers.•Deployment of performance measurement Infrastructure is on-going on world-wide basis.•Lessens are shared with CHEP and KAIST experiments over APII and TransPAC as well as CMM (Common Measurement Machines).

ENP2611Network

Processor Emulator

APII Workshop 2004 October 28-29, 2004

Appendix - VLBI System Transitions

K5 Data AcquisitionTerminal

1st Generation

2nd Generation

1983~Open-Reel TapeHardware Correlator

1990~Cassette TapeHardware Correlatore-VLBI over ATM

3rd Generation

2002~PC-based SystemHard-disk StorageSoftware Correlatore-VLBI over Internet

K3 Correlator (Center)K3 Recorder (Right)

K4 Terminal

K4 Correlator

64Mbps

256Mbps

1 ～ 2Gbps

Nobeyama 45mOC48/ATM Galaxy