This wiki is obsolete, see the NorduGrid web pages for up to date information.

NOX/Tests/HopiResults

From NorduGrid
< NOX‎ | Tests
Jump to navigationJump to search

A: without TLS, without Plexer

The config: 

profile=/usr/local/share/arc/profiles/HopiServiceNonSecure.xml

pidfile=/tmp/arched.pid
logfile=/tmp/arched.log
debug=ERROR

port=60000

[hopi]
documentroot=/tmp/hopi

Run the service: 

$ sudo /usr/local/sbin/arched -f -i HopiServiceNonSecure.ini

A1: empty document root

The directory is empty: 

$ ls /tmp/hopi/
$

Typed http://localhost:60000 into the browser, which displays this: 

Directory list of '/'

With curl: 

$ curl http://localhost:60000
<HTML>
<HEAD>Directory list of '/'</HEAD>
<BODY><UL>
</UL></BODY></HTML>

With arccp: 

$ arccp http://localhost:60000 index.html; cat index.html
<HTML>
<HEAD>Directory list of ''</HEAD>
<BODY><UL>
</UL></BODY></HTML>

A2: non-empty document root

I've created a directory and some files: 

$ ls -lR /tmp/hopi
total 2048
drwxr-xr-x  4 zsombor  wheel      136 Okt 23 14:15 dir
-rw-r--r--  1 zsombor  wheel  1048576 Okt 23 14:15 file

/tmp/hopi/dir:
total 224
-rw-r--r--  1 zsombor  wheel   10240 Okt 23 14:15 file1
-rw-r--r--  1 zsombor  wheel  102400 Okt 23 14:15 file2

Browser with http://localhost:60000 

Directory list of '/'
 * dir
 * file

Browser with http://localhost:60000/dir 

Directory list of '/dir'
 * file1
 * file2

curl: 

$ curl http://localhost:60000
<HTML>
<HEAD>Directory list of '/'</HEAD>
<BODY><UL>
<LI><a href="/dir">dir</a></LI>
<LI><a href="/file">file</a></LI>
</UL></BODY></HTML>
$ curl http://localhost:60000/dir
<HTML>
<HEAD>Directory list of '/dir'</HEAD>
<BODY><UL>
<LI><a href="/dir/file1">file1</a></LI>
<LI><a href="/dir/file2">file2</a></LI>
</UL></BODY></HTML>

arccp: 

$ arccp http://localhost:60000 index.html; cat index.html
<HTML>
<HEAD>Directory list of ''</HEAD>
<BODY><UL>
<LI><a href="/dir">dir</a></LI>
<LI><a href="/file">file</a></LI>
</UL></BODY></HTML>
$ arccp http://localhost:60000/dir index.html; cat index.html
<HTML>
<HEAD>Directory list of '/dir'</HEAD>
<BODY><UL>
<LI><a href="/dir/file1">file1</a></LI>
<LI><a href="/dir/file2">file2</a></LI>
</UL></BODY></HTML>

A3: download files

I've created several files: 

$ ls -lhrS /tmp/hopi/
-rw-r--r--  1 zsombor  wheel   1,0K Okt 23 14:23 1kB
-rw-r--r--  1 zsombor  wheel    10K Okt 23 14:22 10kB
-rw-r--r--  1 zsombor  wheel   100K Okt 23 14:22 100kB
-rw-r--r--  1 zsombor  wheel   1,0M Okt 23 14:23 1MB
-rw-r--r--  1 zsombor  wheel    10M Okt 23 14:22 10MB
-rw-r--r--  1 zsombor  wheel   100M Okt 23 14:22 100MB
-rw-r--r--  1 zsombor  wheel   1,0G Okt 23 14:23 1GB

curl: 

$ time curl -s http://localhost:60000/1kB > file
real	0m0.014s
$ diff file /tmp/hopi/1kB
$ time curl -s http://localhost:60000/10kB > file
real	0m0.026s
$ diff file /tmp/hopi/10kB
$ time curl -s http://localhost:60000/100kB > file
real	0m0.032s
$ diff file /tmp/hopi/100kB
$ time curl -s http://localhost:60000/1MB > file
real	0m0.024s
$ diff file /tmp/hopi/1MB
$ time curl -s http://localhost:60000/10MB > file
real	0m0.744s
$ diff file /tmp/hopi/10MB
$ time curl -s http://localhost:60000/100MB > file
real	0m7.347s
$ diff file /tmp/hopi/100MB
$ time curl -s http://localhost:60000/1GB > file
real	1m24.330s
$ diff file /tmp/hopi/1GB
$

arccp: 

$ time arccp http://localhost:60000/1kB file
real	0m0.398s
$ diff file /tmp/hopi/1kB
$ time arccp http://localhost:60000/10kB file
real	0m0.398s
$ diff file /tmp/hopi/10kB
$ time arccp http://localhost:60000/100kB file
real	0m0.396s
$ diff file /tmp/hopi/100kB
$ time arccp http://localhost:60000/1MB file
real	0m0.408s
$ diff file /tmp/hopi/1MB
$ time arccp http://localhost:60000/10MB file
real	0m0.754s
$ diff file /tmp/hopi/10MB
$ time arccp http://localhost:60000/100MB file
real	0m6.935s
$ diff file /tmp/hopi/100MB
$ time arccp http://localhost:60000/1GB file
real	0m36.785s
$ diff file /tmp/hopi/1GB
$


A4: upload files

The documentroot is empty: 

$ ls /tmp/hopi/
$

Upload several files with curl, then download them, and check the diff: 

$ time curl -T files/1kB http://localhost:60000
real	0m2.445s
$ curl -s http://localhost:60000/1kB > file
$ diff file files/1kB
$ time curl -T files/10kB http://localhost:60000
real	0m2.372s
$ curl -s http://localhost:60000/10kB > file
$ diff file files/10kB
$ time curl -T files/100kB http://localhost:60000
real	0m2.036s
$ curl -s http://localhost:60000/100kB > file
$ diff file files/100kB
$ time curl -T files/1MB http://localhost:60000
real	0m2.060s
$ curl -s http://localhost:60000/1MB > file
$ diff file files/1MB
$ time curl -T files/10MB http://localhost:60000
real	0m2.844s
$ curl -s http://localhost:60000/10MB > file
$ diff file files/10MB
$ time curl -T files/100MB http://localhost:60000
real	0m7.454s
$ curl -s http://localhost:60000/100MB > file
$ diff file files/100MB
$ time curl -T files/1GB http://localhost:60000
real	0m55.171s
$ curl -s http://localhost:60000/1GB > file
$ diff file files/1GB
$

Check the document root: 

$ ls -lrS /tmp/hopi/
total 2324712
-rw-------  1 root  wheel        1024 Okt 24 10:40 1kB
-rw-------  1 root  wheel       10240 Okt 24 10:41 10kB
-rw-------  1 root  wheel      102400 Okt 24 10:41 100kB
-rw-------  1 root  wheel     1048576 Okt 24 10:41 1MB
-rw-------  1 root  wheel    10485760 Okt 24 10:42 10MB
-rw-------  1 root  wheel   104857600 Okt 24 10:42 100MB
-rw-------  1 root  wheel  1073741824 Okt 24 10:43 1GB

Empty the documentroot, and do the same with arccp: 

$ time arccp files/1kB http://localhost:60000/
real	0m1.358s
$ arccp http://localhost:60000/1kB file
$ diff file files/1kB
$ time arccp files/10kB http://localhost:60000/
real	0m1.357s
$ arccp http://localhost:60000/10kB file
$ diff file files/10kB
$ time arccp files/100kB http://localhost:60000/
real	0m1.360s
$ arccp http://localhost:60000/100kB file
$ diff file files/100kB
$ time arccp files/1MB http://localhost:60000/
real	0m1.361s
$ arccp http://localhost:60000/1MB file
$ diff file files/1MB
$ time arccp files/10MB http://localhost:60000/
real	0m1.361s
$ arccp http://localhost:60000/10MB file
$ diff file files/10MB
$ time arccp files/100MB http://localhost:60000/
real	0m7.358s
$ arccp http://localhost:60000/100MB file
$ diff file files/100MB
$ time arccp files/1GB http://localhost:60000/
real	0m37.430s
$ arccp http://localhost:60000/1GB file
$ diff file files/1GB

Check the document root: 

$ ls -lrS /tmp/hopi/
total 2324712
-rw-------  1 root  wheel        1024 Okt 24 10:49 1kB
-rw-------  1 root  wheel       10240 Okt 24 10:49 10kB
-rw-------  1 root  wheel      102400 Okt 24 10:49 100kB
-rw-------  1 root  wheel     1048576 Okt 24 10:50 1MB
-rw-------  1 root  wheel    10485760 Okt 24 10:50 10MB
-rw-------  1 root  wheel   104857600 Okt 24 10:50 100MB
-rw-------  1 root  wheel  1073741824 Okt 24 10:51 1GB

A5: simultaneous downloads

Get the 100MB file with 10 simultaneous clients.

curl: 

$ for i in `gseq 0 9`; do (time curl -s http://localhost:60000/100MB > file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do diff file$i files/100MB; done
$ cat /tmp/results? | grep real
real	0m28.091s
real	0m28.436s
real	0m29.122s
real	0m25.048s
real	0m27.418s
real	0m26.683s
real	0m27.779s
real	0m28.679s
real	0m28.889s
real	0m28.596s

arccp: 

$ for i in `gseq 0 9`; do (time arccp http://localhost:60000/100MB file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do diff file$i files/100MB; done
$ cat /tmp/results? | grep real
real	0m31.390s
real	0m31.910s
real	0m31.268s
real	0m31.880s
real	0m31.202s
real	0m29.603s
real	0m31.315s
real	0m30.265s
real	0m31.154s
real	0m30.877s

A6: simultaneous uploads

Upload 10 100MB file simultaneously.

curl: 

$ for i in `gseq 0 9`; do (time curl -T files/100MB http://localhost:60000/file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do diff files/100MB /tmp/hopi/file$i; done
$ cat /tmp/results? | grep realreal	0m37.094s
real	0m36.540s
real	0m37.075s
real	0m33.138s
real	0m33.797s
real	0m33.910s
real	0m35.081s
real	0m36.457s
real	0m36.524s
real	0m36.183s

arccp: 

$ for i in `gseq 0 9`; do (time arccp files/100MB http://localhost:60000/file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do diff files/100MB /tmp/hopi/file$i; done
$ cat /tmp/results? | grep realreal	0m47.222s
real	0m47.190s
real	0m47.586s
real	0m47.509s
real	0m46.946s
real	0m47.251s
real	0m47.579s
real	0m47.256s
real	0m46.288s
real	0m47.253s

B: without TLS, with Plexer

Config: 

# cat HopiServiceWithPlexerNonSecure.ini
profile=/usr/local/share/arc/profiles/HopiServiceWithPlexerNonSecure.xml

pidfile=/tmp/arched.pid
logfile=/tmp/arched.log
debug=ERROR

port=60000

[hopi]
urlpatter=^/hopi/
documentroot=/tmp/hopi

B1: empty document root

The directory is empty: 

$ ls -l /tmp/hopi/
$

Typed http://localhost:60000/hopi/ into the browser, which shows: 

Directory list of ''

curl: 

$ curl http://localhost:60000/hopi/
<HTML>
<HEAD>Directory list of ''</HEAD>
<BODY><UL>
</UL></BODY></HTML>

arccp: 

$ arccp http://localhost:60000/hopi/ index.html; cat index.html
[2009-10-24 15:48:43] [Arc.arccp] [ERROR] [2886/12619088] Fileset copy to single object is not supported yet

arccp cannot download this way, because it thinks it is a directory

B2: non-empty document root

I've created a directory and some files: 

$ ls -lR /tmp/hopi
total 8
drwxr-xr-x  4 zsombor  wheel  136 Okt 24 15:51 dir
-rw-r--r--  1 zsombor  wheel    5 Okt 24 15:51 file

/tmp/hopi/dir:
total 16
-rw-r--r--  1 zsombor  wheel  6 Okt 24 15:51 file1
-rw-r--r--  1 zsombor  wheel  6 Okt 24 15:51 file2

Browser with http://localhost:60000/hopi/ : 

Directory list of ''
 * dir
 * file

And with http://localhost:60000/hopi/dir : 

Directory list of 'dir'
 * file1
 * file2

curl: 

$ curl http://localhost:60000/hopi/
<HTML>
<HEAD>Directory list of ''</HEAD>
<BODY><UL>
<LI><a href="/hopi//dir">dir</a></LI>
<LI><a href="/hopi//file">file</a></LI>
</UL></BODY></HTML>
$ curl http://localhost:60000/hopi/dir
<HTML>
<HEAD>Directory list of 'dir'</HEAD>
<BODY><UL>
<LI><a href="/hopi/dir/file1">file1</a></LI>
<LI><a href="/hopi/dir/file2">file2</a></LI>
</UL></BODY></HTML>

we can see a small problem with double slashes, but that does not affect the functionality 

$ arccp http://localhost:60000/hopi/ index.html; cat index.html
[2009-10-24 15:56:20] [Arc.arccp] [ERROR] [2973/12619088] Fileset copy to single object is not supported yet
$ arccp http://localhost:60000/hopi/dir index.html; cat index.html
<HTML>
<HEAD>Directory list of 'dir'</HEAD>
<BODY><UL>
<LI><a href="http://localhost:60000/hopi/dir/file1">file1</a></LI>
<LI><a href="http://localhost:60000/hopi/dir/file2">file2</a></LI>
</UL></BODY></HTML>

the same problem with arccp thinking it is a directory

B3: download files

I've created some files: 

$ ls -lrS /tmp/hopi
total 227328
-rw-r--r--  1 zsombor  wheel    1048576 Okt 24 15:59 1MB
-rw-r--r--  1 zsombor  wheel   10485760 Okt 24 15:59 10MB
-rw-r--r--  1 zsombor  wheel  104857600 Okt 24 15:59 100MB

curl: 

$ time curl -s http://localhost:60000/hopi/1MB > file
real	0m0.025s
$ diff file files/1MB
$ time curl -s http://localhost:60000/hopi/10MB > file
real	0m0.442s
$ diff file files/10MB
$ time curl -s http://localhost:60000/hopi/100MB > file
real	0m2.612s
$ diff file files/100MB
$

arccp: 

$ time arccp http://localhost:60000/hopi/1MB file
real	0m0.405s
$ diff file files/1MB
$ time arccp http://localhost:60000/hopi/10MB file
real	0m0.642s
$ diff file files/10MB
$ time arccp http://localhost:60000/hopi/100MB file
real	0m3.328s
$ diff file files/100MB
$

B4: upload files

The documentroot is empty: 

$ ls /tmp/hopi/
$

Upload some files with curl, then check the diff: 

$ time curl -T files/1MB http://localhost:60000/hopi/1MB
real	0m2.022s
$ time curl -T files/10MB http://localhost:60000/hopi/10MB
real	0m2.265s
$ time curl -T files/100MB http://localhost:60000/hopi/100MB
real	0m4.990s
$ ls -lrS /tmp/hopi
total 227328
-rw-------  1 root  wheel    1048576 Okt 24 16:07 1MB
-rw-------  1 root  wheel   10485760 Okt 24 16:07 10MB
-rw-------  1 root  wheel  104857600 Okt 24 16:07 100MB
$ sudo diff files/1MB /tmp/hopi/1MB
$ sudo diff files/10MB /tmp/hopi/10MB
$ sudo diff files/100MB /tmp/hopi/100MB
$

Empty the documentroot, and do the same with arccp: 

$ time arccp files/1MB http://localhost:60000/hopi/1MB
real	0m1.371s
$ time arccp files/10MB http://localhost:60000/hopi/10MB
real	0m1.371s
$ time arccp files/100MB http://localhost:60000/hopi/100MB
real	0m2.977s
$ ls -lrS /tmp/hopi
total 227328
-rw-------  1 root  wheel    1048576 Okt 24 16:09 1MB
-rw-------  1 root  wheel   10485760 Okt 24 16:09 10MB
-rw-------  1 root  wheel  104857600 Okt 24 16:09 100MB
$ sudo diff files/1MB /tmp/hopi/1MB
$ sudo diff files/10MB /tmp/hopi/10MB
$ sudo diff files/100MB /tmp/hopi/100MB
$


B5: simultaneous downloads

Get the 100MB file with 10 simultaneous clients.

curl: 

$ for i in `gseq 0 9`; do (time curl -s http://localhost:60000/hopi/100MB > file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff file$i /tmp/hopi/100MB; done
$ cat /tmp/results? | grep real
real	0m31.865s
real	0m31.084s
real	0m31.374s
real	0m31.741s
real	0m30.416s
real	0m28.664s
real	0m29.790s
real	0m29.569s
real	0m28.568s
real	0m29.395s

arccp: 

$ for i in `gseq 0 9`; do (time arccp http://localhost:60000/hopi/100MB file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff file$i /tmp/hopi/100MB; done
$ cat /tmp/results? | grep real
real	0m33.981s
real	0m35.683s
real	0m34.079s
real	0m34.608s
real	0m34.946s
real	0m35.215s
real	0m34.199s
real	0m34.646s
real	0m32.922s
real	0m30.129s

B6: simultaneous uploads

Upload 10 100MB file simultaneously.

curl: 

$ for i in `gseq 0 9`; do (time curl -T files/100MB http://localhost:60000/hopi/file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff files/100MB /tmp/hopi/file$i; done
$ cat /tmp/results? | grep real
real	0m39.773s
real	0m37.976s
real	0m41.153s
real	0m40.497s
real	0m40.067s
real	0m38.293s
real	0m41.354s
real	0m37.090s
real	0m40.029s
real	0m40.251s

arccp: 

$ for i in `gseq 0 9`; do (time arccp files/100MB http://localhost:60000/hopi/file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff files/100MB /tmp/hopi/file$i; done
$ cat /tmp/results? | grep real
real	0m59.694s
real	0m58.110s
real	0m59.057s
real	0m59.981s
real	0m59.208s
real	1m0.137s
real	0m59.528s
real	0m58.722s
real	0m59.670s
real	0m59.992s

C: with TLS, without Plexer

The config: 

profile=/usr/local/share/arc/profiles/HopiService.xml

pidfile=/tmp/arched.pid
logfile=/tmp/arched.log
debug=ERROR

port=60000

cacert=/etc/grid-security/certificates
host_cert=/etc/grid-security/hostcert.pem
host_key=/etc/grid-security/hostkey.pem

[hopi]
documentroot=/tmp/hopi

Run the service: 

$ sudo /usr/local/sbin/arched -f -i HopiService.ini

Set a convenient alias for curl: 

$ alias curls='curl --insecure --key ~/.arc/userkey.pem --cert ~/.arc/usercert.pem'

Set the user config for arccp: 

$ cat ~/.arc/client.conf 
keypath=/Users/zsombor/.arc/userkey.pem
certificatepath=/Users/zsombor/.arc/usercert.pem
cacertificatesdirectory=/etc/grid-security/certificates

C1: empty document root

The directory is empty: 

$ ls -l /tmp/hopi/
$

curl: 

$ curls https://localhost:60000
<HTML>
<HEAD>Directory list of '/'</HEAD>
<BODY><UL>
</UL></BODY></HTML>

arccp: 

$ arccp https://localhost:60000 index.html; cat index.html
<HTML>
<HEAD>Directory list of ''</HEAD>
<BODY><UL>
</UL></BODY></HTML>

C2: non-empty document root

I've created some files: 

$ ls -lR /tmp/hopi
total 8
drwxr-xr-x  4 zsombor  wheel  136 Okt 24 18:01 dir
-rw-r--r--  1 zsombor  wheel    5 Okt 24 18:00 file

/tmp/hopi/dir:
total 16
-rw-r--r--  1 zsombor  wheel  6 Okt 24 18:00 file1
-rw-r--r--  1 zsombor  wheel  6 Okt 24 18:01 file2

curl: 

$ curls https://localhost:60000
<HTML>
<HEAD>Directory list of '/'</HEAD>
<BODY><UL>
<LI><a href="/dir">dir</a></LI>
<LI><a href="/file">file</a></LI>
</UL></BODY></HTML>
$ curls https://localhost:60000/dir
<HTML>
<HEAD>Directory list of '/dir'</HEAD>
<BODY><UL>
<LI><a href="/dir/file1">file1</a></LI>
<LI><a href="/dir/file2">file2</a></LI>

arccp: 

$ arccp https://localhost:60000 index.html; cat index.html<HTML>
<HEAD>Directory list of ''</HEAD>
<BODY><UL>
<LI><a href="/dir">dir</a></LI>
<LI><a href="/file">file</a></LI>
</UL></BODY></HTML>
$ arccp https://localhost:60000/dir index.html; cat index.htm
<HTML>
<HEAD>Directory list of '/dir'</HEAD>
<BODY><UL>
<LI><a href="/dir/file1">file1</a></LI>
<LI><a href="/dir/file2">file2</a></LI>
</UL></BODY></HTML>

C3: download files

I've created some files: 

$ ls -lrS /tmp/hopi
total 2324480
-rw-r--r--  1 zsombor  wheel     1048576 Okt 24 18:07 1MB
-rw-r--r--  1 zsombor  wheel    10485760 Okt 24 18:07 10MB
-rw-r--r--  1 zsombor  wheel   104857600 Okt 24 18:07 100MB
-rw-r--r--  1 zsombor  wheel  1073741824 Okt 24 18:10 1GB

curl: 

$ time curls -s https://localhost:60000/1MB > file
real	0m0.112s
$ diff file /tmp/hopi/1MB
$ time curls -s https://localhost:60000/10MB > file
real	0m0.874s
$ diff file /tmp/hopi/10MB
$ time curls -s https://localhost:60000/100MB > file
real	0m6.091s
$ diff file /tmp/hopi/100MB
$ time curls -s https://localhost:60000/1GB > file
real	1m52.824s
$ diff file /tmp/hopi/1GB

arccp: 

$ time arccp https://localhost:60000/1MB file
real	0m1.093s
$ diff file /tmp/hopi/1MB
$ time arccp https://localhost:60000/10MB file
real	0m1.936s
$ diff file /tmp/hopi/10MB
$ time arccp https://localhost:60000/100MB file
real	0m17.181s
$ diff file /tmp/hopi/100MB
$ time arccp https://localhost:60000/1GB file
real	4m23.108s
$ diff file /tmp/hopi/1GB
$

with the 1GB file, arccp was much slower then curl

C4: upload files

The documentroot is empty: 

$ ls /tmp/hopi/
$

Upload some files of different sizes, then check the diff:

curl: 

$ time curls -T files/1MB https://localhost:60000
real	0m2.284s
$ sudo diff files/1MB /tmp/hopi/1MB
$ time curls -T files/10MB https://localhost:60000
real	0m2.842s
$ sudo diff files/10MB /tmp/hopi/10MB
$ time curls -T files/100MB https://localhost:60000
real	0m10.955s
$ sudo diff files/100MB /tmp/hopi/100MB
$ time curls -T files/1GB https://localhost:60000
real	2m33.145s
$ sudo diff files/1GB /tmp/hopi/1GB
$


arccp: 

$ time arccp files/1MB https://localhost:60000/
real	0m1.403s
$ sudo diff files/1MB /tmp/hopi/1MB
$ time arccp files/10MB https://localhost:60000/
real	0m2.076s
$ sudo diff files/10MB /tmp/hopi/10MB
$ time arccp files/100MB https://localhost:60000/
real	0m17.225s
$ sudo diff files/100MB /tmp/hopi/100MB
$ time arccp files/1GB https://localhost:60000/
$ time arccp files/1GB https://localhost:60000/
real	4m14.577s
$ sudo diff files/1GB /tmp/hopi/1GB
$

C5: simultaneous downloads

Get the 100MB file with 10 simultaneous clients.

curl: 

$ for i in `gseq 0 9`; do (time curls -s https://localhost:60000/100MB > file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff file$i /tmp/hopi/100MB; done
$ cat /tmp/results? | grep real
real	1m24.003s
real	1m24.149s
real	1m23.944s
real	1m24.132s
real	1m23.416s
real	1m24.127s
real	1m24.463s
real	1m24.371s
real	1m24.300s
real	1m22.381s

It seems ok, however the server log got these messages:: 

[2009-10-24 18:56:16] [Arc.MCC.TLS] [ERROR] [3625/7510448] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:17] [Arc.MCC.TLS] [ERROR] [3625/7503216] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7494032] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7510272] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7531440] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7538496] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7485648] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7545552] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7552608] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 18:56:18] [Arc.MCC.TLS] [ERROR] [3625/7524384] SSL error: 6 - (empty):(empty):EVP lib

arccp: 

$ for i in `gseq 0 9`; do (time arccp https://localhost:60000/100MB file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff file$i /tmp/hopi/100MB; done
$ cat /tmp/results? | grep realreal	3m18.767s
real	3m19.682s
real	3m18.344s
real	3m19.530s
real	3m19.791s
real	3m20.108s
real	3m17.636s
real	3m19.541s
real	3m20.213s
real	3m18.656s

In the arccp case, there is no error log, but it is much slower than in the curl case.

C6: simultaneous uploads

Upload 10 100MB file simultaneously.

curl: 

$ for i in `gseq 0 9`; do (time curls -T files/100MB https://localhost:60000/file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff files/100MB /tmp/hopi/file$i; done
$ cat /tmp/results? | grep realreal	1m58.894s
real	1m56.350s
real	1m40.187s
real	1m59.195s
real	1m55.440s
real	1m57.593s
real	1m56.331s
real	1m58.478s
real	1m57.183s
real	1m57.608s

In the server log, some error messages appeared: 

[2009-10-24 19:06:02] [Arc.MCC.TLS] [ERROR] [3625/7494416] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:17] [Arc.MCC.TLS] [ERROR] [3625/7485648] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:18] [Arc.MCC.TLS] [ERROR] [3625/7619504] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:18] [Arc.MCC.TLS] [ERROR] [3625/7580464] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:19] [Arc.MCC.TLS] [ERROR] [3625/7501664] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:19] [Arc.MCC.TLS] [ERROR] [3625/7568448] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:20] [Arc.MCC.TLS] [ERROR] [3625/7604528] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:20] [Arc.MCC.TLS] [ERROR] [3625/7595552] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:21] [Arc.MCC.TLS] [ERROR] [3625/7612448] SSL error: 6 - (empty):(empty):EVP lib
[2009-10-24 19:06:21] [Arc.MCC.TLS] [ERROR] [3625/7602608] SSL error: 6 - (empty):(empty):EVP lib

arccp: 

$ for i in `gseq 0 9`; do (time arccp files/100MB https://localhost:60000/file$i) 2> /tmp/results$i & done
$ for i in `gseq 0 9`; do sudo diff files/100MB /tmp/hopi/file$i; done
$ cat /tmp/results? | grep real
real	3m10.114s
real	3m9.321s
real	3m9.787s
real	3m10.074s
real	3m9.841s
real	3m8.739s
real	3m9.146s
real	3m8.326s
real	3m9.623s
real	3m9.897s

The arccp was slower then curl in this case also.

D: with TLS, with Plexer

not tested

Slave mode tests

Add slavemode to the ini: 

[hopi]
documentroot=/tmp/hopi
slavemode=1

Run it: 

$ sudo /usr/local/sbin/arched -f -i HopiServiceNonSecure.ini

HTML listing should not work

curl: 

$ curl -v http://localhost:60000
* About to connect() to localhost port 60000 (#0)
*   Trying ::1... Connection refused
*   Trying fe80::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 60000 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: localhost:60000
> Accept: */*
> 
< HTTP/1.1 500 Internal error
< Content-Length: 0
< Connection: keep-alive
< 
* Connection #0 to host localhost left intact
* Closing connection #0

arccp: 

$ arccp http://localhost:60000 index.html; cat index.html
[2009-10-24 11:16:45] [Arc.arccp] [ERROR] [1418/12619072] Transfer FAILED: Failed while reading from source

Second download attempt should fail

Copy a file into the documentroot, and download it twice. Actually the second attempt should only fail if it is after a specific timeout, which is 10 seconds by default.

curl: 

$ curl -s http://localhost:60000/100MB > file; ls -l file; sleep 10; curl -s http://localhost:60000/100MB > file; ls -l file
-rw-r--r--  1 zsombor  staff  104857600 Okt 24 11:22 file
-rw-r--r--  1 zsombor  staff  0 Okt 24 11:22 file
$

arccp: 

$ arccp http://localhost:60000/100MB file; ls -l file; sleep 10; arccp http://localhost:60000/100MB file
-rw-r--r--  1 zsombor  staff  104857600 Okt 24 11:20 file
[2009-10-24 11:20:45] [Arc.arccp] [ERROR] [1453/12619056] Transfer FAILED: Failed while reading from source

Upload to a non-existent path sould fail

curl: 

$ curl -v -T files/100MB http://localhost:60000/not_there
* About to connect() to localhost port 60000 (#0)
*   Trying ::1... Connection refused
*   Trying fe80::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 60000 (#0)
> PUT /not_there HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: localhost:60000
> Accept: */*
> Content-Length: 104857600
> Expect: 100-continue
> 
< HTTP/1.1 500 Internal error
< Content-Length: 0
< Connection: keep-alive
< 
* Connection #0 to host localhost left intact
* Closing connection #0

arccp: 

$ arccp files/100MB http://localhost:60000/not_there
[2009-10-24 15:32:47] [Arc.MCC.TCP] [ERROR] [2734/12692480] Failed to send content of buffer
[2009-10-24 15:32:47] [Arc.MCC.TCP] [ERROR] [2734/12692480] Failed to send content of buffer
[2009-10-24 15:32:47] [Arc.MCC.TCP] [ERROR] [2734/12692480] Failed to send content of buffer
[2009-10-24 15:32:47] [Arc.MCC.TCP] [ERROR] [2734/12692480] Failed to send content of buffer
[2009-10-24 15:32:47] [Arc.MCC.TCP] [ERROR] [2734/12692480] Failed to send content of buffer
[2009-10-24 15:32:48] [Arc.arccp] [ERROR] [2734/12619088] Transfer FAILED: Failed while writing to destination


Second attempt to upload should fail

Touch a file in the documentroot:

curl: 

$ curl -v -T files/100MB http://localhost:60000/put_here
* About to connect() to localhost port 60000 (#0)
*   Trying ::1... Connection refused
*   Trying fe80::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 60000 (#0)
> PUT /put_here HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: localhost:60000
> Accept: */*
> Content-Length: 104857600
> Expect: 100-continue
> 
< HTTP/1.1 200 OK
< Content-Length: 0
< Connection: keep-alive
< 
* Connection #0 to host localhost left intact
* Closing connection #0
$ curl -v -T files/100MB http://localhost:60000/put_here
* About to connect() to localhost port 60000 (#0)
*   Trying ::1... Connection refused
*   Trying fe80::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 60000 (#0)
> PUT /put_here HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: localhost:60000
> Accept: */*
> Content-Length: 104857600
> Expect: 100-continue
> 
< HTTP/1.1 500 Internal error
< Content-Length: 0
< Connection: keep-alive
< 
* Connection #0 to host localhost left intact
* Closing connection #0

arccp: 

$ arccp files/100MB http://localhost:60000/put_here
$ arccp files/100MB http://localhost:60000/put_here
[2009-10-24 15:38:43] [Arc.MCC.TCP] [ERROR] [2780/12692528] Failed to send content of buffer
[2009-10-24 15:38:43] [Arc.MCC.TCP] [ERROR] [2780/12692528] Failed to send content of buffer
[2009-10-24 15:38:43] [Arc.MCC.TCP] [ERROR] [2780/12692528] Failed to send content of buffer
[2009-10-24 15:38:43] [Arc.MCC.TCP] [ERROR] [2780/12692528] Failed to send content of buffer
[2009-10-24 15:38:43] [Arc.MCC.TCP] [ERROR] [2780/12692528] Failed to send content of buffer
[2009-10-24 15:38:44] [Arc.arccp] [ERROR] [2780/12619088] Transfer FAILED: Failed while writing to destination

Conclusion

There are some issues with performance and strange error messages, but otherwise Hopi is working as intended.

Test Review at Egerszalok

  • Nice tests carried out. Zsombor is kindly asked to prepare a technical documentation of the Hopi based on this wiki page
  • Comment from Aleksandr: arccp should be slower than curl by design. Somebody should define how much slower is ok :-)
  • Zsombor is also asked to look into ERROR messages and correct as much as possible (communicate with other developers as well). Use Bugzilla if needed.
Retrieved from "https://wiki.nbi.ku.dk/w/index.php?title=NOX/Tests/HopiResults&oldid=1941"