Here are the results of tests performed in August 2002 to compare
lossless compression of True Color (24-bit) images by all known programs
developed for such purpose, including BMF, JPEG-LS and PNG software,
7-Zip, Pkzip, and archivers with special algorithms for 24-bit images.
See Archive Comparison Test by Jeff Gilchrist for more info:
http://compression.ca
If anybody wants to start or continue such tests,
or can suggest some other sets of images, or other compression programs,
(not sources or algorithm descriptions, executable programs only)
or knows we have missed something important,
(some new fantastic technology, an algorithm or even a program capable
of lossless compression of up to 1000:1 and lossy compression of
up to 10000:1, we hear about it from time to time since PC-XTs)
please let us know immediately: artest@inbox.ru Thank you!
[[1]] COMPRESSION QUALITY
=========================
(see also
[[2]] Speed
[[3]] Details
[[4]] Comments)
The last 13th line shows results for the sum of all 12 sets (627 images, 2 Gb)
Original BMF RKIM UHIC NK ERI ArHanGeL UHARC ACE LSP
312.42% 113.54 115.37 114.15 124.36 100% 125.45 122.15 129.46 131.09
289.12% 100% 124.50 104.88 132.34 114.40 114.61 112.22 121.23 136.11
245.22% 100% 101.39 107.94 109.56 119.38 112.92 118.90 114.34 111.40
238.64% 100% 112.66 104.21 118.11 115.14 110.53 116.65 114.13 122.74
472.81% 102.89 100% 112.25 114.09 136.49 113.07 125.36 120.74 235.68
234.76% 100% 101.90 104.87 106.86 111.69 109.64 116.21 112.66 112.06
300.89% 100% 104.16 104.39 108.19 111.54 111.11 114.53 117.33 123.56
342.65% 100% 102.61 106.05 110.30 116.76 111.61 120.70 118.36 116.50
227.58% 100% 104.53 104.01 108.85 110.58 108.92 114.55 111.00 110.36
547.51% 100% 104.26 104.85 122.41 119.24 107.77 127.08 156.04 132.31
298.76% 108.25 109.28 109.79 118.03 100% 121.55 116.62 124.64 126.27
336.86% 109.13 108.43 110.28 118.18 100% 118.07 110.08 127.27 127.96
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
294.02% 100% 102.58 104.48 109.04 110.21 110.54 115.21 116.54 117.36
SBC DC RK LOCO ZZip PNGCrush RAR Compressia Display Pkzip
135.18 136.17 137.24 169.98 148.92 170.75 138.18 148.45 174.71 190.14
132.44 127.16 127.77 156.42 132.25 151.37 130.67 165.23 164.86 208.56
120.62 123.53 125.13 119.20 130.84 129.46 122.23 160.19 139.54 195.64
125.49 124.69 124.77 127.46 130.69 134.30 124.18 158.01 144.00 184.72
123.48 137.85 134.12 176.37 139.34 145.17 195.16 158.07 157.13 187.70
117.66 118.60 120.00 115.91 125.98 119.56 127.05 131.56 128.84 173.10
118.23 122.63 122.20 120.16 128.24 129.43 137.46 131.27 136.48 169.65
119.24 126.75 127.19 127.12 133.98 134.58 130.89 148.70 143.27 201.55
116.47 120.30 120.62 118.46 126.11 122.16 117.20 141.28 133.11 181.12
118.41 131.16 132.39 151.18 137.54 141.97 130.70 156.88 148.75 193.33
128.38 130.83 130.82 151.91 137.53 156.67 133.92 140.20 161.61 179.17
120.00 127.74 127.52 143.54 129.09 141.79 152.45 123.20 151.10 188.14
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
117.79 122.11 122.59 124.70 128.42 129.61 131.02 135.81 137.89 180.73
[[2]] Speed
===========
A set of 24 Kodak images from ftp://ftp.cipr.rpi.edu/stills/kodak/color/
was used for this test, and a 970MHz PC with 256Mb RAM and Windows98.
Programs, Compression/ Overall Average Users' Compressed
options Extraction, Score Score Size
seconds seconds, % seconds, % bytes , %
no compression 0 0 7865 304 7865 308 28312848 312
NK 139 28 3398 131 3272 128 11270106 124
locoe -c1 13 13 4305 167 4292 168 15404818 170
locoe -c2 15 14 4320 167 4305 169 15446962 170
cbtpc 18 9 4724 183 4706 184 16908835 187
lsp.bat e 13 13 3326 129 3313 130 11880422 131
lsp.bat e -f 11 11 3411 132 3401 133 12203421 135
7za a -tzip 28 2 4584 177 4557 178 16394253 181
7za a -mx -tzip 51 2 4602 178 4552 178 16376722 181
7za a -t7z 75 5 4094 158 4020 157 14449249 159
7za a -mx -t7z 101 5 4094 158 3994 156 14353985 158
ace32 a -d4096 43 10 3305 128 3263 128 11709297 129
ace32 a -d4096 -m1 42 10 3306 128 3264 128 11715709 129
ace32 a -d4096 -m5 50 10 3318 128 3269 128 11730213 129
arh a -mm -1 -2 69 38 3265 126 3197 125 11368943 125
bmf -f 22 8 3087 119 3066 120 11007688 121
bmf -f -q9 104 7 3161 122 3058 120 10979484 121
bmf -f -s 90 78 3026 117 2937 115 10289304 114
bmf -f -s -q9 240 78 3176 123 2938 115 10289304 114
compcl c -b15 -s 63 69 3868 150 3806 149 13451942 148
dc e -b16300 -mb5 58 18 3503 135 3446 135 12339333 136
display -bpng -i --same 42 5 4445 172 4404 172 15833142 175
eri a -3h13 32 37 2585 100 2554 100 9062479 100
eri a -3f13 32 37 2601 101 2570 101 9117643 101
eri a -3e13 32 37 2625 102 2594 102 9203321 102
eri a -3e13 -h90 23 29 2642 102 2619 103 9323447 103
gcac a 47 21 3553 137 3507 137 12547410 138
gcac s 48 21 3606 139 3558 139 12729808 140
imp98 a -mm -2 22 9 3923 152 3901 153 14008321 155
imp98 a -2 -s4 22 9 3926 152 3904 153 14019704 155
pkzip -es 4 1 5750 222 5746 225 20681803 228
pkzip -a 7 1 4837 187 4830 189 17381963 192
pkzip -exx 9 1 4839 187 4830 189 17383851 192
pkzipc -add 7 2 4853 188 4846 190 17436386 192
pkzipc -add -max 8 2 4845 187 4837 189 17405433 192
pkzipc -add -defl 15 2 4816 186 4801 188 17274828 191
pkzipc -add -max -defl 20 2 4809 186 4789 187 17231333 190
pngc-mmx -q 128 - 4427 171 4300 168 15474254 171
pngc-mmx -q -loco 127 - 3733 144 3607 141 12982336 143
pngc-mmx -q -brute 2351 - 6609 256 4282 168 15328841 169
qlfc a 28 21 3764 146 3736 146 13373069 148
rar a -m1 -mcc+ -s 27 6 3507 136 3479 136 12504418 138
rar a -m3 -mcc+ -s 28 6 3507 136 3480 136 12504424 138
rar a -m4 -mcc+ -s 29 7 3509 136 3480 136 12503013 138
rar a -m5 -mcc+ -s 29 7 3510 136 3481 136 12505518 138
rk -mf1 37 26 3924 152 3888 152 13901688 153
rk -mx1 476 478 4429 171 3958 155 12510696 138
rk -mx2 669 674 4769 184 4107 161 12336268 136
rk -mx3 866 868 5116 198 4259 167 12176044 134
rkim c 58 64 3145 122 3087 121 10883083 120
rkim cx 144 76 3123 121 2981 117 10454964 115
sbc c -m2 -b63 63 33 3500 135 3438 135 12254334 135
sbc c -m3 -b63 75 36 3514 136 3440 135 12251188 135
sbc c -m3 -b63 -ad 75 37 3507 136 3433 134 12224642 135
sbc c -m3 -b63 -os 79 36 3510 136 3432 134 12223134 135
uharc a -m1 -md32768 173 26 3573 138 3401 133 12146238 134
uharc a -mz -md32768 35 38 3960 153 3926 154 13994087 154
uharc a -mx -md32768 268 246 3616 140 3350 131 11167998 123
uhic e 55 58 3024 117 2969 116 10479144 116
uhic m 107 110 3091 120 2985 117 10344923 114
zzip a 35 27 3816 148 3781 148 13515967 149
zzip a -mm -mx -26m 41 28 3817 148 3777 148 13495871 149
zzip a -mm -26m 41 28 3817 148 3777 148 13495871 149
Losslessly compressed with ERI32 to 8,891,702 bytes, this set can be found
at http://artest.virtualave.net/ki/ and http://geocities.com/rao-inc/kodak/
Overall score is calculated by adding compression time, extraction time, and
time it would take to transfer the compressed file over a 28,800bps network:
(compressed_size)/3600
Average Users' score is calculated by adding (compress_time/10)+ extract_time +
time it would take to transfer the compressed file over a 28,800bps network.
Compression time is divided by 10 here, because more than 90% of people would
never compress anything during their life (with compression programs), but they
use compressed data almost _every_ time they use computers and/or Internet.
That's why compression time is not so actual for them.
[[3]] Details
=============
are no longer put to this main text
(984 lines reporting 25600 results on 627 files in 12 sets),
but can be found in FULL version with GRAPHICS.DAT and all *.BAT
at http://geocities.com/SiliconValley/Bay/1995/artest24.zip
or http://artest1.tripod.com/artest24.zip
1,2,3,4,9th image sets were taken from ftp://ftp.cipr.rpi.edu/stills/ and
/sequences/ , converted from Sun Raster format to Windows Bitmap,
(this conversion changes image headers only, 32 bytes
are replaced by 54 bytes, but image data is not modified)
This site www.cipr.rpi.edu is described in comp.compression FAQ,
in 29 lines of "Subject: [55] Where can I find Lenna and other images?"
Eight images in the 5th set are from the famous Waterloo Color Set,
( losslessly compressed to 2,801,690 bytes,
it can be found at http://geocities.com/eri32 )
6,7,8,12th image sets are from ftp://download.intel.com/pressroom/images/
They were converted from TIFF format to Windows Bitmap.
10th and 11th sets were downloaded from ftp://ftp.kodak.com/www/images/
Some images were slightly renamed. See GRAPHICS.DAT inside artest24.zip
for more details. Some new images (from download.intel.com) were not added yet.
[[4]] Comments
==============
Links to download programs:
~~~~~~~~~~~~~~~~~~~~~~~~~~~
ACE 2.04 :W http://winace.host.sk/ace204.exe 593K
BMF 1.1f :e http://geocities.com/eri32/bmf11f.zip 76K
ERI 5.1fre :e http://geocities.com/eri32/eri51fre.zip 89K
UHArc 0.4b :eW ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc04.zip 220K
ZZip 0.36c :W http://www.via.ecp.fr/~damien/downloads/zzip-win32.zip 35K
ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip 50K
DC 0.98b :W ftp://ftp.elf.stuba.sk/pub/pc/pack/dc124.zip 55K
Imp 1.1 :e http://www.technelysium.com.au/imp110d.zip 266K
Imp-win 1.12 :W http://www.technelysium.com.au/imp112.exe 122K
Pkzip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K
PkzipC 4.00 :W ftp://ftp.pkware.com/pkzc400s.exe 3470K
RK-dos 1.04.1 :e http://rksoft.virtualave.net/downloads/rk104a1d.exe 461K
RK 1.04.1 :W http://rksoft.virtualave.net/downloads/rk104a1w.exe 380K
RKim 1.06 :W http://rksoft.virtualave.net/downloads/rkim106.zip 57K
UHIC 1.0 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip 52K
NK 2.0beta :W ftp://tu.tusur.ru/pub/ImageTools/NK2beta.ZIP 52K
PngCrush 1.5.10:e http://download.sourceforge.net/pmt/pngcrush-1.5.10-DOS.zip
JPEG-LS 1.00 :W http://www.hpl.hp.com/loco/jlsrefV100.zip 325K
SPIHT Nov'96 :W http://www.cipr.rpi.edu/research/SPIHT/EW_Code/lsp.exe 72K
BTPC 4.1 :W http://www.engr.mun.ca/~john/btpc4.1.zip 715K
Display 1.90b5:e http://www.freenet.edmonton.ab.ca/~crnelson/disp.exe 811K
SEA 1.3 :e ftp://ftp.elf.stuba.sk/pub/pc/graph/sea13.zip 705K
Alchemy 1.10 :a ftp://ftp.elf.stuba.sk/pub/pc/graph/alch110.zip 1462K
PPMN_km b4 :W http://compression.graphicon.ru/ms/ppmn_km.rar
7-Zip 2.30b23 :W http://belnet.dl.sourceforge.net/sevenzip/7z230b23.exe
WinRAR 3.00 :W http://www.rarlab.com/rar/wrar300.exe
RAR32 3.00 :e http://www.rarlab.com/rar/rarx300.exe
SBC_d 0.969br1 :e http://personal.inet.fi/musiikki/sjm/sbc0969b_dos.zip
SBC 0.969br1 :W http://personal.inet.fi/musiikki/sjm/sbc0969b_win32.zip
Compressia 0.98b :W http://www.compressia.com/compcl.exe
:a - any DOS - DOS programs, will run under pure DOS or in a DOS box
:e - extender - DOS programs using DOS extenders like DOS/4GW or CWSDPMI
:W - windows - Windows95/98/NT/etc programs
If direct link doesn't work-most probably newer version of the program appeared
at the same site: visit web page, or read the whole directory from ftp server
(i.e. try the same URL, but without filename).
Homepages:
~~~~~~~~~~
Arhangel : http://geocities.com/SiliconValley/Lab/6606
Eri32 : http://geocities.com/eri32
mirror : http://artest1.tripod.com
RK, RKim : http://rksoft.virtualave.net
Imp,WinImp : http://www.technelysium.com.au/winimp.html
ACE,WinACE : http://www.winace.com
PkZip : http://www.pkware.com
RAR,WinRAR : http://www.rarsoft.com
JPEG-LS : http://www.hpl.hp.com/loco
PngCrush : http://pmt.sourceforge.net/pngcrush
SPIHT : http://www.cipr.rpi.edu/research/SPIHT
BTPC : http://www.engr.mun.ca/~john/btpc.html
Display : http://fn2.freenet.edmonton.ab.ca/~crnelson/display.html
Alchemy : http://www.handmadesw.com/Products/Image_Alchemy.htm
SEA : http://www.photodex.com/products/dos
SBC : http://sbcarchiver.netfirms.com
ZZip : ? http://www.zzip.f2s.com
Compressia : http://www.compressia.com/
Ufa,777,
7-Zip : http://www.7-zip.org
What's new:
~~~~~~~~~~~
6 new programs were tested:
PngCrush 1.5.10
7-zip 2.30 b22
RAR 3.00
SBC 0.969b r1
PPMN_km b4
Compressia 0.98b
7-zip, PngCrush and BZip2 perform exactly as their previous versions
tested in March or earlier.
Latest beta version of DC was available from author by e-mail:
DC: EdgarBinder@t-online.de
Results of IMP, 7-zip, BTPC and PPMN are in the full version only,
see GRAPHICS.DAT.
WARNINGS:
~~~~~~~~~
NK can't losslessly decompress some 8-bit images.
BMF doesn't save all fields of bmp-headers.
ESP, Rkive and some other programs are not tested any more,
their results and links can be found in previous versions:
ESP - artest3...artest12
Rkive - 3...12
UFA - 3...12
777 - 13...14
BOA - 7...14
IMP - 13...22
SEA - 12...19
CompuShow 2000! - 3...11
The LATEST RELEASE, and all previous versions of these tests can be found
at http://geocities.com/SiliconValley/Bay/1995/ and http://artest1.tripod.com/
Send your suggestions, comments to artest@inbox.ru
With best kind regards,
A.Ratushnyak
Back to main ARTest page
