I need to compress small size of data in my project without too heavy side-effect on performance. Many people recommend LZ4 for me since it is almost the fastest compression algorithm at present.
Compression ratio is not the most important element here, but it still need to be evaluated. So I create a 4KB file contains normal text (from a README file from a open source software) and compress it with LZ4 and GZIP.
| LZ4 | GZIP -1 | |
| Compression Ratio | 1.57 | 2.24 |
Hum, Looks like the compression ratio of LZ4 is not bad. But when I run test with this special content (changed from here):
The First 1,000 Primes (the 1,000th is 7919) For more information on primes see http://primes.utm.edu/ 2,,3,,5,,7, 11, 13, 17, 19, 23, 29 31, 37, 41, 43, 47, 53, 59, 61, 67, 71 73, 79, 83, 89, 97, 101, 103, 107, 109, 113 127, 131, 137, 139, 149, 151, 157, 163, 167, 173 179, 181, 191, 193, 197, 199, 211, 223, 227, 229 233, 239, 241, 251, 257, 263, 269, 271, 277, 281 283, 293, 307, 311, 313, 317, 331, 337, 347, 349 353, 359, 367, 373, 379, 383, 389, 397, 401, 409 419, 421, 431, 433, 439, 443, 449, 457, 461, 463 467, 479, 487, 491, 499, 503, 509, 521, 523, 541 547, 557, 563, 569, 571, 577, 587, 593, 599, 601 607, 613, 617, 619, 631, 641, 643, 647, 653, 659 661, 673, 677, 683, 691, 701, 709, 719, 727, 733 739, 743, 751, 757, 761, 769, 773, 787, 797, 809 811, 821, 823, 827, 829, 839, 853, 857, 859, 863 877, 881, 883, 887, 907, 911, 919, 929, 937, 941 947, 953, 967, 971, 977, 983, 991, 997,1009,1013 1019,1021,1031,1033,1039,1049,1051,1061,1063,1069 1087,1091,1093,1097,1103,1109,1117,1123,1129,1151 1153,1163,1171,1181,1187,1193,1201,1213,1217,1223 1229,1231,1237,1249,1259,1277,1279,1283,1289,1291 1297,1301,1303,1307,1319,1321,1327,1361,1367,1373 1381,1399,1409,1423,1427,1429,1433,1439,1447,1451 1453,1459,1471,1481,1483,1487,1489,1493,1499,1511 1523,1531,1543,1549,1553,1559,1567,1571,1579,1583 1597,1601,1607,1609,1613,1619,1621,1627,1637,1657 1663,1667,1669,1693,1697,1699,1709,1721,1723,1733 1741,1747,1753,1759,1777,1783,1787,1789,1801,1811 1823,1831,1847,1861,1867,1871,1873,1877,1879,1889 1901,1907,1913,1931,1933,1949,1951,1973,1979,1987 1993,1997,1999,2003,2011,2017,2027,2029,2039,2053 2063,2069,2081,2083,2087,2089,2099,2111,2113,2129 2131,2137,2141,2143,2153,2161,2179,2203,2207,2213 2221,2237,2239,2243,2251,2267,2269,2273,2281,2287 2293,2297,2309,2311,2333,2339,2341,2347,2351,2357 2371,2377,2381,2383,2389,2393,2399,2411,2417,2423 2437,2441,2447,2459,2467,2473,2477,2503,2521,2531 2539,2543,2549,2551,2557,2579,2591,2593,2609,2617 2621,2633,2647,2657,2659,2663,2671,2677,2683,2687 2689,2693,2699,2707,2711,2713,2719,2729,2731,2741 2749,2753,2767,2777,2789,2791,2797,2801,2803,2819 2833,2837,2843,2851,2857,2861,2879,2887,2897,2903 2909,2917,2927,2939,2953,2957,2963,2969,2971,2999 3001,3011,3019,3023,3037,3041,3049,3061,3067,3079 3083,3089,3109,3119,3121,3137,3163,3167,3169,3181 3187,3191,3203,3209,3217,3221,3229,3251,3253,3257 3259,3271,3299,3301,3307,3313,3319,3323,3329,3331 3343,3347,3359,3361,3371,3373,3389,3391,3407,3413 3433,3449,3457,3461,3463,3467,3469,3491,3499,3511 3517,3527,3529,3533,3539,3541,3547,3557,3559,3571 3581,3583,3593,3607,3613,3617,3623,3631,3637,3643 3659,3671,3673,3677,3691,3697,3701,3709,3719,3727 3733,3739,3761,3767,3769,3779,3793,3797,3803,3821 3823,3833,3847,3851,3853,3863,3877,3881,3889,3907 3911,3917,3919,3923,3929,3931,3943,3947,3967,3989 4001,4003,4007,4013,4019,4021,4027,4049,4051,4057 4073,4079,4091,4093,4099,4111,4127,4129,4133,4139 4153,4157,4159,4177,4201,4211,4217,4219,4229,4231 4241,4243,4253,4259,4261,4271,4273,4283,4289,4297 4327,4337,4339,4349,4357,4363,4373,4391,4397,4409 4421,4423,4441,4447,4451,4457,4463,4481,4483,4493 4507,4513,4517,4519,4523,4547,4549,4561,4567,4583 4591,4597,4603,4621,4637,4639,4643,4649,4651,4657 4663,4673,4679,4691,4703,4721,4723,4729,4733,4751 4759,4783,4787,4789,4793,4799,4801,4813,4817,4831 4861,4871,4877,4889,4903,4909,4919,4931,4933,4937 4943,4951,4957,4967,4969,4973,4987,4993,4999,5003 5009,5011,5021,5023,5039,5051,5059,5077,5081,5087 5099,5101,5107,5113,5119,5147,5153,5167,5171,5179 5189,5197,5209,5227,5231,5233,5237,5261,5273,5279 5281,5297,5303,5309,5323,5333,5347,5351,5381,5387 5393,5399,5407,5413,5417,5419,5431,5437,5441,5443 5449,5471,5477,5479,5483,5501,5503,5507,5519,5521 5527,5531,5557,5563,5569,5573,5581,5591,5623,5639 5641,5647,5651,5653,5657,5659,5669,5683,5689,5693 5701,5711,5717,5737,5741,5743,5749,5779,5783,5791 5801,5807,5813,5821,5827,5839,5843,5849,5851,5857 5861,5867,5869,5879,5881,5897,5903,5923,5927,5939 5953,5981,5987,6007,6011,6029,6037,6043,6
the result became interesting:
| LZ4 | GZIP -1 | |
| Compression Ratio | 0.99 | 2.25 |
GZIP could compress this content unexceptionally, but LZ4 don’t. So, LZ4 can’t compress some special content (like, numbers), although it is faster than any other compression algorithm.
Somebody may ask: why don’t you use special algorithm to compress content of numbers? The answer is: in real situation, we could not know the type of our small data segments.