ts_zip: Text Compression using Large Language Models
The ts_zip utility can
compress (and hopefully decompress) text files using a Large Language
Model. The compression ratio is much higher than with other
compression tools. There are some caveats of course:
A GPU is necessary to get a reasonable speed. 4 GB of RAM is
required.
It is slower than conventional compressors (compression and
decompression speed: up to 1 MB/s on a RTX 4090).
Only text files are supported. Binary files won't be compressed
much. The currently used language model (RWKV 169...
Read more at bellard.org