[HN Gopher] Python - Create large ZIP archives without memory in... ___________________________________________________________________ Python - Create large ZIP archives without memory inflation Author : pythonscripts2 Score : 26 points Date : 2021-01-02 21:04 UTC (1 hours ago) (HTM) web link (github.com) (TXT) w3m dump (github.com) | amelius wrote: | Does anyone know of a tar equivalent which performs | deduplication? | spockz wrote: | Is there something like this for the JVM? I'm not sure whether | with https://github.com/srikanth-lingala/zip4j#adding-entries- | wit... will keep everything it is possible to keep it in | constrained memory. | MaxBarraclough wrote: | Can someone explain what it's doing? Is it using an algorithm | with far superior space complexity than the usual algorithm? | | Python seems a curious choice. Compression is computationally | intensive. | da_big_ghey wrote: | Looks like it just splits by 16MB chunks, so just standard | deflate. Actual compression is handled by the python zipfile | module, which is probably C code underneath. | leiserfg wrote: | How does it compares to https://github.com/kbbdy/zipstream? ___________________________________________________________________ (page generated 2021-01-02 23:00 UTC)