Slow to add new file to ZIP with multile files inside

Question

I have an app that saves an image from IP camera to HDD every 30 seconds. So together in 24 hours I get 2880 images. It takes lot of time to open a folder with that many images so I wish to store them in ZIP file. Like this:

ZipArchive.Add(ZipPath, Path.Combine(KameraPath, _filename), "\\", ArchiveTraversalMode.NonRecursive, ArchiveActionOnExistingFile.OverwriteOlder);

Now the thing is that is works very fast when ZIP file has just few files inside. But at end of day it takes over 1 minute or more to add a single file (around 60kb) inside existing ZIP file with over 2000 small files.

Is there a way to speed this up somehow? Maybe if I disable compression? I just need them all stored in single ZIP files.

Answer 1

Actually, problem is not in adding files into a zip which contains a lot of items (in your case it should took less then 1s), but it is in archive shrinking.

You use ArchiveActionOnExistingFile.OverwriteOlder which can cause the files with the same name are deleted from archive at first, then the more recent file is added into archive. This makes a hole (unused space) within the archive. If the archive has 2800 * 64kb files, it means it is approx. 180MB file.

When using ZipArchive.Add method, the hole is automatically "removed" (archive is shrinked) which can take approx. 1 min.

You can choose between the two: Either you create "large" archives quickly or "small" archives slowly.

To disable automatic archive shrinking use this:

// add file into archive without shrinking
using (ZipArchive zip = new ZipArchive(ZipPath))
{
    // disable auto-shrink
    zip.SaveMode = ArchiveSaveMode.Delayed;

    // add required file
    zip.Add(Path.Combine(KameraPath, _filename), null, 0, 0, ActionOnExistingFiles.OverwriteOlder);

    // save archive and skip shrinking
    zip.Save(ArchiveSaveAction.None);
}

When you are done or when you decide to shrink the archive (e.g. it exceeds 1GB size on disk), you can invoke Shrink like this (but please note, it can take approx. 1 min):

// shrink the archive
using (ZipArchive zip = new ZipArchive(ZipPath))
{
    zip.Save(ArchiveSaveAction.Shrink);
}

Comments

Thank you very much. It is working much faster now.