Eats the Archive as It Goes

A friend at a party said their laptop was too full to extract a download. Three days later I had shipped a thing about it and it was the most-starred new Rust repo on GitHub, and a week after that Google was listing it next to 7-Zip.

I still don't fully know what to do with that.

What I worked on

• hatchzip. The whole week, basically. 100 commits, all of them to this one repo.

  The problem is stupid once you see it. You finish a 180 GB download, you hit extract, and it says not enough disk space, because unzip needs room for the archive and the files at the same time. The archive is already sitting there. The files need their own space on top of it.

  So hatchzip extracts the archive from the end, and after each finished batch it cuts the archive shorter. The files take the archive's place. You only need room for what's inside, not for both at once.

Two things I cared about more than the trick itself. It's crash-safe: kill it or lose power halfway and running it again picks up exactly where it stopped, which I tested by killing it at every commit point. And it checks the space before touching anything, so it refuses rather than stranding you with half an archive and half a folder.

On a disk with 220 MiB free and a 180 MB zip that unpacks to 200 MB, unzip died at file 6 of 21. hatchzip finished and every file came out byte-identical. It's also faster, because it unpacks several files in parallel: 0.25s against 1.48s on that same zip.

One honest limit I put in the launch post itself: 7-Zip's default settings pack everything into a single solid block, so there's nothing to cut off the end and hatchzip can't save you space on those yet. It tells you that before it starts rather than after.

Built in about two hours on Wednesday. The remaining three days were everything that comes after building.

• Six releases in four days, v0.1.0 through v0.1.5. RAR support, encrypted RAR filenames, a Windows installer, Homebrew, Scoop, and a winget PR that's still sitting in Microsoft's queue. 13,401 downloads by Sunday night, about three quarters of them Windows, which tells you exactly who has this problem.

The launch

Posted it Thursday at 6:54pm. Took a screenshot when it had 10 views, because I take a screenshot of everything.

An hour later it was at 77K. By Friday night it was past 131K and still going.

I also put it on Show HN the same night, where it got three points and two comments and sank without trace. Two things launched within hours of each other, same project, same words, and one of them did nothing at all.

Friday is the day the whole thing actually happened. 937 stars in that one day, then another 732 on Saturday. I posted a star screenshot roughly every time the number changed, which in hindsight was a lot of posting.

Followers went 2,222 on Monday to past 3,000 by Sunday. I watched 2,300 turn into 2,400 in three hours and genuinely did not know what to do with myself.

Some of the replies were very kind and I'm keeping them.

Two security researchers went through it properly and found real bugs, not drive-by nitpicks: an archive could plant a resume file to write outside the output folder, and a link entry under another link entry could escape the destination. Both fixed and shipped in v0.1.5 on Saturday with published advisories. That is the part of this week I'm most pleased about, because someone caring enough to audit your three-day-old project is its own kind of compliment.

Strangers in the repo

By Saturday there were pull requests from people I've never met.

• kodejeet cut out a full archive read before ZIP extraction, which matters a lot when the archive is 180 GB.
• 011010 sent four: test coverage for XZ-in-ZIP and Zstandard-in-7z using fixtures from real tools, logging so a macOS flake would name its own cause, an "open another archive" flow after a finished extraction, and a fix treating ENOTTY from F_PUNCHHOLE as a filesystem that just can't punch holes.

They also filed a proper writeup of extraction and resume behaviour on a FAT32 USB stick on macOS, with throughput numbers. Nobody asked them to do that.

Four days old and people were already doing better work on the hard parts than I would have gotten to.

Where it went after that

Writing this on Thursday, five days past the end of the week, and the thing has kept moving without me.

The launch post is at 1.8 million views, 28K likes and 2.1K reposts, which is about seventy times my previous best.

The repo is at 2,308 stars and 72 forks, with 16,889 downloads across every release and eight people in the contributors list. It hit #2 on Trendshift's daily Rust list and #17 on the weekly, and spent Oct 24 and 25 on the overall trending board.

The part I did not expect at all: it went properly viral in Chinese tech Twitter, entirely separately from my post. Six or seven accounts over four days, some of them big, explaining the trick in Chinese with their own rebuilt diagrams and benchmark numbers. One was a "things worth saving this week" roundup that put it alongside a Rust PDF editor with 17,000 stars. I cannot read most of it and I have been running it through translate like a tourist.

There are YouTube Shorts about it now, made by people I have never spoken to. There are two full re-uploads of the repo under other people's accounts, which is less fun but comes with the territory.

And then this, which is the one that genuinely got me:

Google's AI answer for "open source zip file extractor" lists 7-Zip at 27 years old, PeaZip at 20 years old, and hatchzip at 6 days old. (post)

What I found interesting

• Unpackerr : extracts downloads for the *arr stack and deletes the extracted files after import. Found it while checking whether hatchzip already existed.
• star-history : the de facto GitHub star history graph. Used it a lot this week for obvious reasons.
• localflow : native private dictation for mac, Windows and Linux, in Rust, with its own speech engine and its own cleanup model. Nothing leaves your machine.
• betterwispr : local-first dictation for macOS. Hold a shortcut, speak, text appears.
• magicalos : an operating system written in Rust.
• arpchat : texting your friends using only ARP. Answering the question nobody asked, which is the best category.
• jobs : a job board you reach over SSH.
• 23rd.dev : framework-agnostic UI components, copy paste ship.

shawn has been posting isometric pixel scenes and this one has been in my head since Tuesday.

Random

My laptop is cooked. Load average 27.79, 99% CPU at 80°C, memory at 90%, disk at 87% full, which is both the symptom and the joke given what I spent the week building.

A MacBook Pro is arriving next week. I have thought about very little else.

Also burned through all 3,000 GitHub Actions minutes eleven days before the reset. Public CI is free, so this was entirely a private repo doing it to me.

Someone got phished by an account impersonating me, close enough on picture and name and handle that they reported mine first, then messaged to apologise once X sorted it out. A recognisable account turns out to come with a fake one attached.

The desktop on launch night, which is also my favourite wallpaper I've ever had.

Gave the next rounds at Tech Mahindra on Saturday, and I'm through to the interview. I'll probably turn it down come summer, but I'm going to give the interview properly regardless.

By Saturday night I was completely done with the hype and said so out loud, then played GTA V until I wasn't. Earlier in the week it was Drive Mad, which is a browser game I am not going to justify.

Studied operating systems on Sunday, because I did almost no college work all week and at some point that bill arrives too.

Monday's uptime was 60 hours. Tuesday I slept properly for the first time in a long while, which I noted at the time because it was unusual enough to note. Wednesday the wall fan broke, I tried to fix it, made the room measurably dustier, gave up and slept with the AC on.

And then this, which is either about future me or current me, I genuinely can't tell anymore.