ImageOptim2

[kornelski]

An update of what's happening with ImageOptim:

The existing project consists of multiple external executables (for various 3rd party tools), internal framework (forced by Apple's sandboxing requirements), and plug-in (for Apple Mail). This seems to be more than Xcode can handle, and it's causing a constant source of pain with broken Xcode builds, invalid Apple Code Signing signatures, inefficient and fragile archiving, etc. Apple's new Notarization requirements are even harder to satisfy and the errors it reports are Kafkaesque.

The net effect is that Xcode builds are broken more often than not. Making an ImageOptim release that takes hours of debugging, testing, and solving vague unhelpful errors reported by Apple's tooling and OS. This is not fun.

Second, ImageOptim's codebase is old. The project is 14 years old! When I started it multicore CPUs were a novelty, and many modern macOS APIs didn't exist yet. I've retrofitted what I could, but the architecture of the application is in some places messier than I'd like. AppKit/Cocoa is not thread-safe, and Objective-C is unhelpful for ensuring thread safety. In fact, small mistakes lead to very hard to track crashes. This is not fun either.

Third, being tied to Apple's proprietary technologies has been problematic. People keep asking for ImageOptim for Windows, or a CLI version that works on Linux servers, or something that will work on mobile. The current code can't do it. Being tied to the C language and the old AppKit is a problem. It's not even the best solution for macOS any more.

So my plan is:

• The current codebase is dead.
• I'm going to write a new core of ImageOptim using Rust. Rust is a native language with excellent support for reliable parallel code. It works on macOS, iOS, Windows, Linux, Android, and WASM. It doesn't need Xcode!
• There will be a native CLI version for all platforms that Rust supports.
• I'll give it a native UI for macOS using SwiftUI. But if Xcode and Code Signing keep being annoying as they were, you're getting an Electron app instead.
• I may make a Web version via WASM, and maybe GTK for Linux. We'll see.

There's no timeline for it. It's still a hobby project.

[ream88]

Sounds awesome and I feel the mood:

I'll give it a native UI for macOS using SwiftUI. But if Xcode and Code Signing keep being annoying as they were, you're getting an Electron app instead.

[custa1200]

When you say current codebase, I hope you don't mean the Sketch Plugin too... Been totally bummed about my workflow having to remember to go find and compress my files like an animal :)

[kornelski]

The Sketch issue is separate. Please talk to Sketch support to sort it out. Their developer docs have no mention of any changes for plug-in authors after v72, so it's probably just a bug in Sketch, not a plug-in issue.

[sindresorhus]

Great plan 👍

I'll give it a native UI for macOS using SwiftUI.

I'm happy to help with this.

[eexit]

I can't help much on this but I'm happy to donate to show my interest and contribution to this great tool.

Anyone who is interested in the next version should donate as well.

Thanks! ❤️

[ap]

@kornelski Does this mean that the 1.8.9 beta is as far as native M1 support for the old codebase is ever going to go?

[alexchandel]

Electron has poor integration and awful footprint, probably 500% disk usage increase minimum.

[utopiatopia]

Can you please explain why being tied to the C language is an issue? I agree with writing everything in Rust, however I’d prefer ImageOptim in Rust and everything else temporarily in C if it’s faster than rewriting everything in Rust.

[kornelski]

@utopiatopia It's not strictly blocked by things being in C, but more by my limited time to work on the project in general.

I do not plan to rewrite everything in Rust. For example, MozJPEG will stay in C, as changing it is a too big undertaking. I'm just not planning to ever write any new code in C, because for me Rust is more productive and reliable.

I want to stop being tied to macOS and Xcode, so I also need a replacement for the build system. For C there are many options, but none of them are good for cross-platform projects. So I'd like to keep as much in Rust/Cargo as possible. Fortunately I already have mozjpeg in Cargo. I've oxidized pngquant recently too only because it was easier than fixing C build system when cross-compiling (with ARM Macs, cross-compilation is now necessary).

[kishaningithub]

I would suggest using wails for a cross platform UI

https://github.com/wailsapp/wails

Tool similar to this built using wails
https://github.com/Splode/optimus

This one uses lossy compression though

[Melonking906]

This is sad news, ImageOptim is one of the diamonds of the MacOS world; I didn't realize it was all on one developer though, so I'm really grateful for your work kornelski, you've made so many of my web projects so much better.

Hopefully someone will fork the project and continue the original ImageOptim m1 support in the immediate future, as far as I know its the only image compressor that supports GIFs which is 80% of what I use it for.

Best of luck with ImageOptim2, it sounds like its gonna be a lot of work, but also a lot of discovery along the way! (Please don't make it an electron app...)

[kishaningithub]

Since you are planning to use rust, for the UI a proposal i have is to use tauri

https://tauri.studio/

This is built using rust, and can help create a cross platform UI (if possible i would like to use imageoptim even if i am a linux/windows user)

[fulldecent]

I propose a draft API for connecting the command line tool to the SwiftUI front end. If this can be provisionally approved, then maybe one person is able to move forward with the macOS GUI at the same time that you are working on the Rust part.

Of course someone else might even do the Electron app at the same time.

---

Loading the app is a shell call.

imageoptim --api

Work orders are passed to the program as JSON lines on STDIN. Options apply to each file input. (The API may also possibly support sending the entire image contents back and forth encoded in the JSON, if this helps with sandboxing.)

{
  "file": "/full/path/to/image.png",
  "action": "begin",
  "allowLossy": true,
  "jpegQuality": 99,
  "pngQuality": 80
}
{
  "file": "/full/path/to/image.png",
  "action": "abort"
}

And the program returns status updates using JSON lines on STDOUT.

{
    "file": "/full/path/to/image.png",
    "status": "working",
    "size": 23546
}
{
    "file": "/full/path/to/image.png",
    "status": "complete",
    "size": 23546
}

These status lines blast out while work is done and then the program exists with status 0 if everything worked. The "status": "working" lines are limited to no more than 5 per second per concurrent working file.

A schema/taxonomy of options is needed, error conditions must be defined. But this is a starting point that can easily be mocked (in a one line Zsh script?) and tested on both sides. And then ultimately the two sides connect.

[remmah]

Of note, Tauri just shipped 1.0 and passed a code audit. By using the native OS webview for each platform, the bundle size decreases dramatically. Drag-and-drop looks to be possible as well.

[fulldecent]

@kornelski Please advise if the above proposal is acceptable.

If so, I can help define these exact messages, somebody else can help implement a GUI using the API approach (any takers?), and your Rust program can consume/produce according to this API.

The end result will be that you only have to lead effort on Rust (for now) and everybody gets the modern, cross-platform, M2-compatible ImageOptim for the future.

[mrleblanc101]

Nearly 2 years after Apple Silicon launch, and no progress on this.
ImageOptim is my only Intel app left.

[kornelski]

A build for Apple Silicon exists, but it's not Notarized due to Apple's code signing tools being kafkaesque crappy misery, and I'm frustrated and burned out dealing with Xcode's crap.

Xcode does not seem to support signing app bundles that contain command-line executables. All the export and notarization options simply disappear from the UI when there's any CLI subproject (FB9885692), so the simple "oh, just click Archive and you're done" path doesn't exist at all for ImageOptim.app. The other way of doing every step manually by running multiple Apple code signing tools is bizarrely overcomplicated and doesn't work for me, and Apple's tools don't even explain why! codesign says everything is signed correctly, but spctl says it's not signed correctly enough to run. Neither tool says what's wrong, and Apple engineers explained to me that it's the intended behavior, because neither of these tools is intended for troubleshooting of notarization problems. And notarization tools just say it's invalid, and that's it. So that's it. Until apple fixes FB9885692 Xcode bug ImageOptim is dead.

[mrleblanc101]

Thank you for the update, I was not aware of the beta build or if it was stable enough to be used.

I understand the the only issue preventing 1.9 from being proposed via Sparkle auto-updating is the notarization ? Or are there other Apple-Silicon bug/regression I should be aware ?

[kornelski]

There are no regressions. It should work great. The only blocker is that due to the code signing mess, Apple will tell you it's malware and tell you to trash it.

[mrleblanc101]

Perfect, I installed it.
If your right-click and select Open it bypass the message and I don't think it ever appear again. I understand that for many inexperienced user this is scary, but for me it's gonna be fine until the v2 Rust rework is out.

[ickc]

@kornelski, while not speaking for everyone, I think it is ok if it is not notarized. It is not the first open source project that are not notarized and while the user experience is a little bit more annoying (that I have to go into system preferences to confirm I want to open it), it works after the initial annoyance (and every time it is updated). For an open source tool, not notarizing is not really that big of a deal. Supply chain attack risk is higher but there are other ways to mitigate this, as other OSS do.

[kornelski]

This application is pretty widely used, and I know it has users who will be confused or scared by the off-putting user-experience that Apple has created for non-notarized apps, so I can't release it without notarization.

[ickc]

Not to intrude but I've seen more widely used softwares not notarized. A quick search resulted in this: ungoogled-software/ungoogled-chromium#859

But there are more.

From the comments here and elsewhere, people would be pissed off more if they are not updated for arm64 or just not working, then having a non-notarized app.

I know the general public may not be educated enough to think correctly, but notarization can gives a false sense of security for the uninformed. And usually this trust issue is not all that important for open source project. If people really cares about security they would have audit the code and compile it themselves (not much people do it either.)

But of course it is entirely up to you if you want to fight this battle, but it is alarming to me that it is burning you out doing so.

[kornelski]

Among over a million users of ImageOptim it's not just developers who have a GitHub account to comment here to say it's not a big deal to skip a prompt. There are also plenty of less-techy users who just upload a bunch of photos to their WordPress site, or completely noob users who merely want to share a photo on social media without leaking GPS data. On several occasions I even got complaints from users that "Get Info" in Finder shows "Macintosh HD > Users > THEIR NAME > Desktop > Photo.JPG" when I promised them to remove private info from the photo. I don't want to be on hook for explaining to these people that they don't have malware, and it's just Apple eroding software freedoms and executing it poorly.

[ickc]

Completely agree with you and you're the maintainer.

it's just Apple eroding software freedoms and executing it poorly

It is a bit too strong claiming that. The way I often say would be something like "It just works, until it doesn't." or "when it works it is magic, when it doesn't work it is also magic."

There are obvious reasons behind that (notarization) and also it creates extra works for developers to do as it is an extra step comparing to common practice they already knew (gpg, etc.) But on the other hand it really eliminates a class of problems. Apple's fault as usual is to not handle the exceptions—whenever people are doing things deviating from what Apple design it to works, it is a pain or impossible to get around—like here that you mentioned about the cli exception.

But I'm not disagreeing with you. I hope you feel better and not getting burn out. This is more important then what any end users including me would feel if you go one way or another.

[mattpilott]

@kornelski I use ImageOptim all the time, is there a rough timeline for v2? Will there be a milestone we can follow at some point?

[benjaminhoffman]

are you saying that the v2 will work on windows?!? that would be amazing and help me and my team out a LOT. how can i support the cause?

[gingerbeardman]

How is development of the Rust version going?

[BenRacicot]

+1, this project is so awesome and even famous now! Looking forward to any updates.

[gingerbeardman]

1.9.2 beta released

https://imageoptim.com/ImageOptim1.9.2.zip

[rpkoller]

ohhhhh is it really what i think it is? Is it a notarized universal binary now?!

[kornelski]

Yes, finally got notarization working, so I can resume development.

[phripley]

I am super excited to see this project back development.

What can users do to help? Are there specific tests that are needed?

In a quick test it appears that images that were previously losslessly optimized can now be further losslessly optimized. Presumably this is because the underlying libraries have been improved. Is that accurate?

[kornelski]

Yes, oxipng in particular got some improvements.