Isolate is a tool I built for myself to view and organize art reference/inspiration. It’s a power user tool; it does a few things very fast and well.

While scrolling quickly, images sometimes pop in after a delay, rather than being visible immediately as the page scrolls.

Is this worth fixing? Is it worth a third rewrite? Probably not. Talking to users, nobody has ever mentioned it and I maybe was better off spending my vacation drawing or doing something other than coding for 12 hours straight cause that’s what I was supposed to get a break from.

But here were are, and fundamentally the problem is Isolate is loading full sized images to display as thumbnails. Even reading locally from disk is not instant for a dozen images at say ~1mb each. With some judicious css to ensure the position of visible elements is not affected by offscreen images with unknown dimensions size means Chrome can intelligently lazily load these images, but the problem is still visible in these few milliseconds when scrolling.

Similar software will thumbnail the images, saving a resized small version of the image. But this is computationally expensive, and not something I’d want to do on the same thread handling UI interactions (thumbnailing my dir of 500 images takes ~20 seconds), else this goes from a problem of barely perceived rendering lag to this-app-is-slow-hold-on-I-have-a-rant-about-Electron.

Problem statement

To step back, I’ve so far built Isolate in Electron. I want it to be cross-platform because I draw/sculpt in Windows, and everything else in OSX. I wish this was not the case; even cross-platform frameworks require lots of testing/platform specific work.

I’d like images thumbnailed in a background thread upon opening a directory, and a small store with metadata like hash, dimensions and thumbnail for fast querying.

The open questions for me were:

• I’ve heard Javascript is slow, is this too expensive for Javascript?
• How to communicate with the background thread/process
• How to package and manage the application lifecycle

I prototyped some different approaches. I learned a lot, much more than just reading about them. For the most part, are all workable. This is just my reasoning and experience with the different directions I tried:

• C++ prototype (QT, then SDL with intent of imgui)
• Electron with a compiled daemon communicating locally via http
• background threads in Electron: service/web workers, Node cluster/worker, second renderer process

Cross platform fully C++ prototype (QT/SDL)

A task like the above problem statement is straightforward and routine in a reasonably performant language with concurrency, and I could just reimplement the whole app in such a language like C/C++.

I started off with a version in QT as a common recommendation and backbone of heavy apps like Houdini. Then, because Isolate is fairly simple, I tried SDL to build my own abstractions rather than diving too deep into how make QT do what I want.

Thoughts:

• While the background worker is easier, I’d have to reimplement a lot of what I take for granted. E.g. layout, swapping thumbnails with full-sized images based on display size, application packaging, even establishing my own logging because I don’t have console.log. This means I have much more control over the behavior, but also much more code to write, test across platforms and maintain.
• I value not having many abstractions/runtimes in between me and the OS API: it’s nice to be able to just spawn a thread like every other application rather than learn how JS does it
• I am not a productive C++ developer yet. Segfaults are a poor feedback loop. I’m scared of how much of C++ is unchecked statically.
• Isolate is a relatively simple application. Unlike massive 3d applications, it has a few one off components with custom interactions and design. I’m not sure I see the value of another mental model over the underlying OS Api. Instead I could write an OSX and Windows version with maybe as much work.

I like the idea of SDL or equivalent IMGUI. I’d like to try it for a project; It’s slower to develop in, but I find it really enjoyable to work from the ground up and make all my own abstractions.

I stopped after an initial exploration. I don’t want to spend my time building Isolate I just want it to exist, and realistically I’m not going to invest in the development effort involved with this approach.

Even I recognized I’d be better off just living with a few images popping and actually spend time art’ing.

Background daemon, communicating over http

Instead of rebuilding the ui, I tried leaving the ui in Javascript and deferring computational work to a daemon process (which could be any language).

In this prototype, the application execs another binary on startup. The ui communicates with the background daemon locally over http, looking much like a typical client/server webapp.

Thoughts:

• Very quick to prototype, and easy to extend info a functional application. Once I was in Go, all the concurrency, thumbnailing and database interaction was a breeze. But here I get to keep the ease of development everywhere else.
• Managing the lifecycle of an extra process was tricky. I had to take care with startup ordering, and handling multiple instances running/daemon terminations. Originally the ui was nonfunctional without the daemon, but I made it optional to make the app more reliable.
• Security: previously user interaction is the only interface, but now an unauthenticated http server is listening to requests locally. I’ve just been careful to limit the interface to the daemon listing files, not taking action.
• Packaging: with osx/app images the package is just a directory and can include more than one binary. On Windows, I’d need the user to move a folder with both binaries. I overvalued the simplicity of install, but eventually decided this is okay.
• I tried Go because of ease of cross compilation, ease of concurrency and my experience from work. After forking the daemon I saw roughly 2-4ms of overhead on requests.

Background workers in Electron

It would have been prudent to check Javascript is too slow before assuming it. As I understand it, a common pattern in interpreted languages like Python, Ruby, Javascript (Node), is to load a compiled dynamic library and expose it so it can be called like any other module in the host language. In this way, packages exist for Javascript to interact with sqlite and libvips’s compiled C/C++ code, which turns out to as fast as the compiled daemon.

So I tried:

• Service/web workers, the way of having background threads in webapps. Neither of which have access to Node/Electron’s APIs making reading the original images not possible?
• Node’s cluster module to fork a duplicate of the Electron renderer process, but it crashed on startup because the mismatch between Node and Electron’s flags.
• Node workers, but these are still experimental and not available unless Node is explicitly compiled with them.
• There’s a pattern in Electron to start a second renderer process with a hidden window and use IPC as a way of getting a background process. Doing this worked, but the app was obviously sluggish when thumbnailing.

Thoughts:

• Even if I could get it working, communication with workers was challenging to get right. I’m not sure I understand the lifecycle with service workers.
• I thought packaging would be easier in all JS, but native modules need to be compiled for the version of Node Electron is using. This complicates the dev and production build. I’m using a lot of tools that work like magic (webpack, electron-builder), and I’d like to avoid having to support an abnormal usage like I did with bs-platform.

Back to the daemon

It’s fair to say all these approaches are workable, and perhaps I signed some off too early. Ultimately, I’d really like Isolate to be as low maintenance as possible, and the daemon seemed to suit that well.

I still really like Electron; some tasks a lot faster to do with an interactive gui, and it’s really powerful to quickly build tools for myself. As much vitriol as there is around it, I’d love to hear more/look into about how people construct large apps that run well like VSCode and Discord.