Seena Burns

Recreating the OFFF by Night opening shot in Houdini

Thu, 20 Jun 2019 00:00:00 -0700

Dev log time. I’ve been back in Houdini again, mesmerized by the OFFF By Night 2016 title sequence.

There are a ton of amazing shots in here, but today I’m just looking at the opening shot. Director William Arnold and artist eparizi have some behind the scenes experiments, but no indication of to how (other than “sop solver”)

After a few experiments, here’s what I’ve gotten to:

Your browser appears to not support HTML5 video mp4.

I’m decently happy with this recreation, but would love to hear if anyone (esp the artists on it!) can break down the effect better.

The Breakdown

The setup is not that complicated: a bunch of thin grid geo repeated and passed into a SOP solver. A solver gives you the previous frame as input, and you modify how the next frame should look, the basis of simulations.

After the solver, the color and vertical position is applied. There is no shading here, separating the layers vertically is just to fix the z-fighting as they overlap.

SOP level node graph

Well the meat of what’s happening is in the solver itself.

Every frame, we’re sampling some random activating points (from the scatter node above, passed in as an input).
For each point in the stripes, we see if they are within some RADIUS of activating points. These points are now activated.
Activated are given a velocity proportional to their distance to the activating point. There’s a lot of extra logic to make sure the points don’t move back, and only the top line of the stripe moves, the bottom is static. Additionally, only affect points in the hemisphere above the activating point, so points aren’t pushed backwards.

With some drag this gives a neat bursting growth, and as long as the radius of affected points is roughly greater than the max distance travelled by each point, the layers give the allusion of not overlapping.

Solver node graph

Debug viz of velocities

And roughly the VEX involved:

// find_active_and_dir
int pchandle = pcopen(1, "P", @P, ch("radius"), chi("max_points"));

@group_active = 0;
if (pcnumfound(pchandle) > 0) {
    @group_active = 1;

    v@active_center = pcfilter(pchandle, "P"); // avg(centers);
}

// set_vel (only applied to active group)
if (v@active_center[2] > @P.z) {
    vector to_active_center = (@active_center - @P) * {1,0,1};
    v@dir = normalize(to_active_center) * -1 * {0.5, 0, 1};
    if (@dir[2] > 0) {
        @dir[2] *= -1;
    }

    float dist = distance(v@active_center, @P);
    float dist_scaling = `chs("../find_active_and_dir/radius")` - dist;
    v@vel = @dir * ch("vel_scale") * dist_scaling;
}

Your browser appears to not support HTML5 video mp4.

Motion test

Currently this overly uses above (greater z), like to determine the hemisphere. This breaks down if you rotate the points as well. This should be turned into forward, say with a dot product of an initial forward vector

What could be better

Unfortunately when you spend your time recreating something, it becomes more about copying exactly than just making something good in its own way.

I’m not 100% happy with the motion, and I think it’s still simpler than what happens in the real sequence. For example, I’m fundamentally lacking these interactions:

Delay: the bottom yellow stripe grows after the initial movement, but is at the center of impact

Expand: the red stripe grows outward after the inital burst, making it more growth-like and not like a rounded lump

I’m not convinced this is a differential curve growth like where I first saw shots from the title sequence, but the motion is definitely more entrancing than what I have.

Wrote some ARM

Wed, 17 Apr 2019 00:00:00 -0700

Over the weekend I wrote some ARM! Specifically a program to read two numbers from argv, parse, add, convert back to a string, and write back to stdout with a syscall.

I thought it would be peaceful in zen sort of way (it was not) and maybe be a little educational (it was).

This turned out to be a good exercise, going over reading and writing arrays to the stack, looping, function calls and syscalls.

Quick takeaways:

A program is really just a giant mutable array of bytes in memory and some mutable registers. A debugger is pretty useful here, and separately, it’s interesting to code is just data being executed
Compiling and running ARM on x86 (with qemu) demystified some questions I had about emulation and VMs
I overwrote all my stack frames trying to write an array because I got the stack direction wrong. Not recommended
But the stack seems stupid cheap (just add a register and copy, though not sure how expensive the copy is), vs running a whole bunch of allocator code + maybe a syscall

Full gist (quite brittle at points)

Thumbnailing in Isolate: Background workers in Electron

Wed, 02 Jan 2019 00:00:00 -0800

Isolate is a tool I built for myself to view and organize art reference/inspiration. It’s a power user tool; it does a few things very fast and well.

While scrolling quickly, images sometimes pop in after a delay, rather than being visible immediately as the page scrolls.

Your browser appears to not support HTML5 video mp4.

Is this worth fixing? Is it worth a third rewrite? Probably not. Talking to users, nobody has ever mentioned it and I maybe was better off spending my vacation drawing or doing something other than coding for 12 hours straight cause that’s what I was supposed to get a break from.

But here were are, and fundamentally the problem is Isolate is loading full sized images to display as thumbnails. Even reading locally from disk is not instant for a dozen images at say ~1mb each. With some judicious css to ensure the position of visible elements is not affected by offscreen images with unknown dimensions size means Chrome can intelligently lazily load these images, but the problem is still visible in these few milliseconds when scrolling.

Similar software will thumbnail the images, saving a resized small version of the image. But this is computationally expensive, and not something I’d want to do on the same thread handling UI interactions (thumbnailing my dir of 500 images takes ~20 seconds), else this goes from a problem of barely perceived rendering lag to this-app-is-slow-hold-on-I-have-a-rant-about-Electron.

Problem statement

To step back, I’ve so far built Isolate in Electron. I want it to be cross-platform because I draw/sculpt in Windows, and everything else in OSX. I wish this was not the case; even cross-platform frameworks require lots of testing/platform specific work.

I’d like images thumbnailed in a background thread upon opening a directory, and a small store with metadata like hash, dimensions and thumbnail for fast querying.

The open questions for me were:

I’ve heard Javascript is slow, is this too expensive for Javascript?
How to communicate with the background thread/process
How to package and manage the application lifecycle

I prototyped some different approaches. I learned a lot, much more than just reading about them. For the most part, are all workable. This is just my reasoning and experience with the different directions I tried:

C++ prototype (QT, then SDL with intent of imgui)
Electron with a compiled daemon communicating locally via http
background threads in Electron: service/web workers, Node cluster/worker, second renderer process

Cross platform fully C++ prototype (QT/SDL)

A task like the above problem statement is straightforward and routine in a reasonably performant language with concurrency, and I could just reimplement the whole app in such a language like C/C++.

I started off with a version in QT as a common recommendation and backbone of heavy apps like Houdini. Then, because Isolate is fairly simple, I tried SDL to build my own abstractions rather than diving too deep into how make QT do what I want.

Thoughts:

While the background worker is easier, I’d have to reimplement a lot of what I take for granted. E.g. layout, swapping thumbnails with full-sized images based on display size, application packaging, even establishing my own logging because I don’t have console.log. This means I have much more control over the behavior, but also much more code to write, test across platforms and maintain.
I value not having many abstractions/runtimes in between me and the OS API: it’s nice to be able to just spawn a thread like every other application rather than learn how JS does it
I am not a productive C++ developer yet. Segfaults are a poor feedback loop. I’m scared of how much of C++ is unchecked statically.
Isolate is a relatively simple application. Unlike massive 3d applications, it has a few one off components with custom interactions and design. I’m not sure I see the value of another mental model over the underlying OS Api. Instead I could write an OSX and Windows version with maybe as much work.

I like the idea of SDL or equivalent IMGUI. I’d like to try it for a project; It’s slower to develop in, but I find it really enjoyable to work from the ground up and make all my own abstractions.

I stopped after an initial exploration. I don’t want to spend my time building Isolate I just want it to exist, and realistically I’m not going to invest in the development effort involved with this approach.

Even I recognized I’d be better off just living with a few images popping and actually spend time art’ing.

Background daemon, communicating over http

Instead of rebuilding the ui, I tried leaving the ui in Javascript and deferring computational work to a daemon process (which could be any language).

In this prototype, the application execs another binary on startup. The ui communicates with the background daemon locally over http, looking much like a typical client/server webapp.

Thoughts:

Very quick to prototype, and easy to extend info a functional application. Once I was in Go, all the concurrency, thumbnailing and database interaction was a breeze. But here I get to keep the ease of development everywhere else.
Managing the lifecycle of an extra process was tricky. I had to take care with startup ordering, and handling multiple instances running/daemon terminations. Originally the ui was nonfunctional without the daemon, but I made it optional to make the app more reliable.
Security: previously user interaction is the only interface, but now an unauthenticated http server is listening to requests locally. I’ve just been careful to limit the interface to the daemon listing files, not taking action.
Packaging: with osx/app images the package is just a directory and can include more than one binary. On Windows, I’d need the user to move a folder with both binaries. I overvalued the simplicity of install, but eventually decided this is okay.
I tried Go because of ease of cross compilation, ease of concurrency and my experience from work. After forking the daemon I saw roughly 2-4ms of overhead on requests.

Background workers in Electron

It would have been prudent to check Javascript is too slow before assuming it. As I understand it, a common pattern in interpreted languages like Python, Ruby, Javascript (Node), is to load a compiled dynamic library and expose it so it can be called like any other module in the host language. In this way, packages exist for Javascript to interact with sqlite and libvips’s compiled C/C++ code, which turns out to as fast as the compiled daemon.

So I tried:

Service/web workers, the way of having background threads in webapps. Neither of which have access to Node/Electron’s APIs making reading the original images not possible?
Node’s cluster module to fork a duplicate of the Electron renderer process, but it crashed on startup because the mismatch between Node and Electron’s flags.
Node workers, but these are still experimental and not available unless Node is explicitly compiled with them.
There’s a pattern in Electron to start a second renderer process with a hidden window and use IPC as a way of getting a background process. Doing this worked, but the app was obviously sluggish when thumbnailing.

Thoughts:

Even if I could get it working, communication with workers was challenging to get right. I’m not sure I understand the lifecycle with service workers.
I thought packaging would be easier in all JS, but native modules need to be compiled for the version of Node Electron is using. This complicates the dev and production build. I’m using a lot of tools that work like magic (webpack, electron-builder), and I’d like to avoid having to support an abnormal usage like I did with bs-platform.

Back to the daemon

It’s fair to say all these approaches are workable, and perhaps I signed some off too early. Ultimately, I’d really like Isolate to be as low maintenance as possible, and the daemon seemed to suit that well.

I still really like Electron; some tasks a lot faster to do with an interactive gui, and it’s really powerful to quickly build tools for myself. As much vitriol as there is around it, I’d love to hear more/look into about how people construct large apps that run well like VSCode and Discord.

Lessons on exec from 4:40pm on a friday

Sat, 20 Oct 2018 00:00:00 -0700

A few Fridays back I got a message from a friend:

also puzzler for you:

$ ls -la /path/to/my/executable
-r-xr-xr-x 1 root root 6559032 Jan 1 1970 /path/to/my/executable

$ /path/to/my/executable
-bash: /path/to/my/executable: No such file or directory

After a few sad guesses, I asked my coworker and got:

Oh, it’s a dynamically linked file missing the loader

What

When you execute a (dynamically linked) executable, it’s not actually executing the file, the kernel runs the loader. It’s not in the right path so it errors. Here

Okay, but first, context! Before, when we talked about building C programs, we saw when compiling a binary, you can package all object code together in one file (static linked) or distribute libraries/modules independently in shared object archives that are installed separately and loaded at runtime (dynamic linking). When a dynamically executed file is started, it relies on a linker/loaded to load all the shared object files into memory before executing.

Now you’re on the same page as I was: something’s going wrong with loading, but why file not found? It’s the kernel doing all this? How does this all happen?

The demo went something like:

$ docker run -it --rm ubuntu

$ ldd /bin/mv
  ...
	/lib64/ld-linux-x86-64.so.2
  ...

$ apt-get update
$ apt-get install file
$ file /bin/mv
/bin/mv: ELF 64-bit LSB shared object
  x86-64, version 1 (SYSV)
  dynamically linked
  interpreter /lib64/ld-linux-x86-64.so.2
  for GNU/Linux 3.2.0
  ...

# Or with readelf
$ apt-get install binutils
$ readelf /bin/mv -l
Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  ...
  INTERP         0x0000000000000238 0x0000000000000238 0x0000000000000238
                 0x000000000000001c 0x000000000000001c  R      0x1
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  ...

/bin/mv is dynamically linked with /lib64/ld-linux-x86-64.so.2 as the program interpreter. You can run programs with it directly too!

$ /lib64/ld-linux-x86-64.so.2 /bin/ls -al /bin/ls
-rwxr-xr-x 1 root root 133792 Jan 18  2018 /bin/ls

But when it’s not located on the right path, the kernel errors trying to load it.

$ mv /lib64/ld-linux-x86-64.so.2 /tmp/

$ ls
bash: /bin/ls: No such file or directory

$ mv /tmp/ld-linux-x86-64.so.2 /lib64/
bash: /bin/mv: No such file or directory

# uh oh

$ /tmp/ld-linux-x86-64.so.2 /bin/mv /tmp/ld-linux-x86-64.so.2 /lib64/
# Things are okay now

We’ve reproduced our problem! And so ended the workweek.

ld-linux

And thus began weekend reading! To start with, ld-linux(8) points out:

dynamic linker can be run either indirectly by running some dynamically linked program or library (e.g. ELF binaries in the PT_INTERP program header) or directly by running: /lib/ld-linux.so.* [OPTIONS] [PROGRAM [ARGUMENTS]]

I think being able to run directly is for convenience, but hey we saw that.
Also

ld.so and ld-linux.so* find and load the shared libraries needed by a program, prepare the program to run, and then run it.

This hints at how this is happening, namely that ld-linux is actually running the program in addition to loading shared libraries

How programs get run

lwn.net has some pretty good articles on what happens when a program is exec’d:

Namely, a number of executable formats are defined as instances of struct linux_binfmt, the most interesting to me being:

binfmt_script: files that start with #!
binfmt_misc: dynamically defined formats in /proc/sys/fs/binfmt_misc, which Jessie Frazelle dove into to script go, and Java uses to exec .jar files
binfmt_elf: ELF binaries, standard Linux binary format

After some initialization, the kernel iterates over each format, having each attempt to load the binary file, and returning the first that successfully recognizes it without returning ENOEXEC.

For example, binfmt_script matches on the #! at the start, parses the interpreter in the first line, and recursively execs on it (which then matches to another binfmt). The original file is passed in as argv[0] like:

$ cat args.sh
#! /bin/bash
echo "args: $*"

$ cat test
#! args.sh

$ ./test
args: ./test

You could write a much more interesting interpreter like say, python, that actually uses the given file.

Additionally, fun facts:

Because BINPRM_BUF_SIZE is set to 128, the #! interpreter will fail to match an interpreter path longer than 128 bytes (minus like 3 or 4 bytes)
We just recursed into another binfmt_script executable. You can’t do this forever, exec returns ELOOP after 4 levels: -bash: ./5: 4: bad interpreter: Too many levels of symbolic links

But from our demo, we were dealing with binfmt_elf which does a lot more than binfmt_script.

The lwn article covers all the heavy lifting to load and initialize the program and kernel state, but here I’ll focus on the dynamic loading.

If the PT_INTERP program header is present, the interpreter is itself loaded into memory as well. The entry point is replaced with the interpreter’s, and execution begins with the interpreter (the linker here) loading shared libraries and resolving symbols in userspace. After which, the interpreter jumps to the original entry point, and actual execution begins.

Why did this all happen

My friend, being the Nix enthusiast that he is, has all sorts of interesting puzzlers. Nix wants to load it’s own Glibc, and programs must be either compiled with a correct ELF interpreter or patched afterwards with patchelf, which rewrites the interpreter header.

I believe this was a program compiled with Nix, installed into a regular, Docker-built container (so only the regular /lib/ld-linux is present) and boom, thus arises a fun problem most developers won’t ever have to deal with.

Migrating Isolate to ReasonML + React

Mon, 02 Jul 2018 00:00:00 -0700

I’ve just finished moving Isolate to ReasonML and React. Today’s post is about that.

Now, I’m no frontend dev. I spend my time at work thinking about systems and infrastructure, migrating things, building incremental value while working towards some far off vision.

Then I come home and work on Isolate, an app I’ve made for myself, to managing art reference and inspiration. Honestly, I just want to have it, not to build it. None of this thinking hard, following best practices, whatever. I’ve been playing it fast and loose with global variables, redundant code, unclear interfaces. And what do you know I’ve found myself in the odd position of having incurred technical debt in a side project.

One day, when all the problems are solved and there’s nothing left to do, I’ll move this to QT. But for now, I just wanted to clean things up a bit and move back to adding some features.

TL;DR

React: Great!

Reason: Great!

React

Imposes code structure

As I worked, I broke up code into logically grouped modules. This is nice for library-esque modules like path, but did little to organize the application as a whole. Largely it ended up with code reaching across interfaces and no real abstraction.

React imposed some much needed code structure. In React, you define components, essentially functions that define how to construct some HTML. An application is built by composing these components in a hierarchy: a component can render html, or nest calls to other components.

React automagically handles re-drawing components when the arguments (“props”) of the components change, offloading the significant headache of managing how to subscribe to changes, or updating dependents.

This is also really nice because it’s a lot easier to reason about components in isolation. All the dependencies are described as props (and typed with Reason!).

Modal.re is a good example, here it only depends on State.Modal.state and an action to update the modal state State.Modal.action => unit and then defines how to render the HTML given the current state. This is compared to before where the modal module described how to mutate the modal DOM object. You can probably have written a lot better javascript than that without React, but it really pushes down a good structure.

State

I haven’t found a good way to have sibling components affect each other (probably good I haven’t), and so React forced me to accept what is shared state. E.g. if the header contents depends on if there is an active search, that’s not search-local state, it’s either header-local or global. Because Isolate has relatively little state, I’ve just been pushing it up to a global state if it’s shared.

ReasonReact includes a mini Redux model, where a stateful component includes a function (action, state) => state that pattern matches on the possible actions and defines how to update the state.

Unfortunately I’m still struggling to find a good way of composing the reducer function. ReasonReact encourages putting a lot into it, but my main reducer is getting a bit excessive.

Declarative

Previously, the UI would be imperatively toggled, which scaled abysmally. Clicking on an image would trigger adding display: block on the modal container, updating a bunch of fields in the process. New UI elements would each mutate each other’s DOM element, and the DOM became this tangled implicit, global state.

In React, you describe what should be rendered given a component’s props, not how to get there. With JSX (inlining HTML in Javascript), this is incredibly convenient.

All together, previously searching would hide/show elements in the header like:

function search(term) {
  ...
  hide(ui.pwd)
  hide(ui.dirs)
  show(document.querySelector('#search-controls'))
}

document.querySelector('#search-controls a').onclick =
  e => {
    hide(document.querySelector('#search-controls'))
    show(ui.pwd)
    show(ui.dirs)
    ...
  }

Notably, every possible path to updating state would have to update the DOM. Now, instead, the header describes how to be rendered:

render: self => {
  ...
  <header>
    <Search ... />
    (if (!self.state.search)) {
      <div>
        <h3> (ReasonReact.string(pwd)) </h3>
        <Directories ... />
      </div>;
    } else {
      ReasonReact.null;
    })
  </header>;
  ...
}

ReasonML

Reason (OCaml) feels like a really tight, consistent language. The docs are great, and focus on motivating the switching from Javascript.

Having types is wonderful; adding types for relative and absolute filepaths saved me so many bugs. Refactoring is much saner when you repeatedly compile and fix errors until the build succeeds.

Reason does a really good job with Javascript interop. It’s easy to inject raw Javascript, and wrap it in types. The output of Bucklescript (Reason compiler) is just a (surprisingly readable) transpiled javascript file, so it’s easy to include in a Javascript file for the other way around.

Most of Isolate is the UI, so I haven’t gone too deep into Reason. To keep it pragmatic, it says a lot that it added value without getting in my way. If that’s not convincing, I will say I look forward to using it again.

Troubles

Briefly, the pains I faced were:

ReasonML’s compiler errors are very vague. These two errors comprise almost all the errors I ever saw:

File "/Users/seena/Developer/isolate/src/Main.re", line 14, characters 0-1:
Error: 3724: <UNKNOWN SYNTAX ERROR>

This has type:
  string
But somewhere wanted:
  Isolate.Path.base (defined as Isolate.Path.base)

Mutating Javascript value from Reason.

I couldn’t for the life of me find a way to set document.onclick to a function defined in Reason from Reason.

Writing To The Framebuffer

Wed, 04 Apr 2018 00:00:00 -0700

I’ve been thinking about cross platform GUIs, and one question led to the next and eventually I was asking: how do pixels end up on the monitor? A pretty neat stack overflow response answers:

The image displayed on the monitor is stored in your computer’s video RAM on the graphics card in a structure called a framebuffer.

The data in the video RAM can be generated by the GPU or by the CPU. The video RAM is continuously read out by a specialized DMA component on the video card and sent to the monitor. The signal output to the monitor is either an analog signal (VGA) where the color components are sent through digital to analog converters before leaving the card, or a digital signal in the case of HDMI or DVI.

Now, you probably realize that for a 1920x1080 display with 4 bytes per pixel, you only need about 8 MB to store the image, but the video RAM in your computer is probably many times that size. This is because the video RAM is not only intended for storing the framebuffer. The video RAM is directly connected to the GPU, a special purpose processor designed for efficient 3D rendering and video decoding.

Which is super informative, and also explains why my monitors are directly plugged into my GPU via DVI. Another answer I found said no modern operating system will let you access the framebuffer directly. BUT LUCKILY THAT’S WRONG. With another stackexchange tip, it’s as easy as:

> cat /dev/urandom > /dev/fb0
-bash: /dev/fb0: Permission denied
> sudo cat /dev/urandom > /dev/fb0
-bash: /dev/fb0: Permission denied
> ls -al /dev/fb0
crw-rw---- 1 root video 29, 0 Apr  4 00:21 /dev/fb0
> sudo adduser seena video
> cat /dev/urandom > /dev/fb0

Glossing over why sudo would mysteriously not work, this writes a bunch of noise out to the monitor:

Awesome! The kernel framebuffer docs have some general info, but the gist is /dev/fb0 acts like any other memory device in /dev and we can write to it like a file. Since it’s just a memory buffer, any write will overwrite the existing value, which you can see when the console prompt immeidately overwrites the noise.

Also, you can even read from the framebuffer, taking a screenshot with cp /dev/fb0 screenshot which would have been helpful before I took a bunch of pictures with my phone.

Gradients

Alright, so the stackexchange answer also gives a script to write out a red box, but I wanted to take a shot at a gradient. After permuting the following lines a bunch of times, I settled on: the monitor is written to one horizontal line at a time, with one byte for Blue,Green,Red,Alpha(?) per pixel (seems like 24-bit true color).

xsize = 1920
ysize = 1200
with open('data.out', 'wb') as f:
  for y in range(0,ysize):
    for x in range(0,xsize):
      r = int(min(x / (xsize/256),255))
      g = int(min(y / (ysize/256),255))
      b = 0
      f.write((b).to_bytes(1, byteorder='little'))
      f.write((g).to_bytes(1, byteorder='little'))
      f.write((r).to_bytes(1, byteorder='little'))
      f.write((0).to_bytes(1, byteorder='little'))

Text

I’ve been running this from the linux console (rather than x server or another display server), which I believe is just rendering text and basic colors to the framebuffer. I should be able to do that too. I grabbed Lemon, a bitmap font I used to use. Bitmap fonts describe characters as a grid of bits, which can be easily rendered without much thought about the complexity/horrors of text rendering, aliasing and the likes.

I started a janky reader for BDF files, and then started to regret this idea and my python script. Nevertheless, I produced a single “lock” icon from the font, drawn to the framebuffer! This could be extended to write out a text string, but again, it was really janky.

Images

Okay so at this point, it’s not much of a stretch to render out an image to the framebuffer. We’ve already got a smooth color gradient and an image is just a file format we have to read pixel data to and encode/dump to the framebuffer. I exported an image as a TGA file (just an uncompressed format for easy parsing, also tried BMP), and used pyTGA to write the image straight out. What’s neat is you can use seek to skip over pixels, rather than having offset write out black pixels.

Here’s the little TGA renderer python script:

import sys
import os
import tga

if len(sys.argv) < 4:
  print("Usage: tga.py xsize xoff yoff input output")
  sys.exit(1)

g_xsize = int(sys.argv[1])
g_xoff = int(sys.argv[2])
g_yoff = int(sys.argv[3])
g_input = sys.argv[4]
g_output = sys.argv[5]

img = tga.Image()
img.load(g_input)
header = img._header

with open(g_output, 'wb') as f:
  f.seek(g_yoff * 4 * g_xsize, os.SEEK_CUR)
  for y in range(header.image_height):
    f.seek(g_xoff * 4, os.SEEK_CUR)

    for x in range(header.image_width):
      p = img.get_pixel(y,x)
      f.write(p[2].to_bytes(1, byteorder='little'))
      f.write(p[1].to_bytes(1, byteorder='little'))
      f.write(p[0].to_bytes(1, byteorder='little'))
      f.write((0).to_bytes(1, byteorder='little'))

    f.seek((g_xsize - header.image_width - g_xoff) * 4, os.SEEK_CUR)

# Running in bash
> python3 tga.py 1920 750 300 test.tga /dev/fb0

It works! Image rendered straight to the framebuffer

Framebuffer Extras

This has been super useful in connecting a bunch of loose threads I’ve had in my mind. It explains, for example, why the final step of OpenGL is rendering out to a framebuffer, and why double buffering (rendering to a hidden buffer, then swapping) helps (especially given how slow my python program is).

On the GUI side, From what I’ve found, a lot of custom GUI engines mostly just blit together bitmaps, and X11, along with many others (“MPlayer, links2, Netsurf, fbida,[2] and fim[3] and libraries such as GGI, SDL, GTK+, and Qt”) render out to the framebuffer. Some university classes even use Raspberry PIs to mess around with the framebuffer, and there’s some useful reading there because the architecture is very straightforward.

Quad Remapping

Mon, 02 Apr 2018 00:00:00 -0700

Last week I finished with road geometry in Houdini. This alone is pretty bland; I want to be able to map geometry along the road. Specifically, I want to take an arbitrary input quad (four points that form a box) with whatever geometry inside the bounds, and map it to a target quad, preserving the relative positions of the inner geometry.

Input shown, target quad outlined

This really smells like a change of basis to me. If I could express the initial corner positions, and the target positions as vector spaces, I would just need to find the single transformation matrix that would take any point from the intial space to the target space. Here’s where I found out how rusty my linear algebra is.

I sort of got somewhere by expressing a 2d space, with the basis vectors being along the diagonals of the quad. Then any point could be expressed as a linear combination of these orthogonal axes. Remapping is using the same linear combination with the new basis vectors. It worked for corners but lots of distortion around points off the basis vectors.

Remapping a starting grid, intermediate points distorted

Bummer. This is really haunting me, because I’d really like to believe it’s possible and I’m setting it up wrong, but I also question if my example transformation - translating a single corner - is even a linear transformation.

At some point I threw out that idea and started to think that instead of diagonals I could take two parallel sides of the initial quad, and express each point as a percentage along each line, and then a percentage between those two points.

I got a hunch that maybe, like a bezier curve, I could identify a point’s relative position by computing the percentage along two of the parallel sides of the initial quad, getting two P0 and P1. Then I could compute the percentage between P0 and P1. Remapping would involve using the same percentages, but with those parallel sides transformed to the target. This stackexchange answer backed up my hunch, but generalized to more complex polygons.

Then I realized, textures do this everyday! UV mapping takes a 2d input texture (which is just a quad) and maps it to some target geometry. Here it’s a simpler problem, because my target isn’t some complex polygon; UV mapping should easily handle texturing a quad. Turns out, it’s the same technique, but much more formally described: Bilinear interpolation! Finally, the y/vertical axis just needs to be offset along the target quad normal, rather than the world up.

Bilinear interpolation for grid

Original input, repeatedly remapped along a poly strip.

Generating road from a curve in Houdini

Mon, 26 Mar 2018 00:00:00 -0700

Inn the car controller post, you can see I’ve been living the Kenny asset life. I’m a huge fan of being able to focus on prototyping the gameplay.

Moving forward I’d like to be able to make content (race tracks) quickly, so today I’m going to talk about the system I made to transform an arbitrary curve in space into drivable geometry.

Process

I started a curve made with Houdini’s builtin curve SOP, which uses given points as anchors for a Bezier curve (or other curve types). Unfortunately, the curve tool is pretty clunky compared to others, and it might be easier to define elsewhere and bring in as a series of points.

Input curve

Next I sample points along the curve. Corners with tight curvature need a much higher resolution to get smooth geometry, whereas straights can be a rectangle, so I used the facet SOP to consolidate colinear points. Also I only want the point data, so I chopped out the line geometry.

Resampled curve

To transform the curve into a surface, I originally had tried sampling lines an using the sweep SOP, but this twisted the surface uncontrollably when working in all 3 axes. Instead, I now (for each point:

Get a vector diffV along the curve. For a given point i, this can be done by averaging the differences between i-1,i and i,i+1.
Generate a direction vector crossDir, orthogonal to the road. I can get this with diffV crossed with the road surface’s normal (world up since the road shouldn’t twist)
Place new points at pos(point_i) +/- crossDif * width
With the points in place, I can create triangles between every 3 points. I originally did this manually, but the poly patch SOP does this for me.

Points and lines orthogonal to the road, which form the road geometry when connected

And it becomes a drivable surface! But I’d like to make this surface form ramps with walls, not just a paper thin surface. So I follow up by extruding downwards, creating a bottom face. I do another pass, projecting this bottom face onto the XZ ground plane.

Extruding and projecting onto ground

From here, it’s just some optimizations before exporting. I remove the faces on the ground, and weld close points (mainly to clean up the extruded walls where the road is on the ground already), and create smoothing groups.

Banking

Unfortunately when I tried driving it in game, I had all sorts of problems. Collision detection really struggled with the triangulation, but adding a suspension instead of a box collider fixed most of the jank.

Still, curves that move in 3 axes at once were problems. The car handles poorly on them, usually flying off. And, once triangulated, the road would form “walls” because the inner side would rise more steeply than the outer side (the Y/XZ ratio is greater).

Wall artifacts

One solution to both is to try to bank the turns based on the tightness of the turn. I tried banking by a function of the Y/XZ ratio (an estimate of curvature), which I smoothed over several points. Unfortunately, I didn’t have a great way to twist the surface, so it ended up just lowering/smoothing out the ramps instead of banking. It cleaned the walls but made for a dull road.

My second approach was limiting the amount of climb, e.g. for every unit of XZ, the road can at most climb some amount on the Y axis. Critically, I measure this value separately on the inner and outer edges, so if the outer edge travels further, it can climb more.

This means each point is a function of the previous point. In Houdini each attribute wrangle can only access the attributes of others points before execution, so an iteration like this requires a single pass over the whole object, tracking points in an array manually iterating.

Banking. Top image shows the difference between the original points and the banked result

There are still a few artifacts that can be cleaned up, but you can see the surface banks a lot for the tight turn and the walls have be substantially smoothed.

In Game

Isolate

Sun, 25 Mar 2018 00:00:00 -0700

Released Isolate today, a snappy image browser I’ve been working on and using for a few months. The full source and prebuilt binaries is on Github.

Your browser appears to not support HTML5 video mp4.

FZF.el: emacs + fzf

Tue, 06 Mar 2018 00:00:00 -0800

Ripgrep and FZF in Emacs

One of the things I’ve missed in the Vim to Emacs migration is solid recursive fuzzy file search. I tried bling/fzf.el, and ended up spending the weekend working on it.

Rather than just work with fuzzy file search, I got way too into the possibility of using fzf as a general completion engine. Fzf.el already had a decent way of invoking fzf (spawn a terminal buffer and shell out) but the real potential is in the programmability.

My code/fork exposes fzf by saying given a list and a function, the user can select an item with FZF and it will be passed to the function. It becomes trivial to run your own actions:

; Example
(fzf-with-entries (list "a" "b" "c") 'print)

; Open recent file
(defun fzf-recentf ()
  (interactive)
  (fzf-with-entries recentf-list
    (lambda (f) (when (file-exists-p f) (find-file f))))
)

; Recursive fuzzy file search
; (resolve-directory will use the projectile root if set)
(defun fzf-find-file (&optional directory)
  (interactive)
  (let ((d (fzf/resolve-directory directory)))
    (fzf-base
    (lambda (x)
        (let ((f (expand-file-name x d)))
        (when (file-exists-p f)
            (find-file f))))
    d)))