red character 'A' without the horizontal bar Achraf Kassioui

Physics-Based User Interface

Blog / User Interface,

Experiments in building interfaces with physics, where everything is simulated, UI and content alike.

Sticky Regions

A sticky floating control can be made with physics. During drag, the controller can be positioned freely by applying velocity to its physics body. On release, it is attracted back to its rest position.

The rest position is defined by a physics spring field. The visualization below shows the center of the field and the rectangular physics body of the controller.

In fact, the controller itself is made of two parts: a pad and a handle. They are attached with a physics joint. In a physics-based UI, grouping is made using joints across a flat hierarchy of bodies, instead of the nested parent-child relationships used by traditional UI frameworks.

The behavior can be fine-tuned by changing the damping of the physics body and the strength of the physics field. All motion is continuous and interruptible. The user can pick up the controller mid-flight and throw it around.

Different positions can be defined with different fields. Below we can see each field activating as the controller enters or leaves a region:

This behavior is reminiscent of picture-on-picture on iOS or the PencilKit palette. Apple has a project sample that uses UIKit Dynamics fields to implement sticky corners. It's possible that the current iOS PiP behavior was inspired by Apple's experiments with physics in UI. See my note on UIKit Dynamics.

The experiments above were made with SpriteKit physics engine, before I found out about Apple's example.

Zoomable UI

Below is a controller floating on top of a zoomable canvas. Both the UI layer and the content layer are physical spaces with simulated bodies:

To implement a zoomable canvas, either the content layer or the UI layer must visually scale. But physics engines don't like messing with scale. See my note on scaling physics.

The visualisation below shows how all physics bodies live in the same space, preserving their nominal scale. When the camera zooms out, "bringing with it" the controller, the physics body of the controller is still "down there" in physics space:

The physical canvas can handle multiple types of media, such as text, images, video, and particle effects: