Pixel-perfect mouse picking with mousePick. Fifty mixed primitives — boxes, spheres, torii, cones, cylinders — drawn at random positions and tagged with a unique integer id encoded as a CSS hex colour by tag. Each frame the scene renders twice: once into a 1×1 framebuffer with each shape filled by its id, then gl.readPixels returns whichever id sits under the cursor; the visible pass renders normally and lights up the hit. Cost is one extra geometry submission per frame regardless of object count, and the answer is exactly the rendered pixel — through the hole of a torus, behind a partially-occluding box, anywhere a real fragment landed.

Two passes, same geometry

// Pick pass — every object filled by its id, drawn into a 1×1 FBO
hitId = mousePick(() => {
  for (const m of models) {
    push()
    translate(m.position)
    fill(tag(m.id))
    drawShape(m)
    pop()
  }
})

// Render pass — same scene, real materials. Highlight the hit id.
for (const m of models) {
  push()
  translate(m.position)
  // ... lighting, materials, fill ...
  drawShape(m)
  pop()
}

mousePick(drawFn) is shorthand for colorPick(mouseX, mouseY, drawFn). Inside the callback, the library binds a 1×1 framebuffer, replaces the projection with a pick matrix that places (mouseX, mouseY) at the centre of that single pixel, forces noLights() / noStroke() / resetShader() so encoded integer colours reach the framebuffer unmodified, runs your draw, calls gl.readPixels(0, 0, 1, 1, ...), and decodes the four bytes into a 24-bit integer. Returns 0 on miss.

The pick pass is invisible — the FBO is never blitted to the canvas. No flicker, no ghost geometry, no setup ceremony beyond drawing the same scene with fill(tag(id)) instead of real materials.

Tagging objects

fill(tag(1))    // → fill('#010000')
fill(tag(2))    // → fill('#020000')
fill(tag(255))  // → fill('#ff0000')
fill(tag(256))  // → fill('#000100')   // R wraps, G picks up

tag(id) packs the integer into the three colour bytes — R = LSB, B = MSB. The result is a CSS hex string that p5’s fill() accepts regardless of the active colorMode(), because hex always parses as raw RGB bytes irrespective of the current normalisation. id 0 is reserved (decodes as background / miss); valid ids run from 1 to 2²⁴ − 1 ≈ 16.7 million — enough to give every object in any reasonable scene its own.

Strokes are excluded from the pick buffer by default. To make them hit-testable, tag them with the same id as the fill:

fill(tag(id)); stroke(tag(id))   // strokes now hit

Mixing fill and stroke ids on the same shape is a bug — hovering on the stroke would resolve to a different object than hovering on the fill of the same shape.

Pixel-perfect by construction

The pick pass renders the actual geometry, so the answer to “what’s under the cursor?” is whichever pixel was drawn last at that location after depth testing — exactly what the user sees. This means:

Hover through the hole of a torus and you pick the object visible behind it.
Hover the corner of a box that partially occludes a sphere → you get the box; one pixel over → you get the sphere.
Overlapping geometry resolves correctly because depth testing runs in the FBO too.

A CPU-side approximation can’t reproduce any of this without a real ray–mesh intersection per object. Color-ID picking gets it for free because the GPU has already solved the visibility problem the moment it rendered the scene.

Why GPU scales

Per-object proximity testing on the CPU costs a matrix-vector multiply and a 2D distance check per object. For ten clickable handles that’s nothing; for fifty random primitives it’s already heavier than the GPU pass; for five hundred it’s a clear win for the GPU.

The pick pass pays a fixed cost of one geometry submission plus one readPixels round-trip. The geometry submission scales with object count the same way the visible pass does — but with no fragment-side lighting or shading. The readPixels is a 4-byte sync regardless of scene complexity. In practice the GPU pass handles thousands of pickable objects without breaking a sweat.

The other thing the GPU pass gets right is which hit. CPU proximity returns a boolean per object — if the cursor is near two overlapping projected centres, both return true and the caller has to disambiguate. GPU color-ID returns a single id that’s already been depth-resolved to the foreground. No ambiguity, no tie-breaker logic.

When proximity is the right call

The GPU pass is the right default for scene picking. The exception is when you want the hit zone itself to be visible and tunable — UI handles, graph nodes, drag gizmos — where “click anywhere within this circle” is a feature, not a bug. That’s CPU proximity picking: the same size you pass to mouseHit is the size you draw with cross or bullsEye, so the user sees the radius they’re aiming at.

The two strategies coexist naturally in the same sketch — color-ID for the scene’s geometry, proximity for handles overlaid on top. Scene meshes want pixel-perfect picks; drag handles want generous hit zones with visible feedback.

References

p5.tree README — Picking — colorPick, mousePick, tag
WebGL Fundamentals — Picking — the canonical walkthrough of the colour-ID technique
gl.readPixels (MDN) — the GPU→CPU sync the pick pass relies on
p5.Framebuffer reference — what the 1×1 pick FBO is built on
CPU proximity picking — sibling strategy: cheap, gizmo-coupled, per-object
Three.js Raycaster — the third option: CPU ray–mesh intersection, more flexible but heavier than colour-ID
Camera interpolation — uses beginHUD and the p5.tree rendering vocabulary the same way

Two passes, same geometry#

Tagging objects#

Pixel-perfect by construction#

Why GPU scales#

When proximity is the right call#

References#