3D Brush Painting in VR

The following 3D brush painting algorithm integrates depth control for VR experiences by leveraging p5.treegl’s parsePosition function. This allows users to paint dynamically in 3D space, using mouse input and a depth slider to place brush strokes with precision.

code

'use strict'

let color, depth, brush, escorzo = true, fallback = [], points = [], record

function preload() {
  loadJSON('cloud_500.json', json =>
    fallback = json.map(entry => ({
      worldPosition: createVector(entry.x, entry.y, entry.z),
      color: entry.color
    }))
  )
}

function setup() {
  createCanvas(600, 400, WEBGL)
  colorMode(RGB, 1)
  document.oncontextmenu = () => false
  points = [...fallback]
  const o = parsePosition(Tree.ORIGIN, { from: Tree.WORLD, to: Tree.SCREEN })
  depth = createSlider(0, 1, o.z, 0.001)
  depth.position(10, 10)
  depth.style('width', '580px')
  color = createColorPicker('#C7C08D')
  color.position(width - 70, 40)
  brush = sphereBrush
}

function draw() {
  (mouseY >= 30) && orbitControl()
  record && update()
  background('#222226')
  axes({ size: 50, bits: Tree.X | Tree.Y | Tree.Z })
  for (const point of points) {
    push()
    translate(point.worldPosition)
    brush(point)
    pop()
  }
}

function keyPressed() {
  key === 'c' && (points = [])
  key === 'f' && focus()
  key === 'r' && (record = !record)
  key === 's' && saveCloud()
}

function sphereBrush(point) {
  push()
  noStroke()
  fill(point.color)
  sphere(1)
  pop()
}

function update() {
  points.push({
    worldPosition: parsePosition([mouseX, mouseY, depth.value()], { from: Tree.SCREEN, to: Tree.WORLD }),
    color: color.color(),
  })
}

function focus() {
  const center = [0, 0, 0]
  const position = parsePosition()
  const up = parseDirection(Tree.j)
  camera(...position.array(), ...center, ...up.array())
  const o = parsePosition(Tree.ORIGIN, { from: Tree.WORLD, to: Tree.SCREEN })
  depth.value(o.z)
}

function saveCloud() {
  const data = points.map(point => ({
    x: point.worldPosition.x,
    y: point.worldPosition.y,
    z: point.worldPosition.z,
    color: [red(point.color) / 255, green(point.color) / 255, blue(point.color) / 255, alpha(point.color) / 255]
  }))
  saveJSON(data, 'custom_cloud.json')
}

Key Elements of the 3D Brush Algorithm

The primary functionality revolves around transforming user input from screen space to world space, enabling dynamic point placement. Using the parsePosition function, we map 2D screen coordinates and depth values into the 3D world, allowing the brush to paint accurately in VR space.

Depth Control with `parsePosition`

The depth slider controls the z-axis positioning by transforming screen space to world space. This is key to painting in depth and makes the VR brush interactive.

function update() {
  points.push({
    worldPosition: parsePosition([mouseX, mouseY, depth.value()], { from: Tree.SCREEN, to: Tree.WORLD }),
    color: color.color(),
  })
}

Setting the World Origin as the Focal Point

The focus function orients the camera towards the center of the 3D world without reposition it, using parseDirection(Tree.j) to align the camera’s up direction. By recalibrating the depth slider to reflect the z-coordinate of the world origin in screen space, it allows the user to precisely draw points passing through the world origin.

function focus() {
  const center = [0, 0, 0]
  const position = parsePosition()
  const up = parseDirection(Tree.j)
  camera(...position.array(), ...center, ...up.array())
  const o = parsePosition(Tree.ORIGIN, { from: Tree.WORLD, to: Tree.SCREEN })
  depth.value(o.z)
}

Brush Strokes in 3D

Each brush stroke is rendered in 3D, using sphereBrush to draw small spheres at the calculated positions.

function sphereBrush(point) {
  push()
  noStroke()
  fill(point.color)
  sphere(1)
  pop()
}

Camera Controls and Recording

Camera Navigation: Users can freely move around the scene using orbitControl.
Recording & Saving: The keyPressed function toggles recording, and users can save their custom point clouds as JSON.

p5.treegl API reference

For more details on parsePosition and the p5.treegl library, check out:

p5.treegl API Reference

Key Elements of the 3D Brush Algorithm#

Depth Control with parsePosition#

Setting the World Origin as the Focal Point#

Brush Strokes in 3D#

Camera Controls and Recording#

p5.treegl API reference#

Further reading#