Taos Engine ▦ Taos: API Documentation

4 — Anchors & Picking

▶ Live demo: run this tutorial in your browser — built from src/04_anchors_and_picking.ts.

Two complementary skills: putting your content onto the globe at a precise lat/lon (anchors — 3D models, plain meshes, and screen-space text labels), and finding out what's there — the geodetic location under the cursor, or the specific building feature you clicked (picking). Sources: src/geo/anchor.ts, samples/geo_labels.ts, samples/geo_photo.ts, src/geo/pick_feature.ts.

Anchoring a glTF model#

loadAnchor loads a glTF/glb, places it at a geodetic position oriented to the local surface, and returns a TileContent you hand to geo.addStatic:

import { loadAnchor } from 'taos/geo/index.js';

const content = await loadAnchor(device, geo.frame, '/models/duck.glb', {
  lonDeg: -122.4194,
  latDeg: 37.7749,
  height: 50,          // meters above the WGS84 ellipsoid
  scale: 10,           // uniform scale on the model
  headingDeg: 45,      // rotation about local up; 0 = model faces north
});
geo.addStatic(content);

The model stands radially on the surface (its up = the ellipsoid normal at that lat/lon) and headingDeg spins it about that up axis. addStatic registers it as one-off baked content that streams alongside the tilesets; geo.removeStatic(content) takes it back out.

For models on another body (the Moon, say), pass bodyCenter (the body's ECEF position) and ellipsoid ({ a, b } radii) so the up vector is computed about that body, not Earth — see tutorial 10.

Clamping to terrain#

height is above the ellipsoid, not above the ground. To sit a model on the streamed terrain, query the terrain height first and add your offset:

import { DEG2RAD } from 'taos/geo/index.js';

const ground = geo.heightAt(lonDeg * DEG2RAD, latDeg * DEG2RAD);  // null if not loaded yet
if (ground !== null) {
  const content = await loadAnchor(device, geo.frame, url, { lonDeg, latDeg, height: ground, scale });
  geo.addStatic(content);
}

heightAt returns null until the terrain tile covering that point has streamed in, so either retry across frames or place the model once terrain is ready.

Placing a plain GameObject#

If you don't need loadAnchor's glTF handling — e.g. a billboard or a marker mesh — convert the lat/lon to world space yourself (the chain from tutorial 2) and use the normal scene API:

import { lonLatToEcef } from 'taos/geo/index.js';

const world = geo.frame.worldFromEcefPoint(lonLatToEcef(lonDeg, latDeg, height));
markerGO.setPosition(world.x, world.y, world.z);
engine.scene.add(markerGO);

Remember to re-derive the world position after a reanchor if the marker must stay put — the world origin moved underneath it.

Text labels & POI markers#

A 3D mesh marker is one way to mark a place; a screen-space text label is the other. GeoLabelFeature projects each label's ECEF anchor into screen space every frame, lays the text out at a constant on-screen size, and declutters by priority so overlapping labels drop out gracefully. The keyless way to drive it is testLabels (lon/lat points):

import { GeoLabelFeature } from 'taos/geo/geo_label_feature.js';   // direct subpath
import { TonemapFeature } from 'taos/engine/index.js';
import { POI_ICON } from 'taos/geo/poi_icon_data.js';   // direct subpath

const icon = (c: number) => String.fromCodePoint(c);

engine.addFeatureBefore(new GeoLabelFeature({
  scene: { labels: [], frame: geo.frame },   // no streamed labels → falls back to testLabels
  testLabels: [
    { text: `${icon(POI_ICON.star)} One World Trade`, lonDeg: -74.0134, latDeg: 40.7127, rank: 0 },
    { text: `${icon(POI_ICON.tree)} Battery Park`,     lonDeg: -74.0170, latDeg: 40.7033, rank: 2 },
  ],
  pixelSize: 17,
  distanceFade: { near: 600, far: 4000, minOpacity: 0.15 },   // thin out the far ones
}), TonemapFeature.name);

It registers before tonemap (it chains frame.hdr). A leading POI icon glyph — the private-use code points in POI_ICON — is tinted by category, so the star/tree/cup markers read as colored icons rather than plain text. Lower rank = kept first when labels collide; distanceFade fades distant labels by camera range.

Beyond testLabels, the scene option takes any { labels, frame } source: a baked GeoJSON layer's layer.labels(...) (tutorial 7), or a GeoScene directly (its streamed vector-tile place/street/POI labels). Pair the labels with the 3D cone markers from above for a full annotation layer — geo_labels.ts does exactly that over keyless terrain.

Picking 1: screen ray → geodetic location#

"What lat/lon is under the mouse?" Build a world-space ray from the click, march it to a distance (or intersect terrain), then convert the hit back through the frame. This is the core of geo_photo.ts:

import { ecefToGeodetic, RAD2DEG } from 'taos/geo/index.js';
import { Vec3 } from 'taos/math/index.js';

function screenToLonLat(sx: number, sy: number, distMeters: number) {
  const m = cameraGO.localToWorld().data;
  const camPos = new Vec3(m[12], m[13], m[14]);

  // NDC → camera-space ray (camera looks down -Z), rotated into world by the basis.
  const tanV = Math.tan(FOV_Y / 2);
  const tanH = tanV * (engine.ctx.width / engine.ctx.height);
  const ndcx = 2 * sx - 1, ndcy = 1 - 2 * sy;          // sx,sy in [0,1]
  const dx = ndcx * tanH, dy = ndcy * tanV, dz = -1;
  const wx = m[0] * dx + m[4] * dy + m[8] * dz;
  const wy = m[1] * dx + m[5] * dy + m[9] * dz;
  const wz = m[2] * dx + m[6] * dy + m[10] * dz;
  const len = Math.hypot(wx, wy, wz) || 1;

  const hit = new Vec3(
    camPos.x + (wx / len) * distMeters,
    camPos.y + (wy / len) * distMeters,
    camPos.z + (wz / len) * distMeters);

  const g = ecefToGeodetic(geo.frame.ecefFromWorldPoint(hit));
  return { lonDeg: g.lonRad * RAD2DEG, latDeg: g.latRad * RAD2DEG, height: g.height };
}

The key move is the last line: ecefFromWorldPoint lifts the f32 world hit back into f64 ECEF, and ecefToGeodetic turns that into lon/lat/height. (Marching a fixed distance is the simplest version; intersecting terrain or the ellipsoid for the exact ground point follows the same convert-the-hit pattern.)

Picking 2: feature metadata → which building?#

This second technique needs a 3D-Tiles dataset that carries feature tables (b3dm batch tables / EXT_structural_metadata) — e.g. the Cesium OSM Buildings tileset via ion. The keyless OpenFreeMap vector-tile buildings this tutorial streams don't carry that per-feature metadata, so swap the buildings layer for the ion 3D Tiles (see tutorial 3) to follow along here. The geometric "Picking 1" above works with any source.

3D Tiles carry per-feature metadata (every building footprint has properties: height, name, address…). pickFeature ray-tests against pickable meshes and returns which feature you hit, not just a triangle. From pick_feature.ts:

import { pickFeature } from 'taos/geo/pick_feature.js';   // not in the barrel — direct subpath

// `meshes` are PickableMesh: { positions, indices, featureIds, tableIndex? }
// sourced from the streamed tile contents (TileContent.featureTables).
const hit = pickFeature(
  [origin.x, origin.y, origin.z],    // ray origin, world space
  [dir.x, dir.y, dir.z],             // ray direction (unit length)
  meshes,
);

if (hit) {
  // hit.meshIndex, hit.featureId, hit.tableIndex, hit.t, hit.position
  const table = meshes[hit.meshIndex];              // → tile's featureTables[hit.tableIndex]
  console.log('feature', hit.featureId, 'at', hit.position);
  // Look up hit.featureId in that feature table to read its properties.
}

pickFeature returns a FeaturePick ({ meshIndex, featureId, tableIndex, t, position }) or null for a miss. The featureId is the key into the tile's feature table, where the metadata lives. Highlighting the picked feature is then a vertex-color edit on that feature's triangles — the same mechanism tutorial 7's GeoJSON setFeatureColor uses.

Building the PickableMesh list from streamed tiles (collecting positions, indices, and feature ids out of GeoFrameResult.tileContents) is the plumbing part; geo_photo.ts shows it end to end.

Next#