Chapter 1: XVIZ Loader Interface#

Welcome to the streetscape.gl tutorial! In this first chapter, we'll tackle a fundamental concept that makes displaying autonomous vehicle (AV) data possible: the XVIZ Loader Interface.

Imagine you have a bunch of data recorded from an autonomous vehicle – sensor readings, detected objects, map information, and so on. To visualize this data in a web application, you need a way to load it and make it available to the parts of your application that draw things on the screen. This is where the XVIZ Loader Interface comes in.

Think of the XVIZ Loader Interface as a standardized "data pipe". It's the contract or blueprint for any component that's responsible for bringing that AV information (formatted in the XVIZ standard) into your application.

Why do we need a "contract"? Because AV data can come from different places: * Static files saved on your computer. * Streaming data from a server over a network. * A live feed from a running vehicle.

Regardless of the source, you want the rest of your application (the parts that display the data) to interact with it in the same way. The XVIZ Loader Interface ensures this consistency. Any component that implements this interface promises to provide a standard set of actions, ways to get data, and ways to be notified when data changes.

Let's explore what this interface defines.

What the XVIZ Loader Interface Does#

The interface defines a set of methods that any loader component must have. These methods fall into a few categories:

Actions: Starting, Stopping, and Listening#

These methods control the flow of data and allow you to react to important events.

connect(): This method starts the data flow. If you're loading from files, it might start reading. If you're connecting to a server, it establishes the connection.
close(): This method stops the data flow and cleans up any connections or resources.
on(eventType, callback): This allows you to listen for specific events. The loader will emit these events when something important happens.
- ready: The loader has loaded the initial information (like metadata) and is ready to start providing data.
- update: New data (a timeslice) has arrived and is available. This is fired frequently as data loads or streams.
- finish: All available data has been loaded (relevant for file-based or finite stream logs).
- error: Something went wrong during loading or processing.
off(eventType, callback): Stop listening for a specific event.

Think of connect() as turning on the data tap, close() as turning it off, and on()/off() as setting up or removing listeners to know when new water flows or when the tap is ready.

Setters: Controlling What Data You See#

These methods allow you to control the playback and visibility of the data.

seek(timestamp): Jump to a specific moment in time within the log data. This is how you scrub through a recorded log.
setLookAhead(lookAhead): Control how much future data is included in the current view. Sometimes you need to see slightly ahead (e.g., for predicting trajectories).
updateStreamSettings(settings): XVIZ data contains many different streams (like /vehicle/velocity, /perception/object_detections, /lidar/points). This lets you choose which streams are active and should be processed or displayed.

Getters: Accessing Information#

These methods allow you to retrieve information from the loader.

getCurrentTime(): Get the timestamp the loader is currently positioned at.
getMetadata(): Get high-level information about the log (like the coordinate system, stream definitions, start/end times if available).
getStreams(): Get the actual XVIZ data, organized by stream name. This gives you access to the raw messages that have been loaded and processed.
getCurrentFrame(): Get the processed and synchronized data relevant to the getCurrentTime() timestamp, potentially including lookAhead data. This is often what rendering components need.
getLogStartTime(), getLogEndTime(): Get the overall time range of the log, if known (like in a file-based log).
getBufferStartTime(), getBufferEndTime(): Get the time range of the data currently held in the loader's memory buffer.
getBufferedTimeRanges(): Get an array of time ranges [[start1, end1], [start2, end2], ...] that are currently buffered.
getStreamMetadata(): Get metadata specifically for each stream.
getStreamSettings(): Get the current settings indicating which streams are enabled.

This is how you ask the loader questions like "What time is it currently?" or "Show me all the data for the current moment."

Different Types of Loaders#

As mentioned, different loaders implement this same interface to handle different data sources. streetscape.gl provides a few built-in ones:

Loader Class	Data Source	Use Cases
`XVIZFileLoader`	Static Files	Loading recorded logs from disk.
`XVIZStreamLoader`	WebSocket Stream	Playing back recorded logs streamed from a server.
`XVIZLiveLoader`	Live WebSocket	Connecting to a running system for live data.

They all promise to have connect(), seek(), getCurrentFrame(), etc., but their constructors and internal logic differ to handle file reads versus network connections.

Let's look at an example of creating and connecting a XVIZFileLoader.

import {XVIZFileLoader} from 'streetscape.gl';

// Define where your XVIZ files are
const DATA_DIR = './data';

// Create a loader instance
const logLoader = new XVIZFileLoader({
  // Path to a file that lists all the frame timings
  timingsFilePath: `${DATA_DIR}/0-frame.json`,
  // A function that tells the loader how to get the path for each frame file
  getFilePath: frameIndex => `${DATA_DIR}/${frameIndex}-frame.glb`
});

// Now, start the loading process!
logLoader.connect();

// You can optionally listen for events
logLoader.on('ready', () => {
  console.log('Log metadata loaded, ready to display!');
});

logLoader.on('update', () => {
  // New data arrived, something might need to be updated visually
  // console.log('New data update received!'); // (This can be chatty!)
});

logLoader.on('finish', () => {
    console.log('Finished loading all frames from file.');
});

logLoader.on('error', (eventType, error) => {
    console.error(`Loader error: ${error}`);
});

// Other components will now use logLoader.getCurrentFrame(), logLoader.seek(), etc.

In this snippet: 1. We import the specific XVIZFileLoader. 2. We create an instance, providing options like where to find the file index and how to build file paths for each frame. This is specific to the XVIZFileLoader. 3. We call logLoader.connect() to start the loading. 4. We set up event listeners to react when the loader becomes ready (ready), gets new data (update), finishes (finish), or encounters an issue (error).

After connect() is called, the logLoader instance is ready to be used by other streetscape.gl components (like the LogViewer we'll see in the next chapter). Those components don't need to know if it's a file or stream loader; they just interact with its standard methods (seek, getCurrentFrame, etc.).

Under the Hood (A Simple Look)#

How does the XVIZ Loader Interface implementation (like XVIZFileLoader or XVIZLiveLoader) work internally?

Connection/Loading: When you call connect(), the specific loader implementation handles the details: XVIZFileLoader fetches files, XVIZWebsocketLoader (the base for XVIZStreamLoader and XVIZLiveLoader) opens a WebSocket connection.
Parsing: As raw data arrives (from files or WebSocket messages), it's processed by the @xviz/parser library into structured JavaScript objects representing XVIZ metadata, timeslices, etc.
Buffering: The parsed data (timeslices) is stored in an internal buffer, typically an instance of XVIZStreamBuffer. This buffer efficiently manages the data, often keeping only a relevant time window in memory, especially for long logs or live streams.
Synchronization: The loader maintains a current timestamp (getCurrentTime()). When a component asks for getCurrentFrame(), the loader uses a StreamSynchronizer (which works with the XVIZStreamBuffer) to find all data messages across different streams that are relevant for that specific timestamp and potentially within the lookAhead window.
State Management & Events: The loader keeps track of its internal state (current time, stream settings, loaded metadata, etc.). When this state changes or new data arrives, it updates its internal state and emits events (update, ready, etc.) to notify any registered listeners (set up via on()). This is managed via methods like set() and emit() defined in its base classes (LoaderInterface).

Here's a simplified sequence:

sequenceDiagram
    participant App as Your Application
    participant Loader as XVIZ Loader Instance
    participant Source as Data Source<br>(File or Server)
    participant Buffer as XVIZStreamBuffer
    participant Sync as StreamSynchronizer

    App->>Loader: connect()
    Loader->>Source: Request Metadata
    Source->>Loader: XVIZ Metadata
    Loader->>App: emit('ready', metadata)
    Loader->>Source: Request Data
    loop As data arrives
        Source->>Loader: XVIZ Timeslice
        Loader->>Buffer: insert(timeslice)
        Loader->>App: emit('update', timeslice)
    end
    App->>Loader: seek(timestamp)
    Loader->>Buffer: setCurrentTime(timestamp)
    App->>Loader: getCurrentFrame()
    Loader->>Sync: setTime(timestamp)
    Loader->>Sync: setLookAheadTimeOffset(lookAhead)
    Sync->>Buffer: get data for time/lookahead
    Buffer->>Sync: relevant data
    Sync->>Loader: current frame
    Loader->>App: return current frame

Note: The diagram simplifies internal interactions like state updates and event emission details for clarity.

Let's peek at some simplified code from the actual streetscape.gl library (modules/core/src/loaders/xviz-loader-interface.js) to see how the event handling works.

// From XVIZLoaderInterface constructor (simplified)
constructor(options = {}) {
  // ... base class setup ...
  this.callbacks = {}; // Object to store listeners
  // ...
}

// From XVIZLoaderInterface.on
on(eventType, cb) {
  this.callbacks[eventType] = this.callbacks[eventType] || [];
  this.callbacks[eventType].push(cb); // Add callback to the list for this event type
  return this;
}

// From XVIZLoaderInterface.emit
emit(eventType, eventArgs) {
  const callbacks = this.callbacks[eventType];
  if (callbacks) {
    for (const cb of callbacks) {
      cb(eventType, eventArgs); // Call each listener function
    }
  }
}

// From XVIZLoaderInterface.onXVIZMessage (called internally after parsing)
onXVIZMessage = message => {
  switch (message.type) {
    case 'metadata':
      this._onXVIZMetadata(message); // Handle metadata internally
      this.emit('ready', message);    // Fire the 'ready' event
      break;

    case 'timeslice':
      this._onXVIZTimeslice(message); // Handle timeslice internally (buffer data)
      this.emit('update', message);   // Fire the 'update' event
      break;

    // ... other cases ...

    default:
      this.emit('error', message);     // Fire 'error' for unknown message types
  }
};

This shows how the on method registers callbacks, the emit method calls them when an event happens, and the internal onXVIZMessage method decides which event to emit based on the type of XVIZ message received.

The seek method is also relatively straightforward at a high level:

// From XVIZLoaderInterface.seek (simplified)
seek(timestamp) {
  // clamp timestamp to log start/end if known
  // ... boundary checks ...

  this.set('timestamp', timestamp); // Update internal state (triggers re-renders for connected components)

  // Tell the buffer where the current playback head is
  this.streamBuffer.setCurrentTime(timestamp);
}

It updates the internal record of the current time and informs the XVIZStreamBuffer so the buffer can manage which data needs to be kept or potentially loaded around that time.

Finally, the getCurrentFrame method relies on the StreamSynchronizer:

// From XVIZLoaderInterface.getCurrentFrame (simplified selector)
getCurrentFrame = createSelector(
  this,
  [this.getStreamSettings, this.getCurrentTime, this.getLookAhead, this._getDataVersion],
  (streamSettings, timestamp, lookAhead) => {
    const {logSynchronizer} = this; // Get the internal synchronizer instance
    if (logSynchronizer && Number.isFinite(timestamp)) {
      logSynchronizer.setTime(timestamp); // Tell synchronizer the current time
      logSynchronizer.setLookAheadTimeOffset(lookAhead); // Tell synchronizer the lookahead
      // Ask the synchronizer for the combined data frame
      return logSynchronizer.getCurrentFrame(streamSettings);
    }
    return null;
  }
);

This is a bit more complex due to createSelector, but the core idea is that getCurrentFrame doesn't access the raw buffer directly. Instead, it asks the logSynchronizer for the frame corresponding to the current time and lookahead settings.

Conclusion#

The XVIZ Loader Interface provides a crucial abstraction in streetscape.gl. It defines a standard way to load and interact with XVIZ log data, regardless of whether it comes from files, a stream, or a live feed. By implementing this interface, different loader classes allow the rest of the streetscape.gl components to interact with the data consistently.

Understanding this interface is the first step to building applications with streetscape.gl, as it's the piece responsible for getting your valuable AV data into the viewer.

In the next chapter, we'll look at the LogViewer component, which is a high-level React component that uses an instance of an XVIZ Loader to display the data.

LogViewer