u/datascienceharp

a teleop operator reaches for the wrong bin. the reach is clean. no jitter, no hesitation, no correction

every smoothness metric comes back perfect. the episode is still garbage. the robot did the wrong thing smoothly

motion metrics score how an action was executed, never what the action was. so automated scoring has one honest job: pointing your limited attention at the episodes most likely to contain a real problem. triage, not autofilter

i built a fiftyone panel that runs this on multimodal MCAP episodes: motion smoothness, sensor health, outliers, every flag deep-linked to the exact second on the timeline. free and open source

plugin: github.com/harpreetsahota204/demo_quality_scorer

full writeup on what i learned while building this : https://voxel51.com/blog/robot-episode-quality-triage

curious how you're triaging episode data right now.

watching everything? random sampling? trusting a score?

most SLAM datasets mount the lidar level.

tilt it 45 degrees and everything changes: the camera and lidar barely overlap, the upper beams are sparse, and standard odometry starts to drift

that's exactly how compact mobile mapping rigs are built in the real world. the lidar tilts so it sweeps more vertical structure. but almost no benchmark tests this configuration

YUTO MMS from York University: a tilted 32-beam lidar, a 6-lens panoramic camera, and GPS/INS with cm-level ground truth driven through Toronto.

every lidar point is RGB-colorized from the nearest panoramic frame, not a synthetic colormap

loaded as mcap in fiftyone. scrub the timeline and watch the world-frame 3D map build itself progressively alongside the panoramic camera, GPS track, and IMU telemetry

checkout the dataset here: https://huggingface.co/datasets/Voxel51/yuto-mms-multimodal

it's running as a live space too, nothing to install: https://huggingface.co/spaces/harpreetsahota/yuto-mms-multimodal

a robot can navigate a hallway without hitting anyone and still make every person in it uncomfortable.

collision-free and socially aware are two completely different problems

NavWareSet records both. seven social navigation scenarios (frontal approach, blind corner, following, perpendicular crossing), each run twice under matched conditions: once with socially compliant behavior, once without. same room, same people, same starting positions.

the only variable is whether the robot navigates like it knows humans have personal space

robot onboard lidar and camera plus an overhead ground truth station tracking every pedestrian in 3D across the full episode

loaded as native mcap in fiftyone. scrub the robot's camera, both lidar streams, and the annotated pedestrian trajectories on one synced timeline.

filter by scenario and behavior to compare compliant vs non-compliant side by side

start here, read the dataset card: https://huggingface.co/datasets/Voxel51/navwareset

the car in front of you on a wet highway kicks up a spray plume.

your lidar sees it as a wall of false objects. your camera sees a blur through the windshield. your radar barely notices

SemanticSpray++ from Ulm / BMW: 36 vehicle-following episodes on a closed wet airstrip, 50-130 km/h, with per-point semantic labels on both lidar and radar telling you exactly which returns are spray noise and which are the actual vehicle. plus 2D camera boxes and 3D lidar boxes on every frame

loaded as native mcap in fiftyone so you can scrub camera, lidar, and radar together and watch the spray noise light up in the point cloud while the boxes track the lead vehicle through it

checkout the dataset here: https://huggingface.co/datasets/Voxel51/semanticspray-plusplus

or get hands on with this hugging face space: https://huggingface.co/spaces/harpreetsahota/semanticspray-plusplus?logs=build

your gaussian splat looks photorealistic from the trajectory you trained it on. move the camera off that path and the geometry falls apart

this barely gets measured because the ground truth has to be more accurate than the thing you're scoring. that means dragging a survey-grade scanner around the site for days

oxford robotics institute did it for six oxford landmarks. 24 sequences, 125,000 m², a handheld rig with three synchronized fisheye cameras, a 64-beam hesai lidar and an imu, and a leica RTC360 scan of every site as the reference — 1.9mm accurate at 10m, with the trajectories registered at 1-2cm

the novel-view test images aren't held-out frames from the training path. they're a different walk through the same site facing a different direction. that's the part that breaks splats

i packed six episodes into mcap so you can scrub all three cameras, the lidar, the imu and the slam pose on one timeline in fiftyone, with lidar depth painted onto every frame

checkout the dataset here: https://huggingface.co/datasets/Voxel51/oxford-spires-multimodal

it's running as a live space too, nothing to install: https://huggingface.co/spaces/harpreetsahota/oxford-spires-multimodal-explorer

falling snow shows up as thousands of fake objects in lidar point clouds. radar barely notices it's snowing

most self-driving datasets are shot on sunny days in california or phoenix. none of them show what happens once the weather turns

Boreas is UTIAS's answer: 128-beam lidar, 360 degree radar, and 5MP camera driving the same Toronto route for a full year through sun, rain, and snow.

326,180 3D boxes for cars, pedestrians, and cyclists across 7,111 labeled frames

loaded as native mcap in fiftyone so you can scrub camera, lidar, and radar on one synced timeline, and watch the 3D boxes render live on the point cloud and project onto the camera and radar images

checkout the dataset here: https://huggingface.co/datasets/Voxel51/boreas-multimodal

or get hands-on with this hugging face space: https://huggingface.co/spaces/harpreetsahota/boreas-multimodal

a robot mapped a forest road in november. it came back in january and the road was buried under a meter of snow

the same lidar-inertial slam that localized fine before the snowstorm saw its drift jump from 4% to 46% on the exact same route

FoMo is a year-long multi-season robot navigation dataset from a boreal forest in quebec, eh.

2 lidars, an fmcw radar, stereo + mono cameras, dual imus, and gnss ground truth, across 12 deployments from -19°c winters to 18°c summers

i parsed the episodes into fiftyone's new multimodal mcap format so you can scrub camera, lidar, and radar together

watch the ground-truth trajectory move in 3d, and see the same road across six different seasons

start here, read the dataset card: https://huggingface.co/datasets/Voxel51/fomo-multimodal-sample

and get hands-on in this hugging face space: https://huggingface.co/spaces/harpreetsahota/fomo-multimodal-sample

everyone on my timeline is screenshotting Qwen3.8-Max drawing bounding boxes

clean demos, obvious objects, no ground truth to check against

i pointed it at 27,083 real logos and scored every box against actual annotations

here's what nobody is showing you:

• it invents its own pixel canvas even when you tell it the real image dimensions.

• same prompt, same image, same settings: one run matched 3 of 5 logos. the next run matched 0 of 5. nothing changed between calls

• changing one verb in the prompt, "mask out" to "draw segmentation masks," silently switched the model from a 0-1000 grid to normalized [0,1] coordinates.

• thinking mode costs 14x more tokens and doesn't reliably improve accuracy. it just shows you the model doing long division instead of looking at the pixels

full writeup with every trace, every score, and the fiftyone plugin to run it yourself: https://voxel51.com/blog/qwen38-max-visual-grounding-fiftyone

test it yourself here: https://huggingface.co/spaces/harpreetsahota/qwen38-max-openlogo-demo

on a lake there are no lane lines, no fixed landmarks, no other vehicles to localize against

the shoreline shifts with your viewpoint, radar and lidar don't share a clock, and sonar is measuring a world the cameras can't see

CANOE is a multi-sensor USV dataset from UTIAS: 360° radar, 128-beam lidar, stereo camera, sonar, and GPS/INS ground truth across lakes and a reservoir in Ontario

parsed it into fiftyone multimodal so you can scrub every sensor on one synced clock and project lidar straight onto the camera to see where they agree and where they don't

checkout the dataset here: https://huggingface.co/datasets/Voxel51/canoe-multimodal

or get hands on in this hugging face space: https://huggingface.co/spaces/harpreetsahota/canoe-multimodal

this intersection in tokyo ranked second worst in the city for traffic accidents.

six roads converge at a blind hill crest, cars cross centerlines on narrow curves, and the signal phasing has multiple unprotected turns

most autonomous driving datasets give you highways and four-way stops. this is none of that

Hard Intersection Multimodal Sample: 6 synchronized cameras, aggregated LiDAR point cloud, HD map projections, vehicle trajectories, and semantic annotations across 4 driving passes through a single intersection that breaks everything

grouped all 6 camera views with the 3D point cloud, frame-level HD map overlays, and trajectory projections in fiftyone

checkout the dataset here: https://huggingface.co/datasets/Voxel51/hard-intersection-multimodal-sample

or get hands-on in the HF space: https://huggingface.co/spaces/harpreetsahota/hard-intersection-multimodal-sample

lidar and cameras get less reliable exactly when driving gets more dangerous: rain and night. most public driving datasets barely have data from those conditions

CMHT autonomous dataset adds radar and a thermal camera alongside lidar, a color camera, and gps/imu.

4 drives, dusk/clear to night/rain, downtown hamilton, 9,000+ labeled frames with a 3d box, class, and tracking id on every object

i converted the raw ros2 bags into synced mcap episodes in fiftyone so you can scrub camera, thermal, lidar, radar, and gps together frame by frame, with the 3d and 2d boxes playing back in sync

start here, read the dataset card: https://huggingface.co/datasets/Voxel51/cmht-autonomous-driving

then check out the space on hf: https://huggingface.co/spaces/harpreetsahota/cmht-autonomous-driving

try it right now without installing anything.

the fiftyone app is running in a hugging face space for the first time (its a bit hacky atm, but working on polishing it up)

space: https://huggingface.co/spaces/harpreetsahota/fiftyone-app

full walkthrough: https://voxel51.com/blog/view-mcap-files-fiftyone

most plant datasets are top-down RGB images at one point in time

this one tracks individual leaves in 3D at 0.012mm accuracy as they grow, day by day, for 20 days

Pheno4D: 7 maize and 7 tomato plants laser-scanned daily in a greenhouse

sub-millimeter point clouds with per-point instance segmentation where every leaf keeps the same ID across the entire time series

you can track leaf area, leaf length, stem diameter, and growth trajectory for every organ on every plant

223 scans parsed into fiftyone as interactive 3D point clouds. shade by instance label and scrub through the time series to watch each leaf emerge and expand

check it out here: https://huggingface.co/datasets/Voxel51/pheno4d

check it out here: https://huggingface.co/datasets/Voxel51/VineLiDAR

14,800+ functional elements across 710 laser-scanned indoor scenes. every handle, knob, button, and switch labeled with its affordance, motion axis, motion type, and a natural language task description

each scene's 3D point cloud with its iPad video recordings in fiftyone. the affordance annotations are in 3D and project into the video frames

check it out here https://huggingface.co/datasets/Voxel51/SceneFun3D

nobody is driving a real vehicle into a flood to get training data.

but a model that's never seen standing water will drive straight through it

i built a workflow that generates the missing conditions from real frames

comfyui runs inside fiftyone.

open a sample, restyle it to rain, fog, snow, or dusk, save the result back with full generation metadata and a link to the source frame

synthetic data you can trace is an asset. synthetic data you can't trace is a liability

learn the full workflow here: https://voxel51.com/blog/synthetic-off-road-data-comfyui-fiftyone

check it out here: https://huggingface.co/datasets/Voxel51/KITScenes-LongTail

• Saved views — 6 per-camera views (slice__front_left … slice__rear_right) + by_scenario_type_and_instruction.

• Qwen3-VL embeddings — 2048-d clip embeddings on the surround, front, and rear slices.

• Similarity & visualization indexes — sim_* (nearest-neighbour search) and viz_* (UMAP 2D plots) per slice.

• Quality scores — uniqueness_* and representativeness_* per clip (find rare vs. prototypical scenarios).

• 3D threed slice — VGGT-Omega point-cloud scenes (.fo3d) reconstructed from the front clip, plus per-frame depth_map heatmaps and camera-pose fields on the front slice.

get started:

pip install fiftyone

then install this plugin (one-line install): https://github.com/harpreetsahota204/annotation_styles

i saw a post from u/ReturnAdventurous179 earlier today about their lost rc plane

op shared a link to a gdrive folder with a zip, that seems kinda dodgy, but i took one for the team anyway and parsed it into fiftyone

i uploaded to hugging face, you can find it here: https://huggingface.co/datasets/Voxel51/ariel_scans

i enriched this dataset with clip embeddings for visualization and similarity search.

u/Aimforapex shared some code to help op: https://gist.github.com/dwightkelly/05a7d9421486fd2ebda696d315c05481

i took this, modified it a bit, then made it into a fiftyone plugin so it's easier to use and iterate on: https://github.com/harpreetsahota204/plane_finder

there's a starter notebook here: https://github.com/harpreetsahota204/plane_finder/blob/main/Finding_a_Needle_in_the_Desert.ipynb

apparently there is fifty bucks on the line, but don't message me asking for it.

u/ReturnAdventurous179 you should open this as a kaggle competition 😆

PartScan from PinPoint3D: 1,509 scene-level 3D scans with dense per-point part segmentation across 707 scenes.

no manual annotation, fully synthesized pipeline on real-world-style geometry

parsed it into fiftyone as interactive 3D point clouds. every point colored by its part label

https://huggingface.co/datasets/Voxel51/partscan