Locus Prototype: Real object tracking
I'm prototyping the real object tracking and here is what I got so far.
How I made it
- I scanned the mug with Reality Composer on my iPhone Pro.
- I trained a .referenceobject with Create ML on my M4 Pro Mac mini. That took about 5.5 hours.
- Then I imported it into Locus.
What you're seeing
- Desk & Surroundings: Locus tracks the mug and opens a mug-shaped window through the Virtual Space, so you see the real mug through passthrough.
- Crown / full immersion: there's no passthrough there, so I render the 3D model where the real mug is.
Honest notes
- The scan is rough. You can see the flaws in the model in the Crown part of the video: the rim isn't round and the surface is lumpy. Tracking works anyway.
- I've only tried this one mug. Generic cups and printed markers didn't work for me; only training on this exact mug did.
- Tilt the mug and it loses track, which you can see in the video. My guess is that I trained with Create ML's "upright" viewing angles, so tilted views are outside what it learned. Training with all angles might help, but I haven't tested that.
There are better ways to build the model than a quick phone scan. I'm going to try a cleaner reconstruction next and will report back if tracking improves.
I'll include Real Objects as an experimental feature in the next version (Settings › Real Objects, after turning on the Experimental switch) and free, with no paywall. But initially there won't be any official models.
I've tried other things like building a general mug model, it doesn't work. Also the image trackers only works if it's put flat in front of my eyes. When I put them down on the table they stop working.