Lenslist asked Pavlo and me for a simple phone-recorded interview about building AR for Snap Spectacles. We went a bit overboard 😅: we wrote the answers as a script, cloned our voices with qwen3-tts, rendered our Bitmojis in Blender from 12 angles as character references, and let Seedance 2.5 generate the whole thing shot by shot, lip-synced to the cloned audio.

The setting is the 3D cave from our portfolio site, and every prop is one of our shipped projects.

This was definitely not a one-shot. We generated a lot of takes and kept the best ones. The lip sync quality surprised us the most, even on stylized characters with a beard covering half the mouth.

Full making-of with the examples prompts and pipeline: https://pavlo-stijn.dev/blog/posts/creator-spotlight-with-seedance-2-5.html

TL;DR: I built a live digital twin / diorama of Heathrow Airport (Terminal 5 for NOW) for the Spectacles. It sits on your table, with flights landing and taking off ... all based on LIVE public Heathrow flight data. ✈️

For the last 3 or 4 years I’ve been teaching AR and spatial storytelling at the Royal College of Art, including a lot of work with Snap and Spectacles. My students have made some brilliant projects with them, but somewhere along the way I realised I hadn’t really published many of my own.

So here is one.

I was obsessed with planes when I was younger. Growing up we didn’t travel much, so airports always felt like a huge event. As someone who loved making dioramas as a kid, I would collect airplane / aviation souvenirs…by getting relatives who traveled to bring them for me , and then set them up as airports.

And recently, while driving to Heathrow, I noticed a patch by the runway full of people with huge cameras, chairs, binoculars and notepads.

PLANE SPOTTERS!!! 🛫 🧐

It made me think: what if I could shrink Heathrow down and put it on a table? HOW COOL WOULD THAT BE!

So I built a digital twin using geospatial data OpenStreetmap Data, brought it into Blender with a plug in and extruded the buildings. So it’s all pretty accurate. I originally tried a much bigger section of the entire Heathrow Airport, but everything became too tiny and too heavy, so I focused on Terminal 5 (FOR NOW)

What it does

• Uses live Heathrow arrivals and departures data
• Planes land, take off, queue and park
• Labels show flight number, route, aircraft type and status
• Audio effects have been used to make it feel more fun
• Pointing at a plane enlarges the Flight Label
• Pointing and pinching a plane opens a "Spot Card" which is a UI frame with deeper information about that specific aircraft and flight.

With the SPOT CARD, the idea was to go beyond simply watching the airport and give plane spotters the kind of detail they actually care about. Which includes: Flight number, airline, route, aircraft type, aircraft variant, origin/destination, current flight status, gate/stand.

AND it lso includes a bonus fact about the aircraft or route, adding a small bit of aviation trivia to each Spot Card.

A huge part of the work was optimisation. Lots of planes, labels, interactions and shadows get expensive very quickly on glasses, so I ended up building pooling, parked-aircraft limits, hover-only labels, selective shadows and staggered movements.

There is loads more I want to add for actual plane spotters, but seeing a tiny live Heathrow running on my table is making the plane-obsessed younger version of me very happy.

(For this video I’m using demo mode because I recorded it around midnight when Heathrow terminal 5 had gone very quiet and there were no flights from 11:00 pm to 5:00 am. BUT the live version works like a charm, Ill post more soon).

Please let me know what you all think or what else I could add to it :)

I’m interested in the multiplayer potential of Specs. In Snap’s chess demo, though, the remote player’s movements look noticeably laggier than the local player’s. I hope Snap can improve that latency, because shared AR really needs to feel responsive.

Personally, I find experiences like shared basketball or virtual ping-pong more compelling than chess, since they’re much harder to replicate on a phone. Am I expected too much out of specs? But I hope people at specs realize if the multiplayer is a selling point to not make sure it’s laggy in anyway.

I had this last-minute idea for the CLAD Hackathon. I struggled quite a bit with UI visibility, but the overall prototype is working well.

NEXUS is a spatial productivity and collaboration app for Snap Specs that transforms a traditional Kanban board into an interactive mixed-reality workspace.

Teams can create, organize, assign, and move color-coded tasks across a shared workflow. Tasks persist between sessions and synchronize across multiple participants, allowing collaborators to contribute to the same project in real time. Voice input makes it possible to create and edit notes hands-free, while task categories provide immediate visual context.

NEXUS offers two complementary ways to understand a project. Board View provides a familiar Kanban workflow for tracking progress, while Spatial Map turns tasks into movable nodes that can be connected with curved relationship lines. This helps teams visualize dependencies, creative relationships, and larger project loops that are difficult to understand in a flat task list.

Tasks can also become visual workspaces. Users can generate AI concept artwork or 3D artifacts from a task description, keep those assets associated with the task, and place generated 3D models into the surrounding space. Artifact position, rotation, scale, and task relationships are synchronized so collaborators experience the same shared arrangement.

The included multiplayer demo uses the production of a survival game to clearly demonstrate the workflow. Tasks cover environment art, gameplay systems, audio, narrative, QA, crafting, and inventory. A spatial game loop connects exploration, gathering, crafting, survival, and upgrading, while generated concept images and 3D assets demonstrate how teams can move from an idea to a shared visual reference.

Although game development is used as the demonstration, NEXUS is designed for any collaborative project—from product design and filmmaking to education, event planning, architecture, and creative brainstorming.

NEXUS explores a future where project management is no longer confined to a laptop screen. Work becomes spatial, visual, collaborative, and directly connected to the ideas and artifacts a team is creating.

Get your vinyl collection organized. This experience helps you catalog your records, organize them on the shelf and tell you where to find them later.
(Longer post on my LinkedIn)
Enjoy!

Continuity is basically physical version control for real-world setups. You save a workspace as a “Take,” then later compare the current scene against it to see what moved, rotated, or went missing, and restore the setup using spatial 3D ghosts.

Think film sets, creator studios, YouTube or product-shoot setups, retail displays, gallery installations, or any workspace that needs to be rebuilt exactly the way it was.

Under the hood, WorldQuery establishes the physical workspace, Gemini helps understand the objects in the scene, and rigid registration brings the saved state back into the correct real-world position. For this video I’m using a deterministic demo mode with prebuilt 3D props so I can reliably film the full workflow without hardware/environment variability — those objects are just demo stand-ins; the actual system is designed to observe and reconcile real physical objects in your environment.

>

Writing has expanded our ability or organize as groups by allowing us to transfer experiential content into an informationally stable support. Whether it's etching on basalt or ink on paper, this recording process enables us to draw links between concepts, keep track of notions that have faded from working memory and perform operations that overcome some of the limitations of bare minds.

In some ways, writing can be thought of as a process of encoding one's subjectivity, that is, encoding the content of one's experiential stream.
In this case, reading can be seen as a process of decoding this subjectivity. If the encoding is compactly and vividly performed and the reader is keen, the subjectivity of the reader can be merged to that of the writer (see Sasha Capin, "Books as subjectivity merging devices, rather than information transfer devices).

Both the encoding and decoding processes play an crucial role in cognitive development and the upkeep of cognitive acuity.
However, both require sustained attention, which is now widely known to be incurring heavy damage from attention capturing technologies.

Since only so much damage can be undone, since habits can only be changed so much among the broader population, we can think of working within the confines of technology to make thoughtful designs that at least curb the decline in sustained attention, or lean into the minds' peripheral processing to guide attention towards the subject at hand.

The aim of this project is to investigate how ambient virtual content can be used to ease people into the consumption of long form content, written by a conscious subject, thereby helping protect their attentional faculty ( see Joe Darko We Built Technology to Capture Our Attention. What If It Gave It Back? )

One approach is to provide a cognitive mirroring utility.
We can think of modeling the way minds shuffle concepts both between each other, move them and in and out of working memory, and provide the user with some a spatial anchor of the process they're already engaged in whether consciously or not.

To do this we can extract concepts from a text corpus and lay them out into an orderly grid-like structure called lattice.

First, this specific arrangement leans into our minds' natural ease with processing axial movement(horizontal, vertical, and depth)
Additionally, each concept can only be related to 6 other concepts at once which is friendly to memorization at glance and spatial shuffling.
These design decisions create constraints that aim to avoid some of the pitfalls of arbitrary relational graphs which have long been the default mode for representing corpuses of knowledge or simply any data, as is the case in mind maps.

I've been looking to implement this for 3 years. I began with syntactic parsers and categorial grammars but these methods failed to handle longer context and ambiguity.

Harnessed LLMs have essentially unlocked the ability to process language over longer word horizons, to extract meaningful relations and deal with the spuriousness and breath of human language.
Their integration into 3D engines such as Lens Studio with CLAD has greatly reduced the difficulty of generating complex spatial arrangements and translating spatial transformation primitives into code.

As a result is fair to say this might be an LLM-native technological artifact.
Here's a demo of the concept lattice being generated based on a text corpus (The Origin and Development of Markets: A Business History Perspective).

In practice, you would open a book with your Specs on, and have the tool generate the ambient content as you read. This content would be both relevant to the theme of your book and serve as a reference to look back to and potentially help you take reading notes yourself (that would be an ideal outcome)

This could also serve as a tool for AI interpretability, particularly if combined with Jacobian Lenses, to capture and convey language processing primitives spatially and succinctly.

I have documented the technical reasoning and design decisions in this repository, so feel free to give some feedback or direct your token flow towards it.

https://github.com/a-sumo/concept-lattice

For Week 1 of the CLAD Summer Hackathon the theme was organize / plan / be more productive, so I built a feng shui master for Specs, since feng shui is fairly literally the practice of organising a space.

Fengshui Master reads the room you're standing in. You pinch once: it scores your room's chi out of 100, names what's actually blocking it, and repaints the room to show you the fix. You get a checklist of what to move, clear or add — then you tidy up for real, assess again, and watch the score climb toward a target.

What it does:

- One-pinch assessment — capture → AI analysis → AI repaint → a second AI pass that verifies the repaint applied the changes before you see it

- Chi score with a verdict, a target to beat, and your best score kept across sessions

- Five-element (五行) balance so you can see which element the room is starved of

- Chi Plan — the fixes as a tappable checklist for the real room

- Problem markers pinned to real objects — the bare wall, the cluttered shelf

- Chi-flow particles that congest and turn amber where energy is stuck, then clear once you improve it

- Before/after slider, plus a depth-displaced 3D view you can lean into

- Scan Whole Room — turn on the spot and it reads four views, with commentary on each

- Ask the Master — hold a mic button and ask about your actual room, out loud

- Shop — buy the piece it recommends without leaving the Lens

- English and 中文 throughout, including the spoken voice.

Built entirely with CLAD and Claude code.

Full prompt log and source here: https://github.com/ohistudio/fengshui-master

This is my theory for one killer app that will prove all the naysayers wrong about Specs.

Like all new platforms, you need some killer apps right out of the gate to catalyze adoption. Some examples that come to mind are Super Mario for Nintendo, Tetris for Gameboy, Sonic for Sega, YouTube, Pandora, Facebook and Shazam for iPhone (2008).

10 years ago, when Pokemon Go came out on the iPhone there was pandemonium (see video below). Pokemon is still incredibly popular today, with hundreds of millions of fans ripping, trading and collecting cards, playing games, watching content (produced and user generated), engaging with the community — it's become part of our culture.

If Snap can strike a partnership with Pokemon Go, create a new immersive AR experience for catching, collecting and trading Pokemon, and building a world around it, a whole new arm of that community will emerge. This could be the start of real life game play, where you're literally IN the game, running around the actual world, not just a virtual world (VR goggles) or behind a computer screen at a desk, or holding up a tiny phone screen. This could lead to seasonal narrative arcs, competitions, movies, etc. and would be a blue print for other franchises.

And for those who can't get past the current form factor, you may not fit into the category of "early adopter" and that's okay. The people who are passionate about Pokemon could care less about how "cool" they look while running around catching fictional creatures.

The technology is nothing short of revolutionary — it just needs a little push for people to see the potential and to get the ball rolling, and it will just keep improving with each iteration.

$SNAP

2016 Pokemon Go - Vaporeon stampede Central Park, NYC

My new experiment! The pots get saved to Supabase, and in theory, you would be able to see other people's pots. Video sped up 4×.

As an AR consultant for AGOG, one of the (many) fun parts of my job is getting to do some R+D! So, I’ve been experimenting with Gaussian splats on the spectacles and I wanted to share some of the tips and tricks I learned along the way. 

1- When preparing a Gaussian splat for your project, be mindful of your splat count. The first splats I imported made the project suuuppper laggy. I had initially thought it was the file size, but even smaller sized splats with a large splat count produced the same issues. I found success with making sure the splat count was no larger than 500k.

2- It seems you can only import a regular ole’ .PLY file. I thought a compressed .PLY would work as well, but there were some incompatibility issues, at least using Super Splat to export the compressed version. (Let me know if you’ve found a way to successfully use a compressed .PLY file!)

3- Lens Studio uses Draco compression on import to help bring down the file size as well. I would play with the compression settings though to find the right balance between file size and quality. There were times when I tried importing a splat with and without compressing to compare, and I did see a difference in quality. 

4-   360 and 160 gaussian splats work really well! I initially thought the field of view would pull me out of the experience, but seeing the splats fully overlaid on the “real” world added a really ethereal quality in the glasses that could work beautifully for storytelling. 

Combining with Story
Gaussian splats are a lovely way to capture memories, and that can be utilized with the Specs.  I took the above footage of the amazing spread from my Aunt’s Thanksgiving dinner a few years ago. It turned out to be the last Thanksgiving she hosted before she passed. I’m thankful to have capture it in 3D even if it’s not the best quality. But no matter the quality, I know it’s not real. So, I played with those bittersweet feelings using a distance trigger and particle effects. The closer I get, the illusion fades away. I really do miss those dinners tho LOL!

Interactivity and Spatial Audio
With this pipe organ splat (courtesy of Ashley Buschorn), I added a spatial audio effect so that the sound of the organ gets louder as you approach it. I also added some simple buttons on the organ to add a touch of interactivity. 

Pipe Organ

Larger splats
I tried a few larger splats including this beach scene I captured a few years back (when NeRFs were a thing!). I used the Surface Placement tool to ensure I could place the gaussian splat in a convenient location, as well as align the “floor” of the scene with my actual floor. 

Beach Scene

If y’all have also tried out splats and specs, I’d love to see your examples! 

https://github.com/flarb/BrushAR

Ok--I saw so many cool BLE demos lately and wanted to mess around with BLE myself. The only relevant Bluetooth devices I could think of that I might have access to are smart Sonicare tooth brushes. So I bought a kids brush and a 9300 -- two models that don't require bonding and thus work with the Spectacles.

So--this is sort of an attempt to gamify brushing your teeth, as well as taking the typical smart toothbrush app into AR.

Pardon the bad video--but I had to zoom in a lot to see the tiny face anchored interface plus threw some stabilization on it.

The basic flow is:

* Load up the lens

* Wake up your toothbrush so Spectacles can see it

* Change the mode / settings to what you want to use for brushing

* Once the brush is found, stand in front of the mirror until your face is detected

* Power on the brush and start

From there you'll see a HUD bound to your face which gives the current progress and status of your brushing session. With the kids brush, you see a simplified HUD. The time elapsed, time left, and 4 squares representing the quadrants of your mouth. The quadrant fills with green until you finish brushing it and the Sonicare vibrates to tell you to move on to the next quadrant, accompanied by virtual sparkles augmenting your increasingly clean teeth. When all 4 quadrants are done, you get a brush complete message with celebratory fanfare.

With the 9300 brush it's more advanced. When you start brushing you see 6 sections instead of 4. You also see a pressure gauge on the right which turns red if you are using too much pressure (this is bad for your gums). On the left you see the intensity setting you have the brush set to. The same rules apply, move on to the indicated section once you get the signal. This brush as a bunch of modes, so the actual timing and which section you move to is governed by the mode.

The app tracks your streak--brushing once in the morning and once at night. It saves your streak as a score and uploads it to a leaderboard. You can compare your streaks with your friends or global scores.

A couple of notes for Snap:

* BLE needs to not be 'experimental'. It's time to drop all these silly restrictions on Lenses and make a proper permission system. I have some actual real Lenses clients are interested in building with me, but they're gated by the fact that you can't ship a BLE lens.

* I need to be able to use the camera with BLE. If I could do that I could optically track the brush. But you can't access the camera and BLE at the same time according to the documentation

* You need to support bonded devices. I used the 9300 brush which cost over $200 but is discontinued, because all the newer ones are bonded and that's not supported on Spectacles.

Hopefully these issues are addressed on SPECS!

Hey r/Spectacles! 👋

I just published Plan Cub AI, an AI-powered cube coach for Specs

The idea: I never looked up how to solve the cube. I figured it out by thinking, and

that taught me more about logic than any tutorial. So I built a coach that does the

same for you: it never solves the cube for you; it teaches you to solve it.

What it does:

🎓 Learn: a guided lesson through the layer method. The coach reads the REAL state

of your cube, highlights the pieces that matter, demonstrates moves (and undoes them

so YOU do it), explains the WHY behind each stage; and if you're breaking your own

progress, it gently brings the cube back.

💡 Stuck? Tap the bulb for a hint.

🎲 Mix & Play: free solving against the clock, personal best saved on device.

📅 Daily Challenge: everyone in the world gets the same scramble each day. Keep

your streak alive!

🗣️ Full voice coach in English, Español and Français (OpenAI voices via Remote

Service Gateway), plus voice commands.

✋ Natural interaction: swipe any row with your index finger to turn it, pinch +

drag to rotate the whole cube, just like holding a real one.

Tech: Lens Studio 5.15, Spectacles Interaction Kit, RSG for the AI voices. The cube

state detection is derived entirely from piece transforms; the cube itself is the

source of truth.

Try it here: https://www.spectacles.com/lens/c7993aeab3dd4856affd0acaca25df15?type=SNAPCODE&metadata=01

Would love your feedback, especially from anyone who learned the cube the hard way.

What should the coach teach next?

I shared an earlier version of SunSeat here a little while ago, but I didn’t really have a decent place — or video — to show how it actually works.

I built it earlier this summer as my July Spectacles entry, and while I was at the beach this weekend I finally got to use it in exactly the kind of situation I designed it for :)

The idea is simple: SunSeat lets you preview how sunlight and shade will shift around a spot before you settle there.

You calibrate the ground, align the sun’s real-world position, and the Lens uses your location, time and orientation to calculate and anchor the sun’s path around you.

Then you can look at a shade source — an umbrella, wall or tree — and move time forward to see how the shade will shift. You can scrub through time manually or use voice commands, with simple verdicts like SHADE HOLDS or SUN HITS IN 22 MIN.

In this video, I checked where the umbrella shade would be a couple of hours later and moved my lounger accordingly.

It’s not only about beach chairs either — the same idea could be useful for choosing a café table, setting up a picnic, planning an outdoor shoot, or simply knowing where the light is going to be later.

Finally managed to shoot it somewhere that makes sense. Would love to know what you think.

See earlier post here: https://www.reddit.com/r/Spectacles/comments/1vc0n40/sunseat\_preview\_the\_shade\_before\_you\_sit/