u/inchenzo

Hi,

Welcome to the 11th edition of the Sunday Report! I hope you had a great weekend and have a great read!

Week 21 -> Let's go!

Spatial 3D

Being the first to bring realtime spatial 3D conversion to Vision Pro was quite the feat it seems. My mailbox has never been hotter. I'm glad to hear so many of you enjoying it on their Vision Pro's. It's crazy to think this has been possible for such a long time and no one putting in the effort to actually do it.

That said, being the first is fun and all, but taking it to the next level is much more interesting to me. There were just too many technical tradeoffs that had to be taken care of.

The first version was to give you a glimpse of what's possible, because I knew there's much more possible. As some of you know Google's Android XR has realtime 3D conversion systemwide, but it's been limited to 1080p 30fps and it would overheat.

What I've been doing is to take this tech to the next level. And since last Friday I've been able to push it to 1080p 60fps HDR without any compromises!

This means that at 60fps a new depth map is being created for EVERY SINGLE FRAME while maintaining 60fps per eye (so running 120fps actually: since you have to render it per eye) and most importantly: all of this happens while keeping the device cool, so you can play for hours on end -> No concessions!

All of this while improving the visual quality, minimising any visual artefacts or warping.

It's remarkable what's possible on Apple's hardware. What's even more remarkable is that it runs great on both the M2 and M5 version of the Vision Pro.

As far as I know, It's the first and only implementation to achieve this on a consumer processor at this frequency while keeping the quality high, the latency low and the device cool.

Crazy right?

And that's not all: I've always been a believer that stuff should just work, since the more settings you have, the more complexity and uncertainty you bring to the end-user looking for the best setup.

Stuff just needs to work.

Which means it's all been simplified: It's all just one button press for you to get going. The depth slider is gone, the mesh slider is gone, no more end-user finetuning needed -> it just works so you can focus on what matters: The Game.

I can't wait to get this in your hands, and will be available with version 1.3.2 of Asobi, and with version 1.0 of Midori (r/MidoriApp)

....

That said, I now also have it working on iPhones and iPads, so it won't be long for those devices to get in on the Spatial 3D action via a connected 3DTV or 3D glasses.

For those wondering if I can get this to work on the Meta Quest -> I don't know yet, but I'll try. I just hope it won't melt the device 🫠

The Asobi Porting Initiative (I thought this sounded cool)

- Android

Besides being overwhelmed with message for people wanting to join the The PlayTest, It's also about to come to an end. Which means the full release is coming closer. Aim for the full release is currently for week 22. Thank you all for sharing your feedback! And keep an eye out for the announcement on this subreddit.

- Steam (Windows/Linux)

Although the coming PlayTest might not support all devices out of the box just yet, the plan for the full release is to support Windows ARM/x86 and Linux. So all no one should feel left out when the full release arrives. Either way, the PlayTest should start around the time that the Android version has been released.

- Meta Quest

When the Android version is out, I intend to follow up quickly with the Meta Quest PlayTest. It should be very performant and easy to use. But just like the official Xbox remote play client on there, controller will be limited to using a DualSense/DualShock controller. More about this when the time comes for the PlayTest to start.

Either way, there's loads incoming. When these ports are done I'll shift focus to a potential Android TV client.

Update 1.3.2

I'm currently putting the final touches on update version 1.3.2 and will submit it to Apple tomorrow. Which means it won't be long before you can play with all the new toys that come with the update. You'll see the patch notes drop around this time tomorrow.

Midori: The Xbox Remote Play/Cloud play client

I've submitted the latest build to Apple today, and it's now just a matter of time before it gets released. I just had to bring the latest real-time 3D conversion tech to it for the Vision Pro users. I'll keep you posted on the approval status on the Midori subreddit r/MidoriApp, when it's released it'll be the same pricing model as Asobi with a big/deep discount for the Midori bundle unlocks on launch day.

Community

It's crazy to think that Asobi's only been around for 6 months is popping up in the craziest places on the web and in print. With now 10 members shy from reaching 1k members on this subreddit and 3 members shy from reaching 250 members on youtube I can only say that I'm humbled by the support!

I couldn't have done this without you! Thank you so much! 🙇‍♂️

I've still got great things coming, so stay tuned!

In closing

Please keep reporting your bugs, share your setup, and keep that feedback coming! Again: thank for your support!

Thanks for reading and game on! 🔥🔥🔥

/v

Read up on week 20 here

Hi,

Just a head's up, really eager to share this and could't wait till Sunday. But the realtime 3D conversion is getting a big update next week.

As I shared earlier > I've made some crazy progress on the realtime 3D conversion, so next update;

- Artefacts are gone
- No more depth slider needed
- No more mesh slider needed
- And running perfectly on 1080p 60fps HDR (4k will be the next target)
- And will be available on Midori (/r/MidorApp) on launch as well !
- It just works!

Can't wait for you all to get your hands on it!

More news this Sunday! Have a great weekend!

/v

P.S. Gallery has been updated! https://asobiapp.com/spatial-3d.html

Hi,

So I thought I'd give you an update on what's going on at the moment. Since not everyone's keeping tabs on everything.

So let's chat!

Midori

Midori is my current priority. It's been in the works for a while and I know many of you have been waiting. I'm doing final preparations and it'll be launching very soon.

You might notice some shared DNA between the two, but they both have their own paths forward.

Once it's out, I'll have much more bandwidth to push both Asobi and Midori forward with dedicated features.

Current ETA is next week (week 21)

Asobi version 1.3.2

It's currently delayed to week 21 (next week). I've found some interesting last minute features I'd like to add and there's still some finetuning to be done. So bear with me.

Ports

The ports of Asobi are coming, first one to release to their respective app store will be the Android version.

For anyone wanting to join the current closed beta for Android -> well I'm sorry but it's at capacity. I can add people to the list, but I've hit the limit of active users.

So to give you some level of what you can expect.

- Android version will come to the Play Store this month.
- The Steam PlayTest version should be up at the end of this month (aiming for it, might be a week later) with a full release in June.
- Last one will be the Meta Quest PlayTest version -> there's a slim chance it might be skipped due to shared DNA with the android version (sorry), But it does mean it'll have an earlier release -> I'm aiming to release it to the Meta Quest store in June.

In Closing

Hit me up if you have any questions!

/v

Hi,

Welcome to the 10th edition of the Report!

How's everyone? I've got quite some interesting stuff to share. I've made progress on Spatial 3D, which will get a big upgrade soon on Vision Pro, Remote Play updates in relation to Internet Play and stream stability and big updates coming to Cloud streaming. So let's start with that!

Cloud streaming/Library

So I've hit quite a milestone in the last week, every indication is there that you won't have to do any of these steps anymore to add games. After much research I've been able to get my full game library to show on Asobi, just like on the PlayStation Portal and this update isn't just coming to Apple devices, but to Android as well. So this will mean that new additions will show up at the same time as when it shows on the PlayStation Portal.

I've also been working on combining the PS5, PS4 and PS3 game libraries and to unify them. I'm planning to implement this when the new "Owned" game section has been updated.

Spatial 3D

You didn't think I was done with this I hope? I have a big update coming soon for Vision Pro users in relation to the realtime spatial conversion to 3D. I've managed to upgrade my ML model, so the quality and performance of the realtime conversion will be improved and more precise while keeping the wow factor. Incidentally, I've also managed to make it perform much better at 60fps compared to the current model.

I'm also one step closer to a much more improved out of the box experience where I can slowly clean-up the user interface. So less sliders, and more "let's go!"

After this is shipped, the focus will shift to solving those halo artifacts around foreground objects.

Performance

My main focus these days has been on performance. The focus is still "High fidelity & Low Latency", which means to keep the stream quality high, with low latency gameplay and removing any hiccups you have. To achieve this I'll be adding ways to share diagnostics with me when something goes wrong, an updated Performance Overlay and more tools to get to the bottom of your network setup. More on this soon!

I also added more support pages to the website and a dedicated page to explain how to read the Performance Overlay.

And for Mac users who can't change their wifi channels: check this page.

Ports

I've made quite some progress on the Android port, which I'm expecting to ship to the Google Play store within the next 2 weeks. I might still release one more update this coming week (week 20) to test internet remote play, so I'll keep you posted. After that I'll focus on the Steam and Meta versions.

New update incoming (Soon!)

As you might've noticed, there was no update last week. Some features I wanted to ship with version 1.3.2 needed a bit more thought and testing from myself. And I made quite some progress on the R&D side of Asobi, so I had to delay it to week 20. That said: This update will have added features to the Performance Overlay, improved FEC (which results in fewer recovery glitches after brief Wi-Fi blips), and improved streaming stability.

In closing

Please keep reporting your bugs and keep sending all that feedback! As always, thanks for all the support! You guys are the best!

Thanks for reading! See you next week!

/v

Read up on week 19 here.

Hi,

I've got a nice little preview for you!

Since the introduction of the new Performance Overlay it's become much easier to figure out what issues what you might be running in to.

So I decided to add some more info to it to be able to identify issues even better. When the next update releases (version 1.3.2) you'll be able to see the jitter (the wobble in the net stream) of the connection, what you're doing (Cloud play or Remote play) and which server you are connected to (e.g. London server or PS5)

A lot of extra info to figure out what's going on.

FYI: A webpage is up which explains the values: https://asobiapp.com/performance-overlay.html

I hope you guys will like this!

/v

btw: there's a chance that version 1.3.2 is delayed to after the weekend.

Hi,

Welcome to the 9th edition of the Sunday Report! Hope you all had a great weekend. Exciting times ahead!

It's going to be a short one this time around, since I'm quite tired from my weekend in London. Shame I'm heading back home tomorrow, had a great time! Hopefully I'll get to see some of you guys next time I'm here!

Asobi: Remote Play

As you may know from the latest release (version 1.3.1), the focus is currently on performance optimisations and bug fixes. Plan is to release some more performance enhancements with version 1.3.2, which should come available at the end of week 19.

For those still having issues connecting via Internet Remote Play: I haven't forgotten you: I have thought of a couple of methods to reach parity with the official PS Remote Play app. So the goal would be that if it works there it should work for Asobi.

After that I'll start adding more new features to the app. I've got quite some exciting stuff coming!

Cloud Play

Big changes for PS Now and PS Cloud are incoming!

I have planned out a way to unify the PS Now and PS Cloud libraries. This means that soon you won't have to switch between those two modes. I also managed to get one step close to get full parity with the PS Portal library (actually just one step to go). So if a game is available there, it should also show on Asobi with that update

PS Now support will also get some more love, so more people will be able to connect to the servers that support those games.

Next to that there's a couple of under the hood enhancements coming that are already in place with the Android version, which will make connecting blazing fast.

Ports

I expect to release an update next week for the Android version which will feature most of the Asobi core experience. Once that one's done, I'll focus on getting it ready for a full release. If you run into any issues on the Android version then please let me know!

The coming week is going to be a little busy for me, which means that I won't open up the testing yet for the Steam and Meta quest versions just yet. That said, they were approved for closed testing. I'm going to see if I can make the play tests available in week 20.

The Future

There's still a lot to do for Asobi to get it into the state I've envisioned for it. I just hope you all will like what's still to come. Split screen Remote Play, new view modes, screen filters, custom controls, macros to stuff never before seen or thought possible.

Thanks again for all your support and for coming along for the ride! 🙌

Keep your ideas, feedback and kind words coming! It's well received and appreciated! 🙇‍♂️

Game on! 🔥🔥🔥

/v

Hi!

Asobi version 1.3.1 is available now for iPhone, iPad, Apple TV, Mac and Vision Pro for download/update.

I hope this fixes some of the freezes/skips you guys were having. Let me know how it's going!

See release notes;

https://www.reddit.com/r/Asobi/comments/1t055mz/asobi_version_131_release_notes_the_one_with/

Have a great weekend!

/v

P.S. LONDON BABY! 🔥🔥🔥 (Cue the Clash 🎶)

Hi,

I've been hard at work this week on network and video optimisations for Asobi. The result is that his should help most of you with any freezing of the video stream.

Please keep in mind that 'competition' latency mode will still have a high probability of creating freezes and frame-drops if your network is not completely stable.

I expect the update to become available before the weekend.

Changes

# Remote Play

- Networking optimisations

# Cloud gaming

- Networking optimisations

- Added Experimental settings

# General

- Performance Overlay optimisations

# Platform Specific

- (visionOS) Updated Interface for Curved Screen & 3D mode

- (visionOS) 3D Spatial button is now hidden when resolution is set higher than 1080p

- (visionOS) Added game meta data to Spatial Screenshots and Video’s

- (visionOS) Immersive mode optimisations

Platforms (& Apple review status)

• iOS - Out now (May 1st)
• tvOS - Out now (May 1st)
• macOS - Out now (May 1st)
• visionOS - Out now (May 1st)

Be nice and game on! 🔥🔥🔥

/v

For the release notes of version 1.3 click here

Hi,

So I just pushed an update for the Android version of the PlayTest, which should be available now. Let me know what you find in this test.

Changes for version 0.5

# PSCloud

Unsupported countries should be able to connect now.

# General

Multiple UI/UX improvements

Fullscreen fixes

Added on screen touchpads for controllers without them (PS Portal style)

Added Performance Overlay which shows the following;

- (Total) Total latency,
- (Net) Connection latency,
- (Visual) Visual latency,
- (Res) Stream resolution
- (RTT) Network round-trip time
- (DT) Hardware decode time per frame
- (FPS) Decoded frames per second
- (BT) Incoming stream bitrate
- (VL) Video packet loss %
- (Drops) Decoder queue drops

There's still some space left in the PlayTest; so if you want to join, then please DM me your Google Play email address. (make sure your device is Android 13+ with a 64bit processor)

And then follow the instructions written up here: https://www.reddit.com/r/Asobi/comments/1sua1cn/playtest_asobi_remote_play_android_playtest_has/

I'm really curious about the latency numbers you're getting, on Remote play and PS Cloud. So please do share! When you share anything in relation to PS Cloud I'd also like to know the country you're connecting from (maybe DM me that for privacy reasons).

I'm kinda expecting it to perform better in relation to latency than any other Remote Play app out there.

For those in the test, enjoy! More soon!

/v

p.s. no news yet on the Steam version.

Hi,

Welcome to the 8th edition of the Sunday Report! (Time flies)

Let's get to it!

Performance

Let's get the most important thing out of the way first. Now that the performance overlay is a thing, it's become a bit easier to cure issues for users, which has resulted in my focus shifting to getting rid of camera panning stutters and some bitrate congestions issues. Which means we'll see some improvements on frame pacing and network optimisations.

2D->3D Spatial conversion

Well as some of you might've noticed: the realtime spatial conversion mode has been released for Vision Pro. It's still crazy I got to this point already. Which in all honesty to me is a milestone for Asobi. That said, I still see room for improvement which you should see with future updates.

One thing I almost forgot was that about a year and a half ago I responded to a reddit post that I would continue building my own Remote Play app for Vision Pro, which said;

>Stuff I have planned;

>- Realtime 3D conversion if feasible (it induces latency, so not sure if this will be something we'd want although it'll probably look cool)
- Chat audio
- Shareplay
- Play over internet

And well 3 out of 4 are done. So let's see what I can do about bringing shareplay into the equation.

Also, in due time you'll see this feature pop up on other platforms as well.

Screen modes

So I'm looking into the holy grail of Remote Play gaming. Which means something along the lines of expanding the view on screen since remote play is stuck at 16:9, which is annoying since you don't want those black borders on the side. In the past, the solution to this would be to either stretch or zoom in on the action and this has been it for years now.

So not long ago I created the light spill mode to solve this (see video here) which in my opinion has been a fantastic solution for this.

However it's not the holy grail in my opinion. And as you can see for the image gallery attached to this post, I've done some explorations on making new expanded modes. These should all be added to Asobi on iOS soon to give you alternatives to the light spill mode.

That said, as you can see there's also an image of a potential holy grail mode. Especially now that we're living in a world with machine learning and neural engines I've started work on a Asobi specific ML package which would do realtime image expanding.

What the model will do is expand the view based on what's happening on screen. Which would mean that the game you're playing will become fullscreen without having to deal with essential information falling off screen or being stretched in any way.

Of course these things are difficult to achieve, take time and are resource intensive so I currently have no ETA on when this will be properly working. But as I see it, this will most definitely be the Holy Grail/End boss for playing on any device at any aspect ratio.

Asobi controller mount.

Some time ago u/N_schifty responded to a post of mine where he asked if I did anything with my magsafe/stadia mount idea -> which I responded to that I had zero time and should go at it if he wanted. And yeah, he actually did it.

And I actually printed a few of them as well. It works great! I am still gonna bother him of course to maybe make a couple of optimisations and maybe see if he can rename it to "The Asobi Mount". Who knows, maybe we'll see some Asobi merchandise come at some point.

Either way, big shout out to u/N_schifty for bringing it to life!

Porting Asobi

Response to the Android PlayTest has been better than expected! So thank you all to those who are participating! Originally I planned to only have the PlayTest for this weekend, but I've decided against it. The plan's currently to let it run until version 1.0 has been released.

This will also be the case for the Steam and Meta versions. I am however considering changing the plan and do them 1 by 1, or maybe do some kind of waterfall method of play tests. Last Friday showed me that it can be quite overwhelming to handle all the feedback coming in via the reddit post and DM's. So I'll get back to you guys about this on how I'm going to proceed.

The plan is however to update the Android play test version soon with a couple more options and features until it has parity with the core Asobi version.

So more on this soon.

Sunday Report

As I said earlier, this is the 8th edition of the Sunday Report. To be honest, I'm actually surprised that I still have something to write about, but here we are!

So there's 2 things:

1. There might not be a Sunday Report next week due to scheduling, I have a meeting to go to in London next week which means I'll be either in Chinatown eating Korean BBQ or on voluntarily stuck on the Thames on a Uber Boat. I'll try my best to write something, but no promises.

2. Do you guys have any wishes? Stuff you'd like to read more about? More in depth? Other projects? Anything else? Let me know!

In closing I want to add that I'm grateful for all the support! Humbled even by all the support!

Be nice & game on! ❤️

/v

P.S. something green is coming closer. 👀👀

Hi!

Check out this really well done video by u/Jdunrrp ! He does a really great job at conveying the 3D effect (the 3DS shows what a hard sell 3D is, maybe Nintendo should've hired him back then)

🔥🔥🔥

/v

p.s. also check out the post; https://www.reddit.com/r/VisionPro/comments/1svftdz/i_just_play_playstation_games_with_spatial_depth/