What would you actually use an AI that can see your screen for?
I'm researching an AI tool that can understand what's currently happening on your screen and interact with you based on that context.
Instead of taking screenshots and explaining the situation to an AI, the AI could already understand what you're doing and answer questions about it.
I'm curious what people would actually use something like this for.
Would it be useful for:
Troubleshooting software
Learning/programming
Browsing the web
Gaming
Helping with creative software
Explaining things on your screen
Accessibility
Something I haven't thought of?
Privacy is obviously a big concern with something that can see your screen. The idea would include a privacy toggle that completely turns off screen analysis, plus a pause button for temporarily stopping it whenever you don't want it looking at your screen.
So it wouldn't have to be something that's constantly active — you'd be able to control when screen analysis is enabled.
What would make you comfortable or uncomfortable using something like this?
I'm trying to figure out whether there's a genuinely useful product here rather than building something just because "AI that sees your screen" sounds cool.