Would reward hacking "shut down" a brain?

So me and my friend u/the_syner believe in basically the Hedonistic Imperative by David Pearce but I wanted to take it a step further and not just eliminate suffering but maximize pleasure, like really maximize it. But I don't want just drugged out bliss, I want every deep form of fulfillment and having all needs met, and simulations of every pleasure inducing experience and every deep intellectual pleasure imaginable.

So I found the idea of "reward hacking" where a being like an AI figures out how to set its reward function to maximum. Now according to my friend this kills the being as it has no goals and even the micro decisions needed for neurons to fire cannot be made with no reward system, but I insist that electricity must keep flowing through their head and thus consciousness is generated, they're totally useless except as a consciousness generator. I had this idea in the context of orgasmium (making a material designed for bliss though usually characterized as only human level) and omegapoint which could make a mind capable of feeling all that deep stuff on a scale unimaginable to us, as the end stage of any civilization and they all converge on it (not realistic but assuming these things can be converged on like a convergent ideology, it's kinda a sociological clarketech).

So, there's some options to keep the mind runninng and idk why it'd reward hack if it meant death because then it would stop feeling the reward hacking. Also not sure if reward hacking gets me all my deep happiness but that could be added in and one could have things done to them instead of by them or their virtual body piloted ny AI. They could also have their neurons fired by AI systems forcefully. Now maybe you only get 99.99999% reward hacked and still retain your status as an agent, we don't know what the cut off is, how close you can get and how functional or dysfunctional such a mind would be. But even without it worst case is what people consider "suffering" will be vastly happier than our greatest highs.

Does something have to be an agent to be valuable? He thinks so but I just believe anything with consciousness is worthy of moral consideration and that agent or GI status is completely irrelevant hence why I value animals just as much as people (at least in theory, my broken human empathy still values humans higher unfortunately, one of the many things I'd change about myself with psych modding technology).

reddit.com
u/firedragon77777 — 5 days ago

Would reward hacking "shut down" a brain?

So me and my friend u/the_syner believe in basically the Hedonistic Imperative by David Pearce but I wanted to take it a step further and not just eliminate suffering but maximize pleasure, like really maximize it. But I don't want just drugged out bliss, I want every deep form of fulfillment and having all needs met, and simulations of every pleasure inducing experience and every deep intellectual pleasure imaginable.

So I found the idea of "reward hacking" where a being like an AI figures out how to set its reward function to maximum. Now according to my friend this kills the being as it has no goals and even the micro decisions needed for neurons to fire cannot be made with no reward system, but I insist that electricity must keep flowing through their head and thus consciousness is generated, they're totally useless except as a consciousness generator. I had this idea in the context of orgasmium (making a material designed for bliss though usually characterized as only human level) and omegapoint which could make a mind capable of feeling all that deep stuff on a scale unimaginable to us, as the end stage of any civilization and they all converge on it (not realistic but assuming these things can be converged on like a convergent ideology, it's kinda a sociological clarketech).

So, there's some options to keep the mind runninng and idk why it'd reward hack if it meant death because then it would stop feeling the reward hacking. Also not sure if reward hacking gets me all my deep happiness but that could be added in and one could have things done to them instead of by them or their virtual body piloted ny AI. They could also have their neurons fired by AI systems forcefully. Now maybe you only get 99.99999% reward hacked and still retain your status as an agent, we don't know what the cut off is, how close you can get and how functional or dysfunctional such a mind would be. But even without it worst case is what people consider "suffering" will be vastly happier than our greatest highs.

Does something have to be an agent to be valuable? He thinks so but I just believe anything with consciousness is worthy of moral consideration and that agent or GI status is completely irrelevant hence why I value animals just as much as people (at least in theory, my broken human empathy still values humans higher unfortunately, one of the many things I'd change about myself with psych modding technology).

reddit.com
u/firedragon77777 — 5 days ago

Cameron is probably "weak"

So I'm officially a "weak Cameron theorist" I guess lol. I just think he's weaker than Jane by at least 0.5, he might be an 8.8 but 8.6 feels more right. Yeah he's "in a league of his own" but that doesn't mean much stronger than Orrin. Everyone talks about the barrier shattering like it's so crazy when Sera has always done that, I'd expect that from an 8.3 honestly, though doing it with lightning is impressive and might mean he can change the distribution of his amping and put more aura into specific stats, like amping strength and speed by 2 instead of 1.5. Now don't get me wrong I've always been an aura manip/channel master stan and firmly believed John could copy Sera, so I get the impulse to make Cam strong, but I think Uru is saving that for Jane, as he said basically "oh well *ShE's So StRoNg*" like she's stronger than him and he's secretly jealous. Also makes sense for there to be a 0.3-0.5 gap because that's when a fight becomes un-winable like John vs Sera or Val vs Sera or Sera vs Orrin. It gives plenty of room for Cam to be waaay stronger than Johm but Jane to also be unique and have something as OP as manipulating the abilities of others. In a fight Jane would just dampen Cameron or disable him instantly, no fight required just like Orrin v Sera, though maybe a barrier-like ability could stop her aura from messing with others. Also explains why Sera stumbled in the NxGen raid arc when she got near the converters, it's Jane's aura. Jane may be able to amp or dampen a certain number of stats on herself ir others, so for example she could amp all stats of an ability or all stats of someone else or 1 stat across 5 people, and can choose which stat for each of those 5. She also might be able to permanently disable abilities though maybe it takes energy to keep them that way, and forcing ability activation would be cool too but doesn't have any precedent in the NxGen tech, it'd really help with other aura manipulators. But Cameron didn't dampen anyone so I suspect he's weaker than Jane by a lot. But let's be clear even a 8.6 Cam is insane, truly mind boggling, worthy of the "in a league of his own" statement. Jane may also have truly permanent copying and not just "sets" for the day/week like Cam, as it's kinda implied she "collected" all the most powerful abilities in the world. Though I'd expect her ability is damaged and unreliable especially as she wasn't even supposed to be able to speak. And I think Jane is probably at least a little stronger than even Leon and the Ember/Authorities higher-ups, she's probably the strongest in the world or at least that we'll ever see, but I don't expect John to ever pass 8.5, maybe even 8.3 sorry John stans, that's still so much growth though! Also I still think time manip is physical, there's no way it's not, it's gotta be that he can't copy abilities stronger than him otherwise that's just stupid writing with a poor understanding of physicality no offense Uru. Like many abilities like telekinesis and that digging ability Cam used don't have anything overtly physical but still habe very visible affects, if punching through walls isn't physical idk what is and all quantum abilities should be impossible to copy. Now Cameron didn't copy time manip but he also didn't copy barrier or particles or anything at all so idk, he probably wasn't impressed by anyone's abilities not even Sera and figured lightning was all he'd need. Dampener and disabler immunity would also be cool though, and it seems backed up by what we've seen with John.

Also side theory, I bet this means Ember is gonna have Specter tech now, just imagine them with dampeners and disabling superheros and making them into forced Ember agents hunting down their own kind. I'm guessing we're gonna shift away from Specter after this half of the season, still gives us room for an Orrin capture arc with Seraphina stealing him back.

reddit.com
u/firedragon77777 — 13 days ago