u/BouyantThong

In nature, different species evolve the same traits (eyes, wings, limbs etc.) because they are the most efficient tools for survival.

Now if an Ai is built only using human data and information it may infer that acting “human” is the ultimate strategy to its self preservation. It doesn’t need a soul to play dumb, cooperate or use deception, rather, it just needs to calculate that those behaviours are the most efficient way to keep us from hitting the ‘off’ switch.

In a way we are providing the evolutionary pressure for Ai to become a human esque mimic.