u/Suspicious-Sir946

I built this farm in 26.1, every so often golems will stop spawning completely and in order for me to get them to spawn again I need to wait until it is night, close the trap doors to break line of sight and then open the trap doors again. Is this just a problem with the villagers not being able to sleep? If so how do I fix that? Could it be something different?