Were Zant's invading army the Garudo.
Shadow Beasts are made out of people. And it seems they need to be in Twilight to do it. Before the Twili could invade Ordon village, they first wanted to plunge it into Twilight by taking the Spirit Light. This would turn the people into Spirits. The contrasting note here being that Shadow Beasts aren't spirits. So people need to be Spirits before they can become Shadow Beasts. We never see the desert under the curtain of Twilight. But it's safe to assume that the Light Spirit down at Lake Hylia would also have the desert within it's range. Hence why Zant returns there first, to keep the desert and the Mirror of Twilight under Twili control after Link returns the light. Furthermore, there are no Garudo in the desert. And in any timeline where the Garudo reject Ganondorf, he holds a grudge. So he'd want to make them submit. Hyrule Castle is not in Twilight during Zant's invasion. So based on this theory, Zant's army couldn't have been anybody from Castle Town or the Castle. But there's no other settlement between the Mirror of Twilight and Hyrule Castle... unless there was a Garudo settlement, and Zant turned them all into Shadow Beasts.