Have you tried the new Parrot OS optimized repositories?
Parrot OS 7.3 has just been released and now ships whith opt-in repositories with several packages optimized for amd64v3 and arm64v8.2
the documentation page does not simply explain how to enable them, but also goes into the implementation details and describes how it was achieved https://www.parrotsec.org/docs/configuration/parrot-optimized-builds
has anyone tried it yet?