[5e] CE Wizard dropped an AoE on our melee party, getting 2 PCs killed. DM says "resolve it in-game". Is it an overreaction to attack or banish him?
The Context: I’m playing in a long-running campaign where the DM explicitly allows PvP. One of the players is running a Chaotic Evil Wizard. We’ve already had a major red flag with him: we were trying to send evidence of demons to a Paladin order, and the Wizard flat-out attacked us because he wanted to keep the evidence for himself. We handled it in-game and supposedly resolved the issue.
The Incident: Fast forward to our last session. We were fighting a very powerful enemy, and four of our party members were engaged in melee with it. The Wizard decided to drop a fireball right on top of the boss, hitting all four of our melee fighters. The DM allowed it, and everyone took the damage. Because of that friendly fire, two of those characters ended up dying during the combat.
The DM's Stance: The DM says that pvp is allowed and we need to solve it in-game.
My Question: Before anyone suggests it: leaving the group is not an option I am looking for right now. I want to handle this exactly how the DM requested—within the narrative.
Given the circumstances, how would your characters naturally react? Would it be going too far if the surviving characters immediately attack the Wizard or forcefully expel him from the party? It feels completely illogical for our characters to continue trusting or traveling with someone who just got half the team killed for convenience, but I want to know if going full PvP retaliation is a justified in-game response here. How would you play this out?
EDIT for clarification: the DM is a first time DM and all the players except me are playing their first long campaign