Would you have let Marty McFly remember?
Let's assume, from the end of the first Back To The Future film, Marty continues to remember the life he had before he travelled back. He's definitely confused when he sees the different decor, his siblings and how his parents behave.
They will remember different holidays (vacations) and family events from the ones he does. A pretty horrible way to live. Would you have written it to end that way?