These are generally going to be results orientated responses and if the weather is awful (and there's no cover as per 3 games this season) then you remember them much more.
My all time worst was Northampton away at the old ground (separated from the cricket ground by a rope). We drew them in the FA Cup when we were top division and not long after we'd won the FA Cup. It lashed it down for the entire game and we were so poor. They could smell victory and as usual against a relative minnow we duly obliged. Having rested Big Cyrille, he was brought off the bench to rescue us, only to an ironic chorus of "who the fkn hell are you" from the Northampton fans. We trundled out and remained soaked all the way home. I didn't travel to Northampton when it was our home ground for the season and I'm convinced it's raining every time I drive through Northamptonshire for any reason.
I have a few tales regarding the old grounds at places like Cardiff and Millwall away where we didn't fare too well, but that's more to do with us fearing for our health from their intimidation. Fine when you're in the ground giving it the 'big un', not so good when you realise that you have to head out in their direction to get your transport home whilst most of the others were by coach. I'd like to think I've learnt over the years, particularly as I can't run these days, but we occasionally still have our moments