Go to Fenny Stratford. There are a few pubs there, some with car parks, and it is about 20 minutes walk from the ground. When you leave after the game, drive in the opposite direction to the stadium on Watling Street, which takes you down to the A5. Should be fairly straightforward from there...