I get what you are saying, but not all real fans can go to every game. So when you see an average gate of say 25,000 it is not the same 25,000 at every game.
But the basic idea is right, a club with a bigger average attendances will have more fans than that, so more will miss out.