If the average fan doesn't have enough leverage to send a message by threatening to withdraw support, as you say, then the average fans also does not have leverage to send the same message by actually withdrawing support. No matter how you look at it, if your goal is to improve the team, you'd do better by staying engaged with team.