Athletes usually are on scholarship. Just like the band, until they start getting paid (in tuition or other form), they have the perfect right to take their breaks.
Part of it, is that if they live on campus they have to find another place to stay, since the dorms close. I'm not sure what the athletes do.