Where coaches get in trouble is not adapting to changes in the game or stubbornly sticking to schemes or players who cannot put the team in position to win, e.g. Rex Ryan. Some coaches get stale at their job like everyone else and need to move on, and sometimes they move to a new team and find their stride again, you never know.
That said, I think Frazier is doing about all he can with what he has. Average QB at best, 1 veteran receiver and a bunch of young guys, and a horrible secondary does not translate to a 10-6 team...his coaching staff did a great job last year. Now if Ponder continues to have 4 turnovers every game and they don't pull the plug within a month, then I'm going to join in calling for...gulp...Cassel.