Well, Callahan has to be my choice out of those two. Sure, the wounds of Bo are wide open right now and bleeding. But, Callahan simply was never the right fit on any level with Nebraska. And, there is a chance that if Callahan was never here, we would have had a successful coach during that time and Bo never would have been hired then in 2008.
Now, if you really want to dive into what era I wish never would have happened, I would go with the entire 1940s and 1950s. There was a stretch of years from 1942 to 1961 where we had a 68 - 120 record. That's a 36% winning percentage. The only reason why it's even that respectable is because between 1949 and 1955 we almost had a 50% winning percentage. The rest of the years are absolutely pathetic.
Why do I even care about years back then? The stat I found has us currently 41 wins behind Michigan as the winningest program in history (I'm not sure if all the results of this year have been added into this). We would have simply had to win slightly more than two more games per year (on average) during those 20 years to be where Michigan is at the top of the list.
As frustrating as both the Callahan and Pellini eras have been, it could be way way way worse with much more of a negative influence on how we are viewed historically as a program.