I'll probably catch some grief for this, but here it is:
I've been a Duke basketball fan almost as long as I've been a Husker fan. Don't get me wrong, if it ever came to a NU-Duke game, I'd be there in red. When I was little I was accepted into the "gifted and talented" program in our schools, which was associated with program from Duke. So I think that's where it all started.
Also, I now root for some of the teams that NU has played in the past, especially if they have had good fans come over to our board (killer cacti, some VaTech fans, etc). It's not impossible as long as you're not a fair weather fan about it.