Ok, I'm married to a ski fanatic, and I'm a ski fanatic...but that was luck.
I think we all look for many of the right things for compatability, but I think we underestimate making sure our mate is well-suited to our Play Time. I would never be in a serious relationship, for instance, with someone who was into the beach. I hate the salt and sand, and i DETEST heat. As one gets older (yikes), one wants the companionship of someone more and more--careers wind down, "lots of friends" winds down, kids grow up and leave the house. I never want to look at my mate and think "now what??"
I love that my DH loves skiing as much as I do. We are together as many hours as possible (since i teach, i have lots of summer/xmas time), and going on two decades, we are more in love than ever before. Are we older? Yeah. Do I find romance in coming home to a clean house (rather than flowers and candy)? BIG TIME. Do I love coming home to find my DH has located a particulary cute ski jacket for me and bought it for me and it arrives, only to fit perfectly and i love it? YEAH! Am I a jacket slut because he does this ALL THE TIME? Yeah.
So no, it's not weird to me to break up with someone who doesn't share your passion. Isnt passion what it's all about??