I'm not scared of speed (my own) or steeps. What DOES scare me are other people, many of whom are skiing/boarding too fast, without control, and often unaware of the skiers' code. (I've far too often heard approaching skiers who hit or almost hit a downhill skier say "You got in my way"!) This to the point that I'm tending to limit my ski days to non-crowded ones. Non-holiday ones. Non-school-break ones. And days when I can get off the groomers. I'm glad I x-c ski and have other options.
^^This. I came inches of being taken out yesterday. The holiday crowds are beginning to build, and with them, people who haven't a clue of what they're doing. A young man who was totally out of control fell (behind me) and went careening down the hill, nearly knocking me down. I don't think I've ever had that near a miss. Very, very scary.