Quality Over Quantity
If I haven't interacted with the person, I check the profile and see if there are common interests or follows. It seems that the majority are 'follow trolling' or driving up the number of Twitter followers they have. Maybe I just don't get the 'like me and I'll like you back' phenomenon, but I really don't want to add to my feed with somebody whose purpose is to appear popular.
One person who I share an interest with seems to be of another type that I don't understand: like every single artist she likes who happens to be on Twitter, then hound them endlessly with tweets, about half of which are of the 'please follow back' nature. While many artists enjoy the ability to interact with fans, I'm sure they don't want half of their feed flooded with those types of messages!
Now that I'm back tweeting fairly regularly, reading more than tweeting actually, I've been pruning who I follow, but I'm looking forward to taking a bit of time during the semester break to really look at my list and dump the ones that never tweet, the ones I followed for a contest entry and don't really tweet anything that is relevant to me and probably half the celebrity follows I did when I first joined Twitter back in 2007.
Of course, this means my spam inbox is going to be flooded with 'We can give you 5,000 followers for $5!' emails!