To De-friend or not to De-friend?

So, lately I've been toying with the notion of going through my FB friend's list and chopping it down. I mean, WAY down. At the same time, would I be hurting people's feelings? What if they remember me and I have no clue who they are? What if they barely have any friend's and would notice instantly if I were to de-friend them? I hate being de-friended even if I don't know the person (then again, I am way too sensitive for my own good). Of course there's one person out there I *should* de-friend, but I'm unable to bring myself to do so (and therefore find myself stalking his page way more than I'd ever admit just to see how he's doing). But, as for the rest of them...is it hurting them to keep them as friends? What difference does it make either way? Seems like more of a hassle than anything else - at this point I'd rather be lying down with an ice pack on my head (but that could just be the migraine talking).