I think you should have a moment, and plan something for the next year or so.
If I'm not mistaken, you mentioned that you have a patio - now, if the patio is small, and the perennial plants take a lot of space, see if you want to keep them or decide to get something different. if they take small space, and the patio is big, try and keep them.
I agree that plants are alive, and I never have thrown or killed a plant just because I don't want it, or can't take care of it anymore, I (personally) usually put plants that are sick, on the deathbed, or generally quite old or in bad shape, in the public garden near my house, or drive them to my father's place. I really don't want you to make decision just because of me, and my situation.
You see, I only have 2 quite small balconies in my house, which is located in the middle of the town, no space at all. Also, Job and university, create a situation that I can't afford to take care of the Garden every single day. Because of that, I try to have only plants that I really want, and plants that promise to be successful. I know it's a bit harsh, but don't get me wrong, when my cucumbers got sick and then got aphids, I took care of them until they were okay. I'm just saying, that if I were you, and the perennials could have bearly give another year, and they are in bad shape, I (again, personally) would think about moving them out, not killing them, of course not, but moving them to a different place, or trying to save them, but get them out of my small balconies.
Hope you'll find a solution that satisfies you, ask some other people on the forum, they probably have way more experience than me and you together...
Good luck.