I have asthma myself and I've been dealing with it for about 10 years now. My fiance cleans the cage, and we use fleece (we used to use sawdust and hay). But we've found that the fleece actually helps because their hair and the hay both stick to the fleece. As long as I don't go near the cage I'm fine. It also might help if you pet them and give 'em a little grooming before the allergic person touches them.
But if you do have to rehome them - i'm so sorry. I hate it when my pets leave me
