We use a
Shark Navigator (NV105-26) widely available from Amazon, Home Depot, BBB, BJ's etc. Not expensive and does a great job on the hay...as well as general use in the home.
https://www.amazon.com/Shark-Navigator-Upright-Vacuum-NV105/dp/B00LLHXHH4
If you are using one of those corner litter pans, then that says it all. I have never had a cavy that would use it! They are too small and seemingly intimidating to cavies. You need a pan large enough for an adult cavy to stretch out and move around in it. I use what was marketed as a kitten litter pan......it's just a small plastic pan with 4" sides. If the litter pan is large enough, then your guys can easily leap into the pan. My litter boxes (one in the loft, one on main level) also have a layer of hay and the boys love being able to do two things at once. Sparky loves the litter pan SO much that he will often sleep in it for extended periods of time. He has a white belly so the litter box has to be kept very tidy to avoid his belly and bum being soiled.
If you can't find a suitable plastic pan, you may want to try the one(s) pictures below. Be aware that they are a sort of compressed cardboard that will eventually need to be replaced. I often used a small puppy pad to line the litter box and extend its life.
View attachment 83970
View attachment 83971