It really depends on how large you are making your cage and how you want it to be structured. For example, I used the snap together grids for all four sides plus the bottom to make it sturdier. Since I have a cat I also fashioned a lid out of those closet/laundry room racks, but you could also use the grids from the storage cube kits to make a lid if you need one to keep your cavy safe from small children and/or other pets in your home.
A set of 14 panels at Target runs about $20. I had to purchase two kits for my cavy cage. Fleece can run up to $12/yard, although you can find fleece solids (the more expensive fleece fabrics are the prints and licensed prints such as Disney characters or sports teams) for as low as $5/yd.
If you prefer disposable bedding, Care Fresh runs about $20/ large bag. I used two a week in my cavy cage. I'm not sure how much Coroplast typically runs at the local sign shop, as I purchased my sheet for $7 from the rescue I adopted my cavies from.
All in all you should be looking to spend approximately $60+, but put it into persepctive. Many people happily drop that much money on those fancy shmancy hamster habitrails with all of the funky bells and whistles (ya know, exercise wheels that look like a princess crown or space ship and all that sort of buffoonery), so why not spend that much to provide a cavy with a wonderful home they will love?
Edited to Add:
When purchasing the cube storage units, make sure the grids are at least 9x9 squares per panel. Anything larger can cause injury or even death if by chance your cavy were to stick their head through the squares. I erred on the side of caution and purchased some from Target's children's storage department that were 36x36 squares, almost a mesh.