is it really too cold??

Yes, it's too cold. Per Guinea Lynx (I can't get the link open)

Choose a bright draft-free room with a stable temperature range between 65 and 75 degrees F (18 to 24 degrees C), out of direct sunlight,...
Temperature fluctuations can be very hard on your guinea pigs.

Click to expand...

If they're like my piggies, they will huddle and hide like you've just thrown them in a snake pit the moment you take them outside. Mine were so freaked out being outside. Maybe b/c they aren't use to it, maybe bc one of them had a very traumatic 'in the wild' experience. And since I, too, live in a cooler (so far) climate and can't take them out every day, I've given up on taking them out. Maybe I'll try again, but definitely when it's much warmer.

I'm also not sure that the deck is the best place for them. They may not like the feeling on their paws, it may open them to splinters, possibly getting one of their little paws stuck in the space between the boards and I kind of feel like it will make them sitting ducks for birds of prey. I put a net over their play pen the day I took them out as a precaution for the birds.

My guys go out on sunny days anywhere over 12C, about 55F? There's plastic around and over the cage so it warms up more in the sun. Cloudy days they stay in more.

We have a relatively mild winter, temps to -5C overnight and usually averaging 14 during the day, and there's not much difference between the house overnight and outside during the day, so I work more on how dry it is rather than temperature.