On weekdays, my interactions with my pigs are as follows (I'm out of the house from 8:30am and don't get back until 8:30pm):
1. Wake up, spot clean the cage (how thoroughly I do this depends on how pressed for time I am), give them some veggies. Refresh their hay pile. Pet them on the head a few times.
2. Come home from work, give them a few more veggies, spot clean the cage.
3. Give each of them 15-20 minutes of lap time. They live separately, so this is a time sink, unfortunately. Weigh them, pet them, give them some veggies, etc.
4. Their cages are in the living room, so I hang out on the couch and do a little more work or some Internet-y stuff.
5. Before going to sleep, give them some more hay, refill their pellets and water, if necessary.
This comes out to be about half an hour in the morning, and maybe 1-2 hours when I come home (this includes direct interaction and passively sitting nearby). Weekends are more of the same, but I give them about an hour of floor time each day where they hang out in my room (separately, of course).
As mentioned, as long as their cage is large, they have a plentiful and diverse diet, and you clean their cage regularly, floor time is more of a luxury than a necessity. They won't be any worse off and if they have company, they'll be fine. Now, when my girlfriend visits, they get spoiled like there's no tomorrow.