I'm sorry you lost the mom.
No milk for the baby. The main benefit of milk for infants is the immune system support it offers, and since the mom has died, there's no way to get that. Other milks are no substitute.
Unlike kittens and puppies, guinea pig pups are born mature and can eat on their own. You can try cutting things in match-stick sized pieces and offering it to the pup. Probably on your lap would be best, rather than letting it huddle in the cage by itself. I'd start with lettuce and bell pepper, and maybe a slice of apple peeling. You want it to eat, and now's not the time to worry about its sugar intake.
You need a syringe to give it fluids, which you'll need to do several times a day until it learns to drink from the water bottle. Unflavored pedialyte would be a good thing to use, as it has electrolytes in it that the pig needs. If you don't have syringes, tell the pharmacy that you need syringes without needles to feed a motherless baby animal, and they'll probably sell you a few.
If you use the pedialyte, keep it in the refrigerator and don't let it sit on the counter after it's opened. Also wash the syringes very carefully, because that stuff molds in a hurry.
Don't worry about sexing the baby at this point. What s/he needs is comfort and reassurance, not someone poking at his/her private parts.
Let us know how the baby is doing. Be sure to weigh it morning and evening and record the weights. It will probably lose weight the first day or two, but should start gaining if you can get it to eat.