Keep in mind that cavies have NO dexterity with their paws and the world is explored through the mouth. It's not like a cat who can reach out and bat something away. They are prey animals and are easily startled and will often reflexively nip or bite. Now, if they are clamping down and biting hard, that is a different story.
As others have suggested, it is important to move slowly and deliberately with cavies until they are absolutely secure with your voice and your touch. When I come into the room where my cavies live, I always make it a point to speak to them before I make any move to pet or pick them up. Be sure that your hands don't smell of food so that they make a mistake and lunge toward it.
I had a boy who was a real biter......and I mean biter! I knew where he came from and that he likely had never been abused in any way........and yet, he frequently bit me. I would tap him on the nose and say sternly, "NO......no bite". Since I so seldom raise my voice to my cavies, they know that something is wrong if I do. Eventually he stopped biting and became a "licker".......much more pleasant response. It took a long, long time to gentle him because he was a high-strung, nervous little fellow who had difficulty relaxing. When we had him out for lap time, he would "sing" at the top of his lungs! So loudly that we could carry on a conversation or watch t.v. As he got older and became more trusting, the loud singing became this little sing-song voice that was uniquely his.
Keep in mind that cavies are as different as we humans and that this little one just may require more time and patience to become totally trusting.