First of all I would suggest getting them dewormed because that could very likely be the problem.
Also I would take away the milk because many cats are lactose intolerant and the milk gives them an upset stomach, especially since they are so young. If you want to give them milk, get the lactose free one especially made for kittens at the pet store - NOT SOY!
Are you feeding her kitten food, or cat food...kitten food is usually better because it has all the necessary nutrients for kittens and is softer on their stomachs.
Another possible cause is anal gland problems. Anal glands are small glands located at the side of the anus and produce and contain a stinky liquid (used for territorial marking).
If your kitty has this condition, she may need some help in emptying these glands. You first need to get them to the vet to diagnose the problem, so you can figure out what to do. Here's a link with more info about anal glands :
http://cats.about.com/cs/healthissues/a/analglands.htm