You'll often hear your doctor say words "failure to thrive," and failure to thrive really means your baby is not putting on weight, and that can be extremely dangerous in a newborn.
The way to spot it at home is to notice if your baby is not eating, not finishing feeds, and really not gaining weight. If you're concerned, you can buy a scale and measure your baby, or you can take your baby to your doctor as many times as you want to figure out if they are gaining the appropriate weight they should from the hospital.
Typically, babies will be born at their birth weight, drop a little bit of weight when they leave the hospital, and then gain weight at a rapid pace. If you're noticing that your baby is not gaining weight and not finishing their feeds, you really need to go to your doctor and check them out and figure out why they're not thriving.
Is it they can't tolerate the food, they can't keep the food down, or they don't want to eat? So it's really important to monitor that and be in close contact with your doctor.