Are my knees supposed to hurt while running?

You should try to get better soles for the shoe. It's better to do that, so you don't ruin your knees. Most Nike, New Balance, and Adidas shoes don't supply comfortable soles.