You should remove your contact lenses from your eyes whenever you feel pain. The contacts could be inserted the wrong side, they could have a scratch, or maybe they just are not clean enough.
Most of the time contact lenses wearers are delighted with the level of comfort that contact lenses provide after a brief adjustment period, and most of the people don't even feel the lenses in their eyes.
Some others, really few among color contact lens users, might be allergic to the material used to make the lenses, or some discomfort can occurs when using the wrong base curve or when the solution that they are using to disinfect their contacts is not good enough, so they should visit a doctor to solve this problem.
Remember that these circle contact lenses are cosmetic products as eyeliner or foundation are, and whenever your cosmetics give you a rush, you should not wear them anymore.