Primer Not Adhering to Bathroom Wall?

I've been applying primer to my bathroom's formerly yellow walls with relatively good results until I noticed the paint over the toilet section of the wall wasn't adhering as well as the rest. I vaguely recall pulling off a piece of something or other sticking out and then, to my shock and amazement, the primer just kept peeling and peeling, taking what appears to be the previous yellow coat of paint with it.

Underneath lies a lighter shade of yellow. Perhaps this is a tinted primer? Anyway, I've tried sanding a small section of the torn area, wiping away dust with a damp cloth, and reapplying the primer, but as soon as the primer makes contact with the edge of primer/underlying mystery surface, it becomes soft and pliable once more.

I'm not really sure what the problem is. I don't want to peel off the whole wall of primer/paint if that isn't the problem. I'm thinking the primer reacted as it did because I applied too thick a coat in that area. Please help.

By Hannah from IN