Lemme explain all the problems that you may be having:
1) If you are trying to give them to yourself, that's impossible.
2) If the person you are trying to give/take away points with has no - or + button, that person probably disabled the feature.
3) If you have given/taken away a PridePoint within a twenty-four hour limit with one person, you cannot increase/decrease it again.
Hope this helps