A common statement:
“My problem with this is that by calling myself either feminist or MRA, I’m helping the outliers gain more power when I want them to have less.”
The way to handle that is to speak of the specific beliefs and positions that you take, along with describing your behaviour, that is, eschew using verbal shortcuts that end with -ist. The problem is that -ism is a bucket usually filled with a person’s best intentions, rather than elegantly and lucidly stated positions. By speaking of what you think and do, you divorce yourself from those who claim a label while simultaneously putting forth your version of what you think a better world looks like in actionable terms.