Not at all! Voluntary class 2 NICs are probably the best risk-weighted investment you can make in the whole world (not an exaggeration)!
You pay £180 for one year and for that you get £327 per year inflation adjusted (and triple lock protected). You make more than your money back in the first year!