Yes and no; if your employer has automatically enrolled you, there’s a window between 2 and 8 weeks after enrollment where you can opt out. Login to myIR or submit KS10 to the IRD.
If you have chosen to voluntarily join KiwiSaver through BetterSaver or a through a provider, then you are unable to opt out. However, you still have the option to take a contributions holiday (3-12 months). The is no limit to the number of contributions holidays that you can take.