If you are unemployed, you can do freelance work and these payments are deducted from what they pay you.
Just be careful about what the penalties are if you voluntarily quit your job. I don't think you can delay registering as you're supposed to do it before your job ends if you want to claim. If you are declaring yourself as self-employed this may not allow you to claim benefits. So, I don't think you could quit your job, try to set up your business, and then if it's not working claim benefits. You could perhaps quit your job, submit your claim and try setting up your business while unemployed. However, if you want to claim benefits then you'll be expected to be making applications. I was working part-time, freelancing and still expected to meet the required number of applications. The RAV does have a scheme where they give you 6 months or so to set up your own business (and during this time you don't have to make applications). I don't have more information on that. I suspect though that you would have to have a business plan etc and for that to be approved. Perhaps freelancing would be considered, but maybe it's more focused on people looking to form companies.