Can we enforce this verbal contract in NJ?

Verbal contracts are only enforeceable if you have strong evidence that such occurred, such as a tape recording of the verbal contract taking place. Your (multiple) errors were in moving forward 'on a handshake'. Throw this fool out of your store promptly and get back to what you were doing. You are being taken for a goodly ride here.