Most likely the software is locking up because it is trying to talk to the plotter that has gone off in the weeds. If you wait long enough after hitting cancel, the software will eventually time out and give up trying to talk to the hardware and start working normally again.
When you say the cutter resets, does it go back to home and stop? Typically when static is the problem the cutter runs away and feeds vinyl till you stop it, often times with the knife down and cutting a nice long slice through your good vinyl.
Everyone talks about the memory issue, I have a MH and I've never had a problem and have cut some designs with a ridiculous amount of nodes. If the software is working correctly the amount of memory on a cutter should be irrelevant. Software feeds the data to the cutter till the memory buffer is full, then waits for the cutter to request more. In theory, a cutter could work with just a few bytes of memory, it'd be extremely inefficient, but it could be done.
I would try the pen thing mentioned by skeeter first, and if it still behaves weird, then try different software. If it behaves weird with no blade, and different software there's a good chance it's a hardware problem.