Hi Rory,
Would you mind sharing the code snippet for what you are doing in the edit mode and toggling between edit and non-edit mode. Would be easier for me to debug.
Also, when you click on the 'delete' button on the AttributeInspector, I would suggest to close the InfoWindow(map.infoWindow.hide()) right away.