Try to detect the error from the browser DevTools console.
for example on Google Chrome pushing the F12 button from your keyboard opens the following console
in your case you can select the Network Tab in order to analyze the request to your Portal when your are truing to update the Service Definition.
try to add the domain and monitor the request on the Network Tab. When the error occurred something like the following expected
select the updateDefinition request and get the detailed info from the response Tab.
Also the same massage can be found on the ArcGIS Server Manager Logs