Hello,
When a callback URL error is being thrown, it is because the ArcGIS Server itself cannot connect back to itself on the URL specified in the error.
This can be caused by a number of things, but the best method I have found for troubleshooting this particular error is to run your browser as the user that is running your ArcGIS Server service. Then browse to the URL in the error (Linux or Windows.) The reason why this is helpful is this allows you to see what the ArcGIS Server would see when connecting to the ArcGIS Server URL specified in the error message.
For example I have seen two things cause this error in the last month, though there could be many more reasons in the same way that many things can network connections to fail for specific users. Both cases were diagnosed using and resolved using the above method. So far I have yet to see this error caused by a bad install or corrupt data, instead it's been environmental.
1) The proxy settings for the service account running the ArcGIS Server Service did not have the same proxy settings as other users on the domain applied to it. (Normal users were setup to have exceptions for local addresses, but the service account was not.
As a result, the proxy was interfering with the ArcGIS Server reaching its own address while other users could browse the manager page normally. So when attempting to start/stop a service, the callback URL error would be thrown as the server attempts to connect to its HTTP/HTTPS listener.
To change this we ran Internet Explorer as the service account. In Windows Shift+Right click Internet Explorer to bring up the pop up menu to allow us to select "Run as different user." (In Linux you could "su - " to the account running your service, and run your preferred browser.)
Since this case was Windows, we Opened up: Tools > Internet Options > LAN Settings > Advanced, then set our proxy settings to match the settings of the other users. In this case adding an exception for the URL of the ArcGIS Server.
2) The account running the ArcGIS Server Windows Service could not download the Certificate Revocation list for the certificate being used in ArcGIS Server.
So the effect was users could browse the manager page via https and could check that the CRL was fine and the certificate had not been revoked. However, when the account that ArcGIS Server was running under tried to check the CRL, it was not allowed to download the CRL as a result of Group Policy. When ArcGIS Server is setup to use HTTPS, if it cannot verify a Certificate Revocation list (assuming the certificate has one,) it assumes to error on the side of caution and not make the connection to the server with an unverifiable certificate.
Two additional tests to help resolve this issue:
1) Rerun the Configure ArcGIS Server account to have the ArcGIS Server run under your domain account, this way you know AGS has the same settings as you in terms of what you can browse.
2) If you are getting Callback URL errors that reference 6443. You can tell if it's HTTPS related by switching the HTTPS settings for ArcGIS Server back to HTTP only. If the problem goes away, it is something related to your HTTPS/Certificates.
Additionally, issues with HTTPS/SSL will also tend to have to be followed by (WinInet error code = xxxxx) If you google the WinInet error code, it often will tell you what is the problem .
I hope this information helps!