I seem to be running into a related issue. I have an RMD that has been in use for years in v10.3.1 as the basis for an associated image service. I did not modifiy this RMD in 10.5.1 (just viewed its properties), but I am getting the error message "Error opening mosaic dataset" as well as errors with the AGS image service.
As such, what operation on an RMD in ArcCatalog 10.5.1 would upgrade an RMD?
I do recall in a previous upgrade years ago when going from either 9.3 to 10.0 or 10.0 to 10.2, that if I opened the Description tab of a lyr file in the higher version, I was unable to use that lyr file in a lower version of the software as viewing the Description somehow upgraded the lyr file. The RMD seems to be working in the same way here, so it appears you cannot view the Description of the RMD in the higher version of the software without upgrading the RMD.