We seem to have similiar problem with a customer that wants to change from server 10.3 to 10.5.1.
After installing Server 10.5.1 on a new virtual machine the performance of dynamic map-services has decreased.
The datastore (fgdb's) and published map services are exactly the same. Yet on the old server (also a vm) images from map-services are created twice as fast.
We tested both vm's perfomance with the linpac test and microsofts diskspd tool. The new vm performed significantly better in all tests.
The only difference between the servers is the operating system. The old one uses Win Server 2012 and the new one Server 2016. Otherwise the servers are quite the same. No additional software like databases, plenty of unused ram and pagefiles in the 8GB range.
Heap Sizes for ArcGIS Server is 256 MB in both cases.
Is there something special to the 10.5.1 version in regards to performance limiting factors as opposed to the 10.3 version?