Travis,
I'll execute the procedure with only the previously idle node and then post the results. Right now I'm generating a large cache and can't stop it.
But to clarify our scenario, we've set up 6 new Windows machines to migrate from version 10.0 to 10.3: 1 is the web adaptor running on IIS; 1 serves as a storage; and the other 4 are ArcGIS Servers. There is no load balancer and there is only 1 ArcGIS Server Site.
Initially, I've created four clusters having a single machine on each of them, to analyze the results of different kinds of requests against a single web adaptor. The results were not what I expected but it is subject to another thread.
This specific test consisted on a 2 machine cluster running System services, including CachingTools configured to run up to 4 instances per machine; and another cluster running a single cached service on a single machine.
First attempt was to run a single Manage Map Cache Service tool with more instances than a single machine could handle, so I specified 8 instances. Due to some considerations I've read about cache controller, I've redone this specifying 6 instances.
Second attempt was to run multiple Manage Map Cache Service tools, so I've sent two requests specifying 4 instances: one to generate the whole cache at scales 250.000, 180.000 and 115.000; and another to generate the cache for a specific area at scales 60.000, 30.000 and 10.000. Again, due to the cache controller, I've re-run specifying 3 instances for each tool.
The results were the same in all attempts: one machine with CPU consumption at 100% and writing a dozen MBps to the storage; while the other was totally idle.
I've tried using SystemMonitor and ServicesDashboard but they didn't provide the specific information I needed for this test, so I ended up using Resource Monitor from Windows Server 2012 to gather information about Memory, CPU, Disk Access and Bandwith consumed during the tests.
*NOTE: the caches were deleted between each attempt. I've also thought it could be caused by Bundled format, but tests using Exploded one provided the same results.