I've rewritten my code to make multiple, recursive calls to the server via the ?f=json and it's slow as all get out. Yuck!
So, I'm going to create a service acct with GIS admin privs and create a web service to wrap the call to the ?f=sitemap option. That way, the web service will ask as an admin and pass the data I need back much faster.
For gis websites that aren't "ours" (so we don't have admin access), we'll use the new code I've written.