To access an ArcGIS Server site on a different domain than the Silverlight application host site, the ArcGIS Server site must enable remote access to services. This is accomplished using clientaccesspolicy.xml or crossdomain.xml. In most cases, just place the XML file at the root of your website (for example, for IIS, C:\inetpub\wwwroot\).