Try this- copy the ServiceReference.Config to the Viewer.xap file . In order to add the file you need to first rename the xap file to zip file and copy the ServiceReference.Config to the zip file and then rename it back to xap file.
Alternative option is to follow the link below. This is common to any MEF application and not ArcGIS Viewer for Silverlight. http://weblogs.asp.net/manishdalal/archive/2009/02/23/silverlight-servicereferences-clientconfig-alt...