WPF proxy authentication

[LorenCress]

Google tells me that I'm not the only one facing this problem: we are building a WPF application using the Esri WPF API, but in order to consume REST services we have to go through a proxy.  That would be fine, except that the proxy requires authentication.  All of the WebClient libraries support the Proxy object, which in turn supports a Credentials object.  The Esri API classes, though, only offer me a "ProxyURL" string, and a separate Credentials object, which doesn't appear to do anything for the proxy; I wouldn't expect it to.  That object seems to be for providing creds to the REST endpoint in a secured ArcGIS Server setting.

"ProxyURL" is obviously for supporting a "proxy page", which is not applicable for our situation - that's not what we're doing.  We'd just like to consume simple, insecure maps and services, like ArcGIS Online, through a forward web proxy.

One workaround that we've found is to build - in the application itself - our own tiny webserver/proxy that uses WebClient to pass http requests through to the company proxy.  That seems very kludgy, though. How can we get through the proxy without writing our own WebClient-based proxy into our application?

[JenniferNery]

Credentials can be defined by the proxy administrator for both token and Windows/HTTP secured services.

Source: http://blogs.esri.com/Dev/blogs/silverlightwpf/archive/2010/02/15/How-to-use-secure-ArcGIS-Server-se.... When you download ProxyPage, kindly see proxy.config file, there are examples there.

[LorenCress]

Thanks for your reply, Jennifer.  However, as I stated, this is NOT a silverlight application.  This is WPF, a thick-client desktop application.  There is no web server to server a proxy page from.

See this comment on the page you linked.  It says :

[...] this will not work in WPF [...].

The services and tasks available in the WPF API offer only a string ProxyURL property, without any proxy authentication properties.  The Credentials property doesn't work for a proxy, it is for authenticating with secure ArcGIS services, not the proxy itself.

So - what options are available for WPF?

Edit:

For clarification, here is the summary:
1.  WPF desktop application - no web servers involved
2.  Proxy, requiring authentication to get anything from the internet
3.  Non-secured ArcGIS REST endpoints (in this case, we're just using ArcGIS Online)

[LorenCress]

Others on the forum with the same problem, unanswered and apparently unsolved:

http://forums.arcgis.com/threads/29498-ArcGISTiledMapServiceLayer-access-through-a-web-proxy.?highli...

http://forums.arcgis.com/threads/38348-Override-ArcGISTiledMapServiceLayer-proxy-authentication?high...

[RichardWatson]

When I look at the underlying ESRI code, it appears to me that they are simply passing the credentials you provide to the underlying Microsoft class which performs the HTTP operation.  The main class used is WebClient, though they do use other classes when the WebClient is insufficient.

So, I think that one approach is to look at which the Microsoft class does:

http://msdn.microsoft.com/en-us/library/system.net.webclient.credentials.aspx

My suggestion is to use Fiddler and see what the actual communication is between your client and your proxy server.  My guess is that it uses a standard challenge/response protocol.

I have no experience using a proxy server in the manner that you are but I assume that your proxy supports standard authentication protocols, i.e. it does not require one to log on via a web page (say oauth).  The credentials that you should provide are whatever credentials the proxy server requires.

Just trying to help.  Note that I am not an ESRI employee and often surprised to see posters (like you) with an ESRI badge using the forums to ask questions.

[RichardWatson]

Here is another option for you:

http://www.sharpgis.net/post/2012/01/23/Overwriting-the-default-WebRequest-used-by-WebClient.aspx

[LorenCress]

I couldn't figure out how to use Fiddler with an authenticated proxy, since Fiddler overrides the proxy settings itself.  However, I think I know what's going on anyway, for the most part:

client -> proxy: request
proxy -> client: 407 authentication required
client -> proxy: authenticate
proxy -> client: 200 OK
client -> proxy: request
proxy -> server: request
server -> proxy: 200/401/404/whatever
proxy -> client: 200/401/404/whatever

...or something like that.

Regarding overwriting the WebRequest, that would require access to the WebClient, which isn't there.

At this point I suspect that the API functions are using a standard WebRequest; however, if you were to create your own standard WebRequest behind an authenticated proxy, it will automatically pick up the proxy server and port, but doesn't automatically set the credentials.  That must be handled by the specific implementation.  It can be retrieved from default, but isn't automatically set.  The problem here is that since the API doesn't expose the WebRequest object (which in turn exposes the Proxy object, which in turn exposes the Proxy.Credentials object), we have no way to set the proxy credentials to respond to the 407 response from the proxy.

In the end, our solution was to build a small TcpListener-based proxy into the application, which will pick up the default credentials (or we can specify them) and pass them to the real proxy.  We then set the ProxyURL on all of the API objects to "http://localhost:port/" established by this local proxy.

This works, apparently quite well so far, but it would have been significantly easier if the API had allowed for proxy authentication to begin with.  On a side note, it's worthwhile to know that we tried HttpListener first - this was a bad move, as HttpListener has some serious security issues and requires administrator actions to make it usable on each client.  Not cool.  Anyway, I can post code for the proxy for whomever is interested.

[ERICUNDERWOOD]

Could you please post that code?

[LorenCress]

Could you please post that code?

What we used was based on what can be found at http://code.google.com/p/o2platform/source/browse/trunk/O2_Scripts/APIs/O2_WebProxy/ProxyServer.cs?s..., but since we aren't supporting SSL, we took out the https stuff.

    public class ProxyServer
    {
        private static readonly string THIS_PROXY_ADDRESS = "http://localhost";
        private static readonly int THIS_PROXY_PORT = 51212;

        private static readonly int BUFFER_SIZE = 8192;
        private static readonly char[] semiSplit = new char[] { ';' };
        private static readonly char[] equalSplit = new char[] { '=' };
        private static readonly String[] colonSpaceSplit = new string[] { ": " };
        private static readonly char[] spaceSplit = new char[] { ' ' };
        private static readonly char[] commaSplit = new char[] { ',' };
        private static readonly Regex cookieSplitRegEx = new Regex(@",(?! )");
        private static object _outputLockObj = new object();

        private TcpListener _listener;
        private Thread _listenerThread;

        private IWebProxy realProxy;
        private IWebProxy thisProxy;
        private ICredentials realProxyCredentials;

        public ProxyServer()
        {
            realProxy = HttpWebRequest.GetSystemWebProxy();
            realProxyCredentials = CredentialCache.DefaultCredentials;
            thisProxy = new WebProxy() { Address = new Uri(THIS_PROXY_ADDRESS + ":" + THIS_PROXY_PORT + @"/"), };
            _listener = new TcpListener(ListeningIPInterface, ListeningPort);
            this.DumpHeaders = false;
            this.DumpPostData = false;
            this.DumpResponseData = false;
        }

        public ProxyServer(IWebProxy proxy, ICredentials credentials)
        {
            realProxy = proxy;
            realProxyCredentials = credentials;
            thisProxy = new WebProxy() { Address = new Uri(THIS_PROXY_ADDRESS + ":" + THIS_PROXY_PORT + @"/"), };
            _listener = new TcpListener(ListeningIPInterface, ListeningPort);
            this.DumpHeaders = false;
            this.DumpPostData = false;
            this.DumpResponseData = false;
        }

        public IWebProxy Proxy
        {
            get { return thisProxy; }
        }

        public IPAddress ListeningIPInterface
        {
            get
            {
                IPAddress addr = IPAddress.Loopback;

                return addr;
            }
        }

        public Int32 ListeningPort
        {
            get
            {
                Int32 port = THIS_PROXY_PORT;

                return port;
            }
        }

        public Boolean DumpHeaders { get; set; }
        public Boolean DumpPostData { get; set; }
        public Boolean DumpResponseData { get; set; }

        public bool Start()
        {
            try
            {
                _listener.Start();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                return false;
            }

            _listenerThread = new Thread(new ParameterizedThreadStart(Listen));

            _listenerThread.Start(_listener);

            return true;
        }

        public void Stop()
        {
            _listener.Stop();

            //wait for server to finish processing current connections...

            _listenerThread.Abort();
            _listenerThread.Join();
            _listenerThread.Join();
        }

        private void Listen(Object obj)
        {
            TcpListener listener = (TcpListener)obj;
            try
            {
                while (true)
                {
                    TcpClient client = listener.AcceptTcpClient();
                    while (!ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessClient), client)) ;
                }
            }
            catch (ThreadAbortException) { }
            catch (SocketException) { }
        }

        private void ProcessClient(Object obj)
        {
            TcpClient client = (TcpClient)obj;
            try
            {
                DoHttpProcessing(client);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            finally
            {
                client.Close();
            }
        }

        private void DoHttpProcessing(TcpClient client)
        {
            Stream clientStream = client.GetStream();
            Stream outStream = clientStream; //use this stream for writing out - may change if we use ssl
            StreamReader clientStreamReader = new StreamReader(clientStream);

            //if (this.DumpHeaders || this.DumpPostData || this.DumpResponseData)
            //{
            //    //make sure that things print out in order - NOTE: this is bad for performance
            //    Monitor.TryEnter(_outputLockObj, TimeSpan.FromMilliseconds(-1.0));
            //}

            try
            {
                //read the first line HTTP command
                String httpCmd = clientStreamReader.ReadLine();
                if (String.IsNullOrEmpty(httpCmd))
                {
                    clientStreamReader.Close();
                    clientStream.Close();
                    return;
                }
                //break up the line into three components
                String[] splitBuffer = httpCmd.Split(spaceSplit, 3);

                String method = splitBuffer[0];
                String remoteUri = splitBuffer[1];
                if (remoteUri.Substring(0, 2).Equals("/?"))
                {
                    remoteUri = remoteUri.Substring(2);
                }
                remoteUri = Uri.UnescapeDataString(remoteUri);

                HttpWebRequest webReq;
                HttpWebResponse response = null;
                //construct the web request that we are going to issue on behalf of the client.
                webReq = (HttpWebRequest)HttpWebRequest.Create(remoteUri);
                webReq.Method = method;
                //webReq.ProtocolVersion = version;

                //read the request headers from the client and copy them to our request
                int contentLen = ReadRequestHeaders(clientStreamReader, webReq);

                webReq.Proxy = realProxy;
                webReq.Proxy.Credentials = realProxyCredentials;
                webReq.KeepAlive = false;
                //webReq.AllowAutoRedirect = false;
                //webReq.AutomaticDecompression = DecompressionMethods.None;

(continued below)

[LorenCress]

                if (this.DumpHeaders)
                {
                    Console.WriteLine(String.Format("{0} {1} HTTP/{2}", webReq.Method, webReq.RequestUri.AbsoluteUri, webReq.ProtocolVersion));
                    DumpHeaderCollectionToConsole(webReq.Headers);
                }

                //Console.WriteLine(String.Format("ThreadID: {2} Requesting {0} on behalf of client {1}", webReq.RequestUri, client.Client.RemoteEndPoint.ToString(), Thread.CurrentThread.ManagedThreadId));
                webReq.Timeout = 15000;

                try
                {
                    response = (HttpWebResponse)webReq.GetResponse();
                }
                catch (WebException webEx)
                {
                    response = webEx.Response as HttpWebResponse;
                }
                if (response != null)
                {
                    List<Tuple<String,String>> responseHeaders = ProcessResponse(response);
                    StreamWriter myResponseWriter = new StreamWriter(outStream);
                    Stream responseStream = response.GetResponseStream();
                    try
                    {
                        //send the response status and response headers
                        WriteResponseStatus(response.StatusCode, response.StatusDescription, myResponseWriter);
                        WriteResponseHeaders(myResponseWriter, responseHeaders);

                        Byte[] buffer;
                        if (response.ContentLength > 0)
                            buffer = new Byte[response.ContentLength];
                        else
                            buffer = new Byte[BUFFER_SIZE];

                        int bytesRead;

                        while ((bytesRead = responseStream.Read(buffer, 0, buffer.Length)) > 0)
                        {
                            outStream.Write(buffer, 0, bytesRead);
                            if (this.DumpResponseData)
                                Console.Write(UTF8Encoding.UTF8.GetString(buffer, 0, bytesRead));
                        }
                        if (this.DumpResponseData)
                        {
                            Console.WriteLine();
                            Console.WriteLine();
                        }

                        responseStream.Close();

                        outStream.Flush();
                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine(ex.Message);
                    }
                    finally
                    {
                        responseStream.Close();
                        response.Close();
                        myResponseWriter.Close();
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            finally
            {
                if (this.DumpHeaders || this.DumpPostData || this.DumpResponseData)
                {
                    //release the lock
                    Monitor.Exit(_outputLockObj);
                }

                clientStreamReader.Close();
                clientStream.Close();
                outStream.Close();
            }
        }

        private static List<Tuple<String, String>> ProcessResponse(HttpWebResponse response)
        {
            String value=null;
            String header=null;
            List<Tuple<String, String>> returnHeaders = new List<Tuple<String, String>>();
            foreach (String s in response.Headers.Keys)
            {
                if (s.ToLower() == "set-cookie")
                {
                    header = s;
                    value = response.Headers;
                }
                else
                    returnHeaders.Add(new Tuple<String, String>(s, response.Headers));
            }

            if (!String.IsNullOrWhiteSpace(value))
            {
                response.Headers.Remove(header);
                String[] cookies = cookieSplitRegEx.Split(value);
                foreach (String cookie in cookies)
                    returnHeaders.Add(new Tuple<String, String>("Set-Cookie", cookie));

            }
            //returnHeaders.Add(new Tuple<String, String>("X-Proxied-By", "esri-application proxy"));
            return returnHeaders;
        }

        private void WriteResponseStatus(HttpStatusCode code, String description, StreamWriter myResponseWriter)
        {
            String s = String.Format("HTTP/1.0 {0} {1}", (Int32)code, description);
            myResponseWriter.WriteLine(s);
            if (this.DumpHeaders)
                Console.WriteLine(s);
        }

        private void WriteResponseHeaders(StreamWriter myResponseWriter, List<Tuple<String, String>> headers)
        {
            if (headers != null)
            {
                foreach (Tuple<String,String> header in headers)
                    myResponseWriter.WriteLine(String.Format("{0}: {1}", header.Item1, header.Item2));
            }
            myResponseWriter.WriteLine();
            myResponseWriter.Flush();

            if (this.DumpHeaders)
                DumpHeaderCollectionToConsole(headers);
        }

        private static void DumpHeaderCollectionToConsole(WebHeaderCollection headers)
        {
            foreach (String s in headers.AllKeys)
                Console.WriteLine(String.Format("{0}: {1}", s, headers));
            Console.WriteLine();
        }

        private static void DumpHeaderCollectionToConsole(List<Tuple<String, String>> headers)
        {
            foreach (Tuple<String,String> header in headers)
                Console.WriteLine(String.Format("{0}: {1}", header.Item1, header.Item2));
            Console.WriteLine();
        }

        private static int ReadRequestHeaders(StreamReader sr, HttpWebRequest webReq)
        {
            String httpCmd;
            int contentLen = 0;
            do
            {
                httpCmd = sr.ReadLine();
                if (String.IsNullOrEmpty(httpCmd))
                    return contentLen;
                String[] header = httpCmd.Split(colonSpaceSplit, 2, StringSplitOptions.None);
                switch (header[0].ToLower())
                {
                    case "host":
                        // since HttpWebRequests correctly add the "Host" property anyway, and the
                        // requests from the ArcGIS WPF API change the "Host" to the ProxyURL, we
                        // don't want to set that here.  It will just end in tears, I know it.

                        //webReq.Host = header[1];
                        break;
                    case "user-agent":
                        webReq.UserAgent = header[1];
                        break;
                    case "accept":
                        webReq.Accept = header[1];
                        break;
                    case "referer":
                        webReq.Referer = header[1];
                        break;
                    case "cookie":
                        webReq.Headers["Cookie"] = header[1];
                        break;
                    case "proxy-connection":
                    case "connection":
                    case "keep-alive":
                        //ignore these
                        break;
                    case "content-length":
                        int.TryParse(header[1], out contentLen);
                        break;
                    case "content-type":
                        webReq.ContentType = header[1];
                        break;
                    case "if-modified-since":
                        String[] sb = header[1].Trim().Split(semiSplit);
                        DateTime d;
                        if (DateTime.TryParse(sb[0], out d))
                            webReq.IfModifiedSince = d;
                        break;
                    default:
                        try
                        {
                            webReq.Headers.Add(header[0], header[1]);
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine(String.Format("Could not add header {0}.  Exception message:{1}", header[0], ex.Message));
                        }
                        break;
                }
            } while (!String.IsNullOrWhiteSpace(httpCmd));
            return contentLen;
        }
    }