WebClient Object

Long time since the last post...

I've been working on a project that interfaces with numerous HTTP servers to update the firmware on the hosting machine. I opted to use the WebClient object, mostly because I wasn't sure what the best method would be but figured using WebBrowsers would just be wasteful, and ran into a few problems that were tedious enough to make me want to post them on here.

First thing's first, some servers respond back with an "Expect" error, equating to an exception from the WebClient. The fix:

Code:

System.Net.ServicePointManager.Expect100Continue = false;

Secondly, logging into a website. The WebClient object doesn't handle cookies for you and that sucks, a lot. The easiest way to fix that is by creating a new class, inheriting the WebClient class and overriding the GetWebRequest method.

Code:

class WebClientEx : WebClient
{
private CookieContainer m_container = new CookieContainer();

protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = m_container;
}
return request;
}

Credit for this goes to http://codex.bradrowley.net/

Ok, that gets you cookies but submitting to a form wasn't all that easy...

Code:

// Create the POST form parameters
NameValueCollection nvc = new NameValueCollection();
nvc.Add("authUser", username);
nvc.Add("authPass", password);

// Pause until m_client is available
while (m_client.IsBusy)
{
}

result = System.Text.ASCIIEncoding.ASCII.GetString(
m_client.UploadValues("http://...", "POST", nvc));

And finally, the last thing I had to figure out how to do was upload a file, via a form. Fortunately, I was able to find a piece of code from madmik3 that did the trick.

Code:

// Majority of this function from http://www.codeproject.com/KB/cs/uploadfileex.aspx
public string FormUploadFile(string uploadfile, string url, string fileFormName, string contenttype, NameValueCollection querystring)
{
if ((fileFormName == null) ||
(fileFormName.Length == 0))
{
fileFormName = "file";
}

if ((contenttype == null) ||
(contenttype.Length == 0))
{
contenttype = "application/octet-stream";
}

string postdata;
postdata = "?";
if (querystring != null)
{
foreach (string key in querystring.Keys)
{
postdata += key + "=" + querystring.Get(key) + "&";
}
}
Uri uri = new Uri(url + postdata);


string boundary = "----------" + DateTime.Now.Ticks.ToString("x");
HttpWebRequest webrequest = this.GetWebRequest(uri) as HttpWebRequest; // Changed by Austin 5/22/09 - To account for cookies
webrequest.ContentType = "multipart/form-data; boundary=" + boundary;
webrequest.Method = "POST";


// Build up the post message header
StringBuilder sb = new StringBuilder();
sb.Append("--");
sb.Append(boundary);
sb.Append("\r\n");
sb.Append("Content-Disposition: form-data; name=\"");
sb.Append(fileFormName);
sb.Append("\"; filename=\"");
sb.Append(Path.GetFileName(uploadfile));
sb.Append("\"");
sb.Append("\r\n");
sb.Append("Content-Type: ");
sb.Append(contenttype);
sb.Append("\r\n");
sb.Append("\r\n");

string postHeader = sb.ToString();
byte[] postHeaderBytes = Encoding.UTF8.GetBytes(postHeader);

// Build the trailing boundary string as a byte array
// ensuring the boundary appears on a line by itself
byte[] boundaryBytes =
Encoding.ASCII.GetBytes("\r\n--" + boundary + "\r\n");

FileStream fileStream = new FileStream(uploadfile,
FileMode.Open, FileAccess.Read);
long length = postHeaderBytes.Length + fileStream.Length +
boundaryBytes.Length;
webrequest.ContentLength = length;

Stream requestStream = webrequest.GetRequestStream();

// Write out our post header
requestStream.Write(postHeaderBytes, 0, postHeaderBytes.Length);

// Write out the file contents
byte[] buffer = new Byte[checked((uint)Math.Min(4096,
(int)fileStream.Length))];
int bytesRead = 0;
while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
requestStream.Write(buffer, 0, bytesRead);

// Write out the trailing boundary
requestStream.Write(boundaryBytes, 0, boundaryBytes.Length);
WebResponse responce = webrequest.GetResponse();
Stream s = responce.GetResponseStream();
StreamReader sr = new StreamReader(s);

return sr.ReadToEnd();
}

Variable arguments in C#

Ever used variable arguments in C++ and wanted to use them in C# but you could never figure out how? I certainly did a few weeks ago, I was working on a function similar to some of the old standard C IO libraries (though I can't recall what it's purpose was to be) and I spent quite a bit of time trying to learn how, never actually finding anything. Well I just stumbled upon it here at the MSDN C# FAQ page.

If you're like me and would rather just see the code...

Code:

void paramsExample(object arg1, object arg2, params object[] argsRest)
{
    foreach (object arg in argsRest)
    { /* .... */ }
}

Submitting forms on the enter key

Quite a while ago I was working on a form with a few textboxes and a submit button. I was nearly complete with the application but it didn't do one thing that I wanted, I had to either tab to the submit button and hit enter or click on it with the mouse. I'm fairly lazy and I know most people are too, so this little problem would most likely annoy quite a few people.

Well, after searching on google for a few hours and going over my past code (turns out I had found the solution a long time back, then forgot) and came up with the solution.

Code:

this.AcceptButton = button_Submit;

Seems simple enough, the real trick is to change the AcceptButton whenever you have a different form you want to submit. Then all of your various forms will respond accordingly and you don't have to attempt to handle any key downs or other messages to mimic this behavior.

Posting form content with the WebBrowser control

A while ago I was working on one of my many side projects and needed to be able to login to a website using a WebBrowser control. The form used Post rather then Get, so I wasn't able to simply append the appropriate form elements onto the URL.

I did some searching online and ended up with a nice little method to Post form content to a site by sending the correct headers in your Navigate call.

Code:

private void Post(string szUrl, string szPost)
{
    Encoding a = Encoding.UTF8;
    byte[] byte1 = a.GetBytes(szPost);
    webBrowser.Navigate(szUrl, szTarget, byte1,
"Content-Type: application/x-www-form-urlencoded\r\n");
    return;
}

MSHTML, Dom, and copying objects from a WebBrowser Control

While working on grabbing an image from a WebBrowser (talked about in my last post), I stumbled upon this method. It does work; however, it has one draw back: It only works for single framed images (ie. No animated images). Though, to be honest, I didn't play around all that much to see if I could finagle it to work with animated images.

I found this method of a foreign site and half the code was gone so I had to drum up the rest by sorting through the MSDN help files. I'm not quite sure if anyone else has looked up any DOM type stuff there, but as far as I can tell it's all quite dated and any real useful information is sparse and hard to come by. Luckily, with time I was able to piece together enough to determine the appropriate castings.

Alright, you have a WebBrowser control, and there's a particular element on it that you want. If it's text, I'd suggest just parsing the DocumentText variable of the control, but if it's an image, then this is a definite possible solution if you don't feel comfortable with the one here (the previous post going over a little bit simpler method which also handles sessions).

Before posting the code (straight from my project), I would like to point out that I didn't end up using this method and it was mostly testing. I still think that for some people it might pose useful and so I'm posting it. I still suggest looking at the previous post that covers a bit easier method before using this.

Code:

mshtml.IHTMLDocument2 doc = (mshtml.IHTMLDocument2)
    webBrowser1.Document.DomDocument;
mshtml.IHTMLSelectionObject sobj = doc.selection;
mshtml.HTMLBody body = doc.body as mshtml.HTMLBody;
sobj.empty();
mshtml.IHTMLControlRange range = body.createControlRange() as
    mshtml.IHTMLControlRange;
mshtml.IHTMLControlElement img = (mshtml.IHTMLControlElement)     webBrowser1.Document.Images[0].DomElement;

range.add(img);
range.select();
range.execCommand("Copy", false, null);

Bitmap bimg = new Bitmap(Clipboard.GetImage());
pictureBox1.Image = bimg;

Grabbing secure data from the web.

Recently, I was working on a little image recognition for some anti-bot type image. No no, I wasn't working on any spamming type of application, just a little side project of mine.

Regardless, I needed to load the image from a website while being logged into the website. I did a bit of research and learned that there's no real way, or at least no easy way, to pull an image from a WebBrowser control. So I went about Load'ing it with a PictureBox until I found out that the manner in which the image is loaded doesn't retain the session cookie in the WebBrowser. Thus I ran into a rather large dilemma, I needed to load the image while remaining logged in and the PictureBox was definitely not the correct solution nor was trying to pull it out of the WebBrowser control.

I did quite a bit more research and came across the HttpWebRequest and the HttpWebResponse classes. I knew that if I could some how send out the correct cookie data from my WebBrowser control with my request for the image, I'd get what I wanted. So I quick parsed the cookie credentials from my WebBrowser control (which was a bit of a pain because they each have their different forms of storing the cookie information), added it to my request, and got my image perfectly.

Code:

HttpWebRequest h = (HttpWebRequest)WebRequest.Create(URL);
h.CookieContainer = new CookieContainer();

foreach (string Cooky in webBrowser.Document.Cookie.Split(';'))
{
    h.CookieContainer.Add(new Cookie(Cooky.Split('=')[0].Replace(" ", ""), Cooky.Split('=')[1].Replace(" ", ""), DOMAIN, URL));
}

HttpWebResponse response = (HttpWebResponse)h.GetResponse();
Stream simg = response.GetResponseStream();
Image bimg = new Bitmap(simg);

I posted this code at the CodeProject where a few people were looking to do the same thing.