The Lazy Coders

WebClient Object

2009-05-29T13:28:00.022-05:00

Long time since the last post...

I've been working on a project that interfaces with numerous HTTP servers to update the firmware on the hosting machine. I opted to use the WebClient object, mostly because I wasn't sure what the best method would be but figured using WebBrowsers would just be wasteful, and ran into a few problems that were tedious enough to make me want to post them on here.

First thing's first, some servers respond back with an "Expect" error, equating to an exception from the WebClient. The fix:

Code:

System.Net.ServicePointManager.Expect100Continue = false;

Secondly, logging into a website. The WebClient object doesn't handle cookies for you and that sucks, a lot. The easiest way to fix that is by creating a new class, inheriting the WebClient class and overriding the GetWebRequest method.

Code:

class WebClientEx : WebClient
{
    private CookieContainer m_container = new CookieContainer();

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest request = base.GetWebRequest(address);
        if (request is HttpWebRequest)
        {
            (request as HttpWebRequest).CookieContainer = m_container;
        }
    return request;
}

Credit for this goes to http://codex.bradrowley.net/

Ok, that gets you cookies but submitting to a form wasn't all that easy...

Code:

// Create the POST form parameters
        NameValueCollection nvc = new NameValueCollection();
        nvc.Add("authUser", username);
        nvc.Add("authPass", password);

        // Pause until m_client is available
        while (m_client.IsBusy)
        {
        }

        result = System.Text.ASCIIEncoding.ASCII.GetString(
m_client.UploadValues("http://...", "POST", nvc));

And finally, the last thing I had to figure out how to do was upload a file, via a form. Fortunately, I was able to find a piece of code from madmik3 that did the trick.

Code:

// Majority of this function from http://www.codeproject.com/KB/cs/uploadfileex.aspx
public string FormUploadFile(string uploadfile, string url, string fileFormName, string contenttype, NameValueCollection querystring)
{
    if ((fileFormName == null) ||
        (fileFormName.Length == 0))
    {
    fileFormName = "file";
    }

    if ((contenttype == null) ||
    (contenttype.Length == 0))
    {
        contenttype = "application/octet-stream";
    }

    string postdata;
    postdata = "?";
    if (querystring != null)
    {
        foreach (string key in querystring.Keys)
    {
            postdata += key + "=" + querystring.Get(key) + "&";
    }
    }
    Uri uri = new Uri(url + postdata);


    string boundary = "----------" + DateTime.Now.Ticks.ToString("x");
    HttpWebRequest webrequest = this.GetWebRequest(uri) as HttpWebRequest;  // Changed by Austin 5/22/09 - To account for cookies
    webrequest.ContentType = "multipart/form-data; boundary=" + boundary;
    webrequest.Method = "POST";


    // Build up the post message header
    StringBuilder sb = new StringBuilder();
    sb.Append("--");
    sb.Append(boundary);
    sb.Append("\r\n");
    sb.Append("Content-Disposition: form-data; name=\"");
    sb.Append(fileFormName);
    sb.Append("\"; filename=\"");
    sb.Append(Path.GetFileName(uploadfile));
    sb.Append("\"");
    sb.Append("\r\n");
    sb.Append("Content-Type: ");
    sb.Append(contenttype);
    sb.Append("\r\n");
    sb.Append("\r\n");

    string postHeader = sb.ToString();
    byte[] postHeaderBytes = Encoding.UTF8.GetBytes(postHeader);

    // Build the trailing boundary string as a byte array
    // ensuring the boundary appears on a line by itself
    byte[] boundaryBytes =
            Encoding.ASCII.GetBytes("\r\n--" + boundary + "\r\n");

    FileStream fileStream = new FileStream(uploadfile,
            FileMode.Open, FileAccess.Read);
    long length = postHeaderBytes.Length + fileStream.Length +
                                               boundaryBytes.Length;
    webrequest.ContentLength = length;

    Stream requestStream = webrequest.GetRequestStream();

    // Write out our post header
    requestStream.Write(postHeaderBytes, 0, postHeaderBytes.Length);

    // Write out the file contents
    byte[] buffer = new Byte[checked((uint)Math.Min(4096,
            (int)fileStream.Length))];
    int bytesRead = 0;
    while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
        requestStream.Write(buffer, 0, bytesRead);

    // Write out the trailing boundary
    requestStream.Write(boundaryBytes, 0, boundaryBytes.Length);
    WebResponse responce = webrequest.GetResponse();
    Stream s = responce.GetResponseStream();
    StreamReader sr = new StreamReader(s);

    return sr.ReadToEnd();
}

Stripping HTML from text in SQL Server 2005

2007-06-27T09:29:00.000-05:00

Today I came across a problem where a field in a SQL server 2005 database contained encoded HTML (for example, it looked like "<div>Blah blah"). However, as this field was required in a SQL server report, this HTML needed to either be removed or formatted. Since SQL reports don't have a built-in function for this, and trying to create a function for it using their Excel-style expressions didn't sound like much fun, the only option left was to trim it in the SQL code.

Normally, I'd turn to a regular expression to handle this, however in SQL server, to use regular expressions you need to mess about with COM to get a VBScript interface etc, and I'm lazy, so I wanted a simpler way. A bit of research turned up an article at http://blog.sqlauthority.com/ which appeared to do exactly what I needed using just a normal user-defined function, and as luck would have it, it had only been written 11 days ago, so my timing was pretty good ;) Of course, it wasn't exactly what I needed, but it was pretty close, and after a few changes (mostly copy-paste code to handle each HTML entity that was likely to appear), I had a lovely new function which did exactly what I needed it to do. The performance of it is probably pretty dire, but that's acceptable in this case.

Here's the code, should anyone need it:


ALTER FUNCTION [dbo].[udf_StripHTML]
(
@HTMLText varchar(MAX)
)
RETURNS varchar(MAX)
AS
BEGIN
DECLARE @Start  int
DECLARE @End    int
DECLARE @Length int

-- Replace the HTML entity &amp; with the '&' character (this needs to be done first, as
-- '&' might be double encoded as '&amp;amp;')
SET @Start = CHARINDEX('&amp;', @HTMLText)
SET @End = @Start + 4
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, '&')
SET @Start = CHARINDEX('&amp;', @HTMLText)
SET @End = @Start + 4
SET @Length = (@End - @Start) + 1
END

-- Replace the HTML entity &lt; with the '<' character
SET @Start = CHARINDEX('&lt;', @HTMLText)
SET @End = @Start + 3
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, '<')
SET @Start = CHARINDEX('&lt;', @HTMLText)
SET @End = @Start + 3
SET @Length = (@End - @Start) + 1
END

-- Replace the HTML entity &gt; with the '>' character
SET @Start = CHARINDEX('&gt;', @HTMLText)
SET @End = @Start + 3
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, '>')
SET @Start = CHARINDEX('&gt;', @HTMLText)
SET @End = @Start + 3
SET @Length = (@End - @Start) + 1
END

-- Replace the HTML entity &amp; with the '&' character
SET @Start = CHARINDEX('&amp;amp;', @HTMLText)
SET @End = @Start + 4
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, '&')
SET @Start = CHARINDEX('&amp;amp;', @HTMLText)
SET @End = @Start + 4
SET @Length = (@End - @Start) + 1
END

-- Replace the HTML entity &nbsp; with the ' ' character
SET @Start = CHARINDEX('&nbsp;', @HTMLText)
SET @End = @Start + 5
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, ' ')
SET @Start = CHARINDEX('&nbsp;', @HTMLText)
SET @End = @Start + 5
SET @Length = (@End - @Start) + 1
END

-- Replace any <br> tags with a newline
SET @Start = CHARINDEX('<br>', @HTMLText)
SET @End = @Start + 3
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, CHAR(13) + CHAR(10))
SET @Start = CHARINDEX('<br>', @HTMLText)
SET @End = @Start + 3
SET @Length = (@End - @Start) + 1
END

-- Replace any <br/> tags with a newline
SET @Start = CHARINDEX('<br/>', @HTMLText)
SET @End = @Start + 4
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, 'CHAR(13) + CHAR(10)')
SET @Start = CHARINDEX('<br/>', @HTMLText)
SET @End = @Start + 4
SET @Length = (@End - @Start) + 1
END

-- Replace any <br /> tags with a newline
SET @Start = CHARINDEX('<br />', @HTMLText)
SET @End = @Start + 5
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, 'CHAR(13) + CHAR(10)')
SET @Start = CHARINDEX('<br />', @HTMLText)
SET @End = @Start + 5
SET @Length = (@End - @Start) + 1
END

-- Remove anything between <whatever> tags
SET @Start = CHARINDEX('<', @HTMLText)
SET @End = CHARINDEX('>', @HTMLText, CHARINDEX('<', @HTMLText))
SET @Length = (@End - @Start) + 1

WHILE (@Start > 0 AND @End > 0 AND @Length > 0) BEGIN
SET @HTMLText = STUFF(@HTMLText, @Start, @Length, '')
SET @Start = CHARINDEX('<', @HTMLText)
SET @End = CHARINDEX('>', @HTMLText, CHARINDEX('<', @HTMLText))
SET @Length = (@End - @Start) + 1
END

RETURN LTRIM(RTRIM(@HTMLText))

END

Of course, most of the credit for this goes to Pinal Dave of SQLAuthority.com, I just hacked his code around a bit to suit my particular needs.

Note the formatting is going a bit awry, mostly due to HTML entities in the code. I'll fix this when I get a chance

SQL Reports: Execution '' cannot be found.

2007-06-26T08:58:00.000-05:00

While developing a page to generate a weekly report using SQL server reporting services, I came across a bit of a funny error. The ReportViewer control was giving an error of "Execution '' cannot be found.", though this error was appearing as part of the report itself, and not being thrown back to the ASP.NET application.
After a fair bit of pondering and head-scratching, a working solution was discovered - albeit one that I don't really understand why it works, since the original error is still a bit of a mystery.

Basically, it all boils down to the fact that the ReportViewer control was on the same .aspx page as the page that the user selects the parameters for the report (in this case, the department). So once the user chose a department, the page would post-back to itself, the parameter selection panel would be hidden, and the ReportViewer panel would be shown. Once this was seperated into two seperate pages, with the first using a cross-page postback to call the page with the report, it all worked perfectly. I can only assume this to be an issue caused by the report doing something even while hidden that later messes things up. It works now though, which is all that matters ;)

Variable arguments in C#

2007-06-04T15:28:00.001-05:00

Ever used variable arguments in C++ and wanted to use them in C# but you could never figure out how? I certainly did a few weeks ago, I was working on a function similar to some of the old standard C IO libraries (though I can't recall what it's purpose was to be) and I spent quite a bit of time trying to learn how, never actually finding anything. Well I just stumbled upon it here at the MSDN C# FAQ page.

If you're like me and would rather just see the code...

Code:

void paramsExample(object arg1, object arg2, params object[] argsRest)
{
foreach (object arg in argsRest)
{ /* .... */ }
}

Submitting forms on the enter key

2007-06-04T12:07:00.000-05:00

Quite a while ago I was working on a form with a few textboxes and a submit button. I was nearly complete with the application but it didn't do one thing that I wanted, I had to either tab to the submit button and hit enter or click on it with the mouse. I'm fairly lazy and I know most people are too, so this little problem would most likely annoy quite a few people.

Well, after searching on google for a few hours and going over my past code (turns out I had found the solution a long time back, then forgot) and came up with the solution.

Code:

this.AcceptButton = button_Submit;

Seems simple enough, the real trick is to change the AcceptButton whenever you have a different form you want to submit. Then all of your various forms will respond accordingly and you don't have to attempt to handle any key downs or other messages to mimic this behavior.

Posting form content with the WebBrowser control

2007-06-04T00:42:00.000-05:00

A while ago I was working on one of my many side projects and needed to be able to login to a website using a WebBrowser control. The form used Post rather then Get, so I wasn't able to simply append the appropriate form elements onto the URL.

I did some searching online and ended up with a nice little method to Post form content to a site by sending the correct headers in your Navigate call.

Code:

private void Post(string szUrl, string szPost)
{
    Encoding a = Encoding.UTF8;
    byte[] byte1 = a.GetBytes(szPost);
    webBrowser.Navigate(szUrl, szTarget, byte1,
"Content-Type: application/x-www-form-urlencoded\r\n");
    return;
}

MSHTML, Dom, and copying objects from a WebBrowser Control

2007-06-03T23:52:00.000-05:00

While working on grabbing an image from a WebBrowser (talked about in my last post), I stumbled upon this method. It does work; however, it has one draw back: It only works for single framed images (ie. No animated images). Though, to be honest, I didn't play around all that much to see if I could finagle it to work with animated images.

I found this method of a foreign site and half the code was gone so I had to drum up the rest by sorting through the MSDN help files. I'm not quite sure if anyone else has looked up any DOM type stuff there, but as far as I can tell it's all quite dated and any real useful information is sparse and hard to come by. Luckily, with time I was able to piece together enough to determine the appropriate castings.

Alright, you have a WebBrowser control, and there's a particular element on it that you want. If it's text, I'd suggest just parsing the DocumentText variable of the control, but if it's an image, then this is a definite possible solution if you don't feel comfortable with the one here (the previous post going over a little bit simpler method which also handles sessions).

Before posting the code (straight from my project), I would like to point out that I didn't end up using this method and it was mostly testing. I still think that for some people it might pose useful and so I'm posting it. I still suggest looking at the previous post that covers a bit easier method before using this.

Code:

mshtml.IHTMLDocument2 doc = (mshtml.IHTMLDocument2)
    webBrowser1.Document.DomDocument;
mshtml.IHTMLSelectionObject sobj = doc.selection;
mshtml.HTMLBody body = doc.body as mshtml.HTMLBody;
sobj.empty();
mshtml.IHTMLControlRange range = body.createControlRange() as
    mshtml.IHTMLControlRange;
mshtml.IHTMLControlElement img = (mshtml.IHTMLControlElement)     webBrowser1.Document.Images[0].DomElement;

range.add(img);
range.select();
range.execCommand("Copy", false, null);

Bitmap bimg = new Bitmap(Clipboard.GetImage());
pictureBox1.Image = bimg;

Grabbing secure data from the web.

2007-06-03T22:58:00.001-05:00

Recently, I was working on a little image recognition for some anti-bot type image. No no, I wasn't working on any spamming type of application, just a little side project of mine.

Regardless, I needed to load the image from a website while being logged into the website. I did a bit of research and learned that there's no real way, or at least no easy way, to pull an image from a WebBrowser control. So I went about Load'ing it with a PictureBox until I found out that the manner in which the image is loaded doesn't retain the session cookie in the WebBrowser. Thus I ran into a rather large dilemma, I needed to load the image while remaining logged in and the PictureBox was definitely not the correct solution nor was trying to pull it out of the WebBrowser control.

I did quite a bit more research and came across the HttpWebRequest and the HttpWebResponse classes. I knew that if I could some how send out the correct cookie data from my WebBrowser control with my request for the image, I'd get what I wanted. So I quick parsed the cookie credentials from my WebBrowser control (which was a bit of a pain because they each have their different forms of storing the cookie information), added it to my request, and got my image perfectly.

Code:

HttpWebRequest h = (HttpWebRequest)WebRequest.Create(URL);
h.CookieContainer = new CookieContainer();

foreach (string Cooky in webBrowser.Document.Cookie.Split(';'))
{
    h.CookieContainer.Add(new Cookie(Cooky.Split('=')[0].Replace(" ", ""),         Cooky.Split('=')[1].Replace(" ", ""), DOMAIN, URL));
}

HttpWebResponse response = (HttpWebResponse)h.GetResponse();
Stream simg = response.GetResponseStream();
Image bimg = new Bitmap(simg);

I posted this code at the CodeProject where a few people were looking to do the same thing.