Search all CodeProject articles by single click using SpringSys OrchidGrid.

Download

Project:
SearchCodeProject_Exe.zip

Source code:
SearchCodeProject_Src

Summary

This article describes how to search all CodeProject articles from client application, and to parse / extract the searching result using OrchidGrid automatically.

The benefit is obvious: you need not to navigate between web pages anymore if there are multiple pages of results, searching will be performed by a single click. The result can be further processed in grid, you can highlight rows with rating greater than a value, you can sort the rows, you can export the result into text file or Excel file etc.

Below is a screen snapshot of the sample application in this article.

Send searching request

It's very simple to send searching request to CodeProject's web site. What you need to do is to send http request to the below URL
http://www.codeproject.com/info/search.asp?cats=CAT&searchkw=KW&Page=PI

where
CAT is category in number
KW is the keywords to search
PI is the page index

Well, WebClient class in Microsoft .NET framework is exactly the thing we need to send http request to and get response from server. By calling the WebClient. DownloadFileAsync() method with url and the file name as parameters, we can get response from server in asynchronous manner. When downloading is processing or finished, WebClient fires DownloadFileCompleted and DownloadProgressChanged events as notification. For more information about WebClient class, please read the refferrence on WebClient class in MSDN.

The below code will download the content of a URL to a file named ¡°cache.txt¡±. What we download will be a HTML text file. Suppose the URL is above CodeProject's URL, then the response we get from server will be the searching result in HTML.

WebClient client = new WebClient ();
// wire up the download complete event handler.
// The operation will either complete successfully or with an
// error code.
client.DownloadFileCompleted += client_DownloadFileCompleted;

// wire up the download progress event handler.
client.DownloadProgressChanged += client_DownloadProgressChanged;
Uri uri = new Uri (url);
client.DownloadFileAsync(uri, "cache.txt" );

Parse the webpage result

Ok, we already have the searching result in text/html file, how to parse and extract the useful data from it?

SpringSys OrchidGrid is exactly the component we can use. OrchidGrid is not only a data binding grid control, it but also has the powerful text parsing functions that can parse/extract/tabulate data from delimited text file, fixed-width text file, or text file with particular pattern.

OrchidGrid parses and extracts text data according to the information specified in the TextDataAdapter. TextDataAdaper is the joint channel between text source and grid, it defines the location of the source text, the type of conetent, the delimiters for delimited file, the width of each field for fixed-width file, or the pattern for pattern file.

You need not to know detail about TextDataAdapter, there is a short cut which can navigate you setting up the TextDataAdapter in friendly steps at design time - Import Wizard. Right click the control and select ¡°Import Wizart¡± from the popup menu, you can see it.

Before starting up the Import Wizard, we a temparary search result file from CodeProject. For example, open the www.codeproject.com , search a word ¡° AJAX ¡± then save the result web page into a temparary text file say ¡°cache.txt¡±.

Right click the OrchidGrid at design time, choose ¡°Import Wizard¡± from the pop up menu. The first step of the wizard starts. Here we specify the cache.txt file as the source file, please see the picture-1.

Click next, in this step we will specify the type of the raw text. Here we set the ¡°Raw Data Type¡± to ¡°Pattern¡±. Please see picture-2 .

Click next, in this step we define the data pattern appears in the text file. A pattern is actually a sequence of markers and fields (including general field, fixed width field and ignore field). Marker is a string that can indicate the start point of a field in the source text. Any content between two Markers is treated as data for field and will be extracted. A data field must has a unique field name and data type, except the ignore field. This is similar to the concept in a data base.

After anylizing the text, we set the pattern with 5 fields, they are ¡°Topic Link¡±, ¡°Topic¡±, ¡°Author¡±, ¡°Update Data¡±, ¡°Rating¡±. Please see picture-3.

Click next to go to the last step of wizard, in which we can specify the data type for each field.Please see picture-4.

If you'd like, you can click the ¡°Preview¡± button in step 3 or 4 to check whether data can be properly extracted, you can do nessasary adjustment if not.

If everything is fine, clicking ¡°Done¡± button to finish the wizard. All the information we set through the wizard will be persisted into the OrchidGrid.DataAdapter property automatically. In addition, 5 columns will be created, each corresponds to a field we defined above.

We can call OrchidGrid .FillWithAdapter() method to fill the data into grid.

Put searching and parsing together in loops

Now, we can put the searching/downloading step together with the parsing step. Searching is started from page 1, after getting the result, we call OrchidGrid.FillWithAdapter() to append the data rows. Then increse the page index parameter to search and parse next page, untill there is no new data found.

In the sample project, I added some logic to high-light the rows that have rating value greater than a threshold. Also, double clicking a result will open the web page corresponding to that article.

Please see the screen shot of the this application.