For a long time, I was looking for a tool that can extract data from newsgroup server. This may help me analyzing information like what topic people mostly focus on, who are concerning with what content, where they come from, who are the most active people etc. But unfortunately, I could not get a suitable one. So, why not write one myself?
NNTP Wrapper Class
As we know, NNTP defines the communication protocol between the client and newsgroup server. So, internally, we use a helper class NewsgroupClient to communicate withe newsgroup server. This class wraps the methods that can send commands to server and retrieve data correspondingly.
For example, ListGroup() method lists the available newsgroup on the server. SelectGroup() method selects and retrieves article ranges. DownloadHeaders() method retrieve article headers from the current active group.
User Interface
The user interface of this tool can be divided into two parts: the Newsgroups tree on the left side, and the article list on the right. Both of them are actually SpringSys OrchidGrid control. As we know, OrchidGrid can work in tree mode and grid mode.
After adding a Newsgroup server, all the groups on this server will be added as child under that server name node. The data of the tree will be persisted into a text file when the application is closed. When next we run the tool again, and data will be restored from the text file.
Each group has a checkbox, by which we can select the groups that are going to be explored before downloading the article headers. Here we use the check box node feature of the grid.
When you choose the groups and click the "Download Message Headers" button, the basic message data will be listed on the right side grid.
OrchidGrid has built-in data exporting abilities, calling the grid's methods ExportToDelimitedFile() and ExportToExcel() can export the grid data into text file of excel file. Simple?
You can also write your own exporting code if you'd like. In this application, I commented some code in the source that can export the email address of the article author to a text file. (Search button1_Click method in Form1.cs file), thus, this tool can be extended as an email address extractor.
Try a Sample Server
Let's try a newsgroup server for example – “msnews.microsoft.com”, it has a bunch of newgroups, and some contain thousands of articles.
Input the server address “msnews.microsoft.com” and press the Enter key, the server and the newsgroups on that server are listed on the left tree. Check some newsgroups as you like and click the button “Download Message Headers”, you will get all the headers in the selected newsgroups. Then, you can export the headers to text or Excel file.
Download the code
Click below link to download this tool in source code.
NewsgroupExtractor.zip (475.43 kb)