Classes

Class Description
ProcessingStateEventArgs

Summary

Trace argument that wraps a ProcessingState.
ProcessingStateMessageEventArgs

Summary

Wraps a ProcessingState and a System.String message.
Request

Summary

A request which represents a Uri and all the links processed to get to that Uri.
UrlCompletedEventArgs

Summary

An System.EventArgs that is used when firing events to notify recipients that a Url was processed succesfully.
UrlFailedEventArgs

Summary

An System.EventArgs which is fired if a Url could not be processed.

Interfaces

Interface Description
CompletedRequestHistory

Summary

The interface used by the WebCrawlerEngine to store a history of previously completed requests, so that they are not visited again.
LinkExtractor

Summary

Extracts links from a processed page.
LinkSelector

Summary

Determines what uris (links) should be crawled.
ProcessingState

Summary

Stores state that is accumulated while processing a request.
RequestStore

Summary

Stores Requests which are intended to be processed by the crawler.
TemporaryStorage

Summary

Abstracts away temporary storage of the contents of a stream so that it can be accessed later.
TemporaryStorageFactory

Summary

Creates instances of TemporaryStorage.
WebCrawlerComponent

Summary

Classes which implement this interface will be initialized by the WebCrawlerContext into which they are passed.
WebCrawlerContext

Summary

Context used by web crawler during processing.
WebCrawlerEngine

Summary

The main web crawler engine. This interface is used to run the web crawler.
WebCrawlerEvents

Summary

Contains the collection of events fired while crawling.
WebCrawlerSettings

Summary

Implementations of this class are used to create impementations of WebCrawlerEngine.
WebRequestInterface

Summary

Wraps details about a web request.
WebResponseInterface

Summary

Wraps a System.Net.WebResponse returned from a request to a System.Uri.

Enumerations

Enumeration Description
TraceLevel

Summary

The level of tracing to use while processing.
WorkerState

Summary

The different states of processing in which a worker can be.