Summary
The interface used by the WebCrawlerEngine to store a history of previously completed requests, so that they are not visited again.
For a list of all members of this type, see
CompletedRequestHistory Members
.
Declaration Syntax
Public Interface CompletedRequestHistory
Inherits WebCrawlerComponent, IDisposable
public interface CompletedRequestHistory : WebCrawlerComponent, IDisposable
public __gc __interface CompletedRequestHistory : public WebCrawlerComponent, IDisposable
public interface CompletedRequestHistory implements WebCrawlerComponent, IDisposable
Remarks
When the WebCrawlerEngine completes visiting a url, it notifies this class by calling AddCompletedRequest. When the engine begins processing in the future, it will store by loading previously completed requests by calling GetCompletedRequests. Implementations can, therefore, choose if they want to prevents duplicates across runs. Please note that this class does not prevent duplicate processing inside of one run. This is the job of the RequestStore.Requirements
Namespace: Eas.WebCrawler.InterfacesAssembly: Eas.WebCrawler.Interfaces (eas.webcrawler.interfaces.dll)