Download HttpCrawler

Description

A simple, extensible and multi-threaded HTTP Crawler written in C# using Visual Studio 2010 and .NET 4.0.

Source Files

The download file httpcrawler.zip has the following entries.


BuildProcessTemplates/DefaultTemplate.11.1.xaml
BuildProcessTemplates/DefaultTemplate.xaml
BuildProcessTemplates/LabDefaultTemplate.11.xaml
BuildProcessTemplates/UpgradeTemplate.xaml
HttpCrawler.Demo/HttpCrawler.Demo.csproj
HttpCrawler.Demo/HttpCrawler.Demo.csproj.vspscc
HttpCrawler.Demo/Program.cs/*from   w  w w.j av a2 s .  c om*/
HttpCrawler.Demo/Properties/AssemblyInfo.cs
HttpCrawler.Demo/app.config
HttpCrawler.Engine.Test/CssHttpCrawlerContentProcessorTests.cs
HttpCrawler.Engine.Test/HttpCrawler.Engine.Test.csproj
HttpCrawler.Engine.Test/HttpCrawler.Engine.Test.csproj.vspscc
HttpCrawler.Engine.Test/Internal/StringHttpCrawlerContent.cs
HttpCrawler.Engine.Test/Properties/AssemblyInfo.cs
HttpCrawler.Engine/ConsoleHttpCrawlerEngineLogger.cs
HttpCrawler.Engine/CssHttpCrawlerContentProcessor.cs
HttpCrawler.Engine/DoNothingHttpCrawlerContentPersister.cs
HttpCrawler.Engine/DoNothingHttpCrawlerEngineLogger.cs
HttpCrawler.Engine/FileSystemHttpCrawlerContentPersister.cs
HttpCrawler.Engine/HtmlHttpCrawlerContent.cs
HttpCrawler.Engine/HtmlHttpCrawlerContentProcessor.cs
HttpCrawler.Engine/HttpCrawler.Engine.csproj
HttpCrawler.Engine/HttpCrawler.Engine.csproj.vspscc
HttpCrawler.Engine/HttpCrawlerContent.cs
HttpCrawler.Engine/HttpCrawlerContentFactory.cs
HttpCrawler.Engine/HttpCrawlerEngine.cs
HttpCrawler.Engine/IHttpCrawlerContentPersister.cs
HttpCrawler.Engine/IHttpCrawlerContentProcessor.cs
HttpCrawler.Engine/IHttpCrawlerEngine.cs
HttpCrawler.Engine/IHttpCrawlerEngineLogger.cs
HttpCrawler.Engine/IHttpCrawlerUriFollowDeterminer.cs
HttpCrawler.Engine/Properties/AssemblyInfo.cs
HttpCrawler.Engine/SimilarOriginHttpCrawlerUriFollowDeterminer.cs
HttpCrawler.Engine/XmlHttpCrawlerContent.cs
HttpCrawler.Extensions.Tests/HttpCrawler.Extensions.Tests.csproj
HttpCrawler.Extensions.Tests/HttpCrawler.Extensions.Tests.csproj.vspscc
HttpCrawler.Extensions.Tests/ParallelHelperTests.cs
HttpCrawler.Extensions.Tests/Properties/AssemblyInfo.cs
HttpCrawler.Extensions/ConcurrentQueueExtensions.cs
HttpCrawler.Extensions/HashSetExtensions.cs
HttpCrawler.Extensions/HttpCrawler.Extensions.csproj
HttpCrawler.Extensions/HttpCrawler.Extensions.csproj.vspscc
HttpCrawler.Extensions/ParallelHelper.cs
HttpCrawler.Extensions/PathHelper.cs
HttpCrawler.Extensions/Properties/AssemblyInfo.cs
HttpCrawler.Extensions/UriExtensions.cs
HttpCrawler.Extensions/UriHelper.cs
HttpCrawler.Extensions/WebResponseExtensions.cs
HttpCrawler.sln
HttpCrawler.vsmdi
HttpCrawler.vssscc
Libraries/HtmlAgilityPack.License.txt
Libraries/HtmlAgilityPack.dll
Local.testsettings
TraceAndTestImpact.testsettings

Download

Click the following link to download httpcrawler.zip.

httpcrawler.zip




















Home »
  C# Free Code »
    Development »




Algorithm
ANTLR
AOP
Application
Barcode
CHM
Design Patterns
Desktop
DotNet
DotNet Engine
DotNet Application
DotNet Assembly
DotNet Library
Engine
Epub
Example
File
File Library
Framework
GIS
Intellisense
IoC
Library
log4net
Matlab
MSBuild
MVC
MVP
MVVM
Numerical Library
NUnit
OpenCL
Plugin
Quartz
Resx File
SDK
SMS
Thread
UnitTest
Utility
VisualStudio
Windows
XML