Download NRobots.txt

Description

The Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website which is otherwise publicly viewable.

Source Files

The download file nrobots.zip has the following entries.


BuildProcessTemplates/DefaultTemplate.11.1.xaml
BuildProcessTemplates/LabDefaultTemplate.11.xaml
BuildProcessTemplates/UpgradeTemplate.xaml
Robots/Lib/xunit.dll//from  w w w . j av  a 2 s .  c  o  m
Robots/LocalTestRun.testrunconfig
Robots/Robots.sln
Robots/Robots.vsmdi
Robots/Robots.vssscc
Robots/Robots/Fluent/RobotsBuilder.cs
Robots/Robots/IRobots.cs
Robots/Robots/Models/AccessUrlEntry.cs
Robots/Robots/Models/AllowEntry.cs
Robots/Robots/Models/CommentEntry.cs
Robots/Robots/Models/CrawlDelayEntry.cs
Robots/Robots/Models/DisallowEntry.cs
Robots/Robots/Models/Entry.cs
Robots/Robots/Models/EntryType.cs
Robots/Robots/Models/SiteMapEntry.cs
Robots/Robots/Models/UrlEntry.cs
Robots/Robots/Models/UserAgentEntry.cs
Robots/Robots/Properties/AssemblyInfo.cs
Robots/Robots/Robots.cs
Robots/Robots/Robots.csproj
Robots/Robots/Robots.csproj.vspscc
Robots/Robots/UserAgents/BotsUserAgents.cs
Robots/Robots/UserAgents/UserAgents.cs
Robots/Robots/app.config
Robots/RobotsTests/AccessUrlEntryTest.cs
Robots/RobotsTests/AllowEntryTest.cs
Robots/RobotsTests/CommentEntryTest.cs
Robots/RobotsTests/CrawlDelayEntryTest.cs
Robots/RobotsTests/DisallowEntryTest.cs
Robots/RobotsTests/EntryTest.cs
Robots/RobotsTests/Properties/AssemblyInfo.cs
Robots/RobotsTests/RobotsBuilderTest.cs
Robots/RobotsTests/RobotsTest.cs
Robots/RobotsTests/RobotsTests.csproj
Robots/RobotsTests/RobotsTests.csproj.vspscc
Robots/RobotsTests/SiteMapEntryTest.cs
Robots/RobotsTests/Test References/Robots.accessor
Robots/RobotsTests/UrlEntryTest.cs
Robots/RobotsTests/UserAgentEntryTest.cs

Download

Click the following link to download nrobots.zip.

nrobots.zip




















Home »
  C# Free Code »
    Web »




AJAX
Blog
DotNetNuke
Email
HTML
Http
IIS
JavaScript
jQuery
RSS
URL
VoIP
Web
WebSite
WebSocket