Packages

package robot

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. case class Directive (userAgent: String, disallow: Seq[String] = Nil, allow: Seq[String] = Nil, crawlDelay: Int = 0) extends Product with Serializable
  2. case class RobotsTxt (directive: Seq[Directive] = Nil, sitemap: Seq[String] = Nil) extends Product with Serializable
  3. sealed abstract class Sitemap extends AnyRef
  4. case class Sitemapindex (sitemap: Seq[Sitemapindex.Sitemap]) extends Sitemap with Product with Serializable
  5. case class Urlset (url: Seq[Url]) extends Sitemap with Product with Serializable

Value Members

  1. object RobotsTxtParser

    The object of parsing the robots.txt.

  2. object SitemapParser

    The object of parsing the sitemap.

  3. object Sitemapindex extends Serializable
  4. object Urlset extends Serializable

Ungrouped