org.languagetool.dev.dumpcheck

Class WikipediaSentenceSource

java.lang.Object

org.languagetool.dev.dumpcheck.SentenceSource

org.languagetool.dev.dumpcheck.WikipediaSentenceSource

All Implemented Interfaces:

Iterator<Sentence>

public class WikipediaSentenceSource
extends SentenceSource

Provides access to the sentences of a Wikipedia XML dump. Note that conversion exceptions are logged to STDERR and are otherwise ignored. To get an XML dump, download pages-articles.xml.bz2 from http://download.wikimedia.org/backup-index.html, e.g. http://download.wikimedia.org/dewiki/latest/dewiki-latest-pages-articles.xml.bz2.

Since:

2.4

Constructor Summary

Constructors

Constructor and Description
WikipediaSentenceSource(InputStream xmlInput, Language language)
WikipediaSentenceSource(InputStream xmlInput, Language language, Pattern filter)

Method Summary

Methods

Modifier and Type	Method and Description
String	getSource()
boolean	hasNext()
Sentence	next() Return the next sentence.

Methods inherited from class org.languagetool.dev.dumpcheck.SentenceSource

acceptSentence, remove, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

WikipediaSentenceSource

public WikipediaSentenceSource(InputStream xmlInput,
                       Language language)

WikipediaSentenceSource

public WikipediaSentenceSource(InputStream xmlInput,
                       Language language,
                       Pattern filter)

Since:

3.0

Method Detail

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<Sentence>

Specified by:

hasNext in class SentenceSource

next

public Sentence next()

Description copied from class: SentenceSource

Return the next sentence. Sentences from the source are filtered by length to remove very short and very long sentences.

Specified by:

next in interface Iterator<Sentence>

Specified by:

next in class SentenceSource

getSource

public String getSource()

Specified by:

getSource in class SentenceSource