edu.berkeley.nlp.lm.io

Class GoogleLmReader<W>

java.lang.Object

edu.berkeley.nlp.lm.io.GoogleLmReader<W>

All Implemented Interfaces:

LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>

public class GoogleLmReader<W>
extends java.lang.Object
implements LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>

Reads in n-gram count collections in the format that the Google n-grams Web1T corpus comes in.

Author:

adampauls

Constructor Summary

Constructors

Constructor and Description
GoogleLmReader(java.lang.String rootDir, WordIndexer<W> wordIndexer, ConfigOptions opts)

Method Summary

Modifier and Type	Method and Description
static <W> void	addToIndexer(WordIndexer<W> wordIndexer, java.lang.String sortedVocabPath)
int	getLmOrder()
void	parse(NgramOrderedLmReaderCallback<LongRef> callback)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GoogleLmReader

public GoogleLmReader(java.lang.String rootDir,
                      WordIndexer<W> wordIndexer,
                      ConfigOptions opts)

Method Detail

getLmOrder

public int getLmOrder()

parse

public void parse(NgramOrderedLmReaderCallback<LongRef> callback)

Specified by:

parse in interface LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>

addToIndexer

public static <W> void addToIndexer(WordIndexer<W> wordIndexer,
                                    java.lang.String sortedVocabPath)

Parameters:

sortedVocabPath -