public class StaticTokenTreeBuilder extends AbstractTokenTreeBuilder
DynamicTokenTreeBuilder
when multiple index segments produced by PerSSTableIndexWriter are stitched together
by PerSSTableIndexWriter.complete().
This class uses the RangeIterator, now provided by
CombinedTerm.getTokenIterator(), to iterate the data twice.
The first iteration builds the tree with leaves that contain only enough
information to build the upper layers -- these leaves do not store more
than their minimum and maximum tokens plus their total size, which makes them
un-serializable.
When the tree is written to disk the final layer is not
written. Its at this point the data is iterated once again to write
the leaves to disk. This (logarithmically) reduces copying of the
token values while building and writing upper layers of the tree,
removes the use of SortedMap when combining SAs, and relies on the
memory mapped SAs otherwise, greatly improving performance and no
longer causing OOMs when TokenTree sizes are big.
See https://issues.apache.org/jira/browse/CASSANDRA-11383 for more details.AbstractTokenTreeBuilder.InteriorNode, AbstractTokenTreeBuilder.Leaf, AbstractTokenTreeBuilder.LevelIterator, AbstractTokenTreeBuilder.NodeTokenTreeBuilder.EntryTypeleftmostLeaf, numBlocks, rightmostLeaf, rightmostParent, root, tokenCount, treeMaxToken, treeMinTokenAB_MAGIC, BLOCK_BYTES, BLOCK_ENTRY_BYTES, BLOCK_HEADER_BYTES, ENTRY_TYPE_MASK, LAST_LEAF_SHIFT, MAX_OFFSET, OVERFLOW_ENTRY_BYTES, OVERFLOW_TRAILER_BYTES, OVERFLOW_TRAILER_CAPACITY, SHARED_HEADER_BYTES, TOKENS_PER_BLOCK| Constructor and Description |
|---|
StaticTokenTreeBuilder(CombinedTerm term) |
| Modifier and Type | Method and Description |
|---|---|
void |
add(java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> data) |
void |
add(java.lang.Long token,
long keyPosition) |
void |
add(java.util.SortedMap<java.lang.Long,com.carrotsearch.hppc.LongSet> data) |
protected void |
constructTree() |
long |
getTokenCount() |
boolean |
isEmpty() |
java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> |
iterator() |
void |
write(DataOutputPlus out) |
add, alignBuffer, finish, flushBuffer, serializedSizepublic StaticTokenTreeBuilder(CombinedTerm term)
public void add(java.lang.Long token,
long keyPosition)
public void add(java.util.SortedMap<java.lang.Long,com.carrotsearch.hppc.LongSet> data)
public void add(java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> data)
public boolean isEmpty()
public java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> iterator()
public long getTokenCount()
getTokenCount in interface TokenTreeBuildergetTokenCount in class AbstractTokenTreeBuilderpublic void write(DataOutputPlus out) throws java.io.IOException
write in interface TokenTreeBuilderwrite in class AbstractTokenTreeBuilderjava.io.IOExceptionprotected void constructTree()
constructTree in class AbstractTokenTreeBuilderCopyright © 2018 The Apache Software Foundation