public class PhysicalFsWriter extends Object implements PhysicalWriter
| Modifier and Type | Class and Description |
|---|---|
protected static class |
PhysicalFsWriter.VariantTracker
Record the information about each column encryption variant.
|
PhysicalWriter.OutputReceiver| Constructor and Description |
|---|
PhysicalFsWriter(FileSystem fs,
Path path,
OrcFile.WriterOptions opts) |
PhysicalFsWriter(FileSystem fs,
Path path,
OrcFile.WriterOptions opts,
WriterEncryptionVariant[] encryption) |
PhysicalFsWriter(FSDataOutputStream outputStream,
OrcFile.WriterOptions opts,
WriterEncryptionVariant[] encryption) |
| Modifier and Type | Method and Description |
|---|---|
void |
appendRawStripe(ByteBuffer buffer,
OrcProto.StripeInformation.Builder dirEntry)
Appends raw stripe data (e.g.
|
void |
close()
Closes the writer.
|
org.apache.orc.impl.PhysicalFsWriter.BufferedStream |
createDataStream(StreamName name)
Create an OutputReceiver for the given name.
|
protected OutputStream |
createIndexStream(StreamName name) |
void |
finalizeStripe(OrcProto.StripeFooter.Builder footerBuilder,
OrcProto.StripeInformation.Builder dirEntry)
Flushes the data in all the streams, spills them to disk, write out stripe
footer.
|
void |
flush()
Flushes the writer so that readers can see the preceding postscripts.
|
long |
getFileBytes(int column,
WriterEncryptionVariant variant)
Get the number of bytes for a file in a given column
by finding all the streams (not suppressed)
for a given column and returning the sum of their sizes.
|
StreamOptions |
getStreamOptions()
Get the unencrypted stream options for this file.
|
String |
toString() |
void |
writeBloomFilter(StreamName name,
OrcProto.BloomFilterIndex.Builder bloom)
Write a bloom filter index in the given stream name.
|
void |
writeFileFooter(OrcProto.Footer.Builder builder)
Writes out the file footer.
|
void |
writeFileMetadata(OrcProto.Metadata.Builder builder)
Writes out the file metadata.
|
void |
writeHeader()
Writes the header of the file, which consists of the magic "ORC" bytes.
|
void |
writeIndex(StreamName name,
OrcProto.RowIndex.Builder index)
Write an index in the given stream name.
|
long |
writePostScript(OrcProto.PostScript.Builder builder)
Writes out the postscript (including the size byte if needed).
|
void |
writeStatistics(StreamName name,
OrcProto.ColumnStatistics.Builder statistics)
Write a stripe or file statistics to the file.
|
public PhysicalFsWriter(FileSystem fs, Path path, OrcFile.WriterOptions opts) throws IOException
IOExceptionpublic PhysicalFsWriter(FileSystem fs, Path path, OrcFile.WriterOptions opts, WriterEncryptionVariant[] encryption) throws IOException
IOExceptionpublic PhysicalFsWriter(FSDataOutputStream outputStream, OrcFile.WriterOptions opts, WriterEncryptionVariant[] encryption) throws IOException
IOExceptionpublic long getFileBytes(int column,
WriterEncryptionVariant variant)
getFileBytes in interface PhysicalWritercolumn - column from which to get file sizevariant - the encryption variant to checkpublic StreamOptions getStreamOptions()
PhysicalWritergetStreamOptions in interface PhysicalWriterpublic void writeFileMetadata(OrcProto.Metadata.Builder builder) throws IOException
PhysicalWriterwriteFileMetadata in interface PhysicalWriterbuilder - Metadata builder to finalize and write.IOExceptionpublic void writeFileFooter(OrcProto.Footer.Builder builder) throws IOException
PhysicalWriterwriteFileFooter in interface PhysicalWriterbuilder - Footer builder to finalize and write.IOExceptionpublic long writePostScript(OrcProto.PostScript.Builder builder) throws IOException
PhysicalWriterwritePostScript in interface PhysicalWriterbuilder - Postscript builder to finalize and write.IOExceptionpublic void close()
throws IOException
PhysicalWriterclose in interface PhysicalWriterIOExceptionpublic void flush()
throws IOException
PhysicalWriterflush in interface PhysicalWriterIOExceptionpublic void appendRawStripe(ByteBuffer buffer, OrcProto.StripeInformation.Builder dirEntry) throws IOException
PhysicalWriterappendRawStripe in interface PhysicalWriterbuffer - Stripe data buffer.dirEntry - File metadata entry for the stripe, to be updated with
relevant data.IOExceptionpublic void finalizeStripe(OrcProto.StripeFooter.Builder footerBuilder, OrcProto.StripeInformation.Builder dirEntry) throws IOException
PhysicalWriterfinalizeStripe in interface PhysicalWriterfooterBuilder - Stripe footer to be updated with relevant data and written out.dirEntry - File metadata entry for the stripe, to be updated with
relevant data.IOExceptionpublic void writeHeader()
throws IOException
PhysicalWriterwriteHeader in interface PhysicalWriterIOExceptionpublic org.apache.orc.impl.PhysicalFsWriter.BufferedStream createDataStream(StreamName name)
PhysicalWritercreateDataStream in interface PhysicalWritername - the name of the streamprotected OutputStream createIndexStream(StreamName name)
public void writeIndex(StreamName name, OrcProto.RowIndex.Builder index) throws IOException
PhysicalWriterwriteIndex in interface PhysicalWritername - the name of the streamindex - the bloom filter to writeIOExceptionpublic void writeBloomFilter(StreamName name, OrcProto.BloomFilterIndex.Builder bloom) throws IOException
PhysicalWriterwriteBloomFilter in interface PhysicalWritername - the name of the streambloom - the bloom filter to writeIOExceptionpublic void writeStatistics(StreamName name, OrcProto.ColumnStatistics.Builder statistics)
PhysicalWriterwriteStatistics in interface PhysicalWritername - the name of the streamstatistics - the statistics to writeCopyright © 2013–2023 The Apache Software Foundation. All rights reserved.