ParquetFormatter::Impl Struct Reference

Public Member Functions
void	checkAttr (const SumoXMLAttr attr)
template<class ATTR_TYPE , class BUILDER >
void	checkBuilder (const ATTR_TYPE &attr, const std::shared_ptr< arrow::DataType > &(*dataType)())
std::string	getAttrString (const std::string &attrString) const
	column-name lookup honoring the headerFormat option
	Impl (const std::string &columnNames, const int batchSize)

Data Fields
const int	myBatchSize
	the number of rows to write per batch
std::vector< std::shared_ptr< arrow::ArrayBuilder > >	myBuilders
	the content array builders for the table
bool	myCheckColumns = false
	whether the columns should be checked for completeness
parquet::Compression::type	myCompression = parquet::Compression::UNCOMPRESSED
	the compression to use
std::string	myCurrentTag
	the currently read tag (only valid when generating the header)
SumoXMLAttrMask	myExpectedAttrs
	the attributes which are expected for a complete row (including null values)
const std::string	myHeaderFormat
	the format to use for the column names
int	myMaxDepth = 2
	the maximum depth of the XML hierarchy
bool	myNeedsWrite = false
	whether there is still unwritten data
std::unique_ptr< parquet::arrow::FileWriter >	myParquetWriter
	the output stream writer
std::shared_ptr< arrow::Schema >	mySchema = arrow::schema({})
	the table schema
SumoXMLAttrMask	mySeenAttrs
	the attributes already seen (including null values)
std::vector< std::shared_ptr< arrow::Scalar > >	myValues
	the current attribute / column values
bool	myWroteHeader = false
	whether the schema has been constructed completely
std::vector< int >	myXMLStack
	The number of attributes in the currently open XML elements.

Detailed Description

Definition at line 97 of file ParquetFormatter.cpp.

Constructor & Destructor Documentation

Impl()

ParquetFormatter::Impl::Impl ( const std::string &  columnNames, const int  batchSize  )

inline

Definition at line 98 of file ParquetFormatter.cpp.

Member Function Documentation

checkAttr()

void ParquetFormatter::Impl::checkAttr ( const SumoXMLAttr  attr )

inline

Definition at line 162 of file ParquetFormatter.cpp.

References myCheckColumns, myExpectedAttrs, myMaxDepth, mySeenAttrs, myXMLStack, TLF, and toString().

checkBuilder()

template<class ATTR_TYPE , class BUILDER >

void ParquetFormatter::Impl::checkBuilder ( const ATTR_TYPE &  attr, const std::shared_ptr< arrow::DataType > &(*)()  dataType  )

inline

Definition at line 172 of file ParquetFormatter.cpp.

References getAttrString(), myBuilders, myNeedsWrite, mySchema, myValues, myWroteHeader, and toString().

getAttrString()

std::string ParquetFormatter::Impl::getAttrString ( const std::string &  attrString ) const

inline

column-name lookup honoring the headerFormat option

Definition at line 147 of file ParquetFormatter.cpp.

References myCurrentTag, myHeaderFormat, and mySchema.

Referenced by checkBuilder().

Here is the caller graph for this function:

Field Documentation

myBatchSize

const int ParquetFormatter::Impl::myBatchSize

the number of rows to write per batch

Definition at line 108 of file ParquetFormatter.cpp.

myBuilders

std::vector<std::shared_ptr<arrow::ArrayBuilder> > ParquetFormatter::Impl::myBuilders

the content array builders for the table

Definition at line 120 of file ParquetFormatter.cpp.

Referenced by checkBuilder().

myCheckColumns

bool ParquetFormatter::Impl::myCheckColumns = false

whether the columns should be checked for completeness

Definition at line 135 of file ParquetFormatter.cpp.

Referenced by checkAttr().

myCompression

parquet::Compression::type ParquetFormatter::Impl::myCompression = parquet::Compression::UNCOMPRESSED

the compression to use

Definition at line 105 of file ParquetFormatter.cpp.

myCurrentTag

std::string ParquetFormatter::Impl::myCurrentTag

the currently read tag (only valid when generating the header)

Definition at line 111 of file ParquetFormatter.cpp.

Referenced by getAttrString().

myExpectedAttrs

SumoXMLAttrMask ParquetFormatter::Impl::myExpectedAttrs

the attributes which are expected for a complete row (including null values)

Definition at line 141 of file ParquetFormatter.cpp.

Referenced by checkAttr().

myHeaderFormat

const std::string ParquetFormatter::Impl::myHeaderFormat

the format to use for the column names

Definition at line 102 of file ParquetFormatter.cpp.

Referenced by getAttrString().

myMaxDepth

int ParquetFormatter::Impl::myMaxDepth = 2

the maximum depth of the XML hierarchy

Definition at line 129 of file ParquetFormatter.cpp.

Referenced by checkAttr().

myNeedsWrite

bool ParquetFormatter::Impl::myNeedsWrite = false

whether there is still unwritten data

Definition at line 138 of file ParquetFormatter.cpp.

Referenced by checkBuilder().

myParquetWriter

std::unique_ptr<parquet::arrow::FileWriter> ParquetFormatter::Impl::myParquetWriter

the output stream writer

Definition at line 117 of file ParquetFormatter.cpp.

mySchema

std::shared_ptr<arrow::Schema> ParquetFormatter::Impl::mySchema = arrow::schema({})

the table schema

Definition at line 114 of file ParquetFormatter.cpp.

Referenced by checkBuilder(), and getAttrString().

mySeenAttrs

SumoXMLAttrMask ParquetFormatter::Impl::mySeenAttrs

the attributes already seen (including null values)

Definition at line 144 of file ParquetFormatter.cpp.

Referenced by checkAttr().

myValues

std::vector<std::shared_ptr<arrow::Scalar> > ParquetFormatter::Impl::myValues

the current attribute / column values

Definition at line 126 of file ParquetFormatter.cpp.

Referenced by checkBuilder().

myWroteHeader

bool ParquetFormatter::Impl::myWroteHeader = false

whether the schema has been constructed completely

Definition at line 132 of file ParquetFormatter.cpp.

Referenced by checkBuilder().

myXMLStack

std::vector<int> ParquetFormatter::Impl::myXMLStack

The number of attributes in the currently open XML elements.

Definition at line 123 of file ParquetFormatter.cpp.

Referenced by checkAttr().

---

The documentation for this struct was generated from the following file:

• ParquetFormatter.cpp