Package com.univocity.parsers.common

Class CommonParserSettings<F extends Format>

java.lang.Object
com.univocity.parsers.common.CommonSettings<F>
com.univocity.parsers.common.CommonParserSettings<F>
Type Parameters:
F - the format supported by this parser.
All Implemented Interfaces:
Cloneable
Direct Known Subclasses:
CsvParserSettings, FixedWidthParserSettings, TsvParserSettings
public abstract class CommonParserSettings<F extends Format> extends CommonSettings<F>
This is the parent class for all configuration classes used by parsers (AbstractParser)

By default, all parsers work with, at least, the following configuration options in addition to the ones provided by CommonSettings:

  • rowProcessor: a callback implementation of the interface RowProcessor which handles the life cycle of the parsing process and processes each record extracted from the input
  • headerExtractionEnabled (defaults to false): indicates whether or not the first valid record parsed from the input should be considered as the row containing the names of each column
  • columnReorderingEnabled (defaults to true): indicates whether fields selected using the field selection methods (defined by the parent class CommonSettings) should be reordered.

    When disabled, each parsed record will contain values for all columns, in the order they occur in the input. Fields which were not selected will not be parsed but and the record will contain empty values.

    When enabled, each parsed record will contain values only for the selected columns. The values will be ordered according to the selection.

  • inputBufferSize (defaults to 1024*1024 characters): The number of characters held by the parser's buffer when processing the input.
  • readInputOnSeparateThread (defaults true if the number of available processors at runtime is greater than 1):

    When enabled, a reading thread (in input.concurrent.ConcurrentCharInputReader) will be started and load characters from the input, while the parser is processing its input buffer. This yields better performance, especially when reading from big input (greater than 100 mb)

    When disabled, the parsing process will briefly pause so the buffer can be replenished every time it is exhausted (in DefaultCharInputReader it is not as bad or slow as it sounds, and can even be (slightly) more efficient if your input is small)

  • numberOfRecordsToRead (defaults to -1): Defines how many (valid) records are to be parsed before the process is stopped. A negative value indicates there's no limit.
  • lineSeparatorDetectionEnabled (defaults to false): Attempts to identify what is the line separator being used in the input. The first row of the input will be read until a sequence of '\r\n', or characters '\r' or '\n' is found. If a match is found, then it will be used as the line separator to use to parse the input
See Also:

  • Field Details

    • headerExtractionEnabled

      protected Boolean headerExtractionEnabled

    • processor

      private Processor<? extends Context> processor

    • columnReorderingEnabled

      private boolean columnReorderingEnabled

    • inputBufferSize

      private int inputBufferSize

    • readInputOnSeparateThread

      private boolean readInputOnSeparateThread

    • numberOfRecordsToRead

      private long numberOfRecordsToRead

    • lineSeparatorDetectionEnabled

      private boolean lineSeparatorDetectionEnabled

    • numberOfRowsToSkip

      private long numberOfRowsToSkip

    • commentCollectionEnabled

      private boolean commentCollectionEnabled

    • autoClosingEnabled

      private boolean autoClosingEnabled

    • commentProcessingEnabled

      private boolean commentProcessingEnabled

    • inputAnalysisProcesses

      private List<InputAnalysisProcess> inputAnalysisProcesses

  • Constructor Details

    • CommonParserSettings

      public CommonParserSettings()

  • Method Details

    • getReadInputOnSeparateThread

      public boolean getReadInputOnSeparateThread()
      Indicates whether or not a separate thread will be used to read characters from the input while parsing (defaults true if the number of available processors at runtime is greater than 1)

      When enabled, a reading thread (in com.univocity.parsers.common.input.concurrent.ConcurrentCharInputReader) will be started and load characters from the input, while the parser is processing its input buffer. This yields better performance, especially when reading from big input (greater than 100 mb)

      When disabled, the parsing process will briefly pause so the buffer can be replenished every time it is exhausted (in DefaultCharInputReader it is not as bad or slow as it sounds, and can even be (slightly) more efficient if your input is small)

      Returns:
      true if the input should be read on a separate thread, false otherwise

    • setReadInputOnSeparateThread

      public void setReadInputOnSeparateThread(boolean readInputOnSeparateThread)
      Defines whether or not a separate thread will be used to read characters from the input while parsing (defaults true if the number of available processors at runtime is greater than 1)

      When enabled, a reading thread (in com.univocity.parsers.common.input.concurrent.ConcurrentCharInputReader) will be started and load characters from the input, while the parser is processing its input buffer. This yields better performance, especially when reading from big input (greater than 100 mb)

      When disabled, the parsing process will briefly pause so the buffer can be replenished every time it is exhausted (in DefaultCharInputReader it is not as bad or slow as it sounds, and can even be (slightly) more efficient if your input is small)

      Parameters:
      readInputOnSeparateThread - the flag indicating whether or not the input should be read on a separate thread

    • isHeaderExtractionEnabled

      public boolean isHeaderExtractionEnabled()
      Indicates whether or not the first valid record parsed from the input should be considered as the row containing the names of each column
      Returns:
      true if the first valid record parsed from the input should be considered as the row containing the names of each column, false otherwise

    • setHeaderExtractionEnabled

      public void setHeaderExtractionEnabled(boolean headerExtractionEnabled)
      Defines whether or not the first valid record parsed from the input should be considered as the row containing the names of each column
      Parameters:
      headerExtractionEnabled - a flag indicating whether the first valid record parsed from the input should be considered as the row containing the names of each column

    • getRowProcessor

      @Deprecated public RowProcessor getRowProcessor()
      Deprecated.
      Use the getProcessor() method as it allows format-specific processors to be built to work with different implementations of Context. Implementations based on RowProcessor allow only parsers who provide a ParsingContext to be used.
      Returns the callback implementation of the interface RowProcessor which handles the lifecycle of the parsing process and processes each record extracted from the input
      Returns:
      Returns the RowProcessor used by the parser to handle each record
      See Also:

    • setRowProcessor

      @Deprecated public void setRowProcessor(RowProcessor processor)
      Deprecated.
      Use the setProcessor(Processor) method as it allows format-specific processors to be built to work with different implementations of Context. Implementations based on RowProcessor allow only parsers who provide a ParsingContext to be used.
      Defines the callback implementation of the interface RowProcessor which handles the lifecycle of the parsing process and processes each record extracted from the input
      Parameters:
      processor - the RowProcessor instance which should used by the parser to handle each record
      See Also:

    • getProcessor

      public <T extends Context> Processor<T> getProcessor()
      Returns the callback implementation of the interface Processor which handles the lifecycle of the parsing process and processes each record extracted from the input
      Type Parameters:
      T - the context type supported by the parser implementation.
      Returns:
      Returns the Processor used by the parser to handle each record
      See Also:

    • setProcessor

      public void setProcessor(Processor<? extends Context> processor)
      Defines the callback implementation of the interface Processor which handles the lifecycle of the parsing process and processes each record extracted from the input
      Parameters:
      processor - the Processor instance which should used by the parser to handle each record
      See Also:

    • newCharInputReader

      protected CharInputReader newCharInputReader(int whitespaceRangeStart)
      An implementation of CharInputReader which loads the parser buffer in parallel or sequentially, as defined by the readInputOnSeparateThread property
      Parameters:
      whitespaceRangeStart - starting range of characters considered to be whitespace.
      Returns:
      The input reader as chosen with the readInputOnSeparateThread property.

    • getNumberOfRecordsToRead

      public long getNumberOfRecordsToRead()
      The number of valid records to be parsed before the process is stopped. A negative value indicates there's no limit (defaults to -1).
      Returns:
      the number of records to read before stopping the parsing process.

    • setNumberOfRecordsToRead

      public void setNumberOfRecordsToRead(long numberOfRecordsToRead)
      Defines the number of valid records to be parsed before the process is stopped. A negative value indicates there's no limit (defaults to -1).
      Parameters:
      numberOfRecordsToRead - the number of records to read before stopping the parsing process.

    • isColumnReorderingEnabled

      public boolean isColumnReorderingEnabled()
      Indicates whether fields selected using the field selection methods (defined by the parent class CommonSettings) should be reordered (defaults to true).

      When disabled, each parsed record will contain values for all columns, in the order they occur in the input. Fields which were not selected will not be parsed but and the record will contain empty values.

      When enabled, each parsed record will contain values only for the selected columns. The values will be ordered according to the selection.

      Returns:
      true if the selected fields should be reordered and returned by the parser, false otherwise

    • getFieldSet

      FieldSet<?> getFieldSet()
      Returns the set of selected fields, if any
      Overrides:
      getFieldSet in class CommonSettings<F extends Format>
      Returns:
      the set of selected fields. Null if no field was selected/excluded

    • getFieldSelector

      FieldSelector getFieldSelector()
      Returns the FieldSelector object, which handles selected fields.
      Overrides:
      getFieldSelector in class CommonSettings<F extends Format>
      Returns:
      the FieldSelector object, which handles selected fields. Null if no field was selected/excluded

    • setColumnReorderingEnabled

      public void setColumnReorderingEnabled(boolean columnReorderingEnabled)
      Defines whether fields selected using the field selection methods (defined by the parent class CommonSettings) should be reordered (defaults to true).

      When disabled, each parsed record will contain values for all columns, in the order they occur in the input. Fields which were not selected will not be parsed but the record will contain empty values.

      When enabled, each parsed record will contain values only for the selected columns. The values will be ordered according to the selection.

      Parameters:
      columnReorderingEnabled - the flag indicating whether or not selected fields should be reordered and returned by the parser

    • getInputBufferSize

      public int getInputBufferSize()
      Informs the number of characters held by the parser's buffer when processing the input (defaults to 1024*1024 characters).
      Returns:
      the number of characters held by the parser's buffer when processing the input

    • setInputBufferSize

      public void setInputBufferSize(int inputBufferSize)
      Defines the number of characters held by the parser's buffer when processing the input (defaults to 1024*1024 characters).
      Parameters:
      inputBufferSize - the new input buffer size (in number of characters)

    • newCharAppender

      protected CharAppender newCharAppender()
      Returns an instance of CharAppender with the configured limit of maximum characters per column and the default value used to represent a null value (when the String parsed from the input is empty)
      Returns:
      an instance of CharAppender with the configured limit of maximum characters per column and the default value used to represent a null value (when the String parsed from the input is empty)

    • isLineSeparatorDetectionEnabled

      public final boolean isLineSeparatorDetectionEnabled()
      Indicates whether the parser should detect the line separator automatically.
      Returns:
      true if the first line of the input should be used to search for common line separator sequences (the matching sequence will be used as the line separator for parsing). Otherwise false.

    • setLineSeparatorDetectionEnabled

      public final void setLineSeparatorDetectionEnabled(boolean lineSeparatorDetectionEnabled)
      Defines whether the parser should detect the line separator automatically.
      Parameters:
      lineSeparatorDetectionEnabled - a flag indicating whether the first line of the input should be used to search for common line separator sequences (the matching sequence will be used as the line separator for parsing).

    • getNumberOfRowsToSkip

      public final long getNumberOfRowsToSkip()
      Returns the number of rows to skip from the input before the parser can begin to execute.
      Returns:
      number of rows to skip before parsing

    • setNumberOfRowsToSkip

      public final void setNumberOfRowsToSkip(long numberOfRowsToSkip)
      Defines a number of rows to skip from the input before the parser can begin to execute.
      Parameters:
      numberOfRowsToSkip - number of rows to skip before parsing

    • addConfiguration

      protected void addConfiguration(Map<String,Object> out)
      Overrides:
      addConfiguration in class CommonSettings<F extends Format>

    • preventReordering

      private boolean preventReordering()

    • isCommentCollectionEnabled

      public boolean isCommentCollectionEnabled()
      Indicates that comments found in the input must be collected (disabled by default). If enabled, comment lines will be stored by the parser and made available via AbstractParser.getContext().comments() and AbstractParser.getContext().lastComment()
      Returns:
      a flag indicating whether or not to enable collection of comments.

    • setCommentCollectionEnabled

      public void setCommentCollectionEnabled(boolean commentCollectionEnabled)
      Enables collection of comments found in the input (disabled by default). If enabled, comment lines will be stored by the parser and made available via AbstractParser.getContext().comments() and AbstractParser.getContext().lastComment()
      Parameters:
      commentCollectionEnabled - flag indicating whether or not to enable collection of comments.

    • runAutomaticConfiguration

      final void runAutomaticConfiguration()
      Overrides:
      runAutomaticConfiguration in class CommonSettings<F extends Format>

    • configureFromAnnotations

      protected void configureFromAnnotations(Class<?> beanClass)
      Configures the parser based on the annotations provided in a given class
      Parameters:
      beanClass - the classes whose annotations will be processed to derive configurations for parsing

    • clone

      protected CommonParserSettings clone(boolean clearInputSpecificSettings)
      Description copied from class: CommonSettings
      Clones this configuration object to reuse user-provided settings. Properties that are specific to a given input (such as header names and selection of fields) can be reset to their defaults if the clearInputSpecificSettings flag is set to true
      Overrides:
      clone in class CommonSettings<F extends Format>
      Parameters:
      clearInputSpecificSettings - flag indicating whether to clear settings that are likely to be associated with a given input.
      Returns:
      a copy of the configurations applied to the current instance.

    • clone

      protected CommonParserSettings clone()
      Description copied from class: CommonSettings
      Clones this configuration object. Use alternative CommonSettings.clone(boolean) method to reset properties that are specific to a given input, such as header names and selection of fields.
      Overrides:
      clone in class CommonSettings<F extends Format>
      Returns:
      a copy of all configurations applied to the current instance.

    • clearInputSpecificSettings

      protected void clearInputSpecificSettings()
      Description copied from class: CommonSettings
      Clears settings that are likely to be specific to a given input.
      Overrides:
      clearInputSpecificSettings in class CommonSettings<F extends Format>

    • isAutoClosingEnabled

      public boolean isAutoClosingEnabled()
      Indicates whether automatic closing of the input (reader, stream, etc) is enabled. If true, the parser will always close the input automatically when all records have been parsed or when an error occurs. Defaults to true
      Returns:
      flag indicating whether automatic input closing is enabled.

    • setAutoClosingEnabled

      public void setAutoClosingEnabled(boolean autoClosingEnabled)
      Configures whether the parser should always close the input (reader, stream, etc) automatically when all records have been parsed or when an error occurs. Defaults to true
      Parameters:
      autoClosingEnabled - flag determining whether automatic input closing should be enabled.

    • isCommentProcessingEnabled

      public boolean isCommentProcessingEnabled()
      Indicates whether code will check for comment line in the data file is enabled. If true, the parser will always check for the comment line default value for comment check is # Defaults to true
      Returns:
      flag indicating whether parser will check for the comment line If disabled/false then parser wont treat any line as comment line including default(#) this condition will supersede the comment character(#)

    • setCommentProcessingEnabled

      public void setCommentProcessingEnabled(boolean commentProcessingEnabled)
      Configures whether the parser will check for the comment line in the file Defaults to true
      Parameters:
      commentProcessingEnabled - flag determining whether comment line check should be performed If disabled/false then parser wont treat any line as comment line including default(#) this condition will supersede the comment character(#)

    • addInputAnalysisProcess

      public void addInputAnalysisProcess(InputAnalysisProcess inputAnalysisProcess)
      Provides a custom InputAnalysisProcess to analyze the input buffer and potentially discover configuration options such as column separators is CSV, data formats, etc. The process will be execute only once.
      Parameters:
      inputAnalysisProcess - a custom process to analyze the contents of the first input buffer loaded when the parsing starts.

    • getInputAnalysisProcesses

      public List<InputAnalysisProcess> getInputAnalysisProcesses()
      Returns the sequence of InputAnalysisProcess to be used for analyzing the input buffer and potentially discover configuration options such as column separators is CSV, data formats, etc. Each process will be execute only once.
      Returns:
      the list of custom processes to analyze the contents of the first input buffer loaded when the parsing starts.