ext.sulis.data

Class CsvData

Object

CsvData

public class CsvData
extends Object

class to read text data from a file, with flexible support for all kinds of delimiters, comments, repeated header rows. allows to return a list or I*E group, or configure a callback method for processing row by row NOTE: default is to treat lines starting with a # sign as comments TODO: more examples

Author:

mkr

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAJOR
static int	MINOR

Constructor Summary

Constructors

Constructor and Description
CsvData()
CsvData(File file)
CsvData(String file)
CsvData(String[] hdrs, String file)

Method Summary

Methods

Modifier and Type	Method and Description
static void	addValsFromFile(String f, Vector v) add values from a file to a vector - each line is considered a value (for DiscreteSets)
String[]	getCharsets()
Group	getGroupFromCSV(String filename) return a group from a csv file, auto-detecting charset (unless explicitely specified) if no record delimiter char is explicitely set cr cr/nl and nl are accepted.
List<String>	getHeaders()
List<Map<String,String>>	getListFromCSV(String filename) return a group from a csv file, auto-detecting charset (unless explicitely specified) if no record delimiter char is explicitely set cr cr/nl and nl are accepted.
Group	getStringData(String data) return a group from a csv String if no record delimiter char is explicitely set cr cr/nl and nl are accepted.
static void	main(String[] args) commandline invocation to print version, usage, expose methods.
static String	nn(Object o)
static String	nn(Object o, String d)
void	process(String filename) process csv set by set, if CharConversionException occurrs will retry, skipping the already processed data sets.
int	process(String filename, Object... more) process csv set by set, if CharConversionException occurrs will retry, skipping the already processed data sets.
CsvData	setAttributeExcludeSet(Set<String> filterSet) set a Set to identify attributes to exclude - overrides include set.
CsvData	setAttributeIncludeSet(Set<String> filterSet) set a Set to identify attributes to include - overrides exclude set.
CsvData	setCallback(String classMethod) define per-row callback method, signature is (Map*lt;String, String>)
CsvData	setCallback2(String classMethod) define per-row callback method, signature is (Map*lt;String, String>)
CsvData	setCharsets(String[] charsets) set list of charsets to try reading the file with (defaults to utf-8, then latin1)
CsvData	setCommentSignature(String signature) set the character signature which identifies a comment line
CsvData	setFieldDelimiter(String newDelim) set the field delimiter character. if totally numeric make char from int value, else use first char;
CsvData	setFirstLineHeaders(boolean b) flag if first line contains column headers
void	setForceUniqueHeaders(boolean forceUniqueHeaders) switch on/off Exception if header line has empty or duplicate entries (default on)
void	setHeaders(List newHeaders) set the list of header names
void	setHeaders(String[] newHeaders) set the list of header names
CsvData	setMaxRows(int maxRows) set max number of rows to process
CsvData	setNewHeaderStart(String token) set the string which at the beginning of a line indicates new headers; it will be stripped off.
CsvData	setQuoteCharacter(String newQuote) set the quoting character (default "); if totally numeric make char from int value, else use first char; if String passed in is empty sets quoted to false and quoteChar back to ", else quoted is set to true.
CsvData	setQuoted(boolean b) flag if data is quoted with ""
CsvData	setRecordDelimiter(String newDelim) set the record delimiter character. if totally numeric make char from int value, else use first char;
void	setReportError(boolean flag) set flag if encountering an exception during process() should re-throw the exception or continue and return the number of bad invocations
CsvData	setTrim(boolean trim) toggle trim of whitespace around attribute values on/off
static void	testCallback(Map row)
void	write(List<List<Object>> data)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAJOR

public static final int MAJOR

See Also:

Constant Field Values

MINOR

public static final int MINOR

See Also:

Constant Field Values

Constructor Detail

CsvData

public CsvData()

CsvData

public CsvData(File file)
        throws Exception

Throws:

Exception

CsvData

public CsvData(String file)
        throws Exception

Throws:

Exception

CsvData

public CsvData(String[] hdrs,
       String file)
        throws Exception

Throws:

Exception

Method Detail

write

public void write(List<List<Object>> data)
           throws Exception

Throws:

Exception

getGroupFromCSV

public Group getGroupFromCSV(String filename)
                      throws Exception,
                             CharConversionException

return a group from a csv file, auto-detecting charset (unless explicitely specified)
if no record delimiter char is explicitely set cr cr/nl and nl are accepted.

Parameters:

filename - input data file name

Returns:

Group

Throws:

Exception - , CharConversionException

CharConversionException

getListFromCSV

public List<Map<String,String>> getListFromCSV(String filename)
                                        throws Exception,
                                               CharConversionException

return a group from a csv file, auto-detecting charset (unless explicitely specified)
if no record delimiter char is explicitely set cr cr/nl and nl are accepted.

Parameters:

filename - input data file name

Returns:

Group

Throws:

Exception - , CharConversionException

CharConversionException

process

public void process(String filename)
             throws Exception,
                    CharConversionException

process csv set by set, if CharConversionException occurrs will retry, skipping the already processed data sets.

Parameters:

filename - input data file name

Throws:

Exception - , CharConversionException

CharConversionException

process

public int process(String filename,
          Object... more)
            throws Exception,
                   CharConversionException

process csv set by set, if CharConversionException occurrs will retry, skipping the already processed data sets. uses callback method 2 with signature (Object[] args) - slot 0 holds the rows data map,m slots 1..n the args passed to process(filename, arg1, arg2 ...);

Parameters:

filename - input data file name

more - objects from the caller passed on to callback method - e.g. a list to accumulate the data read

Returns:

Throws:

Exception - , CharConversionException

CharConversionException

testCallback

public static void testCallback(Map row)

getStringData

public Group getStringData(String data)
                    throws Exception

return a group from a csv String
if no record delimiter char is explicitely set cr cr/nl and nl are accepted.

Parameters:

data - input String

Returns:

Group

Throws:

Exception

setMaxRows

public CsvData setMaxRows(int maxRows)

set max number of rows to process

Parameters:

maxRows - max, if 0 read unlimited

Returns:

self

setRecordDelimiter

public CsvData setRecordDelimiter(String newDelim)
                           throws Exception

set the record delimiter character. if totally numeric make char from int value, else use first char;

Parameters:

newDelim - delimiter string (\d+ or first char)

Returns:

self

Throws:

Exception

setQuoteCharacter

public CsvData setQuoteCharacter(String newQuote)
                          throws Exception

set the quoting character (default "); if totally numeric make char from int value, else use first char;
if String passed in is empty sets quoted to false and quoteChar back to ", else quoted is set to true.

Parameters:

newQuote - string (\d+ or first char)

Returns:

self

Throws:

Exception

setQuoted

public CsvData setQuoted(boolean b)
                  throws Exception

flag if data is quoted with ""

Parameters:

b -

Returns:

self

Throws:

Exception

setHeaders

public void setHeaders(List newHeaders)

set the list of header names

Parameters:

newHeaders - list of column names - make sure there are as many as values in the longest row!

setHeaders

public void setHeaders(String[] newHeaders)

set the list of header names

Parameters:

newHeaders - list of column names - make sure there are as many as values in the longest row!

setReportError

public void setReportError(boolean flag)

set flag if encountering an exception during process() should re-throw the exception or continue and return the number of bad invocations

Parameters:

flag -

setCallback

public CsvData setCallback(String classMethod)
                    throws Exception

define per-row callback method, signature is (Map*lt;String, String>)

Parameters:

className -

methodName - callback method name - needs signature methodName(Map<String, String>) csvData)

Returns:

self

Throws:

Exception - if method not found.

setCallback2

public CsvData setCallback2(String classMethod)
                     throws Exception

define per-row callback method, signature is (Map*lt;String, String>)

Parameters:

className -

methodName - callback method name - needs signature methodName(Map<String, String>) csvData)

Returns:

self

Throws:

Exception - if method not found.

setAttributeIncludeSet

public CsvData setAttributeIncludeSet(Set<String> filterSet)

set a Set to identify attributes to include - overrides exclude set.

Parameters:

filterSet -

Returns:

self

setAttributeExcludeSet

public CsvData setAttributeExcludeSet(Set<String> filterSet)

set a Set to identify attributes to exclude - overrides include set.

Parameters:

filterSet -

Returns:

self

setForceUniqueHeaders

public void setForceUniqueHeaders(boolean forceUniqueHeaders)

switch on/off Exception if header line has empty or duplicate entries (default on)

Parameters:

forceUniqueHeaders -

setTrim

public CsvData setTrim(boolean trim)

toggle trim of whitespace around attribute values on/off

Parameters:

trim -

Returns:

self

getCharsets

public String[] getCharsets()

getHeaders

public List<String> getHeaders()

setCharsets

public CsvData setCharsets(String[] charsets)

set list of charsets to try reading the file with (defaults to utf-8, then latin1)

Parameters:

charsets -

Returns:

self

setCommentSignature

public CsvData setCommentSignature(String signature)

set the character signature which identifies a comment line

Parameters:

signature -

Returns:

self

setFirstLineHeaders

public CsvData setFirstLineHeaders(boolean b)
                            throws Exception

flag if first line contains column headers

Parameters:

b -

Returns:

self

Throws:

Exception

setNewHeaderStart

public CsvData setNewHeaderStart(String token)
                          throws Exception

set the string which at the beginning of a line indicates new headers; it will be stripped off. e.g.,

 __NEWHDR__first\tsecond\third
 

will force the following lines into 3 atts first/second/third
this allows to build non-uniform groups from the data file.

Parameters:

token -

Returns:

self

Throws:

Exception

setFieldDelimiter

public CsvData setFieldDelimiter(String newDelim)
                          throws Exception

set the field delimiter character. if totally numeric make char from int value, else use first char;

Parameters:

newDelim - delimiter string (\d+ or first char)

Returns:

self

Throws:

Exception

addValsFromFile

public static void addValsFromFile(String f,
                   Vector v)
                            throws Exception

add values from a file to a vector - each line is considered a value (for DiscreteSets)

Parameters:

f - filename

v - vector to append to

Throws:

Exception

nn

public static String nn(Object o)

nn

public static String nn(Object o,
        String d)

main

public static void main(String[] args)
                 throws Exception

commandline invocation to print version, usage, expose methods.

Parameters:

args -

Throws:

Exception