karld.record_reader Package

karld.record_reader Package

When your data can be divided into logical units, but is each unit takes up varying amounts of multiple lines of a file, use this to consume them in those units. Just provide a function that takes a line and tells if it’s a start line or not.

karld.record_reader.__init__.multi_line_records(lines, is_line_start=None)

Iterate over lines, yielding a sequence for group of lines that end where the next multi-line record begins. The beginning of the record is determined by calling the given is_line_delimiter function, which is called on the every line.

Parameters:

• lines – An iterator of unicode lines.
• is_line_start (callable that returns if a line is the beginning of a record.) – determine the beginning line of a record.

Yields:

deque of lines.

app_engine Module

karld.record_reader.app_engine.is_log_start_line(line)

Is the line the start of a request log from Google App Engine.

Parameters:line – A string. Returns:True if the line doesn’t start with a tab.

karld.record_reader.app_engine.log_reader(file_path)

Iterate over request logs as written by a Google App Engine app.

Parameters:file_path – Path to an App Engine log file. Returns:An iterator of multi-line log records.