package core:bufio

Types

Loadahead_Reader ¶

Loadahead_Reader :: struct {
	r:   io.Reader,
	buf: []u8,
	n:   int,
}
 

Loadahead_Reader provides io lookahead. This is useful for tokenizers/parsers. Loadahead_Reader is similar to bufio.Reader, but unlike bufio.Reader, Loadahead_Reader's buffer size will EXACTLY match the specified size, whereas bufio.Reader's buffer size may differ from the specified size. This makes sure that the buffer will not be accidentally read beyond the expected size.

Read_Writer ¶

Read_Writer :: struct {
	r: ^Reader,
	w: ^Writer,
}
 

Read_Writer stores pointers to a Reader and a Writer

Reader ¶

Reader :: struct {
	buf:                         []u8,
	buf_allocator:               runtime.Allocator,
	rd:                          io.Reader,
	// reader
	r,
	w:                           int,
	// read and write positions for buf
	err:                         io.Error,
	last_byte:                   int,
	// last byte read, invalid is -1
	last_rune_size:              int,
	// size of last rune read, invalid is -1
	max_consecutive_empty_reads: int,
}
 

Reader is a buffered wrapper for an io.Reader

Scanner ¶

Scanner :: struct {
	r:                            io.Reader,
	split:                        Split_Proc,
	buf:                          [dynamic]u8,
	max_token_size:               int,
	start:                        int,
	end:                          int,
	token:                        []u8,
	_err:                         Scanner_Error,
	max_consecutive_empty_reads:  int,
	successive_empty_token_count: int,
	scan_called:                  bool,
	done:                         bool,
}

Scanner_Error ¶

Scanner_Error :: union {
	io.Error, 
	Scanner_Extra_Error, 
}

Scanner_Extra_Error ¶

Scanner_Extra_Error :: enum i32 {
	None, 
	Negative_Advance, 
	Advanced_Too_Far, 
	Bad_Read_Count, 
	Too_Long, 
	Too_Short, 
}
 

Extra errors returns by scanning procedures

Split_Proc ¶

Split_Proc :: proc "odin" (data: []u8, at_eof: bool) -> (advance: int, token: []u8, err: Scanner_Error, final_token: bool)
 

Split_Proc is the signature of the split procedure used to tokenize the input.

Writer ¶

Writer :: struct {
	buf:                          []u8,
	buf_allocator:                runtime.Allocator,
	wr:                           io.Writer,
	n:                            int,
	err:                          io.Error,
	max_consecutive_empty_writes: int,
}
 

Writer is a buffered wrapper for an io.Writer

Constants

DEFAULT_BUF_SIZE ¶

DEFAULT_BUF_SIZE :: 4096

DEFAULT_MAX_SCAN_TOKEN_SIZE ¶

DEFAULT_MAX_SCAN_TOKEN_SIZE :: 1 << 16

Variables

This section is empty.

Procedures

lookahead_reader_buffer ¶

lookahead_reader_buffer :: proc "odin" (lr: ^Loadahead_Reader) -> []u8 {…}

lookahead_reader_consume ¶

lookahead_reader_consume :: proc "odin" (lr: ^Loadahead_Reader, n: int) -> io.Error {…}
 

lookahead_reader_consume drops the first n populated bytes from the Lookahead_Reader.

lookahead_reader_consume_all ¶

lookahead_reader_consume_all :: proc "odin" (lr: ^Loadahead_Reader) -> io.Error {…}

lookahead_reader_init ¶

lookahead_reader_init :: proc "odin" (lr: ^Loadahead_Reader, r: io.Reader, buf: []u8) -> ^Loadahead_Reader {…}

lookahead_reader_peek ¶

lookahead_reader_peek :: proc "odin" (lr: ^Loadahead_Reader, n: int) -> ([]u8, io.Error) {…}
 

lookahead_reader_peek returns a slice of the Lookahead_Reader which holds n bytes If the Lookahead_Reader cannot hold enough bytes, it will read from the underlying reader to populate the rest. NOTE: The returned buffer is not a copy of the underlying buffer

lookahead_reader_peek_all ¶

lookahead_reader_peek_all :: proc "odin" (lr: ^Loadahead_Reader) -> ([]u8, io.Error) {…}
 

lookahead_reader_peek_all returns a slice of the Lookahead_Reader populating the full buffer If the Lookahead_Reader cannot hold enough bytes, it will read from the underlying reader to populate the rest. NOTE: The returned buffer is not a copy of the underlying buffer

read_writer_init ¶

read_writer_init :: proc "odin" (rw: ^Read_Writer, r: ^Reader, w: ^Writer) {…}

read_writer_to_stream ¶

read_writer_to_stream :: proc "odin" (rw: ^Read_Writer) -> (s: io.Stream) {…}

reader_buffered ¶

reader_buffered :: proc "odin" (b: ^Reader) -> int {…}
 

reader_buffered returns the number of bytes that can be read from the current buffer

reader_destroy ¶

reader_destroy :: proc "odin" (b: ^Reader) {…}
 

reader_destroy destroys the underlying buffer with its associated allocator IFF that allocator has been set

reader_discard ¶

reader_discard :: proc "odin" (b: ^Reader, n: int) -> (discarded: int, err: io.Error) {…}
 

reader_discard skips the next n bytes, and returns the number of bytes that were discarded

reader_init ¶

reader_init :: proc "odin" (b: ^Reader, rd: io.Reader, size: int = DEFAULT_BUF_SIZE, allocator := context.allocator) {…}

reader_init_with_buf ¶

reader_init_with_buf :: proc "odin" (b: ^Reader, rd: io.Reader, buf: []u8) {…}

reader_peek ¶

reader_peek :: proc "odin" (b: ^Reader, n: int) -> (data: []u8, err: io.Error) {…}
 

reader_peek returns the next n bytes without advancing the reader The bytes stop being valid on the next read call If reader_peek returns fewer than n bytes, it also return an error explaining why the read is short The error will be .Buffer_Full if n is larger than the internal buffer size

reader_read ¶

reader_read :: proc "odin" (b: ^Reader, p: []u8) -> (n: int, err: io.Error) {…}
 

reader_read reads data into p The bytes are taken from at most one read on the underlying Reader, which means n may be less than len(p)

reader_read_byte ¶

reader_read_byte :: proc "odin" (b: ^Reader) -> (u8, io.Error) {…}
 

reader_read_byte reads and returns a single byte If no byte is available, it return an error

reader_read_bytes ¶

reader_read_bytes :: proc "odin" (b: ^Reader, delim: u8, allocator := context.allocator) -> (buf: []u8, err: io.Error) {…}
 

reader_read_bytes reads until the first occurrence of delim from the Reader It returns an allocated slice containing the data up to and including the delimiter

reader_read_rune ¶

reader_read_rune :: proc "odin" (b: ^Reader) -> (r: rune, size: int, err: io.Error) {…}
 

reader_read_rune reads a single UTF-8 encoded unicode character and returns the rune and its size in bytes If the encoded rune is invalid, it consumes one byte and returns utf8.RUNE_ERROR (U+FFFD) with a size of 1

reader_read_slice ¶

reader_read_slice :: proc "odin" (b: ^Reader, delim: u8) -> (line: []u8, err: io.Error) {…}
 

reader_read_slice reads until the first occurrence of delim from the reader It returns a slice pointing at the bytes in the buffer The bytes stop being valid at the next read If reader_read_slice encounters an error before finding a delimiter reader_read_slice fails with error .Buffer_Full if the buffer fills without a delim Because the data returned from reader_read_slice will be overwritten on the next IO operation, reader_read_bytes or reader_read_string is usually preferred

reader_read_slice returns err != nil if and only if line does not end in delim

reader_read_string ¶

reader_read_string :: proc "odin" (b: ^Reader, delim: u8, allocator := context.allocator) -> (string, io.Error) {…}
 

reader_read_string reads until the first occurrence of delim from the Reader It returns an allocated string containing the data up to and including the delimiter

reader_reset ¶

reader_reset :: proc "odin" (b: ^Reader, r: io.Reader) {…}

reader_size ¶

reader_size :: proc "odin" (b: ^Reader) -> int {…}

reader_to_stream ¶

reader_to_stream :: proc "odin" (b: ^Reader) -> (s: io.Stream) {…}
 

reader_to_stream converts a Reader into an io.Stream

reader_unread_byte ¶

reader_unread_byte :: proc "odin" (b: ^Reader) -> io.Error {…}
 

reader_unread_byte unreads the last byte. Only the most recently read byte can be unread

reader_unread_rune ¶

reader_unread_rune :: proc "odin" (b: ^Reader) -> io.Error {…}
 

reader_unread_rune unreads the last rune. Only the most recently read rune can be unread

reader_write_to ¶

reader_write_to :: proc "odin" (b: ^Reader, w: io.Writer) -> (n: i64, err: io.Error) {…}

scan_bytes ¶

scan_bytes :: proc "odin" (data: []u8, at_eof: bool) -> (advance: int, token: []u8, err: Scanner_Error, final_token: bool) {…}

scan_lines ¶

scan_lines :: proc "odin" (data: []u8, at_eof: bool) -> (advance: int, token: []u8, err: Scanner_Error, final_token: bool) {…}

scan_runes ¶

scan_runes :: proc "odin" (data: []u8, at_eof: bool) -> (advance: int, token: []u8, err: Scanner_Error, final_token: bool) {…}

scan_words ¶

scan_words :: proc "odin" (data: []u8, at_eof: bool) -> (advance: int, token: []u8, err: Scanner_Error, final_token: bool) {…}

scanner_bytes ¶

scanner_bytes :: proc "odin" (s: ^Scanner) -> []u8 {…}
 

Returns the most recent token created by scanner_scan. The underlying array may point to data that may be overwritten by another call to scanner_scan. Treat the returned value as if it is immutable.

scanner_destroy ¶

scanner_destroy :: proc "odin" (s: ^Scanner) {…}

scanner_error ¶

scanner_error :: proc "odin" (s: ^Scanner) -> Scanner_Error {…}
 

Returns the first non-EOF error that was encountered by the scanner

scanner_init ¶

scanner_init :: proc "odin" (s: ^Scanner, r: io.Reader, buf_allocator := context.allocator) -> ^Scanner {…}

scanner_init_with_buffer ¶

scanner_init_with_buffer :: proc "odin" (s: ^Scanner, r: io.Reader, buf: []u8) -> ^Scanner {…}

scanner_scan ¶

scanner_scan :: proc "odin" (s: ^Scanner) -> bool {…}
 

scanner_scan advances the scanner

scanner_text ¶

scanner_text :: proc "odin" (s: ^Scanner) -> string {…}
 

Returns the most recent token created by scanner_scan. The underlying array may point to data that may be overwritten by another call to scanner_scan. Treat the returned value as if it is immutable.

writer_available ¶

writer_available :: proc "odin" (b: ^Writer) -> int {…}
 

writer_available returns how many bytes are unused in the buffer

writer_buffered ¶

writer_buffered :: proc "odin" (b: ^Writer) -> int {…}
 

writer_buffered returns the number of bytes that have been writted into the current buffer

writer_destroy ¶

writer_destroy :: proc "odin" (b: ^Writer) {…}
 

writer_destroy destroys the underlying buffer with its associated allocator IFF that allocator has been set

writer_flush ¶

writer_flush :: proc "odin" (b: ^Writer) -> io.Error {…}
 

writer_flush writes any buffered data into the underlying io.Writer

writer_init ¶

writer_init :: proc "odin" (b: ^Writer, wr: io.Writer, size: int = DEFAULT_BUF_SIZE, allocator := context.allocator) {…}

writer_init_with_buf ¶

writer_init_with_buf :: proc "odin" (b: ^Writer, wr: io.Writer, buf: []u8) {…}

writer_read_from ¶

writer_read_from :: proc "odin" (b: ^Writer, r: io.Reader) -> (n: i64, err: io.Error) {…}
 

writer_read_from is to support io.Reader_From types If the underlying writer supports the io,read_from, and b has no buffered data yet, this procedure calls the underlying read_from implementation without buffering

writer_reset ¶

writer_reset :: proc "odin" (b: ^Writer, w: io.Writer) {…}

writer_size ¶

writer_size :: proc "odin" (b: ^Writer) -> int {…}
 

writer_size returns the size of underlying buffer in bytes

writer_to_stream ¶

writer_to_stream :: proc "odin" (b: ^Writer) -> (s: io.Stream) {…}
 

writer_to_stream converts a Writer into an io.Stream

writer_write ¶

writer_write :: proc "odin" (b: ^Writer, p: []u8) -> (n: int, err: io.Error) {…}
 

writer_write writes the contents of p into the buffer It returns the number of bytes written If n < len(p), it will return an error explaining why the write is short

writer_write_byte ¶

writer_write_byte :: proc "odin" (b: ^Writer, c: u8) -> io.Error {…}
 

writer_write_byte writes a single byte

writer_write_rune ¶

writer_write_rune :: proc "odin" (b: ^Writer, r: rune) -> (size: int, err: io.Error) {…}
 

writer_write_rune writes a single unicode code point, and returns the number of bytes written with any error

writer_write_string ¶

writer_write_string :: proc "odin" (b: ^Writer, s: string) -> (int, io.Error) {…}
 

writer_write writes a string into the buffer It returns the number of bytes written If n < len(p), it will return an error explaining why the write is short

Procedure Groups

This section is empty.

Source Files

Generation Information

Generated with odin version dev-2022-10 (vendor "odin") Windows_amd64 @ 2022-10-05 21:11:47.289279100 +0000 UTC