Synopsis - Synopsis::Lexer

namespace Synopsis
class Lexer
File: ../Synopsis/Lexer.hh

a Lexer reads tokens from a stream.

Public Typedefs Summary:
std::vector<Token> Comments
Public Structs Summary:
struct InvalidChar
Public Enums Summary:
TokenSet C, CXX, GCC, MSVC

Define sets of token that are to be recognized as special keywords (as opposed to identifiers).

Public Member functions Summary:
constructor Lexer(Buffer*, int tokenset = CXX | GCC)

Construct a Lexer on the given Buffer using the given token set.

Token::Type get_token(Token&)
Token::Type look_ahead(size_t)
Token::Type look_ahead(size_t, Token&)
const char* save()
void restore(const char*)
Comments get_comments()
unsigned long origin(const char*, std::string&) const

Return the origin of the given pointer (filename and line number)

Private Classes Summary:
class Queue

a Queue is used to read in tokens from a stream without consuming them

Private Typedefs Summary:
std::map<std::string, Token::Type> Dictionary
Private Member functions Summary:
void rewind(const char*)
Token::Type read_token(const char*&, size_t&)
bool fill(size_t o)

try to fill the token cache to contain at least o tokens.

void skip_paren()

skip till end of paren

void skip_line()

skip till end of line

void skip_attribute()

skip __attribute__(...), ___asm__(...), ...

Token::Type skip_extension(const char*&, size_t&)

skip __extension__(...).

void skip_asm()

skip __asm ...

void skip_declspec()

skip __declspec(...).

void skip_pragma()

skip __pragma(...);.

char get_next_non_white_char()
Token::Type read_line()
bool read_char_const(unsigned long top)
bool read_str_const(unsigned long top)
Token::Type read_number(char c, unsigned long top)
Token::Type read_float(unsigned long top)
Token::Type read_identifier(unsigned long top)
Token::Type screen(const char* identifier, size_t len)
Token::Type read_separator(char c, unsigned long top)
Token::Type single_char_op(unsigned char c)
Token::Type read_comment(char c, unsigned long top)
Private Data members Summary:
Buffer* my_buffer
Queue my_tokens
Dictionary my_keywords
Token my_token
Comments my_comments
Enumerators Summary:
C
CXX
GCC
MSVC
Public Enums Details:
enum TokenSet
C = 0x0
CXX = 0x01
GCC = 0x02
MSVC = 0x04

Define sets of token that are to be recognized as special keywords (as opposed to identifiers). They can be or'ed. If CXX is not specified, the Lexer will operate in 'C mode'.

Public Member functions Details:
constructor Lexer(Buffer*, int tokenset = CXX | GCC)

Construct a Lexer on the given Buffer using the given token set. The default token set is CXX with GCC extensions.

unsigned long origin(const char*, std::string&) const

Return the origin of the given pointer (filename and line number)

Private Classes Details:
class Queue

a Queue is used to read in tokens from a stream without consuming them

Private Member functions Details:
bool fill(size_t o)

try to fill the token cache to contain at least o tokens. Returns false if there are not enough tokens.

void skip_paren()

skip till end of paren

void skip_line()

skip till end of line

void skip_attribute()

skip __attribute__(...), ___asm__(...), ...

Token::Type skip_extension(const char*&, size_t&)

skip __extension__(...).

void skip_asm()

skip __asm ...

void skip_declspec()

skip __declspec(...).

void skip_pragma()

skip __pragma(...);.

Generated on Thu Apr 16 16:27:34 2009 by
synopsis (version devel)