Castle Game Engine: X3DLexer: Class TX3DLexer

# Class TX3DLexer
Description | Hierarchy | Fields | Methods | Properties
## Unit X3DLexer ## Declaration `type TX3DLexer = class(TObject)` ## Description VRML/X3D (classic encoding) lexer. The lexer always "looks" (i.e. contains in Token and TokenXxx fields) at the next not yet interpreted token. Remember that VRML is case-sensitive, so TokenName and TokenString should be compared in case-sensitive manner. Also note that for VRML/X3D >= 2.0 these fields contain UTF-8 encoded strings. Note that this lexer can read only from TPeekCharStream, not just from any TStream. You may have to wrap your stream in some TPeekCharStream descendant (for example create TFileStream and then wrap it inside TBufferedReadStream). ## Hierarchy TObject TX3DLexer ## Overview ### Methods
| `constructor Create(AStream: TPeekCharStream; AOwnsStream: boolean);`
| `constructor CreateForPartialStream( AStream: TPeekCharStream; AOwnsStream: boolean; const AVersion: TX3DVersion); overload;`
| `constructor CreateForPartialStream(const S: string; const AVersion: TX3DVersion); overload;`
| `destructor Destroy; override;`
| `function NextToken: TX3DToken;`
| `procedure NextTokenForceVTName;`
| `procedure NextTokenForceVTString;`
| `function TokenIsKeyword(const Keyword: TX3DKeyword): boolean; overload;`
| `function TokenIsKeyword(const Keywords: TX3DKeywords): boolean; overload;`
| `function DescribeToken: string;`
| `procedure CheckTokenIs(Tok: TX3DToken); overload;`
| `procedure CheckTokenIs(Tok: TX3DToken; const TokDescription: string); overload;`
| `procedure CheckTokenIs(const Toks: TX3DTokens; const ToksDescription: string); overload;`
| `procedure CheckTokenIsKeyword(const Keyword: TX3DKeyword);`
### Properties
| `property Stream: TPeekCharStream read FStream;`
| `property Version: TX3DVersion read fVersion;`
| `property Token: TX3DToken read fToken;`
| `property TokenKeyword: TX3DKeyword read fTokenKeyword;`
| `property TokenName: string read fTokenName;`
| `property TokenFloat: Float read fTokenFloat;`
| `property TokenInteger: Int64 read fTokenInteger;`
| `property TokenString: string read fTokenString;`
## Description ### Methods
| `constructor Create(AStream: TPeekCharStream; AOwnsStream: boolean);`
Standard constructor. After constructor call, Version is already set, it's checked that file is not compressed by gzip, and the first Token is already read. ###### Exceptions raised EX3DGzipCompressed If the Stream starts with gzip file header.
| `constructor CreateForPartialStream( AStream: TPeekCharStream; AOwnsStream: boolean; const AVersion: TX3DVersion); overload;`
Constructor for the case when you only have part of normal VRML tokens stream. This is particularly useful to parse fields of X3D in XML encoding. Inside XML attributes we have then a text that can parsed with a classical VRML lexer, to parse fields contents. This creates a lexer that works quite like a normal lexer. At creation time it doesn't expect header line (like `#VRML 2.0 utf8`), that why you have to supply VRML major and minor version as parameters here. Also it doesn't try to detect gzip header. It simply behaves like we're in the middle of VRML tokens stream. Overloaded version with a first parameter as string simply reads tokens from this string (wrapping it in TStringStream and TPeekCharStream).
| `constructor CreateForPartialStream(const S: string; const AVersion: TX3DVersion); overload;`
| `destructor Destroy; override;`
| `function NextToken: TX3DToken;`
`NextToken` reads next token from stream, initializing appropriately all Token* properties. For comfort, this returs the new value of Token property.
| `procedure NextTokenForceVTName;`
Read the next token, knowing that it must be vtName token. This is basically a dirty hack to read some incorrect VRML files, that use not allowed characters in VRML names. This allows us to accept as a vtName some characters that normally (when using normal NextToken) would get interpreted as other token. For example, mgf2inv can write name `0` (a zero, that would be read as vtInteger token in normal circumstances), on some WWW page I found sample VRML models with node name `"Crab!"` (yes, with exclamation mark and double quotes as part of the node name). ###### Exceptions raised EX3DParserError When we really really cannot interpret contents as vtName token here — currently this may happen only if end of stream is reached. Note that this is reported as a parsing error.
| `procedure NextTokenForceVTString;`
Read the next token, knowing that it must be vtString token. Similiar to NextTokenForceVTName: use this like a shortcut for
      NextToken;
      CheckTokenIs(vtString);
but it is not equivalent to such instructions. This is because VRML 1.0 allowed rather strange thing: string may be not enclosed in double quotes if it does not contain a space. This "feature" is not present in VRML >= 2.0, but, unfortunately, I'm trying to handle VRML 1.0 here so I have to conform to this specification. In particular, Blender generates VRML 1.0 files with Texture2.filename fields not enclosed in double quotes. So this "feature" is actually used by someone... So I have to implement this. Usual NextToken will not be able to return vtString if it approaches a string not enclosed in double quotes. But THIS function will be able to handle it. So always use this function when you expect a string, this ensures that we will correctly parse any valid VRML 1.0 file. (unfortunately I'm not doing this now when parsing MFString, this would just require too "unclean" code; I'm using this function only before calling parse on SFString field from TX3DNode.Parse.)
| `function TokenIsKeyword(const Keyword: TX3DKeyword): boolean; overload;`
Returns if Token is vtKeyword and TokenKeyword is given Keyword.
| `function TokenIsKeyword(const Keywords: TX3DKeywords): boolean; overload;`
| `function DescribeToken: string;`
Nice textual description of current token, suitable to show to user.
| `procedure CheckTokenIs(Tok: TX3DToken); overload;`
Check is token = Tok, if not -> parser error "expected token 'tok'". You can provide your own description for Tok or default desciption for token will be used.
| `procedure CheckTokenIs(Tok: TX3DToken; const TokDescription: string); overload;`
| `procedure CheckTokenIs(const Toks: TX3DTokens; const ToksDescription: string); overload;`
| `procedure CheckTokenIsKeyword(const Keyword: TX3DKeyword);`
### Properties
| `property Stream: TPeekCharStream read FStream;`
The stream we're reading. This is simply the AStream that you passed to the constructor of this class. Note that you can't operate on this stream from outside while lexer works, this could confuse the lexer. But you're free to read some stream properties, e.g. check Stream.Position.
| `property Version: TX3DVersion read fVersion;`
VRML/X3D version, as recorded in the file header. All VRML 1.0, 2.0, X3D (various 3.x) are recognized correctly. For Inventor 1.0 ascii, we set Version.Major to 0 (as historically Inventor is a predecessor to VRML 1.0).
| `property Token: TX3DToken read fToken;`
Token we're currently standing on. TokenKeyword, TokenName, TokenFloat and TokenInteger have defined values only when token type is appropriate.
| `property TokenKeyword: TX3DKeyword read fTokenKeyword;`
When Token = vtKeyword, `TokenKeyword` points to appropriate keyword. When Version.Major = 1, then you can be sure that `TokenKeyword` is in VRML10Keywords. Analogous for VRML20Keywords and X3DKeywords. So e.g. in VRML 1.0 "PROTO" will be treated like a normal name, not a start of prototype.
| `property TokenName: string read fTokenName;`
When Token = vtName, `TokenName` contains appropriate VRML name. Name syntax as in specification on page 24 (really 32 in pdf) of vrml97specification.pdf. It can be a user name for something (for a node, for example) but it can also be a name of a node type or a node field or an enumerated field constant ... it can be anything except keyword. Note that this is supposed to contain UTF-8 encoded string for VRML >= 2.0.
| `property TokenFloat: Float read fTokenFloat;`
When Token = vtFloat or vtInteger, `TokenFloat` contains a value of this token. For vtInteger you have the same thing in TokenInteger, `TokenFloat` is also initialized to the same value for your comfort (every integer value is also a float, after all).
| `property TokenInteger: Int64 read fTokenInteger;`
When Token = vtInteger, `TokenInteger` contains appropriate value.
| `property TokenString: string read fTokenString;`
When Token = vtString, `TokenString` contains string value.
Generated by PasDoc 0.13.0 on 2015-06-15 04:43:14