AbstractLexer Class Reference

Search Toolkit Book for AbstractLexer

#include <alexer.hpp>

Inheritance diagram for AbstractLexer:

Inheritance graph
[legend]
Collaboration diagram for AbstractLexer:

Collaboration graph
[legend]
List of all members.

Detailed Description

Definition at line 46 of file alexer.hpp.

Public Member Functions

 AbstractLexer (CNcbiIstream &in, const string &name)
virtual ~AbstractLexer (void)
void SetParser (AbstractParser *parser)
const AbstractToken & NextToken (void)
void FillComments (void)
bool CheckSymbol (char symbol)
void Consume (void)
const string & ConsumeAndValue (void)
int CurrentLine (void) const
int LastTokenLine (void) const
virtual void LexerError (const char *error)
virtual void LexerWarning (const char *error, int err_subcode=0)
bool HaveComments (void) const
const CComment & NextComment (void) const
void SkipNextComment (void)
void FlushComments (void)
void FlushCommentsTo (CComments &comments)
void FlushCommentsTo (AbstractLexer &lexer)
const string & GetName (void) const
bool TokenStarted (void) const

Protected Member Functions

virtual TToken LookupToken (void)=0
virtual void LookupComments (void)=0
void NextLine (void)
void StartToken (void)
void AddChars (size_t count)
void AddChar (void)
void SkipChars (size_t count)
void SkipChar (void)
char Char (size_t index)
char Char (void)
bool Eof (void)
const char * CurrentTokenStart (void) const
const char * CurrentTokenEnd (void) const
size_t CurrentTokenLength (void) const
void EndCommentBlock ()
CComment & AddComment (void)
void RemoveLastComment (void)

Protected Attributes

AbstractParser * m_Parser

Private Member Functions

const AbstractToken & FillNextToken (void)
char FillChar (size_t index)

Private Attributes

CNcbiIstream & m_Input
int m_Line
char * m_Buffer
char * m_AllocEnd
char * m_Position
char * m_DataEnd
char * m_TokenStart
AbstractToken m_NextToken
string m_TokenText
list< CComment > m_Comments
string m_Name

Classes

class  CComment

Constructor & Destructor Documentation

AbstractLexer::AbstractLexer (  CNcbiIstream &  in,
const string &  name
) 
 

Definition at line 48 of file alexer.cpp.

AbstractLexer::~AbstractLexer (  void   )  [virtual]
 

Definition at line 56 of file alexer.cpp.

References m_Buffer.

Member Function Documentation

void AbstractLexer::AddChar (  void   )  [inline, protected]
 

Definition at line 156 of file alexer.hpp.

References AddChars().

Referenced by XSDLexer::AddElement(), FillNextToken(), ASNLexer::LookupBinHexString(), XSDLexer::LookupEndOfTag(), DTDLexer::LookupEntity(), ASNLexer::LookupIdentifier(), DTDLexer::LookupIdentifier(), XSDLexer::LookupLexeme(), ASNLexer::LookupNumber(), ASNLexer::LookupString(), DTDLexer::LookupString(), ASNLexer::LookupTag(), ASNLexer::LookupToken(), and XSDLexer::Skip().

void AbstractLexer::AddChars (  size_t  count  )  [inline, protected]
 

Definition at line 150 of file alexer.hpp.

References _ASSERT, m_DataEnd, m_Position, and TokenStarted().

Referenced by AddChar(), ASNLexer::LookupBinHexString(), XSDLexer::LookupEndOfTag(), ASNLexer::LookupIdentifier(), ASNLexer::LookupString(), ASNLexer::LookupToken(), and XSDLexer::Skip().

AbstractLexer::CComment & AbstractLexer::AddComment (  void   )  [protected]
 

Definition at line 197 of file alexer.cpp.

References CurrentLine(), and m_Comments.

Referenced by DTDLexer::ProcessComment(), XSDLexer::ProcessDocumentation(), and ASNLexer::SkipComment().

char AbstractLexer::Char (  void   )  [inline, protected]
 

Definition at line 178 of file alexer.hpp.

Referenced by XSDLexer::AddElement(), CheckSymbol(), ASNLexer::LookupBinHexString(), ASNLexer::LookupComments(), DTDLexer::LookupComments(), XSDLexer::LookupEndOfTag(), DTDLexer::LookupEntity(), ASNLexer::LookupIdentifier(), DTDLexer::LookupIdentifier(), XSDLexer::LookupLexeme(), ASNLexer::LookupNumber(), ASNLexer::LookupString(), DTDLexer::LookupString(), ASNLexer::LookupTag(), XSDLexer::LookupToken(), ASNLexer::LookupToken(), DTDLexer::LookupToken(), DTDLexer::ProcessComment(), XSDLexer::ProcessDocumentation(), XSDLexer::Skip(), and ASNLexer::SkipComment().

char AbstractLexer::Char (  size_t  index  )  [inline, protected]
 

Definition at line 170 of file alexer.hpp.

References FillChar(), m_DataEnd, m_Position, and pos.

bool AbstractLexer::CheckSymbol (  char  symbol  ) 
 

Definition at line 78 of file alexer.cpp.

References _ASSERT, Char(), FillNextToken(), AbstractToken::GetSymbol(), AbstractToken::GetToken(), LookupComments(), m_NextToken, T_SYMBOL, and TokenStarted().

Referenced by AbstractParser::CheckSymbol().

void AbstractLexer::Consume (  void   )  [inline]
 

Definition at line 69 of file alexer.hpp.

References LexerError(), m_TokenStart, and TokenStarted().

Referenced by AbstractParser::Consume().

const string & AbstractLexer::ConsumeAndValue (  void   ) 
 

Definition at line 96 of file alexer.cpp.

References CurrentTokenEnd(), CurrentTokenStart(), LexerError(), m_TokenStart, m_TokenText, and TokenStarted().

Referenced by AbstractParser::ConsumeAndValue().

int AbstractLexer::CurrentLine (  void   )  const [inline]
 

Definition at line 77 of file alexer.hpp.

References m_Line.

Referenced by AddComment(), FillNextToken(), LexerError(), LexerWarning(), XSDParser::ParseElementContent(), StartToken(), and ASNParser::TypesBlock().

const char* AbstractLexer::CurrentTokenEnd (  void   )  const [inline, protected]
 

Definition at line 190 of file alexer.hpp.

References m_Position.

Referenced by ConsumeAndValue(), CurrentTokenLength(), XSDLexer::LookupKeyword(), and WSDLLexer::LookupKeyword().

size_t AbstractLexer::CurrentTokenLength (  void   )  const [inline, protected]
 

Definition at line 194 of file alexer.hpp.

References CurrentTokenEnd(), and CurrentTokenStart().

Referenced by FillNextToken(), XSDLexer::LookupKeyword(), WSDLLexer::LookupKeyword(), ASNLexer::LookupKeyword(), and DTDLexer::LookupKeyword().

const char* AbstractLexer::CurrentTokenStart (  void   )  const [inline, protected]
 

Definition at line 186 of file alexer.hpp.

References m_TokenStart.

Referenced by ConsumeAndValue(), CurrentTokenLength(), FillNextToken(), XSDLexer::LookupKeyword(), WSDLLexer::LookupKeyword(), ASNLexer::LookupKeyword(), DTDLexer::LookupKeyword(), and XSDLexer::LookupLexeme().

void AbstractLexer::EndCommentBlock (   )  [protected]
 

Definition at line 232 of file alexer.cpp.

References AbstractParser::EndCommentBlock(), and m_Parser.

Referenced by DTDLexer::LookupComments().

bool AbstractLexer::Eof (  void   )  [inline, protected]
 

Definition at line 182 of file alexer.hpp.

References m_Input.

Referenced by FillChar(), ASNLexer::LookupBinHexString(), ASNLexer::LookupString(), ASNLexer::LookupTag(), DTDLexer::ProcessComment(), XSDLexer::ProcessDocumentation(), and ASNLexer::SkipComment().

char AbstractLexer::FillChar (  size_t  index  )  [private]
 

Definition at line 128 of file alexer.cpp.

References buffer, Eof(), m_AllocEnd, m_Buffer, m_DataEnd, m_Input, m_Position, m_TokenStart, memmove, pos, READ_AHEAD, and skip.

Referenced by Char().

void AbstractLexer::FillComments (  void   )  [inline]
 

Definition at line 62 of file alexer.hpp.

References LookupComments(), and TokenStarted().

Referenced by AbstractParser::CopyLineComment(), and FillNextToken().

const AbstractToken & AbstractLexer::FillNextToken (  void   )  [private]
 

Definition at line 105 of file alexer.cpp.

References _ASSERT, AddChar(), CurrentLine(), CurrentTokenLength(), CurrentTokenStart(), FillComments(), AbstractToken::length, AbstractToken::line, LookupToken(), m_DataEnd, m_NextToken, m_Position, m_TokenStart, AbstractToken::start, T_EOF, T_SYMBOL, AbstractToken::token, and TokenStarted().

Referenced by CheckSymbol(), and NextToken().

void AbstractLexer::FlushComments (  void   ) 
 

Definition at line 213 of file alexer.cpp.

References m_Comments.

Referenced by DTDParser::EndCommentBlock(), and FlushCommentsTo().

void AbstractLexer::FlushCommentsTo (  AbstractLexer &  lexer  ) 
 

Definition at line 226 of file alexer.cpp.

References FlushComments(), and m_Comments.

void AbstractLexer::FlushCommentsTo (  CComments &  comments  ) 
 

Definition at line 218 of file alexer.cpp.

References CComments::Add(), FlushComments(), ITERATE, and m_Comments.

Referenced by AbstractParser::CopyComments(), XSDParser::ParseAttributeGroup(), XSDParser::ParseGroup(), and XSDParser::ParseTypeDefinition().

const string& AbstractLexer::GetName (  void   )  const [inline]
 

Definition at line 127 of file alexer.hpp.

References m_Name.

bool AbstractLexer::HaveComments (  void   )  const [inline]
 

Definition at line 113 of file alexer.hpp.

References m_Comments.

int AbstractLexer::LastTokenLine (  void   )  const [inline]
 

Definition at line 81 of file alexer.hpp.

References AbstractToken::GetLine(), and m_NextToken.

Referenced by AbstractParser::LastTokenLine().

void AbstractLexer::LexerError (  const char *  error  )  [virtual]
 

Definition at line 61 of file alexer.cpp.

References CurrentLine(), NStr::IntToString(), m_Name, m_TokenText, and NCBI_THROW.

Referenced by Consume(), ConsumeAndValue(), ASNParser::Identifier(), XSDLexer::LookupEndOfTag(), DTDLexer::LookupEntity(), XSDLexer::LookupLexeme(), DTDLexer::LookupString(), DTDLexer::LookupToken(), ASNParser::ModuleReference(), and ASNParser::TypeReference().

void AbstractLexer::LexerWarning (  const char *  error,
int  err_subcode = 0
)  [virtual]
 

Definition at line 70 of file alexer.cpp.

References CurrentLine(), ERR_POST_EX, m_TokenText, and NCBI_ERRCODE_X.

Referenced by ASNLexer::LookupBinHexString(), ASNLexer::LookupString(), and ASNLexer::LookupTag().

virtual void AbstractLexer::LookupComments (  void   )  [protected, pure virtual]
 

Implemented in DTDLexer, and ASNLexer.

Referenced by CheckSymbol(), and FillComments().

virtual TToken AbstractLexer::LookupToken (  void   )  [protected, pure virtual]
 

Implemented in DTDLexer, ASNLexer, and XSDLexer.

Referenced by FillNextToken().

const CComment& AbstractLexer::NextComment (  void   )  const [inline]
 

Definition at line 117 of file alexer.hpp.

References m_Comments.

Referenced by AbstractParser::CopyLineComment().

void AbstractLexer::NextLine (  void   )  [inline, protected]
 

Definition at line 140 of file alexer.hpp.

References m_Line.

Referenced by ASNLexer::LookupComments(), DTDLexer::LookupComments(), XSDLexer::LookupEndOfTag(), DTDLexer::LookupString(), DTDLexer::ProcessComment(), XSDLexer::ProcessDocumentation(), XSDLexer::Skip(), and ASNLexer::SkipComment().

const AbstractToken& AbstractLexer::NextToken (  void   )  [inline]
 

Definition at line 55 of file alexer.hpp.

References FillNextToken(), m_NextToken, and TokenStarted().

Referenced by AbstractParser::NextToken().

void AbstractLexer::RemoveLastComment (  void   )  [protected]
 

Definition at line 203 of file alexer.cpp.

References m_Comments.

Referenced by DTDLexer::ProcessComment(), and XSDLexer::ProcessDocumentation().

void AbstractLexer::SetParser (  AbstractParser *  parser  )  [inline]
 

Definition at line 51 of file alexer.hpp.

References m_Parser.

Referenced by DTDParser::DTDParser().

void AbstractLexer::SkipChar (  void   )  [inline, protected]
 

Definition at line 166 of file alexer.hpp.

References SkipChars().

Referenced by ASNLexer::LookupComments(), DTDLexer::LookupComments(), XSDLexer::LookupEndOfTag(), DTDLexer::LookupEntity(), DTDLexer::LookupString(), XSDLexer::LookupToken(), DTDLexer::LookupToken(), DTDLexer::ProcessComment(), XSDLexer::ProcessDocumentation(), XSDLexer::Skip(), and ASNLexer::SkipComment().

void AbstractLexer::SkipChars (  size_t  count  )  [inline, protected]
 

Definition at line 160 of file alexer.hpp.

References _ASSERT, m_DataEnd, m_Position, and TokenStarted().

Referenced by DTDLexer::EndPrevToken(), ASNLexer::LookupComments(), DTDLexer::LookupComments(), DTDLexer::LookupToken(), DTDLexer::ProcessComment(), SkipChar(), and ASNLexer::SkipComment().

void AbstractLexer::SkipNextComment (  void   ) 
 

Definition at line 192 of file alexer.cpp.

References m_Comments.

Referenced by AbstractParser::CopyLineComment().

void AbstractLexer::StartToken (  void   )  [inline, protected]
 

Definition at line 144 of file alexer.hpp.

References _ASSERT, CurrentLine(), AbstractToken::line, m_NextToken, m_Position, m_TokenStart, and TokenStarted().

Referenced by XSDLexer::LookupEndOfTag(), DTDLexer::LookupEntity(), DTDLexer::LookupIdentifier(), XSDLexer::LookupLexeme(), DTDLexer::LookupString(), ASNLexer::LookupToken(), and XSDLexer::Skip().

bool AbstractLexer::TokenStarted (  void   )  const [inline]
 

Definition at line 131 of file alexer.hpp.

References m_TokenStart.

Referenced by AddChars(), ASNLexer::AddStringChar(), CheckSymbol(), Consume(), ConsumeAndValue(), FillComments(), FillNextToken(), NextToken(), SkipChars(), ASNLexer::StartString(), and StartToken().

Member Data Documentation

char* AbstractLexer::m_AllocEnd [private]
 

Definition at line 212 of file alexer.hpp.

Referenced by FillChar().

char* AbstractLexer::m_Buffer [private]
 

Definition at line 211 of file alexer.hpp.

Referenced by FillChar(), and ~AbstractLexer().

list<CComment> AbstractLexer::m_Comments [private]
 

Definition at line 218 of file alexer.hpp.

Referenced by AddComment(), FlushComments(), FlushCommentsTo(), HaveComments(), NextComment(), RemoveLastComment(), and SkipNextComment().

char* AbstractLexer::m_DataEnd [private]
 

Definition at line 214 of file alexer.hpp.

Referenced by AddChars(), Char(), FillChar(), FillNextToken(), and SkipChars().

CNcbiIstream& AbstractLexer::m_Input [private]
 

Definition at line 209 of file alexer.hpp.

Referenced by Eof(), and FillChar().

int AbstractLexer::m_Line [private]
 

Definition at line 210 of file alexer.hpp.

Referenced by CurrentLine(), and NextLine().

string AbstractLexer::m_Name [private]
 

Definition at line 219 of file alexer.hpp.

Referenced by GetName(), and LexerError().

AbstractToken AbstractLexer::m_NextToken [private]
 

Definition at line 216 of file alexer.hpp.

Referenced by CheckSymbol(), FillNextToken(), LastTokenLine(), NextToken(), and StartToken().

AbstractParser* AbstractLexer::m_Parser [protected]
 

Definition at line 203 of file alexer.hpp.

Referenced by EndCommentBlock(), and SetParser().

char* AbstractLexer::m_Position [private]
 

Definition at line 213 of file alexer.hpp.

Referenced by AddChars(), Char(), CurrentTokenEnd(), FillChar(), FillNextToken(), SkipChars(), and StartToken().

char* AbstractLexer::m_TokenStart [private]
 

Definition at line 215 of file alexer.hpp.

Referenced by Consume(), ConsumeAndValue(), CurrentTokenStart(), FillChar(), FillNextToken(), StartToken(), and TokenStarted().

string AbstractLexer::m_TokenText [private]
 

Definition at line 217 of file alexer.hpp.

Referenced by ConsumeAndValue(), LexerError(), and LexerWarning().

The documentation for this class was generated from the following files:
Generated on Mon Dec 7 09:48:39 2009 for NCBI C++ ToolKit by  doxygen 1.4.6
Modified on Mon Dec 07 16:22:18 2009 by modify_doxy.py rev. 173732