NaturalDocs:: SymbolTable

A package that handles all the gory details of managing symbols.  It handles where they are defined, which files reference them, if any are undefined or duplicated, and loading and saving them to a file.

Usage and Dependencies

Summary
A package that handles all the gory details of managing symbols.
A hash of all symbols, both those that are defined and those which are simply potential interpretations of references.
A hash of all references in the project.
A hash of all the files that define symbols and references in the project.
A NaturalDocs::SymbolTable::File object of the file being watched for changes.
The file name of the watched file, if any.
A hashref of the symbol definition information for all the symbols in the watched file.
A hash of generated symbol indexes.
An existence hash of all the indexes that have changed.
The storage file for the symbol table.
Loads the symbol table from disk.
Purges the symbol table of all symbols and references from files that no longer have Natural Docs content.
Saves the symbol table to disk.
Adds a symbol definition to the table, if it doesn’t already exist.
Adds a reference to the table, if it doesn’t already exist.
Tracks a file to see if any symbols or references were changed or deleted in ways that would require other files to be rebuilt.
Handles any changes found when reparsing a file using WatchFileForChanges().
Purges unnecessary information from the symbol table after it is fully resolved.
Clears all generated indexes.
These functions should not be called until the symbol table is fully resolved.
Returns the symbol the passed class and name define.
Returns what the passed symbol and scope reference, if anything.
Returns the global definition of the passed symbol.
Returns a symbol index.
Determines which indexes out of a list actually have content.
Returns whether the specified index has changed.
Encodes the symbol information as a string.
Encodes the reference information as a string.
Returns the class and symbol encoded in a symbol string.
Returns the scope and reference encoded in a reference string.
Takes a symbol, reference, class, or scope and removes any minor features that could interfere with matching.
Removes a symbol definition from the table.
Deletes a reference from the table.
Generates the list of interpretations for the passed reference.
Generates singular interpretations of a symbol if it can be interpreted as a plural.
Adds an interpretation to an existing reference.
Interprets the passed reference, matching it to the defined symbol with the highest score.
Generates a symbol index.
Converts a prototype string for internal use.

Variables

symbols

my %symbols

A hash of all symbols, both those that are defined and those which are simply potential interpretations of references.  The keys are the symbol strings, generated by by MakeSymbolString(), and the values are NaturalDocs::SymbolTable::Symbol objects.  Note that because references can have many interpretations, there will probably be more symbols here that aren’t defined than that are.

references

my %references

A hash of all references in the project.  The keys are the reference strings, generated by by MakeReferenceString(), and the values are NaturalDocs::SymbolTable::Reference objects.

files

my %files

A hash of all the files that define symbols and references in the project.  The keys are the file names, and the values are NaturalDocs::SymbolTable::File objects.

watchedFile

my $watchedFile

A NaturalDocs::SymbolTable::File object of the file being watched for changes.  This is compared to the version in files to see if anything was changed since the last parse.

watchedFileName

my $watchedFileName

The file name of the watched file, if any.  If there is no watched file, this will be undef.

watchedFileSymbolDefinitions

my %watchedFileSymbolDefinitions

A hashref of the symbol definition information for all the symbols in the watched file.  The keys are the symbol strings, and the values are NaturalDocs::SymbolTable::SymbolDefinition objects.

indexes

my %indexes

A hash of generated symbol indexes.  The keys are <Topic Types> or *, and the values are sorted arrayrefs of NaturalDocs::SymbolTable::IndexElements.  A key of * means the index is not limited to a topic type.

indexChanges

my %indexChanges

An existence hash of all the indexes that have changed.  The keys are the <Topic Types>.  Unlike indexes, there is no * entry.  Simply check if there are any defined keys instead.

Files

SymbolTable.nd

The storage file for the symbol table.

Format

The beginning of the file is the application version that generated it.  Handle it using the text file functions in NaturalDocs::Version.

[app version]

The first stage of the file is for symbol definitions, analogous to symbols.  For this stage, each entry continues until it reaches a blank line.  The first line is the symbol string, and each subsequent line is a tab separated list of the definition, type, prototype, and summary.  The first file on this list will be the one that has the global definition.  Undefined symbols are not included here.

[symbol string]
[global definition file] tab [symbol type] tab [prototype] tab [summary]
[definition file] tab [symbol type] tab [prototype] tab [summary]
blank line

The two stages are separated by a blank line.  That works out to meaning a symbol definition followed by two blank lines.

blank line

The second stage is for references, which is analogous to references.  For this stage, each is a pair of lines.  The first line is the reference string, and the second is a tab separated list of the files that define it.  The interpretations will be regenerated.

[reference string]
[definition file] tab [definition file] tab [definition file] ...

Revisions

1.16

  • All file names are now absolute.  Prior to 1.16 they were relative since there was only one input directory allowed.

1.14

  • The file was renamed from NaturalDocs.sym to SymbolTable.nd and moved to the project data subdirectory..

1.1

  • There was no summary saved in the symbol definition lines.

0.95

  • The file version is now the same as the app version.  Prior to 0.95, the version line was 1.  Test for “1” instead of “1.0” to distinguish.

File Functions

Load

sub Load

Loads the symbol table from disk.

Purge

sub Purge

Purges the symbol table of all symbols and references from files that no longer have Natural Docs content.

Save

sub Save

Saves the symbol table to disk.  It is written to SymbolTable.nd.

Modification Functions

AddSymbol

sub AddSymbol #(class,
symbol,
file,
type,
prototype,
summary)

Adds a symbol definition to the table, if it doesn’t already exist.  If the definition changes or otherwise requires the files that reference it to be updated, the function will call NaturalDocs::Project->RebuildFile() to make sure that they are.

Parameters

classThe symbol’s class.  If none, set to undef.
symbolThe symbol’s name.
fileThe file name where it’s defined.
typeThe symbol’s type.
prototypeThe symbol’s prototype, if applicable.
summaryThe symbol’s summary, if applicable.

AddReference

sub AddReference #(scope,
reference,
file)

Adds a reference to the table, if it doesn’t already exist.

Parameters

scopeThe class scope it appears in.  Set to undef if none.
referenceThe reference text.
fileThe file name where the reference is.

WatchFileForChanges

sub WatchFileForChanges #(file)

Tracks a file to see if any symbols or references were changed or deleted in ways that would require other files to be rebuilt.  Assumes that after this function call, the entire file will be parsed again, and thus every symbol and reference will go through AddSymbol() and AddReference().  Afterwards, call AnalyzeChanges() to handle any differences.

Parameters

fileThe file to watch.

AnalyzeChanges

sub AnalyzeChanges

Handles any changes found when reparsing a file using WatchFileForChanges().

PurgeResolvingInfo

sub PurgeResolvingInfo

Purges unnecessary information from the symbol table after it is fully resolved.  This will reduce the memory footprint for the build stage.  After calling this function, you can only call the Information Functions and Save().

PurgeIndexes

sub PurgeIndexes

Clears all generated indexes.

Information Functions

These functions should not be called until the symbol table is fully resolved.

Defines

sub Defines #(class,
symbol)

Returns the symbol the passed class and name define.  This is not exactly the same as what was passed.  Certain elements may be taken out to make pattern matching more tolerant, so you must use this function when defining symbols in output files to match the tolerance of this package.

For example, say there’s a symbol definition A-space-space-B.  SymbolTable condenses whitespace, so the symbol can also be referenced by A-space-B, A-tab-B, A-space-space-space-B, etc.  In effect, A-space-space-B really defines the symbol A-space-B.  If you encoded the anchor as A-space-space-B in the output file, when References() returns A-space-B, they won’t match.  If you run it through this function first, they always will.

Note that this is only required when encoding symbol definitions in output files.  Every other SymbolTable function handles this automatically, so you don’t need to run symbols through this to call AddSymbol() etc.

Parameters

classThe symbol’s class.  Remember that if you got this value direct from NDMarkup, you must run it through NaturalDocs::NDMarkup->RestoreAmpChars() first.  Use undef if the symbol is global.
symbolThe symbol’s name.  You also need to run this through NaturalDocs::NDMarkup->RestoreAmpChars() if it was gotten from NDMarkup.

Returns

The defined symbol as the array ( class, symbol ).  If class was undef in the parameters, it will be undef here too.

References

sub References #(scope,
reference,
file)

Returns what the passed symbol and scope reference, if anything.  Note that this only works if the reference had been previously added to the table via AddReference().

Parameters

scopeThe scope the symbol appears in.  Note that if you got this value direct from NDMarkup, you must run it through NaturalDocs::NDMarkup->RestoreAmpChars() first.  Use undef if the reference appears as global.
referenceThe reference text.  You need to run this through NaturalDocs::NDMarkup->RestoreAmpChars() too if necessary.
fileThe source file the reference appears in.  This is important because if a symbol is defined in multiple places, the definition in the same file as the reference gets priority.

Returns

A NaturalDocs::SymbolTable::ReferenceTarget object, or undef if the reference doesn’t resolve to anything.

GlobalDefinition

sub GlobalDefinition #(class,
symbol)

Returns the global definition of the passed symbol.  This does not require AddReference() to have been called beforehand.  However, the symbol must be an exact match, and anything generated from it will not be automatically updated by this package if the definition changes.

Parameters

classThe class of the symbol, or undef if none.
symbolThe exact name of the symbol.

Returns

A NaturalDocs::SymbolTable::ReferenceTarget object, or undef if the symbol doesn’t exist.

Index

sub Index #(type)

Returns a symbol index.

Indexes are generated on demand, but they are stored so subsequent calls for the same index will be fast.  Call PurgeIndexes() to clear the generated indexes.

Parameters

typeThe type of symbol to limit the index to, or undef for none.  Should be one of the <Topic Types>.

Returns

A sorted arrayref of NaturalDocs::SymbolTable::IndexElement objects.

HasIndexes

sub HasIndexes #(types)

Determines which indexes out of a list actually have content.

Parameters

typesAn existence hashref of the symbols to check for indexes.  The keys are the <Topic Types> or * for general.

Returns

An existence hashref of all the specified indexes that have content.  Will return an empty hashref if none.

IndexChanged

sub IndexChanged #(type)

Returns whether the specified index has changed.

Parameters

typeThe type of symbol to limit the index to, or undef if none.  Should be one of the <Topic Types>.

Support Functions

MakeSymbolString

sub MakeSymbolString #(class,
symbol)

Encodes the symbol information as a string.  The format of the string should be treated as opaque and the string should not be used for anything other than direct comparison, file I/O, and hash keys.  The only characteristic of the encoding you can depend on is that it will not contain line breaks, which should make using them in text files easier.

Parameters

classThe class the symbol is part of.  Use undef if it is global.
symbolThe name of the symbol itself.

Returns

The encoded symbol string.

MakeReferenceString

sub MakeReferenceString #(scope,
reference)

Encodes the reference information as a string.  The format of the string should be treated as opaque and the string should not be used for anything other than direct comparison, file I/O, and hash keys.  The only characteristic of the encoding you can depend on is that it will not contain line breaks, which should make using them in text files easier.

Parameters

scopeThe scope the reference appears in.  Use undef if it is global.  Note that just because the reference appears in a scope does not necessarily mean the reference is to something in that scope.  Also, it doesn’t mean the reference doesn’t already have a scope specified within it.
referenceThe text of the reference itself.

Returns

The encoded reference string.

DecodeSymbolString

sub DecodeSymbolString #(symbolString)

Returns the class and symbol encoded in a symbol string.

Parameters

symbolStringThe symbol string to decode.

Returns

The array ( class, symbol ).  If the symbol is global, class will be undef.

DecodeReferenceString

sub DecodeReferenceString #(referenceString)

Returns the scope and reference encoded in a reference string.

Parameters

referenceStringThe reference string to decode.

Returns

The array ( scope, reference ).  If the reference is global, scope will be undef.

PrepareString

sub PrepareString #(string)

Takes a symbol, reference, class, or scope and removes any minor features that could interfere with matching.

Parameters

stringThe symbol, reference, class, or scope to be cleaned.

Returns

The cleaned string ready for output.

DeleteSymbol

sub DeleteSymbol #(symbolString,
file)

Removes a symbol definition from the table.  If any files reference it, the function will call NaturalDocs::Project->RebuildFile() for them.

External code should not attempt to delete symbols using this function.  Instead it should call <WatchFileFoChanges()>, reparse the file, and call AnalyzeChanges().

Parameters

symbolStringThe symbol string made from MakeSymbolString().
fileThe file name where it was defined.

DeleteReference

sub DeleteReference #(referenceString,
file)

Deletes a reference from the table.  Be careful with this function, as deleting a reference means there are no more of them in the file at all.  The tables do not keep track of how many times references appear in a file.

External code should not attempt to delete references using this function.  Instead it should call <WatchFileFoChanges()>, reparse the file, and call AnalyzeChanges().

Parameters

referenceStringThe reference string generated by MakeReferenceString().
fileThe file name where the reference is.

GenerateInterpretations

sub GenerateInterpretations #(scope,
reference)

Generates the list of interpretations for the passed reference.  Also creates potential symbols as necessary.

Parameters

scopeThe scope the reference appeared in.  Set to undef if global.
referenceThe text of the reference.

GenerateSingularInterpretations

sub GenerateSingularInterpretations #(symbol)

Generates singular interpretations of a symbol if it can be interpreted as a plural.  Will strip extensions such as -s, -es, and -’s.

Parameters

symbolThe symbol part of the reference.

Returns

An array of potential singular interpretations, in no particular order.  If the symbol can’t be interpreted as a plural, returns an empty array.

AddInterpretation

sub AddInterpretation #(referenceString,
class,
symbol,
score)

Adds an interpretation to an existing reference.  Creates potential symbols as necessary.

Parameters

referenceStringThe reference string to add the interpretation to.
classThe class of the symbol the reference can be interpreted as.  Undef if global.
symbolThe symbol the reference can be interpreted as.
scoreThe score of the interpretation

InterpretReference

sub InterpretReference #(referenceString)

Interprets the passed reference, matching it to the defined symbol with the highest score.  If the symbol is already interpreted, it will reinterpret it.  If there are no matches, it will make it an undefined reference.

Parameters

referenceStringThe string of the reference to interpret.

MakeIndex

sub MakeIndex #(type)

Generates a symbol index.

Parameters

typeThe type to limit the index to, or undef if none.  Should be one of the <Topic Types>.

Returns

An arrayref of sections.  The first represents all the symbols, the second the numbers, and the rest A through Z.  Each section is a sorted arrayref of NaturalDocs::SymbolTable::IndexElement objects.  If a section has no content, it will be undef.

PreparePrototype

sub PreparePrototype #(prototype)

Converts a prototype string for internal use.  Specifically, it strips out tabs and line breaks so that they can be saved in SymbolTable.nd properly.

Parameters

prototypeThe original prototype string.

Returns

The prepared prototype.

my %symbols
A hash of all symbols, both those that are defined and those which are simply potential interpretations of references.
my %references
A hash of all references in the project.
my %files
A hash of all the files that define symbols and references in the project.
my $watchedFile
A NaturalDocs::SymbolTable::File object of the file being watched for changes.
A class representing a file, keeping track of what symbols and references are defined in it.
my $watchedFileName
The file name of the watched file, if any.
my %watchedFileSymbolDefinitions
A hashref of the symbol definition information for all the symbols in the watched file.
my %indexes
A hash of generated symbol indexes.
my %indexChanges
An existence hash of all the indexes that have changed.
sub Load
Loads the symbol table from disk.
sub Purge
Purges the symbol table of all symbols and references from files that no longer have Natural Docs content.
sub Save
Saves the symbol table to disk.
sub AddSymbol #(class,
symbol,
file,
type,
prototype,
summary)
Adds a symbol definition to the table, if it doesn’t already exist.
sub AddReference #(scope,
reference,
file)
Adds a reference to the table, if it doesn’t already exist.
sub WatchFileForChanges #(file)
Tracks a file to see if any symbols or references were changed or deleted in ways that would require other files to be rebuilt.
sub AnalyzeChanges
Handles any changes found when reparsing a file using WatchFileForChanges().
sub PurgeResolvingInfo
Purges unnecessary information from the symbol table after it is fully resolved.
sub PurgeIndexes
Clears all generated indexes.
sub Defines #(class,
symbol)
Returns the symbol the passed class and name define.
sub References #(scope,
reference,
file)
Returns what the passed symbol and scope reference, if anything.
sub GlobalDefinition #(class,
symbol)
Returns the global definition of the passed symbol.
sub Index #(type)
Returns a symbol index.
sub HasIndexes #(types)
Determines which indexes out of a list actually have content.
sub IndexChanged #(type)
Returns whether the specified index has changed.
sub MakeSymbolString #(class,
symbol)
Encodes the symbol information as a string.
sub MakeReferenceString #(scope,
reference)
Encodes the reference information as a string.
sub DecodeSymbolString #(symbolString)
Returns the class and symbol encoded in a symbol string.
sub DecodeReferenceString #(referenceString)
Returns the scope and reference encoded in a reference string.
sub PrepareString #(string)
Takes a symbol, reference, class, or scope and removes any minor features that could interfere with matching.
sub DeleteSymbol #(symbolString,
file)
Removes a symbol definition from the table.
sub DeleteReference #(referenceString,
file)
Deletes a reference from the table.
sub GenerateInterpretations #(scope,
reference)
Generates the list of interpretations for the passed reference.
sub GenerateSingularInterpretations #(symbol)
Generates singular interpretations of a symbol if it can be interpreted as a plural.
sub AddInterpretation #(referenceString,
class,
symbol,
score)
Adds an interpretation to an existing reference.
sub InterpretReference #(referenceString)
Interprets the passed reference, matching it to the defined symbol with the highest score.
sub MakeIndex #(type)
Generates a symbol index.
sub PreparePrototype #(prototype)
Converts a prototype string for internal use.
A package to handle the command line and various other program settings.
A package to manage all the programming languages Natural Docs supports.
A package that manages information about the files in the source tree, as well as the list of files that have to be parsed and built.
These functions should not be called until the symbol table is fully resolved.
sub ParseForInformation #(file)
Parses the input file for information.
A class representing a symbol or a potential symbol.
A class representing a symbol or a potential symbol.
A class representing a symbol definition.
A class representing part of an indexed symbol.
A package for handling version information.
The storage file for the symbol table.
sub RebuildFile #(file)
Adds the file to the list of files to build.
A markup format used by the parser, both internally and in NaturalDocs::Parser::ParsedTopic objects.
sub RestoreAmpChars #(text)
Replaces NDMarkup amp chars with their original symbols.
A class for storing information about a reference target.