source.visitor module

class module_dependencies.source.visitor.ParentedNodeVisitor

Bases: NodeVisitor

Subclass of NodeVisitor which adds the parent attribute to every node traversed via generic_visit. This attribute points to the parent node in the AST.

__init__() → None

Return type:

None

generic_visit(node: AST)

Called if no explicit visitor function exists for a node. Places parent attribute on the child node.

Parameters:

node (ast.AST) – Node in an AST.

visit(node)

Visit a node.

visit_Constant(node)

class module_dependencies.source.visitor.ParserVisitor

Bases: ParentedNodeVisitor

An AST NodeVisitor for determining usage of a module.

__init__(tree: AST)

Initialize a ParserVisitor responsible for determining usage of the module module in an AST.

Parameters:

• module (str) – The name of the module to determine the usage of, e.g. "nltk" or "nltk.tokenize", optional. If None, then the usage of all imported modules is gathered.

• tree (AST) –

Variables:

• import_names (Set[Variable]) –

  Set of names of imported objects, e.g.:

  {('product',), ('chain',), ('groupby',), ('np',), ('word_tokenize',)}
  

  when the file used to create the AST contains:

  from itertools import chain, groupby, product
  from nltk.tokenize import word_tokenize
  import numpy as np
  

• import_modules (Set[Variable]) –

  Set of names of modules from which objects are imported, e.g.:

  {('itertools',), ('nltk', 'tokenize'), ('numpy',)}
  

  when the file used to create the AST contains:

  from itertools import chain, groupby, product
  from nltk.tokenize import word_tokenize
  import numpy as np
  

• prefixes (Dict[Variable, Variable]) –

  Dictionary as mapping from imported object to imported module, e.g.:

  {('chain',): ('itertools',), ('groupby',): ('itertools',), ('product',): ('itertools',), ('word_tokenize',): ('nltk', 'tokenize')}
  

  when the file used to create the AST contains:

  from itertools import chain, groupby, product
  from nltk.tokenize import word_tokenize
  import numpy as np
  

• aliases (Dict[Variable, Variable]) –

  Dictionary as mapping from module alias to original module name, e.g.:

  {('np',): ('numpy',)}
  

  when the file used to create the AST contains:

  from itertools import chain, groupby, product
  from nltk.tokenize import word_tokenize
  import numpy as np
  

• uses (Set[Variable]) – Set of objects from module used in an AST.

propagate_attributes(node: AST) → Tuple[str, ...]

Recursively propagate upwards through the AST starting from node, gathering the full name of the variable. This method is meant to be called from the parent of an ast.Name node, which helps produce e.g. ('nltk', 'tokenize', 'PunktSentenceTokenizer') when the ast.Name only contained 'PunktSentenceTokenizer'.

Parameters:

node (ast.AST) – The node where we start propogating upwards.

Return Variable:

The names of the tokens that occur before node, if any.

Return type:

Tuple[str, …]

visit_Import(node: Import)

Called when encountering an Import node, e.g.:

import numpy as np

In this example, ('numpy',) is added to self.import_modules, ('np',) is added to self.import_names, and the mapping of ('numpy',) to ('np',) is added to self.aliases.

Parameters:

node (ast.Import) – The node with the import information.

visit_ImportFrom(node: ImportFrom)

Called when encountering an ImportFrom node, e.g.:

from nltk.corpus import wordnet as wn

In this example, ('nltk', 'corpus') is added to self.import_modules, (‘wn’,) is added to self.import_names, the mapping of ('wn',) to ('wordnet',) is added to self.aliases, and the mapping of ('wordnet',) to ('nltk', 'corpus') is added to self.prefixes.

Parameters:

node (ast.ImportFrom) – The node with the import-from information.

visit_Name(node: Name)

Called when encountering a Name node, which exists whenever a variable is used in some capacity. This Name node consists only of the most left-most part of a variable, e.g. 'nltk' when the variable was nltk.tokenize.TweetTokenizer.

This method extracts the full Variable, e.g. ('nltk', 'tokenize', 'TweetTokenizer'). It uses self.aliases and self.prefixes from visit_Import and visit_ImportFrom to deal with aliases (e.g. ('np',) to ('numpy',)) and extending prefixes, e.g. ('word_tokenize',) to ('nltk', 'tokenize', 'word_tokenize').

Parameters:

node (ast.Name) – The Name node representing the use of a variable in an AST.

get_imports() → Set[Tuple[str, ...]]

Return the set of names of modules from which objects are imported, e.g.:

{('itertools',), ('nltk', 'tokenize'), ('numpy',)}

when the file used to create the AST contains::

    from itertools import chain, groupby, product
    from nltk.tokenize import word_tokenize
    import numpy as np

Returns:

Set of names of modules from which objects are imported.

Return type:

Set[Variable]

get_uses(modules: Iterable[str] | str | None = None) → Iterator[Tuple[str, ...]]

Generate the reported uses of objects imported from modules in modules.

If modules is None, then all uses of objects that were imported are yielded.

For example:

>>> tree = ast.parse("from nltk.tokenize import word_tokenize

word_tokenize(‘Hello there!’)”)

>>> visitor = ParserVisitor(tree)
>>> list(visitor.get_uses("nltk"))
[('nltk', 'tokenize', 'word_tokenize')]

param module:

Module string or list of module strings. For example: 'nltk', 'nltk.parse' or ['nltk.parse', 'nltk.stem']. If module is None, then all uses of imported variables, functions and classes are returned. Defaults to None.

type module:

Union[Iterable[str], str], optional

yield:

A tuple of tokens representing the use of an imported object.

rtype:

Iterator[Variable]

Parameters:

modules (Iterable[str] | str | None) –

Return type:

Iterator[Tuple[str, …]]

generic_visit(node: AST)

Called if no explicit visitor function exists for a node. Places parent attribute on the child node.

Parameters:

node (ast.AST) – Node in an AST.

visit(node)

Visit a node.

visit_Constant(node)