schema_salad.fetcher

Resource fetching.

Module Contents

Classes

Fetcher

Fetch resources from URIs.

MemoryCachingFetcher

Fetcher that caches resources in memory after retrieval.

DefaultFetcher

The default Fetcher implementation.
class schema_salad.fetcher.Fetcher

Bases: abc.ABC

digraph inheritancef27fba63b1 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "ABC" [URL="https://docs.python.org/3/library/abc.html#abc.ABC",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Helper class that provides a standard way to create an ABC using"]; "Fetcher" [URL="#schema_salad.fetcher.Fetcher",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Fetch resources from URIs."]; "ABC" -> "Fetcher" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Fetch resources from URIs.

schemes = ['file', 'http', 'https', 'mailto']
abstract fetch_text(url, content_types=None)

Retrieve the given resource as a string.

Parameters:

  • url (str) –

  • content_types (Optional[List[str]]) –

Return type:

str

abstract check_exists(url)

Check if the given resource exists.

Parameters:

url (str) –

Return type:

bool

abstract urljoin(base_url, url)

Construct a full (“absolute”) URL by combining a “base URL” with another URL.

Parameters:

  • base_url (str) –

  • url (str) –

Return type:

str

supported_schemes()

Return the list of supported URI schemes.

Return type:

List[str]

class schema_salad.fetcher.MemoryCachingFetcher(cache)

Bases: Fetcher

digraph inheritancede13b772e8 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "ABC" [URL="https://docs.python.org/3/library/abc.html#abc.ABC",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Helper class that provides a standard way to create an ABC using"]; "Fetcher" [URL="#schema_salad.fetcher.Fetcher",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Fetch resources from URIs."]; "ABC" -> "Fetcher" [arrowsize=0.5,style="setlinewidth(0.5)"]; "MemoryCachingFetcher" [URL="#schema_salad.fetcher.MemoryCachingFetcher",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Fetcher that caches resources in memory after retrieval."]; "Fetcher" -> "MemoryCachingFetcher" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Fetcher that caches resources in memory after retrieval.

Parameters:

cache (schema_salad.utils.CacheType) –

class schema_salad.fetcher.DefaultFetcher(cache, session)

Bases: MemoryCachingFetcher

digraph inheritance4470177d80 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "ABC" [URL="https://docs.python.org/3/library/abc.html#abc.ABC",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Helper class that provides a standard way to create an ABC using"]; "DefaultFetcher" [URL="#schema_salad.fetcher.DefaultFetcher",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="The default Fetcher implementation."]; "MemoryCachingFetcher" -> "DefaultFetcher" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Fetcher" [URL="#schema_salad.fetcher.Fetcher",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Fetch resources from URIs."]; "ABC" -> "Fetcher" [arrowsize=0.5,style="setlinewidth(0.5)"]; "MemoryCachingFetcher" [URL="#schema_salad.fetcher.MemoryCachingFetcher",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Fetcher that caches resources in memory after retrieval."]; "Fetcher" -> "MemoryCachingFetcher" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

The default Fetcher implementation.

Parameters:
fetch_text(url, content_types=None)

Retrieve the given resource as a string.

Parameters:

  • url (str) –

  • content_types (Optional[List[str]]) –

Return type:

str

check_exists(url)

Check if the given resource exists.

Parameters:

url (str) –

Return type:

bool

urljoin(base_url, url)

Construct a full (“absolute”) URL by combining a “base URL” with another URL.

Parameters:

  • base_url (str) –

  • url (str) –

Return type:

str