schema_salad.avro.schema

Contains the Schema classes.

A schema may be one of:

A record, mapping field names to field value data; An enum, containing one of a small set of symbols; An array of values, all of the same schema; A union of other schemas; A unicode string; A 32-bit signed int; A 64-bit signed long; A 32-bit floating-point float; A 64-bit floating-point double; A boolean; or Null.

Module Contents

Classes

Schema

Base class for all Schema classes.

Name

Class to describe Avro name.

Names

Track name set and default namespace during parsing.

NamedSchema

Named Schemas specified in NAMED_TYPES.

Field

PrimitiveSchema

Valid primitive types are in PRIMITIVE_TYPES.

EnumSchema

Named Schemas specified in NAMED_TYPES.

ArraySchema

Base class for all Schema classes.

UnionSchema

Base class for all Schema classes.

RecordSchema

Named Schemas specified in NAMED_TYPES.

Functions

get_other_props(all_props, reserved_props)

Retrieve the non-reserved properties from a dictionary of properties.

make_avsc_object(json_data[, names])

Build Avro Schema from data parsed out of JSON string.

is_subtype(existing, new)

Check if a new type specification is compatible with an existing type spec.

Attributes

PRIMITIVE_TYPES

NAMED_TYPES

VALID_TYPES

SCHEMA_RESERVED_PROPS

JsonDataType

AtomicPropType

PropType

PropsType

FIELD_RESERVED_PROPS

VALID_FIELD_SORT_ORDERS

schema_salad.avro.schema.PRIMITIVE_TYPES = ('null', 'boolean', 'string', 'int', 'long', 'float', 'double')
schema_salad.avro.schema.NAMED_TYPES = ('enum', 'record')
schema_salad.avro.schema.VALID_TYPES
schema_salad.avro.schema.SCHEMA_RESERVED_PROPS = ('type', 'name', 'namespace', 'fields', 'items', 'symbols', 'doc')
schema_salad.avro.schema.JsonDataType
schema_salad.avro.schema.AtomicPropType
schema_salad.avro.schema.PropType
schema_salad.avro.schema.PropsType
schema_salad.avro.schema.FIELD_RESERVED_PROPS = ('default', 'name', 'doc', 'order', 'type')
schema_salad.avro.schema.VALID_FIELD_SORT_ORDERS = ('ascending', 'descending', 'ignore')
exception schema_salad.avro.schema.AvroException(msg, sl=None, children=None, bullet_for_children='')

Bases: schema_salad.exceptions.SchemaException

digraph inheritance9342b6adff { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AvroException" [URL="#schema_salad.avro.schema.AvroException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "SchemaException" -> "AvroException" [arrowsize=0.5,style="setlinewidth(0.5)"]; "SchemaException" [URL="../../exceptions/index.html#schema_salad.exceptions.SchemaException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Indicates error with the provided schema definition."]; "SchemaSaladException" -> "SchemaException" [arrowsize=0.5,style="setlinewidth(0.5)"]; "SchemaSaladException" [URL="../../exceptions/index.html#schema_salad.exceptions.SchemaSaladException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all schema-salad exceptions."]; }

Indicates error with the provided schema definition.

Parameters:
exception schema_salad.avro.schema.SchemaParseException(msg, sl=None, children=None, bullet_for_children='')

Bases: AvroException

digraph inheritanced169f71d5f { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AvroException" [URL="#schema_salad.avro.schema.AvroException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "SchemaException" -> "AvroException" [arrowsize=0.5,style="setlinewidth(0.5)"]; "SchemaException" [URL="../../exceptions/index.html#schema_salad.exceptions.SchemaException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Indicates error with the provided schema definition."]; "SchemaSaladException" -> "SchemaException" [arrowsize=0.5,style="setlinewidth(0.5)"]; "SchemaParseException" [URL="#schema_salad.avro.schema.SchemaParseException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AvroException" -> "SchemaParseException" [arrowsize=0.5,style="setlinewidth(0.5)"]; "SchemaSaladException" [URL="../../exceptions/index.html#schema_salad.exceptions.SchemaSaladException",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all schema-salad exceptions."]; }

Indicates error with the provided schema definition.

Parameters:
class schema_salad.avro.schema.Schema(atype, other_props=None)

Base class for all Schema classes.

Parameters:

  • atype (str) –

  • other_props (Optional[PropsType]) –

property props: PropsType
Return type:

PropsType

get_prop(key)
Parameters:

key (str) –

Return type:

Optional[PropType]

set_prop(key, value)
Parameters:

  • key (str) –

  • value (Optional[PropType]) –

Return type:

None

class schema_salad.avro.schema.Name(name_attr=None, space_attr=None, default_space=None)

Class to describe Avro name.

Parameters:

  • name_attr (Optional[str]) –

  • space_attr (Optional[str]) –

  • default_space (Optional[str]) –

property fullname: str | None
Return type:

Optional[str]

get_space()

Back out a namespace from full name.

Return type:

Optional[str]

class schema_salad.avro.schema.Names(default_namespace=None)

Track name set and default namespace during parsing.

Parameters:

default_namespace (Optional[str]) –

has_name(name_attr, space_attr)
Parameters:

  • name_attr (str) –

  • space_attr (Optional[str]) –

Return type:

bool

get_name(name_attr, space_attr)

Fetch the stored schema for the given namespace.

Parameters:

  • name_attr (str) –

  • space_attr (Optional[str]) –

Return type:

Optional[NamedSchema]

add_name(name_attr, space_attr, new_schema)

Add a new schema object to the name set.

Parameters:

  • name_attr (str) – name value read in schema

  • space_attr (Optional[str]) – namespace value read in schema.

  • new_schema (NamedSchema) –

Returns:

the Name that was just added.

Return type:

Name

class schema_salad.avro.schema.NamedSchema(atype, name, namespace=None, names=None, other_props=None)

Bases: Schema

digraph inheritancecfa20f43d8 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "NamedSchema" [URL="#schema_salad.avro.schema.NamedSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Named Schemas specified in NAMED_TYPES."]; "Schema" -> "NamedSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Schema" [URL="#schema_salad.avro.schema.Schema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all Schema classes."]; }

Named Schemas specified in NAMED_TYPES.

Parameters:

  • atype (str) –

  • name (str) –

  • namespace (Optional[str]) –

  • names (Optional[Names]) –

  • other_props (Optional[PropsType]) –

property name: str
Return type:

str

class schema_salad.avro.schema.Field(atype, name, has_default, default=None, order=None, names=None, doc=None, other_props=None)
Parameters:

  • atype (JsonDataType) –

  • name (str) –

  • has_default (bool) –

  • default (Optional[Any]) –

  • order (Optional[str]) –

  • names (Optional[Names]) –

  • doc (Optional[Union[str, List[str]]]) –

  • other_props (Optional[PropsType]) –

property default: Any | None
Return type:

Optional[Any]

get_prop(key)
Parameters:

key (str) –

Return type:

Optional[PropType]

set_prop(key, value)
Parameters:

  • key (str) –

  • value (Optional[PropType]) –

Return type:

None

class schema_salad.avro.schema.PrimitiveSchema(atype, other_props=None)

Bases: Schema

digraph inheritance309a015514 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "PrimitiveSchema" [URL="#schema_salad.avro.schema.PrimitiveSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Valid primitive types are in PRIMITIVE_TYPES."]; "Schema" -> "PrimitiveSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Schema" [URL="#schema_salad.avro.schema.Schema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all Schema classes."]; }

Valid primitive types are in PRIMITIVE_TYPES.

Parameters:

  • atype (str) –

  • other_props (Optional[PropsType]) –

class schema_salad.avro.schema.EnumSchema(name, namespace, symbols, names=None, doc=None, other_props=None)

Bases: NamedSchema

digraph inheritance881b940a13 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "EnumSchema" [URL="#schema_salad.avro.schema.EnumSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "NamedSchema" -> "EnumSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "NamedSchema" [URL="#schema_salad.avro.schema.NamedSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Named Schemas specified in NAMED_TYPES."]; "Schema" -> "NamedSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Schema" [URL="#schema_salad.avro.schema.Schema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all Schema classes."]; }

Named Schemas specified in NAMED_TYPES.

Parameters:

  • name (str) –

  • namespace (Optional[str]) –

  • symbols (List[str]) –

  • names (Optional[Names]) –

  • doc (Optional[Union[str, List[str]]]) –

  • other_props (Optional[PropsType]) –

property symbols: List[str]
Return type:

List[str]

class schema_salad.avro.schema.ArraySchema(items, names, other_props=None)

Bases: Schema

digraph inheritance34e8329cb5 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "ArraySchema" [URL="#schema_salad.avro.schema.ArraySchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Schema" -> "ArraySchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Schema" [URL="#schema_salad.avro.schema.Schema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all Schema classes."]; }

Base class for all Schema classes.

Parameters:

  • items (JsonDataType) –

  • names (Names) –

  • other_props (Optional[PropsType]) –

property items: Schema
Return type:

Schema

class schema_salad.avro.schema.UnionSchema(schemas, names)

Bases: Schema

digraph inheritance3e42bb20c3 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "Schema" [URL="#schema_salad.avro.schema.Schema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all Schema classes."]; "UnionSchema" [URL="#schema_salad.avro.schema.UnionSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Schema" -> "UnionSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Base class for all Schema classes.

Parameters:

  • schemas (List[JsonDataType]) –

  • names (Names) –

property schemas: List[Schema]
Return type:

List[Schema]

class schema_salad.avro.schema.RecordSchema(name, namespace, fields, names, schema_type='record', doc=None, other_props=None)

Bases: NamedSchema

digraph inheritance9f432fd7f3 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "NamedSchema" [URL="#schema_salad.avro.schema.NamedSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Named Schemas specified in NAMED_TYPES."]; "Schema" -> "NamedSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "RecordSchema" [URL="#schema_salad.avro.schema.RecordSchema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "NamedSchema" -> "RecordSchema" [arrowsize=0.5,style="setlinewidth(0.5)"]; "Schema" [URL="#schema_salad.avro.schema.Schema",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class for all Schema classes."]; }

Named Schemas specified in NAMED_TYPES.

Parameters:

  • name (str) –

  • namespace (Optional[str]) –

  • fields (List[PropsType]) –

  • names (Names) –

  • schema_type (str) –

  • doc (Optional[Union[str, List[str]]]) –

  • other_props (Optional[PropsType]) –

property fields: List[Field]
Return type:

List[Field]

static make_field_objects(field_data, names)

We’re going to need to make message parameters too.

Parameters:

  • field_data (List[PropsType]) –

  • names (Names) –

Return type:

List[Field]

schema_salad.avro.schema.get_other_props(all_props, reserved_props)

Retrieve the non-reserved properties from a dictionary of properties.

Parameters:

  • reserved_props (Tuple[str, Ellipsis]) – The set of reserved properties to exclude

  • all_props (PropsType) –

Return type:

Optional[PropsType]

schema_salad.avro.schema.make_avsc_object(json_data, names=None)

Build Avro Schema from data parsed out of JSON string.

Parameters:

  • names (Optional[Names]) – A Name object (tracks seen names and default space)

  • json_data (JsonDataType) –

Return type:

Schema

schema_salad.avro.schema.is_subtype(existing, new)

Check if a new type specification is compatible with an existing type spec.

Parameters:

  • existing (PropType) –

  • new (PropType) –

Return type:

bool