bookish.parser package¶

Submodules¶

bookish.parser.bootstrap module¶

class bookish.parser.bootstrap.Parser(imports, rules, main='grammar', ns=None)¶

as_module(name)¶

as_python_source()¶

context()¶

parse(stream, debug=False, pos=0)¶

parse_parser(stream, debug=False, pos=0, main='grammar', ns=None)¶

write_python_file(filepath)¶

bookish.parser.bootstrap.boot_parse_grammar(filepath, debug=False, pos=0, main='grammar', ns=None)¶

bookish.parser.bootstrap.compile_grammar_file(inpath, outpath=None, pos=0, main='grammar', ns=None)¶

bookish.parser.bootstrap.make_namespace(imports, rules, ns=None, snap=True)¶

bookish.parser.bootstrap.make_rule_dict(localdict)¶

bookish.parser.bootstrap.parse_grammar_file(filepath, pos=0, main='grammar', ns=None)¶

bookish.parser.builder module¶

class bookish.parser.builder.Builder(imports, rules, main_rule, context)¶

Bases: object

add_constant(prefix, value)¶

add_indent(n)¶

add_regex(pattern, expr)¶

assign(name, value)¶

call(rule, indent=0, external=False)¶

call_by_name(name, indent=0, context_name='context')¶

effective_name(rule)¶

generate_id(prefix, save=False, top_level=False)¶

indented(amount)¶

line(line, indent=0)¶

make_function(rule)¶

run()¶

sub_indent(n)¶

class bookish.parser.builder.Frame(name, counter=0, indent=4)¶

Bases: object

inc(name)¶

class bookish.parser.builder.Indentation(builder, amount)¶: Bases: object

bookish.parser.rules module¶

This module defines the basic building blocks for a PEG parser.

class bookish.parser.rules.AlphaNum¶

Bases: bookish.parser.rules.SingletonRule

Matches any alphanumeric character.

build(bld)¶

inline = True¶

class bookish.parser.rules.Among(items)¶

Bases: bookish.parser.rules.Rule

Matches any of a set of characters.

build(bld)¶

first_chars(pctx)¶

inline = True¶

class bookish.parser.rules.Any¶

Bases: bookish.parser.rules.SingletonRule

Matches any character.

build(bld)¶

inline = True¶

class bookish.parser.rules.ApplicationArgs¶

Bases: bookish.parser.rules.Rule

Matches a bracketed list of space-separated arguments. This is just a Rule object wrapper for the take_app_args function.

build(bld)¶

inline = True¶

class bookish.parser.rules.Bind(name, rule)¶

Bases: bookish.parser.rules.Wrapper

If the sub-rule matches, set a variable in the context to the output value.

build(bld)¶

has_binding(bld)¶

class bookish.parser.rules.BlockBreak¶

Bases: bookish.parser.rules.SingletonRule

Matches the end of a “block”. So, the end of the input, or a newline followed by one or more blank lines, or a newline followed by a change in indentation. This is a specialized rule to encapsulate commonly desired wiki behavior without requiring the grammar author to define it manually.

This rule sets the “indent” variable in the context to the indent of the new “block”.

build(bld)¶

first_chars(pctx)¶

inline = True¶

class bookish.parser.rules.Call(name, argexprs=())¶

Bases: bookish.parser.rules.Rule

Invokes another rule by name. This is how we implement circular/recursive definitions in the grammar.

build(bld)¶

dump(pctx, level=0)¶

first_chars(pctx)¶

fixed_length(pctx=None)¶

has_binding(bld)¶

resolve(pctx)¶

snap(pctx, seen)¶

class bookish.parser.rules.Call2(modname, name, argexprs=())¶

Bases: bookish.parser.rules.Call

Invokes another rule in another module by name.

build(bld)¶

dump(pctx, level=0)¶

fixed_length(pctx=None)¶

inline = True¶

resolve(pctx)¶

class bookish.parser.rules.Cond(fn)¶

Bases: bookish.parser.rules.Run

Runs a function (using the context to fill in arguments by name) and matches if the function returns a truthy value, or misses otherwise.

build(bld)¶

class bookish.parser.rules.Do(source)¶

Bases: bookish.parser.rules.Rule

Runs a compiled Python expression (using the context as the environment) and outputs the result.

build(bld)¶

class bookish.parser.rules.DoCode(source)¶

Bases: bookish.parser.rules.Rule

build(bld)¶

inline = True¶

class bookish.parser.rules.Empty¶: Rules always need to return a value. This class serves as a “token” to be returned when a match doesn’t actually correspond to text at the given position (for example, a lookbehind match). This allows the system to distinguish textual matches that happen to be empty (‘’) from “virtual” matches.

class bookish.parser.rules.Extent(rule)¶

Bases: bookish.parser.rules.Wrapper

If the sub-rule matches, annotate the output span/block with an “extent” key indicating the start and end character indices.

build(bld)¶

class bookish.parser.rules.FailIf(rule)¶

Bases: bookish.parser.rules.Rule

If the sub-rule matches, returns a Failure value, indicating to the parent Mixed rule that it does not match.

build(bld)¶

children()¶

class bookish.parser.rules.Failure¶: This class servers as a token to be returned by a FailIf rule inside a Mixed rule. Whereas “Miss” means “this rule did not match at the current position” this value means “this rule matched so the entire parent Mixed rule is a Miss”.

class bookish.parser.rules.FirstChars(chars)¶

Bases: bookish.parser.rules.Rule

A “utility” rule that lets you explicitly set the “first chars” at the start of a sequence, for cases where the system isn’t smart enough to figure them out itself.

build(bld)¶

first_chars(pctx)¶

class bookish.parser.rules.Get(name, default=None)¶

Bases: bookish.parser.rules.Rule

Always matches, outputs the value of a variable in the context.

build(bld)¶

inline = True¶

class bookish.parser.rules.If(source)¶

Bases: bookish.parser.rules.Do

Runs a compiled Python expression (using the context as the environment) and matches if the expression returns a truthy value, or misses otherwise.

build(bld)¶

class bookish.parser.rules.IfCode(source)¶

Bases: bookish.parser.rules.DoCode

build(bld)¶

inline = True¶

class bookish.parser.rules.LineEnd¶

Bases: bookish.parser.rules.SingletonRule

Matches the end of a line (so, at the end of the input or a newline). Note that this rule consumes the newline.

build(bld)¶

first_chars(pctx)¶

inline = True¶

class bookish.parser.rules.LineStart¶

Bases: bookish.parser.rules.SingletonRule

Matches at the start of a line (so, at the start of the input, or right after a newline character).

build(bld)¶

inline = True¶

class bookish.parser.rules.LookBehind(rule)¶

Bases: bookish.parser.rules.Rule

Matches if the sub-rule matches leading up to the given position.

build(bld)¶

children()¶

class bookish.parser.rules.Match(item)¶

Bases: bookish.parser.rules.Rule

Matches a given character.

build(bld)¶

first_chars(pctx)¶

inline = True¶

class bookish.parser.rules.Miss¶: This class serves as a “token” to be returned to indicate a rule did not match at the given position (it is not meant to be instantiated). This is a performance hack; returning a singleton is so much faster than using exceptions it’s worth the convoluted un-Pythonic code.

class bookish.parser.rules.Mixed(until, rule=None)¶

Bases: bookish.parser.rules.Rule

Takes text until a certain rule (the “until” rule) matches. This rule does not consume the “until” match (it acts like a Peek). Before the until rule, you can optionally specify a “content” rule. The output of any matches of this rule are interspersed with the text.

This allows parsing wiki text where inline markup such as links and styling are interspersed with text. As with BlockBreak, you could specify this behavior using lower-level rules, but it would be tedious and less efficient.

build(bld)¶

children()¶

snap(pctx, seen)¶

class bookish.parser.rules.MultiRule(rules)¶

Bases: bookish.parser.rules.Rule

Base class for rules that encapsulate multiple sub-rules (Or and Seq).

children()¶

has_binding(bld)¶

snap(pctx, seen)¶

class bookish.parser.rules.Not(rule)¶

Bases: bookish.parser.rules.Rule

Matches if a sub-rule doesn’t match.

build(bld)¶

children()¶

first_chars(pctx)¶

class bookish.parser.rules.Opt(rule)¶

Bases: bookish.parser.rules.Rule

Matches the sub-rule zero or one times.

build(bld)¶

children()¶

is_optional()¶

class bookish.parser.rules.Or(*rules)¶

Bases: bookish.parser.rules.MultiRule

Checks multiple sub-rules in sequence, returns the output of the first one that matches, or Miss if none of the sub-rules match.

build(bld)¶

first_chars(pctx)¶

fixed_length(pctx=None)¶

is_optional()¶

class bookish.parser.rules.Params(rule, argnames)¶

Bases: bookish.parser.rules.Wrapper

This rule indicates that its sub-rule should be run in a context where certain parameters are filled in. This rule doesn’t actually do anything; the Call rule checks whether its target is a Params rule and fills in the necessary parameters if it is.

build(bld)¶

class bookish.parser.rules.Peek(rule)¶

Bases: bookish.parser.rules.Rule

Matches if the sub-rule would match at the given position, but does not move the the position forward.

build(bld)¶

children()¶

first_chars(pctx)¶

class bookish.parser.rules.Plus(rule)¶

Bases: bookish.parser.rules.Repeat

Matches the sub-rule matches one or more times.

class bookish.parser.rules.Put(name, value)¶

Bases: bookish.parser.rules.Rule

Always matches, sets a value in the context.

build(bld)¶

inline = True¶

class bookish.parser.rules.PythonExpr(ends)¶

Bases: bookish.parser.rules.Rule

Matches a Python expression, up to one of the characters in the “ends” set. This is a Rule object wrapper for the take_python_expr() function.

build(bld)¶

class bookish.parser.rules.Regex(pattern, groups=True)¶

Bases: bookish.parser.rules.Rule

Matches a regular expression. The output is the whole match, and any named groups are added to the context as variables.

build(bld)¶

inline = True¶

class bookish.parser.rules.Repeat(rule, mintimes=0, maxtimes=None)¶

Bases: bookish.parser.rules.Rule

Matches if a sub-rules matches within a certain range of repititions.

build(bld)¶

children()¶

first_chars(pctx)¶

is_optional()¶

class bookish.parser.rules.Replace(rule, output)¶

Bases: bookish.parser.rules.Wrapper

If the sub-rule matches, outputs a given value instead of the sub-rule’s output.

build(bld)¶

class bookish.parser.rules.Rule¶

Bases: object

Base class for all rules.

accept(stream, i, context)¶

build(bld)¶

children()¶

dump(pctx, level=0)¶

first_chars(pctx)¶

fixed_length(pctx=None)¶

has_binding(bld)¶

is_optional()¶

rulename()¶

set_name(name)¶

snap(pctx, seen)¶

class bookish.parser.rules.Run(fn)¶

Bases: bookish.parser.rules.Rule

Runs a function (using the context to fill in arguments by name) and outputs the result.

build(bld)¶

class bookish.parser.rules.Seq(*rules)¶

Bases: bookish.parser.rules.MultiRule

Checks that a sequence of sub-rules match one after the other. This rule only matches if all sub-rules match in sequence.

build(bld)¶

first_chars(pctx)¶

fixed_length(pctx=None)¶

is_optional()¶

class bookish.parser.rules.SingletonRule¶

Bases: bookish.parser.rules.Rule

Base class for rules which only ever have one instance.

class bookish.parser.rules.Star(rule)¶

Bases: bookish.parser.rules.Repeat

Matches the sub-rule matches zero or more times.

class bookish.parser.rules.StreamEnd¶

Bases: bookish.parser.rules.SingletonRule

Matches at the end of the input.

build(bld)¶

first_chars(pctx)¶

inline = True¶

class bookish.parser.rules.StreamStart¶

Bases: bookish.parser.rules.SingletonRule

Matches at the start of the input.

build(bld)¶

inline = True¶

class bookish.parser.rules.String(s)¶

Bases: bookish.parser.rules.Rule

Matches a given string.

build(bld)¶

first_chars(pctx)¶

fixed_length(pctx=None)¶

inline = True¶

class bookish.parser.rules.Take(rule)¶

Bases: bookish.parser.rules.Wrapper

If the sub-rule matches, output the text corresponding to the start and end of the match, instead of the sub-rule’s output.

build(bld)¶

class bookish.parser.rules.Value(value)¶

Bases: bookish.parser.rules.Rule

Always matches, outputs a given value.

build(bld)¶

inline = True¶

class bookish.parser.rules.Wall(name)¶

Bases: bookish.parser.rules.Rule

This rule doesn’t do anything, but it serves as a marker in a sequence that past this point, the rest of the sequence must match, otherwise there’s an error in the input file (the parent Seq rule will raise an exception).

build(bld)¶

class bookish.parser.rules.Wrapper(rule)¶

Bases: bookish.parser.rules.Rule

Base class for rules that “wrap” a sub-rule, modifying their behavior or output somehow.

build(bld)¶

children()¶

first_chars(pctx)¶

fixed_length(pctx=None)¶

has_binding(bld)¶

is_optional()¶

snap(pctx, seen)¶

bookish.parser.rules.charset_string(chars)¶: Returns a Python source code representation of a set of characters.

bookish.parser.rules.compile_expr(expr)¶: Compiles a Python source code string into a Python expression code object.

bookish.parser.rules.ensure(rule)¶: Ensures an argument to a rule is itself a rule. Basically, lets you use a string value “x” and turn it into a String(“x”) rule automatically.

bookish.parser.rules.firstmap_string(builder, firstmap)¶: Returns a Python source code string representation of a dictionary as created by make_firstmap().

bookish.parser.rules.make_firstmap(rules, pctx)¶

Given a list of rules, generates a mapping between initial characters and the rules that could match starting with that character. The None key maps to a list of rules that could match starting with any character.

This is performance optimization. Figuring out what characters a rule can possibly start with, and only trying the rule if we’re on one of those characters, gives a massive performance boost.

bookish.parser.rules.name_rules(d)¶

bookish.parser.rules.row_and_col(stream, i)¶: Given a string and an index into the string, returns a tuple of the line number and column number corresponding to that position.

bookish.parser.rules.take_app_args(stream, i)¶: Starting at the given position, parses a string corresponding to a bracketed, space-separated list of arguments (e.g. “(a b ‘c’)”) and returns a list of argument strings (e.g. [“a”, “b”, “‘c’”]).

bookish.parser.rules.take_python_expr(stream, i, ends)¶: Starting at a given position, takes a string corresponding to a Python expression, stopping when it sees one of the characters in the “ends” set. Returns None if there is not a valid Python expression at the given position.

Module contents¶

class bookish.parser.ParserContext(m=None, parent=None, namespace=None, debug=False)¶

Bases: bookish.util.Context

cache¶

debug¶

namespace¶

set_debug(v)¶

exception bookish.parser.ParserError¶: Bases: exceptions.Exception

bookish.parser.condition_string(c, add_eot=True)¶