* Fixes#1147: template strings not obeying -b ascii_only true
* Allow evaluation of template expressions by adding optimizers and
walkers
* Make sure tagged templates are never changed
* Remove template tokenizer in parser, add template tokenizer in
tokenizer. It is using a brace counter to track brace position of
templates
* Add tokens `template_head` and `template_substitution` but parsing
tokens stays mostly the same
* Do not output strings anymore in AST_TemplateString, instead use
AST_TemplateSegment
* Fix parsing tagged templates, allowing multiple templates behind
as spec allows this
These changes don't influence tagged templates because raw content
may influence code execution, however they are safe to do in normal
templates:
* Allow basic string concatenation of templates where possible
* Allow custom character escape style similar to strings, except in
tagged templates
Note that expressions are still compressed in tagged templates.
Optional things that may be improved later:
* Custom quote style for templates if it doesn't have expressions.
Making it obey the quote_style option if this is the case.
- Partially reverting 91cdb93e57 and eaf3911c31 and reimplement
- Add generators support for objects and classes
- Only classes can have static methods so restrict use of it
Special thanks to @rvanvelzen and @kzc for reviewing this patch and
providing constructive feedback over and over again.
It's now available during tree walking, i.e. walker.has_directive("use
asm"), rather than as part of the scope. It's thus no longer necessary
to call `figure_out_scope` before codegen. Added special bits in the
code generator to overcome the fact that it doesn't inherit from
TreeWalker.
Fix#861
