Skip to content

Brython development

Jump to bottom Edit New page
Pierre Quentel edited this page Apr 2, 2024 · 13 revisions

This page provides information for those who would like to modify the core Brython scripts or Python/Javascript modules in Brython standard library.

The scripts used in Brython programs are located in folder /www/src.

At the root of this folder we find two categories of scripts : core scripts and generated files.

Core scripts

They are the Javascript programs in directory /www/src of the repository. They are described in the section How Brython works. Developers should only edit these scripts.

Generated files

The generated scripts are not meant to be modified by developers : most of them are generated by a CPython program, located at /scripts/make_dist.py. The main generated scripts are :

  • brython.js: compilation of the core scripts in /www/src
  • brython_stdlib.js: compilation of Python and Javascript modules and packages in /www/src/Lib and /www/src/libs
  • stdlib_paths.js: a list of the stdlib scripts, used to speed up imports of modules in the stdlib when brython_dist.js is not used
  • version_info.js: various informations on the Brython version

A few other files related to Unicode management are generated by other scripts in directory /scripts.

  • /scripts/make_unicode_tables.py
    1. sends HTTP requests to _https://www.unicode.org/Public/UCD/latest/ucd_/ to load Unicode tables (UnicodeData.txt, CaseFolding.txt, DerivedCoreProperties.txt)
    2. stores them in /scripts/ucd
    3. uses them to generate /www/src/unicode_data.js
  • /scripts/make_unicode_file.py uses UnicodeData.txt to generate a more compact file at /www/src/unicode.txt

Development cycle

When a developer wants to modify something in Brython, to fix a bug, improve an implementation, add new features etc., he should only edit core scripts.

To test the modifications, the most convenient is to create a test page (eg test.html in folder /www/tests) and, instead of loading brython.js, load all the core scripts. The list of files can be found in /scripts/make_dist.py. At time of writing, this means that the section <head> of test.html should start with

<script src="/src/brython_builtins.js"></script>

<script src="/src/py_ast_classes.js"></script>
<script src="/src/stdlib_paths.js"></script>
<script src="/src/unicode_data.js"></script>
<script src="/src/version_info.js"></script>

<script src="/src/py_tokens.js"></script>
<script src="/src/python_tokenizer.js"></script>
<script src="/src/py_ast.js"></script>
<script src="/src/py2js.js"></script>
<script src="/src/loaders.js"></script>
<script src="/src/py_utils.js"></script>
<script src="/src/py_object.js"></script>
<script src="/src/py_type.js"></script>
<script src="/src/py_builtin_functions.js"></script>
<script src="/src/py_sort.js"></script>
<script src="/src/py_exceptions.js"></script>
<script src="/src/py_range_slice.js"></script>
<script src="/src/py_bytes.js"></script>
<script src="/src/py_set.js"></script>
<script src="/src/py_import.js"></script>
<script src="/src/py_string.js"></script>
<script src="/src/py_int.js"></script>
<script src="/src/py_long_int.js"></script>
<script src="/src/py_float.js"></script>
<script src="/src/py_complex.js"></script>
<script src="/src/py_dict.js"></script>
<script src="/src/py_list.js"></script>
<script src="/src/js_objects.js"></script>
<script src="/src/py_generator.js"></script>
<script src="/src/py_dom.js"></script>
<script src="/src/py_pattern_matching.js"></script>
<script src="/src/async.js"></script>
<script src="/src/py_flags.js"></script>
<script src="/src/builtin_modules.js"></script>
<script src="/src/ast_to_js.js"></script>
<script src="/src/symtable.js"></script>

<!-- scripts for PEG parser -->
<script src="/src/action_helpers_generated_version.js"></script>
<script src="/src/string_parser.js"></script>
<script src="/src/number_parser.js"></script>
<script src="/src/python_parser_peg_version.js"></script>
<script src="/src/pegen.js"></script>
<script src="/src/gen_parse.js"></script>

<script src="/src/brython_ready.js"></script>

Note that the order is important, since some of the scripts at the end of the list use objects defined in previous scripts.

The Brython test program can then be inserted in test.html; each time the developer changes one of the core scripts, the effect can be seen immediately by reloading the page, instead of generating brython.js.

If the developer is modifying a module in the standard library, then brython_stdlib.js should not be loaded in the test page, because modifications of programs in the stdlib would not be reflected when reloading the page.

Built-in test suite

Brython has a built-in test suite, the page located at /www/tests/index.html. It is mandatory to run the whole test suite before submitting a Pull Request, to make sure that no regression was introduced.

This page uses the individual core scripts (not brython.js), so changes to core scripts are reflected. But, for performance reasons, this page loads brython_stdlib.js, so if changes have been made to the standard distribution, /scripts/make_dist.py must be executed before running the test suite.

Building brython.js and brython_stdlib.js

As explained above, these scripts are generated by running /scripts/make_dist.py. Note that it must be executed with the same CPython version as the Brython version (ie, to generate the scripts for Brython 3.12.x, use CPython 3.12). This is because the generation of brython_stdlib.js requires to parse the Python modules in the stdlib with the CPython ast module, and this would fail with older versions of CPython.

Documentation files

The documentation pages, written in markdown format, are located in folders /doc and /tutorial.

For performance reasons, they are translated into static HTML pages by the CPython script /scripts/make_doc.py. The pages are stored in static_doc and static_tutorial.

Python parser

Since version 3.12.3, Brython uses a parser for Python source code generated by a program based on the official Python grammar.

The steps taken to generate this parser are:

  • run program /scripts/downloads.py. This script downloads python.gram and Tokens from the CPython Github site, and Unicode-related files from https://www.unicode.org
  • run make_javascript_gen_parse.py : it generates the parser script located at /src/gen_parse.js

Debugging issues

For each issue, put the code that fails in a file such as test.html above and try to reproduce the bug. Set the debug level to 2 (<script type="text/python" debug="2">).

Examine the debug trace. The Javascript trace should indicate in which core script(s) the error happens. Edit these scripts by adding the odd console.log() expression.

Another strategy is to copy the Javascript translation to a script called eg module1.js; replace the first lines

// Javascript code generated from ast
var $B = __BRYTHON__,
    _b_ = $B.builtins,
    locals___main__ = $B.imported["__main__"],
    locals = locals___main__,
    frame = ["__main__", locals, "__main__", locals]

with

// Javascript code generated from ast
var $B = __BRYTHON__,
    _b_ = $B.builtins,
    locals___main__ = $B.imported["__main__"] = {},
    locals = locals___main__,
    frame = ["__main__", locals, "__main__", locals]
brython()

Then disable the Python script by setting its type to an unknow type (<script type="text/xxx">) and add

<script src="module1.js"></script>

Reload the html page, you should get more or less the same error message. Then edit module1.js to insert console.log() in the places that will help locating the error.

Add a custom footer
Add a custom sidebar
Clone this wiki locally