From 1b8dcf17bd13476838fb90609928e4538b50b0b4 Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Fri, 26 Dec 2014 22:01:30 +1100 Subject: [PATCH 1/8] notes on object layout --- doc/devdocs/julia.rst | 1 + doc/devdocs/object.rst | 68 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100644 doc/devdocs/object.rst diff --git a/doc/devdocs/julia.rst b/doc/devdocs/julia.rst index e7b7f13fe6b5b..b1227138fff87 100644 --- a/doc/devdocs/julia.rst +++ b/doc/devdocs/julia.rst @@ -9,6 +9,7 @@ .. toctree:: :maxdepth: 1 + object cartesian meta subarrays diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst new file mode 100644 index 0000000000000..6bc7bcc537a95 --- /dev/null +++ b/doc/devdocs/object.rst @@ -0,0 +1,68 @@ +****************************** +Memory layout of Julia Objects +****************************** + +Object layout (jl_value_t) +-------------------------- + +The :code:`jl_value_t` struct defines the minimal header for a Julia +object in memory. Every object begins with a pointer to a +:code:`jl_datatype_t` object:: + + typedef struct _jl_value_t { + struct _jl_value_t *type; + } jl_value_t; + +The layout of the rest of the object is dependant on its type. + +e.g. a :func:`Base.tuple` object has an array of pointers to the +objects contained by the tuple:: + + typedef struct { + struct _jl_value_t *type; + size_t length; + jl_value_t *data[]; + } jl_tuple_t; + +e.g. a "boxed" uint16_t (created by :func:`jl_box_uint16`) is stored as +follows:: + + struct { + struct _jl_value_t *type; -- 8 bytes + uint16_t data; -- 2 bytes + -- 6 bytes padding + }; + + +Garbage collector mark bit +-------------------------- + +The garbage collector uses the low bit of the :code:`jl_value_t.type` +pointer as a flag to mark reachable objects (see :code:`gcval_t`). +During each mark/sweep cycle, the gc sets the mark bit of each +reachable object, deallocates objects that are not marked, then +clears the mark bits. While the mark/sweep is in progress the +:code:`jl_value_t.type` pointer is altered by the mark bit. The gc +uses the :func:`gc_typeof` macro to retrieve the original type +pointer. + + #define gc_typeof(v) ((jl_value_t*)(((uptrint_t)jl_typeof(v))&~1UL)) + + +Object allocation +----------------- + +Storage for new objects is allocated by :func:`newobj` in julia_internal.h:: + + STATIC_INLINE jl_value_t *newobj(jl_value_t *type, size_t nfields) + { + jl_value_t *jv = (jl_value_t*)allocobj((1+nfields) * sizeof(void*)); + jv->type = type; + return jv; + } + +Note that all objects are allocated in multiples of 8 bytes, so the +smallest object size is 16 bytes (8 byte type pointer + 8 bytes +data). :func:`allocobj` in gc.c allocates memory for new objects. +Memory is allocated from a pool for objects up to 2048 bytes, or +by malloc() otherwise.:: From af7bd4d7e59968462402190460ed61ac37193a00 Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Sat, 27 Dec 2014 15:52:30 +1100 Subject: [PATCH 2/8] Notes on runtime init. --- doc/devdocs/init.rst | 246 +++++++++++++++++++++++++++++++++++++++++ doc/devdocs/julia.rst | 1 + doc/devdocs/object.rst | 4 +- doc/devdocs/sysimg.rst | 2 + 4 files changed, 252 insertions(+), 1 deletion(-) create mode 100644 doc/devdocs/init.rst diff --git a/doc/devdocs/init.rst b/doc/devdocs/init.rst new file mode 100644 index 0000000000000..23053333ea917 --- /dev/null +++ b/doc/devdocs/init.rst @@ -0,0 +1,246 @@ +*********************************** +Initialisation of the Julia runtime +*********************************** + +How does the Julia runtime execute :code:`julia -e 'println("Hello World!")'` ? + +main() +------ + +Execution starts at `main() in julia/ui/repl.c +`_. + +main() calls `libsupport_init() +`_ +to set the C library locale and to initialise the "ios" library +(see `ios_init_stdstreams() +`_ +and :ref:`dev-ios`). + +Next `parse_opts() +`_ +is called to process command line options. Note that parse_opts() +only deals with options that effect early initialisation. Other +options are handled later by `process_options() in base/client.jl +`_ + +parse_opts() stores command line options in the `global jl_compileropts +struct +`_. + + +julia_init() +------------ + + +`julia_init() in task.c +`_ is +called by main() and calls `_julia_init() in init.c +`_. + +_julia_init() begins by calling libsupport_init() again (it does +nothing the second time). + +`restore_signals() +`_ is +called to zero the signal handler mask. + +`jl_resolve_sysimg_location() +`_ searches +configured paths for the base system image. See :ref:`dev-sysimg`. + +`jl_gc_init() +`_ +sets up allocation pools and lists for: weak refs, preserved values +and finalization. + +`jl_init_frontend() +`_ +loads and initialises a pre-compiled femptolisp image containing +the scanner/paser; + +`jl_init_types() +`_ +creates jl_datatype_t type description objects for the `built-in +types defined in julia.h +`_. e.g.:: + + jl_any_type = jl_new_abstracttype(jl_symbol("Any"), NULL, jl_null); + jl_any_type->super = jl_any_type; + + jl_type_type = jl_new_abstracttype(jl_symbol("Type"), jl_any_type, jl_null); + + jl_int32_type = jl_new_bitstype(jl_symbol("Int32"), + jl_any_type, jl_null, 32); + +`jl_init_tasks() +`_ creates +the jl_datatype_t* jl_task_type object; initialises the global +`jl_root_task struct +`_; and +sets jl_current_task to the root task. + +`jl_init_codegen() +`_ +initialises the `LLVM libray `_. + +`jl_init_serializer() +`_ +initialises 8-bit serialisation tags for 256 frequently used +jl_value_t values. The serialisation mechanism uses these tags as +shorthand (in lieu of storing whole objects) to save storage space. + +If there is no sysimg file (:code:`!jl_compileropts.image_file`) then +then "Core" and "Main" modules are created and "boot.jl" is evaluated: + + - :code:`jl_core_module = jl_new_module(jl_symbol("Core"))` creates + the Julia "Core" module. + + - `jl_init_intrinsic_functions() + `_ + creates a new Julia module "Intrinsics" containing constant + jl_intrinsic_type symbols. These that define an integer code for + each `intrinsic function + `_. + `emit_intrinsic() + `_ + translates these symbols into LLVM instructions during code generation. + + - `jl_init_primitives() + `_ + hooks C functions up to Julia function symbols. e.g. the symbol + :func:`Base.is` is bound to C function pointer :code:`jl_f_is` + by calling :code:`add_builtin_func("eval", jl_f_top_eval)`, which does:: + + jl_set_const(jl_core_module, + jl_symbol("is"), + jl_new_closure(jl_f_top_eval, jl_symbol("eval"), NULL)); + + + - `jl_new_main_module() + `_ + creates the global "Main" module and sets + :code:`jl_current_task->current_module = jl_main_module`. + + - Note: _julia_init() `then sets `_ :code:`jl_root_task->current_module = jl_core_module`. :code:`jl_root_task` is an alias of :code:`jl_current_task` at this point, so the current_module set by jl_new_main_module() above is overwritten. + + - `jl_load("boot.jl") `_ calls `jl_parse_eval_all("boot.jl") `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute `boot.jl `_. TODO -- dill down into eval? + + - `jl_get_builtin_hooks() `_ initialises global C pointers to Julia globals defined in boot.jl. + + + - `jl_init_box_caches() `_ pre-allocates global boxed integer value objects for values up to 1024. This speeds up allocation of boxed ints later on. e.g.:: + + jl_value_t *jl_box_uint8(uint32_t x) + { + return boxed_uint8_cache[(uint8_t)x]; + } + +If there is a sysimg file, it contains a pre-cooked image of the "Core" and "Main" modules (and whatever else is created above by "boot.jl"). See :ref:`dev-sysimg`. + + - `jl_restore_system_image() `_ de-serialises the saved sysimg into the current Julia runtime environment. + - Note: `jl_restore_system_image() (and dump.c in general) `_ uses the :ref:`dev-ios`.G + +`_julia_init() iterates `_ over the :code:`jl_core_module->bindings.table` looking for :code:`jl_datatype_t` values and sets the type name's module prefix to :code:`jl_core_module`. + +`jl_add_standard_imports(jl_main_module) `_ does "using Base" in the "Main" module. + +Note: _julia_init() `now reverts `_ to :code:`jl_root_task->current_module = jl_main_module` as it was before being `set to jl_core_module `_ above. + +Platform specific signal handlers are initialised for SIGSEGV (OSX, Linux), and SIGFPE (Windows). + +Other signals (SIGINFO, SIGBUS, SIGILL, SIGTERM, SIGABRT, SIGQUIT, SIGSYS and SIGPIPE) are hooked up to `sigdie_handler() `_ which prints a backtrace. + +`jl_init_restored_modules() `_ calls `jl_module_run_initializer() `_ for each de-serialised module to run the "__init__" function. + +Finally `sigint_handler() `_ is hooked up to SIGINT and calls :code:`jl_throw(jl_interrupt_exception)`. + +_julia_init() the returns `back to main() in julia/ui/repl.c +`_ and main() calls :code:`true_main(argc, (char**)argv)`. + +true_main() +----------- + +`true_main() `_ loads the contents of :code:`argv[]` into :data:`Base.ARGS`. + +If a .jl "program" file was supplied on the command line, then `exec_program() `_ calls `jl_load(program) `_ which calls `jl_parse_eval_all() `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute the program. + +However, in our :code:`julia -e 'println("Hello World!")'` example, `jl_get_global(jl_base_module, jl_symbol("_start")) `_ looks up `Base._start `_ and `jl_apply() `_ executes it. + + +Base._start +----------- + +`Base._start `_ calls `Base.process_options `_ which calls `jl_parse_input_line("println(\"Hello World!\")") `_ to create an expression object and :func:`Base.eval` to execute it. + + +Base.eval +--------- + +Base.eval was `mapped to jl_f_top_eval `_ by jl_init_primitives(). + +`jl_f_top_eval() `_ calls `jl_toplevel_eval_in(jl_main_module, ex) `_, where "ex" is the parsed expression :code:`println("Hello World!")`. + +`jl_toplevel_eval_in() `_ calls `jl_toplevel_eval_flex() `_ which calls `eval() in interpreter.c https://github.com/JuliaLang/julia/blob/master/src/interpreter.c#L112`_. + +The stack dump below shows how the interpreter works its way through methods of println, print before arriving at `write{T}(s::AsyncStream, a::Array{T}) `_ which does :code:`ccall(jl_write_no_copy())`. + +`jl_write_no_copy() `_ +calls uv_write() to write "Hello World!" to JL_STDOUT. See :ref:`dev-libuv`.:: + + Hello World! + + +============================ ================= =============================================== +Stack frame Source code Notes +============================ ================= =============================================== +jl_write_no_copy() jl_uv.c:552 called though :func:`Base.ccall` +julia_write_282942 stream.jl:734 function write!{T}(s::AsyncStream, a::Array{T}) +julia_print_284639 ascii.jl:93 print(io::IO, s::ASCIIString) = (write(io, s);nothing) +jlcall_print_284639 +jl_apply() julia.h:989 +jl_trampoline() builtins.c:835 +jl_apply() julia.h:989 +jl_apply_generic() gf.c:1624 Base.print(Base.TTY, ASCIIString) +jl_apply() julia.h:989 +jl_trampoline() builtins.c:835 +jl_apply() julia.h:989 +jl_apply_generic() gf.c:1643 Base.print(Base.TTY, ASCIIString, Char, Char...) +jl_apply() julia.h:989 +jl_f_apply() builtins.c:374 +jl_apply() julia.h:989 +jl_trampoline() builtins.c:835 +jl_apply() julia.h:989 +jl_apply_generic() gf.c:1643 Base.println(Base.TTY, ASCIIString, ASCIIString...) +jl_apply() julia.h:989 +jl_trampoline() builtins.c:835 +jl_apply() julia.h:989 +jl_apply_generic() gf.c:1643 Base.println(ASCIIString,) +jl_apply() julia.h:989 +do_call() interpreter.c:70 +eval() interpreter.c:210 +jl_interpret_toplevel_expr() interpreter.c:25 +jl_toplevel_eval_flex() toplevel.c:498 +jl_toplevel_eval() toplevel.c:521 +jl_toplevel_eval_in() builtins.c:440 +jl_f_top_eval() builtins.c:469 +============================ ================= =============================================== + +Since our example has just one function call, which has done its +job of printing "Hello World!" the stack now rapidly unwinds back to main(). + + +jl_atexit_hook() +---------------- + +main() calls `jl_atexit_hook() +`_. This +calls _atexit for each module, calls `jl_gc_run_all_finalizers() +`_ +and cleans up libuv handles. + + +julia_save() +------------ + +Finally main() calls `julia_save() `_, which if requested on the command line, saves the runtime state to a new system image. See `jl_compile_all() `_ and `jl_save_system_image() `_. diff --git a/doc/devdocs/julia.rst b/doc/devdocs/julia.rst index b1227138fff87..152ee89807b1b 100644 --- a/doc/devdocs/julia.rst +++ b/doc/devdocs/julia.rst @@ -9,6 +9,7 @@ .. toctree:: :maxdepth: 1 + init object cartesian meta diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst index 6bc7bcc537a95..56b20267c572c 100644 --- a/doc/devdocs/object.rst +++ b/doc/devdocs/object.rst @@ -7,7 +7,7 @@ Object layout (jl_value_t) The :code:`jl_value_t` struct defines the minimal header for a Julia object in memory. Every object begins with a pointer to a -:code:`jl_datatype_t` object:: +`jl_datatype_t `_ object:: typedef struct _jl_value_t { struct _jl_value_t *type; @@ -33,6 +33,8 @@ follows:: -- 6 bytes padding }; +Structs for the built-in types are `defined in julia.h `_. The corresponding global jl_datatype_t objects are created by `jl_init_types() `_. + Garbage collector mark bit -------------------------- diff --git a/doc/devdocs/sysimg.rst b/doc/devdocs/sysimg.rst index 15654c8125205..98a666b95fec2 100644 --- a/doc/devdocs/sysimg.rst +++ b/doc/devdocs/sysimg.rst @@ -2,6 +2,8 @@ System Image Building ********************* +.. _dev-sysimg: + Building the Julia system image ------------------------------- From 2ec44e928ae3b68b4f4d2de813beedf5e8834531 Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Sat, 27 Dec 2014 23:11:16 +1100 Subject: [PATCH 3/8] fix typos --- doc/devdocs/init.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/devdocs/init.rst b/doc/devdocs/init.rst index 23053333ea917..a79e560df757c 100644 --- a/doc/devdocs/init.rst +++ b/doc/devdocs/init.rst @@ -82,7 +82,7 @@ sets jl_current_task to the root task. `jl_init_codegen() `_ -initialises the `LLVM libray `_. +initialises the `LLVM library `_. `jl_init_serializer() `_ @@ -99,7 +99,7 @@ then "Core" and "Main" modules are created and "boot.jl" is evaluated: - `jl_init_intrinsic_functions() `_ creates a new Julia module "Intrinsics" containing constant - jl_intrinsic_type symbols. These that define an integer code for + jl_intrinsic_type symbols. These define an integer code for each `intrinsic function `_. `emit_intrinsic() @@ -124,7 +124,7 @@ then "Core" and "Main" modules are created and "boot.jl" is evaluated: - Note: _julia_init() `then sets `_ :code:`jl_root_task->current_module = jl_core_module`. :code:`jl_root_task` is an alias of :code:`jl_current_task` at this point, so the current_module set by jl_new_main_module() above is overwritten. - - `jl_load("boot.jl") `_ calls `jl_parse_eval_all("boot.jl") `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute `boot.jl `_. TODO -- dill down into eval? + - `jl_load("boot.jl") `_ calls `jl_parse_eval_all("boot.jl") `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute `boot.jl `_. TODO -- drill down into eval? - `jl_get_builtin_hooks() `_ initialises global C pointers to Julia globals defined in boot.jl. @@ -139,7 +139,7 @@ then "Core" and "Main" modules are created and "boot.jl" is evaluated: If there is a sysimg file, it contains a pre-cooked image of the "Core" and "Main" modules (and whatever else is created above by "boot.jl"). See :ref:`dev-sysimg`. - `jl_restore_system_image() `_ de-serialises the saved sysimg into the current Julia runtime environment. - - Note: `jl_restore_system_image() (and dump.c in general) `_ uses the :ref:`dev-ios`.G + - Note: `jl_restore_system_image() (and dump.c in general) `_ uses the :ref:`dev-ios`. `_julia_init() iterates `_ over the :code:`jl_core_module->bindings.table` looking for :code:`jl_datatype_t` values and sets the type name's module prefix to :code:`jl_core_module`. @@ -165,7 +165,7 @@ true_main() If a .jl "program" file was supplied on the command line, then `exec_program() `_ calls `jl_load(program) `_ which calls `jl_parse_eval_all() `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute the program. -However, in our :code:`julia -e 'println("Hello World!")'` example, `jl_get_global(jl_base_module, jl_symbol("_start")) `_ looks up `Base._start `_ and `jl_apply() `_ executes it. +However, in our example (:code:`julia -e 'println("Hello World!")'`), `jl_get_global(jl_base_module, jl_symbol("_start")) `_ looks up `Base._start `_ and `jl_apply() `_ executes it. Base._start @@ -181,9 +181,9 @@ Base.eval was `mapped to jl_f_top_eval `_ calls `jl_toplevel_eval_in(jl_main_module, ex) `_, where "ex" is the parsed expression :code:`println("Hello World!")`. -`jl_toplevel_eval_in() `_ calls `jl_toplevel_eval_flex() `_ which calls `eval() in interpreter.c https://github.com/JuliaLang/julia/blob/master/src/interpreter.c#L112`_. +`jl_toplevel_eval_in() `_ calls `jl_toplevel_eval_flex() `_ which calls `eval() in interpreter.c `_. -The stack dump below shows how the interpreter works its way through methods of println, print before arriving at `write{T}(s::AsyncStream, a::Array{T}) `_ which does :code:`ccall(jl_write_no_copy())`. +The stack dump below shows how the interpreter works its way through various methods of :func:`Base.println` and :func:`Base.print` before arriving at `write{T}(s::AsyncStream, a::Array{T}) `_ which does :code:`ccall(jl_write_no_copy())`. `jl_write_no_copy() `_ calls uv_write() to write "Hello World!" to JL_STDOUT. See :ref:`dev-libuv`.:: @@ -235,7 +235,7 @@ jl_atexit_hook() main() calls `jl_atexit_hook() `_. This -calls _atexit for each module, calls `jl_gc_run_all_finalizers() +calls _atexit for each module, then calls `jl_gc_run_all_finalizers() `_ and cleans up libuv handles. From 78474c461b5e3b8a0a29784fb91121420580b95f Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Sun, 28 Dec 2014 10:06:36 +1100 Subject: [PATCH 4/8] Upadates per @Keno's feedback. --- doc/devdocs/init.rst | 68 ++++++++++++++++++++++-------------------- doc/devdocs/object.rst | 11 +++++-- 2 files changed, 45 insertions(+), 34 deletions(-) diff --git a/doc/devdocs/init.rst b/doc/devdocs/init.rst index a79e560df757c..6bbf4ec4ec86b 100644 --- a/doc/devdocs/init.rst +++ b/doc/devdocs/init.rst @@ -20,7 +20,7 @@ and :ref:`dev-ios`). Next `parse_opts() `_ is called to process command line options. Note that parse_opts() -only deals with options that effect early initialisation. Other +only deals with options that affect code generation or early initialisation. Other options are handled later by `process_options() in base/client.jl `_ @@ -90,57 +90,61 @@ initialises 8-bit serialisation tags for 256 frequently used jl_value_t values. The serialisation mechanism uses these tags as shorthand (in lieu of storing whole objects) to save storage space. +.. sidebar:: sysimg + + If there is a sysimg file, it contains a pre-cooked image of the "Core" and "Main" modules (and whatever else is created by "boot.jl"). See :ref:`dev-sysimg`. + + `jl_restore_system_image() `_ de-serialises the saved sysimg into the current Julia runtime environment and initialisation continues after jl_init_box_caches() below... + + Note: `jl_restore_system_image() (and dump.c in general) `_ uses the :ref:`dev-ios`. + + If there is no sysimg file (:code:`!jl_compileropts.image_file`) then then "Core" and "Main" modules are created and "boot.jl" is evaluated: - - :code:`jl_core_module = jl_new_module(jl_symbol("Core"))` creates - the Julia "Core" module. - - - `jl_init_intrinsic_functions() - `_ - creates a new Julia module "Intrinsics" containing constant - jl_intrinsic_type symbols. These define an integer code for - each `intrinsic function - `_. - `emit_intrinsic() - `_ - translates these symbols into LLVM instructions during code generation. - - - `jl_init_primitives() - `_ - hooks C functions up to Julia function symbols. e.g. the symbol - :func:`Base.is` is bound to C function pointer :code:`jl_f_is` - by calling :code:`add_builtin_func("eval", jl_f_top_eval)`, which does:: +:code:`jl_core_module = jl_new_module(jl_symbol("Core"))` creates +the Julia "Core" module. + +`jl_init_intrinsic_functions() +`_ +creates a new Julia module "Intrinsics" containing constant +jl_intrinsic_type symbols. These define an integer code for +each `intrinsic function +`_. +`emit_intrinsic() +`_ +translates these symbols into LLVM instructions during code generation. + +`jl_init_primitives() +`_ +hooks C functions up to Julia function symbols. e.g. the symbol +:func:`Base.is` is bound to C function pointer :code:`jl_f_is` +by calling :code:`add_builtin_func("eval", jl_f_top_eval)`, which does:: jl_set_const(jl_core_module, jl_symbol("is"), jl_new_closure(jl_f_top_eval, jl_symbol("eval"), NULL)); - - `jl_new_main_module() - `_ - creates the global "Main" module and sets - :code:`jl_current_task->current_module = jl_main_module`. +`jl_new_main_module() +`_ +creates the global "Main" module and sets +:code:`jl_current_task->current_module = jl_main_module`. - - Note: _julia_init() `then sets `_ :code:`jl_root_task->current_module = jl_core_module`. :code:`jl_root_task` is an alias of :code:`jl_current_task` at this point, so the current_module set by jl_new_main_module() above is overwritten. +Note: _julia_init() `then sets `_ :code:`jl_root_task->current_module = jl_core_module`. :code:`jl_root_task` is an alias of :code:`jl_current_task` at this point, so the current_module set by jl_new_main_module() above is overwritten. - - `jl_load("boot.jl") `_ calls `jl_parse_eval_all("boot.jl") `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute `boot.jl `_. TODO -- drill down into eval? +`jl_load("boot.jl") `_ calls `jl_parse_eval_all("boot.jl") `_ which repeatedly calls `jl_parse_next() `_ and `jl_toplevel_eval_flex() `_ to parse and execute `boot.jl `_. TODO -- drill down into eval? - - `jl_get_builtin_hooks() `_ initialises global C pointers to Julia globals defined in boot.jl. +`jl_get_builtin_hooks() `_ initialises global C pointers to Julia globals defined in boot.jl. - - `jl_init_box_caches() `_ pre-allocates global boxed integer value objects for values up to 1024. This speeds up allocation of boxed ints later on. e.g.:: +`jl_init_box_caches() `_ pre-allocates global boxed integer value objects for values up to 1024. This speeds up allocation of boxed ints later on. e.g.:: jl_value_t *jl_box_uint8(uint32_t x) { return boxed_uint8_cache[(uint8_t)x]; } -If there is a sysimg file, it contains a pre-cooked image of the "Core" and "Main" modules (and whatever else is created above by "boot.jl"). See :ref:`dev-sysimg`. - - - `jl_restore_system_image() `_ de-serialises the saved sysimg into the current Julia runtime environment. - - Note: `jl_restore_system_image() (and dump.c in general) `_ uses the :ref:`dev-ios`. - `_julia_init() iterates `_ over the :code:`jl_core_module->bindings.table` looking for :code:`jl_datatype_t` values and sets the type name's module prefix to :code:`jl_core_module`. `jl_add_standard_imports(jl_main_module) `_ does "using Base" in the "Main" module. diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst index 56b20267c572c..5a6d5a1270782 100644 --- a/doc/devdocs/object.rst +++ b/doc/devdocs/object.rst @@ -5,14 +5,21 @@ Memory layout of Julia Objects Object layout (jl_value_t) -------------------------- +.. sidebar:: `special case. `_ + + :code:`jl_tuple_type->type = jl_tuple_type` + The :code:`jl_value_t` struct defines the minimal header for a Julia -object in memory. Every object begins with a pointer to a +object in memory. +The :code:`type` field points to a `jl_datatype_t `_ object:: typedef struct _jl_value_t { struct _jl_value_t *type; } jl_value_t; + + The layout of the rest of the object is dependant on its type. e.g. a :func:`Base.tuple` object has an array of pointers to the @@ -46,7 +53,7 @@ reachable object, deallocates objects that are not marked, then clears the mark bits. While the mark/sweep is in progress the :code:`jl_value_t.type` pointer is altered by the mark bit. The gc uses the :func:`gc_typeof` macro to retrieve the original type -pointer. +pointer:: #define gc_typeof(v) ((jl_value_t*)(((uptrint_t)jl_typeof(v))&~1UL)) From 25006aa152af67482cd31c878d7184b8dd1e9c87 Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Sun, 28 Dec 2014 18:30:14 +1100 Subject: [PATCH 5/8] tweaks per feedback --- doc/devdocs/object.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst index 5a6d5a1270782..5f20250dd938a 100644 --- a/doc/devdocs/object.rst +++ b/doc/devdocs/object.rst @@ -12,18 +12,20 @@ Object layout (jl_value_t) The :code:`jl_value_t` struct defines the minimal header for a Julia object in memory. The :code:`type` field points to a -`jl_datatype_t `_ object:: +`jl_datatype_t `_ object, +(the jl_typeof() macro should be used to query it):: typedef struct _jl_value_t { struct _jl_value_t *type; } jl_value_t; + #define jl_typeof(v) (((jl_value_t*)(v))->type) The layout of the rest of the object is dependant on its type. e.g. a :func:`Base.tuple` object has an array of pointers to the -objects contained by the tuple:: +objects contained by the tuple (or an array of raw values for un-boxed bitstype tuples):: typedef struct { struct _jl_value_t *type; @@ -74,4 +76,4 @@ Note that all objects are allocated in multiples of 8 bytes, so the smallest object size is 16 bytes (8 byte type pointer + 8 bytes data). :func:`allocobj` in gc.c allocates memory for new objects. Memory is allocated from a pool for objects up to 2048 bytes, or -by malloc() otherwise.:: +by malloc() otherwise. From c7e4ff709246dabb476f728425bb7834ecac286b Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Sun, 28 Dec 2014 18:39:11 +1100 Subject: [PATCH 6/8] 64-bit note --- doc/devdocs/object.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst index 5f20250dd938a..52689819696bd 100644 --- a/doc/devdocs/object.rst +++ b/doc/devdocs/object.rst @@ -34,7 +34,7 @@ objects contained by the tuple (or an array of raw values for un-boxed bitstype } jl_tuple_t; e.g. a "boxed" uint16_t (created by :func:`jl_box_uint16`) is stored as -follows:: +follows (assuming machine is 64-bit):: struct { struct _jl_value_t *type; -- 8 bytes From 8b885932c49a68f7a075d2ee6853a17d2ca0f496 Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Mon, 29 Dec 2014 10:11:55 +1100 Subject: [PATCH 7/8] note about 8-byte "nothing" object --- doc/devdocs/object.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst index 52689819696bd..703f6f228aa54 100644 --- a/doc/devdocs/object.rst +++ b/doc/devdocs/object.rst @@ -25,7 +25,7 @@ The :code:`type` field points to a The layout of the rest of the object is dependant on its type. e.g. a :func:`Base.tuple` object has an array of pointers to the -objects contained by the tuple (or an array of raw values for un-boxed bitstype tuples):: +objects contained by the tuple:: typedef struct { struct _jl_value_t *type; @@ -72,6 +72,12 @@ Storage for new objects is allocated by :func:`newobj` in julia_internal.h:: return jv; } +.. sidebar:: `special case. `_ + + The special singleton object :data:`nothing` is "void". + It has no data and consumes only 8 bytes. + :code:`jl_nothing = newstruct(jl_void_type)` + Note that all objects are allocated in multiples of 8 bytes, so the smallest object size is 16 bytes (8 byte type pointer + 8 bytes data). :func:`allocobj` in gc.c allocates memory for new objects. From 7acbfc6661aa015469a18a76b5e89451f209cc35 Mon Sep 17 00:00:00 2001 From: Sam O'Connor Date: Mon, 29 Dec 2014 14:24:08 +1100 Subject: [PATCH 8/8] Clarificaiton of nothing and singletons --- doc/devdocs/object.rst | 11 +++++++---- doc/manual/faq.rst | 2 ++ 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/doc/devdocs/object.rst b/doc/devdocs/object.rst index 703f6f228aa54..6eda9816584be 100644 --- a/doc/devdocs/object.rst +++ b/doc/devdocs/object.rst @@ -72,11 +72,14 @@ Storage for new objects is allocated by :func:`newobj` in julia_internal.h:: return jv; } -.. sidebar:: `special case. `_ +.. sidebar:: :ref:`man-singleton-types` - The special singleton object :data:`nothing` is "void". - It has no data and consumes only 8 bytes. - :code:`jl_nothing = newstruct(jl_void_type)` + + Singleton types have only one instance and no data fields. + Singleton instances use only 8 bytes. + e.g. :data:`nothing::Void`. + + See :ref:`man-singleton-types` and :ref:`man-nothing` Note that all objects are allocated in multiples of 8 bytes, so the smallest object size is 16 bytes (8 byte type pointer + 8 bytes diff --git a/doc/manual/faq.rst b/doc/manual/faq.rst index d13ce1a88a54f..d6abfd403ef9c 100644 --- a/doc/manual/faq.rst +++ b/doc/manual/faq.rst @@ -746,6 +746,8 @@ To prevent this, we can add an inner constructor:: The inner constructor requires that the element type of ``A`` be ``T``. +.. _man-nothing: + Nothingness and missing values ------------------------------