Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lowercase builtin vars #956

Merged
merged 36 commits into from
Jul 8, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
90cde68
Remove uppercase vars in dune files
rgrinberg Jul 5, 2018
f2f0d24
Add tests for lower/uppercase vars
rgrinberg Jul 5, 2018
9d7107d
Add renamed vars to migration
rgrinberg Jul 5, 2018
998ff8c
Clarify rename test
rgrinberg Jul 6, 2018
1a37977
Fix error message
rgrinberg Jul 6, 2018
a0a92d9
Introduce a Dir primitive to Value
rgrinberg Jul 6, 2018
54ff98b
Rename ROOT to root
rgrinberg Jul 6, 2018
8ef6af2
Add some helpers to String_with_loc
rgrinberg Jul 7, 2018
9811031
Implement a more elaborate variable expansion mechanism
rgrinberg Jul 7, 2018
3f7ad78
Replace more code with the variable system
rgrinberg Jul 7, 2018
3a7b62a
Use the variable for dynamic expansions as well
rgrinberg Jul 7, 2018
6159a29
Remove one off function
rgrinberg Jul 7, 2018
2a71439
Toyish gadtization to remove a few assert falses
rgrinberg Jul 7, 2018
972406a
Fix static dependencies
rgrinberg Jul 7, 2018
73cc3b1
Simplify Var.t definition
rgrinberg Jul 7, 2018
2f86a4d
Rename the expand family of functions
rgrinberg Jul 7, 2018
43b15bf
Add Var module to Super_context
rgrinberg Jul 7, 2018
843792a
Switch from maps to tables for variables
rgrinberg Jul 7, 2018
9750f84
Remove String_with_var.Loc.fail
rgrinberg Jul 8, 2018
7b05fc3
Add String_with_vars.Var.name
rgrinberg Jul 8, 2018
5f783be
Use maps instead of hash tables for variable lookup
rgrinberg Jul 8, 2018
44dc039
s/Nothing/No_info/
rgrinberg Jul 8, 2018
f95c9e0
Simplify some checks with an is_form function
rgrinberg Jul 8, 2018
c244fa9
Share error messages for renamed/since/deleted etc.
rgrinberg Jul 8, 2018
78140ca
Remove the targets helper function
rgrinberg Jul 8, 2018
1b918ec
Improve error message for renamed forms
rgrinberg Jul 8, 2018
647f68d
Move Super_context.Var into own module
rgrinberg Jul 8, 2018
813b8d9
Use correct syntax for error
rgrinberg Jul 8, 2018
7cb068d
Improve error messages for all forms
rgrinberg Jul 8, 2018
1a9895a
Consistent naming for vars and forms
rgrinberg Jul 8, 2018
1fb3af8
Rename var to pform
rgrinberg Jul 8, 2018
bfd246c
Rename Kind to Var in Pform
rgrinberg Jul 8, 2018
a24b55e
Add error for %{read:x} in wrong place
rgrinberg Jul 8, 2018
ccabeb7
Hack to fix printing of errors
rgrinberg Jul 8, 2018
a215353
Fix error messages when percent macros are used incorrectly
rgrinberg Jul 8, 2018
78370d3
Rename test not to use outdated form term
rgrinberg Jul 8, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 48 additions & 48 deletions doc/dune-files.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Stanzas
(rule
(targets foo.ml)
(deps generator/gen.exe)
(action (run ${<} -o ${@})))
(action (run %{<} -o %{@})))

The following sections describe the available stanzas and their meaning.

Expand Down Expand Up @@ -336,16 +336,16 @@ compilation mode binary kind extensions
---------------- ------------- -----------------
byte exe .bc and .bc.js
native/best exe .exe
byte object .bc${ext_obj}
native/best object .exe${ext_obj}
byte shared_object .bc${ext_dll}
native/best shared_object ${ext_dll}
byte object .bc%{ext_obj}
native/best object .exe%{ext_obj}
byte shared_object .bc%{ext_dll}
native/best shared_object %{ext_dll}
================ ============= =================

Where ``${ext_obj}`` and ``${ext_dll}`` are the extensions for object
Where ``%{ext_obj}`` and ``%{ext_dll}`` are the extensions for object
and shared object files. Their value depends on the OS, for instance
on Unix ``${ext_obj}`` is usually ``.o`` and ``${ext_dll}`` is usually
``.so`` while on Windows ``${ext_obj}`` is ``.obj`` and ``${ext_dll}``
on Unix ``%{ext_obj}`` is usually ``.o`` and ``%{ext_dll}`` is usually
``.so`` while on Windows ``%{ext_obj}`` is ``.obj`` and ``%{ext_dll}``
is ``.dll``.

Note that when ``(byte exe)`` is specified but neither ``(best exe)``
Expand Down Expand Up @@ -459,7 +459,7 @@ For instance:
(rule
(targets b
(deps a
(action (copy ${<} ${@})))))
(action (copy %{<} %{@})))))

In this example it is obvious by inspecting the action what the
dependencies and targets are. When this is the case you can use the
Expand All @@ -483,7 +483,7 @@ stanza is rejected by dune:

.. code:: scheme

(rule (copy a b.${read:file}))
(rule (copy a b.%{read:file}))

ocamllex
--------
Expand All @@ -495,7 +495,7 @@ ocamllex
(rule
(targets <name>.ml)
(deps <name>.mll)
(action (chdir ${ROOT} (run ${bin:ocamllex} -q -o ${<}))))
(action (chdir %{root} (run %{bin:ocamllex} -q -o %{<}))))

To use a different rule mode, use the long form:

Expand All @@ -515,7 +515,7 @@ ocamlyacc
(rule
(targets <name>.ml <name>.mli)
(deps <name>.mly)
(action (chdir ${ROOT} (run ${bin:ocamlyacc} ${<}))))
(action (chdir %{root} (run %{bin:ocamlyacc} %{<}))))

To use a different rule mode, use the long form:

Expand Down Expand Up @@ -573,7 +573,7 @@ The typical use of the ``alias`` stanza is to define tests:

(alias
(name runtest)
(action (run ${exe:my-test-program.exe} blah)))
(action (run %{exe:my-test-program.exe} blah)))

See the section about :ref:`running-tests` for details.

Expand Down Expand Up @@ -825,32 +825,32 @@ Variables are expanded after the set language is interpreted.
Variables expansion
-------------------

Some fields can contains variables of the form ``$(var)`` or ``${var}`` that are
Some fields can contains variables of the form ``$(var)`` or ``%{var}`` that are
expanded by dune.

Dune supports the following variables:

- ``ROOT`` is the relative path to the root of the build
context. Note that ``ROOT`` depends on the worksace
configuration. As such you shouldn't use ``ROOT`` to denote the
root of your project. Use ``SCOPE_ROOT`` instead for this purpose
- ``SCOPE_ROOT`` is the root of the current scope. It is typically
- ``root`` is the relative path to the root of the build
context. Note that ``root`` depends on the workspace
configuration. As such you shouldn't use ``root`` to denote the
root of your project. Use ``project_root`` instead for this purpose
- ``project_root`` is the root of the current scope. It is typically
the toplevel directory of your project and as long as you have at
least one ``<package>.opam`` file there, ``SCOPE_ROOT`` is
least one ``<package>.opam`` file there, ``project_root`` is
independent of the workspace configuration
- ``CC`` is the C compiler command line (list made of the compiler
name followed by its flags) that was used to compile OCaml in the
current build context
- ``CXX`` is the C++ compiler command line being used in the
current build context
- ``ocaml_bin`` is the path where ``ocamlc`` lives
- ``OCAML`` is the ``ocaml`` binary
- ``OCAMLC`` is the ``ocamlc`` binary
- ``OCAMLOPT`` is the ``ocamlopt`` binary
- ``ocaml`` is the ``ocaml`` binary
- ``ocamlc`` is the ``ocamlc`` binary
- ``ocamlopt`` is the ``ocamlopt`` binary
- ``ocaml_version`` is the version of the compiler used in the
current build context
- ``ocaml_where`` is the output of ``ocamlc -where``
- ``ARCH_SIXTYFOUR`` is ``true`` if using a compiler targeting a
- ``arch_sixtyfour`` is ``true`` if using a compiler targeting a
64 bit architecture and ``false`` otherwise
- ``null`` is ``/dev/null`` on Unix or ``nul`` on Windows
- ``ext_obj``, ``ext_asm``, ``ext_lib``, ``ext_dll`` and ``ext_exe``
Expand Down Expand Up @@ -878,8 +878,8 @@ In addition, ``(action ...)`` fields support the following special variables:
is installed by a package in the workspace (see `install`_ stanzas),
the locally built binary will be used, otherwise it will be searched
in the ``PATH`` of the current build context. Note that ``(run
${bin:program} ...)`` and ``(run program ...)`` behave in the same
way. ``${bin:...}`` is only necessary when you are using ``(bash
%{bin:program} ...)`` and ``(run program ...)`` behave in the same
way. ``%{bin:...}`` is only necessary when you are using ``(bash
...)`` or ``(system ...)``
- ``lib:<public-library-name>:<file>`` expands to a path to file ``<file>`` of
library ``<public-library-name>``. If ``<public-library-name>`` is available
Expand All @@ -906,10 +906,10 @@ In addition, ``(action ...)`` fields support the following special variables:
- ``read-strings:<path>`` expands to the list of lines in the given
file, unescaped using OCaml lexical convention

The ``${<kind>:...}`` forms are what allows you to write custom rules that work
The ``%{<kind>:...}`` forms are what allows you to write custom rules that work
transparently whether things are installed or not.

Note that aliases are ignored by both ``${<}`` and ``${^}``.
Note that aliases are ignored by both ``%{<}`` and ``%{^}``.

The intent of this last form is to reliably read a list of strings
generated by an OCaml program via:
Expand All @@ -920,13 +920,13 @@ generated by an OCaml program via:

#. Expansion of lists

Forms that expands to list of items, such as ``${CC}``, ``${^}``,
``${@}`` or ``${read-lines:...}``, are suitable to be used in, say,
Forms that expands to list of items, such as ``%{cc}``, ``%{^}``,
``%{@}`` or ``%{read-lines:...}``, are suitable to be used in, say,
``(run <prog> <arguments>)``. For instance in:

.. code:: scheme

(run foo ${^})
(run foo %{^})

if there are two dependencies ``a`` and ``b``, the produced command
will be equivalent to the shell command:
Expand All @@ -940,7 +940,7 @@ you have to quote the variable as in:

.. code:: scheme

(run foo "${^}")
(run foo "%{^}")

which is equivalent to the following shell command:

Expand All @@ -949,15 +949,15 @@ which is equivalent to the following shell command:
$ foo "a b"

(the items of the list are concatenated with space).
Note that, since ``${^}`` is a list of items, the first one may be
Note that, since ``%{^}`` is a list of items, the first one may be
used as a program name, for instance:

.. code:: scheme

(rule
(targets result.txt)
(deps foo.exe (glob_files *.txt))
(action (run ${^})))
(action (run %{^})))

Here is another example:

Expand All @@ -966,7 +966,7 @@ Here is another example:
(rule
(targets foo.exe)
(deps foo.c)
(action (run ${CC} -o ${@} ${<} -lfoolib)))
(action (run %{cc} -o %{@} %{<} -lfoolib)))


Library dependencies
Expand Down Expand Up @@ -1052,10 +1052,10 @@ you had setup a rule for every file of the form:
(rule
(targets file.pp.ml)
(deps file.ml)
(action (with-stdout-to ${@} (chdir ${ROOT} <action>))))
(action (with-stdout-to %{@} (chdir %{root} <action>))))

The equivalent of a ``-pp <command>`` option passed to the OCaml compiler is
``(system "<command> ${<}")``.
``(system "<command> %{<}")``.

Preprocessing with ppx rewriters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -1105,8 +1105,8 @@ For instance:
.. code:: scheme

(preprocess (per_module
(((action (run ./pp.sh X=1 ${<})) (foo bar)))
(((action (run ./pp.sh X=2 ${<})) (baz)))))
(((action (run ./pp.sh X=1 %{<})) (foo bar)))
(((action (run ./pp.sh X=2 %{<})) (baz)))))

.. _deps-field:

Expand Down Expand Up @@ -1231,7 +1231,7 @@ in ``src/foo/dune`` will be run from ``_build/<context>/src/foo``.
The argument of ``(action ...)`` fields is a small DSL that is interpreted by
dune directly and doesn't require an external shell. All atoms in the DSL
support `Variables expansion`_. Moreover, you don't need to specify dependencies
explicitly for the special ``${<kind>:...}`` forms, these are recognized and
explicitly for the special ``%{<kind>:...}`` forms, these are recognized and
automatically handled by dune.

The DSL is currently quite limited, so if you want to do something complicated
Expand Down Expand Up @@ -1289,15 +1289,15 @@ called ``copy-and-add-line-directive``. However, most of time one
wants this behavior rather than a bare copy, so it was renamed to
something shorter.

Note: expansion of the special ``${<kind>:...}`` is done relative to the current
Note: expansion of the special ``%{<kind>:...}`` is done relative to the current
working directory of the part of the DSL being executed. So for instance if you
have this action in a ``src/foo/dune``:

.. code:: scheme

(action (chdir ../../.. (echo ${path:dune})))
(action (chdir ../../.. (echo %{path:dune})))

Then ``${path:dune}`` will expand to ``src/foo/dune``. When you run various
Then ``%{path:dune}`` will expand to ``src/foo/dune``. When you run various
tools, they often use the filename given on the command line in error messages.
As a result, if you execute the command from the original directory, it will
only see the basename.
Expand All @@ -1310,7 +1310,7 @@ To understand why this is important, let's consider this dune file living in
(rule
(targets blah.ml)
(deps blah.mll)
(action (run ocamllex -o ${@} ${<})))
(action (run ocamllex -o %{@} %{<})))

Here the command that will be executed is:

Expand All @@ -1334,7 +1334,7 @@ of your project. What you should write instead is:
(rule
(targets blah.ml)
(deps blah.mll)
(action (chdir ${ROOT} (run ocamllex -o ${@} ${<}))))
(action (chdir %{root} (run ocamllex -o %{@} %{<}))))

Locks
-----
Expand All @@ -1357,13 +1357,13 @@ same lock:
(name runtest)
(deps foo)
(locks m)
(action (run test.exe ${<})))
(action (run test.exe %{<})))

(alias
(name runtest)
(deps bar)
(locks m)
(action (run test.exe ${<})))
(action (run test.exe %{<})))

Dune will make sure that the executions of ``test.exe foo`` and
``test.exe bar`` are serialized.
Expand All @@ -1383,7 +1383,7 @@ simply use an absolute filename:
(name runtest)
(deps foo)
(locks /tcp-port/1042)
(action (run test.exe ${<})))
(action (run test.exe %{<})))

.. _ocaml-syntax:

Expand Down
8 changes: 8 additions & 0 deletions doc/migration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,14 @@ Jbuild Dune
``${path:file}`` ``%{dep:file}``
``${SCOPE_ROOT}`` ``%{project_root}``
``${findlib:..}`` ``%{lib:..}``
``${CPP}`` ``%{cpp}``
``${CC}`` ``%{cc}``
``${CXX}`` ``%{cxx}``
``${OCAML}`` ``%{ocaml}``
``${OCAMLC}`` ``%{ocamlc}``
``${OCAMLOPT}`` ``%{ocamlopt}``
``${ARCH_SIXTYFOUR}`` ``%{arch_sixtyfour}``
``${MAKE}`` ``%{make}``
======================== ============

Removed Variables
Expand Down
3 changes: 3 additions & 0 deletions src/action.ml
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,9 @@ end
let prog_and_args_of_values p ~dir =
match p with
| [] -> (Unresolved.Program.Search "", [])
| Value.Dir p :: _ ->
die "%s is a directory and cannot be used as an executable"
(Path.to_string_maybe_quoted p)
| Value.Path p :: xs -> (This p, Value.L.to_strings ~dir xs)
| String s :: xs ->
(Unresolved.Program.of_string ~dir s, Value.L.to_strings ~dir xs)
Expand Down
4 changes: 2 additions & 2 deletions src/gen_rules.ml
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ module Gen(P : Install_rules.Params) = struct
let copy_files_rules (def: Copy_files.t) ~src_dir ~dir ~scope =
let loc = String_with_vars.loc def.glob in
let glob_in_src =
let src_glob = SC.expand_vars sctx ~dir def.glob ~scope in
let src_glob = SC.expand_vars_string sctx ~dir def.glob ~scope in
Path.relative src_dir src_glob ~error_loc:loc
in
(* The following condition is required for merlin to work.
Expand Down Expand Up @@ -1042,7 +1042,7 @@ module Gen(P : Install_rules.Params) = struct
| Copy_files { glob; _ } ->
let src_dir =
let loc = String_with_vars.loc glob in
let src_glob = SC.expand_vars sctx ~dir glob ~scope in
let src_glob = SC.expand_vars_string sctx ~dir glob ~scope in
Path.parent_exn (Path.relative src_dir src_glob ~error_loc:loc)
in
Some
Expand Down
4 changes: 2 additions & 2 deletions src/jbuild.ml
Original file line number Diff line number Diff line change
Expand Up @@ -1238,7 +1238,7 @@ module Rule = struct
; action =
(loc,
Chdir
(S.virt_var __POS__ "ROOT",
(S.virt_var __POS__ "root",
Run (S.virt_text __POS__ "ocamllex",
[ S.virt_text __POS__ "-q"
; S.virt_text __POS__ "-o"
Expand All @@ -1259,7 +1259,7 @@ module Rule = struct
; action =
(loc,
Chdir
(S.virt_var __POS__ "ROOT",
(S.virt_var __POS__ "root",
Run (S.virt_text __POS__ "ocamlyacc",
[S.virt_var __POS__ "first-dep"])))
; mode
Expand Down
2 changes: 1 addition & 1 deletion src/menhir.ml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ module Run (P : PARAMS) = struct
let sources ms =
List.map ~f:source ms

(* Expand special variables, such as ${ROOT}, in the stanza's flags. *)
(* Expand special variables, such as %{root}, in the stanza's flags. *)

let flags =
SC.expand_and_eval_set
Expand Down
Loading