Bitfield constants

doc/autogen/types/bitfield.rst

@@ -1,5 +1,9 @@
 .. rubric:: Operators
 
+.. spicy:operator:: bitfield::HasMember bool t:bitfield <sp> op:?. <sp> t:<field>
+
+    Returns true if the bitfield's element has a value.
+
 .. spicy:operator:: bitfield::Member <field~type> t:bitfield <sp> op:. <sp> t:<attribute>
 
     Retrieves the value of a bitfield's attribute. This is the value of

doc/examples/_skip.spicy

@@ -0,0 +1,9 @@
+# Automatically generated; edit in Sphinx source code, not here.
+module Test;
+
+public type Foo = unit {
+    x: int8;
+     : skip bytes &size=5;
+    y: int8;
+    on %done { print self; }
+};

doc/programming/language/types.rst

@@ -40,6 +40,18 @@ parsed inside a unit.
 - Each ``RANGE`` has one of the forms ``LABEL: A`` or ``LABEL: A..B``
   where ``A`` and ``B`` are bit numbers.
 
+.. rubric:: Constants
+
+- ``bitfield(N) { RANGE_1 [= VALUE_1]; ...; RANGE_N [= VALUE_N] }``
+
+A bitfield constant represents expected values for all or some of the
+individual bitranges. They can be used only for parsing inside a unit
+field, not as values to otherwise operate with. To define such a
+constant with expected values, add ``= VALUE`` to the bitranges inside
+the type definition as suitable (with ``VALUE`` representing the final
+value after applying any ``&bit-order`` attribute, if present). See
+:ref:`parse_bitfield` for more information.
+
 .. include:: /autogen/types/bitfield.rst
 
 .. _type_bool:

doc/programming/parsing.rst

@@ -969,12 +969,12 @@ Example:
 Bitfield
 ^^^^^^^^
 
-Bitfields parse an integer value of a given size, and then make
-selected smaller bit ranges within that value available individually
-through dedicated identifiers. For example, the following unit parses
-4 bytes as an ``uint32`` and then makes the value of bit 0 available
-as ``f.x1``, bits 1 to 2 as ``f.x2``, and bits 3 to 4 as ``f.x3``,
-respectively:
+:ref:`Bitfields <type_bitfield>` parse an integer value of a given
+size, and then make selected smaller bit ranges within that value
+available individually through dedicated identifiers. For example, the
+following unit parses 4 bytes as an ``uint32`` and then makes the
+value of bit 0 available as ``f.x1``, bits 1 to 2 as ``f.x2``, and
+bits 3 to 4 as ``f.x3``, respectively:
 
 .. spicy-code:: parse-bitfield.spicy
 
@@ -1037,6 +1037,25 @@ range to an enum, using ``$$`` to access the parsed value:
     :exec: printf '\x21' | spicy-driver %INPUT
     :show-with: foo.spicy
 
+When parsing a bitfield, you can enforce expected values for some
+or all of the bitranges through an assignment-style syntax:
+
+.. spicy-code::
+
+    type Foo = unit {
+        f: bitfield(8) {
+            x1: 0..3 = 2;
+            x2: 4..5;
+            x3: 6..7 = 3;
+        }
+    };
+
+Now parsing will fail if values of ``x1`` and ``x3`` aren't ``2`` and
+``3``, respectively. Internally, Spicy treats bitfields with such
+expected values similar to constants of other types, meaning they
+operate as valid look-ahead symbols as well (see
+:ref:`parse_lookahead`).
+
 .. _parse_bytes:
 
 Bytes

hilti/runtime/include/types/bitfield.h

@@ -19,7 +19,7 @@ namespace hilti::rt {
 /// printing.
 template<typename... Ts>
 struct Bitfield {
-    std::tuple<Ts...> value;
+    std::tuple<std::optional<Ts>...> value;
 };
 
 template<typename... Ts>

hilti/toolchain/include/ast/ctors/all.h

@@ -3,6 +3,7 @@
 #pragma once
 
 #include <hilti/ast/ctors/address.h>
+#include <hilti/ast/ctors/bitfield.h>
 #include <hilti/ast/ctors/bool.h>
 #include <hilti/ast/ctors/bytes.h>
 #include <hilti/ast/ctors/coerced.h>

hilti/toolchain/include/ast/ctors/bitfield.h

@@ -0,0 +1,75 @@
+// Copyright (c) 2020-2023 by the Zeek Project. See LICENSE for details.
+
+#pragma once
+
+#include <algorithm>
+#include <utility>
+#include <vector>
+
+#include <hilti/ast/ctor.h>
+#include <hilti/ast/expression.h>
+#include <hilti/ast/id.h>
+#include <hilti/ast/types/bitfield.h>
+
+namespace hilti::ctor {
+
+namespace bitfield {
+/** AST node for a bitfield element value. */
+class Bits : public NodeBase {
+public:
+    Bits(ID id, Expression e, Meta m = Meta()) : NodeBase(nodes(std::move(id), std::move(e)), std::move(m)) {}
+    Bits(Meta m = Meta()) : NodeBase(nodes(node::none, node::none), std::move(m)) {}
+
+    const auto& id() const { return child<ID>(0); }
+    const auto& expression() const { return child<Expression>(1); }
+
+    /** Implements the `Node` interface. */
+    auto properties() const { return node::Properties{}; }
+
+    bool operator==(const Bits& other) const { return id() == other.id() && expression() == other.expression(); }
+};
+
+inline Node to_node(Bits f) { return Node(std::move(f)); }
+} // namespace bitfield
+
+/** AST node for a bitfield constructor. */
+class Bitfield : public NodeBase, public hilti::trait::isCtor {
+public:
+    Bitfield(std::vector<bitfield::Bits> bits, type::Bitfield type, Meta m = Meta())
+        : NodeBase(nodes(std::move(type), std::move(bits)), std::move(m)) {}
+
+    /** Returns all bits that the constructors initializes. */
+    auto bits() const { return children<bitfield::Bits>(1, -1); }
+
+    /** Returns the underlying bitfield type. */
+    const auto& btype() const { return child<type::Bitfield>(0); }
+
+    /** Returns a field initialized by the constructor by its ID. */
+    hilti::optional_ref<const bitfield::Bits> bits(const ID& id) const {
+        for ( const auto& b : bits() ) {
+            if ( b.id() == id )
+                return b;
+        }
+
+        return {};
+    }
+
+    bool operator==(const Bitfield& other) const { return bits() == other.bits() && btype() == other.btype(); }
+
+    /** Implements `Ctor` interface. */
+    const auto& type() const { return child<Type>(0); }
+
+    /** Implements `Ctor` interface. */
+    bool isConstant() const { return true; }
+    /** Implements `Ctor` interface. */
+    auto isLhs() const { return false; }
+    /** Implements `Ctor` interface. */
+    auto isTemporary() const { return true; }
+    /** Implements `Ctor` interface. */
+    auto isEqual(const Ctor& other) const { return node::isEqual(this, other); }
+
+    /** Implements `Node` interface. */
+    auto properties() const { return node::Properties{}; }
+};
+
+} // namespace hilti::ctor

hilti/toolchain/include/ast/nodes.decl

@@ -25,6 +25,7 @@ hilti::type::enum_::Label : isNode
 hilti::type::tuple::Element : isNode
 
 hilti::ctor::Address : isCtor
+hilti::ctor::Bitfield : isCtor
 hilti::ctor::Bool : isCtor
 hilti::ctor::Bytes : isCtor
 hilti::ctor::Coerced : isCtor

hilti/toolchain/include/ast/operators/bitfield.h

@@ -10,6 +10,7 @@
 #include <hilti/ast/operators/common.h>
 #include <hilti/ast/types/any.h>
 #include <hilti/ast/types/bitfield.h>
+#include <hilti/ast/types/bool.h>
 #include <hilti/ast/types/integer.h>
 #include <hilti/ast/types/unknown.h>
 
@@ -75,4 +76,24 @@ right.
     }
 END_OPERATOR_CUSTOM_x
 
+BEGIN_OPERATOR_CUSTOM(bitfield, HasMember)
+    Type result(const hilti::node::Range<Expression>& /* ops */) const { return type::Bool(); }
+
+    bool isLhs() const { return false; }
+    auto priority() const { return hilti::operator_::Priority::Normal; }
+
+    const std::vector<Operand>& operands() const {
+        static std::vector<Operand> _operands =
+            {{{}, type::constant(type::Bitfield(type::Wildcard())), false, {}, "bitfield"},
+             {{}, type::Member(type::Wildcard()), false, {}, "<field>"}};
+        return _operands;
+    }
+
+    void validate(const expression::ResolvedOperator& i, operator_::position_t p) const {
+        detail::checkName(i.op0(), i.op1(), p.node);
+    }
+
+    std::string doc() const { return "Returns true if the bitfield's element has a value."; }
+END_OPERATOR_CUSTOM
+
 } // namespace hilti::operator_

hilti/toolchain/include/ast/types/bitfield.h

@@ -24,11 +24,23 @@ namespace bitfield {
 class Bits : public hilti::NodeBase {
 public:
     Bits() : NodeBase({ID("<no id>"), hilti::node::none}, Meta()) {}
+
     Bits(ID id, int lower, int upper, int field_width, std::optional<AttributeSet> attrs = {}, Meta m = Meta())
         : hilti::NodeBase(nodes(std::move(id),
                                 hilti::expression::Keyword::createDollarDollarDeclaration(
                                     hilti::type::UnsignedInteger(field_width)),
-                                hilti::type::auto_, std::move(attrs)),
+                                hilti::type::auto_, std::move(attrs), hilti::node::none),
+                          std::move(m)),
+          _lower(lower),
+          _upper(upper),
+          _field_width(field_width) {}
+
+    Bits(ID id, int lower, int upper, int field_width, std::optional<AttributeSet> attrs = {},
+         std::optional<Expression> ctor_value = {}, Meta m = Meta())
+        : hilti::NodeBase(nodes(std::move(id),
+                                hilti::expression::Keyword::createDollarDollarDeclaration(
+                                    hilti::type::UnsignedInteger(field_width)),
+                                hilti::type::auto_, std::move(attrs), std::move(ctor_value)),
                           std::move(m)),
           _lower(lower),
           _upper(upper),
@@ -39,6 +51,7 @@ class Bits : public hilti::NodeBase {
     auto upper() const { return _upper; }
     auto fieldWidth() const { return _field_width; }
     auto attributes() const { return children()[3].tryAs<AttributeSet>(); }
+    auto ctorValue() const { return children()[4].tryAs<Expression>(); }
 
     const Type& ddType() const { return children()[1].as<hilti::declaration::Expression>().expression().type(); }
     NodeRef ddRef() const { return NodeRef(children()[1]); }
@@ -55,10 +68,11 @@ class Bits : public hilti::NodeBase {
 
     void setAttributes(const AttributeSet& attrs) { children()[3] = attrs; }
     void setItemType(const Type& t) { children()[2] = t; }
+    void setCtorValue(const Expression& e) { children()[4] = e; }
 
     bool operator==(const Bits& other) const {
         return id() == other.id() && _lower == other._lower && _upper == other._upper &&
-               _field_width == other._field_width && itemType() == other.itemType();
+               _field_width == other._field_width && itemType() == other.itemType() && ctorValue() == other.ctorValue();
         // TODO: Attributes don't quite compare correctly, see e.g., spicy.types.bitfield.parse-enum failure
         // && attributes() == other.attributes();
     }
@@ -95,6 +109,13 @@ class Bitfield : public hilti::TypeBase,
     std::optional<int> bitsIndex(const ID& id) const;
     auto attributes() const { return children()[1].tryAs<AttributeSet>(); }
 
+    /**
+     * If at least one of the bits comes with a pre-defined value, this builds
+     * a bitfield ctor value that corresponds to all values defined by any of
+     * the bits. If none does, return nothing.
+     */
+    std::optional<Ctor> ctorValue() const;
+
     bool operator==(const Bitfield& other) const { return width() == other.width() && bits() == other.bits(); }
 
     /** Implements the `Type` interface. */

hilti/toolchain/src/ast/types/bitfield.cc

@@ -1,5 +1,6 @@
 // Copyright (c) 2020-2023 by the Zeek Project. See LICENSE for details.
 
+#include <hilti/ast/ctors/bitfield.h>
 #include <hilti/ast/types/bitfield.h>
 #include <hilti/ast/types/tuple.h>
 
@@ -22,3 +23,17 @@ std::optional<int> type::Bitfield::bitsIndex(const ID& id) const {
 
     return {};
 }
+
+std::optional<Ctor> type::Bitfield::ctorValue() const {
+    std::vector<ctor::bitfield::Bits> values;
+
+    for ( const auto& b : bits() ) {
+        if ( auto v = b.ctorValue() )
+            values.emplace_back(b.id(), *v, meta());
+    }
+
+    if ( ! values.empty() )
+        return ctor::Bitfield(std::move(values), *this, meta());
+    else
+        return {};
+}

hilti/toolchain/src/compiler/codegen/ctors.cc

@@ -23,6 +23,22 @@ struct Visitor : hilti::visitor::PreOrder<cxx::Expression, Visitor> {
 
     result_t operator()(const ctor::Address& n) { return fmt("::hilti::rt::Address(\"%s\")", n.value()); }
 
+    result_t operator()(const ctor::Bitfield& n) {
+        std::vector<cxx::Type> types;
+        std::vector<cxx::Expression> values;
+        for ( const auto& b : n.btype().bits(true) ) {
+            auto itype = cg->compile(n.btype().bits(b.id())->itemType(), codegen::TypeUsage::Storage);
+            types.emplace_back(itype);
+
+            if ( auto x = n.bits(b.id()) )
+                values.emplace_back(cg->compile(x->expression()));
+            else
+                values.emplace_back("std::nullopt");
+        }
+
+        return fmt("hilti::rt::Bitfield<%s>{std::make_tuple(%s)}", util::join(types, ", "), util::join(values, ", "));
+    }
+
     result_t operator()(const ctor::Bool& n) { return fmt("::hilti::rt::Bool(%s)", n.value() ? "true" : "false"); }
 
     result_t operator()(const ctor::Bytes& n) { return fmt("\"%s\"_b", util::escapeBytesForCxx(n.value())); }

hilti/toolchain/src/compiler/codegen/operators.cc

@@ -111,7 +111,14 @@ struct Visitor : hilti::visitor::PreOrder<cxx::Expression, Visitor> {
         auto id = n.op1().as<expression::Member>().id();
         auto elem = n.op0().type().as<type::Bitfield>().bitsIndex(id);
         assert(elem);
-        return {fmt("std::get<%u>(%s.value)", *elem, op0(n)), cxx::Side::RHS};
+        return {fmt("(hilti::rt::optional::value(std::get<%u>(%s.value)))", *elem, op0(n)), cxx::Side::RHS};
+    }
+
+    result_t operator()(const operator_::bitfield::HasMember& n) {
+        auto id = n.op1().as<expression::Member>().id();
+        auto elem = n.op0().type().as<type::Bitfield>().bitsIndex(id);
+        assert(elem);
+        return {fmt("std::get<%u>(%s.value).has_value()", *elem, op0(n)), cxx::Side::RHS};
     }
 
     /// bytes::Iterator

hilti/toolchain/src/compiler/coercion.cc

@@ -374,6 +374,42 @@ struct VisitorCtor : public visitor::PreOrder<std::optional<Ctor>, VisitorCtor>
             return ctor::Struct(std::move(nf), dst_, c.meta());
         }
 
+        if ( auto dtype = dst_.tryAs<type::Bitfield>() ) {
+            if ( ! dst_.typeID() )
+                // Wait for this to be resolved.
+                return {};
+
+            auto stype = c.type().as<type::Struct>();
+
+            std::set<ID> src_fields;
+            for ( const auto& f : stype.fields() )
+                src_fields.insert(f.id());
+
+            std::set<ID> dst_fields;
+            for ( const auto& f : dtype->bits() )
+                dst_fields.insert(f.id());
+
+            // Check for fields in ctor that type does not have.
+            if ( ! util::set_difference(src_fields, dst_fields).empty() )
+                return {};
+
+            // Coerce each field.
+            std::vector<ctor::bitfield::Bits> bits;
+
+            for ( const auto& sf : stype.fields() ) {
+                const auto& dbits = dtype->bits(sf.id());
+                const auto& se = c.field(sf.id());
+                assert(dbits && se);
+                if ( const auto& ne = hilti::coerceExpression(se->expression(), dbits->itemType(), style) )
+                    bits.emplace_back(sf.id(), *ne.coerced);
+                else
+                    // Cannot coerce.
+                    return {};
+            }
+
+            return ctor::Bitfield(std::move(bits), *dtype, c.meta());
+        }
+
         return {};
     }
 };