Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: extern records cannot have a different ordering of fields #26235

Open
jabraham17 opened this issue Nov 14, 2024 · 1 comment
Open

[Bug]: extern records cannot have a different ordering of fields #26235

jabraham17 opened this issue Nov 14, 2024 · 1 comment

Comments

@jabraham17
Copy link
Member

According to the spec on extern records, an extern record can specify fields in a differen't order than the actual struct definition.

From https://chapel-lang.org/docs/main/language/spec/interoperability.html#referring-to-external-c-structs-and-unions:

Within the Chapel declaration, some or all of the fields from the C structure or union may be omitted. The order of these fields need not match the order they were specified within the C code. Any fields that are not specified (or that cannot be specified because there is no equivalent Chapel type) cannot be referenced within the Chapel code. Some effort is made to preserve the values of the undefined fields when copying these structs but Chapel cannot guarantee the contents or memory story of fields of which it has no knowledge.

However, when using CHPL_TARGET_COMPILER=llvm with --verify, this is not true.

Given the following Chapel code:

require "R.h";
extern record R {
  var x: int(32);
  var y: int(32);
}
var r: R;
writeln(r);

And associated C code in R.h:

typedef struct {
  int y;
  int x;
} R;

Compiling this as chpl --target-compiler=llvm --verify foo.chpl fails with the following

Offsets must be increasing!
  %61 = load i32, ptr %60, align 4, !tbaa !542
!543 = !{!"R", !7, i64 4, !7, i64 0}
internal error: COD-CG--BOL-3129 chpl version 2.2.0

Internal errors indicate a bug in the Chapel compiler,
and we're sorry for the hassle.  We would appreciate your reporting this bug --
please see https://chapel-lang.org/bugs.html for instructions.

However, using --target-compiler=clang compiles fine. This is also fine without --verify

Weirdly, using --comm=gasnet and --target-compiler=llvm does work fine. So this only seems to be a problem with CHPL_COMM=none and CHPL_TARGET_COMPILER=llvm

Failing chplenv:

CHPL_TARGET_PLATFORM: darwin
CHPL_TARGET_COMPILER: llvm
CHPL_TARGET_ARCH: arm64
CHPL_TARGET_CPU: native
CHPL_LOCALE_MODEL: flat
CHPL_COMM: none
CHPL_TASKS: qthreads
CHPL_LAUNCHER: none
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc +
CHPL_ATOMICS: cstdlib
CHPL_GMP: system +
CHPL_HWLOC: system +
CHPL_RE2: bundled +
CHPL_LLVM: system
CHPL_AUX_FILESYS: none

Passing chplenv (using C backend):

CHPL_TARGET_PLATFORM: darwin
CHPL_TARGET_COMPILER: clang *
CHPL_TARGET_ARCH: arm64
CHPL_TARGET_CPU: native
CHPL_LOCALE_MODEL: flat
CHPL_COMM: none
CHPL_TASKS: qthreads
CHPL_LAUNCHER: none
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc +
CHPL_ATOMICS: cstdlib
CHPL_GMP: system +
CHPL_HWLOC: system +
CHPL_RE2: bundled +
CHPL_LLVM: system
CHPL_AUX_FILESYS: none

Passing chplenv (using LLVM and gasnet)

CHPL_TARGET_PLATFORM: darwin
CHPL_TARGET_COMPILER: llvm
CHPL_TARGET_ARCH: arm64
CHPL_TARGET_CPU: native *
CHPL_LOCALE_MODEL: flat
CHPL_COMM: gasnet *
  CHPL_COMM_SUBSTRATE: udp
  CHPL_GASNET_SEGMENT: everything
CHPL_TASKS: qthreads
CHPL_LAUNCHER: amudprun
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc
CHPL_ATOMICS: cstdlib
  CHPL_NETWORK_ATOMICS: none
CHPL_GMP: system *
CHPL_HWLOC: system *
CHPL_RE2: bundled *
CHPL_LLVM: system
CHPL_AUX_FILESYS: none
@bradcray
Copy link
Member

The spec text dates to when the C back-end was dominant, possibly even before the LLVM back-end existed. I wasn't aware that LLVM was sensitive to this until now.

As to why llvm + gasnet works… we don't fall back to the C back-end in that configuration for some reason do we? (the only semi-plausible explanation I could come up with).

jabraham17 added a commit that referenced this issue Nov 21, 2024
Adds a new package module, `Python`, which supports calling Python code
from Chapel.

For example, the following Chapel progam use the python interface to
`BeautifulSoup` to parse HTML

```chapel
import Python;
use URL;
use List;

proc main() {

  const url = 'https://chapel-lang.org/docs/main/language/spec/interoperability.html';
  var htmlReader = openUrlReader(url);
  var html = htmlReader.readAll(string);

  var interp = new Python.Interpreter();
  var mod = new Python.Module(interp, "bs4");

  var cls = new Python.Class(mod, "BeautifulSoup");
  var soup = cls(html, 'html.parser');

  var res: list(owned Python.ClassObject?);
  res = soup.callMethod(res.type, "find_all", "h3");
  for c in res {
    writeln(c!.getAttr(string, "text"));
  }
}
```

As an another example, this simple program compiles an runs a Python
lambda from Chapel code
```chapel
import Python;

config const n = 10;
config const func = "lambda x,: x + 1 if x % 2 != 0 else x";

proc apply(interp: borrowed, type t, arr, l) {
  var lambdaFunc = new Python.Function(interp, l);
  var res: [arr.domain] t;
  for i in arr.domain {
    res(i) = lambdaFunc(t, arr(i));
  }
  return res;
}
proc main() {
  var interp = new Python.Interpreter();

  var data: [1..#n] int = 1..#n;
  writeln(" data: ", data);

  var res = apply(interp, int, data, func);
  writeln("res: ", res);
}

```

Future work:
- Convert classes to a hierarchy under `Value` so that more
implementation can be shared and the Chapel interface can better imitate
python
- Add special support for common python types as sub-classes of value,
to avoid round tripping values through chapel in many cases
- Support common operators like add. These can be called today as dunder
methods (`.call("__add__", ...)`), but it would be nice to support
native operators like `+`
- Add the ability to compile arbitrary chapel strings, beyond just
lambdas
- Setup python to use Chapel stdout/stderr
- Add custom adapters for Chapel arrays to allow python functions to
operate on Chapel arrays, without copying
- Support python context managers as Chapel context managers
- Fully support Python sets, Python dictionaries, Chapel sets, Chapel
maps, Chapel associative arrays, and Chapel associative domains
- Make sure all python objects are properly reference counted
- Resolve TODO with LLVM IR verification:
#26235


[Reviewed by @DanilaFe]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants