cgo: Nowhere to specify C/C++ source files

[mckelvin]

1. What version of Go are you using (go version)?

   go version go1.5 darwin/amd64

2. What operating system and processor architecture are you using?

   Darwin PANs-MacBook-Pro.local 15.0.0 Darwin Kernel Version 15.0.0: Wed Aug 26 16:57:32 PDT 2015; root:xnu-3247.1.106~1/RELEASE_X86_64 x86_64

3. What did you do?

   Support there's a go project based on cgo. Both of the C/C++ code are in the same repo. All c header files are listed in the include directory and c source files listed in the src directory. An ideal way to place the golang code is put it in a golibmc directory:

   .
   ├── golibmc
   │   └── golibmc.go
   ├── include
   │   ├── hashkit
   │   │   └── crc32.h
   │   └── libmc.h
   └── src
       ├── hashkit
       │   └── crc32.c
       └── libmc.c
   

4. What did you expect to see?

   I expected there was a directive (e.g.: #cgo SOURCES ../src/*.c ../src/hashkit/*.c) for me to specify all C/C++ source files need to be compiled. But cgo can only find source files in the same directory where the .go file exists.

5. What did you see instead?

   I cannot find the expected directive mentioned in 4, then I have to re-organize the source tree in a smelly way:

   .
   ├── golibmc -> src
   ├── include
   │   ├── hashkit
   │   │   └── crc32.h
   │   └── libmc.h
   └── src
       ├── golibmc.go
       ├── hashkit
       ├── hashkit_crc32.c
       └── libmc.c
   

---

An example in the wild world is http://github.com/douban/libmc

[adg]

The go tool isn't a tool for compiling C code. It may compile .c files as part of a Go package that uses cgo, but at root that's still Go code.

Am I right in interpreting your request? Do you want the go tool to understand and build C libraries that contain no Go code?

[mckelvin]

@adg If golang can't do such thing, I have to build the c part manually, maybe something like:

mkdir build
cd build
cmake ..
make install # suppose /usr/lib/libmc.a is generated
cd ../golibmc
go build

instead of:

go build

Python has the ability to do such work, so I wish golang can also do.

[adg]

Executing arbitrary code is something the go tool will never do as part of a go build or go install. This is by design.

You could use a go generate directive to achieve this, though.

[mckelvin]

The purpose of the PR is not to Executing arbitrary code but to make it configurable for cgo to work on any source tree. It will also be no painful for users using the library, they don't need to care about the compilation any more. If they want to use the pkg, just go install like all other pkgs.

Please consider again.

[adg]

configurable for cgo to work on any source tree.

I believe that if you follow this goal through to its logical conclusion then you end up with arbitrary code execution. Please correct me if I'm wrong.

[bradfitz]

Go is not a build system. Check out http://bazel.io/

[mckelvin]

@adg Would you please explain the details? I don't think this will bring any extra security issues(arbitrary code execution).

Given the feature, all C/C++ source files are specified by Peter who write the golang code. If he want to execute any evil C++ code in the go project, he can do it without the proposed feature. One concern is: Peter should not add files outside the project (#cgo SOURCE_FILES /opt/foo/bar.cpp). If someone deliberately add evil code to the file, I think Peter rather than the go project should be blame. Anyway this is a rare case I think.
I'm not sure if I got you idea.

---

@bradfitz That's true, python is not a build system neither. In my view, go is to python as cgo is to cython. Cython can do such stuff while cgo can't. Go is not a build system but actually it(cgo) does, though not doing very tell. Why not trying to make it better.

Libmc, As a open-source library (not a standlone program), I wish there's no pain for users to use it. Peter can install the library pip install libmc just like a normal python library, without worrying about compiling the C/C++ part. Cython did the job for Peter but these's no need for Peter to know anything about Cython. Bazel is great but I don't want it be a dependency of libmc.(If Peter want to use libmc, he has to install Bazel first. -- I think that's terrible.)

[minux]

go build is designed to build Go packages, not C/C++ libraries. If a Go package happens to use cgo, go build will build the C/C++/Obj-C source code in the same directory as the Go package (this is necessary because not all cgo code can be put in the cgo preamble in Go source files.) However, go build will not attempt to build arbitrary C/C++ source code outside the Go package it's building.

[mckelvin]

go build will build the C/C++/Obj-C source code in the same directory as the Go package

Ideally, I don't think C/C++ source code should be treated as part of a go pkg, but the library built using source files is. Different C project has different source tree. You can't demand all existed C* library to be organised the way golang like: Put all source file in a single directory , not even a sub-directory!

[minux]

Different C/C++ library have different build procedures, for example, some require running a configure script before make, should go build support building those? The C/C++ library that the Go package interfaces with should be built separately.

[mckelvin]

Different C/C++ library have different build procedures, for example,
some require running a configure script before make, should go build
support building those?

should go build support building those? You can build libmc using cmake, that does not mean you have to use cmake. You can even use autotools to to build it if you like. These are just build tools. c go is also shipped with a build tool, and the point is, the build tool is not working well.

The C/C++ library that the Go package interfaces with should be built
separately.

ANSWER is in : #12953 (comment) , in addition:

That's true and is the common case. In such way, I have to release the C++ part as a separate library, and resolve version problems between the Python/Golang wrapper and the base library. If the base library is upgraded, programs running under the old version of the Python/Golang wrapper may break. That's terrible. Now that I can build source files from the scratch in Python and Golang, why not?

[ianlancetaylor]

Use a Makefile that runs "go build".

[minux]

Or add a go generate rule that create stub C files that simply include the actual files.