Fixes for recent gcc and gforth

[bendlas]

About first patch: On recent gcc versions (presumably), <stdio.h> needs to be included before <readline/readline.h>

About second patch: Not sure how this worked before, but s>number? thinks - is a number. Thus I implemented a custom version, to check for this case specifically.

[mschuldt]

I can't come up with a case in which s>number? thinks - is a number, I'm running 7.9, maybe our gforth versions are different.
Can you give me the version you are running and an example that reproduces this problem?

this is what I'm seeing:

$ gforth
Gforth 0.7.9_20210218
...
s" -" s>number?  ok 3
.s <3> 0 0 0  ok 3

while trying to reproduce this I discovered that negative numbers are completely broken...

[bendlas]

Huh, interesting, I'm still on 0.7.3, where this seems to be a persistent issue:

% gforth
Gforth 0.7.3, Copyright (C) 1995-2008 Free Software Foundation, Inc.
Gforth comes with ABSOLUTELY NO WARRANTY; for details type `license'
Type `bye' to exit
s" -" s>number? .s <3> 0 0 -1  ok

According to https://www.gnu.org/software/gforth/, 0.7.3 is still the most recent version, but since you mentioned it, I was able to find https://github.com/forthy42/gforth, which has more recent versions, and has presumably had its behavior changed to what you observed. I'm pretty new to forth, do you have any insight into the relation there?

So maybe for the number issue, it would be sufficient to mention the requirement on more recent versions into the readme.

Have you had a chance to reproduce the include order issue, which I observed when trying to build forth.c with gcc 10?

If you want to pull just that, I can split up this PR, please let me know ...

[bendlas]

while trying to reproduce this I discovered that negative numbers are completely broken...

Yep, can confirm that for 0.7.3 as well.

[bendlas]

Found the fix for s" -" s>number?: forthy42/gforth@babed75

Sorry for wasting your time on this one.

[mschuldt]

You're welcome to split the PR, I've got no issue with the fix for the include order (although I can't replicate that either, seems to work both ways for me)

I build gforth from source (the forthy42 repo), because in my experience the version of gforth in the Ubuntu package manager is old, and the development progress is fast enough for it to matter.

Because most people are likely to be running older versions of gforth, I suggest we conditionally redefine s>number? when we detect this bug, place it near the top, something like this:

s" -" s>number? nip nip [if]
: s>number?   your version here
[then]

I do appreciate this PR, not a waste of time!

[mschuldt]

The new issue with negative numbers speaks to the general stability of this language, there is much work to do. At the point I stopped spending real time working on this I had decided that I needed to redesign the dictionary and memory models. This program grew somewhat organically, features got added without consideration of how that would impact the implementation of future features. It's all further complicated by the fact that Raillisp maps lisp to forth as closely as possible to take advantage of the speed that gforth and other implementations offer - and each implementation has it's own dictionary format. Backtraces are currently broken and disassembly is now less useful because gforth does not fully understand the way functions are compiled into the dictionary (at this point, I'm not sure I understand either).

Despite the lack of attention I've given this project recently, I do still feel its potential. Something that would let us program in lisp while taking advantage of the speed of gforth on the desktop and the ubiquity of fast assembly implementations of forth on micro-controllers. One language to rule them all...

Anyways, if you're interested in working on this more, I have some documentation and notes, somewhere, that I can pull together which might be helpful. You're of course welcome to play around with this in any way that you feel like, but if you want to add support for something like garbage collection or another forth implementation, I think it's going to require some deliberate (re)design and widespread changes. I'd be happy to collaborate about the design. Although, from what I've seen of the community, the most forthy thing for you to do is design your own from scratch...

[bendlas]

Because most people are likely to be running older versions of gforth, I suggest we conditionally redefine s>number? when we detect this bug, place it near the top, something like this:

Taking on workarounds like this is a judgement call, and I suspect that for a fledgling language, the scale may tip heavily towards staying minimal. But a note in the readme, or even a test case with fix instructions, would definitely have helped.

s" -" s>number? nip nip [if]
: s>number?   your version here
[then]

I do appreciate this PR, not a waste of time!

Thanks for the kind words! If we are going to do a polyfill for this, however, I'd still vote for backporting the fixed version, instead of pasting my quickfix ;-)

[bendlas]

The new issue with negative numbers speaks to the general stability of this language, there is much work to do. At the point I stopped spending real time working on this I had decided that I needed to redesign the dictionary and memory models. This program grew somewhat organically, features got added without consideration of how that would impact the implementation of future features.

You see, I'm looking at raillisp as a learning vehicle to get into forth. And as such, it's already doing an amazing job, showing me how to build a non-trivial system in forth. So, thank you for sharing it!

Regarding memory model: I don't know enough about forth yet, to offer any insight, though I did notice that forth implementations can differ rather drastically. I think I read a Chuck Moore quote, that a forth should be molded into its host platform, and that different platforms are supposed to have different forths.

It's all further complicated by the fact that Raillisp maps lisp to forth as closely as possible to take advantage of the speed that gforth and other implementations offer - and each implementation has it's own dictionary format. Backtraces are currently broken and disassembly is now less useful because gforth does not fully understand the way functions are compiled into the dictionary (at this point, I'm not sure I understand either).

Mapping closely to the platform at the cost of fixable inconveniences, is something that I strongly agree to. Though I may be biased coming from Clojure, where staying at platform level is the norm; and where, as you must know, no issue with platform stacktraces has been being voiced by any member of the community, ever ;-)

Though I think the issue you're describing runs deeper than "fixable": What's the platform to map to? I understand, that ANS wants to fill that role, but it seems to be unpopular with embedded forths. Any idea why? Is it not modular enough? Inefficient?

To me, as a newbie, the answer to "What platform am I currently on, and how can I get my toys here?", within forth seems to boil down to: "Whatever implementation I can get on my hardware, and whatever portability layer I'm accruing".

Then again, there is an inherent tension between portability and efficiency, and I came to forth for erring in the side of efficiency.

Despite the lack of attention I've given this project recently, I do still feel its potential. Something that would let us program in lisp while taking advantage of the speed of gforth on the desktop and the ubiquity of fast assembly implementations of forth on micro-controllers. One language to rule them all...

Yep, microcontrollers are a use case that's sorely overlooked by most language design. I feel like ferret is so close, and I can easily see it grow more sophisticated GC, defprotocol, read-cond, and all kinds of nifty sourcery. But what it probably won't be able to do is take a gcc with it to my arduino. So ... interpreter or no repl. I resent that. And actually, I resent having a C compiler in my bootstraps at all. Those are just more gigabytes of code, I'll never read.

Anyways, if you're interested in working on this more, I have some documentation and notes, somewhere, that I can pull together which might be helpful.

Well, at least I'd be interested in helping you publish them along with the project, by proof reading them, and maybe offering what perspective I can.

You're of course welcome to play around with this in any way that you feel like, but if you want to add support for something like garbage collection or another forth implementation, I think it's going to require some deliberate (re)design and widespread changes. I'd be happy to collaborate about the design. Although, from what I've seen of the community, the most forthy thing for you to do is design your own from scratch...

I'd definitely be interested in your lessons learned from the project so far, and your vision for its futures.

I have noticed the distinct lack of free in the codebase, when cons is just based on malloc ;) but I'm not planning on just plunging in and adding a GC. I mean, I have checked, but unfortunately, Anton Ertl's GC is conservative (i.e. non-compacting), and MPS is 1.4 MB.

Regarding porting to another forth: Right now, I still have a hard time figuring out which one that would be ... the best candidate I found so far, is circleforth.

But what I'm most concerned with right now, from a language design POV, is method dispatch: ClojureScript has proven, that you can bootstrap a production ready lisp from defprotocol/deftype, also being on an OO platform, this is the main interop producer and GC interface. It seems crucial to me to have an answer for this in any new language. So far, I've found a couple of libraries for generating dispatch tables, as well as gforth's object package(s) obviously.

Designing my own is certainly something I can enjoy, but OTOH, I already crossed that off my list when I wrote an interpreter for textbook lambda calculus. Still have to get around to publishing it, one day ..

And as I said I'm pretty new to forth, so for now I'm just grateful for any guidance and collaboration, I can find.