Allot word headers from data space vs allocate from host memory?

[SirWumpus]

Post4 currently allocates word headers from host memory (via C malloc() et. al) and are outside the of data space. A more traditional / historical Forth would allot the word headers from the data space (embedded system). @ruv do you have an opinion on this?

Originally I wanted to Post4 to arbitrarily grow the data space as words were added, link list of words each with its own portion of the data space needed to define that word and allotted space, so growing the data space would happen "organically". However this made using HERE a little tricky for which I had added >HERE to give an offset from the current word definition. Of course this means that data space is fragmented in host memory.

After experimenting with that for a long while, I switched to allocating a fixed size data space (option -m) and have word headers (still allocated) to point to their definitions consuming the data space. This is the current state and simplifies lots of the memory management and models traditional (historical?) 16-bit Forth systems. (If I ever want to make the data space grow or snapshot compiled images, I would have to add a level of indirection to handle when realloc() moves the memory block or loading a snapshot.)

So NOW I'm considering moving the word headers into the data space, which would be closer to historical design (I assume) and simplifies some word / memory management (I hope). Certainly would avoid host memory leaks (not that I currently have any).

Anyway @ruv do you (or others) have an opinion on this? Or is it no biggie as long as either system works?

[ruv]

A standard program does not depend on the memory layout used by the Forth system. So, it's up to the implementation what memory layout is better for it.

link list of words each with its own portion of the data space needed to define that word and allotted space,

NB: here shall return the same address before and after switching the compilation word list with set-current or definitions, among the word lists created with wordlist or vocabulary. So, we cannot associate own address range with a word list created with wordlist.

It means, if we want to switch a dictionary section (so here returns different addresses in different sections), we have two options: use a special word to switch sections, or use a specially created word lists.

[SirWumpus]

A standard program does not depend on the memory layout used by the Forth system. So, it's up to the implementation what memory layout is better for it.

I know the internal implementation doesn't matter to a standard program, though I think there are assumptions concerning memory that make some designs better than others.

link list of words each with its own portion of the data space needed to define that word and allotted space,

Don't think I expressed my prior design very well, a word header would be allocated, along with enough data space to hold that word's definition (realloc() to release unused spaced) until the next word is defined. The design worked fine in most cases and would work fine with wordlists. The problem I struggled with was how much space does a typical word need? If a word grows, the C realloc() can move the memory affecting memory addresses obtained from HERE. Its easy enough to shrink a memory block to release unused memory when a new word is started, but growing a word can get messy.

I have since moved away from that design (maybe 3 years ago) to slightly more historical design assuming a fixed size contiguous memory at startup, which will get chopped up as words are defined.

[ruv]

The problem I struggled with was how much space does a typical word need? If a word grows, the C realloc() can move the memory affecting memory addresses obtained from HERE.

I see. Typical words are small. But there may be not only words, but also program's data. So, you either provide large enough buffer for each word or data block, say, 1 MiB, or provide a command line argument to configure this size.

[SirWumpus]

I suppose, though I have -m option for overall memory at startup, not the same as having a max. word definition size that is reallocated to shrink at the start of the next word. I might have considered that long ago (~2007) and discarded it for some reason.

I think I was enticed by the idea I had read about some Forth systems being able to snapshot an image that could later be reloaded. Kinda cool idea, needs contiguous memory and a memory base pointer though; haven't reached that point, but still neat direction to consider.