Introduce MemoryPointer #2196

Kenzzer · 2024-07-27T12:22:47Z

This whole PR is still an heavy draft, nothing has been field tested.

Introduction

As of today no immediate solution to PseudoAddress has been explored. Only third-party solutions like NotnHeavy/SM-Address64 & skial-com/port64 have been made, but no major adoption so far. And while those solutions can work, I believe the friction for plugin authors is quite high. Now having to worry about the server architecture, and the confusing concept of treating ptrs as array of int/any.

Explanation

Given that there's no timeline as for when enum struct natives will ever be a thing. I propose instead to expose a new handle type that shall act as an opaque pointer towards abitrary memory regions. The underlying C++ class IMemoryPointer contained by the handle type is exposed as a public header. Allowing third-party extensions to wrap their pointers into it, no longer having to rely and depend on cell_t capacity to store the pointer's value on 32 bits.

IMemoryPointer shall become the defacto standard to pass abitrary pointers around for plugins & extensions.

List of natives

MemoryPointer.MemoryPointer(int size)
Allocates an abitrary amount of memory.

MemoryPointer.Load
Similar to LoadFromAddress native.

MemoryPointer.Store
Similar to StoreToAddress native.

MemoryPointer.Offset
Offsets the base pointer, and creates a new MemoryPointer handle.

MemoryPointer.LoadMemoryPointer
Interprets the loaded data as an address and wraps it into a new MemoryPointer handle.

MemoryPointer.StoreMemoryPointer
Retrieves the pointer's value wrapped inside the MemoryPointer handle, and stores it.

MemoryPointer.LoadEntityFromHandle
Similar to LoadEntityFromHandleAddress native.

MemoryPointer.StoreEntityToHandle
Similar to StoreEntityToHandleAddress native.

GameData.GetMemSigEx
Similar to GameData.GetMemSig but wraps the pointer into a MemoryPointer handle.

GameData.GetAddressEx
Similar to GameData.GetAddress but wraps the pointer into a MemoryPointer handle.

GetEntityMemoryPointer
Similar to GetEntityAddress but wraps the entity pointer into a MemoryPointer handle.

SDKCall_MemoryPointer
New sdkcall type to allow member function calls from the ptr contained by a MemoryPointer handle.

SDKType_MemoryPointer
New sdkparam type, to retrieve the abitrary ptr contained inside a MemoryPointer handle. INVALID_HANDLE / null are allowed values to represent nullptr, if allow null flag was passed.

PrepSDKCall_SetAddressFromMemoryPointer
Similar to PrepSDKCall_SetAddress native.

Ultimately this PR aims to supersede #2159 & #2112

NotnHeavy · 2024-07-27T12:58:51Z

I think this is a decent work-around, considering the current cell_t limitations of SourcePawn and the numerous extensions that currently exist as a result of 64-bit address workarounds - including mine (which I hoped would've been the standard until an official SM native of some kind came around).

However, I feel like it's still missing out on some features, such as being able to wrap around existing addresses rather than only being able to use the handle to allocate a new block of memory.

Perhaps natives for converting between pseudo addresses and MemoryPointer would also be ideal?

In the end as well this obviously creates a new issue of having to delete your MemoryPointer objects every time you use them as well, but I suppose active use of it wouldn't necessarily be encouraged.

KaelaSavia · 2024-07-28T13:09:04Z

In the end as well this obviously creates a new issue of having to delete your MemoryPointer objects every time you use them as well, but I suppose active use of it wouldn't necessarily be encouraged.

Perhaps it could be possible to have argument in MemoryPointer constructor that would decide whether pointer is freed on plugin end, or requires manual freeing.

Also, would it be possible for "reasons" to have those two natives implemented?

Convert two 32-bit ints (array) into MemoryPointer
Convert MemoryPointer into array of two 32-bits

Other than that, current solution is pretty decent, people should be able to easily adapt. Although in future also having wiki page just like we had for transistional syntax will make things even smoother in that regard!

Kenzzer · 2024-07-29T08:40:28Z

Perhaps it could be possible to have argument in MemoryPointer constructor that would decide whether pointer is freed on plugin end, or requires manual freeing.

Purposefully leaking memory isn't something I'd advise. The standard in Sourcemod is that all plugin-instanced ressources are freed on unloading, this shouldn't be an exception. If a plugin or extension wants to keep a memorypointer around for longer, they're free to create their own and copy the data over.

FortyTwoFortyTwo · 2024-07-29T20:15:04Z

plugins/include/sourcemod.inc

+	//
+	// @param name          Name of the property to find.
+	// @return              New MemoryPointer handle containing the address calculated on success, or null on failure.
+	public native Address GetAddressEx(const char[] name);


Missed here and GetMemSigEx having return field as MemoryPointer?

Headline

Most is nit picky or have already been pointed out

core/logic/MemoryPointer.cpp

core/logic/smn_core.cpp

core/logic/smn_gameconfigs.cpp

plugins/include/sourcemod.inc

NotnHeavy · 2024-08-04T15:24:04Z

There's issues with using SDKType_MemoryPointer. These are addressed in a PR I made to Kenzzer's PR.

Kenzzer · 2024-08-19T11:52:05Z

We should be good to merge this time. Though maybe we should wait for @psychonic to split branches, as mentioned on discord. Stabilise 1.12, start 1.13 and merge this over there.

NotnHeavy · 2024-08-22T14:28:14Z

The thing with 1.12 at the moment is that dvander is currently refactoring the SourcePawn compiler completely, to the point that int64 hypothetically may be introduced. I'm not sure if it will make sense to start 1.13 now.

dvander · 2024-08-22T15:18:49Z

It'll be a while. It's a slog if there ever was a slog. I'd branch if you need to.On Aug 22, 2024 10:28 AM, NotnHeavy ***@***.***> wrote: The thing with 1.12 at the moment is that dvander is currently refactoring the SourcePawn compiler completely, to the point that int64 hypothetically may be introduced. I'm not sure if it will make sense to start 1.13 now. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

Kenzzer · 2024-08-22T21:11:06Z

The thing with 1.12 at the moment is that dvander is currently refactoring the SourcePawn compiler completely, to the point that int64 hypothetically may be introduced. I'm not sure if it will make sense to start 1.13 now.

Branching makes the most sense, because we can make breaking changes on the dev branch. This was suggested because int64 on SP is a possibility. And when it is, we can destroy MemoryPointer without really breaking bcompat, because its dev branch.

NotnHeavy · 2024-08-24T14:18:19Z

Ah, I suppose I haven't really thought about that. That makes more sense then.

Kenzzer

Leaving a self-review; before I forget again what I was missing on this PR.
I plan on fixing all of this later this week.

Kenzzer · 2024-10-21T09:09:32Z

extensions/sdktools/vdecoder.cpp

+			}
+
+			HandleError err = HandleError_None;
+			Handle_t hndl = handlesys->CreateHandle(g_MemPtrHandle, new ForeignMemoryPointer(ptr), pContext->GetIdentity(), myself->GetIdentity(), &err);


Don't leak memory.

Kenzzer · 2024-10-21T09:11:11Z

extensions/sdktools/vdecoder.cpp

+					*(void **)buffer = nullptr;
+					return Data_Okay;
+				}
+				pContext->ThrowNativeError("Null/Invalid Handle MemoryPointer isn't allowed");


Remove "invalid", we can never branch here with an invalid handle. And it'd be weird to imply invalid handles can be used as "nullptr".

Kenzzer · 2024-10-21T09:16:30Z

plugins/include/sourcemod.inc

+	// @param size          Offset from base pointer in bytes.
+	// @return              New Handle to a memory pointer.
+	public native MemoryPointer Offset(int offset);
+};


Missing equal native; i.e if two MemoryPointer handles hold the same pointer value.

NotnHeavy · 2024-10-30T19:43:59Z

I decided to work on these for you. My code should be fine, although I can't test on my laptop because both my laptop and internet are horrible in the place I am currently staying at.

Kenzzer · 2024-12-15T23:56:17Z

Closing in favor of #2226 . The consensus internally seems to be that we avoid introducing a new handle type.

Will re-open if cited PR fails.

Introduce MemoryPointer

3310259

Kenzzer added 2 commits July 27, 2024 19:49

complete the natives

b02445b

Fix linux compilation

9a4aa49

FortyTwoFortyTwo reviewed Jul 29, 2024

View reviewed changes

psychonic approved these changes Jul 30, 2024

View reviewed changes

Headline requested changes Jul 30, 2024

View reviewed changes

NotnHeavy mentioned this pull request Jul 30, 2024

WIP: Generate an IR (aka int64 part 1/4). alliedmodders/sourcepawn#983

Open

Kenzzer mentioned this pull request Jul 31, 2024

Add new address natives #2112

Closed

PR review changes

0949fc9

Kenzzer requested a review from Headline August 3, 2024 12:39

Kenzzer marked this pull request as ready for review August 3, 2024 12:40

Headline approved these changes Aug 3, 2024

View reviewed changes

Fix SDKType_MemoryPointer issues

0a8fab4

Kenzzer commented Oct 21, 2024

View reviewed changes

Kenzzer mentioned this pull request Oct 29, 2024

Setup enum-struct address #2213

Draft

Kenzzer closed this Dec 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce MemoryPointer #2196

Introduce MemoryPointer #2196

Kenzzer commented Jul 27, 2024 •

edited

Loading

NotnHeavy commented Jul 27, 2024

KaelaSavia commented Jul 28, 2024 •

edited

Loading

Kenzzer commented Jul 29, 2024

FortyTwoFortyTwo Jul 29, 2024

Headline left a comment

NotnHeavy commented Aug 4, 2024

Kenzzer commented Aug 19, 2024 •

edited

Loading

NotnHeavy commented Aug 22, 2024

dvander commented Aug 22, 2024 via email

Kenzzer commented Aug 22, 2024

NotnHeavy commented Aug 24, 2024 •

edited

Loading

Kenzzer left a comment

Kenzzer Oct 21, 2024

Kenzzer Oct 21, 2024

Kenzzer Oct 21, 2024

NotnHeavy commented Oct 30, 2024

Kenzzer commented Dec 15, 2024

Introduce MemoryPointer #2196

Introduce MemoryPointer #2196

Conversation

Kenzzer commented Jul 27, 2024 • edited Loading

Introduction

Explanation

List of natives

NotnHeavy commented Jul 27, 2024

KaelaSavia commented Jul 28, 2024 • edited Loading

Kenzzer commented Jul 29, 2024

FortyTwoFortyTwo Jul 29, 2024

Choose a reason for hiding this comment

Headline left a comment

Choose a reason for hiding this comment

NotnHeavy commented Aug 4, 2024

Kenzzer commented Aug 19, 2024 • edited Loading

NotnHeavy commented Aug 22, 2024

dvander commented Aug 22, 2024 via email

Kenzzer commented Aug 22, 2024

NotnHeavy commented Aug 24, 2024 • edited Loading

Kenzzer left a comment

Choose a reason for hiding this comment

Kenzzer Oct 21, 2024

Choose a reason for hiding this comment

Kenzzer Oct 21, 2024

Choose a reason for hiding this comment

Kenzzer Oct 21, 2024

Choose a reason for hiding this comment

NotnHeavy commented Oct 30, 2024

Kenzzer commented Dec 15, 2024

Kenzzer commented Jul 27, 2024 •

edited

Loading

KaelaSavia commented Jul 28, 2024 •

edited

Loading

Kenzzer commented Aug 19, 2024 •

edited

Loading

NotnHeavy commented Aug 24, 2024 •

edited

Loading