fix: types with protocol qualified are not built-ins #451

t0rr3sp3dr0 · 2024-04-22T07:36:57Z

I made a mistake in #443 and the code detects things like id <OS_dispatch_queue> as built-ins. While id is a built-in, OS_dispatch_queue is not.

This patch addresses the problem by looking for built-ins only when we have an unqualified type. When we have a qualified type, we split the type in two and check both parts separately:

ipsw/internal/commands/macho/objc.go

Lines 1014 to 1017 in 7c77867

    
           before, after, _ := strings.Cut(typ, "<") 
        
           o.fillImportsForType(before, className, classNames, protoNames, imp) 
        
           o.fillImportsForType("<"+after, className, classNames, protoNames, imp)

blacktop · 2024-04-23T02:26:05Z

Not sure this is a good test, because it's Foundation, but: (check out)

Foundation/NSAKDeserializerStream.h
Foundation/NSKeyedUnarchiver.h
Foundation/NSOperation.h
Foundation/NSOperationQueue.h
Foundation/NSXPCConnection.h
Foundation/NSXPCListener.h

Might be time to start using regep vs. strings.Contains

t0rr3sp3dr0 · 2024-04-23T09:37:02Z

Okay, we have two problems:

We have a mix of Objective-C encoded types like @"NSObject<OS_dispatch_group>" and regular types like NSString *. Trying to parse both at the same time is not pretty straightforward.
There is no exhaustive grammar for Objective-C type encoding besides the compiler implementation itself. I couldn't find a place that documents A or %, for example. The only thing we can 100% rely on is 14 thousand lines of C++.

t0rr3sp3dr0 · 2024-04-23T09:42:58Z

I believe to have addressed the problems you listed.

But, unrelated to this PR, I found that we are not representing some more obscure types properly.

ipsw Header:

@interface NSDecimalNumber : NSNumber {
    /* instance variables */
    unsigned int x :8 _exponent;
    unsigned int x :4 _length;
    unsigned int x :1 _isNegative;
    unsigned int x :1 _isCompact;
    unsigned int x :1 _reserved;
    unsigned int x :1 _hasExternalRefCount;
    unsigned int x :16 _refs;
    unsigned short * _mantissa;
}

Official Apple SDK Header:

@interface NSDecimalNumber : NSNumber {
@private
    signed   int _exponent:8;
    unsigned int _length:4;
    unsigned int _isNegative:1;
    unsigned int _isCompact:1;
    unsigned int _reserved:1;
    unsigned int _hasExternalRefCount:1;
    unsigned int _refs:16;
    unsigned short _mantissa[];
}

t0rr3sp3dr0 · 2024-04-23T10:22:59Z

Another example.

ipsw:

@interface _NSObserverList : NSObject {
    /* instance variables */
    id _owner;
    id _observers;
    atomic unsigned int _observerCount;
    struct os_unfair_lock_s { unsigned int _os_unfair_lock_opaque; } _observersLock;
}

Expected:

@interface _NSObserverList : NSObject {
    /* instance variables */
    id _owner;
    id _observers;
    _Atomic unsigned int _observerCount;
    struct os_unfair_lock_s { unsigned int _os_unfair_lock_opaque; } _observersLock;
}

blacktop · 2024-04-23T16:29:07Z

Another example.

ipsw:

@interface _NSObserverList : NSObject {
    /* instance variables */
    id _owner;
    id _observers;
    atomic unsigned int _observerCount;
    struct os_unfair_lock_s { unsigned int _os_unfair_lock_opaque; } _observersLock;
}

Expected:

@interface _NSObserverList : NSObject {
    /* instance variables */
    id _owner;
    id _observers;
    _Atomic unsigned int _observerCount;
    struct os_unfair_lock_s { unsigned int _os_unfair_lock_opaque; } _observersLock;
}

Is atomic not a valid alias for _Atomic ?

blacktop · 2024-04-23T18:39:46Z

I believe to have addressed the problems you listed.

But, unrelated to this PR, I found that we are not representing some more obscure types properly.

ipsw Header:

@interface NSDecimalNumber : NSNumber {
    /* instance variables */
    unsigned int x :8 _exponent;
    unsigned int x :4 _length;
    unsigned int x :1 _isNegative;
    unsigned int x :1 _isCompact;
    unsigned int x :1 _reserved;
    unsigned int x :1 _hasExternalRefCount;
    unsigned int x :16 _refs;
    unsigned short * _mantissa;
}

Official Apple SDK Header:

@interface NSDecimalNumber : NSNumber {
@private
    signed   int _exponent:8;
    unsigned int _length:4;
    unsigned int _isNegative:1;
    unsigned int _isCompact:1;
    unsigned int _reserved:1;
    unsigned int _hasExternalRefCount:1;
    unsigned int _refs:16;
    unsigned short _mantissa[];
}

I believed this is fixed in go-macho

blacktop · 2024-04-23T18:40:14Z

Another example.

ipsw:

@interface _NSObserverList : NSObject {
    /* instance variables */
    id _owner;
    id _observers;
    atomic unsigned int _observerCount;
    struct os_unfair_lock_s { unsigned int _os_unfair_lock_opaque; } _observersLock;
}

Expected:

@interface _NSObserverList : NSObject {
    /* instance variables */
    id _owner;
    id _observers;
    _Atomic unsigned int _observerCount;
    struct os_unfair_lock_s { unsigned int _os_unfair_lock_opaque; } _observersLock;
}

replaced atomic w/ _Atomic in go-macho

t0rr3sp3dr0 · 2024-04-23T18:55:24Z

_Atomic and atomic are actually different. The former is a C type qualifier and it is used with C types, while the latter is a Objective-C property attribute and cannot be used outside property definitions.

t0rr3sp3dr0 · 2024-04-23T18:55:46Z

Just resolved the conflicts, PTAL

blacktop · 2024-04-23T19:35:32Z

Just resolved the conflicts, PTAL

Can you also rebase from master to pull in the go-macho fixes? I'm still seeing a lot of weird things in those Foundation headers

blacktop · 2024-04-23T19:45:31Z

Foundation/NSConcreteFileHandle.h

@class out unsigned long long;

blacktop · 2024-04-23T19:47:27Z

Foundation/NSOperationQueue.h

@class __backingQueue"@"NSObject;

blacktop · 2024-04-23T19:48:29Z

I forgot to ask about this but in Foundation/NSProgressRegistrar-Protocol.h this PR removes: @protocol OS_dispatch_queue;

Maybe it doesn't make sense to have forward declarations of protocols in protocol headers?

blacktop · 2024-04-23T19:49:28Z

Foundation/NSXPCConnection.h

@class remote"@"OS_xpc_remote_connection");

and Foundation/NSXPCListener.h is similar

t0rr3sp3dr0 · 2024-04-27T08:36:54Z

I’ll try to address these problems this weekend.

t0rr3sp3dr0 · 2024-04-29T00:28:42Z

I'll have to fix some stuff in go-macho to make this work properly. I'll send the PR shortly.

blacktop · 2024-04-29T01:57:38Z

I'll have to fix some stuff in go-macho to make this work properly. I'll send the PR shortly.

merged your go-macho PR and cut a release 👍

t0rr3sp3dr0 · 2024-04-29T03:20:40Z

Okay, just tested against Foundation and the only thing weird I found was @class by copy id; in NSSpellServer and @class undefined; in multiple places. But that's because of go-macho as well.

I'm reviewing these maps and will submit a PR there. Beyond spaces in bycopy, byref, and oneway, there are other problems like int128 instead of __int128 and BOOL instead of _Bool. But I'm having a hard time validating these types:

	"z": "size_t",
	"Z": "int32",
	"w": "wchar_t",
	"%": "NXAtom",
	"!":  "vector",
	"Vv": "void",

I don't know if zZw%!V are legacy types but, with Xcode 15.3, I get unsupported type encoding spec when using them with [[NSMethodSignature signatureWithObjCTypes:s] debugDescription]. Do you know anything about them or have an example where they are used?

blacktop · 2024-04-29T05:30:55Z

Okay, just tested against Foundation and the only thing weird I found was @class by copy id; in NSSpellServer and @class undefined; in multiple places. But that's because of go-macho as well.

I'm reviewing these maps and will submit a PR there. Beyond spaces in bycopy, byref, and oneway, there are other problems like int128 instead of __int128 and BOOL instead of _Bool. But I'm having a hard time validating these types:
	"z": "size_t",
	"Z": "int32",
	"w": "wchar_t",
	"%": "NXAtom",
	"!":  "vector",
	"Vv": "void",
I don't know if zZw%!V are legacy types but, with Xcode 15.3, I get unsupported type encoding spec when using them with [[NSMethodSignature signatureWithObjCTypes:s] debugDescription]. Do you know anything about them or have an example where they are used?

I can't remember where I saw some of those originally, they were added 3-4yrs ago.

Do you know how 'gnu register' should be represented?

blacktop · 2024-04-29T05:33:20Z

https://github.com/apple-oss-distributions/objc4/blob/196363c165b175ed925ef6b9b99f558717923c47/runtime/runtime.h

! and % are real though

t0rr3sp3dr0 · 2024-04-29T06:07:21Z

Take a look at blacktop/go-macho#52. I've removed Vv from there (because it should be oneway void) and left the other types untouched.

blacktop · 2024-04-29T06:10:54Z

I pushed some fixes including some that you mentioned to go-macho and the output is looking pretty good to me now 👍

t0rr3sp3dr0 · 2024-04-29T06:12:02Z

About %, this is interesting: https://github.com/gcc-mirror/gcc/blob/11c13111ac64a035d6c4ea6c118eff4ece7a9d9b/libobjc/objc/runtime.h#L116-L118.

Apparently, GCC also has |: https://github.com/gcc-mirror/gcc/blob/11c13111ac64a035d6c4ea6c118eff4ece7a9d9b/libobjc/objc/runtime.h#L129C1-L129C27.

I would love to have some code that actually uses these things.

t0rr3sp3dr0 · 2024-04-29T06:17:43Z

Let me rebase my PR in go-macho, there are some changes worth picking there.

blacktop · 2024-04-29T06:18:46Z

Let me rebase my PR in go-macho, there are some changes worth picking there.

agreed

blacktop · 2024-04-29T06:26:37Z

About %, this is interesting: https://github.com/gcc-mirror/gcc/blob/11c13111ac64a035d6c4ea6c118eff4ece7a9d9b/libobjc/objc/runtime.h#L116-L118.

Apparently, GCC also has |: https://github.com/gcc-mirror/gcc/blob/11c13111ac64a035d6c4ea6c118eff4ece7a9d9b/libobjc/objc/runtime.h#L129C1-L129C27.

I would love to have some code that actually uses these things.

nice find!

blacktop · 2024-04-29T17:33:58Z

Tested a few of my fav dylibs and it looks good 👍

fix: types with protocol qualified are not built in

7c77867

fix: parsing encoded Objective-C types

81dd1c6

fix: support int128

926f476

Merge branch 'master' into t0rr3sp3dr0-patch-7

f60cf32

blacktop mentioned this pull request Apr 24, 2024

class dump objc categories class name missed for NSObject/UIView etc. #445

Closed

2 tasks

t0rr3sp3dr0 added 2 commits April 28, 2024 19:59

Merge branch 'blacktop:master' into t0rr3sp3dr0-patch-7

43ae919

fix: detection of built ins

eced8f6

blacktop merged commit 5ad4bc9 into blacktop:master Apr 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: types with protocol qualified are not built-ins #451

fix: types with protocol qualified are not built-ins #451

t0rr3sp3dr0 commented Apr 22, 2024

blacktop commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024 •

edited

Loading

t0rr3sp3dr0 commented Apr 27, 2024

t0rr3sp3dr0 commented Apr 29, 2024

blacktop commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024 •

edited

Loading

blacktop commented Apr 29, 2024

blacktop commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024

blacktop commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024

blacktop commented Apr 29, 2024

blacktop commented Apr 29, 2024

blacktop commented Apr 29, 2024

	before, after, _ := strings.Cut(typ, "<")

	o.fillImportsForType(before, className, classNames, protoNames, imp)
	o.fillImportsForType("<"+after, className, classNames, protoNames, imp)

fix: types with protocol qualified are not built-ins #451

fix: types with protocol qualified are not built-ins #451

Conversation

t0rr3sp3dr0 commented Apr 22, 2024

blacktop commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

t0rr3sp3dr0 commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024

blacktop commented Apr 23, 2024 • edited Loading

t0rr3sp3dr0 commented Apr 27, 2024

t0rr3sp3dr0 commented Apr 29, 2024

blacktop commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024 • edited Loading

blacktop commented Apr 29, 2024

blacktop commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024

blacktop commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024

t0rr3sp3dr0 commented Apr 29, 2024

blacktop commented Apr 29, 2024

blacktop commented Apr 29, 2024

blacktop commented Apr 29, 2024

blacktop commented Apr 23, 2024 •

edited

Loading

t0rr3sp3dr0 commented Apr 29, 2024 •

edited

Loading