Debugger implementation #159

dkimitsa · 2017-06-12T12:39:45Z

it seems to be more-less stable, to extend the audience let make it a part of snapshot.
changes:

JDWP server/hooks side implementation itself.
it requires modified soot. Modification were done to fetch local variable debug data. Modified version of soot is located here https://github.com/dkimitsa/soot
LLMV code was extended with one to receive dwarf debug information for local variables. compiled library included in this PR
ClassCompiler was modified to include debug information into binary during the build

Optional:
there is an issue with moe sim launcher https://github.com/multi-os-engine/moe-ios-simulator-launcher/issues/1 and workaround https://github.com/dkimitsa/moe-ios-simulator-launcher/tree/fix_for_pipe_hangs_on_long_msg

Added launch plugin with arguments Integrated to idea

…nary

…er of modifiers

…primitive ones, not tested to be runnable

…ucture

…ome cases it crashes with message "Block was not compiled using a compiler that inserts type information about arguments"

* origin/master: Removed unused old simulator binaries. Exclude some Entitlement keys (MobiVM#144) Update README.md Update README.md Fix issue MobiVM#140 (MobiVM#142) Add the "framework" compiler target (issue MobiVM#132) (MobiVM#141) Added the ability to select one of several USB-connected physical devices with the robovm.device.udid parameter in the Gradle plugin. (MobiVM#139) Update poms to next development version

…s in tar, otherwise on each launch idea will rebuild all classes in case of snapshot builds

* origin/master: Ios10.3 bindings (MobiVM#152) IDEA -- fixed resource unpack to maintain the timestamp same as it was in tar, otherwise on each launch idea will rebuild all classes in case of snapshot builds (MobiVM#149) Remove leftovers of old simulator handling, bringing to foreground. Remove old commercial license manager eclipse ui elements and gradle tasks Android studio/Idea fixes (MobiVM#148) blocks implementation -- added signature generation to blocks as in some cases it crashes with message "Block was not compiled using a compiler that inserts type information about arguments" (MobiVM#147)

…at return null pointer

florianf · 2017-06-12T19:47:06Z

That looks awesome!

Regarding LLVM: did you patch the lib? I think we should fork the robovm llvm repo with your patches applied @too, so that we can rebuild it? Same goes for moe-ios-simulator launcher.

So that would add up to the following new repos:

Soot from your fork
LLVM from original RoboVM?
moe-ios-simulator launcher from moe with your pr applied

dkimitsa · 2017-06-13T10:19:15Z

hi,
there is no LLVM changes itself, but robovm llvm library was changed, there was code for dwarf support (dylib already included in PR). But it would be good idea to fork LLVM as well as it is being picked up currently from RoboVM AB account (but this is not required).

mod-sim-simulator launcher -- there is dirty workaround that works for me. I just recommend to use it as on big log outputs applications will hang.

florianf · 2017-06-13T18:23:26Z

Allright, thanks for the info.

We started forking all necessary repos under the MobiVM organisation, see: https://github.com/MobiVM
We'll move all other repos to this organisation in the future, so the fork isn't tied to @MobiDevelop's personal account anymore.

How should we proceed further? Releasing a SOOT version in the com.mobidevelop namespace?

dkimitsa · 2017-06-14T07:09:56Z

yep, after SOOT under mobidev is only missing dependency.
regarding LLVM: I'd built only MAC dylib so Linux ones are old. I don't know the status of RoboVM for linux, probably these libs could be removed for now at all to reduce package size

florianf · 2017-06-14T07:52:10Z

Okay. @Tom-Ski could you release the forked SOOT on Maven?

Regarding Linux: I would like to keep it, since my primary platform in use is Linux. I can recompile the .so for linux, so that's not a problem.

…obiVM@3d0d71e

Tom-Ski · 2017-06-17T08:57:33Z

I've pushed 2.5.0-4-SNAPSHOT of soot, if you could give that a whirl and let me know how it goes i can push a release to work from on stable.

florianf · 2017-06-17T15:17:10Z

Perfect, thanks! @dkimitsa can you take a look at this? I would merge this after your okay.

dkimitsa · 2017-06-19T07:48:33Z

@florianf built with, no issues so far

florianf · 2017-06-19T15:36:42Z

Okay, than we're good to merge?

florianf · 2017-06-20T08:28:53Z

Got the debugger to work in Eclipse, very cool! Currently I'm encountering an exception when trying to set a breakpoint:

Exception in thread "org.eclipse.jdt.debug: JDI Event Dispatcher" com.sun.jdi.InternalException: Invalid ReferenceTypeID tag encountered:76
	at org.eclipse.jdi.internal.ReferenceTypeImpl.readWithTypeTag(ReferenceTypeImpl.java:1350)
	at org.eclipse.jdi.internal.MethodImpl.readWithReferenceTypeWithTag(MethodImpl.java:697)
	at org.eclipse.jdi.internal.LocationImpl.read(LocationImpl.java:175)
	at org.eclipse.jdi.internal.event.LocatableEventImpl.readThreadAndLocation(LocatableEventImpl.java:48)
	at org.eclipse.jdi.internal.event.BreakpointEventImpl.read(BreakpointEventImpl.java:55)
	at org.eclipse.jdi.internal.event.EventImpl.read(EventImpl.java:124)
	at org.eclipse.jdi.internal.event.EventSetImpl.read(EventSetImpl.java:167)
	at org.eclipse.jdi.internal.event.EventQueueImpl.remove(EventQueueImpl.java:71)
	at org.eclipse.jdt.internal.debug.core.EventDispatcher.run(EventDispatcher.java:240)
	at java.lang.Thread.run(Thread.java:745)

This happens here in the Eclipse JDT tools:
https://github.com/eclipse/eclipse.jdt.debug/blob/master/org.eclipse.jdt.debug/jdi/org/eclipse/jdi/internal/ReferenceTypeImpl.java#L1342

Any idea what this might cause, since 76 seems to be the ReferenceID for an object type? Im running the debugger in client mode for eclipse (opposed to server mode for IDEA).

EDIT:
The Type Tag constants are defined as follows:

	public static final byte TYPE_TAG_CLASS = 1; // ReferenceType is a class.
	public static final byte TYPE_TAG_INTERFACE = 2; // ReferenceType is an
														// interface.
	public static final byte TYPE_TAG_ARRAY = 3; // ReferenceType is an array.

so in the method

	public static ReferenceTypeImpl readWithTypeTag(MirrorImpl target,
			DataInputStream in) throws IOException {
		byte typeTag = target.readByte("type tag", JdwpID.typeTagMap(), in); //$NON-NLS-1$
		switch (typeTag) {
		case 0:
			return null;
		case ArrayTypeImpl.typeTag:
			return ArrayTypeImpl.read(target, in);
		case ClassTypeImpl.typeTag:
			return ClassTypeImpl.read(target, in);
		case InterfaceTypeImpl.typeTag:
			return InterfaceTypeImpl.read(target, in);
		}
		throw new InternalException(
				JDIMessages.ReferenceTypeImpl_Invalid_ReferenceTypeID_tag_encountered___8
						+ typeTag);
	}

there shouldn't be a type 76, because it's not a reference type.
Reading through the eclipse source, it should be 1 for a class reference here.

The stack leading to it:

ReferenceTypeImpl.readWithTypeTag(MirrorImpl, DataInputStream) line: 1350	
MethodImpl.readWithReferenceTypeWithTag(MirrorImpl, DataInputStream) line: 697	
LocationImpl.read(MirrorImpl, DataInputStream) line: 175	
BreakpointEventImpl(LocatableEventImpl).readThreadAndLocation(MirrorImpl, DataInputStream) line: 48	
BreakpointEventImpl.read(MirrorImpl, RequestID, DataInputStream) line: 55	
EventImpl.read(MirrorImpl, DataInputStream) line: 124	
EventSetImpl.read(MirrorImpl, DataInputStream) line: 167	
EventQueueImpl.remove(long) line: 71	
EventDispatcher.run() line: 240

…e to complain. +some more copyrights

dkimitsa · 2017-06-20T09:35:24Z

fixed, btw idea also uses client mode

florianf · 2017-06-20T09:46:27Z

I meant server mode. I need to pass clientmode=false when launchin, since eclipse seems to be expecting a running jdwp server.

I'll test your fixes, thanks!

florianf · 2017-06-20T09:51:03Z

Oh yes, it works now! Currently I'm having problems with the source lookup in eclipse, but breakpoints, stackvars and the like work. Nice work.

florianf · 2017-06-22T20:32:06Z

Hey @dkimitsa, spent some time trying to get the eclipse source lookup to work. I encountered the following: I'm creating a simple ConsoleProject with the following simple main method:

    public static void main(String[] args) throws Exception, IllegalAccessException, IllegalArgumentException, InvocationTargetException, NoSuchMethodException, NoSuchFieldException {	
		System.out.println("Entering sleep ...");
		for (int i=0;i<10;i++) {
			Thread.currentThread().sleep(1000);
		}
		
		System.out.println("okk");
    }

For some reason I need some time before setting a breakpoint works. If I do this, setting the breakpoint works but than the Thread HooksChannel crashes. Following log:

22:27:32.236 INFORMATION: JdwpDebugServer: handling req: Method(6).LineTable(1)
22:27:32.236 INFORMATION: JdwpDebugServer: handling req: Method(6).LineTable(1)
22:27:32.236 INFORMATION: JdwpDebugServer: handling req: Method(6).LineTable(1)
22:27:32.236 INFORMATION: JdwpDebugServer: handling req: Method(6).LineTable(1)
22:27:32.237 INFORMATION: JdwpDebugServer: handling req: EventRequest(15).Set(1)
22:27:32.237 INFORMATION: JdwpEventCenterDelegate: jdwpSetEventRequest: JdwpEventRequest{requestId=113, eventKind=BREAKPOINT, suspendPolicy=EVENT_THREAD, predicates=[Location{classId=5735, methodId=1057, index=56} ]}
22:27:32.237 INFORMATION: JdwpDebugServer: handling req: ReferenceType(2).Signature(1)
22:27:39.537 INFORMATION: JdwpDebugServer: handling req: ThreadReference(11).Resume(3)
22:27:39.537 INFORMATION: JdwpEventCenterDelegate: THREAD_RESUMED: thread (33, main, main)
22:27:39.538 INFORMATION: JdwpEventCenterDelegate: THREAD_DETTACHED: thread (33, main, main)
22:27:39.539 SCHWERWIEGEND: Debugger: Thread HooksChannel socket thread crashed
org.robovm.debugger.DebuggerException: java.nio.BufferOverflowException
	at org.robovm.debugger.hooks.HooksChannel.doSocketWork(HooksChannel.java:152)
	at org.robovm.debugger.hooks.HooksChannel.lambda$0(HooksChannel.java:65)
	at java.lang.Thread.run(Thread.java:745)
	at org.robovm.debugger.utils.DebuggerThread.run(DebuggerThread.java:42)
Caused by: java.nio.BufferOverflowException
	at org.robovm.debugger.utils.bytebuffer.ByteBufferPacket.fillFromInputStream(ByteBufferPacket.java:187)
	at org.robovm.debugger.hooks.HooksChannel.doSocketWork(HooksChannel.java:120)
	... 3 more
22:27:40.042 SCHWERWIEGEND: Debugger: Thread JDWP server socket thread crashed
org.robovm.debugger.DebuggerException: java.net.SocketException: Socket closed
	at org.robovm.debugger.jdwp.JdwpDebugServer.doSocketWork(JdwpDebugServer.java:211)
	at org.robovm.debugger.jdwp.JdwpDebugServer.lambda$0(JdwpDebugServer.java:129)
	at java.lang.Thread.run(Thread.java:745)
	at org.robovm.debugger.utils.DebuggerThread.run(DebuggerThread.java:42)
Caused by: java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.robovm.debugger.utils.bytebuffer.ByteBufferPacket.fillFromInputStream(ByteBufferPacket.java:185)
	at org.robovm.debugger.jdwp.protocol.JdwpRequestHeader.readFromStream(JdwpRequestHeader.java:36)
	at org.robovm.debugger.jdwp.JdwpDebugServer.doSocketWork(JdwpDebugServer.java:182)
	... 3 more

Any idea what's going wrong?

dkimitsa · 2017-06-23T07:14:14Z

can you please add following parameters.getArguments().add("-rvm:log=debug"); to see at device side ?
also I can see that your application lives only 10 sec

there is known limitation of application that run without system debugger attached -- they shall complete DidFinishLaunching in 20 seconds. so if you keep main on breakpoint -- system will terminate you in 20 sec

florianf · 2017-06-24T18:53:52Z

Hey @dkimitsa, thanks for your help! I'm gonna merge this as it is, since I can't get the eclipse debugging to work right now. I'm going to open a seperate issue for this, but in the meantime we can bring this amazing stuff to the people, and maybe someone steps up helping with eclipse.

dkimitsa and others added 30 commits March 2, 2017 14:20

XCode project generator + integration with Idea plugin

d934ffa

added support for @IBInspectable annotation

fdbb65f

added IBInspectable support to XCode project generator

e57a2af

reverted version to match trunk

43b59e8

updated dependency in Idea pom file to include ibxcode module

e5f5247

Merge branch 'xcode_project_and_ibinspectable' into debugger

6b3c690

Debugger module added

c9a1059

Added launch plugin with arguments Integrated to idea

Debugger work in progress

e2090a9

JDWP work in progress. Lack of class info data

af7f20b

JDWP work in progress. jdb able to connect to this stub

344fb04

JDWP work in progress. more JDWP command handlers

a1b76d7

JDWP work in progress. added Mach-o parser to get information from bi…

3dcec6d

…nary

JDWP work in progress. class info is parsed from mach-o data

9497f3b

JDWP work in progress. field info and method info parsing added

d5a5f3f

JDWP work in progress. Field/Method/Class info reworked a bit

51bd27d

JDWP work in progress. Capabilities handlers added

dcab276

debugger work in progress

128968c

debugger work in progress

8077f12

debugger work in progress

469802a

debugger work in progress - picked list of interfaces for class

5c1095e

debugger work in progress - added injection of bcHookInstrumented calls

41b4440

debugger work in progress - JDWP event filtering reworked to keep ord…

d67fdf2

…er of modifiers

debugger work in progress - huge rethink/rework

0aa7cad

debugger work in progress - adapting shutdowns, partial rework

712a51f

debugger work in progress - added basic synchronization around state

1360a70

debugger work in progress - class info reworked to include array and …

f96713e

…primitive ones, not tested to be runnable

debugger work in progress - rework in progress. finding the final str…

6d84b9e

…ucture

debugger work in progress - thread part, can compile now

b8fc9a0

debugger work in progress - day of instance variables

748321b

debugger work in progress - thread/threadgroup/string instances

347b295

dkimitsa added 7 commits May 25, 2017 09:44

blocks implementation -- added signature generation to blocks as in s…

135a72e

…ome cases it crashes with message "Block was not compiled using a compiler that inserts type information about arguments"

IDEA -- fixed resource unpack to maintain the timestamp same as it wa…

51ab398

…s in tar, otherwise on each launch idea will rebuild all classes in case of snapshot builds

debugger -- fixed resume VM handler.

d8b35e1

debugger: fixed crash when trying instantiating result of invoke() th…

244564c

…at return null pointer

debugger: copyright time

0aff827

florianf self-requested a review June 12, 2017 19:44

dkimitsa added 3 commits June 14, 2017 13:13

retroperspective: this change is missing but present in master, check M…

c5a006f

…obiVM@3d0d71e

debugger: fixed crash array get Object element on 64bit target

1175f64

debugger: corrected getter name

a580c34

Fixed wrong TypeTag returned once breakpoint is hit that cause eclips…

962b0f2

…e to complain. +some more copyrights

florianf merged commit 557af56 into MobiVM:master Jun 24, 2017

dkimitsa deleted the debugger branch July 19, 2017 14:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Debugger implementation #159

Debugger implementation #159

dkimitsa commented Jun 12, 2017

florianf commented Jun 12, 2017 •

edited

Loading

dkimitsa commented Jun 13, 2017

florianf commented Jun 13, 2017 •

edited

Loading

dkimitsa commented Jun 14, 2017

florianf commented Jun 14, 2017

Tom-Ski commented Jun 17, 2017

florianf commented Jun 17, 2017 •

edited

Loading

dkimitsa commented Jun 19, 2017

florianf commented Jun 19, 2017

florianf commented Jun 20, 2017 •

edited

Loading

dkimitsa commented Jun 20, 2017

florianf commented Jun 20, 2017 •

edited

Loading

florianf commented Jun 20, 2017

florianf commented Jun 22, 2017

dkimitsa commented Jun 23, 2017 •

edited

Loading

florianf commented Jun 24, 2017

Debugger implementation #159

Debugger implementation #159

Conversation

dkimitsa commented Jun 12, 2017

florianf commented Jun 12, 2017 • edited Loading

dkimitsa commented Jun 13, 2017

florianf commented Jun 13, 2017 • edited Loading

dkimitsa commented Jun 14, 2017

florianf commented Jun 14, 2017

Tom-Ski commented Jun 17, 2017

florianf commented Jun 17, 2017 • edited Loading

dkimitsa commented Jun 19, 2017

florianf commented Jun 19, 2017

florianf commented Jun 20, 2017 • edited Loading

dkimitsa commented Jun 20, 2017

florianf commented Jun 20, 2017 • edited Loading

florianf commented Jun 20, 2017

florianf commented Jun 22, 2017

dkimitsa commented Jun 23, 2017 • edited Loading

florianf commented Jun 24, 2017

florianf commented Jun 12, 2017 •

edited

Loading

florianf commented Jun 13, 2017 •

edited

Loading

florianf commented Jun 17, 2017 •

edited

Loading

florianf commented Jun 20, 2017 •

edited

Loading

florianf commented Jun 20, 2017 •

edited

Loading

dkimitsa commented Jun 23, 2017 •

edited

Loading