Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue with using @SystemMessage(fromResource=...) in dev mode. #618

Open
fmatar opened this issue May 23, 2024 · 9 comments
Open

Issue with using @SystemMessage(fromResource=...) in dev mode. #618

fmatar opened this issue May 23, 2024 · 9 comments
Labels
bug Something isn't working

Comments

@fmatar
Copy link
Contributor

fmatar commented May 23, 2024

I have moved my prompt content to a file named my_prompt.txt to resources/dev/langchain4j/services

My service looks as follows:

interface MyService {
  @SystemMessage(fromResource = "my_prompt.txt")
  String chat();
}

The following error happens in dev mode only, I can get around it by building and running it in prod, not fun!

2024-05-23 09:36:27,410 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (Aesh InputStream Reader) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor#cardPage threw an exception: java.lang.NullPointerException: Cannot invoke "org.jboss.jandex.AnnotationValue.asStringArray()" because the return value of "org.jboss.jandex.AnnotationInstance.value()" is null
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.lambda$addChatPage$4(LangChain4jDevUIProcessor.java:98)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
        at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
        at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.addChatPage(LangChain4jDevUIProcessor.java:99)
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.cardPage(LangChain4jDevUIProcessor.java:51)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1570)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)

        at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:337)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.reloadExistingApplication(AugmentActionImpl.java:267)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.reloadExistingApplication(AugmentActionImpl.java:60)
        at io.quarkus.deployment.dev.IsolatedDevModeMain.restartApp(IsolatedDevModeMain.java:190)
        at io.quarkus.deployment.dev.IsolatedDevModeMain.restartCallback(IsolatedDevModeMain.java:173)
        at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:542)
        at io.quarkus.deployment.console.ConsoleStateManager.forceRestart(ConsoleStateManager.java:175)
        at io.quarkus.deployment.console.ConsoleStateManager.lambda$installBuiltins$0(ConsoleStateManager.java:112)
        at io.quarkus.deployment.console.ConsoleStateManager$1.accept(ConsoleStateManager.java:77)
        at io.quarkus.deployment.console.ConsoleStateManager$1.accept(ConsoleStateManager.java:49)
        at io.quarkus.deployment.console.AeshConsole.lambda$setup$1(AeshConsole.java:278)
        at org.aesh.terminal.EventDecoder.accept(EventDecoder.java:118)
        at org.aesh.terminal.EventDecoder.accept(EventDecoder.java:31)
        at org.aesh.terminal.io.Decoder.write(Decoder.java:133)
        at org.aesh.readline.tty.terminal.TerminalConnection.openBlocking(TerminalConnection.java:216)
        at org.aesh.readline.tty.terminal.TerminalConnection.openBlocking(TerminalConnection.java:203)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor#cardPage threw an exception: java.lang.NullPointerException: Cannot invoke "org.jboss.jandex.AnnotationValue.asStringArray()" because the return value of "org.jboss.jandex.AnnotationInstance.value()" is null
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.lambda$addChatPage$4(LangChain4jDevUIProcessor.java:98)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
        at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
        at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.addChatPage(LangChain4jDevUIProcessor.java:99)
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.cardPage(LangChain4jDevUIProcessor.java:51)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1570)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)

        at io.quarkus.builder.Execution.run(Execution.java:123)
        at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
        at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:160)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:333)
        ... 18 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.jboss.jandex.AnnotationValue.asStringArray()" because the return value of "org.jboss.jandex.AnnotationInstance.value()" is null
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.lambda$addChatPage$4(LangChain4jDevUIProcessor.java:98)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipelin212)va:
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
        at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
        at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.addChatPage(LangChain4jDevUIProcessor.java:99)
        at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.cardPage(LangChain4jDevUIProcessor.java:51)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1570)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
@fmatar
Copy link
Contributor Author

fmatar commented May 23, 2024

Looking at the issue I think it's quite an easy fix:

quarkus-langchain4j/core/deployment/src/main/java/io/quarkiverse/langchain4j/deployment/devui/LangChain4jDevUIProcessor.java

Line 97 in 2ac7810

.filter(a -> a.name().equals(LangChain4jDotNames.SYSTEM_MESSAGE))

The filtering could include the non null values. Possibly update

    private void addChatPage(CardPageBuildItem card, List<DeclarativeAiServiceBuildItem> aiServices) {
        List<String> systemMessages = aiServices.stream()
                .map(s -> s.getServiceClassInfo())
                .flatMap(c -> c.annotations().stream()) //This includes method annotations
                .filter(a -> a.name().equals(LangChain4jDotNames.SYSTEM_MESSAGE))
                .map(a -> String.join("", a.value().asStringArray()))
                .toList();
        ...
    }

to:

    private void addChatPage(CardPageBuildItem card, List<DeclarativeAiServiceBuildItem> aiServices) {
        List<String> systemMessages = aiServices.stream()
                .map(s -> s.getServiceClassInfo())
                .flatMap(c -> c.annotations().stream()) //This includes method annotations
                .filter(a -> a.name().equals(LangChain4jDotNames.SYSTEM_MESSAGE) && a.value()!=null)
                .map(a -> String.join("", a.value().asStringArray()))
                .toList();
        ...
    }

Thoughts?

@gsmet
Copy link
Member

gsmet commented May 23, 2024

I think you would have to resolve the system message instead and reading it from the file. Not sure where it's done for runtime but you need something similar (or a way to propagate the value) in Dev UI.

@fmatar
Copy link
Contributor Author

fmatar commented May 24, 2024

I'm looking a bit more into it, thank you for the suggestion

@geoand
Copy link
Collaborator

geoand commented May 24, 2024

cc @jmartisk

@fmatar
Copy link
Contributor Author

fmatar commented May 24, 2024

found a super hacky way to make it work.

@SystemMessage(value="", fromResource="my_prompt.txt")

fromResource still overrides the value.

@fmatar
Copy link
Contributor Author

fmatar commented May 24, 2024 •
edited

That also uncovered another issue. In dev mode a message changed from an external file won't be changed unless I rebuild the app

@jmartisk
Copy link
Collaborator

That also uncovered another issue. In dev mode a message changed from an external file won't be changed unless I rebuild the app

Yeah the extension will have to declare that file to be watched for changes, this should be doable.
Sorry for the delay in answering, I'm traveling all week and on wifi that mostly doesn't work, but I'll have a look into this as soon as I can.

@geoand
Copy link
Collaborator

geoand commented May 29, 2024

this should be doable.

Definitely doable and easy 😜

geoand added a commit that referenced this issue May 30, 2024
@geoand
Watch files configured in @SystemMessage or @Usermessage
af9d844 
Relates to: #618 (comment)
@geoand geoand mentioned this issue May 30, 2024
Merged
@geoand
Copy link
Collaborator

geoand commented May 30, 2024

#639 takes care of the watches resource files issue

geoand added a commit that referenced this issue May 30, 2024
@geoand
Watch files configured in @SystemMessage or @Usermessage
90b8916 
Relates to: #618 (comment)
jmartisk added a commit that referenced this issue Jun 3, 2024
@jmartisk
Merge pull request #639 from quarkiverse/#618
881962c 
Watch files configured in `@SystemMessage` or `@UserMessage`
@geoand geoand added the bug Something isn't working label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fmatar @jmartisk @gsmet @geoand