can't get field offset on a record class using quarkus-junit5

[Eternal-night-blip]

Describe the bug

I am new to Quarkus and want to hand on effective java test.
When I used the parameterization test on the ItemWait class by quarkus-junit5:3.9.3, relied XStream library reported an error as below. how can I do, can I replace Xstream? Note that I donnot want to replace the record Class with a regular class Class for ItemDetail.

Version and Build tool

quarkus 3.9.3
maven auto controlled by quarkus

Error Message

com.thoughtworks.xstream.converters.ConversionException:
---- Debugging information ----
cause-exception : java.lang.UnsupportedOperationException
cause-message : can't get field offset on a record class: private final java.lang.String org.todo.domain.item.ItemDetail.name
class : org.todo.domain.item.ItemDetail
required-type : org.todo.domain.item.ItemDetail
converter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
path : /org.todo.domain.item.ItemWait/detail/name
line number : 6
class[1] : org.todo.domain.item.ItemWait
required-type[1] : org.todo.domain.item.ItemWait
version : 1.4.20

Code

@QuarkusTest
class ItemWaitTest {
    @Test
    void shouldOfWork() {
        ItemDetail detail = new ItemDetail("todo1", "content", LocalDateTime.now(), LocalDateTime.now().plusDays(1));
        ItemWait wait = ItemWait.of("publisher", "performer", detail);
        assertThat(wait.getId()).isNull();
        assertThat(wait.getStatus()).isEqualTo(ItemWaitingStatus.WAIT_FOR_RECEPTION);
    }

    /*
     * 测试等效主键与主键,等效主键必须与主键保持一对一关系
     * 等效主键是publisher,performer,detail.name的复合,注意这三个值不可以为null
     * 主键是id,id可能为null,这是因为可能还没持久化到数据库中
     */

    @ParameterizedTest
    @MethodSource("equalsTestCases")
    void shouldEqualsWork(ItemWait wait1, ItemWait wait2, boolean expected) {
        assertThat(wait1.equals(wait2)).isEqualTo(expected);
    }

    @ParameterizedTest
    @MethodSource("equalsTestCasesAssertion")
    void shouldEqualsAssert(ItemWait wait1, ItemWait wait2,String errorMessage){
        boolean asserted = false;
        try{
            wait1.equals(wait2);
        }catch(AssertionError e){
             asserted = true;
            assertThat(e.getMessage()).isEqualTo(errorMessage);
        }
        assertTrue(asserted);
    }

    static Stream<Arguments> equalsTestCases() {

        return Stream.of(
                of(wait("a", "b", "c", null),
                        wait("a", "b", "c", null),
                        true),
                of(wait("x", "b", "c", null),
                        wait("a", "b", "c", null),
                        false),
                of(wait("a", "x", "c", null),
                        wait("a", "b", "c", null),
                        false),
                of(wait("a", "b", "x", null),
                        wait("a", "b", "c", null),
                        false),
                of(wait("x", "y", "c", null),
                        wait("a", "b", "c", null),
                        false),
                of(wait("x", "b", "y", null),
                        wait("a", "b", "c", null),
                        false),
                of(wait("a", "x", "y", null),
                        wait("a", "b", "c", null),
                        false),
                of(wait("x", "y", "z", null),
                        wait("a", "b", "c", null),
                        false),

                of(wait("a", "b", "c", 1L),
                        wait("a", "b", "c", null),
                        true),
                of(wait("x", "b", "c", 1L),
                        wait("a", "b", "c", null),
                        false),
                of(wait("a", "x", "c", 1L),
                        wait("a", "b", "c", null),
                        false),
                of(wait("a", "b", "x", 1L),
                        wait("a", "b", "c", null),
                        false),
                of(wait("x", "y", "c", 1L),
                        wait("a", "b", "c", null),
                        false),
                of(wait("x", "b", "y", 1L),
                        wait("a", "b", "c", null),
                        false),
                of(wait("a", "x", "y", 1L),
                        wait("a", "b", "c", null),
                        false),
                of(wait("x", "y", "z", 1L),
                        wait("a", "b", "c", null),
                        false),
                //等效主键判断与主键判断一致
                of(wait("a", "b", "c", 1L),
                        wait("a", "b", "c", 1L),
                        true),
                of(wait("x", "b", "c", 2L),
                        wait("a", "b", "c", 1L),
                        false),
                of(wait("a", "x", "c", 2L),
                        wait("a", "b", "c", 1L),
                        false),
                of(wait("a", "b", "x", 2L),
                        wait("a", "b", "c", 1L),
                        false),
                of(wait("x", "y", "c", 2L),
                        wait("a", "b", "c", 1L),
                        false),
                of(wait("x", "b", "y", 2L),
                        wait("a", "b", "c", 1L),
                        false),
                of(wait("a", "x", "y", 2L),
                        wait("a", "b", "c", 1L),
                        false),
                of(wait("x", "y", "z", 2L),
                        wait("a", "b", "c", 1L),
                        false)
                );
                
                
    }

    static Stream<Arguments> equalsTestCasesAssertion() {
        String error = "ItemWait的主键与虚拟等效主键不一致";
        return Stream.of(

                of(wait("a", "b", "c", 2L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("x", "b", "c", 1L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("a", "x", "c", 1L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("a", "b", "x", 1L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("x", "y", "c", 1L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("x", "b", "y", 1L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("a", "x", "y", 1L),
                        wait("a", "b", "c", 1L),
                        error),
                of(wait("x", "y", "z", 1L),
                        wait("a", "b", "c", 1L),
                        error)

        );
    }

    // Simplify the creation of ItemWait for parameterized testing of the equals method
    static ItemWait wait(String publisher, String performer, String name, Long id) {
        var begin = LocalDateTime.of(2099, 12, 1, 10, 10, 0);
        var end = LocalDateTime.of(2099, 12, 2, 10, 10, 0);
        var content = "content";

        var wait = ItemWait.of(publisher, performer, new ItemDetail(name, content, begin, end));
        try {
            Field idField = ItemWait.class.getDeclaredField("id");
            idField.setAccessible(true);
            idField.set(wait, id);

        } catch (Exception e) {
            e.printStackTrace();
        }
        return wait;
    }

    @Test
    void shouldWaitInThisFileWork(){
        ItemWait wait = wait("a","b","c",1L);
        var begin = LocalDateTime.of(2099, 12, 1, 10, 10, 0);
        var end = LocalDateTime.of(2099, 12, 2, 10, 10, 0);
        assertEquals(1L, wait.getId());
        assertEquals("a", wait.getPublisher());
        assertEquals("b", wait.getPerformer());
        assertEquals("c", wait.getDetail().name());
        assertEquals("content", wait.getDetail().content());
        assertEquals(wait.getDetail().begin(), begin);
        assertEquals(wait.getDetail().end(), end);
    }    
}

@Entity
public class ItemWait {

    @Id
    @GeneratedValue
    private Long id;

    @NotBlank(message = "事项发布者的名称不可以为空")
    private String publisher;

    @NotBlank(message = "事项执行者的名称不可以为空")
    private String performer;

    @Embedded
    @NotNull(message = "事项详细内容不可以为Null")
    private ItemDetail detail;

    private ItemWaitingStatus status;
    public static ItemWait of(@NotBlank String publisher, @NotBlank String performer, @Valid ItemDetail detail) {
        return new ItemWait(null, publisher, performer, detail, ItemWaitingStatus.WAIT_FOR_RECEPTION);
    }

    private ItemWait(Long id, String publisher, String performer, ItemDetail detail, ItemWaitingStatus status) {
        this.id = id;
        this.publisher = publisher;
        this.performer = performer;
        this.detail = detail;
        this.status = status;
    }
   //omit  other methods
}

@Embeddable
public record ItemDetail(@NotBlank(message = "事项名称不可以为空") String name,
                @NotBlank(message = "事项内容不可以为空") String content,
                @NotNull(message = "事项起始时间不可以为NULL") LocalDateTime begin,
                @NotNull(message = "事项结束时间不可以为Null") LocalDateTime end) {

        public ItemDetail {
                LocalDateTime rightBoundary = LocalDateTime.now();
                LocalDateTime leftBoundary = rightBoundary.minusSeconds(20);
                assert begin.isAfter(leftBoundary) : "无效的事项起始时间";
                assert begin.isBefore(end) : "事项起始时间必须在结束时间之前";
        }
}

[geoand]

This has long been a thorn in our side unfortunately...

It's one that @holly-cummins was looking into, but it seems very very complicated

[holly-cummins]

Yes, I think this is a duplicate of ... actually, we have several issues in this area. I got quite far with the fix and then got a bit stuck between a number of undesirable options - but still looking at it.

[geoand]

Thanks for the update @holly-cummins!

[holly-cummins]

@Eternal-night-blip, there is a workaround for this issue, which is to use a version of Java below 16. Obviously, that causes all sorts of other issues, though. The minimum version of Java for recent Quarkus releases is 17, so you might even need to go to a lower version of Quarkus. So it's definitely not a nice workaround.

[Eternal-night-blip]

@holly-cummins,well, I hope you fix this bug someday,good luck to you. So for now I just have to manually build the parametrized tests for myself.

[holly-cummins]

May be fixed by #40601

[yk-littlepay]

If someone has the same problem, you can make a record Serializable as a temporary solution.

[cyrilcolinet]

Hi there, any update on this ? (cc @holly-cummins)

It seems moving from xStream to another marshaller fixes the problem but its not possible for me in that case.

[holly-cummins]

Hi @cyrilcolinet, we're waiting for a release of JUnit 5.11, or a backport of junit-team/junit5#3820 to JUnit 5.10.3. Once that's done, we can merge #40906, which should fix this issue.

[cyrilcolinet]

Perfect.

If this issue is still not fixed, you might need to instantiate a "custom" xStream converter which handles the records perfectly.

This is the code converter:

import com.thoughtworks.xstream.converters.Converter;
import com.thoughtworks.xstream.converters.MarshallingContext;
import com.thoughtworks.xstream.converters.UnmarshallingContext;
import com.thoughtworks.xstream.io.HierarchicalStreamReader;
import com.thoughtworks.xstream.io.HierarchicalStreamWriter;

import java.lang.reflect.Constructor;
import java.lang.reflect.RecordComponent;
import java.util.Arrays;

public class RecordConverter implements Converter {

    @Override
    public boolean canConvert(Class type) {
        return type.isRecord();
    }

    @Override
    public void marshal(Object source, HierarchicalStreamWriter writer, MarshallingContext context) {
        Class<?> recordClass = source.getClass();
        RecordComponent[] components = recordClass.getRecordComponents();
        try {
            for (RecordComponent component : components) {
                writer.startNode(component.getName());
                Object value = component.getAccessor().invoke(source);
                if (value != null) {
                    context.convertAnother(value);
                }
                writer.endNode();
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public Object unmarshal(HierarchicalStreamReader reader, UnmarshallingContext context) {
        Class<?> recordClass = context.getRequiredType();
        RecordComponent[] components = recordClass.getRecordComponents();
        Object[] values = new Object[components.length];
        try {
            int i = 0;
            while (reader.hasMoreChildren()) {
                reader.moveDown();
                for (RecordComponent component : components) {
                    if (component.getName().equals(reader.getNodeName())) {
                        values[i] = context.convertAnother(null, component.getType());
                        i++;
                        break;
                    }
                }
                reader.moveUp();
            }
            Constructor<?> constructor = recordClass.getDeclaredConstructor(
                    Arrays.stream(components).map(RecordComponent::getType).toArray(Class[]::new)
            );
            return constructor.newInstance(values);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

and register it via xStream.registerConverter(new RecordConverter());.
This worked well for me.

The made me continue

[nscuro]

We are running into a similar issue with Quarkus 3.13.0, which switched the serializer as outlined in the earlier comments of this thread. In our case, the class where @ParameterizedTest is used includes a record class with package-private visibility. The class already implements Serializable as suggested in #40148 (comment). We're building with Java 21.

This worked fine with Quarkus v3.12.3, but fails with v3.13.0.

Accessing methods of the record inside the test fails with:

java.lang.IllegalAccessException: symbolic reference class is not accessible: class org.dependencytrack.vulnanalyzer.processor.scanner.internal.InternalScannerProcessorTest$Range, from class org.jboss.marshalling.reflect.JDKSpecific

Complete Stacktrace

2024-07-25T05:43:36.5573364Z java.lang.IllegalStateException: Cannot invoke record getter endExcluding in object Range[startIncluding=null, startExcluding=null, endIncluding=null, endExcluding=null]
2024-07-25T05:43:36.5575732Z 	at org.jboss.marshalling.reflect.JDKSpecific.getRecordComponentValue(JDKSpecific.java:89)
2024-07-25T05:43:36.5577779Z 	at org.jboss.marshalling.reflect.SerializableField.getRecordComponentValue(SerializableField.java:644)
2024-07-25T05:43:36.5579600Z 	at org.jboss.marshalling.cloner.SerializingCloner.clone(SerializingCloner.java:244)
2024-07-25T05:43:36.5581269Z 	at org.jboss.marshalling.cloner.SerializingCloner.clone(SerializingCloner.java:132)
2024-07-25T05:43:36.5583096Z 	at io.quarkus.test.junit.internal.NewSerializingDeepClone.clone(NewSerializingDeepClone.java:128)
2024-07-25T05:43:36.5585033Z 	at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:963)
2024-07-25T05:43:36.5586865Z 	at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:903)
2024-07-25T05:43:36.5588607Z 	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestTemplateMethod(QuarkusTestExtension.java:863)
2024-07-25T05:43:36.5590170Z 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
2024-07-25T05:43:36.5591512Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5592865Z 	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
2024-07-25T05:43:36.5594467Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5595820Z 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
2024-07-25T05:43:36.5597142Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5598953Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5600413Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5601756Z 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
2024-07-25T05:43:36.5603137Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5604585Z 	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
2024-07-25T05:43:36.5606040Z 	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
2024-07-25T05:43:36.5607424Z 	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
2024-07-25T05:43:36.5608770Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5610087Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5611408Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5612799Z 	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
2024-07-25T05:43:36.5614242Z 	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
2024-07-25T05:43:36.5615871Z 	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
2024-07-25T05:43:36.5617321Z 	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
2024-07-25T05:43:36.5618769Z 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
2024-07-25T05:43:36.5620155Z 	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2024-07-25T05:43:36.5621461Z 	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
2024-07-25T05:43:36.5622799Z 	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
2024-07-25T05:43:36.5624130Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5625442Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5626748Z 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2024-07-25T05:43:36.5628227Z 	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
2024-07-25T05:43:36.5629640Z 	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
2024-07-25T05:43:36.5631029Z 	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
2024-07-25T05:43:36.5632464Z 	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
2024-07-25T05:43:36.5633904Z 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
2024-07-25T05:43:36.5653462Z 	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2024-07-25T05:43:36.5654819Z 	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
2024-07-25T05:43:36.5656161Z 	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
2024-07-25T05:43:36.5657554Z 	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
2024-07-25T05:43:36.5658919Z 	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
2024-07-25T05:43:36.5660260Z 	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
2024-07-25T05:43:36.5661666Z 	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
2024-07-25T05:43:36.5663069Z 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
2024-07-25T05:43:36.5664419Z 	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2024-07-25T05:43:36.5666053Z 	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
2024-07-25T05:43:36.5667154Z 	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2024-07-25T05:43:36.5668043Z 	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2024-07-25T05:43:36.5670678Z Caused by: java.lang.IllegalAccessException: symbolic reference class is not accessible: class org.dependencytrack.vulnanalyzer.processor.scanner.internal.InternalScannerProcessorTest$Range, from class org.jboss.marshalling.reflect.JDKSpecific (unnamed module @3f270e0a)
2024-07-25T05:43:36.5673491Z 	at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:894)
2024-07-25T05:43:36.5675158Z 	at java.base/java.lang.invoke.MethodHandles$Lookup.checkSymbolicClass(MethodHandles.java:3787)
2024-07-25T05:43:36.5676524Z 	at java.base/java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:3747)
2024-07-25T05:43:36.5677868Z 	at java.base/java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:2767)
2024-07-25T05:43:36.5679270Z 	at org.jboss.marshalling.reflect.JDKSpecific.getRecordComponentValue(JDKSpecific.java:85)
2024-07-25T05:43:36.5680251Z 	... 51 more

The fix is to make the record class public. It is a bit unintuitive since JUnit 5 classes and test methods themselves can be package-private, but it fixes the test failures for now.

[holly-cummins]

@nscuro, do you mind raising a new defect for the issue you're seeing in 3.13.0?
And @cyrilcolinet, your issue should be fixed in 3.13.0. If you wanted to try it out, 3.13.0.CR1 is available now.

[yk-littlepay]

@holly-cummins the newest version 3.13.0 brakes everything even with Serializable

[gsmet]

Could someone provide a small reproducer for the issue?

[gsmet]

@dmlloyd FYI ^

[holly-cummins]

@yk-littlepay, is your code the same code as what @Eternal-night-blip provided in the initial description, or is your code slightly different? Was your code working with 3.12.x, or has it always been broken?

There are some scenarios that used to work that have stopped working with 3.13.0, for example, #42006 and #42098. If your code used to work before 3.13.0, I think it probably needs a new issue, or a comment on #42098 or #42006.

[yk-littlepay]

@holly-cummins the code works fine with 3.12.3 and Serializable for ever record but when I switch to 3.13.0 I got very strange error message IllegalState Error calling onstructor on record class class com.yuko.MySomeClasss if you notice there is even some typo in this message 'onstructor' and two times class class

[holly-cummins]

Thanks, @yk-littlepay. That definitely sounds like a new problem, then. The scenario being tracked by this original issue didn't work with 3.12.3, but now does work with 3.13.0. The serialization changes which fixed the original problem broke some other things. Could you please raise a new issue? As @gsmet says, if you can provide a reproducer, that's really helpful.

[yk-littlepay]

@holly-cummins & @gsmet in a nutshell "Records that contain Enum are still unsupported with @ParameterizedTest even with Serializable in 3.13.0". See the link immediately after your message, there is also a very simple project with a reproducible error.