Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4433 bug wrong source informations #4481

Merged
merged 32 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
22b9b33
clean up some elements
YohannParis Aug 13, 2024
8cb0a37
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 13, 2024
24ba8ff
remove the divider and some clean up
YohannParis Aug 13, 2024
b3078be
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 13, 2024
2e0d6ec
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 14, 2024
2902133
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 14, 2024
bb35cc0
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 14, 2024
08e1dab
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 14, 2024
fcb986a
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 14, 2024
40854fb
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 15, 2024
36e1928
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 15, 2024
ac3be57
Update service to fetch provenance for model configuration extracted …
YohannParis Aug 16, 2024
de11ca3
improve the provenance call
YohannParis Aug 16, 2024
f9ea6cb
Add endpoint
YohannParis Aug 16, 2024
afb0ba6
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 16, 2024
f9e8126
random cleaning up
YohannParis Aug 16, 2024
9bce3f2
Add service endpoint
YohannParis Aug 16, 2024
8e7db0a
Remove the front-end service
YohannParis Aug 16, 2024
a705fdd
Add source to the configuration call
YohannParis Aug 16, 2024
eddfcb9
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 16, 2024
7f49b89
improve the handler
YohannParis Aug 16, 2024
a4f2125
Update the dataset extraction as well
YohannParis Aug 16, 2024
3dbbd6c
Remove observables overwrites
YohannParis Aug 16, 2024
5e0a599
Override setSource to add a source if not there.
YohannParis Aug 16, 2024
02d76b9
clean up
YohannParis Aug 16, 2024
0480fb3
Remove unused methods
YohannParis Aug 16, 2024
4873621
Update ModelConfigurationController.java
YohannParis Aug 16, 2024
9b4181b
Update ConfigureFromDatasetResponseHandler.java
YohannParis Aug 16, 2024
33fd00a
Update ConfigureFromDatasetResponseHandler.java
YohannParis Aug 16, 2024
17fc01f
oopsie, it's Friday evening.
YohannParis Aug 16, 2024
6bf216e
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Aug 16, 2024
afd4933
Merge branch 'main' into 4433-bug-wrong-source-informations
YohannParis Sep 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -50,35 +50,32 @@
<span>{{ baseParameter }}</span>
<Button label="Open Matrix" text size="small" @click.stop="matrixModalId = baseParameter" />
</template>
<div class="flex">
<Divider layout="vertical" type="solid" />
<div class="stratified">
<ul>
<li v-for="{ referenceId } in childParameters" :key="referenceId">
<div class="flex gap-4">
<Checkbox
v-if="
isAddingUncertainty &&
getParameterDistribution(modelConfiguration, referenceId).type === DistributionType.Constant
"
binary
:model-value="selectedParameters.includes(referenceId)"
@change="onSelect(referenceId)"
/>
<tera-parameter-entry
:model="model"
:model-configuration="props.modelConfiguration"
:model-configurations="props.modelConfigurations"
:parameter-id="referenceId"
@update-parameter="emit('update-parameters', [$event])"
@update-source="emit('update-source', $event)"
/>
</div>
<Divider type="solid" />
<Checkbox
v-if="
isAddingUncertainty &&
getParameterDistribution(modelConfiguration, referenceId).type === DistributionType.Constant
"
binary
:model-value="selectedParameters.includes(referenceId)"
@change="onSelect(referenceId)"
/>
<tera-parameter-entry
:model="model"
:model-configuration="props.modelConfiguration"
:model-configurations="props.modelConfigurations"
:parameter-id="referenceId"
@update-parameter="emit('update-parameters', [$event])"
@update-source="emit('update-source', $event)"
/>
</li>
</ul>
</div>
</AccordionTab>
</Accordion>

<!-- Unstratified -->
<div v-else class="flex gap-4">
<Checkbox
Expand All @@ -99,7 +96,6 @@
@update-source="emit('update-source', $event)"
/>
</div>
<Divider type="solid" />
</li>
</ul>
</AccordionTab>
Expand Down Expand Up @@ -137,7 +133,6 @@ import { MiraModel, MiraTemplateParams } from '@/model-representation/mira/mira-
import Accordion from 'primevue/accordion';
import AccordionTab from 'primevue/accordiontab';
import Button from 'primevue/button';
import Divider from 'primevue/divider';
import { DistributionType, DistributionTypeLabel, distributionTypeOptions } from '@/services/distribution';
import InputNumber from 'primevue/inputnumber';
import Dropdown from 'primevue/dropdown';
Expand Down Expand Up @@ -236,28 +231,41 @@ const onUpdateDistributions = () => {
<style scoped>
ul {
flex-grow: 1;

li {
list-style: none;
}

li + li {
border-top: 1px solid var(--gray-300);
margin-top: var(--gap-1-5);
padding-top: var(--gap-4);
}

li:last-child {
margin-bottom: var(--gap-4);
}
}

:deep(.p-divider) {
&.p-divider-horizontal {
margin-top: 0;
margin-bottom: var(--gap);
color: var(--gray-300);
.stratified {
ul {
border-left: 1px solid var(--gray-300);
margin-left: var(--gap-2);
padding-left: var(--gap-4);
}
&.p-divider-vertical {
margin-left: var(--gap-small);
margin-right: var(--gap);

li {
display: flex;
gap: var(--gap-4);
}
}

.artifact-amount {
font-size: var(--font-caption);
color: var(--text-color-subdued);
margin-left: 0.25rem;
margin-left: var(--gap-1);
}

:deep(.uncertainty-percentage) > input {
width: 4rem;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@
<Button label="Save as..." outlined severity="secondary" @click="showSaveModal = true" />
<Button class="mr-3" :disabled="isSaveDisabled" label="Save" @click="onSaveConfiguration" />
</template>

<Accordion multiple :active-index="[0, 1]">
<AccordionTab>
<template #header>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ public ResponseEntity<ModelConfiguration> getModelConfiguration(
if (modelConfiguration.isEmpty()) {
throw new ResponseStatusException(HttpStatus.NOT_FOUND, messages.get("modelconfig.not-found"));
}

return ResponseEntity.ok(modelConfiguration.get());
} catch (final Exception e) {
log.error("Unable to get model configuration from postgres db", e);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,11 @@ public InitialSemantic clone() {
clone.expressionMathml = this.expressionMathml;
return clone;
}

@Override
public void setSource(String source) {
if ((this.expression != null || this.expressionMathml != null) && this.getSource() != null) {
super.setSource(source);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,7 @@ public class ModelConfiguration extends TerariumAsset {
* This field is only populated if simulationId is not null, it is meant as a sampling of the
* configured space, but not necessarily the true distributions. It will set once and should
* be readonly afterward.
*
* We will designated a dummy distribution type
* We will designate a dummy distribution type
* {
* type: 'inferred',
* parameters: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,11 @@ public ParameterSemantic clone() {
clone.isDefault = this.isDefault;
return clone;
}

@Override
public void setSource(String source) {
if (!this.getDistribution().getParameters().isEmpty() && this.getSource() != null) {
super.setSource(source);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -96,66 +96,6 @@ public ProvenanceSearchResult concept(final ProvenanceQueryParam payload) {
}
}

/**
* Identifies the document from which a model configuration was extracted
*
* @param payload - Search param payload.
* @return
*/
public Set<String> modelConfigFromDocument(final ProvenanceQueryParam payload) {
if (payload.getRootType() != ProvenanceType.MODEL_CONFIGURATION) {
throw new IllegalArgumentException(
"Document used for model-configuration extraction can only be found by providing a model-confirguration"
);
}

try (final Session session = neo4jService.getSession()) {
final UUID modelConfigurationId = payload.getRootId();

final String query = String.format(
"MATCH (d:Document)<-[r:EXTRACTED_FROM]-(m:ModelConfiguration {id: '%s'}) RETURN d",
modelConfigurationId
);

final Result response = session.run(query);
final Set<String> responseData = new HashSet<>();
while (response.hasNext()) {
responseData.add(response.next().get("d").get("id").asString());
}
return responseData;
}
}

/**
* Identifies the dataset from which a model configuration was extracted
*
* @param payload - Search param payload.
* @return
*/
public Set<String> modelConfigFromDataset(final ProvenanceQueryParam payload) {
if (payload.getRootType() != ProvenanceType.MODEL_CONFIGURATION) {
throw new IllegalArgumentException(
"Dataset used for model-configuration extraction can only be found by providing a model-confirguration"
);
}

try (final Session session = neo4jService.getSession()) {
final UUID modelConfigurationId = payload.getRootId();

final String query = String.format(
"MATCH (d:Dataset)<-[r:EXTRACTED_FROM]-(m:ModelConfiguration {id: '%s'}) RETURN d",
modelConfigurationId
);

final Result response = session.run(query);
final Set<String> responseData = new HashSet<>();
while (response.hasNext()) {
responseData.add(response.next().get("d").get("id").asString());
}
return responseData;
}
}

/**
* Identifies the document from which a model was extracted
*
Expand Down Expand Up @@ -258,7 +198,7 @@ public static List<ProvenanceEdge> filterRelationshipTypes(
final List<ProvenanceEdge> relationships,
final List<ProvenanceType> includedTypes
) {
if (includedTypes.size() == 0) {
if (includedTypes.isEmpty()) {
return relationships;
}
final List<ProvenanceEdge> clipped = new ArrayList<>();
Expand All @@ -276,7 +216,7 @@ public static List<ProvenanceNode> filterNodeTypes(
final List<ProvenanceNode> nodes,
final List<ProvenanceType> includedTypes
) {
if (includedTypes.size() == 0) {
if (includedTypes.isEmpty()) {
return nodes;
}
final List<ProvenanceNode> res = new ArrayList<>();
Expand All @@ -301,25 +241,11 @@ public static List<ProvenanceNode> formattedNodes(final Graph graph) {
nodes.add(formatted);
} catch (final NoSuchElementException e) {
log.warn("No element found: " + e);
continue;
}
}
return nodes;
}

public static String dynamicRelationshipDirection(final String direction, final String relationshipType) {
switch (direction) {
case "all":
return String.format("-['%s']-", relationshipType);
case "child":
return String.format("<-['%s']-", relationshipType);
case "parent":
return String.format("-['%s']->", relationshipType);
default:
throw new IllegalArgumentException("Relationship direction is not allowed.");
}
}

public String matchNodeBuilder(final ProvenanceType nodeType) {
final String nodeTypeCharacter = returnNodeAbbr(nodeType);
return String.format("MATCH (%s:%s)", nodeTypeCharacter, nodeType);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,15 @@
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import lombok.Data;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import software.uncharted.terarium.hmiserver.models.dataservice.dataset.Dataset;
import software.uncharted.terarium.hmiserver.models.dataservice.model.Model;
import software.uncharted.terarium.hmiserver.models.dataservice.model.configurations.ModelConfiguration;
import software.uncharted.terarium.hmiserver.models.dataservice.modelparts.ModelParameter;
Expand All @@ -18,6 +21,7 @@
import software.uncharted.terarium.hmiserver.models.dataservice.provenance.ProvenanceRelationType;
import software.uncharted.terarium.hmiserver.models.dataservice.provenance.ProvenanceType;
import software.uncharted.terarium.hmiserver.models.task.TaskResponse;
import software.uncharted.terarium.hmiserver.service.data.DatasetService;
import software.uncharted.terarium.hmiserver.service.data.ModelConfigurationService;
import software.uncharted.terarium.hmiserver.service.data.ModelService;
import software.uncharted.terarium.hmiserver.service.data.ProvenanceService;
Expand All @@ -34,6 +38,7 @@ public class ConfigureFromDatasetResponseHandler extends TaskResponseHandler {
private final ModelService modelService;
private final ModelConfigurationService modelConfigurationService;
private final ProvenanceService provenanceService;
private final DatasetService datasetService;

@Override
public String getName() {
Expand Down Expand Up @@ -85,21 +90,36 @@ public TaskResponse onSuccess(final TaskResponse resp) {
ScenarioExtraction.setNullDefaultModelParameters(modelCopy);

final JsonNode condition = configurations.getResponse().get("values");
final List<ModelParameter> modelParameters = ScenarioExtraction.getModelParameters(condition, modelCopy);
final List<Initial> modelInitials = ScenarioExtraction.getModelInitials(condition, modelCopy);

if (modelCopy.isRegnet()) {
modelCopy.getModel().put("parameters", objectMapper.convertValue(modelParameters, JsonNode.class));
modelCopy.getModel().put("initials", objectMapper.convertValue(modelInitials, JsonNode.class));
// Map the parameters values to the model
if (condition.has("parameters")) {
ScenarioExtraction.getModelParameters(condition.get("parameters"), modelCopy);
}

// Map the initials values to the model
if (condition.has("initials")) {
ScenarioExtraction.getModelInitials(condition.get("initials"), modelCopy);
}

// Fetch the dataset names
final List<String> datasetsNames = new ArrayList<>(List.of());
props.datasetIds.forEach(datasetId -> {
final Optional<Dataset> dataset = datasetService.getAsset(datasetId, ASSUME_WRITE_PERMISSION_ON_BEHALF_OF_USER);
dataset.ifPresent(value -> datasetsNames.add(value.getName()));
});
final String source = String.join(", ", datasetsNames);

// Create the new configuration
final ModelConfiguration modelConfiguration = ModelConfigurationService.modelConfigurationFromAMR(
modelCopy,
"New configuration from dataset",
""
"Configuration from dataset(s)",
"This configuration was created from the dataset(s): " + source
);

// Update the source of the model-configuration with the datasets names
modelConfiguration.getInitialSemanticList().forEach(initial -> initial.setSource(source));
modelConfiguration.getParameterSemanticList().forEach(parameter -> parameter.setSource(source));

try {
for (final UUID datasetId : props.datasetIds) {
final ModelConfiguration newConfig = modelConfigurationService.createAsset(
Expand Down
Loading
Loading