On Study View handle larger number of resource files #5034
Conversation
✅ Deploy Preview for cbioportalfrontend ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
…ej/cbioportal-frontend into reduce-api-calls-files-and-links-tab
| idMap: { [key: string]: ResourceData[] }, | ||
| resource: ResourceData | ||
| ) => { | ||
| const { resourceType } = resource.resourceDefinition; |
There was a problem hiding this comment.
this whole operation would be more clearly handled by lodash's keyBy function. we like to use these kind of named helpers because they make the operation more explicit.
There was a problem hiding this comment.
Hm, I couldn't make lodash keyBy function to work the way I wanted it to for this use case. In python this would be a case for collections.defaultDict. But I couldn't make this work where I could create new arrays and append to them.
LMK if you want me to work on this part some more.
There was a problem hiding this comment.
@hweej sorry, i mean _.groupBy
There was a problem hiding this comment.
I have replaced these lines with a single lodash groupBy('patientId'). Thanks @alisman !
filtering, clean up code superfluous comments.
inodb
changed the title
| const resourcesForPatientsAndSamples = resourcesForEntireStudy | ||
| // Filter the resources to consist of only studyView selected samples | ||
| // Also keep patient level resources (e.g. Those don't have a sampleId) | ||
| .filter( |
There was a problem hiding this comment.
one more thing here: this is operation is Rsamples+Rpatients. could easily get to billion. easily optimized by putting selectedSamples in a map so you can use IN operator.
There was a problem hiding this comment.
I have replaced this with Maps instead of arrays to remove the linear lookup. I don't think you can use the IN operator with Maps so we're opting for the .has('key') expression instead.
filter and reduce with lodash groupBy.
| .filter(resource => | ||
| selectedIds.has(resource.sampleId || resource.patientId) | ||
| ) | ||
| .groupBy('patientId') |
There was a problem hiding this comment.
@hweej one more little tweak. i'm pretty sure that using the shortcut 'patientId' is not typesafe., so as a rule we use
(r)=>r.patientId)
That way if there is no propertyId, ts will catch it.
|
|
||
| // sampleIds (+patientIds) for the selectedSamples | ||
| const selectedIds = new Map([ | ||
| ...new Map(selectedSamples.map(item => [item.sampleId, item.studyId])), |
There was a problem hiding this comment.
this is strange. why instantiate maps inside of the map? i think just
new Map([...selectedSamples.map(xxx)])
Fixes cBioPortal/cbioportal#11004 (associated backend PR is merged, see cBioPortal/cbioportal#11051). This PR modifies the FilesAndLinks table on the Study View to use the new resource-data-all endpoint, which reduces the number of API requests and enables showing thousands of resource files without performance degradation