clarify TASK docs #550
Merged
clarify TASK docs #550
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
martinjaggi
requested changes
Dec 1, 2022
| loss: 'categoricalCrossentropy', | ||
| metrics: ['accuracy'] | ||
| }, | ||
| modelID: 'my_new_task-model', |
There was a problem hiding this comment.
a bit confused why there are two tasks here, simple-face and my_new_task?
| In this case, your model and task will be uploaded and stored on our DISCO servers. You will have to make the task visible to the API. For your custom model, the JSON model architecture is necessary, but the .bin weight file is optional : if you include the weights file, your model will be loaded with the passed weights. If a weights file is not specified, the weights for the model will be initialized randomly. | ||
| If you want to add a new task to our production DISCO server you have two possibilities: | ||
| * using the user interface as described above | ||
| * exporting your own `TaskProvider` from `discojs/discojs-core/src/default_tasks/` and adding a new default task by contributing to the code. |
There was a problem hiding this comment.
maybe say first one is without any coding required, second one is with describing the ML task in your own code
| export { titanic } from './titanic' | ||
| export { simpleFace } from './simple_face' | ||
| export { myNewTask } as my_new_task from './my_new_task' // <---- including our new custom task! | ||
| export { geotags } from './geotags' |
|
|
||
| ### Making the task visible to the API | ||
| > Note that `discojs-core` must only contain platform-agnostic code that works both in the browser and on Node.js. | ||
| > Thus, if your task requires reading some file from your local file system, you need to define the task in `discojs-node` only: `discojs/discojs-node/src/default_tasks/my_new_task.ts` |
There was a problem hiding this comment.
could you comment on the easiest case first, where is the model usually read from (and mention can be initial model, or pre-trained model)
|
|
||
| > Note, it's important to add the `export` tag to the model as well as to the subsequent objects that we define. | ||
| The interface let you load your model however you want, as long as you return a `tf.LayersModel` at the end. If you use a | ||
| pre-trained model, you can simply load and return said model in the function via `tf.loadLayersModel(modelPath)`. |
There was a problem hiding this comment.
(same comment as above, is the default recommendation some URL, or file-system access on the phone, or server?)
| .set(defaultTasks.simple_face.task.taskID, simpleFaceModelPath) | ||
|
|
||
| ``` | ||
| At runtime, the models are stored in `disco/server/models/`, and it is also in the server side that we let disco know where exactly they are saved. In particular, |
There was a problem hiding this comment.
do you mean the initial models for each task, or even the freshly trained ones? (i guess it's true for FL but not for decentralized?)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I clarified some part of the tasks docs to better reflect the task workflow after the refactoring (
TaskProviderinterface which wrapsTaskandtf.LayersModel).I also tried to remove some duplicated information and fix broken links.