Update docs: add lib development guide #178

JoeyHwong-gk · 2021-08-30T06:55:08Z

add lib development guide
Supplementary quickstart documentation
fix known issue

Signed-off-by: JoeyHwong joeyhwong@gknow.cn

JoeyHwong-gk · 2021-08-30T06:55:21Z

/assign @llhuii @jaypume

JoeyHwong-gk · 2021-08-30T06:55:36Z

/cc @ugvddm @TymonXie @JimmyYang20

ugvddm · 2021-08-30T11:15:07Z

examples/storage/s3/incremental_learning/README.md

-  SECRET_ACCESS_KEY: XXXXXXXX
+stringData:
+  ACCESS_KEY_ID: Q3AM3UQ867SPQQA43P2F
+  SECRET_ACCESS_KEY: zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG


please remove a specific account information, and replace it by {USER_AK}.

sorry about this mistake. I have fix the rebase error

ugvddm · 2021-08-30T11:22:24Z

pkg/apis/sedna/v1alpha1/incrementallearningjob_types.go

@@ -96,6 +96,15 @@ type InitialModel struct {

 type DeployModel struct {
 	Name string `json:"name"`
+	// HotUpdateEnabled will enable the model hot update feature if its value is true.


Please create a new pr, don't mix into one pr with docs update

sorry about this mistake. I have fix the rebase error

MooreZheng

The writing of Quick Start still needs more efforts to make it more straightforward and reader-friendly.

MooreZheng · 2021-08-30T10:36:32Z

docs/quickstart.md


-### Prepare
+- Install Sedna on a cluster Step By Step: [guide here](setup/install.md).
+- Install Sedna AllinOne : [guide here](setup/local-up.md).


Need to briefly explain what is the difference between the "step by step" and "AllinOne".

Do users really need to read "step by step" first before "AllinOne"? If so, it should be stated. If not, we shall make "AllinOne" self-contained: that is because readers may directly clicked into this link without reading and conducting "step by step".

Do users really need to read "step by step" first before "AllinOne"
In the previous summary, the document explains that sedna provides two ways。

MooreZheng · 2021-08-30T10:45:29Z

docs/quickstart.md


-todo
+### Component


A "Quick-start" is suggested to be "quick". The introductions of components and system designs take too much space here which are in fact not necessary to run a increamental learning job.

If we need to use some notations in the following paragraphs, e.g., local controller, it would be much smoother to introduce them until they are used, instead of putting all notations at the beginning, forcing them to remember all these. People usually forget, and they hate to read too much in a "Quick-start".

If we must put some notations here, the writing could be much more straightforward by providing some notation links to the "Introduction" sections.

After understanding of the positioning of components in sedna can help users (not only developers) know their personal responsibilities during application development, and what and why it is necessary to be split during development

Same opinion, the introductions of framework really distracted non-k8s users, and it is impossible to understand by a beginner in this stage.

MooreZheng · 2021-08-30T11:04:46Z

docs/quickstart.md

+
+#### 1.1 Deployment Planning
+
+In this example, there is only one host with two nodes, which had creating a Kubernetes cluster with `kind`.


kind is unfamiliar to non-k8s users. Besides, do we really need to mention Kubernetes cluster and kind here, given that Sedna is already deployed?

If not
Remove the sentence of "which had creating a Kubernetes cluster with kind" (and put a link to "Deploying Sedna" when needed).

If so
2.1. Notation explaination is needed.
2.2. Shall it be "which is a Kubernetes cluster created with kind." instead of "which had creating a Kubernetes cluster with kind"?

In the prepare chapter, kind is one of the ways to install sedna. This section describes the service environment in our demo which prepared by kind.

MooreZheng · 2021-08-30T11:10:30Z

docs/quickstart.md

+##### 2.3.1 Encapsulate an Estimators 
+
+Sedna implements several pre-made Estimators in [example](/examples), your can find them from the python scripts called `interface`.
+Sedna supports the Estimators build from popular AI frameworks, such as TensorFlow, Pytorch, PaddlePaddle, MindSpore. Also Custom estimators can be used according to our interface document. 


Custom estimator is important, but it can take quite a few effort in the "Quick Start" and not needed to detail it with so much space here: a link is already sufficient.

Custom estimator is important, but it can take quite a few effort in the "Quick Start" and not needed to detail it with so much space here: a link is already sufficient.

estimator is an abstract concept that's been introduced by sedna, without detail, the fear is that it will not be understood.

MooreZheng · 2021-08-30T11:13:17Z

docs/quickstart.md

+
+```
+
+- `Context` provides the capability of obtaining the context from CRD


CRD is unfamiliar to non-k8s users. A brief notation explaination or link is needed.

ok, I will replace it with an internal link

MooreZheng · 2021-08-30T11:17:09Z

docs/quickstart.md

+     train_data.y.append(item["label"].asnumpy())
+```
+
+-  `sedna.core` contain all edge-cloud features,  Please note that each feature has its own parameters.


Typo. Please -> please

"Each feature has its own parameters." It is confusing for a machine-learning developer, taking seconds to learn what is happening here. We do not mean the machine-learning feature of samples, right? Just say the incremental-learning job.

feature here means incremental-learning , lifelong-learning... In different feature, the initial parameters is need to pay attention. I'm eager for your opinion on "how to avoid the misunderstanding that users think this is a public parameter".

MooreZheng · 2021-08-30T11:26:24Z

docs/quickstart.md

+
+You can also find demos [here](/examples/incremental_learning/helmet_detection).
+
+Some interfaces should be learn in job pipeline:


The sentence has some thing wrong and hard to understand. We mean that there are APIs here?

Here are the APIs that require the minimum necessary of developers

MooreZheng · 2021-08-30T11:32:33Z

docs/quickstart.md

+This image is generated by the script [build_images.sh](/examples/build_image.sh), used for creating training, eval and inference worker.  
+
+##### 3.2 Create Incremental Job
+In this example, `$WORKER_NODE` is a custom node, you can fill it which you actually run.  


Typo:
In this example, $WORKER_NODE is a custom node, you can fill it which you actually run.
->
In this example, $WORKER_NODE is an environment variable indicating a customized node: you can fill its value with the one you actually run.

Copy from examples， shall we need to modify on both sides

MooreZheng · 2021-08-30T11:33:39Z

docs/quickstart.md

+### 4. Run
+
+* incremental learning supports hot model updates and cold model updates. Job support 
+cold model updates default. If you want to use hot model updates, please to add the following fields:  


Typo:
Job support cold model updates default.
->
Job support cold model updates by default.

Copy from examples， shall we need to modify on both sides

MooreZheng · 2021-08-30T11:35:42Z

examples/incremental_learning/helmet_detection/README.md

 ```

 This image is generated by the script [build_images.sh](/examples/build_image.sh), used for creating training, eval and inference worker.  

 ### Create Incremental Job
-in this example, `$WORKER_NODE` is a custom node, you can fill it which you actually run.  
+In this example, `$WORKER_NODE` is a custom node, you can fill it which you actually run.  


Typo:
In this example, $WORKER_NODE is a custom node, you can fill it which you actually run.
->
In this example, $WORKER_NODE is an environment variable indicating a customized node: you can fill its value with the one you actually run.

sorry about this mistake. I have fix the rebase error

- add lib development guide - Supplementary quickstart documentation - fix known issue Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

llhuii · 2021-09-15T09:35:38Z

docs/quickstart.md

+
+This image is generated by the script [build_images.sh](/examples/build_image.sh), used for creating training, eval and inference worker.  
+
+##### 3.2 Create Incremental Job


Why do you duplicate the example of incremental job here? How about just linking here?

This document describes how to quickly develop an application based on sedna. After scripting is complete, the following describes how to run. Diff to example, we removed requirements and results.

llhuii · 2021-09-15T09:38:04Z

docs/quickstart.md

+- `datasources` base class, as that core feature of sedna require identifying the features and labels from data input, we specify that the first parameter for train/evaluate of the ML framework
+
+```python
+from sedna.datasources import BaseDataSource


Too many code here, and look like the quick start of lib.

Can our community provide a description or discussion: what information should be included in quick start ?

llhuii · 2021-09-15T09:41:10Z

docs/quickstart.md

+
+### 1. Prepare
+
+#### 1.1 Deployment Planning


The Deployment section is duplicate with https://github.com/kubeedge/sedna/blob/main/docs/setup/install.md
Better to let user just refer it.

jaypume · 2021-11-05T02:33:20Z

/lgtm

JoeyHwong-gk · 2021-11-05T03:01:18Z

/hold

JoeyHwong-gk · 2021-11-09T06:52:19Z

/unhold

JoeyHwong-gk · 2021-11-09T06:52:34Z

/ping @jaypume

jaypume · 2021-11-09T07:00:18Z

/lgtm

jaypume · 2021-11-09T07:07:53Z

/approve

kubeedge-bot · 2021-11-09T07:08:00Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jaypume

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jaypume]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kubeedge-bot requested review from JimmyYang20 and TymonXie August 30, 2021 06:55

kubeedge-bot assigned jaypume and llhuii Aug 30, 2021

kubeedge-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Aug 30, 2021

kubeedge-bot requested a review from ugvddm August 30, 2021 06:55

JoeyHwong-gk force-pushed the docs branch from 142f94b to 3faf253 Compare August 30, 2021 09:57

kubeedge-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 30, 2021

ugvddm reviewed Aug 30, 2021

View reviewed changes

MooreZheng reviewed Aug 30, 2021

View reviewed changes

JoeyHwong-gk force-pushed the docs branch from 3faf253 to 94ee48f Compare August 30, 2021 11:54

kubeedge-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 30, 2021

Update docs

8b782ae

- add lib development guide - Supplementary quickstart documentation - fix known issue Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

JoeyHwong-gk force-pushed the docs branch 2 times, most recently from 5702b63 to 8b782ae Compare September 2, 2021 13:12

llhuii reviewed Sep 15, 2021

View reviewed changes

kubeedge-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 5, 2021

kubeedge-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 5, 2021

kubeedge-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 9, 2021

kubeedge-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 9, 2021

kubeedge-bot merged commit 1d0f39b into kubeedge:main Nov 9, 2021

MooreZheng mentioned this pull request May 5, 2022

REQUEST: New membership for MooreZheng kubeedge/community#76

Closed

7 tasks

JoeyHwong-gk mentioned this pull request Sep 18, 2023

REQUEST: New membership for <JoeyHwong-gk> kubeedge/community#51

Closed

6 tasks


		#### 1.1 Deployment Planning

		In this example, there is only one host with two nodes, which had creating a Kubernetes cluster with `kind`.


		```

		- `Context` provides the capability of obtaining the context from CRD


		You can also find demos [here](/examples/incremental_learning/helmet_detection).

		Some interfaces should be learn in job pipeline:


		This image is generated by the script [build_images.sh](/examples/build_image.sh), used for creating training, eval and inference worker.

		##### 3.2 Create Incremental Job

Update docs: add lib development guide #178

Update docs: add lib development guide #178

Conversation

JoeyHwong-gk commented Aug 30, 2021

JoeyHwong-gk commented Aug 30, 2021

JoeyHwong-gk commented Aug 30, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MooreZheng left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoeyHwong-gk Aug 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoeyHwong-gk Aug 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoeyHwong-gk Aug 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MooreZheng Aug 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MooreZheng Aug 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MooreZheng Aug 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jaypume commented Nov 5, 2021

JoeyHwong-gk commented Nov 5, 2021

JoeyHwong-gk commented Nov 9, 2021

JoeyHwong-gk commented Nov 9, 2021

jaypume commented Nov 9, 2021

jaypume commented Nov 9, 2021

kubeedge-bot commented Nov 9, 2021

MooreZheng left a comment •

edited

Loading

JoeyHwong-gk Aug 30, 2021 •

edited

Loading

JoeyHwong-gk Aug 30, 2021 •

edited

Loading

JoeyHwong-gk Aug 30, 2021 •

edited

Loading

MooreZheng Aug 30, 2021 •

edited

Loading

MooreZheng Aug 30, 2021 •

edited

Loading

MooreZheng Aug 30, 2021 •

edited

Loading