Update README With UDP Demo Steps #506
Conversation
Docs - Updating Data Flow Diagrams
|
Ready for review @dan-du-car |
README.md
Outdated
| With the PPM module running, all filtered BSMs that are uploaded through the web interface will be captured and processed. You will see an output of both submitted BSM and processed data unless the entire record was filtered out. | ||
|
|
||
| <b>For testing decoding with all supported ODE message types:</b> | ||
|
|
||
| To test decoding all supported ODE messages, the UDP endpoints must be utilized. These endpoints specifically take hex ASN1 encoded message data. These messages are allowed to have headers but do not need to. |
There was a problem hiding this comment.
Is this hex ASN.1 encoding UPER encoded? or?
There was a problem hiding this comment.
Whichever one the ASN.1 library supports. Not actually certain. The headers are trimmed out at the moment so the type of header isn't super important. How would you like me to specify this?
There was a problem hiding this comment.
@dan-du-car I have gone ahead and made adjustments to specify these.
| @@ -20,6 +20,8 @@ The purpose of these diagrams is to show: | |||
| 1. The Asn1DecodeDataRouter class pulls from the Asn1DecodeOutput topic and deposits messages into the Pojo Messages group of topics and the Json Messages group of topics. | |||
| 1. The PPM pulls from the Json Messages group of topics and pushes filtered messages to the Filtered Json Messages group of topics. | |||
| 1. The FileUploadController class pulls from the Json Messages & Filtered Json Messages groups of topics and offloads them. | |||
| 1. The GeoJSON Converter pulls from the Json Messages group of topics, converts the messages and pushes them to the Processed Spat/Map group of topics. | |||
| 1. The Conflict Monitor pulls from the Processed Map/Spat group of topics and pushes to the Conflict Monitor Output Topics group. | |||
|
|
|||
| ### Overview Data Flow 2 | |||
There was a problem hiding this comment.
Do we need bullet point 9 and 10 at "Overview Data Flow 2"?
There was a problem hiding this comment.
Yes we do! These have been added.
There was a problem hiding this comment.
Sounds good. Could we update the diagram to represent the rest of the steps? It looks like the diagram ends at step 8.
There was a problem hiding this comment.
Step 1 involves pushing the messages to the Json Messages group of topics. Once step 8 concludes and the data flows into the SDX, step 9 focuses on what occurs to the messages pushed to that group of topics after the initial step.
docs/data-flow-diagrams/README.md
Outdated
| @@ -20,6 +20,8 @@ The purpose of these diagrams is to show: | |||
| 1. The Asn1DecodeDataRouter class pulls from the Asn1DecodeOutput topic and deposits messages into the Pojo Messages group of topics and the Json Messages group of topics. | |||
| 1. The PPM pulls from the Json Messages group of topics and pushes filtered messages to the Filtered Json Messages group of topics. | |||
| 1. The FileUploadController class pulls from the Json Messages & Filtered Json Messages groups of topics and offloads them. | |||
| 1. The GeoJSON Converter pulls from the Json Messages group of topics, converts the messages and pushes them to the Processed Spat/Map group of topics. | |||
There was a problem hiding this comment.
Could we add GitHub hyperlink to each submodule? It is hard to identify/navigate to the submodules in the GitHub repos if someone is not familiar with all the modules pushed to GitHub. For example: ACM is from https://github.com/usdot-jpo-ode/asn1_codec/tree/develop/src. Another example: I do not know where this GeoJSON Converter submodule is located.
There was a problem hiding this comment.
Hyperlinks have been added!
docs/images/readme/figure1.png
Outdated
There was a problem hiding this comment.
Can we leave the old diagram as it or mark it old architecture diagram? I think the old diagram still correctly describes part of the system, but it is not up to date as we are adding more features. Also I do like to know what topics we have in the system, and which topics are used by which submodules. Additionally, In the old diagram, it used the submodule name like S3 depositor and SDC depositor. This submodule names match the GitHub repos we used, and it is easy to understand what roles the GitHub repos play in this TMC system environment. On the other hand, the new diagram has some new names for submodules like situation data exchange, VSD/ISD, SDC etc... I am not sure how to match these to the series of repos we have in the GitHub.
There was a problem hiding this comment.
The previous diagram, named figure1.old.png, has been reintroduced into the docs\images\readme directory.
The purpose of this diagram is to provide an overview of the system, focusing on high-level concepts. The specific relationships between topics and submodules are considered lower-level details and are better represented in data flow diagrams. We can include a low level diagram in the README that contains these details if you would like. This can be a modified version of figure1.old.png or even the ODE Data Flow Overview diagram.
Regarding the submodule names:
- The S3 depositor, which is used for SDC and Firehose, is now positioned on the left side of the Data Export Services section.
- The Situation Data Exchange (SDX) serves as a data destination where the jpo-sdw-depositor submodule deposits its data. We can change the name of the depositor on the diagram to SDW Depositor, but in the future this will likely be changed to SDX Depositor anyway.
- VSD and ISD are types of data, as opposed to being submodules or components.
There was a problem hiding this comment.
Make sense. I guess if we go with high level concepts, in the architecture.md, do we have some brief descriptions about each high-level concepts/components and where those components are located?
I just realized that the user guide also used the architecture diagram: https://github.com/usdot-jpo-ode/jpo-ode/blob/develop/docs/UserGuide.md#3---system-overview. Could you also update that with the new one?
There was a problem hiding this comment.
Four of the submodules are displayed in https://github.com/usdot-jpo-ode/jpo-ode/blob/readme-update/docs/Architecture.md#31---ode-technology-stack, but I don't see any brief descriptions or location indicators at this time.
There was a problem hiding this comment.
The image file being used for figure 1 in the user guide has been updated.
Added figure1.old.png to `docs\images\readme`.
Updated TMC environment diagram used in User Guide.
|
@dan-du-car when you believe this PR is ready, please go ahead and merge it. I don't have permission to do so. Thanks! |
PR Details
Description
Includes updates to the README to provide detail on how to test the decoding of all ODE supported message types through the UDP endpoints. This included modifying the README, adding an image and removing misleading data examples for SPaT.
Related Issue
#481
Motivation and Context
The ODE Demo site only supports decoding BSM and TIM messages. SPaT, MAP, SRM and SSM messages require the UDP endpoints to demonstrate proper decoding. This update will make that clear in the documentation and walk users through how to perform this test.
How Has This Been Tested?
Manually walked through the steps myself and verified they successfully performed the test they describe.
Types of changes
Checklist:
ODE Contributing Guide