cyber performance test plan

[daohu527]

Testing purposes

Autonomous driving requires middleware to have good performance and stability. We decided to test cyber, avoiding detailed understanding of cyber's performance and stability, and listing detailed indicators.

Test environment

• OS: Ubuntu 18.04
• Hardware: X86 industrial computer (16 cores 32G)
• Version: Apollo 9.0

Testcase

Overall test

• cyber memory, CPU usage
• Single node (writer\reader) memory and CPU usage
• Single service memory and CPU usage

Performance test

• 1K, 2K, 1M size message sending and receiving latency (shared memory)
• 1K, 2K, 1M size message sending and receiving latency (zero-copy)
• 1K, 2K, 1M size message sending and receiving latency (network)
• Multiple nodes send and receive message latency (multiple readers)
• Multiple nodes send and receive message latency (multiple writers)
• Multiple timer trigger accuracy
• Node startup time
• Service delivery latency

Stability test

• 1K, 2K, 1M size message sending and receiving packet loss rate (shared memory)
• 1K, 2K, 1M size message sending and receiving packet loss rate (zero-copy)
• 1K, 2K, 1M size message sending and receiving packet loss rate (network)
• Multiple nodes send and receive message packet loss rate (multiple readers)
• Multiple nodes send and receive message packet loss rate (multiple writers)
• Node shutdown and startup failure rate
• Service shutdown and startup failure rate
• Service sending packet loss rate

Tools test

• cyber_recorder recording performance
• cyber_recorder parsing performance
• cyber_recorder packet broadcast performance

Pressure test

• Task real-time test
• Maximum message bandwidth
• Maximum number of timers on a single core
• Maximum number of services on a single core