Tools

Tools for performance analysis.

Benchmark Demo
Latency Demo

Benchmark Demo 

Abstract

This demo sets up a simulation with various command line arguments for benchmarking purposes. A configurable amount of participants are spawned by a single process on different threads. The demo calculates averaged running times, throughput, speed-up and message rates for performance evaluation.

Sources

BenchmarkDemo.cpp

Requirements

None (The demo starts its own instance of the registry and system controller).

Positional Parameters

[numberOfSimulationRuns] Sets the number of simulation runs to perform.
[simulationDuration] Sets the virtual simulation duration <S>.
[simulationStepSize] Sets the virtual simulation step size <T>.
[service] Sets the communication service type (pubsub, can or ethernet).
[numberOfParticipants] Sets the number of simulation participants <N>.
[messageCount] Sets the number of messages <M> to be send in each simulation step.
[messageSizeInBytes] Sets the message size <B>.
[registryURi] The URI of the registry to start.

Optional Parameters

--help Show the help message.
--registry-uri The URI of the registry to start. Default: silkit://localhost:8500
--message-size Sets the message size <B> in bytes. Default: 1000
--message-count Sets the number of messages <M> to be send in each simulation step. Default: 50
--number-participants Sets the number of simulation participants <N>. Default: 2
--number-simulation-runs Sets the number of simulation runs to perform. Default: 4
--simulation-step-size Sets the simulation step size <T> (virtual time). Default: 1ms
--simulation-duration Sets the simulation duration <S> (virtual time). Default: 1s
--service Sets the communication service type (pubsub, can or ethernet). Default: pubsub
--configuration Path and filename of the participant configuration YAML file. Default: empty
--write-csv Path and filename of CSV file with benchmark results. Default: empty

System Examples

Launch the benchmark demo with default arguments but 3 participants:
```
./SilKitDemoBenchmark --number-participants 3
```

Launch the benchmark demo with positional arguments and a configuration file that enforces TCP communication:

./SilKitDemoBenchmark 4 1 1 ethernet 2 1 10 --configuration ./SilKit-Demos/Benchmark/DemoBenchmarkDomainSocketsOff.silkit.yaml

Notes

This benchmark demo produces timings of a configurable simulation setup. <N> participants exchange <M> messages of <B> bytes per simulation step with a simulation step size of <T> ms and run for <S> seconds (virtual time).
This simulation run is repeated <K> times and averages over all runs are calculated. Results for average runtime, speedup (virtual time/runtime), throughput (data size/runtime), message rate (count/runtime) including the standard deviation are printed.
The demo uses publish/subscribe, can or ethernet controllers. In the publish/subscribe case, the same topic for the message exchange is used, so each participant broadcasts the messages to all other participants. The configuration file DemoBenchmarkDomainSocketsOff.silkit.yaml can be used to disable domain socket usage for more realistic timings of TCP/IP traffic. With DemoBenchmarkTCPNagleOff.silkit.yaml, Nagle’s algorithm and domain sockets are switched off.
The demo can be wrapped in helper scripts to run parameter scans, e.g., for performance analysis regarding different message sizes. See .\SilKit-Demos\Benchmark\msg-size-scaling\Readme.md and .\SilKit-Demos\Benchmark\performance-diff\Readme.md for further information.

Latency Demo 