Configure steps from file

[dominikjeske]

I would like to share my customization idea for nbomber. Idea is to use CustomSettings as array of steps that can be configured. Each can have its own section format in 'StepSettings'. Step is implemented as plugin with common interface and can deserilize its own config section.

"ScenariosSettings": [
      {
        "ScenarioName": "token_scenario",
        "WarmUpDuration": "00:00:05",

        "LoadSimulationsSettings": [
          { "KeepConstant": [ 4, "00:00:10" ] }
        ],

        "CustomSettings": [
          {
            "Name": "Test",
            "Type": "HttpClient",
            "TimeOut": "00:00:05",
            "CsvFeedPath": "C:\\_users.csv",
            "StepSettings": {
              "Uri": "https://10.23.3.85:5001/connect/token",
              "Method": "POST",
              "ContentType": "UrlEncoded",
              "GenerateTraceId": true,
              "Content": {
                "username": "<username>",
                "password": "<password>"
              },
              "ClientCertificate": {
                "CertificatePath": "xxx.pfx",
                "CertificatePassword": "password"
              }
            }
          }
        ]
      }
    ],

Test runner could be generic and use config section as whole configuration - so it is creating scenarios and steps. Each step is created from factory (I had to do dirty hack to read config before scenario was created). In final solution step could be loaded from separate dll like plugin and no recompilation should be needed - only steps plugins and config file.

public async Task Run(TestOptions testOptions)
        {
            var stepFactory = new StepFactory();

            var scenariosConfig = GetScenarios(configPath);
            var scenarios = new List<Scenario>();

            foreach (var scenarioConfig in scenariosConfig)
            {
                var steps = new List<IStep>();
                foreach (var stepConfig in scenarioConfig.Steps)
                {
                    var step = await stepFactory.Create(stepConfig);
                    steps.Add(step);
                }

                scenarios.Add(ScenarioBuilder.CreateScenario(scenarioConfig.Name, steps.ToArray()));
            }

            var context = NBomberRunner.RegisterScenarios(scenarios.ToArray())
                                                   .LoadConfig(configPath);

            var stats = context.Run();
        }

        private IEnumerable<ScenarioDescription> GetScenarios(string path)
        {
            var dummyContext = new NBomberContext("", "", null, null, null, null, null, null, null, false);
            var config = NBomberRunner.LoadConfig(dummyContext, path);

            return config.NBomberConfig.Value.GlobalSettings.Value.ScenariosSettings.Value.Select(x => new ScenarioDescription
            {
                Name = x.ScenarioName,
                Steps = JsonSerializer.Deserialize<StepDescription[]>(x.CustomSettings.Value).ToList()
            });
        }
internal class StepFactory
    {
        private Dictionary<string, IStepPlugin> _plugins = new Dictionary<string, IStepPlugin>();

        public StepFactory()
        {
            //TODO load at runtime as plugin
            _plugins.Add("HttpClient", new HttpStepPlugin());
        }

        public async Task<IStep> Create(StepDescription stepDescription)
        {
            var plugin = _plugins[stepDescription.Type];
            var step = await plugin.Create(stepDescription);
            return step;
        }
    }

[AntyaDev]

Hi @dominikjeske
Can you pls show your dirty hack on how do you apply custom settings?

[AntyaDev]

I suggest not structuring config via step hierarchy
better create a simple DTO with named settings like:
{
MongoConnection: string
TargetHostUrl1: string
TargetHostUrl2: string
RetryCount: int
}

[dominikjeske]

My final approach to explain the point ;)

 public class CustomSettingsExample
    {
        static CustomScenarioSettings _customSettings = new CustomScenarioSettings();

        static Task ScenarioInit(IScenarioContext context)
        {
            // 3. Here we have access to config but is is to late - steps are already created. Beside this even if I would read them here the only way to pass config to step is by global static value
            _customSettings = context.CustomSettings.Get<CustomScenarioSettings>();

            return Task.CompletedTask;
        }

        public static void Run()
        {
            var step = Step.Create("step", async context =>
            {
                await Task.Delay(TimeSpan.FromSeconds(0.1));
                return Response.Ok();
            });

            var customPause = Step.CreatePause(() => _customSettings.PauseMs);

            //1. To create scenario below I need: 'my_scenario', step, customPause - to compose everything from config 
            // I need to cerate steps here basing on config which I don't have and not like above by hardcoding them in code. 
           // I need to have config and read scenario name and list of steps and then create them 
          // in order to pass to scenario builder.
            var scenario = ScenarioBuilder
                .CreateScenario("my_scenario", step, customPause)
                .WithInit(ScenarioInit);

            //2. Here we read NBomber config but still have co access to Custom section
            var context = NBomberRunner
                .RegisterScenarios(scenario)
                .LoadConfig("./config.json");


            context.Run();
        }
    }

[AntyaDev]

Hi @dominikjeske
Did you plan to create steps on the fly? I mean, autogenerate them based on config?

[dominikjeske]

Yes 👍. Keeping this in mind. My code

    ```
      var stepFactory = new StepFactory();
       var scenariosConfig = GetScenarios(configPath);  // 1. Read config on start (including steps)
       var scenarios = new List<Scenario>();

        // 2. Iterate over scenarios
        foreach (var scenarioConfig in scenariosConfig)
        {
            var steps = new List<IStep>();
            // 3. Iterate over scenarios
            foreach (var stepConfig in scenarioConfig.Steps)
            {
                // 4. dynamicly create step from config
                var step = await stepFactory.Create(stepConfig);
                steps.Add(step);
            }

            scenarios.Add(ScenarioBuilder.CreateScenario(scenarioConfig.Name, steps.ToArray()));
        }

        5. Run code (from here I can use existing API as is)
        var context = NBomberRunner.RegisterScenarios(scenarios.ToArray())
                                               .LoadConfig(configPath);

        var stats = context.Run();

[AntyaDev]

Hi @dominikjeske
That's what I wanted to get from you when asked:

Explain please what you want to run?

Because now it's clear:) In short, NBomber was not designed for this, and such usage is what we call - technology abuse. I would argue with you about this approach since: it's very implicit and hard to debug, analyze, trace, etc.

In addition, your proposition about steps hierarchy is bringing accidental complexity into NBomber. It actually remembered me how bad is OOO and inheritance:)