Running JMeter, Gatling and Locust tests as they are

If you already have JMeter plans, Gatling simulations or Locust files, you can run them without converting them to Spitfire tests. The test runs on its own engine, on runners started from that engine's image. The results are reported in Spitfire like any other run: every sampler or request shows up as a step, and thresholds, comparison reports and notifications work as usual.

#Turning an engine on (administrator)

External engines are off by default. An uploaded JMeter plan (JSR223, BeanShell, OS Process), Gatling simulation or locustfile is a program: it runs with the runner's permissions and can reach whatever the runner can.

1. An installation administrator opens the External engines card on the Runners In Spitfire: /runners page.
2. They turn each engine to be used (JMeter, Gatling, Locust) on separately.
3. On saving they confirm the warning "these tests can run code on the runner". The confirmation and the change are written to the audit log (engines.settings).

While an engine is off, its files cannot be uploaded and its tests cannot start.

#Runner images

Each engine has its own runner image: the engine's name is appended to the runner image's tag.

A runner started from one of these images finds the engine itself and advertises it with a label such as engine-jmeter. External engine tests go only to runners with that label; other tests can run on those runners too. With on-demand runners on Kubernetes, the run starts pods from the engine's image. For another image name set SPITFIRE_ONDEMAND_IMAGE_JMETER, SPITFIRE_ONDEMAND_IMAGE_GATLING or SPITFIRE_ONDEMAND_IMAGE_LOCUST.

#Nothing is downloaded from the internet

During a run the runner downloads nothing from the internet:

• JMeter: plugins are uploaded with the test as jars. The image has no Plugins Manager.
• Gatling: the compiled project is uploaded (classes or a jar, plus the dependency jars). Nothing is compiled on the runner.
• Locust: the packages in requirements.txt are installed only from the wheel (.whl) files uploaded with the test, or from the internal PyPI mirror the administrator set on the External engines card.

#Creating the test

1. On the Tests In Spitfire: /tests page, press JMeter / Gatling / Locust.
2. On the Scenarios tab, pick the engine.
3. Upload the test with the files it needs, with Upload files or Upload a folder:
   • JMeter: the .jmx plan, the CSV files it reads and the plugin jars.
   • Gatling: the target/ folder after mvn package, or a zip of it.
   • Locust: the locustfile, requirements.txt and the wheel files.
4. Spitfire reads the files and shows a summary: the scenarios (thread groups, simulations, user classes), the target addresses, the users, the ramp-up and the duration. When there is more than one plan or simulation, you pick the one to run.
5. Read the summary's warnings: elements that run code, a CSV file that was not uploaded, plugins without their jars and the like.

At most 100 MB and 5000 files can be uploaded. The files are kept in your installation's database and sent only to the runners that take the run.

#Load: Spitfire's or the file's

There are two choices:

• Spitfire's load (the default): the scenario's load gives the users, ramp-up and duration. In JMeter the thread groups run with these values. In Gatling the simulation must read the spitfire.users, spitfire.rampUp and spitfire.duration system properties. Locust gets -u, -r and -t.
• Keep the file's load: the thread groups, injection profile or LoadTestShape in the file drive the load as written. When the JMeter plan's load can be read, the switch fills the scenario's load with the file's.

Either way the scenario's VUs and duration are the ceiling. The license's VU limit applies to concurrent users and its duration limit to the run time. When the measured concurrent users go above the ceiling or the time is up, the runner stops the engine and the run ends with the reason.

Properties go to the engine as they are: to JMeter as -J, to Gatling as -D (Java system properties), to Locust as environment variables.

#Results

The runner reads the engine's result file (JMeter JTL, Gatling simulation.log, Locust request events) and turns it into Spitfire metrics. The step name is the sampler's or the request's name (at most 200 different names per test; the rest are summed in one row). Response times, error rate and request rate are reported as in other runs, and thresholds apply to them.

#Limits

• Try (a one-VU trial) is not available for external engine tests; save the test and start a short run.
• Comparison runs (A/B, sequential) do not support external engine tests yet.
• A test has one scenario; Spitfire steps, data files and the adaptive guard cannot be added.
• Tests that run code run with the runner's permissions. Run the runners on a network of their own, limited by network policies, apart from the systems you test.