Running JMeter, Gatling, Locust and k6 tests as they are in Spitfire
If you have a test suite that works, you do not need to rewrite it. A JMeter plan (.jmx), a Gatling simulation, a locustfile or a k6 script runs in Spitfire on its own engine: the test runs in that engine's runner image, on your own runners, and the result is reported in Spitfire like any other run. Every sampler or request shows up as a step; thresholds, reports and notifications work as usual. This page covers the steps, how the load is given and the limits.
Convert, or run as it is?
Spitfire offers two ways. Convert: .jmx and k6 files become Spitfire steps with Tests → Open file, and the test is then changed in the editor without code (importing .jmx plans, importing k6 scripts). Run as it is: the file runs unchanged on its own engine; custom code, plugins and custom metrics in it work as they are. Gatling and Locust tests come in this way only.
The Run natively with k6 (or with JMeter) button on the conversion report uploads the same file to the engine and switches the test to this way.
1. Turn the engine on (installation administrator)
External engines are off by default: an uploaded plan or script is a program and runs with the runner's permissions.
- On the Runners page, open the External engines card.
- Turn each engine you will use (JMeter, Gatling, Locust, k6) on separately.
- On saving, confirm the warning "these tests can run code on the runner". The confirmation and the change go into the audit log (
engines.settings).
2. Start a runner from the engine's image
Each engine has its own runner image: the engine's name is appended to the runner image's tag. A runner started from that image finds the engine itself and advertises it with a label such as engine-jmeter; external engine tests go only to runners with that label. With on-demand runners on Kubernetes, the run starts pods from the engine's image (distributed load testing on Kubernetes).
| Engine | Image | What to upload | Contains |
|---|---|---|---|
| JMeter | usespitfire/spitfire:<version>-jmeter | The .jmx plan, the CSV files it reads, the jars of the plugins it uses | JMeter 5.6.3 + Custom Thread Groups |
| Gatling | usespitfire/spitfire:<version>-gatling | The target/ folder after mvn package, or a zip of it (classes or a jar + dependency jars) | The Gatling 3.11 libraries |
| Locust | usespitfire/spitfire:<version>-locust | The locustfile, requirements.txt and the packages' wheel (.whl) files | Locust 2.32 |
| k6 | usespitfire/spitfire:<version>-k6 | The script, its local modules and data files; or a k6 archive (.tar) | k6 1.0 (no browser, no xk6 extensions) |
Nothing is downloaded from the internet during a run. JMeter plugins are uploaded with the test as jars; the Gatling project is uploaded compiled and nothing is compiled on the runner; Locust packages are installed only from the uploaded wheel files or from the internal PyPI mirror the administrator set; in k6 remote imports (https://…) are not downloaded, they are uploaded with the script.
3. Prepare the files
Run the commands on your own machine; replace the /full/path/... parts with the full paths of your files.
Gatling
mvn -f /full/path/project/pom.xml -DskipTests packageIf you want Spitfire to give the load, the simulation must read the spitfire.users, spitfire.rampUp and spitfire.duration system properties (seconds; the duration includes the ramp). If it does not, its own injection profile runs; the scenario's VUs and duration are still the ceiling.
// Simulation (Java DSL): read the load Spitfire gives this runner.
// spitfire.duration is the whole run in seconds, the ramp included.
int users = Integer.getInteger("spitfire.users", 10);
int rampUp = Integer.getInteger("spitfire.rampUp", 30);
int duration = Integer.getInteger("spitfire.duration", 300);
setUp(scn.injectClosed(
rampConcurrentUsers(0).to(users).during(rampUp),
constantConcurrentUsers(users).during(Math.max(duration - rampUp, 1))
)).protocols(httpProtocol);Locust
Download the packages' wheel files, then upload the locustfile, requirements.txt and the wheels folder together:
pip download -r /full/path/requirements.txt -d /full/path/wheels --only-binary=:all:k6
Inspect the script; if it uses local modules or data files, put it all into one archive:
k6 inspect /full/path/script.jsk6 archive --output /full/path/archive.tar /full/path/script.jsJMeter
Nothing to prepare: put the .jmx plan, the CSVs it reads and the plugin jars in one folder.
4. Create the test
- On the Tests page, press JMeter / Gatling / Locust / k6.
- On the Scenarios tab, pick the engine.
- Upload the files with Upload files or Upload a folder (at most 100 MB and 5000 files). The files are kept in your installation's database and sent only to the runners that take the run.
- Read the summary: the scenarios (thread groups, simulations, user classes), the target addresses, the users, the ramp-up and the duration. If there is more than one plan or simulation, pick the one to run.
- Read the warnings: elements that run code, a CSV file that was not uploaded, plugins without their jars and the like.
5. Pick the load: Spitfire's or the file's
Spitfire's load (the default): the scenario's users, ramp-up and duration go to the engine, each runner its share. Keep the file's load: the thread groups, injection profile, LoadTestShape or k6 scenarios in the file drive the load as written (k6 splits it across the runners with --execution-segment). Either way the scenario's VUs and duration are the ceiling: when the measured concurrent users go above it or the time is up, the runner stops the engine and the run ends with the reason.
| Engine | Spitfire's load | Properties |
|---|---|---|
| JMeter | -Jspitfire.users · -Jspitfire.rampUp · -Jspitfire.duration | -J |
| Gatling | -Dspitfire.users · -Dspitfire.rampUp · -Dspitfire.duration | -D |
| Locust | -u · -r · -t | environment variable |
| k6 | --vus · --duration (with a ramp --stage) | -e (__ENV) |
The license's VU limit applies to concurrent users and its duration limit to the run time.
6. Save and start a short run
External engine tests have no Try (a one-VU trial): the engine runs on a runner only, never on the controller. Save the test and start a 1-minute run.
Success check: the run's Steps table lists the plan's samplers or the script's requests by name (at most 200 different names per test; the rest are summed in one row). For a k6 script the Checks table also has the script's checks and this row:
checks rate==1 check=k6 thresholdsWhere the results come from
The runner reads the engine's result file (JMeter JTL, Gatling simulation.log, Locust request events, k6 --out json) and turns it into Spitfire metrics. Response times, percentiles, error rate and request rate are reported as in other runs, and thresholds apply to them. k6's own thresholds come into the Thresholds tab; the ones Spitfire cannot evaluate (custom metrics, other tags) are left to k6. Custom metrics (Trend, Counter, Rate, Gauge) show on the run's Script metrics card.
The saved test starts like any other; to start it from a pipeline, see performance testing in CI/CD.
Limits
- Comparison runs (A/B, sequential) do not support external engine tests yet.
- A test has one scenario; Spitfire steps, data files and the adaptive guard cannot be added.
- k6: browser scenarios (
k6/browser) and xk6 extensions do not run on the runner-k6 image. - Tests that run code run with the runner's permissions. Run the runners on a network of their own, limited by network policies, apart from the systems you test.
Frequently asked questions
Do I have to rewrite my JMeter plan to use it in Spitfire?Tests
Are Gatling and Locust tests converted into Spitfire steps too?Tests
Does the runner go to the internet during a run?Security
Can I split the load across several runners?Tests
--execution-segment. The results come together in one run.Why are external engines off by default?Tests
Spitfire installs on Docker or Kubernetes with one command; every testing feature and protocol is open in the free edition.