Introduction
In any ecommerce, it is necessary to ensure that navigational and transactional services work as expected under high traffic load (this refers to thousands of simultaneous site transactions and visits). This is an important aspect to consider especially during sales periods such as Black Friday, Cyber Monday and Christmas.
The way to guarantee that an ecommerce site performs as expected in these circumstances, is to simulate the concurrent access of visitors in QA or Preprod environments and measure the results. For this purpose, performance tests are carried out.
Pain points in QA testing
One of the tools that allows us to simulate load and perform load testing on an ecommerce system is JMeter. The execution of test sessions provide a log with the results obtained from each round of testing. This can be for example: the response time, number of concurrent users, response code, URL to which the request has been made, etc.
As such, how can this results log be analyzed effectively to obtain a conclusion about the results? The output of results is presented in plain text and can therefore be very difficult to analyze, which is why we have created LongBOARD.
LongBOARD features
So what exactly is LongBOARD?
LongBOARD is a set of dashboards powered and developed by Orienteed; they provide real-time visualizations of the results obtained during the execution of the performance and load tests.
The platform behind the dashboard (Grafana + ElasticSearch + LogStash) is available and deployable with a single click through the AWS Marketplace.
LongBOARD consists of 2 main parts:
- Logstash: This is a pipeline for collecting, filtering and sending the results generated from the execution of the performance/load test.
- 3 dashboards in Grafana where you can visualize, in real time:
- A summary of the results.
- Statistics of the requests.
- An analysis of the errors.
Below is a screenshot of the results summary dashboard for LongBOARD. In this example, you can see a summary of different KPIs that are evaluated within a load test session.
As mentioned above, there are also two additional dashboards that report the requests statistics and the analysis of the errors found.
These dashboards are made up of dynamic graphs. This means that through the use of dynamic filters, you can select the transactions that you want to see. For example, you can filter to view only login requests, requests with a 503 response code, as well as to filter results by the error message, etc.
Using LongBOARD
In this section, we will go into some technical details about the specific steps to configure LongBOARD and the steps to start using it. So if you are interested in understanding how LongBOARD is set up, be sure to read the below.
How to configure the tool?
First of all, the following requirements must be met:
- JMeter: This will be the application through which the tests will be run, and where the results will be generated and collected for LongBoard’s use.
- Deploy our AMI from AWS Marketplace.
How to configure the tool?
1. Create the EC2 instance from the AMI.
Once the instance is created, we will have:
- Logstash: A tool for ingesting, filtering and sending data to OpenSearch.
- Logstash configuration file
- Index Template: This template will tell OpenSearch the type of data to be entered and which indexes it has to create in the database.
- Grafana docker image ready to run.
- Executable file: This file will automatically run Logstash with the necessary configuration parameters.
2. Connect via SSH to the created instance.
3. Run the Grafana image with the following command: $ docker start Grafana
4. Open Grafana in the browser: http://public_instance_ip:3000
- User: admin
- Password: orienteed
5. Configure the data source, via Configuration > Data sources > OpenSearch. Configure the following fields:
- URL: OpenSearch_domain_endpoint:443
- User: openSearch_user
- Password: openSearch_password
- Index name: (the name of the index you put in line 30 of the configuration file to be mentioned later (logstash_csv.fonf)
For example: stress-myWeb-logstash
6. Click on Save & test. Below are some of the possible errors you may find:
- No index exists: this is because logstash has not yet been run and has not created the index.
- The date field called timeStamp does not exist: this is because the index does not have an associated template or is empty.
NOTE: When creating the data source and not having the index created in Opensearch, these errors are normal. It is OK if they appear, as soon as you insert data they will be fixed automatically.
If everything is OK and there are no errors, the following message will appear:
7. Copy all the files located in the root directory of the instance to the directory where JMeter is installed (it may be in another instance, server, or PC).
The following steps will be performed in the new destination of the files, the directory where JMeter is located.
8. Configure the Logstash configuration file. This file will run Logstash with all the necessary configurations.
- Line 3: Here will be the path where the file to be indexed is located. For example: “/home/ec2-user/jmeter/apache-jmeter-5.4.3/bin/results/test_results_myWeb.csv“.
IMPORTANT: In your JMeter performance test, you must have the listener: View Results Tree, and configure the path and file name where the results will be saved. Here is a screenshot of how it might look like:
- Line 27: OpenSearch domain endpoint. The port to use is 443. For example: "OpenSearch_domain_endpoint:443"
- Line 28: Your OpenSearch user
- Line 29: Your OpenSearch password
- Line 30: Name of the index to be created in OpenSearch and where the data to be indexed will be saved. For example: “stress-myWeb-logstash”
How to use it?
Once everything is configured, you only need to perform 3 steps to start using LongBOARD.
- Run logstash: To do this, you simply need to run the following file (from those copied in step 7). It is important to make sure that the file has execution permissions.
$ ./exec_logstash.sh - Run the test with JMeter.
- Go to the grafana url and you are ready to visualize the results in the dashboards! (http://public_instance_ip:3000)
Conclusion
It is a well-known fact that Quality Assurance helps ecommerce stores create and provide online services that meet the needs, expectations and requirements of customers. QA practices help online businesses sell better and create a better online purchase experience, and also builds trust and loyalty with customers during significant campaigns such as Black Friday.
Hence, it is vital not only to optimize one’s ecommerce in preparation of such important sales events, but also to be able to effectively analyze the data from stress & performance testing. The actionable insight that LongBOARD provides with its dashboard visualizations, allows both business & technology leaders to understand bottlenecks and make the necessary improvements for their business.
Do you want to know more about LongBOARD? Get in touch with us!