New BPS 9.22 Feature HAR Simulation
HTTP Archive Record (HAR) is a JSON-formatted capture of HTTP application data produced by most of the modern web browsers that can be used to conveniently analyze the HTTP transactions in sophisticated web applications. In Keysight Technologies BPS 9.22 software release for BreakingPoint System comes with a new feature called “HTTP Archive Record (HAR) Simulation” which empowers our customers to create rapid simulation of web applications using any HAR capture file conforming to the HTTP Archive v1.2 specification (W3C standard).
In this blog we will talk about how to create a custom HAR Simulation Superflow in BPS, its different features, ATI BPS provided sample HAR Simulation Superflows and some anomalies we have found in the HAR file.
Creating a HAR simulation Superflow in BPS:
The following image displays an example Superflow created from a single instance of the HTTP Archive Record (HAR) Simulation flow - this is the most common scenario for a test. The Superflows that use this flow are only compatible with the AppSim test component and support NAT and Proxy mode testing.
Any canned Superflow using this feature will have the “NAT” and “Proxy” tags and a new tag, “HARSimulation”, to facilitate Superflow search in the UI.
Currently, this flow only supports two actions:
- Replaying the HTTP data in a HAR capture over HTTP 1.1 using the TLS 1.2 transport.
- Replaying the HTTP data in a HAR capture over HTTP 1.1 using the TLS 1.3 transport.
Note: This example displays the parameters for the TLS 1.2 transport. The parameters for TLS 1.3 transport are similar.
HAR Simulation Features:
In both the HAR Simulation flow actions, we have added some features like:
- The users can upload their own HAR file (.har) or its .gz compressed format to simulate its traffic from BPS.
- The users can give the maximum number of unique hosts (unique TCP connections) and HTTP requests-responses as input.
- We also enable users to use their own Client and Server-side certificates and keys to encrypt the traffic.
- They can choose their preferred cipher suites for TLS encryption.
- Also, the users can easily get the decrypted traffic by running the Superflow in decryption mode i.e., choosing the destination port as “80” from the flow parameter.
HAR Simulation Canned Superflows in ATI:
We have published 2 new canned Superflows for HAR simulation:
- Bandwidth HAR Replay HTTP1.1 over TLS1.2
This Superflow simulates HAR produced from random crawling of a Wikipedia web page as of May 2022. There are 3 HTTP hosts and about 150 HTTP transactions (excluding browser cache retrievals) which are replayed in HTTP1.1 over TLS1.2.
This Superflow simulates HAR produced from random crawling of a Wikipedia web page as of May 2022. There are 3 HTTP hosts and about 150 HTTP transactions (excluding browser cache retrievals) which are replayed in HTTP1.1 over TLS1.3.
Note: Future Strike Pack updates will add new actions and features to support more sophisticated test scenarios.
Anomalies in HAR capture:
After capturing a couple of HAR files of different popular websites like Wikipedia, YouTube, Concur Solutions etc. using different web browsers like Google Chrome, Firefox etc. and analyzing them, we have found some anomalies:
1. Empty response body, but content-length is > 0
HAR can only capture the current browser’s traffic. For the HTTP responses with status code 301 i.e., “Moved permanently”, since it redirects into a new URL in a new browser tab, it can only capture the response body length, not the actual response body.
For that reason, in those cases the response body/content is empty though some content-length (> 0) is mentioned in the response header.
2. Transfer-encoding chunked, but no content found
We have seen some HTTP responses where the “Chunked transfer-encoding” is used, but there is no response payload present in the HAR file though the total size of all the chunks is mentioned inside the “content” -> “size” field.
3. Chrome vs Firefox HAR Capture
After analyzing HAR captures of the same webpage taken using both Google Chrome and Firefox browsers, we have found some mismatches in the HTTP request-response header field names like –
Please check the image below to get more information about it:
- Bad HTTP Response with Status Code “0”
In some cases, a HAR file contains no HTTP response with status code “0”. This indicates the request is being timed out due to any reason like slow internet connection, blocking firewall, empty response body etc.
Note: In our HAR Simulation v1 release, we are discarding the HTTP request-response pairs which contain at least one anomaly from the above-mentioned list. But in future, we will tackle and add support for these. So, stay tuned for the upcoming ATI StrikePack releases.
- HTTP Response 304 Contains Response Body:
The HTTP 304 Not Modified response indicates that there is no need to resend the response body since its latest version is already available in client cache.
Ideally, the HTTP 304 response should not contain the response body. While capturing the HAR file from browsers, if we don’t disable the cache, then the HAR captures the response body with the HTTP 304 response.
So, it is always recommended to enable the “Disable cache” checkbox while capturing the HAR file.
Note: If the above-mentioned anomaly is present inside the input HAR file, then we are ignoring the response body and setting the “content-length” field as “0” inside the HTTP 304 response header during HAR simulation from BPS.
Leverage Subscription Service to Stay Ahead of Attacks
Keysight's Application and Threat Intelligence subscription provides daily malware and bi-weekly updates of the latest application protocols and vulnerabilities for use with Keysight test platforms. The ATI Research Centre continuously monitors threats as they appear in the wild. Customers of BreakingPointnow have access to attack campaigns for different advanced persistent threats, allowing them to test their currently deployed security control's ability to detect or block such attacks.