Fingerprinting the HTTP Request — Beyond User-Agent, JA4, and QUIC
User-Agent is trivial to fake. Source IPs rotate. JA4 names the TLS library. HTTP/2 and QUIC fingerprints describe the transport frame. None of them capture how that client actually talks HTTP — which headers it sends, in what order, with what content types, at what path depth.
That gap is why we built the request fingerprint (req_fp). It is a shape of the HTTP exchange: method, path structure, headers, params, cookies, language, auth — not the values themselves. Scanners, libraries, and browsers leave different shapes even when they all copy a Chrome UA string.
A typical log value looks like:
l_ge_20_01_nil_nil_0_0000_07_ahuxxxx_0000_n_n_nil_n_200_20482
Seventeen underscore-separated fields. coreX also splits them into http.request.fingerprint.* so you can write rules on one piece instead of the whole string.
How the 17 fields group
Request line — who is calling what
| # | Log field | CoreX field | What it is |
|---|---|---|---|
| 1 | req_fp_path_b62 | (in full http.request.fingerprint) | Path encoded as base62. Same path → same token; useful for “this exact URL shape,” not human reading. |
| 2 | req_fp_method | — | GET / POST / … (ge, po, …). |
| 3 | req_fp_http_ver | http.request.version_numeric | 11 = HTTP/1.1, 20 = HTTP/2, 30 = HTTP/3. |
| 4 | req_fp_path_depth | http.request.fingerprint.path_depth | Number of / in the path. |
Policy use: Browsers on a marketing site almost always GET a shallow path over HTTP/2. A scanner POSTs to a deep API endpoint like /api/v1/batch/items at depth 4 over HTTP/1.1. Path-traversal and “secret file” dumps show up as unusual depth or as a path_b62 you have never seen on that host.
Do not put the full fingerprint in a request-phase block list if you care about path: field 1 changes per URL, so a probe for /config.json and a probe for /.env are different full strings. Use subfields (header_list, header_count, …) so one rule covers the whole campaign.
Parameters — what they sent, not the secrets
| # | Log field | CoreX field | What it is |
|---|---|---|---|
| 5 | req_fp_param_keys | http.request.fingerprint.param_keys | First letter of each query/body param name, sorted. nil if none. |
| 6 | req_fp_param_types | http.request.fingerprint.param_types | Type of each value: i int, s string, o object, l list, e empty, … |
| 7 | req_fp_param_lens | http.request.fingerprint.param_lens | Length of each value (5-10). |
| 8 | req_fp_ctype | http.request.fingerprint.content_type | First 4 letters of Content-Type subtype (json, xwww, 0000 if none). |
Policy use: A login POST is usually param_keys like pu (password, username), types ss, modest lengths, ctype xwww or json. An exploit often sends an object/list where a string belongs, a 10k-char field, or JSON on an endpoint that never takes a body. SQLi and log-injection scanners show up as “wrong type / huge length” more reliably than as a specific payload string.
http.request.fingerprint.body_depth (JSON nesting) is the same idea for bodies: {a:{b:{c:…}}} used in parser bombs.
Headers — the best “who is this client” signal
| # | Log field | CoreX field | What it is |
|---|---|---|---|
| 9 | req_fp_hdr_count | http.request.fingerprint.header_count | How many headers. |
| 10 | req_fp_hdr_list | http.request.fingerprint.header_list | Sorted first letters of header names (capped). Example: ahuxxxx. |
This is the field used for scanner_hdr_lists. A real Chrome request has Accept, Accept-Encoding, Accept-Language, Cookie, Sec-Fetch-*, Priority, etc. → a long list like aaacccccchpruuxxxx. curl, Python requests, and cheap scanners send Accept + Host + User-Agent and little else → ahuxxxx and count 7.
Policy use:
- Exact
header_listis a stable client ID. One credential-stuffing campaign we tracked was 100%ahuxxxx; a second, separate campaign was 100%aaaccccchruxxxx. Put those in a pattern list and you block the campaign even when IP and UA change. header_countalone is too blunt. A well-known vendor’s crawler sat at 8; aheader_count < 10challenge is too aggressive for that reason. Combine count with list, language, or JA4.- Libraries rarely send
Accept-LanguageorCookie; browsers almost always do.
Client context — browser vs script
| # | Log field | CoreX field | What it is |
|---|---|---|---|
| 11 | req_fp_accept_lang | (also http.request.geo_lang_mismatch) | First 4 alpha chars of Accept-Language (enus, zhcn, 0000 if missing). |
| 12 | req_fp_auth_type | http.request.fingerprint.auth_type | n none, b basic, t bearer, d digest, o other. |
| 13–14 | req_fp_cookie / cookie_fields | — | Cookie present? Initials of cookie names (not values). |
| 15 | req_fp_referer | — | n none, s same-origin, x cross-origin. |
Policy use:
- Missing language (
0000) + no cookie + no referer is the default scanner triple. A large search-engine’s crawler was the opposite tell:zhcnon a mobile UA from that provider’s ASN — internally consistent, where a fakeenuson the same ASN would be the mismatch. geo_lang_mismatch(language vs GeoIP country) catches VPN/scanner farms that forget to align headers with the exit IP.- Cookie names (
sid,session,__cf) fingerprint your app’s real users without storing session tokens. A “browser” with no cookies on a page that always sets them is a bot. - Auth type: GET
/adminwithbasicfrom the internet is a different event than a bearer token on/api.
Response — only after the app answers
| # | Log field | CoreX field | What it is |
|---|---|---|---|
| 16 | req_fp_status | http.response.fingerprint.status | Status code. |
| 17 | req_fp_body_bytes | http.response.fingerprint.body_bytes | Body size. |
These are response-phase. You cannot block with them on the way in; you can use them for allowlists of known-good “this client + this endpoint + this size” combinations, or to spot scanners that always get 200 HTML error pages of a fixed length.
The full http.request.fingerprint string is also response-phase ({partial}_{status}_{body_bytes}).
Why this is better than UA / IP / JA4 / h2-QUIC fingerprinting
| Signal | Easy to fake? | Stable across a campaign? | What it actually identifies |
|---|---|---|---|
| User-Agent | Yes | Until they rotate it | Marketing string |
| Source IP / ASN | Somewhat | No (they rotate) | Network, not client |
| JA4 | Hard (need a different TLS stack) | Yes for that binary | TLS library |
| HTTP/2 fingerprint (Akamai h2) | Hard (need a different h2 stack) | Yes for that binary | HTTP/2 frame settings/order |
| QUIC fingerprint | Hard (need a different QUIC impl) | Yes for that binary | QUIC transport |
header_list + header_count + lang/cookie | Have to reimplement a browser | Yes | HTTP client implementation |
param_types / param_lens | Have to change the exploit | Yes per exploit family | Attack shape |
The credential-stuffing campaign above proved the point: truncated Chrome UA, rotating paths, but every request was hdr_list=ahuxxxx, hdr_count=7, accept_lang=0000, cookie absent. One list entry covers hundreds of URLs.
JA4 and req_fp complement each other. One botnet’s JA4 used TLS 1.3 with HTTP/1.1 ALPN (…h1_…) — browsers on TLS 1.3 speak h2. That JA4 plus param_types on a batch API POST is stronger than either signal alone.
How to write policies with it
Prefer lists of subfields, not one giant fingerprint.
http.request.fingerprint.header_list in $pattern:scanner_hdr_lists
http.request.fingerprint.header_count < 8
and http.request.fingerprint.auth_type = "n"
http.request.fingerprint.path_depth > 8
http.request.fingerprint.param_types contains "o"
and http.request.method = "GET"
Good patterns
- Header list / JA4 lists you grow over time.
- Combine two cheap signals: low header count and no Accept-Language and not a known good bot.
- Param shape on a specific path: login must look like
ss+ form/json, not a 50-key object. - Path depth or body depth as a ceiling (
path_depth > 8,body_depth > 10) — already used in default risk rules.
Bad patterns
header_count < 10with no other condition (legitimate crawlers, some APIs, HTTP/2 with few headers).- Matching the full
req_fpstring for a scanner (path_b62and status/bytes change every hit). - Using generic Chrome JA4 as if it were a client ID (too many real browsers share it).
- Response fields in a request-phase block rule (they are not set yet).
Mental model: JA4 = “which TLS library.” h2/QUIC fingerprint = “which transport stack.” req_fp = “how that client builds an HTTP request.” Security lists on header_list, param_types, and auth_type stay small and stay valid when the attacker changes IP, UA, or even rotates TLS and transport fingerprints.