Handling RAW less JSON Lines in file

[exploit-io]

I use katana for gathering passive and active crawling.

There is no raw field in passive data:

{"timestamp":"2024-05-02T16:28:34.259259353Z","request":{"method":"GET","endpoint":"http://testphp.vulnweb.com:80/artists.php?artist=2"},"response":{"status_code":200,"headers":{}},"passive":{"source":"waybackarchive","reference":"http://web.archive.org/cdx/search/cdx?url=*.http://testphp.vulnweb.com/*\u0026output=txt\u0026fl=original\u0026collapse=urlkey"}}

and when we run nuclei on these requests, it can't find the raw and does not crawl the endpoint which is a simple GET request that nuclei should be able to reconstrcut it.

nuclei Comand:

nuclei -l tests.jsonl -input-mode jsonl -t template.yaml -v

Warning:

[WRN] jsonl: Could not parse raw request http://testphp.vulnweb.com:80/artists.php?artist=2: failed to read method line: EOF

[dogancanbakir]

You are currently using passive crawl mode, which does not provide an actual request/response. See the details at projectdiscovery/katana#781.

[exploit-io]

I think, It'd be good if nuclei, katana could reconstrcut the raw request using the data in json!
because everything is almost ready