jq – filtering JSON by searching key values

12 views
0

Data sample – shodan_data.json (full sample: https://pastebin.com/KFkVmc2M)

{
      "ip": 3301234701,
      "_shodan": {
        "options": {
          "referrer": "7ae15507-f5cc-4353-b72e-5cc0b1c34c5e"
        },
      },
      "hash": -1056085507,
      "os": null,
      "title": "WHM Login",
      "opts": {
        "vulns": ["!CVE-2014-0160"],
        "heartbleed": "2017/08/29 09:57:30 196.196.216.13:2087 - SAFE
        }
      },
      "isp": "Fiber Grid Inc",
      "http": {
        "redirects": [],
        "title": "WHM Login",
        "robots": null,
        "favicon": null,
        "host": "196.196.216.13",
        "html":
}

My jq code:

jq -r 'select((.opts.vulns[0] | contains("!CVE-2014-0160")))? | ['.ip_str', '.isp', '.timestamp', .opts.vulns[0]] | @csv' shodan_data.json

Example output when using full dataset:

"165.231.171.237","Fiber Grid Inc","2017-08-22T02:24:07.658547","!CVE-2014-0160"

Right now what it does is that it’s searching for a specific JSON object .opts.vulns[0] with a value !CVE-2014-0160 then filters the entire object where the value was found and prints the object values specified .ip_str', '.isp', '.timestamp', .opts.vulns[0] as csv. Also suppresses any errors where the searchable value isn’t found which allows me to write the output to a .csv file.

What I would like it to do is to take whatever value I would like to find, like in the example !CVE-2014-0160 (like a search engine) and be able to recursively go through the entire JSON dataset and have the same end result as my code atm.

I’ve been trying to figure this one out and have been trying out .. recursive descent and other methods but the way I’m filtering the data seems to indicate that what I’m trying to accomplish might need a different approach if even possible?