Issues updating CVE database behind https proxy

[budric]

Hi,

I have a firewall that blocks all outbound ports except port 80 (firewall can't be changed). For HTTPS there's a squid proxy available that allows HTTP tunnel using CONNECT method. Environment variables for http_proxy and https_proxy are set on the server/shell running vuls. When updating with go-cve-dictionary fetchnvd -last2y it stalls with timeout error:

[Apr 20 11:24:24]  INFO Fetching... http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz
 0 / 2 [-----------------------------------------------]   0.00%
[Apr 20 11:24:24]  INFO Fetching... http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
[Apr 20 11:26:31] ERROR Failed to fetch cve data from NVD. err: HTTP error. errs: [Get https://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz: dial tcp 129.6.13.177:443: connection timed out], url: http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz

wget tool works:

$ wget http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
--2016-04-20 11:30:04--  http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
Resolving my.org.proxy.address (my.org.proxy.address)... x.x.x.x
Connecting to my.org.proxy.address (my.org.proxy.address)|x.x.x.x|:3128... connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: https://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz [following]
--2016-04-20 11:30:04--  https://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
Connecting to my.org.proxy.address (my.org.proxy.address)|x.x.x.x|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1505541 (1.4M) [application/x-gzip]
Saving to: ‘nvdcve-2.0-2015.xml.gz’

Not sure if it's a bug, or enhancement request to make vuls work behind proxy like wget/curl tools work on the system.

Also notice the HTTP 301 move from http://static.nvd.nist.gov address to https://static.nvd.nist.gov . Perhaps that's tripping up the vuls code somewhere. Sorry I'm not a go programmer I have no idea where to start looking.

Thank you.

[kotakanbe]

Hi, @budric
Thanks for reporting.
It's a bug.
I will fix it today.

[kotakanbe]

Hi, @budric

I Fixed the bug.
see vulsio/go-cve-dictionary#9

Update and build go-cve-dictionary and vuls

$ go get -u github.com/kotakanbe/go-cve-dictionary
$ go get -u github.com/future-architect/vuls

Then

$ go-cve-dictionary fetchnvd -last2y -http-proxy=http://your-proxy-server.com:80

OS Environment variable is not supported yet.

[budric]

Thanks very much!
Works.