-
Notifications
You must be signed in to change notification settings - Fork 62
Timestamp
CIF supports three separate timestamps per record or observation: (reporttime, lasttime, firsttime). A record should have at least one timestamp associated with it and could have up to three timestamps.
This is the timestamp of when the record or observation was given to you.
This is a machine generated timestamp of the last time the source observed the behavior. This would be the most recent timestamp found in machine generated logs where the host is leveraging clock synchronization (NTP).
This is a machine generated timestamp of the first time the source observed the behavior. This would be the earliest timestamp found in machine generated logs where the host is leveraging clock synchronization (NTP).
An information sharing partner may give you the following intelligence:
address portlist protocol firsttime lasttime description
192.168.1.1 22 tcp 2016-06-18T00:00:00Z 2016-06-18T10:10:00Z scanner
If you were to ingest this record into CIF at 2016-06-18T12:00:00Z
, you could associate these three timestamps with this single observation:
firsttime: 2016-06-18T00:00:00Z
lasttime: 2016-06-18T10:10:00Z
reporttime: 2016-06-18T12:00:00Z
When using cif-smrt to ingest intelligence into CIF, cif-smrt will automatically fill in lastime, reporttime and firsttime if those values are not specified. It's not uncommon to see the exact same timestamp when dealing with a feed that does not give any timestamps. Here's an example of the alienvault feed:
{
"lasttime" : "2016-05-24T13:01:52Z",
"firsttime" : "2016-05-24T13:01:52Z",
"reporttime" : "2016-05-24T13:01:51Z",
"tlp" : "white",
"tags" : ["suspicious"],
"altid" : "https://reputation.alienvault.com/reputation.data",
"description" : "Scanning Host",
"altid_tlp" : "white",
"asn" : "8075",
"confidence" : 65,
"group" : ["everyone"],
"provider" : "reputation.alienvault.com",
"observable" : "13.84.219.191",
"otype" : "ipv4",
}
A typical CIF query is to return x data over y period. The "y period" can be nuanced as you have two common choices:
- lasttime
- reporttime
If the set of records have lasttime
and reporttime
specified and the delta between those values is large, the data returned could be rather different when choosing to filter on lasttime
vs reporttime
.
The CIFv2 CLI clients default timestamp choice is almost always reporttime
. When the lasttime
and reporttime
values are the same as the alienvault example above, the returned results are very likely what you expect. In the scenario where you know there is a large delta between lasttime
and reporttime
and you know you want the period to be based on lasttime
you'll want to make sure you are being specific in your queries.
Under the hood these queries use the API parameters reporttime
and reporttimeend
-
--today
return results for the current day starting atT00:00:00Z
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --today
-
--last-hour
return results for the current day and the current hour between00:00Z
-59:59Z
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --last-hour
-
--last-day
return results for the previous 24 hours from the current time.
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --last-day
-
--days [int]
return results for the previous two days from the current time.
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --days 2
When you know that you want to query on the machine generated timestamp (lasttime
) you will need to leverage the filters lasttime
and firsttime
.
- return results for the current day starting at
T00:00:00Z
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --firsttime 2016-05-24T00:00:00Z --lasttime 2016-05-24T23:59:59Z
- return results for the current day and the current hour between 00:00Z - 59:59Z
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --firsttime 2016-05-24T15:00:00Z --lasttime 2016-05-24T15:59:59Z
- return results for the previous 24 hours from the current time.
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --firsttime 2016-05-23T15:13:59Z --lasttime 2016-05-24T15:14:00Z
- return results for the previous two days from the current time.
$ cif --otype fqdn -c 85 --provider osint.bambenekconsulting.com --firsttime 2016-05-22T15:13:59Z --lasttime 2016-05-24T15:14:00Z