For the purposes of building the effective Cyber Intelligence process relevance and completeness of the received data plays the crucial role. In most of the cases the work with the Threat Intelligence (TI) data starts with the adding of open source feeds. Regarding the 2021 SANS Cyber Threat Intelligence (CTI) Survey 66.3 % of the companies use open sources for the collecting of indicators of compromise (IoC) data and they strive to work with multiple sources in parallel.
On one hand using several sources seems the most simple and obvious way to start collecting data quickly but on the flipside there is a big issue in numerous detections when you upload these indicators to the security tools. And it makes the process of the data processing by the analyst almost impossible. We’d like to note also that if you want to create block lists for security tools from IoCs or a collection for search queries on the side of EDR solutions there will be a limitation by the number of entities. This means that anyway there has to be the manual work to prepare such collection of IoCs. Besides you have to keep in mind that only indicators itselves are useless.
The real value comes when the data related to threats is analyzed and there is a link with the additional context. One more problem is that not all the sources are relevant and “fresh” even if they come from commercial feeds. That’s why for the purposes of working with TI it’s important not only to receive data on a regular basis but also to get rid of useless data. That is the reason why TI platforms have implemented features for the evaluation of quality of sources and IoCs. This helps analysts to work only with really important and relevant information.
In the Platform for threat information analysis Defensys Threat Intelligence Platform (TIP) this capability is represented by the IoC rating which helps to exсlude from the mass of IoCs only those that are really needed to be analyzed to withstand external threats (see the picture below).
Picture 1. IoC rating
IoCs ratings
IoC rating is a conditional indicator of the IoC’s severity. With its help TI analysts underline the most relevant and dangerous IoCs for the future work with them filtering them from the useless ones.
And it’s also important to understand that if the TI platform stands only on the data coming from the feed or enrichment service when setting the rating so this rating is not reliable because only few of the sources gives you the severity level of IoCs. Besides a lot of feeds use different approaches in severity evaluation.
Keeping in mind the users’ need to use the unified approach to calculate the rating we analyzed the research from the University of Amsterdam which stands on several basic principles and brought it in Defensys TIP while developing our own rating calculation feature.
IoC rating calculation in Defensys TIP works using 3 principles including both feed source properties and IoC’s details:
- Data source quality
- Attenuation of the IoC’s rating
- White lists
Below we’re going to talk about each of them.
Principle 1. Data source quality
Defensys TIP works with opinions of each IoCs data feed and other entitles and doesn’t give the priority to some particular source. The rating calculation model of Defensys TIP enables the evaluation of each source from the quality angle. That is why every source has its own trust level. This parameter stands on several important metrics: extensiveness, promptness and completeness.
Extensiveness
Extensiveness reflects the level of IoC’s connectivity with additional context (ASN, Whois data, Geo data, open ports data, running services data etc.) which the feed gives along with the included IoC value. The more this metric is (the more extensive is the connectivity with additional context) the more is the probability to forecast that the data has been already analyzed and IoCs have its attribution – the relation with the source – threat author.
Promptness
The promptness metric is calculated with the update frequency of a particular feed comparing to other feeds. The more frequently and promptly the feed provides the data related to IoCs the higher is this metric value.
Completeness
The completeness metric reflects conditional feed’s “contribution” in the total volume of the IoCs’ data received from all the feeds. This means the amount of different IoCs received from the particular source in comparison with overall number of IoCs received from all the added feeds in Defensys TIP. The higher this metric is the more the “contribution” level of the feed and the higher is the completeness metric.
Each of the metrics described above has its own weight. And the average value is calculated based on the sums of all these metrics that defines the trust level of the particular feed. The higher is the value of each metric and respectively of the total trust level the more qualitative and reliable the feed is.
Principle 2. Attenuation of the IoC’s rating
At this stage of rating calculation starts the concept of the IoC relevance attenuation during the dedicated period of time. The attenuation function used in Defensys TIP rating calculation mechanism depends on the following parameters
- The time of existence in the system
- IoC last update time
- IoC lifetime duration – the time after which the rating reduces to 0 (means that IoC is no more relevant and respectively malicious)
- IoC attenuation speed
Principle 3. White lists
There come millions of IoCs from feeds but not each is malicious. Defensys TIP can work with misp-warninglists – the so called “white lists” – lists of known indicators which can be related to potential false positives and errors (see the picture below).
Picture 2. MISP Warninglists
If the received IoC exists in these lists so the source trust level will be lower.
Many companies have their own white lists. And entities from these lists shouldn’t be considered as malicious by the System. Defensys TIP gives the opportunity to users to exclude entities not related with malicious activity from the list of indicators in accordance with particular company’s expertise.
This given methodic is pretty much universal and helps to reduce the number of false positives when underlining and calculating the IoCs’ ratings. Furthermore you can implement it to the existing own TI management system for the calculation of IoC reputation and this enables you to make a proper decision on how to deal with IoCs.
Defensys TIP rating model
The final IoC rating in Defensys TIP is calculated simultaneously taking into account the mentioned above source trust level and the rating set by the particular source.
The rating can be set from 0 to 100. The higher is the rating value the more reliable IoC is and this means that it has the higher probability to be active and malicious.
But Defensys TIP users can leverage the process of rating calculation when they choose the built-in settings. Each setting enables you to pick up the particular set of metrics combinations that will be used to calculate IoCs’ ratings. (see the picture below).
- Standard – all the metrics have the same priority
- Extensiveness calculation – source’s context plays the main role
- Timeliness – the more frequently updated sources will be considered by the System as the more qualified
Picture 3. Threat ranking
This way the rating calculation approach developed by Defensys helps you to evaluate IoCs more flexibly and also leverage the way for their aging. And you can work only with those indicators that are the most dangerous for your infrastructure at the moment.
You can study all the other Defensys TIP advantages on the product’s page.