This topic contains 0 replies, has 1 voice, and was last updated by Greg Makowski 1 year, 5 months ago.
April 7, 2016 at 5:44 pm #14481
I wanted to send some brief background on diagnosis algorithms. See Bayesian Networks (BN) or similar names for the same family: Bayesian Belief Network (BBN) or Graphical Belief Networks (GBM).
How could a BN be used for cyber security?
* You can give it a training set of data, such as network or web logs. The BN algorithm does not depend on having each record labelled as “bad” or “normal.”
* By running the BN training algorithm, a network is built.
….. Each network node is a variable or field, which can take different categories of values. Continuous fields, such as age or income would be binned first.
…… connections between nodes are built. The edges are like rules.
…… A major improvement over Expert Systems (ES) is that the connections and weights are made globally consistent, with something called a “Markov Blanket”. For details on ES, see below.
* Then a new situation, or record, can be diagnosed or “scored”, applying the trained BN. By “asserting” the field values for the record to be scored into the network, probabilities are propagated through the network. States of variables for different kinds of fraud or bad behavior will be asserted to a certain probability level, resulting in the score.
One challenge in security can be a high rate of false alerts. One way to reduce false alerts would be to configure two BN’s, like a prosecuting attorney (assuming the worst) and a defense attorney (erring on the side of assuming innocence) when scoring the same record (or case). Let a strong attack score with a weak defense score get through. If you are getting too many false alerts, then give more weight to plausible defense scores. Or based on end user feedback at the Security Operations Center (SOC), train a model to give weights to the attack and defense BN scores.
http://www.kdnuggets.com/software/bayesian.html (software list, free and commercial)
Predominant commercial software for Bayesian Networks
http://www.bayesia.com/ (researchers & founders from France
http://www.bayesialab.com/ (US distributors of Bayesia) ~$10k for PC version
http://www.bayesia.com/book Free 300+ page ebook download
http://www.bayesia.com/breast-cancer-diagnostics (see chapter 6 of the ebook)
The diagnosis process is the same, for cancer or cyber security.
Java packages for Bayesian Networks
R packages for Bayesian Networks
—– optional history on diagnosis systems and Expert Systems (ES) —–
In the 80’s, Expert Systems were used, named for interviewing experts and encoding their experience in chained rules. Also, backward chaining was used in expert system software or Prolog to solve diagnosis problems. I deployed an expert system on identifying unknowns, as asbestos or not. It was using “polarized crystallography”, or putting the unknown sample under a microscope, in different oils, using polarized lenses.
One challenge with Expert Systems could be getting conflicting information from different people. That could be addressed by using Machine Learning techniques, such as a Decision Tree, repeatedly go through the labelled training data, to build a set of rules in a tree structure.
In 1996, Gates claimed that Bayesian Networks was a competitive advantage of Microsoft.
because of Stanford’s CS research into BN, donate $6mm