Explore Anomaly Detection Techniques – offer two different types of techniques (what are they and what do they do)?
Machine Learning has four common classes of applications: classification, predicting next value, anomaly detection, and discovering structure. Among them, Anomaly detection detects data points in data that does not fit well with the rest of the data. It has a wide range of applications such as fraud detection, surveillance, diagnosis, data cleanup, and predictive maintenance.
Although it has been studied in detail in academia, applications of anomaly detection have been limited to niche domains like banks, financial institutions, auditing, and medical diagnosis etc. However, with the advent of IoT, anomaly detection would likely to play a key role in IoT use cases such as monitoring and predictive maintenance.
What is Anomaly Detection?
Anomalies or outliers come in three types.
Point Anomalies. If an individual data instance can be considered as anomalous with respect to the rest of the data (e.g. purchase with large transaction value)
Contextual Anomalies. if a data instance is anomalous in a specific context, but not otherwise (anomaly if occur at a certain time or a certain region. e.g. a large spike at the middle of the night)
Collective Anomalies. If a collection of related data instances is anomalous with respect to the entire dataset, but not individual values. They have two variations.
Events in unexpected order (ordered. e.g. breaking rhythm in ECG)
Unexpected value combinations (unordered. e.g. buying a large number of expensive items)
Information Theory: The main idea is that anomalies have high information content due to irregularities and this approach tries to find a subset of data points that have highest irregularities.
Dimension Reduction: The main idea is that after applying dimension reduction, normal data can be easily expressed as a combination of dimensions while anomalies tend to create complex combinations.
Graph Analysis: Some processes would have interaction between different players. For example, money transfers would create a dependency graph among participants. Flow analysis of such graphs might show anomalies. On some other use cases such as insurance, stock markets, corporate payment fraud etc., similarities between player’s transactions might suggest anomalous behavior.
Perera, S. (2018, November 06). Introduction to Anomaly Detection: Concepts and Techniques. Retrieved from https://iwringer.wordpress.com/2015/11/17/anomaly-…
Anomaly Detection is an approach used to detect unusual and unexpected patterns which fail to obey with the expected behavior referred to as outliers. Anomaly detection is widely applied in areas such as in business as well as in the system health monitoring which involve spotting and monitoring of malignant tumors in patients. Moreover, anomaly detection is widely applied in fraud detection in credit card-transactions (Huang, Mehrotra & Mohan, 2018).
The two commonly used Anomaly Detection Techniques are: – using Simple Statistical Methods and Density-Based Anomaly Detection. The simple statistical technique involves identifying irregularities within the given data through flag and ignoring all data points deviating from statistical properties of the data distribution such as median, mean, quantiles and mode. The second approach used in detecting data anomalies is referred to as: – Machine Learning-Based techniques. A good example of such techniques is referred to as Density-Based Anomaly-Detection identifies outliers within data sample by use of k-nearest neighbors-algorithm. According to this technique, normal data points are experienced near dense neighborhood whereas the abnormalities are much far away. Moreover, the technique involves evaluating the nearest data set points based their scores (Huang, Mehrotra & Mohan, 2018). Thirdly, we have clustering-Based Anomaly-Detection which assumes that all data points which are like and similar belong to same groups or clusters based on the data distance from local centroids.
Huang, H., Mehrotra, K., & Mohan, C. (2018). Anomaly Detection Principles and Algorithms. New York: Springer.
Do you need a similar assignment done for you from scratch? We have qualified writers to help you. We assure you an A+ quality paper that is free from plagiarism. Order now for an Amazing Discount!Use Discount Code “Newclient” for a 15% Discount!NB: We do not resell papers. Upon ordering, we do an original paper exclusively for you.