Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [anomaly-detection]

Anomaly detection refers to the problem of finding patterns in data that do not conform to expected behaviour. This is also known as outlier detection.

355 questions
4
votes
1 answer

How to detect anomalies in web log data

I have the following challenge: We have the web logs of a platform where people can download publications and we need to detect anomalies. From time to time and only by chance we observe spikes in usage over a day or so, where there are many more…
jfix
  • 143
  • 4
2
votes
1 answer

ML Approach for Graph Anomaly Detection

Very new to ML. I am trying to create an anomaly detector. I have thousands of graphs like the one I have attached. I am interested in the pink line. If the pink line's behavior changes drastically from the overall pattern of the line (i.e. the…
dfahsjdahfsudaf
  • 121
  • 2
2
votes
1 answer

Outlier detection with non normal distribution

What are some techniques that I can use for anomaly detection given a non-Normal distribution? I have less than twenty available observations.
youngam
  • 41
  • 3
2
votes
2 answers

Industry Standard Process for Fraud/Outlier/Anomaly Detection

I want to write my thesis about Fraud Detection in ERP Databases. I'm looking for a Industry Standard Processs such as CRISP-DM for Data Mining Projects, in order to justify my approach in solving the issue of finding outliers/anomalies in the data…
Nex
  • 285
  • 2
  • 6
1
vote
0 answers

Reporting false alarm rate and detection rate in shilling attack detection without given any labels

I have a dataset of ratings given to movies by users. I've applied the method mentioned in this paper to detect fake votes(I've used $H(X)$ and $M(X)$ measures). But the dataset I'm using doesn't have any label for checking the false alarm rate and…
Kamran Hosseini
  • 111
  • 3
1
vote
1 answer

What are the techniques for anomaly detection of Unsupervised learning problem

I have sufficient and properly formatted data in millions without labels. I have to find out the anomalies. Heard Isolation forest, Mahalanobis distance about identifying anomalies in unsupervised learning. Are these ok to try? Are their any other…
GSKR
  • 13
  • 4
1
vote
0 answers

What do you call an anomaly/signature detection algorithm in data science

If I have an algorithm for detecting a set of data points that indicate with a high level of certainty that some event has occured OR that behaviors outside of a set model are occuring. What would you call the algorithm used to detect these…
Jay Hawk
  • 11
  • 1
1
vote
1 answer

Isolation forest: how to deal with identical values?

I am trying to develop my own implementation of isolation forest algorithm. However I don't know how to deal with points that have the same value for a given feature. To better understand the problem, consider this example: in my dataset I have the…
gagarine
  • 15
  • 3
1
vote
1 answer

Detect a bot in a audit log

I need to develop a report that will show automated queries in an audit log of queries on a system of the company. The logs have this fashion: query_id id query_time 1 1 2018-02-01 00:09:02 2 1 2018-02-01…
Allan
  • 131
  • 3
0
votes
2 answers

Anomaly detection in a database

We have a production database. The load on the database varies at different times. I want to identify anomalies; for e.g, the number of database processes responding to user queries at 9 am is 100 for a given day. If the number is 200, then it's an…
Jerry Kross
  • 3
  • 1
0
votes
1 answer

is there a way to check if i got a "good price" on something?

I'm looking at some data. Actually, the Boston Housing dataset is probably a good proxy for it: https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html I'm wondering if there's a way to predict if I got a "good price" given certain…
Mohammad Athar
  • 291
  • 1
  • 13
0
votes
1 answer

RCA on top of existing anomalies

When we work with timeseries data containing multiple features eg. a sensor data. we can detect anomalies using cluster, supervised and semi supervised based approaches (Eg. Isolation, Autoencoder Etc), but those approaches only detect anomalies and…
duck
  • 111
  • 2
-1
votes
1 answer

How does outlier detection work if there are multiple distribution clusters?

In my case, All the time-series observations are with high dimensions. Very likely, they will fall in multiple clusters(meaning multiple distribution patterns) other than a single cluster(meaning single distribution pattern). Will anomaly detection…
Shengjie
  • 231
  • 1
  • 2
  • 7