Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [svm]

Support Vector Machines (SVM) are a popular supervised machine learning algorithm that can be used for classification or regression.

580 questions
14
votes
1 answer

Intuition for the regularization parameter in SVM

How does varying the regularization parameter in an SVM change the decision boundary for a non-separable dataset? A visual answer and/or some commentary on the limiting behaviors (for large and small regularization) would be very helpful.
ASX
  • 451
  • 2
  • 4
  • 7
7
votes
1 answer

Please enlighten me with Platt's SMO algorithm (for SVM)

From A_Roadmap_to_SVM_SMO.pdf, pg 12. (source: postimg.org) Assume I am using linear kernel, how will I be able to get both the first and second inner product? My guess, inner product of datapoint with datapoint j labelled class A for the first…
Vincent Benedict Victor
  • 103
  • 5
5
votes
1 answer

SVM on sparse data

In this paper related to factorization machine, the author compares factorization machine (FM) with SVM. As FM performs better than SVM, it's considered state of the art for sparse data. Why SVM is considered so important algorithm for sparse…
Ravikrn
  • 205
  • 2
  • 6
4
votes
1 answer

SVM vs RVM, when to use what?

I'm currently working on a project where I'm supposed to compare the efficiency of SVM vs RVM, there seems to be a lot of information to be gathered about RVM whereas I find rather old documents about RVM. It's the first time I use this forum so I…
Jan Erst
  • 41
  • 2
4
votes
2 answers

Why do we use +1 and -1 for marginal decision boundaries in SVM

While using support vector machines (SVM), we encounter 3 types of lines (for a 2D case). One is the decision boundary and the other 2 are margins: Why do we use $+1$ and $-1$ as the values after the $=$ sign while writing the equations for the SVM…
user1825567
  • 1,396
  • 1
  • 12
  • 22
3
votes
0 answers

feature weights in structured support vector machine

I like to find the weight vector for input-space features in a structured SVM. The idea is to identify the most important set of input-space features (based on the magnitude of their corresponding weights). I know that in a binary SVM the weight…
imk
  • 31
  • 2
3
votes
1 answer

Implementing SVM from scratch?

I am trying to implement the rbf kernel for SVM from scratch as practice for my coming interviews. I attempted to use cvxopt to solve the optimization problem. However, when I compute the accuracy and compare it to the actual SVM library on sklearn,…
user70145
  • 49
  • 1
  • 4
3
votes
1 answer

How regularization parameter in SVM affects hyperplane parameters

While learning the SVM classification I came across the regularization parameter $\lambda$: $F(w,b) = \left\lVert w\right\lVert_2^2 +\lambda \sum_{i=1}^n max(0,1-y_i(w^Tx_i +b)).$ So from what I understand, the main point of SVM is to find the…
dxdydz
  • 173
  • 5
3
votes
1 answer

Doubt with SVM math

I have a question about SVM that some of you may help me with… I know that y(xi), by convention, would be -1 or 1 depending on which class the Xi belongs to. But I don't fully understand why it's stablished that the hyperplane equation should…
JZarzuela
  • 133
  • 2
3
votes
1 answer

Does the "laplacian" kernel used in the SVM context come from a Hilbert space inner product?

The "kernel" $k(x,y) = e^{-\|x-y\|} $ is used in the context of SVM. Here $x,y \in \mathbb R^n$ and $\|.\|$ is the Euclidean norm. Is there a Hilbert space H and a map $\varphi:\mathbb R^n \rightarrow H$ such that $k(x,y) = <\varphi(x),\varphi(y)>…
VictorZurkowski
  • 438
  • 2
  • 8
3
votes
2 answers

Why SVM works well with high dimensional data?

I'm having troubles trying to understand why SVM works well with high dimensional data, the case when p >> n. I read the following: SVM is automatically regularized. You don't have to pick a regularization parameter because picking the widest…
Norhther
  • 97
  • 1
  • 6
2
votes
0 answers

Solution of quadratic optimization in support vector machines

In support vector machines, the minimization problem with inequality constraints can be converted to a minimization problem of Lagrange multipliers with equality constraints by KKT condition and Lagrange duality. The Lagrange multipliers have to be…
feynman
  • 237
  • 1
  • 8
2
votes
1 answer

How to plot train test error for classification models like Support Vector Classification(SVC)

How to plot train test error for classification models like Support Vector Classification(SVC). I am using SVC from sklearn module, not able to get train and test errors to plot
Harika M
  • 345
  • 1
  • 4
  • 7
2
votes
0 answers

Non-linear transformations input dataset for support vector machines

I have two classes (A,B) that I would like to classify using a SVM. Say that I have a class C and a function f. Can I do this: A' = f(A,C) = |A-C| B' = f(B,C) = |B-C| and then perform the classification on A' and B' instead? In the context of my…
Diego
  • 121
  • 1
2
votes
2 answers

Why does an SVM model store the support vectors, and not just the separating hyperplane?

In every explanation of SVMs, we're shown how training finds a hyperplane that best separates the data. Presumably then for inference, you just check which side of the plane a point is on. However, all the "disadvantages of SVMs" posts [1, 2]…
Doctor J
  • 213
  • 2
  • 4
1
2 3