A Practical Guide to Support Vector Classi

Only available on StudyMode
  • Topic: Support vector machine, Machine learning, Data set
  • Pages : 14 (3972 words )
  • Download(s) : 54
  • Published : April 19, 2013
Open Document
Text Preview
A Practical Guide to Support Vector Classification
Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin Department of Computer Science National Taiwan University, Taipei 106, Taiwan http://www.csie.ntu.edu.tw/~cjlin Initial version: 2003 Last updated: April 15, 2010 Abstract The support vector machine (SVM) is a popular classification technique. However, beginners who are not familiar with SVM often get unsatisfactory results since they miss some easy but significant steps. In this guide, we propose a simple procedure which usually gives reasonable results.

1

Introduction

SVMs (Support Vector Machines) are a useful technique for data classification. Although SVM is considered easier to use than Neural Networks, users not familiar with it often get unsatisfactory results at first. Here we outline a “cookbook” approach which usually gives reasonable results. Note that this guide is not for SVM researchers nor do we guarantee you will achieve the highest accuracy. Also, we do not intend to solve challenging or difficult problems. Our purpose is to give SVM novices a recipe for rapidly obtaining acceptable results. Although users do not need to understand the underlying theory behind SVM, we briefly introduce the basics necessary for explaining our procedure. A classification task usually involves separating data into training and testing sets. Each instance in the training set contains one “target value” (i.e. the class labels) and several “attributes” (i.e. the features or observed variables). The goal of SVM is to produce a model (based on the training data) which predicts the target values of the test data given only the test data attributes. Given a training set of instance-label pairs (xi , yi ), i = 1, . . . , l where xi ∈ Rn and y ∈ {1, −1}l , the support vector machines (SVM) (Boser et al., 1992; Cortes and Vapnik, 1995) require the solution of the following optimization problem: min 1 T w w+C 2 l

w,b,ξ

ξi
i=1

subject to

yi (wT φ(xi ) + b) ≥ 1 − ξi , ξi ≥ 0. 1

(1)

Table 1: Problem characteristics and performance comparisons. Applications #training #testing #features #classes Accuracy data data by users 3,089 391 1,243 4,000 04 41 4 20 21 2 3 2 Accuracy by our procedure 75.2% 96.9% 36% 85.2% 4.88% 87.8%

Astroparticle1 Bioinformatics2 Vehicle3

Here training vectors xi are mapped into a higher (maybe infinite) dimensional space by the function φ. SVM finds a linear separating hyperplane with the maximal margin in this higher dimensional space. C > 0 is the penalty parameter of the error term. Furthermore, K(xi , xj ) ≡ φ(xi )T φ(xj ) is called the kernel function. Though new kernels are being proposed by researchers, beginners may find in SVM books the following four basic kernels: • linear: K(xi , xj ) = xT xj . i • polynomial: K(xi , xj ) = (γxi T xj + r)d , γ > 0. • radial basis function (RBF): K(xi , xj ) = exp(−γ xi − xj 2 ), γ > 0. • sigmoid: K(xi , xj ) = tanh(γxi T xj + r). Here, γ, r, and d are kernel parameters.

1.1

Real-World Examples

Table 1 presents some real-world examples. These data sets are supplied by our users who could not obtain reasonable accuracy in the beginning. Using the procedure illustrated in this guide, we help them to achieve better performance. Details are in Appendix A. These data sets are at http://www.csie.ntu.edu.tw/~cjlin/papers/guide/ data/ Courtesy of Jan Conrad from Uppsala University, Sweden. Courtesy of Cory Spencer from Simon Fraser University, Canada (Gardy et al., 2003). 3 Courtesy of a user from Germany. 4 As there are no testing data, cross-validation instead of testing accuracy is presented here. Details of cross-validation are in Section 3.2. 2 1

2

1.2

Proposed Procedure

Many beginners use the following procedure now: • Transform data to the format of an SVM package • Randomly try a few kernels and parameters • Test We propose that beginners try the following procedure first: • Transform data to the format of an SVM package • Conduct...
tracking img