Tikhonov or lasso regularization: Which is better and when

Fei Wang, Sanjay Chawla, Wei Liu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations

Abstract

It is well known that supervised learning problems with ℓ1 (Lasso) and ℓ2 (Tikhonov or Ridge) regularizers will result in very different solutions. Forexample, the ℓ1 solution vector will be sparser and can potentially beused both for prediction and feature selection. However, given a data set it isoften hard to determine which form of regularizationis more applicable in a given context. In this paper we use mathematical propertiesof the two regularization methods followed by detailed experimentation to understand their impact basedon four characteristics: non-stationarity of the data generating process, level of noise in the data sensingmechanism, degree of correlation between dependent and independent variables and the shape of the data set. The practical outcome of our research is that it can serve as a guide forpractitioners of large scale data mining and machine learning tools in their day-to-day practice.

Original languageEnglish
Title of host publicationProceedings - 25th International Conference on Tools with Artificial Intelligence, ICTAI 2013
Pages795-802
Number of pages8
DOIs
StatePublished - 2013
Externally publishedYes
Event25th IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2013 - Washington, DC, United States
Duration: Nov 4 2013Nov 6 2013

Publication series

NameProceedings - International Conference on Tools with Artificial Intelligence, ICTAI
ISSN (Print)1082-3409

Conference

Conference25th IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2013
Country/TerritoryUnited States
CityWashington, DC
Period11/4/1311/6/13

Keywords

  • Classification
  • Lasso
  • Regularization

Fingerprint

Dive into the research topics of 'Tikhonov or lasso regularization: Which is better and when'. Together they form a unique fingerprint.

Cite this