Independent Component Analysis a Tutorial Introduction
INDEPENDENT COMPONENT ANALYSIS
A Tutorial Introduction James V. Stone
Independent Component Analysis
Independent Component Analysis
A Tutorial Introduction
James V. Stone
A Bradford Book The MIT Press Cambridge, Massachusetts London, England
© 2004 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.
A Typeset by the author using L TEX∂ 2ε .
Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Stone, James V . Independent component analysis : a tutorial introduction / James V. Stone. p. cm. “A Bradford book” Includes bibliographical references and index. ISBN 0-262-69315-1 (pbk.: alk. paper) 1. Neural networks (Computer science) 2. Multivariate analysis. I. Title. QA76.87.S78 2004 006.3'2—dc22 2004042589 10 9 8 7 6 5 4 3 2 1
To Nikki, Sebastian, and Teleri
Contents
Preface Acknowledgments Abbreviations Mathematical Symbols I 1 1.1 1.2 1.3 1.4 1.5 1.6 2 2.1 2.2 2.3 2.4 2.5 2.6 II Independent Component Analysis and Blind Source Separation Overview of Independent Component Analysis Introduction Independent Component Analysis: What Is It? How Independent Component Analysis Works Independent Component Analysis and Perception Principal Component Analysis and Factor Analysis Independent Component Analysis: What Is It Good For? Strategies for Blind Source Separation Introduction Mixing Signals Unmixing Signals The Number of Sources and Mixtures Comparing Strategies Summary The Geometry of Mixtures
xi xiii xv xvii 1 5 5 5 8 8 9 10 13 13 13 14 17 18 18 19 21 21 21 21 22 24 24 27 29 31 31 33 34 35 38 39
3 Mixing and Unmixing 3.1 Introduction 3.2 Signals, Variables, and Scalars 3.2.1 Images as Signals 3.2.2 Representing Signals: Vectors and Vector Variables 3.3 The
A Tutorial Introduction James V. Stone
Independent Component Analysis
Independent Component Analysis
A Tutorial Introduction
James V. Stone
A Bradford Book The MIT Press Cambridge, Massachusetts London, England
© 2004 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.
A Typeset by the author using L TEX∂ 2ε .
Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Stone, James V . Independent component analysis : a tutorial introduction / James V. Stone. p. cm. “A Bradford book” Includes bibliographical references and index. ISBN 0-262-69315-1 (pbk.: alk. paper) 1. Neural networks (Computer science) 2. Multivariate analysis. I. Title. QA76.87.S78 2004 006.3'2—dc22 2004042589 10 9 8 7 6 5 4 3 2 1
To Nikki, Sebastian, and Teleri
Contents
Preface Acknowledgments Abbreviations Mathematical Symbols I 1 1.1 1.2 1.3 1.4 1.5 1.6 2 2.1 2.2 2.3 2.4 2.5 2.6 II Independent Component Analysis and Blind Source Separation Overview of Independent Component Analysis Introduction Independent Component Analysis: What Is It? How Independent Component Analysis Works Independent Component Analysis and Perception Principal Component Analysis and Factor Analysis Independent Component Analysis: What Is It Good For? Strategies for Blind Source Separation Introduction Mixing Signals Unmixing Signals The Number of Sources and Mixtures Comparing Strategies Summary The Geometry of Mixtures
xi xiii xv xvii 1 5 5 5 8 8 9 10 13 13 13 14 17 18 18 19 21 21 21 21 22 24 24 27 29 31 31 33 34 35 38 39
3 Mixing and Unmixing 3.1 Introduction 3.2 Signals, Variables, and Scalars 3.2.1 Images as Signals 3.2.2 Representing Signals: Vectors and Vector Variables 3.3 The