Convex Optimization

Stephen Boyd

Department of Electrical Engineering

Stanford University

Lieven Vandenberghe

Electrical Engineering Department

University of California, Los Angeles

cambridge university press

Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, S˜o Paolo, Delhi a

Cambridge University Press

The Edinburgh Building, Cambridge, CB2 8RU, UK

Published in the United States of America by Cambridge University Press, New York http://www.cambridge.org

Information on this title: www.cambridge.org/9780521833783

c Cambridge University Press 2004

This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without

the written permission of Cambridge University Press.

First published 2004

Seventh printing with corrections 2009

Printed in the United Kingdom at the University Press, Cambridge A catalogue record for this publication is available from the British Library Library of Congress Cataloguing-in-Publication data

Boyd, Stephen P.

Convex Optimization / Stephen Boyd & Lieven Vandenberghe

p. cm.

Includes bibliographical references and index.

ISBN 0 521 83378 7

1. Mathematical optimization. 2. Convex functions. I. Vandenberghe, Lieven. II. Title. QA402.5.B69 2004

519.6–dc22

2003063284

ISBN 978-0-521-83378-3 hardback

Cambridge University Press has no responsiblity for the persistency or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

For

Anna, Nicholas, and Nora

Dani¨l and Margriet

e

Contents

Preface

1 Introduction

1.1 Mathematical optimization . . . . . .

1.2 Least-squares and linear programming

1.3 Convex optimization . . . . . . . . . .

1.4 Nonlinear optimization . . . . . . . .

1.5 Outline . . . . . . . . . . . . . . . . .

1.6 Notation . . . . . . . . . . . . . . . .

Bibliography . . . . . . . . . . . . . . . . .

I

xi

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

1

.1

.4

.7

.9

. 11

. 14

. 16

Theory

2 Convex sets

2.1 Aﬃne and convex sets . . . . . . . . .

2.2 Some important examples . . . . . . .

2.3 Operations that preserve convexity . .

2.4 Generalized inequalities . . . . . . . .

2.5 Separating and supporting hyperplanes

2.6 Dual cones and generalized inequalities

Bibliography . . . . . . . . . . . . . . . . .

Exercises . . . . . . . . . . . . . . . . . . .

19

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

21

21

27

35

43

46

51

59

60