Coordinated Multiuser Communications

COORDINATED MULTIUSER COMMUNICATIONS Coordinated Multiuser Communications by CHRISTIAN SCHLEGEL University of Albert...

Author: Christian Schlegel | Alex Grant

41 downloads 744 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

COORDINATED MULTIUSER COMMUNICATIONS

Coordinated Multiuser Communications by

CHRISTIAN SCHLEGEL University of Alberta, Edmonton, Canada and

ALEX GRANT University of South Australia, Adelaide, Australia

A C.I.P. Catalogue record for this book is available from the Library of Congress.

ISBN-10 ISBN-13 ISBN-10 ISBN-13

1-4020-4074-1 (HB) 978-1-4020-4074-0 (HB) 1-4020-4075-X ( e-book) 978-1-4020-4075-7 (e-book)

Published by Springer, P.O. Box 17, 3300 AA Dordrecht, The Netherlands. www.springer.com

Printed on acid-free paper

All Rights Reserved © 2006 Springer No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Printed in the Netherlands.

to Rhonda and Robyn

Contents

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 The Dawn of Digital Communications . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Multiple Terminal Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Multiple-Access Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 Degrees of Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.4.1 Transmitter and Receiver Cooperation . . . . . . . . . . . . . . . 7 1.4.2 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.4.3 Fixed Allocation Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.5 Network vs. Signal Processing Complexity . . . . . . . . . . . . . . . . . . 10 1.6 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2

Linear Multiple-Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Continuous Time Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Discrete Time Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Matrix-Algebraic Representation . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Symbol Synchronous Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Principles of Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Suﬃcient Statistics and Matched Filters . . . . . . . . . . . . . . 2.5.2 The Correlation Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3 Single-User Matched Filter Detector . . . . . . . . . . . . . . . . . 2.5.4 Optimal Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.5 Individually Optimal Detection . . . . . . . . . . . . . . . . . . . . . 2.6 Access Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Time and Frequency Division Multiple-Access . . . . . . . . . 2.6.2 Direct-Sequence Code Division Multiple Access . . . . . . . 2.6.3 Narrow Band Multiple-Access . . . . . . . . . . . . . . . . . . . . . . . 2.6.4 Multiple Antenna Channels . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.5 Cellular Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.6 Satellite Spot-Beams Channels . . . . . . . . . . . . . . . . . . . . . . 2.7 Sequence Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Orthogonal and Unitary Sequences . . . . . . . . . . . . . . . . . .

13 14 17 18 21 22 24 25 27 29 30 31 31 32 36 37 39 41 42 42

VIII

Contents

2.7.2 Hadamard Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3

Multiuser Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 The Multiple-Access Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Probabilistic Channel Model . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 The Capacity Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Binary-Input Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Binary Adder Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Binary Multiplier Channel . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Gaussian Multiple-Access Channels . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Scalar Gaussian Multiple-Access Channel . . . . . . . . . . . . . 3.4.2 Code-Division Multiple-Access . . . . . . . . . . . . . . . . . . . . . . 3.5 Multiple-Access Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Convolutional and Trellis Codes . . . . . . . . . . . . . . . . . . . . . 3.6 Superposition and Layering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Asynchronous Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 45 46 46 48 54 54 59 59 59 63 73 75 81 81 84 90

4

Multiuser Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.2 Optimal Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.2.1 Jointly Optimal Detection . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.2.2 Individually Optimal Detection: APP Detection . . . . . . . 107 4.2.3 Performance Bounds – The Minimum Distance . . . . . . . . 109 4.3 Sub-Exponential Complexity Signature Sequences . . . . . . . . . . . 112 4.4 Signal Layering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 4.4.1 Correlation Detection – Matched Filtering . . . . . . . . . . . . 118 4.4.2 Decorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.4.3 Error Probabilities and Geometry . . . . . . . . . . . . . . . . . . . 120 4.4.4 The Decorrelator with Random Spreading Codes . . . . . . 122 4.4.5 Minimum-Mean Square Error (MMSE) Filter . . . . . . . . . 124 4.4.6 Error Performance of the MMSE . . . . . . . . . . . . . . . . . . . . 126 4.4.7 The MMSE Receiver with Random Spreading Codes . . . 127 4.4.8 Whitening Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.4.9 Whitening Filter for the Asynchronous Channel . . . . . . . 132 4.5 Diﬀerent Received Power Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 4.5.1 The Matched Filter Detector . . . . . . . . . . . . . . . . . . . . . . . 134 4.5.2 The MMSE Filter Detector . . . . . . . . . . . . . . . . . . . . . . . . . 135

Contents

IX

5

Implementation of Multiuser Detectors . . . . . . . . . . . . . . . . . . . . 139 5.1 Iterative Filter Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 5.1.1 Multistage Receivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 5.1.2 Iterative Matrix Solution Methods . . . . . . . . . . . . . . . . . . . 142 5.1.3 Jacobi Iteration and Parallel Cancellation Methods . . . . 143 5.1.4 Stationary Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . 147 5.1.5 Successive Relaxation and Serial Cancellation Methods . 148 5.1.6 Performance of Iterative Multistage Filters . . . . . . . . . . . 151 5.2 Approximate Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . . . . 158 5.2.1 Monotonic Metrics via the QR-Decomposition . . . . . . . . 159 5.2.2 Tree-Search Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 5.2.3 Lattice Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 5.3 Approximate APP Computation . . . . . . . . . . . . . . . . . . . . . . . . . . 170 5.4 List Sphere Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 5.4.1 Modiﬁed Geometry List Sphere Detector . . . . . . . . . . . . . 172 5.4.2 Other Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

6

Joint Multiuser Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 6.2 Single-User Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 6.2.1 The Projection Receiver (PR) . . . . . . . . . . . . . . . . . . . . . . . 179 6.2.2 PR Receiver Geometry and Metric Generation . . . . . . . . 182 6.2.3 Performance of the Projection Receiver . . . . . . . . . . . . . . 185 6.3 Iterative Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.3.1 Signal Cancellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 6.3.2 Convergence – Variance Transfer Analysis . . . . . . . . . . . . 195 6.3.3 Simple FEC Codes – Good Codeword Estimators . . . . . . 202 6.4 Filters in the Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 6.4.1 Per-User MMSE Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 6.4.2 Low-Complexity Iterative Loop Filters . . . . . . . . . . . . . . . 214 6.4.3 Examples and Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . 217 6.5 Asymmetric Operating Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 219 6.5.1 Unequal Received Power Levels . . . . . . . . . . . . . . . . . . . . . 220 6.5.2 Optimal Power Proﬁles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 6.5.3 Unequal Rate Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 228 6.5.4 Finite Numbers of Power Groups . . . . . . . . . . . . . . . . . . . . 232 6.6 Proof of Lemma 6.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

A

Estimation and Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 A.1 Bayesian Estimation and Detection . . . . . . . . . . . . . . . . . . . . . . . . 237 A.2 Suﬃciency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 A.3 Linear Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 A.4 Quadratic Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 A.4.1 Minimum Mean Squared Error . . . . . . . . . . . . . . . . . . . . . . 242 A.4.2 Cram´er-Rao Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

X

Contents

A.4.3 Jointly Gaussian Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 A.4.4 Linear MMSE Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 245 A.5 Hamming Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 A.5.1 Minimum probability of Error . . . . . . . . . . . . . . . . . . . . . . . 246 A.5.2 Relation to the MMSE Estimator . . . . . . . . . . . . . . . . . . . 246 A.5.3 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . 246 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

List of Figures

1.1 1.2 1.3 1.4 1.5

Basic setup for Shannon’s channel coding theorem. . . . . . . . . . . . Multi-terminal networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A historical overview of multiuser communications. . . . . . . . . . . . Multiple-access channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Degrees of cooperation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 3 5 6 8

2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8

Simpliﬁed two-user linear multiple-access channel. . . . . . . . . . . . . Continuous time linear multiple-access channel. . . . . . . . . . . . . . . Sampling of the modulation waveform. . . . . . . . . . . . . . . . . . . . . . . The modulation vectors sk [i]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Synchronous model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Symbol synchronous matched ﬁltered model. . . . . . . . . . . . . . . . . . Structure of the cross-correlation matrix. . . . . . . . . . . . . . . . . . . . . Symbol synchronous single-user correlation detection for antipodal modulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimal joint detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modulating waveform built from chip waveforms. . . . . . . . . . . . . . Chip match-ﬁltered model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiple transmit and receive antennas. . . . . . . . . . . . . . . . . . . . . . Simpliﬁed cellular system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Satellite spot beam up-link. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13 15 19 20 23 25 26

Two-user multiple-access channel. . . . . . . . . . . . . . . . . . . . . . . . . . . Example of a discrete memoryless multiple-access channel. . . . . . Coded multiple-access system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Two-user achievable rate region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Three-user achievable rate region. . . . . . . . . . . . . . . . . . . . . . . . . . . Two-user binary adder channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Convex hull of two achievable rate regions for the two-user binary adder channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Capacity region of the two-user binary adder channel. . . . . . . . . .

47 49 50 52 52 54

2.9 2.10 2.11 2.12 2.13 2.14 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8

28 30 33 34 37 40 41

56 57

XII

List of Figures

3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25 3.26 3.27 3.28 4.1 4.2 4.3 4.4 4.5 4.6

4.7 4.8 4.9

Channel as seen by user two. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Capacity region of the two-user binary multiplier channel. . . . . . Example of Gaussian multiple-access channel capacity region. . . Rates achievable with orthogonal multiple-access. . . . . . . . . . . . . . Convergence of spectral density. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Spectral eﬃciency of DS-CDMA with optimal, orthogonal and random spreading. Eb /N0 = 10 dB. . . . . . . . . . . . . . . . . . . . . . . . . . Spectral eﬃciency of DS-CDMA with random spreading. . . . . . . Random sequence capacity with Rayleigh fading. . . . . . . . . . . . . . Finding the asymptotic spectral eﬃciency via a geometric construction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rates achieved by some existing codes for the BAC . . . . . . . . . . . Combined 2 user trellis for the BAC . . . . . . . . . . . . . . . . . . . . . . . . Two user nonlinear trellis code for the BAC . . . . . . . . . . . . . . . . . Successive cancellation approach to achieve vertex of capacity region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Two-user MAC with perfect feedback. . . . . . . . . . . . . . . . . . . . . . . Capacity region for the two-user binary adder channel with feedback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple feedback scheme for the binary adder channel. . . . . . . . . . Channel seen by V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Capacity region for the two-user GMAC channel with feedback. Capacity region for symbol-asynchronous two-user Gaussian multiple-access channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Capacity region for two-user collision channel without feedback.

58 60 61 63 66 68 68 73 74 75 82 82 83 85 88 89 89 90 94 95

Classiﬁcation of multiuser detection and decoding methods. . . . . 98 A joint detector considers all available information. . . . . . . . . . . . 100 Matched ﬁlter bank serving as a front-end for an optimal multiuser CDMA detector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Illustration of the correlation matrix R for three asynchronous users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Illustration of the recursive computation of the quadratic form in (4.11). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Illustration of a section of the CDMA trellis used by the optimal decoder, shown for three interfering users, i.e. K = 3, causing 8 states. Illustrated is the merger at state s, where each of the path arrives with the metric (4.12). . . . . . . . . . . . . . . 105 Illustration of the forward and backward recursion of the APP algorithm for individually optimal detection. . . . . . . . . . . . . . . . . . 108 Bounded tree search. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Histograms of the distribution of the minimum distances of a CDMA system with length-31 random spreading sequences, for K = 31 (dashed lines), and K = 20 users (solid lines), and maximum width of the search tree. . . . . . . . . . . . . . . . . . . . . . . . . . 113

List of Figures

XIII

4.10 Linear preprocessing used to condition the channel for a given user (shaded). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.11 Information theoretic capacities of various preprocessing ﬁlters. 117 4.12 Geometry of the decorrelator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.13 Shannon Bounds for the AWGN channel and the random CDMA decorrelator-layered channel. Compare with Figure 4.11. 124 4.14 Shannon bounds for the AWGN channel and the MMSE layered single-user channel for random CDMA. Compare with Figures 4.11 and 4.13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 4.15 The partial decorrelating feedback detector uses a whitened matrix ﬁlter as a linear processor, followed by successive cancellation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 4.16 Shannon Bounds for an MMSE joint detector for the unequal received power scenarios of one strong user, and equal power for the remaining users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 4.17 Shannon Bounds for an MMSE joint detector the case of two power classes with equal numbers of users in each group. . . . . . . 138 5.1 5.2 5.3 5.4 5.5

5.6 5.7

5.8 5.9 5.10 5.11 5.12 5.13

Illustration of the asynchronous blocks in the correlation matrix R. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 The multistage receiver for synchronous and asynchronous CDMA systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Example performance of a parallel cancellation implementation of the decorrelator as a function of the number of iteration steps.144 BER Performance of Jacobi Receivers versus system load for an equal power system, i.e. A = I. . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Visualization of the Gauss-Seidel update method as iterative minimization procedure, minimizing one variable at a time. The algorithm starts at point A. . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Iterative MMSE ﬁlter implementations for random CDMA systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Shannon bounds for the AWGN channel and multistage ﬁlter approximations of the MMSE ﬁlter for a random CDMA system with load β = 0.5. The iteration constant τ was chosen according to (5.39). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Similar Shannon bounds for multistage ﬁlter approximations of the decorrelator with load β = 0.5. . . . . . . . . . . . . . . . . . . . . . . . 157 Three-user binary tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 Performance of the IDDFD for a system with 20 active users and random spreading sequences of length 31. . . . . . . . . . . . . . . . . 164 Performance of the IDDFD under an unequal received power situation with one strong user. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Performance of diﬀerent multiuser decoding algorithms as a function of the number of active users at Eb /σ 2 = 7 dB. . . . . . . . 166 Sphere detector performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

XIV

List of Figures

5.14 Sphere detector average complexity. . . . . . . . . . . . . . . . . . . . . . . . . 170 6.1

6.2 6.3 6.4 6.5 6.6

6.7 6.8 6.9 6.10

6.11 6.12

6.13

6.14 6.15 6.16 6.17 6.18

6.19

Comparison of the per-dimension capacity of optimal and linearly processed random CDMA channels. The solid lines are for β = 0.5 for both linear and optimal processing, the dashed lines are for a full load β = 1. . . . . . . . . . . . . . . . . . . . . . . . 177 Diagram of a “coded CDMA” system, i.e. a CDMA system using FEC coding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 Projection Receiver block diagram using an embedded decorrelator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Projection Receiver diagram using an embedded decorrelator. . . 183 Lower bound on the performance loss of the PR. . . . . . . . . . . . . . 186 Performance examples of the PR for random CDMA. The dashed lines are from applying the bound from Theorem 6.1. The values of Eb /N0 is in dB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Iterative multiuser decoder with soft information exchange. . . . . 193 Soft cancellation variance transfer curve. . . . . . . . . . . . . . . . . . . . . 196 Code VT curves for a selection of low-complexity FEC codes. . . 197 VT chart and iteration example for a highly loaded CDMA system. FEC code VT is dashed, the cancellation VT curve is the solid curve. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Illustration of the turbo eﬀect of iterative joint CDMA detection.199 Illustration of variance transfer curves of various powerful error control codes, such as practical-sized LDPC codes of length N = 5000 and code rate R = 0.5, as well as two serially concatenated turbo codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 VT transfer chart and iteration example for a highly loaded CDMA system using a strong serially concatenated turbo code (SCC 2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Determination of the limiting performance of FEC coded CDMA systems via VT curve matching for Eb /N0 → ∞. . . . . . . 203 Bit error performance of SCC2 from Table 6.1 as a function of the number of iterations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Interference cancellation with a weak rate R = 1/3 repetition code, acting as non-linear layering ﬁlter. . . . . . . . . . . . . . . . . . . . . . 205 Achievable spectral eﬃciencies using linear and nonlinear layering processing in equal power CDMA systems. . . . . . . . . . . . 210 Variance transfer curves for matched ﬁlter (simple) cancellation and per-user MMSE ﬁlter cancellation (dashed lines) for β = 2 and Es /N0 = 0dB and Es /N0 → ∞. . . . . . . . . . . 213 Variance transfer curves for various multi-stage loop ﬁlters for a β = 2 and two values of the signal-to-noise ratio: P/σ 2 = 3dB and P/σ 2 = 23dB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

List of Figures

XV

6.20 Variance transfer chart for an iterative decoder using convolutional error control codes and a two-stage loop ﬁlter, showing an open channel at Eb /N0 = 4.5dB. . . . . . . . . . . . . . . . . . 217 6.21 Bit error rate performance of an iterative canceler with a two-stage loop ﬁlter for 1,10,20, and 30 iterations, compared to the performance of an MMSE loop ﬁlter. . . . . . . . . . . . . . . . . . . 218 6.22 Optimal and linear preprocessing capacities for various system loads for random CDMA signaling. . . . . . . . . . . . . . . . . . . . . . . . . . 218 6.23 Performance of low-rate repetition codes in high-load CDMA systems, compared to single-user layered capacities for matched and MMSE ﬁlter systems. . . . . . . . . . . . . . . . . . . . . . . . . . 220 6.24 CDMA spectral eﬃciencies achievable with iterative decoding with diﬀerent power groups assuming ideal FEC coding with rates R = 1/3 for simple cancellation as well as MMSE cancellation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 6.25 Capacity polytope illustrated for a three-dimensional multiple-access channels. User 2 is decoded ﬁrst, than user 1, and ﬁnally user 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 6.26 CDMA spectral eﬃciencies achievable with iterative decoding with equal power groups assuming ideal FEC coding with optimized rates according to (6.124). . . . . . . . . . . . . . . . . . . . . . . . . 231 6.27 Illustration of diﬀerent power levels and average VT characteristics shown for both a serial turbo code (on the left) and a convolutional code (on the right). The system parameters are K1 = 22, K2 = 18, K3 = 16 for the SCC system at an Eb /N0 = 13.45dB, and K1 = K2 = K3 = 20 at and Eb /N0 = 8.88dB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

List of Tables

3.1 3.2 3.3 3.4

6.1

Coding schemes shown in Figure 3.18. . . . . . . . . . . . . . . . . . . . . . . Uniquely decodeable rate 1.29 code for the two-user BAC. . . . . . Non uniquely decodeable code for the two-user BAC. . . . . . . . . . Rate R = (0.571, 0.558) uniquely decodeable code for the two-user binary adder channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

76 76 77 78

Serially Concatenated Codes of total rate 1/3 whose VT curves are shown in Figure 6.12. For details on serial concatenated turbo codes, see [120, Chapter 11]. . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Preface

Mathematical communications theory as we know it today is a fairly young, but rapidly maturing science just over 50 years old. Multiple-user theories extend back to the same recent birthplace, but are only recently showing the ﬁrst early signs of maturation. The goal of this book is to present both classical and new approaches to the problems of designing co-ordinated communications systems for large numbers of users. The problems of reliable information transfer are in most cases intertwined with the problems of allocating the sparse resources available for use. The multiuser philosophy attempts to optimize whole systems, by combining the multiple-access and information transmission aspects. It is the purpose of this book to introduce the reader to the concepts involved in designing multiple-user communications systems. To achieve this goal, conventional multiple-access strategies are contrasted with newer techniques, such as joint detection and interference cancellation. Emphasizing the theory and practice of unifying accessing and transmission aspects of communications, we hope that this book will be a valuable reference for students and engineers, presenting many years of research in book format. Chapter 2 sets out the main area of interest of the book, namely the linear multiple-access channel. The emphasis is on obtaining a general model with wide application. Chapter 3 gives an overview of results from multiuser information theory, concentrating on the multiple-access channel. The remainder of the book, Chapters 4–6 are devoted to the design and analysis of multiuser detectors and decoders. Chapter 4 describes joint detection strategies for uncoded systems, and implementation details for such detectors are considered in Chapter 5. Joint decoders for systems with error control coding is the subject of Chapter 6, which concentrates on the iterative decoding paradigm. The multiple-user communications philosophy does not solve the world’s communications needs. With every problem addressed, others are peering out of dark places under the guise of complexity. It is the goal of this book to put the tools, techniques and most importantly the philosophy of multiple-user communications systems into the hands and minds of the reader.

XX

Preface

As we write these ﬁnal words, it seems that the information and communications theory community is embarking on a renewed multiple-user revolution, far beyond the scope of this book. We look forward with great anticipation to what the future holds for communications networks.

Park City, Utah and Adelaide, South Australia May 2005

Christian Schlegel Alex Grant

1 Introduction

1.1 The Dawn of Digital Communications Early in the last century, a fundamental result by Nyquist, the sampling theorem, ushered in the era of digital communications. Nyquist [91] showed that a band-limited signal, that is, a signal whose spectral representation was sharply contained in a given frequency band, could be represented by a ﬁnite number of samples. These samples can be used to exactly reconstruct the original signal. In other words, the sampling theorem showed that it is suﬃcient to know a signal at discrete time intervals, and there is no need to store an entire signal waveform. This discretization of time for purposes of information transmission was a very important starting point for the sweeping success of digital information representation and communications later in the 20th century. In 1948, Shannon [123] showed that the time-discrete samples used to represent a communications signal could also be discretized in amplitude, and that the number of levels of amplitude discretization depended on the noise present in the communications channel. This is in essence Shannon’s celebrated channel coding theorem, which assigns to any given communications channel a capacity, which represents the largest number of (digital) information bits that can be reliably transported through this channel. Combined with Nyquist’s sampling theorem, Shannon’s channel coding theorem states that information can be transported in discrete amounts at discrete time intervals without compromising optimality. That is, packaging information into time-discrete units with discrete (possibly fractional) numbers of bits in each unit is the optimal way of transmitting information. This realization has had a profound impact on communications and information processing. Virtually all information nowadays is represented, processed, and transported in discrete digital form. The Shannon channel coding theorem clearly played a pivotal role in this drive towards digital signaling. It quantiﬁes the fundamental limit of the information carrying capacity of a communications channel between a single transmitter and a single user. This set-up is illustrated in Figure 1.1, where

2

1 Introduction

a transmitter sends time-discrete symbols from a (typically) ﬁnite signaling alphabet through a transmission channel. The channel is the sum total of all that happens to the signal from transmitter to receiver. It includes distortion, noise, interference, and other eﬀects the environment has on the signal. The receiver extracts the transmitted information from the channel output signal. It can do so only if the transmission rate R, in bits per symbol, is smaller than the channel capacity C, also measured in bits per symbol.

Channel

Information Source

Message

Signal Transmitter

+

Received Signal

Message Receiver

Destination

Noise Source

Fig. 1.1. Basic setup for Shannon’s channel coding theorem.

Shannon’s channel coding theorem says more, however. The above statement is generally known as the converse to the channel coding theorem, stating what is not possible, i.e. where the limits are in terms of admissible rates. The direct part of the theorem states that there exist encoding and decoding procedures that allow the transmitted rate R approach the channel capacity arbitrarily closely with an error rate that can be made arbitrarily small. The cost to achieve this lies in the length of the code, that is, the block of data that is processed as a unit has to grow in size. Additionally, the complexity of the decoding algorithm increases as well, leading to ever more complex error control decoding circuits which can push rates closer to the capacity of the channel. Typically, the computation of the channel capacity is a fairly straightforward exercise, while the design, study, and analysis of capacity achieving encoding and decoding methods can be exceedingly diﬃcult. For example, if the transmitter is restricted to transmit with average power P and the channel is aﬀected only by additive white Gaussian noise with power N , and has a bandwidth of W Hz, its capacity is given by P C = W log2 1 + , [bits/s/Hz]. (1.1) N Equation (1.1) is arguably the most famous of Shannon’s formulas, and its simplicity belies the depth of the channel coding theorem associated with it.

1.2 Multiple Terminal Networks

3

We will encounter channels such as that one in Figure 1.1 repeatedly throughout this book. It behooves us to recall that the tremendous progress towards realizing the potential of a communications channel via complex encoding and decoding algorithms was mainly fueled by the enormous strides that the technology of very large-scale integrated (VLSI) circuits and systems has made over the last 5 decades since the invention of the transistor, also in 1948.

1.2 Multiple Terminal Networks In real-world situations, the clean arrangement of a single transmitter and a single receiver is more and more becoming a special case, as most transmissions are occurring in a multi-terminal network environment, which consists of a potentially large number of transmitters and receivers. Communication takes place over a common physical channel. The messages from each transmitter are only intended for some subset of the receivers, but may be received by all receivers. This is illustrated in Figure 1.2 in which network nodes are shown as circles and transmissions are the links..

8 2 9

1

5 4 3 7

6

Fig. 1.2. Multi-terminal networks.

Transmitters cause interference for the non-intended receivers. Traditionally, this interference has been lumped into the channel noise, and a set of single-channel models has been applied to multiple terminal communications. This is still the case with modern spread-spectrum based multiple-access systems such as the cdma2000 standard [134].

4

1 Introduction

Multiple terminal networks can be decomposed into more basic components depending on the functionality of the system or the service. •

•

•

• •

In a multiple-access channel (MAC) a number of terminals attempt to reach a common receiver via a shared joint physical channel (e.g. nodes 1, 3, 4 and 2 in Figure 1.2). This scenario is the closest to a single-user channel and is best understood theoretically. In broadcast channels the reverse is the case. A common transmitter attempts to reach a multitude of receivers (e.g. nodes 5, 8, and 9). The goal is to reach each receiver with possibly diﬀerent messages at diﬀerent data rates, at a minimum of resources. The simple description of the broadcast channel belies its theoretical challenge, and very little is known for general broadcast channels. In a relay channel information is relayed over many communications links from a transmitter to a receiver (e.g. nodes 4, 5, and 8). The relays may have no information of their own to transmit, and exist to assist the transmission of other users. In the simplest case, these links are single-user channels. The Internet uses such channels, where information traverses many communications links from source to destination. In an interference channel each transmitter has one intended receiver and one or more unintended receiver (e.g. nodes 4, 5, 6 and 7). In two-way channels both terminals act as transmitters and receivers and share a common physical channel which allows transmission to occur in both directions.

A data network comprises an arbitrary combination of all of these channels, and a general theory of network communication is still in its infancy. An early overview of multi-terminal channels can be found in [87]. A more recent treatment can be found in [24]. In this book we will deal almost exclusively with the multiple-access channel. This has several reasons. First, the multiple-access channel is the most natural extension of the single-user channel, and Shannon’s fundamental results are fairly easily extendible to this case. The multiple-access channel is also the most basic physical level channel, since, even in a more general data network, it describes the behavior of a single receiver node which is within reach of several transmitters, and the fundamental limits of the multiple-access channel apply to this situation. Lastly, the information transmission problem for the multiple-access channel, in contrast to the other multiple terminal network arrangements, can be addressed by transmitter and receiver designs conceptually analogously to single-channel designs largely by designing appropriate physical layer systems. The multiple-access model includes some very important modern-day examples, such as the code-division multiple-access (CDMA) channel and the multiple-antenna channel, a recently popularized example of a multiple-input multiple-output (MIMO) channel (see Chapter 3).

1.2 Multiple Terminal Networks

5

Figure 1.3 shows a coarse time-line of some major developments in multiple-access information theory, signaling, and receiver design for the multiple-access channel. Multiple-terminal information theory began with Shannon, who considered the two way channel. He also claimed to have found the capacity region of the multiple access channel, but this was never published, and it was not until the early 1970s that research into multiple-terminal information theory became widespread. Successively more detailed and more general channel models have been considered since then, and this progress is subject of Chapter 3, which also describes some of the early attempts at code design for the multiple-access channel. In the mid 1970s it was realized that the performance of multiple-access receivers for uncoded transmissions could be improved using joint detection, at a cost of increased implementation complexity. This motivated much research into practical signal processing methods for joint detection, particularly linear ﬁltering methods. This is the subject of Chapter 4 and 5, the latter dealing speciﬁcally with implementation details. After about 1996, much of the receiver design and analysis has focused on iterative turbo-type receivers for coded transmissions. These methods are discussed in detail in Chapter 6 of this book.

4WO7AY#HANNEL 3HANNON

-ULTIPLE!CCESS !HLSWEDE ,IAO

"ROADCAST #OVER

'AUSSIAN-!# 7YNER #OVER

#ORRELATED3OURCES 3LEPIAN7OLF

)NTERFERENCE#HANNEL #ARLEIAL "ERGMANS 3ATO

-!#WITH&EEDBACK 'AARDER7OLF #OVER,EUNG

&RAME!SYNCH-!# 0OLTYREV (UI(UMBLET

'AUSSIAN-!# &EEDBACK /ZAROW

!SYNCH'AUSSIAN-!# 6ERDU

-!#WITH-EMORY 6ERDU

#$-!#APACITY 6ERDU

'-!#WITH-EMORY #HENG6ERDU

#$-!2ANDOM3EQUENCES 'RANT!LEXANDER 6ERDU3HAMAI

--3%2ANDOM3EQUENCES 4SE(ANLY

)TERATIVE0ERFORMANCE!NALYSIS "OUTROS#AIRE

)NFORMATIONTHEORY

"!#"LOCK#ODES +ASAMI,IN "!##ONVOLUTIONAL#ODES 0ETERSON#OSTELLO

/PTIMAL$ETECTION VAN%TTEN 3CHNEIDER 6ERDU

$ECORRELATOR ,UPAS6ERDU

"!#4RELLIS#ODES #HEVILLAT

-ULTIPLE!CCESS#ODES

,--3% 8IEETAL

0ROJECTION2ECEIVER 3CHLEGEL!LEXANDER

)TERATIVE)MPLEMENATIONS %LDERS "OLLETAL 2ASMUSSENETAL

)NTERFERENCE#ANCELLATION 0ATEL(OLTZMAN

/PTIMAL$ECODING 'IALLORENZI7ILSON

)NTERFERENCE#ANCELLATION (AGENAUER !LEXANDERETAL

-ULTIUSER4URBO -OHER 2EEDETAL

#ANCELLATIONWITH--3% 7ANG0OOR

!002ANDOM3EQUENCES 4ANAKA

$ETECTION-ETHODS

$ECODING-ETHODS

Fig. 1.3. A historical overview of multiuser communications.

6

1 Introduction

1.3 Multiple-Access Channel Figure 1.4 shows the basic situation of a multiple-access channel where a number of terminals communicate to a joint receiver using the same channel. As mentioned earlier, this situation arises naturally in a number of important practical cases, such a cellular telephone networks. Much is known about the multiple access channel, such as its capacity, or more appropriately its capacity region discussed in Chapter 3, which is the analog of the channel capacity in the single-channel case. The information theoretic properties of the multiple-access channel were ﬁrst investigated in [2]. An important innovation that grew out of the information theoretic results for the multiple-access channel is that users should be decoded jointly, rather than independently, treating other users as interference. The joint receiver makes use of the known structure of all the users’ transmitted signals, and potentially signiﬁcant gains can be accomplished by doing this. The how and why of joint decoding is the major theme of this book. As can be appreciated from Figure 1.4, the capacity limits of the multiple-access channel, and algorithmic ways to approach them, is of major importance in any kind of multi-terminal network, since the data ﬂow through the multi-access node is limited by the multiple-access channel it forms with the transmitting terminals. These limits apply whether data is destined for the multi-access node or not. We wish to restate that most of the concepts and results ﬁnd application in the physical layer of a communications system, albeit the next higher layer, the medium access layer, will also have to be involved in eﬃcient overall network design. Functions which are to be executed at the medium access layer include the selection of power levels, transmission rates, and transmission formats.

Information Source 1

Message 1

Information Source 2

Message 2

Information Source 3

Message 3

Transmitter 1

Multiple-Access Channel

Message 1

Received Signal

Transmitter 2

+

Transmitter 3

Noise Source

Receiver

Fig. 1.4. Multiple-access channel.

Destination 1

Message 2 Destination 2

Message 3

Destination 3

1.4 Degrees of Coordination

7

1.4 Degrees of Coordination One important conceptual and practical extension that needs to be added to the multiple-access communications problem is the coordination among the diﬀerent communicating terminals. Diﬀerent levels of coordination have a strong impact on the ability of the multi-access receiver to operate eﬃciently, even though the information theoretic capacity of the channel can be quite insensitive to these diﬀerent levels of cooperation. The main cooperation concepts are i) source cooperation and ii) synchronization: 1.4.1 Transmitter and Receiver Cooperation Diﬀerent levels of transmitter and receiver cooperation correspond to diﬀerent statistical assumptions concerning the users’ transmissions, and diﬀerent assumptions about the type of receiver signal processing. Three types of cooperation are represented in Figure 1.5 No cooperation: Each source independently transmits information. The received signal is decoded separately by diﬀerent receivers. The decoder for each user treats every other user as unwanted noise. This situation essentially turns each of the channels into an independent single-user channel, where all the eﬀects of concurrent users are lumped into channel impairments. Current cellular telephony systems and the vast majority of communications systems to date apply this methodology. Receiver cooperation: Each source independently transmits information. The receiver makes full use of the received signal by performing joint decoding. This is usually what we mean by coordinated multiuser communications. The resulting channel is the multiple-access channel with independent transmissions. Full cooperation: The sources may fully coordinate their transmissions. Joint decoding is used at the receiver. If full cooperation is allowed (requiring communication between users), there is no restriction of independence on the joint transmission of the diﬀerent users. Full cooperation allows the channel resource to be used as if by a single “super-source”, which can be used as a benchmark. In certain cases, full cooperation may not add to the fundamental capacity (region) of many practically important multipleaccess channels (such as the Gaussian multiple-access channel). In such cases, from a theoretical perspective, it is unimportant if the communicating terminals are physically separated and independent or are co-located and can coordinate their transmissions. Full cooperation can however be very important in practice to reduce the complexity burden of the receiver. The remaining possibility - transmitter cooperation, without receiver cooperation is also of interest. This is to the broadcast channel, which we do not consider in this book. Receiver cooperation with independent transmitters is the main theme of this book.

8

1 Introduction

Transmitter 1

Receiver 1

Channel

Transmitter 2

Receiver 2

(a) No cooperation.

Transmitter 1

Channel

Receiver

Transmitter 2

(b) Receiver cooperation.

Transmitter

Channel

(c) Full cooperation. Fig. 1.5. Degrees of cooperation.

Receiver

1.4 Degrees of Coordination

9

1.4.2 Synchronization The other important type of coordination between users in a network is synchronization. Whereas the diﬀerent levels of cooperation described above are concerned with the sharing of transmitted data, or coordinating transmission signals, various degrees of synchronism are a consequence of the availability of a common time reference. Depending upon the time reference available, users may align their symbols or frames. Symbol Synchronism: Users strictly align their symbol epochs to common time boundaries. It is important to note that the reference location is the receiver, i.e. the transmitted symbols must align when received. Frame Synchronism: The users align their codewords to common time boundaries. This is a less restrictive level of synchronization and therefore easier to accomplish than symbol synchronism. Phase/Frequency Synchronism: Especially in wireless transmissions the phase and frequency of the carrier signal are very important. Knowing these, a receiver can be built with signiﬁcantly less eﬀort than without that knowledge. Phase and frequency synchronization are typically only feasible when full cooperation is possible. This is the case for multipleantenna transmission systems for example. We shall see that loss of either type of synchronism may change the information rates that can be achieved. 1.4.3 Fixed Allocation Schemes One conceptually simple way to share a common channel is to use some kind of ﬁxed allocation. This is the current state of the art, and the following allocation methods are widely used: Time-Division Multiple-Access (TDMA): Users do not transmit simultaneously, they transmit in orthogonal time periods, and therefore, from a “Shannon” point of view, they all use single channels in the conventional sense. This does require that transmissions are (frame) synchronized in time, which typically adds signiﬁcant system overhead. Frequency-Division Multiple-Access (FDMA): Users transmit in orthogonal frequency bands instead of time intervals. The available channel is sliced up into a number of frequency channels which are used by a single user at a time. This requires coordination of the transmissions and frequency synchronization of the transmissions. Ideally FDMA and TDMA behave similarly on an ideal channel, however, on mobile channels an FDMA system may suﬀer more signiﬁcantly from signal fading. The pan-European cellular telephony system GSM [37] uses a combination of FDMA and TDMA.

10

1 Introduction

Code-Division Multiple-Access (CDMA): This relative newcomer uses orthogonal, or nearly orthogonal, signaling waveforms for each of the users. It should properly be called signal-space-division multiple-access, but historically, its most popular representative was CDMA, where the signal waveforms were generated by spreading codes, hence the terminology. Both time- and frequency-division can be viewed as special instances of signal-space-division. Code-division multiple access also requires transmitter synchronization. Furthermore, the signals used must be orthogonal at the receiver. Many real-world channels destroy this orthogonality, and the channel turns into a proper multiple-access channel. CDMA ﬁnds application in cellular-based telephony systems such as IS-95 [135], cdma2000 [134], and future third generation systems. Mathematical and physical details of these accessing schemes are discussed in detail in Chapter 2.

1.5 Network vs. Signal Processing Complexity State-of-the art networks such a cellular telephony systems use mostly orthogonal allocation schemes to simplify the transmission technology. This, however, puts organizational and computational burden on the network controls. Networks using ﬁxed allocation schemes must maintain synchronism – this requires reliable feedback channels from the terminals. The network must also perform the resource allocation, which requires that the complete state of the network is known to the controller. These requirements can lead to very complex network control operations, which will become more and more complex as wireless networks migrate to packet transmission formats and ad-hoc operation. The precise power control mechanism required in CDMA cellular networks is an example of such a centralized network function [154]. Joint accessing of a MAC using multiple-user coding strategies has the potential to signiﬁcantly increase the system capacity while reducing the overhead associated with network control functions. Complexity is transfered into the receiver, which is equipped with signal processing functions which can extract information streams from the correlated composite signal. With the rapid advances of VLSI technology, receiver designs which seemed impossible just years ago, are now well within reach as VLSI chips push processing powers into the tera ﬂop region. A well-designed multiuser receiver will relax or obviate the requirements on network synchronization and coordination. The properties of the signal processing functions provide for access coordination. For example, since a multiuser detector can process signals with diﬀerent received power levels (see Chapter 6), power control does not need to be realized by the network anymore. Since a multiuser detector can potentially approach the capacity of the multiple-access channel, the physical channel resources can be used optimally. This can lead to dramatic increases in system capacity.

1.6 Future Directions

11

Clearly, we are not getting anything for free, and the complexity of a multiuser receiver can be signiﬁcantly higher than that of single-channel receivers. In Chapter 5 we will present joint receiver structures which can be viewed as bridges between single-channel receivers and joint receivers, in that they rely on only moderately complex extensions of conventional single-channel receivers. In general, however, it is fair to say that complexity is transferred from the network to the receiver, where is is realized in VLSI circuits. The ability to implement complex systems with comparable ease allows the exploitation of channel resources right at the receiver, rather than having to try to compensate for ineﬃcient receiver processing with expensive network-level measures.

1.6 Future Directions As pressure rises to use expensive channel resources such as power and bandwidth more eﬃciently, highly resource-eﬃcient systems will replace less efﬁcient solutions. Multiuser receivers are required to exploit these resources optimally at the physical layer of a multiple-access channel. A level higher up, resources will likely have to allocated dynamically as needed rather than as ﬁxed allocations. This requires a high level of coordination. Intelligent medium-access control layer algorithms have to select resource allocations, but physical layer multiuser receivers will allow the optimal effect of a chosen allocation. Such receivers will also alleviate the pressure on the medium-access control layer by allowing the receivers to adjust to many channel aspects, such as diﬀerent received power levels. The dropping cost of signal processing allows eﬃcient exploitation of channel resources right at the receiver, making it possible to approach the fundamental limits of the communications channel. Joint signal processing at the receiver, combined with novel medium-access control layer control protocols will allow future data networks to harness the intrinsic capacity of the multitude of channels in a multi-terminal network. Laying the foundations of the physical layer processing methodologies for such future networks is the purpose of this book.

2 Linear Multiple-Access

Most of the multiple-access channel models of interest in this book are linear, meaning that the channel output is a linear transformation of the user’s input signals, aﬀected by additive noise. A simpliﬁed example of a two-user linear multiple-access channel is shown in Figure 2.1. The corresponding mathematical model is r(t) = d1 (t) + d2 (t) + z(t), where r(t) is the received signal, d1 (t) and d2 (t) are the signals transmitted by user 1 and 2, and z(t) is an additive noise process.

User 1

d1 (t)

r(t)

User 2

d2 (t)

Receiver

z(t)

Noise

Fig. 2.1. Simpliﬁed two-user linear multiple-access channel.

The assumption of linearity may be based for example on the underlying linear superposition of signals which occurs in radio transmissions. In many real-world applications however, various non-linear eﬀects (for example due to ampliﬁer non-linearities) may be present. Such non-linearities will be ignored in this book. The purpose of this Chapter is to develop a reasonably generic mathematical model for the linear multiple-access channel, to be adopted for the remain-

14

2 Linear Multiple-Access

der of this book. This model describes a number of users accessing a common channel by modulating their data signals with specially designed waveforms. We shall begin in Section 2.1 by describing the channel model in continuous time, and further develop a signal-space representation. We shall then discretize time in Section 2.2, via sampling and derive a linear matrix-algebraic model in Section 2.3 that will form the basis for much of the discussion in the remainder of this book. This matrix-algebraic model applies to both the asynchronous and synchronous case, which facilitates the later discussion of detection algorithms. The special case of the symbol synchronous model will be discussed in Section 2.4. Following on from this in Section 2.5 we give a brief overview of the principles of single-user and multiple-user detection for these channels. In particular, the single-user correlation detector, described in Section 2.5.3 has motivated the design of many existing multiple-access systems (historically due to its low implementation complexity). In Section 2.6 we describe some of these multiple-access systems, including time- and frequency-division methods, and show how the discrete-time linear model is specialized to each case. Of particular interest is the direct-spread code-division multiple-access channel of Section 2.6.2, which forms a common theme for the remainder of this book. The Chapter concludes with an investigation into the design of signaling waveforms for single-user detection in Section 2.7.

2.1 Continuous Time Model Figure 2.1 shows a schematic representation of a K user continuous time linear multiple-access channel. Each user’s transmitter consists of a data source, an encoder (i.e. forward error correction), and a modulator. Typically the data source and encoder output binary symbols (bits). It will be assumed that any data compression is contained within the data source, and we do not consider it explicitly. Modulation is performed in two stages, ﬁrst a baseband modulator maps the coded data onto a complex signaling constellation. The resulting complex samples are then ampliﬁed and multiplied by a channel modulation waveform, which is the method used to access the common channel. The receiver observes a signal which is the sum of delayed versions of the modulated users’ signals, together with noise. In Figure 2.1, a dashed box shows the portion of the system that we refer to as the multiple-access channel. Strictly speaking, from an information theoretic point of view, the channel modulation waveforms should not be regarded as part of the channel. For the purposes of channel modeling however, we will consider the modulation waveforms as part of the channel. We will now develop a detailed mathematical model for this linear multiple-access channel. Considering the output of the baseband modulator, each user k = 1, . . . , K generates a sequence of n data symbols dk [i], i = 1, 2, . . . , n. The subscript k

2.1 Continuous Time Model √ Data Source

Data Source

Encoder

Encoder

Baseband Modulator

Baseband Modulator

P1 s1 (t)

d1 [i] √

Multiple-Access Channel

τ1

x1 (t)

P2 s2 (t)

d2 [i] τ2

x2 (t) r(t)

Encoder

Baseband Modulator

Noise

τK dK [i]

Receiver

z(t)

√ PK sK (t) Data Source

15

xK (t)

Fig. 2.2. Continuous time linear multiple-access channel.

indexes the user and the square-bracket indexing i denotes the symbol time index. Each data symbol is selected from a baseband modulation alphabet, dk [i] ∈ Dk . The users’ alphabets need not be the same for all users, and may be discrete or continuous. In general, they may be subsets of the complex numbers, Dk ⊂ C, which allows us to use the sequences dk [i] to model various forms of complex baseband modulation. For example antipodal modulation, such as binary phase-shift keying can be modeled by Dk = {−1, +1}. Alternatively, bi-orthogonal, e.g. quaternary phase-shift keyed data can be modeled √ by Dk = {1 + j, −1 + j, −1 − j, 1 − j}/ 2. See [104, 167] for basic concepts of signal space and the complex baseband representation of signals. In Figure 2.1 the thicker lines indicate complex signal paths. For the purposes of channel modeling, we shall not be concerned with how the discrete-time symbol sequences dk [i] are generated. For example they could represent either uncoded or coded information sequences. In Chapter 3, we shall be more concerned with statistical models for the users’ messages, which is of relevance for information theory. Each data symbol has duration T seconds. We turn the discrete-time symbol sequences dk [i] into continuous-time sequences (a function of the continuous time variable t ≥ 0) by writing t , dk (t) = dk T where · is the ceiling function which returns the smallest integer that is greater than or equal to its argument. We shall use the convention that discrete-time sequences are indicated with square bracket indexing by i. Integer indices such as k and i shall start

16

2 Linear Multiple-Access

from 1. Continuous time waveforms will be indicated with the round bracket indexing by t ≥ 0. The symbol √ sequences for each user are ampliﬁed via multiplication by the amplitudes Pk ≥ 0. The resulting scaled sequences are then linearly modulated with a real or complex continuous-time waveform sk (t). It will be assumed without loss of generality that the per-symbol energy of the modulating waveforms is normalized so that 1 nT

nT

|sk (t)|2 dt = 1.

0

(recalling that n consecutive symbols are transmitted with total duration nT ). It will also be assumed that sk (t) = 0,

t < 0, t > nT.

Under the further assumption that the modulation waveforms sk (t) are independent of the data sequences dk [i] this results in a per-symbol transmit energy P . Appropriate selection of the modulation waveforms allows us to represent a wide variety of linear modulation schemes. Some examples will be developed in Section 2.6. Each transmitted waveform experiences a time delay of τk seconds. These delays can model the propagation delays inherent in the channel, and the fact that the individual users may not be synchronized in time. Without loss of generality, we can assume that 0 ≤ τk < T (delays longer than one symbol period can be modeled by re-indexing the users’ symbols). Thus (in the absence of delay) the signal contribution xk (t) of user k = 1, 2, . . . , K is t xk (t) = dk Pk sk (t). T The delay and noise-aﬀected channel output waveform r(t) is therefore given by r(t) =

K

xk (t − τk ) + z(t)

k=1

=

K

k=1

dk

t − τk T

Pk sk (t − τk ) + z(t),

(2.1)

(2.2)

where z(t) is a continuous time noise process, which is usually assumed to be Gaussian (real or complex to match the modulating waveforms and data alphabet). See [49] for a discussion of the Gaussian noise process in relation to communications systems.

2.2 Discrete Time Model

17

2.2 Discrete Time Model For the purposes of analysis and indeed implementation, it can be more convenient to work with an equivalent discrete-time model. By a discrete-time model, we mean a representation of r(t) in terms of an integer-indexed sequence r[j], j = 1, 2, . . . . By equivalent, we require r[j] to be a suﬃcient statistic for detection of the users’ data, according to Section A.2. A discrete-time model is obtained by representing the received signal r(t) using a complete orthonormal basis (see [49, Chapter 8]). Let the functions φj (t), j = 1, 2, . . . be a complete orthonormal basis. Then the discrete-time representation is

r(t) = r[j]φj (t) where r[j] = r∗ (t)φj (t) d(t). ∗

where (·) denotes complex conjugation. There are many possible choices for the basis φj , depending on the structure of the transmitted waveforms. In any practical system, the modulating waveforms sk (t) will be (at least approximately) band-limited to a interval of frequencies [−W, W ], and (approximately) time-limited to an interval [0, n T ] and in this case, one common choice of basis is the set of sampling functions sin 2W π t − j−1 2W , j = 1, 2, . . . , 2W nT + 1. (2.3) φj (t) = √ 2W π t − j−1 2W Using this basis (2.3), a ﬁnite dimension discrete-time model is obtained by sampling the received waveform (2.2) at intervals of 1/(2W ) seconds. j−1 (2.4) r[j] = r 2W

K j−1

j−1 j−1 2W − τk = , (2.5) − τk + z dk Pk sk T 2W 2W k=1

where j = 1, 2, . . . , 2W nT + 1 is the sample time index (recalling our convention to start integer indices from 1). With reference to Figure 2.3, let Tk = 2W τk be the delay of user k, rounded up and measured in samples, and for future reference deﬁne Tmax = max Tk . k

The ﬁrst non-zero sample instant for user k is at an oﬀset of τk0 =

Tk − τk 2W

18

2 Linear Multiple-Access

seconds from the start of the modulating waveform. Deﬁne the sampled modulation waveforms j−1 sk [j] = s − τk . (2.6) 2W which are zero for j < Tk + 1 and j ≥ nL + Tk + 1, where L = 2W T is the number of samples per symbol period. Using these deﬁnitions, we arrive at the following discrete-time model, r[j] =

K

dk [i] Pk sk [j] + z[j].

(2.7)

k=1

In the above equation, the data symbol index i of user k corresponding to sample number j is j 2W − τk i= , T valid for j > Tk . The sequence z[j] = z ((j − 1)/(2W )) are noise samples. Throughout this book, the additive noise will be modeled by a zero mean, circularly symmetric complex white Gaussian process with power spectral density N0 /L. In this case, the noise samples z(t) have independent zero mean Gaussian real and imaginary parts, each with variance N0 /(2L), and these samples are also independent and identically distributed (i.i.d.) over time. The received signal to noise ratio γk for user k is therefore for complex modulation Pk /N0 γk = 2Pk /N0 for real modulation.

2.3 Matrix-Algebraic Representation Much insight into the behavior of the linear multiple-access channel and the design of multiple-user detectors is gained through expressing the discretetime model (2.7) in terms of matrix-vector operations. In order to accomplish this, we need to deﬁne some appropriate vectors and matrices. Modulation Matrix Collect each user’s sampled modulation waveforms into length nL + Tmax column vectors sk [i], one vector for each symbol period of each user, with zero padding corresponding to the delays τk . The vector for user k and symbol i is

2.3 Matrix-Algebraic Representation

19

τk0 sk [5] sk (t) sk [4]

sk [1]

sk [2]

sk [3]

0

1 2W

2 2W

τk

3 2W

t

4 2W

Fig. 2.3. Sampling of the modulation waveform. sk [i] = ( 0, . . . , 0 , sk [L(i − 1) + Tk + 1], sk [L(i − 1) + Tk + 2], . . . , sk [Li + Tk ], 0, . . . )t . | {z } L(i−1)+Tk

(2.8)

Note that

n

t

sk [i] = (s[1], sk[1] , . . . , sk[nL + Tmax ])

i=1

is a vector containing the entire sampled modulation waveform for user k. Example 2.1. Figure 2.4 shows a schematic representation of the modulation vectors for user k in a system with L = 3, Tk = 2, Tmax = 3 and n = 3. The zero part of each vector is omitted for clarity. Now deﬁne the (Ln + Tmax ) × Kn modulation matrix S, which takes the vectors sk [i], k = 1, 2, . . . , K, i = 1, 2, . . . , n as columns, S = s1 [1]s2 [1] . . . sK [1]s1 [2]s2 [2] . . . sK [2] . . . s1 [i]s2 [i] . . . sK [i] . . . i.e. all the users’ vectors for symbol 1 (in order of user number), followed by all the vectors for symbol 2 and so on. Column l of S is the sampled modulation sequence of user k = ((l − 1) mod K) + 1 (2.9) corresponding to symbol i = l/K .

(2.10)

Amplitude Matrix Deﬁne a diagonal Kn × Kn matrix √ √ √ √ √ √ A = diag P1 , P2 , . . . , PK , P1 , P2 , . . . , PK , . . . which contains the user amplitudes, repeated n times. Note that time-varying amplitudes are also easily accommodated.

20

2 Linear Multiple-Access

sk [1]

sk [2]

sk [3]

j=0

Tk

sk [2] sk [3] sk [4]

Tk + L

sk [5] sk [6] sk [7]

Tk + 2L

sk [8] sk [9] sk [10]

Tk + 3L Fig. 2.4. The modulation vectors sk [i].

Data Vector Collect the users’ symbols into a single length n column vector t

d = (d1 [1], d2 [1], . . . , dK [1], . . . , d1 [i], d2 [i], . . . , dK [i], . . . , dK [n]) . The elements of this vector are in the same ordering as the columns of S, namely data symbol 1 for each user followed by data symbol 2 for each user and so on. The Basic Model for the Received Vector Finally deﬁne the length nL + Tmax column vectors t

r = (r[1], r[2], . . . , r[nL + Tmax ])

t

z = (z[1], z[2], . . . , z[nL + Tmax ])

which respectively contain the received samples and the noise samples. We can now re-write the discrete-time model (2.7) using the following matrix-vector representation

2.4 Symbol Synchronous Model

r = SAd + z.

21

(2.11)

This is the canonical linear-algebraic model for the linear-multiple-access channel. Using the notation established so far, there are two ways to refer to elements of these matrices. In cases where we wish to emphasize user and time indexes, we will use the sequence notation sk [i]. When we wish to emphasize the linear algebraic structure, we will use matrix-vector indices, sil . As explained above, the conversion between user/time indexes k, i and row/column indexes is via (2.9) and (2.10). Example 2.2. The equation below shows the structure of this linear model, for four users, K = 4 and two symbols n = 2 with T1 = 0, T2 = 1, T3 = 3, T4 = 0 and L = 4. All zero elements are omitted for clarity. ⎞ ⎛ ⎞ ⎛ ⎞ z[1] r[1] s4 [1] s1 [1] ⎞ ⎛√ ⎞⎛ ⎟ ⎜ ⎜ r[2] ⎟ ⎜s1 [2] s2 [2] ⎟ P1 √ d1 [1] s4 [2] ⎟ ⎜ z[2] ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜d2 [1]⎟ ⎜ z[3] ⎟ ⎜ r[3] ⎟ ⎜s1 [3] s2 [3] s3 [3] s4 [3] P2 √ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎟ ⎜d3 [1]⎟ ⎜ z[4] ⎟ ⎜ r[4] ⎟ ⎜s1 [4] s2 [4] s3 [4] s4 [4] P3 √ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎟ ⎜d4 [1]⎟ ⎜ z[5] ⎟ ⎜ r[5] ⎟ ⎜ [5] s [5] s [5] s [5] P s 2 3 1 4 4 ⎟ ⎜ ⎟⎜ ⎟⎜ ⎜ ⎟ ⎜ ⎟ √ ⎟⎜ ⎟ ⎜d1 [2]⎟ + ⎜ z[6] ⎟ ⎜ r[6] ⎟ = ⎜ s [6] [6] s [6] s [6] s P 2 1 3 4 1 ⎟ ⎜ ⎟⎜ ⎟⎜ ⎜ ⎟ ⎜ ⎟ √ ⎟⎜ ⎟ ⎜d2 [2]⎟ ⎜ z[7] ⎟ ⎜ r[7] ⎟ ⎜ [7] s [7] s [7] s [7] s P 1 2 3 4 2 ⎟ ⎜ ⎟⎜ ⎟⎜ ⎜ ⎟ ⎜ ⎟ √ ⎜ r[8] ⎟ ⎜ ⎟ ⎝ s1 [8] s2 [8] s3 [8] s4 [8]⎟ P3 √ ⎠ ⎝d3 [2]⎠ ⎜ ⎟ ⎜ z[8] ⎟ ⎜ ⎟ ⎜ ⎠ ⎝ z[9] ⎠ ⎝ r[9] ⎠ ⎝ s2 [9] s3 [9] d4 [2] P4 s3 [10] r[10] z[10] ⎛

2.4 Symbol Synchronous Model In the case τ1 = τ2 = · · · = τK = 0, the system is symbol synchronous. This is obviously a less general model to the asynchronous model presented above, but it allows us to write a per-symbol matrix model that is very convenient for the development of the main ideas of this book. The degree to which symbol synchronism can be achieved is a technological issue, and in practice it is typically diﬃcult to achieve perfect synchronism across all users. For the symbol synchronous model we have the following symbol-wise discrete-time matrix model. r[i] = S[i]Ad[i] + z[i],

(2.12)

where t

r[i] = (r[(i − 1)L + 1], r[(i − 1)L + 2], . . . , r[iL])

t

z[i] = (z[(i − 1)L + 1], z[(i − 1)L + 2], . . . , z[iL]) t

d[i] = (d1 [i], d2 [i], . . . , dK [i]) A = diag( P1 , . . . , PK )

22

2 Linear Multiple-Access

and S[i] is a L × K matrix with column k being the modulation waveform samples for user k corresponding to symbol i, t

sk [i] = (sk [(i − 1)L + 1], sk [(i − 1)L + 2], . . . , sk [iL]) . It is important to note that the speciﬁc waveform samples are diﬀerent to those used in the asynchronous model. Since the users are completely synchronous, τk0 = 0 for each user and j−1 , (2.13) s[j] = s 2W which are diﬀerent sampling instants compared to the asynchronous model (2.11) (compare (2.13) to (2.6)). Note that the symbol-synchronous (2.12) and the general asynchronous (2.11) models share the same basic mathematical form. In both cases, the observation (whether it is for a single symbol, or and entire block of symbols) is a noise-aﬀected linear transformation of the corresponding input. The only modiﬁcation required is in the speciﬁcation of the matrices and vectors involved. In the case of the symbol synchronous model, it is common practice to drop the symbol indexing when it does not cause confusion, and write r = SAd + z.

(2.14)

With this convention, we can use row/column indexing as explained above to write expressions that apply to both the synchronous (2.14) and asynchronous (2.11) models. This is very useful, since it allows easy translation of results between the two cases. in particular, detection algorithms developed for the synchronous channel can usually be directly applied to the asynchronous channel simply via a re-deﬁnition of the linear model. The symbol-synchronous discrete-time channel model is shown in Figure 2.5, where the summation is now a vector sum. Example 2.3. The equation below shows the structure of the symbol synchronous linear model at symbol i, for three users, K = 3 and L = 4. All zero elements are omitted for clarity. ⎞ ⎛ ⎛ ⎞ ⎞ ⎛ ⎞ ⎞⎛ r[(i − 1)L + 1] z[(i − 1)L + 1] s1 [(i − 1)L + 1] s2 [(i − 1)L + 1] s3 [(i − 1)L + 1] s4 [(i − 1)L + 1] ⎛√ P1 √ d1 [i] ⎜z[(i − 1)L + 2]⎟ ⎜r[(i − 1)L + 2]⎟ ⎜s1 [(i − 1)L + 2] s2 [(i − 1)L + 2] s3 [(i − 1)L + 2] s4 [(i − 1)L + 2]⎟ ⎟⎝ ⎜ ⎟ ⎟ ⎜ P2 √ ⎠ ⎝d2 [i]⎠ + ⎜ ⎝z[(i − 1)L + 3]⎠ ⎝r[(i − 1)L + 3]⎠ = ⎝s1 [(i − 1)L + 3] s2 [(i − 1)L + 3] s3 [(i − 1)L + 3] s4 [(i − 1)L + 3]⎠ d3 [i] P3 s2 [iL] s3 [iL] s4 [iL] s1 [iL] r[iL+] z[iL]

2.5 Principles of Detection So far we have developed sample-level discrete time models for both the asynchronous and the symbol synchronous linear multiple-access channel. Using these models, we may apply existing results from detection theory to develop a variety of detection techniques with diﬀerent levels of performance, at the expense of diﬀerent levels of implementation complexity. In almost all cases

2.5 Principles of Detection √ Baseband Modulator

Baseband Modulator

P1 s1 [i]

d1 [i] √

P2 s2 [i]

d2 [i] r[i]

√ Baseband Modulator

23

Receiver

z[i] PK sK [i]

Noise

dK [i]

Fig. 2.5. Synchronous model.

of interest, implementation of the optimal detector is prohibitively complex – its complexity increases exponentially with the number of users. The high cost of implementation for the optimal detector motivates the development of reduced-complexity sub-optimal detectors. See Appendix A for a brief introduction to Bayesian estimation and detection. Most existing multiple-access strategies have been designed with a particular sub-optimal detection method in mind, namely single-user correlation detection, also known as the single-user matched ﬁlter. Historically, the correlation detector pre-dates the optimal detector, and was the motivation behind the development of many diﬀerent multiple-access techniques, for example direct-sequence code-division multiple-access. The purpose of this section is to give a brief introduction to the principles of detection as they apply to the linear multiple-access channel. In particular, we will introduce the single-user correlation receiver as motivation for the access strategies to be developed in Section 2.6. Chapters 4 – 6 develop in further detail a whole range of detection and decoding strategies, providing a trade-oﬀ between performance and complexity. As explained in Sections 2.3 and 2.4, the discrete time asynchronous and symbol-synchronous models share the mathematical formulation r = SAd + z

(2.15)

under diﬀerent deﬁnitions of these matrices and vectors. In the asynchronous case, the modulation matrix S is nK × (nL + Tmax ) and has as column l the zero-padded portion of the sampled modulation waveform of user ((l − 1) mod K) + 1 associated with symbol l/K . The nK × nK diagonal matrix A has l-th diagonal element P((l−1) mod K)+1 . The length nK data vector d has as l-th element dl/K [((l − 1) mod K) + 1].

24

2 Linear Multiple-Access

In the symbol synchronous case, (2.15) is a symbol-wise model and at each symbol i, the L × K modulation matrix S has as columns the users’ sampled modulation waveforms √ associated with symbol i. The K × K amplitude matrix A = diag Pk and the length K data vector contains the users’ data symbols for time i. Subject to the Gaussian noise model, in both cases, the noise vector z contains either real or circularly symmetric complex i.i.d Gaussian noise samples with N0 I for complex noise, ∗ E [zz ] = N0 /2I for real noise. We will assume complex modulation. In cases where it causes no confusion, we shall simply refer to the linear model without specifying the degree of synchronism. In this way, the principles can be clearly explained without the need to describe a proliferation of special cases. This is one of the main advantages of this linear model. The basic design objective for the multiple-user detector is to ﬁnd estimates dˆk [i] satisfying various optimality criteria. 2.5.1 Suﬃcient Statistics and Matched Filters According to (2.15), the observation r, conditioned on knowledge of S, A, d and N0 is Gaussian, with mean SAd and covariance N0 I, denoted r ∼ N (SAd, N0 I). The matched ﬁlter output y, obtained by multiplying the channel output with the Hermitian transpose of the modulation sequence matrix, y = S∗ r = S∗ SAd + S∗ Az ˜ = Rd + z is a suﬃcient statistic for the detection of the transmitted symbols d (see Section A.2). The matrix R = S∗ S is called the correlation matrix. The ˜ aﬀecting the matched ﬁlter output is correlated according to resulting noise z ˜∗ ] = RN0 . E [˜ zz The matched-ﬁlter model is shown schematically in Figure 2.6 (for the symbol synchronous case). A suﬃcient statistic preserves the statistical properties of the observation. Hence the statistic y is an information lossless, reduced dimension representation of the original observation r. In the asynchronous case, R is nK × nK. For the synchronous model, R is K × K. The correlation matrix is of particular importance in multiuser detection and its structure will be described in further detail in Section 2.5.2.

2.5 Principles of Detection

25

Matched Filter Front End

√ Baseband Modulator

d1 [i] √

Baseband Modulator

s1 [i]

P1 s1 [i]

Reset

P y1 [i] s2 [i]

P2 s2 [i]

d2 [i]

P y2 [i] r[i]

Receiver

z[i] √ Baseband Modulator

PK sK [i]

Noise

dK [i]

sK [i] P yK [i]

Fig. 2.6. Symbol synchronous matched ﬁltered model.

Using either r or y we can ﬁnd minimum probability of error, or minimum variance estimates of d (see Section A.2). In many cases, it is more convenient to work with y, but this is not necessary. In fact, any invertible linear transformation Ar is also a suﬃcient statistic. We shall later see other transformations that are useful for coded systems. 2.5.2 The Correlation Matrix The matrix R = S∗ S is called the correlation matrix, and appears frequently in the design and analysis of multiple-user detectors. We shall therefore consider this matrix in a little more detail. Asynchronous Case In the asynchronous case, column l of S is the spreading sequence for user k = ((l − 1) mod K) + 1 at symbol period i = l/K (reproducing (2.8) for reference), sk [i] = ( 0, . . . , 0 , sk [L(i − 1) + Tk + 1], sk [L(i − 1) + Tk + 2], . . . , sk [Li + Tk ], 0, . . . )t . | {z } L(i−1)+Tk

(2.16)

The correlation matrix R of an asynchronous system has elements

26

2 Linear Multiple-Access

Rlm = s∗k [i]sk [i ], where l = K(i − 1) + k m = K(i − 1) + k . If symbol i for user k does not overlap in time with symbol i for user k then the resulting Rlm = 0. Thus the correlation matrix is band-diagonal with bandwidth 2K + 1, since each symbol can only overlap with at most two symbols from any one other user. Example 2.4. Figure 2.7 shows the structure of R for a K = 3 system and i = 1, 2, 3. We have without loss of generality assumed τ1 ≤ τ2 ≤ τ3 . Hence R is a 9 × 9 band-diagonal matrix with bandwidth 5. The columns of the matrix are labeled with values for k and i. The rows are labeled with values for k and i . The entries correspond to the cross-correlation between user k symbol i and user k , symbol i . Note that the matrix consists of a block-diagonal part, together with lower and upper triangular matrices to ﬁll in the band-diagonal structure. The square matrices on the diagonal are the cross-correlations between all the users for a given symbol interval. The upper-triangular matrices contain the inter-symbol interference terms from users at the previous symbol interval. The lower-triangular matrices contain the terms for the ISI from users at the next symbol interval. i k i'

1 1

2

2 3

1

2

3 3

1

2

3

k' 1

1

2 3 1

2

2 3 1

3

2 3

Fig. 2.7. Structure of the cross-correlation matrix.

2.5 Principles of Detection

27

Symbol Synchronous Case For a symbol-synchronous model, there is no inter-symbol interference and the correlation matrix R is block-diagonal R = diag (R[1], R[2], . . . , ) where R[i] is a K × K symmetric matrix and has elements Rkk [i] = s∗k [i]sk [i]. With reference to Figure 2.7, if the system is symbol synchronous, the upperand lower-triangular components of R are zero, and the remaining square matrices on the diagonal are exactly R[1], R[2], etc. 2.5.3 Single-User Matched Filter Detector The correlation receiver, also known as the single-user matched ﬁlter receiver holds a special place in the history of multiple-user detection. It is an essentially single-user technique that forms its decision dˆk for user k based only on the matched ﬁlter output for user k. Now the matched ﬁlter output for user k consists of the symbol of user k as well as additive multiple-access interference, yk [i] = s∗k [i]r = dk [i] +

(2.17) s∗k [i]sk [i ]dk [i ]

+ z˜k [i]

(2.18)

k =k,i =i

= dk [i] +

Rlm dk [i ]

k =k,i =i

Multiple-access Interference

+

z˜k [i]

.

(2.19)

Thermal Noise

The last line above shows how the entries of the correlation matrix aﬀect the amount of multiple-access interference (where as before, l = Ki + k and m = Ki + k ). The correlator receiver is motivated by a Gaussian assumption regarding the distribution of the multiple-access interference. The folklore argument in favor of the correlator receiver goes as follows. “The matched ﬁlter output is optimal (in terms of minimizing each user’s error probability) when the multiple-access interference is uncorrelated Gaussian noise. If there are many users, the central limit theorem ensures that this interference is in fact asymptotically Gaussian distributed.” Invoking the Gaussian assumption on the multiple-access interference, the correlator receiver applies standard detection methods individually to the output of each matched ﬁlter. In the case of binary antipodal modulation, Dk = {−1, +1}, each user’s symbol estimate is obtained by hard-limiting the matched ﬁlter output as follows.

28

2 Linear Multiple-Access

dˆk [i] = sgn(yk [i]) . Figure 2.8 shows the structure of the correlator receiver for the symbolsynchronous channel and antipodal modulation. In the case of other modulation alphabets, the threshold device is replaced by the appropriate decision device (based on the Gaussian assumption).

s1 [i]

Reset

P

y1 [i]

P

y2 [i]

P

yK [i]

dˆ1 [i]

s2 [i] dˆ2 [i]

r[i]

sK [i] dˆK [i]

Fig. 2.8. Symbol synchronous single-user correlation detection for antipodal modulation.

Largely due to the pioneering work of [121, 142, 147], the Gaussian assumption argument has been refuted, and since then, a proliferation of multiple-user detection strategies, exploiting the inherent structure of the multiple-access interference has occurred. The sub-optimality of the single-user correlator is basically due to the fact that although the vector y is a suﬃcient statistic for the detection of d, the single element yk is not (in general) a suﬃcient statistic for the detection of dk . By ignoring the other matched ﬁlter outputs, information essential for optimal detection is lost. In Chapter 3 we shall see that not only is the correlator sub-optimal from an uncoded bit-error probability point of view, but if each user performs single-user decoding based on the output of the correlator, a penalty is paid in terms of spectral eﬃciency. Nevertheless, the simplicity of implementation of the correlator makes it an attractive choice for system designers, and it is the method of choice for most contemporary existing systems.

2.5 Principles of Detection

29

2.5.4 Optimal Detection We shall now develop the optimal detector for d, given the observation r. Whenever we talk about optimality, we need to clearly specify what function is being optimized. Of particular interest in this detection problem is the probability of error, and there are two main criteria of interest, corresponding respectively to joint or individual optimality. The jointly optimal detector minimizes the probability of a sequence error, ˆ = d[i] , Pr d[i]

(2.20)

whereas the individually optimal detector minimizes the individual error probabilities, Pr dˆk [i] = dk [i] . Let us now develop the jointly optimal detector [121, 142] (the individually optimal detector will be developed in the next section). According to the discussion of minimum error probability detection in Section A.5.1, the jointly optimal detector outputs the data vector with maximum a-posteriori probability. Using Bayes rule, the jointly optimal detector1 is given by ˆ MAP = arg max Pr (d) f (r | d) , d d∈D K

(2.21)

where Pr(d) is the prior probability distribution on the transmitted data and (in the case of complex noise), 1 −L 2 r − SAd2 f (r | d) = (2πN0 ) exp − N0 is the conditional density of the channel output. Alternatively, this detector could have been deﬁned as a function of the matched ﬁlter output, since y is a suﬃcient statistic. The development of this detector, as well as its extension to the asynchronous channel will be given in Chapter 4. Note that the estimate of the entire vector d is based on the entire vector r, or equivalently the output of every matched ﬁlter, [142], as depicted in Figure 2.9. In the case that Pr(d) is the uniform distribution, as might be the case for uncoded data, the joint MAP detector becomes the joint maximum likelihood detector. For Gaussian noise, the joint ML detector minimizes Euclidean distance, ˆ ML = arg min r − SAd2 . (2.22) d 2 d∈D K

The general problem of MAP or ML estimation is NP-hard and brute force evaluation is exponentially complex in K, [149]. The complexity is due 1

Also assuming identical modulation alphabets Dk = D for each user.

30

2 Linear Multiple-Access √ Baseband Modulator

Baseband Modulator

P1 s1 [i] y1 [i]

d1 [i] √

P2 s2 [i]

d2 [i]

y2 [i] r[i]

Matched Filter Front End

z[i] √ Baseband Modulator

dˆ1 [i]

PK sK [i]

dK [i]

Noise

dˆ2 [i] Optimal Detector

S∗

yK [i]

dˆK [i]

Fig. 2.9. Optimal joint detection.

to the discrete nature of the support which has |D|K elements. Brute force evaluation involves calculation of f (y|d) for each element of the support. The ML estimator may be implemented using an exponentially complex trellis search. This prohibitive level of complexity motivates many sub-optimal reduced-complexity detection methods, which is the subject of most of the remainder of this book. In particular Chapter 5 describes reduced complexity methods, mostly based on reduced tree searches for near-optimal detection, which approximate the action of either the jointly optimal, or individually optimal detectors. 2.5.5 Individually Optimal Detection The individually optimal detector minimizes the symbol error probability, Pr dˆk [i] = dk [i] , rather than the sequence error probability (2.20). This is accomplished by outputting the symbol which maximizes the marginal a-posteriori probability, Pr(dk [i] | y). For the symbol synchronous system, this corresponds to

Pr(d)f (y[i] | d). (2.23) dˆk [i] = arg max d∈D

d:dk =d

There are DK−1 terms in the above summation. Brute-force evaluation of the individually optimal decision is exponentially complex in the number of users. Note also that the individually optimal decision is still a function of the entire channel output.

2.6 Access Strategies

31

2.6 Access Strategies There are many diﬀerent existing multiple-access strategies. Each strategy can be described by a speciﬁc choice of modulation waveforms, usually intended to have properties particularly suitable for the application of the single-user matched ﬁlter receiver. In this section we describe how to formulate several well-known multiple-access schemes within the generic linear multiple-access framework (2.2). We will focus on symbol synchronous discrete-time versions of each channel, and hence the various access strategies will be parameterized by their choices of the modulation matrix S. 2.6.1 Time and Frequency Division Multiple-Access Perhaps the most obvious way to share a given bandwidth is to use timedivision or frequency division multiple-access. The basic idea is to allocate non-overlapping subsets of the entire time/bandwidth resource to each user. Time division multiple-access (TDMA) allows each user to transmit using the entire available bandwidth, restricted however to non-overlapping time intervals. Assuming, without loss of generality that each user transmits one complex baseband symbol per allocated time interval, the resulting modulation matrix for the K-user symbol synchronous TDMA channel is STDMA = IK . This is the most obvious example of an orthogonal modulation matrix, i.e. RTDMA = IK . With reference to (2.19), this means that there is no multiple-access interference. In this case the single-user matched ﬁlter receiver is indeed optimal. Frequency division multiple-access (FDMA) allows each user to transmit continuously in non-overlapping frequency bands. Assuming again without loss of generality that each user transmits a single symbol per allocated frequency band, the resulting modulation matrix for the symbol synchronous channel has elements √ 1 (SFDMA )jk = √ exp 2π −1 jk/K . K This is in fact the Fourier transform matrix, F, and once again RFDMA = IK , resulting in optimality of the correlation receiver. Time and frequency-division multiple-access are duals of each other via the Fourier transform, indeed (somewhat trivially)

32

2 Linear Multiple-Access

SFDMA = FSTDMA STDMA = F ∗ SFDMA . The TDMA and FDMA methods just described are two examples of orthogonal multiple-access, in which the modulation matrix is orthogonal, S∗ S = I. Dividing time or frequency between the users makes a certain amount of “intuitive” sense, especially for engineers who are familiar with time-frequency representations of signals. More generally however, what is going on is that there are 2W T signaling dimensions to be shared among K users and there are many more ways to do this. 2.6.2 Direct-Sequence Code Division Multiple Access Direct-sequence code-division multiple-access (DS-CDMA) uses modulating waveforms sk (t) that are built from component waveforms ϕ(t) called chip waveforms. These chip waveforms are of short duration compared to the length of the modulating waveforms themselves. We now specialize the linear multiple-access model to the case of chip-synchronous direct-sequence modulation. Let ϕ(t) be a chip waveform satisfying ϕ(t)ϕ∗ (t − jTc ) = 0 0 = j ∈ Z (2.24) |ϕ(t)|2 dt = 1, (2.25) i.e. the chip waveform is unit energy and is orthogonal to any translation of itself by integer multiples of Tc , the chip period. We assume an integer number of chips L per symbol period, T = LTc . For theoretical purposes, we may consider rectangular chip waveforms, with support [0, Tc ), which however results in an inﬁnite bandwidth. In practice, band-limited pulses, such as pulses with a raised cosine spectrum may be used. The modulating waveforms for each user is composed out of copies of the chip waveform, nL

sk [j] ϕ(t − jTc ), (2.26) sk (t) = j=0

where the integer j denotes the chip index and the chip-rate sequences sk [j] are the real or complex chip amplitudes. Note that we consider the case in which each user has the same chip waveform, and therefore these waveforms are not indexed by the user number k. The resulting modulating waveforms are made diﬀerent through the multiplication of the chip waveforms by the chip amplitudes, which can be diﬀerent from user to user. Code-division multiple-access was motivated by spread-spectrum communications and typically, each user’s modulation waveform sk (t) occupies a

2.6 Access Strategies

33

bandwidth considerably larger than that required by Shannon theory for transmission of the data alone, [83]. This is the case when Tc T , or equivalently, L 1. These sequences sk [j] are sometimes referred to as spreading sequences, signature sequences or spreading codes (we will however reserve the word code to mean forward error control codes). The elements of the spreading √ sequence sk [j] are usually chosen from a ﬁnite alphabet, e.g. {−1, +1}/ L. The concept of spectrum spreading can be viewed more abstractly as the scenario when each user occupies only a small fraction of the total available signaling dimensions. In the context of multiple-access communications however, the entire signal space may be occupied, even though each user spans only a small subspace. These concepts are better understood from an information theoretic point of view, and in Chapter 3 we will discuss this issue in greater depth. Multiplication of the symbol-rate data dk by the spreading sequence sk is called direct-sequence spreading and the resulting form of multiple-access is known as direct-sequence code-division multiple-access. Example 2.5. Figure 2.10 shows an example modulating waveform built from chips, corresponding to a Nyquist chip waveform with roll-oﬀ 0.3 and the chip amplitude sequence sk [1] , . . . , sk [10] = 1, 1, 1, 1, −1, −1, 1, 1, −1, 1.

Fig. 2.10. Modulating waveform built from chip waveforms.

With the above deﬁnitions, each user’s undelayed noise-free contribution is given by nL

sk [j] ϕ(t − jTc ), xk (t) = dk [i] Pk j=0

where i = t/T . Under the convenient (albeit unrealistic) assumption of chip synchronism (but not symbol synchronism) across the users, we assume that the integers τk measure the delay in whole number of chip periods. Although this assumption would rarely hold true in practice, it is a model commonly used in the

34

2 Linear Multiple-Access

literature, and suﬃces for the purposes of the development of the material in this book. The output of the chip-synchronous direct-sequence modulated multipleaccess channel is therefore given by r(t) =

K

xk (t − τk Tc ) + z(t)

k=1

=

K

dk

k=1

t − τk Tc T

nL Pk sk [j] ϕ(t − (j + τk )Tc ) + z(t). j=1

Previously, we obtained a discrete-time model (2.11) from the continuous time model (2.2) using the sampling functions (2.3) as a complete orthonormal basis. Of course, any orthonormal basis will do, and in the case of chip synchronous DS-CDMA, it is convenient to use the set of delayed chip functions, φj (t) = ϕ (t − jTc ) , which according to (2.24) and (2.25) are orthonormal. Furthermore, since we have by design constructed the modulation waveforms as linear combinations (2.26) of φj (t), and the users are chip synchronous, they are a complete orthonormal basis. We can therefore obtain a discrete-time model for the chip synchronous channel by applying a chip matched ﬁlter (assuming that the receiver knows the chip boundaries and therefore the optimal sampling instance). The resulting sequence r[j] is a suﬃcient statistic for d. This set-up is shown in Figure 2.11. √ Baseband Modulator

Baseband Modulator

P1 s1 (t)

d1 [i] √

τ1

x1 (t)

P2 s2 (t)

d2 [i]

τ2

Chip-rate sampling

x2 (t) r(t)

ϕ∗ (−t)

z(t)

Chip Matched Filter

√ PK sK (t) Baseband Modulator

dK [i]

Noise

τK

xK (t)

Fig. 2.11. Chip match-ﬁltered model.

r[i] Receiver

2.6 Access Strategies

35

From (2.26) we can see that the modulation waveform samples resulting from the chip matched ﬁlter are precisely the chip amplitudes sk [j]. Application of the chip match ﬁlter and a sampling at a frequency of 1/Tc samples per second results in the following discrete-time chip synchronous model. r[j] =

K

k=1

dk

j − τk L

Pk sk [j − τk ] + z[j],

(2.27)

where z[j] is a sampled Gaussian noise sequence. Note that this model only applies in the case of no inter-chip interference, which in addition to chip synchronism, requires that the chip waveforms ϕ(t) and ϕ(t + Tc ) oﬀset by and integer number of chip periods are orthogonal. This is true by our assumption (2.24). Note that this discrete-time CDMA channel model (2.27) is identical in form to the generic discrete-time asynchronous model (2.7) presented earlier. The only diﬀerence is in the deﬁnition of the modulation waveform samples. This discrete time model can be re-written in the familiar matrix-vector form (2.11) r = SAd + z, where the deﬁnitions of A, d and z are identical to those given in Section 2.3. The modulation matrix has exactly the same structure as (2.8), except that the entries are the chip amplitudes rather than the sampled waveforms (this is a somewhat pedantic diﬀerence, since here we are in fact using the chip matched ﬁlter to achieve the same goal as sampling). If the delays τk are known to the transmitter, and do not change over time, the modulation matrix S can be chosen (via selection of the sk [j]) to have desired properties. For example, it may be possible to choose an orthogonal S such that R = I, resulting in zero multiple-access interference and optimality of the single-user matched ﬁlter. We have already encountered two possible orthogonal matrices, I and F, corresponding to time- and frequency-division multiple-access. There is nothing particularly special about these choices, and it is clear that any orthogonal (or unitary) matrix can be used with the same result. Rather than assigning speciﬁc time or frequency dimensions to individual users, an arbitrary orthogonal modulation matrix assigns orthogonal signaling dimensions, from a total of 2W T dimensions. All of these orthogonal multiple-access strategies are equivalent via change of basis. One problem with orthogonal modulation however is maintaining synchronism. A modulation matrix S designed for one set of user delays may no longer have the desired properties if the delays change. In particular, orthogonality may be hard to maintain over a wide range of possible delays. There are also many cases in which it is not reasonable to expect that the transmitter either knows the user delays, or that it has the option to adapt the modulation sequences (it may be desired that the modulation sequences are ﬁxed for all time).

36

2 Linear Multiple-Access

We shall distinguish between two broad classes of spreading sequences: periodic sequences, where the same sequence is used to modulate each data symbol for a given user (but diﬀerent sequences for each user); and random sequences, in which the spreading sequence corresponding to each symbol is randomly selected. Deﬁnition 2.1 (Periodic Spreading). The spreading sequence sk [j] is periodic if the same sequence is used for each symbol period sk [j] = sk [j + L] Periodic spreading may be used when symbol synchronism can be enforced and the sequences may be designed to have desired properties, such as low cross-correlation. Deﬁnition 2.2 (Random Spreading). The spreading sequence is random if each element sk [j] is selected i.i.d. from a ﬁxed distribution with zero mean, unit variance, and ﬁnite higher moments.2 Note that this deﬁnition includes uniform selection of chips from an appropriately normalized binary (or any other ﬁnite cardinality) alphabet. It also includes such choices as Gaussian distributed chips. Random spreading may be approximated by using long pseudo-random sequences, such as m-sequences. The use of the term “random spreading” does not mean that the sequences are unknown to the receiver. In fact we usually assume that the receiver also knows the sequences. This is possible when the sequences are only really pseudo-random (i.e. the receiver can generate the same pseudo-random sequence and the transmitter). Random spreading is a useful model for systems in which the period of the spreading is much greater than the symbol duration. As we shall also see, random spreading is a useful theoretical tool for system analysis. 2.6.3 Narrow Band Multiple-Access Rather than assigning wide-band signaling waveforms, we can consider a system in which each user transmits using the same modulation waveform s(t). In the symbol synchronous case the signal model is K

t r(t) = d Pk s(t) + z(t), T

(2.28)

k=1

2

This mild restriction on higher moments is a technical requirement for largesystem capacity results.

2.6 Access Strategies

37

and the corresponding discrete-time matrix model is r[i] = S[i]A[i]d[i] + z[i] where the modulation matrix S[i] has identical columns. In that case, the correlation matrix R is the all-ones matrix, ∗

y[i] = 1A[i]d[i] + S[i] z[i] and the output of each user’s matched ﬁlter is identical and is given by y[i] =

K

Pk dk [i] + z˜[i].

(2.29)

k=1

Note that we could have obtained (2.29) directly from (2.28) via matched ﬁltering for s(t). 2.6.4 Multiple Antenna Channels In certain scenarios, such as in the presence of multipath propagation, it may be advantageous for the receiver and each transmitter to use multiple antennas. Let us develop an idealized channel model for a K user system in which each user has M transmit antennas, and the receiver has N antennas. Extension to diﬀerent number of antennas for each transmitter is straightforward. Figure 2.12 shows a two-user example in which each user has two transmit antennas, and there are three receive antennas. The ﬁgure is simpliﬁed to concentrate on the multiple antenna aspect of the system. x11 (t) h11 1 User 1

h31 1 12 x21 (t) h1 h32 1

r1 (t) h21 1

h22 1 r2 (t) Receiver

h11 2 x12 (t) User 2

h31 2 h12 2

x22 (t)

h21 2 h22 2

r3 (t)

h32 2 Fig. 2.12. Multiple transmit and receive antennas.

38

2 Linear Multiple-Access

To allow for the most general transmission strategies, let each user employ a diﬀerent modulation waveform for each transmit antenna and transmit different data symbols over each antenna. To this end, let sμk (t) and dμk [i] be the respective modulation waveforms and data symbols for user k = 1, 2, . . . , K, and antenna μ = 1, 2, . . . , M . Each user may also transmit using diﬀerent symbol energies from each transmit antenna. Let Pkμ denote the symbol energy for antenna μ of user k. We can therefore write the undelayed transmitted signals for antenna μ of user k as t μ μ xk (t) = dk Pkμ sμk (t). T Let hνμ k be the channel gain between transmit antenna μ of user k and receive antenna ν = 1, 2, . . . , N . Then the signal received at antenna ν is rν (t) =

M K

μ ν hνμ k (t)xk (t − τk ) + z (t)

k=1 μ=1

=

M K

k=1 μ=1

μ hνμ k (t)dk

t − τk T

Pkμ sμk (t − τk ) + z ν (t).

The received signal at each antenna shares the same formulation as a KM user system in which each user has a single transmit antenna (consider the mapping (k, μ) → K(μ − 1) + k). Sampling the output of each receive antenna leads to a discrete-time model which may be represented in matrix form. Under the assumption of symbol synchronism we can write V[i] = S[i]AD[i]H[i] + Z[i]

(2.30)

where the various matrices in (2.30) are deﬁned as follows. The L × N matrix V[i] contains the received samples for symbol period i. Element (V)jν of this matrix is sample j from antenna ν. Thus each column is the output of one antenna. The L × KM matrix S[i] is deﬁned in a similar way to (2.12). Its columns are the sampled modulation waveforms for each transmit antenna of each user. More speciﬁcally, column l is the sampled waveform for transmit antenna μ of user k, according to k = l/M μ = ((l − 1) mod M ) + 1,

(2.31) (2.32)

i.e. we get each transmit antenna of user 1 followed by each antenna of user 2 and so on. The KM × KM matrices A[i] and D[i] are both diagonal. The diagonal elements of A are the transmit amplitudes for each transmit antenna of each

2.6 Access Strategies

39

user. The diagonal elements of D are the transmitted symbols for each antenna of each user. The ordering is the same as that used for S, i.e. subject to (2.31) and (2.32) (D[i])ll = dμk [i] (A[i])ll = Pkμ [i]. Subject to the assumption that the channel gains hμν k are constant for each symbol interval, but that they may change from symbol interval to symbol interval, the KM × N matrix H[i] contains the channel gains, ordered according to (2.31) and (2.32), (H[i])lν = hμν k [i]. With the introduction of appropriate statistical models, the channel gains may model eﬀects such as fast or slow ﬂat fading, or they may be used to model phased arrays. Note that we may recover the single antenna model of (2.12) via M = N = 1 and H[i] = 1, a K × 1 all-ones vector. With these deﬁnitions, (2.30) is identical to (2.12) since d[i] = D[i]1. 2.6.5 Cellular Networks Cellular networks spatially re-use signaling dimensions (time and frequency) in order to increase the overall system capacity. The idea is that given enough spatial separation, the signal attenuation due to radio propagation will be suﬃcient to limit the eﬀects of multiple-access interference from other cells operating in the same signal space. An idealized model of a narrow band cellular multiple-access channel was developed in [170]. In this model, a user contributes to the received signal of a base station only if it is in that base station’s cell, or if it is in an immediately adjacent cell. This is shown schematically in Figure 2.13. This ﬁgure shows ten hexagonal cells each with a base station represented by a circle. Two mobile terminals are shown. Each transmits an intended signal to its own base station (solid lines) and interference to adjacent base stations (dashed lines). Consider the up-link of a cellular network in which there are L cells, each with a single base station. A total of K users populate the network. Assume that each user transmits using identical modulation waveforms and that the system is symbol synchronous. Then (according to the narrow-band model developed in Section 2.6.3), the discrete-time signal model is r[i] = S[i]Ad[i] + z[i]. In this model, rj [i], j = 1, 2, . . . , L is the signal received at base station j. According to Wyner’s idealized model, the matrix S deﬁnes the connection gains between user k and base station j according to

40

2 Linear Multiple-Access

1

2

3

4

5 2

1 6 7

8 9

10

Fig. 2.13. Simpliﬁed cellular system.

sjk

⎧ ⎪ ⎨1 = α ⎪ ⎩ 0

User k is in cell j User k is in a cell adjacent to cell j otherwise.

Thus each user is received in its own cell with symbol energy P and in adjacent cells with symbol energy P α2 . In this simpliﬁed model, the modulation matrix acts as a connection matrix. This is a crude model for perfect power control within each cell, and a ﬁxed path loss between cells. In a more general model, the gains sjk would be random variables and would include the eﬀect of path loss, shadowing and fading. Example 2.6. The cellular arrangement shown in Figure 2.13 is modeled by ⎞ ⎛ α0 ⎜α 0 ⎟ ⎟ ⎜ ⎜α α⎟ ⎟ ⎜ ⎜ 1 α⎟ ⎟ ⎜ ⎜ 0 α⎟ ⎟. ⎜ S=⎜ ⎟ ⎜α 0 ⎟ ⎜α 1 ⎟ ⎟ ⎜ ⎜α α ⎟ ⎟ ⎜ ⎝ 0 α⎠ 0α

2.6 Access Strategies

41

2.6.6 Satellite Spot-Beams Channels Satellite spot-beam channels, [89, 90] share a similar formulation to cellular networks. With reference to Figure 2.14, terrestrial user terminals communicate with a ground station via a satellite which employs a multi-beam antenna (usually implemented using a phased array). The satellite spatially re-uses time and frequency resources by using a spot beam antenna. Each beam of the spot beam antenna illuminates a geographical region and the antenna is designed to provide isolation between each beam. It is however diﬃcult to achieve perfect isolation between beams and multiple-access interference results. In the cellular model, the average gain between each user and base station pair is determined largely by path loss considerations. In the spotbeam channel, the gain between each user and beam is determined by the antenna roll-oﬀ. Under the assumption of symbol synchronism and identical modulating waveforms for each user, the satellite spot-beam channel is the same as the cellular network described in Section 2.6.5. The modulation matrix now contains the gain from each mobile each mobile earth terminal to each antenna beam.

Ground Station

Satellite

Fig. 2.14. Satellite spot beam up-link.

42

2 Linear Multiple-Access

Example 2.7. The seven-beam example shown in Figure 2.14 is modeled by ⎞ ⎛ 1αααααα ⎜α 1 α 0 0 0 α⎟ ⎟ ⎜ ⎜α α 1 α 0 0 0 ⎟ ⎟ ⎜ ⎟ S=⎜ ⎜α 0 α 1 α 0 0 ⎟ ⎜α 0 0 α 1 α 0 ⎟ ⎟ ⎜ ⎝α 0 0 0 α 1 α ⎠ αα0 0 0α1

2.7 Sequence Design Most modulation sequence design is motivated by the use of sub-optimal detection strategies, in particular the correlation receiver described in Section 2.5.3. Conditioned on this choice of sub-optimal detection strategy, it is natural to try to optimize the performance of the receiver by carefully choosing the modulation sequences. To this end, one could try to choose sequences that minimize the average error probability or that minimize the total cross correlation. 2.7.1 Orthogonal and Unitary Sequences If L ≥ K, it is always possible to ﬁnd modulation matrices S that result in zero multiple-access interference. In the case of real matrices, this means R = St S = I and S is called orthogonal. In the case of complex matrices, R = S∗ S = I, and S is called unitary. Obviously orthogonal matrices are also unitary if we consider the real elements to be complex numbers with zero imaginary part. Unitary matrices have the property that each column is orthogonal to every other column, and these columns are the modulating vectors for each user. Starting from an given vector, it is possible to ﬁnd K − 1 other vectors while maintaining orthogonality, via a Gram-Schmidt process. In the case of an unitary S, the matched-ﬁlter output is y = S∗ r = S∗ Sd + S∗ z ˜. =d+z It is a property of the Gaussian density that it is invariant to unitary transformations (i.e. it is isotropic). This means that unlike most transformations, ˜ has the same distribution as z. Therefore each user k may be detected based z only on its own matched ﬁlter output. Some examples of unitary matrices that we have already encountered include I, the identity matrix and F, the Fourier transform matrix. Use of these matrices result respectively in time and frequency-division multiple-access.

2.7 Sequence Design

43

Although it is always possible to construct unitary matrices for L ≥ K, it is not always possible to ensure that the elements of S are members of a given modulation alphabet. It may be required for instance that the elements are binary, sk [j] ∈ {−1, +1}. 2.7.2 Hadamard Sequences Let us now consider the problem of designing modulation sequences to optimize the performance of the correlation receiver. The total variance σk2 of the multiple-access interference as seen by user k is the k-th row (or column) sum of R, minus the diagonal term σk2 =

K

2 Rjk − 1,

j=1

and a reasonable goal for sequence design is to ﬁnd binary modulation sequences S that minimize the total squared cross-correlation 2 = σtot

K

σk2 ,

i=1 2 /K. subject to a fairness condition σk2 = σtot 2 Now if K ≤ L, it is obvious that we can achieve σtot = 0 by using orthogonal modulation sequences, since in that case R = I. The correlation receiver is in fact optimal for orthogonal sequences. Binary matrices S with the property S∗ S = I are called Hadamard Matrices, and can only exist for L = 1, 2 or a multiple of 4. (It is not know whether they exist for every multiple of 4). For arbitrary given K and L (i.e. K not necessarily smaller than L), how close to orthogonal can we get? The following theorem gives a bound on the total squared cross-correlation [159].

2

Theorem 2.1 (Welch’s Bound). Let sk ∈ CL and sk = 1, k = 1, 2, . . . , K then K

K

i=1 j=1

2 Rij ≥

K2 . L

Equality is achieved if and only if S has orthogonal rows. Equality implies K

K 2 Rij = . L j=1

44

2 Linear Multiple-Access

It has been shown in [85] that sequences that meet the Welch bound with equality are optimal if correlation detection is being used. We also see in Section 3.4.2 that such sequences are also are optimal from an information theoretic point of view. Example 2.8 (Welch bound equality sequences). The following non-orthogonal sequence set achieves Welch’s bound (where for clarity, + denotes a +1 and − denotes −1). ⎞ ⎛ ++++++++ ⎜+ + + + − − − −⎟ ⎟ S=⎜ ⎝+ + − − + + − −⎠ +−+−+−+− The requirement for equality in Welch’s bound is the orthogonality of the rows of S. Recall that the modulation sequences themselves form the columns of S. In the case K ≤ L, we can have orthogonality both of the rows, and the columns. If however K > L, the sequences themselves cannot be orthogonal, yet the correlator performance is optimized by having orthogonal rows.

3 Multiuser Information Theory

3.1 Introduction In this chapter we consider some of the Shannon theoretic results and coding strategies for the multiple-access channel. The underlying assumption is that signals are encoded, and that no restrictions will be placed on the capability of the receiver processing. Information theory provides fundamental limits on data compression, reliable transmission and encryption for single user or point-to-point channels. In the context of multiple-user communications, there are many interesting and relevant questions that we could ask: • • • • •

What are the fundamental limits for multiple-user communications? What guidance can information theory give for system design? What are the impact of system features such as feedback, or synchronism? What modulation and/or coding techniques should we use? How should we deal with multiple-access interference from an engineering point of view? • What is the cast of using sub-optimal reception techniques? We will approach some of these questions from an information theoretic perspective, concentrating on the multiple-access channel. We begin in Section 3.2 by formalizing our deﬁnition of a multiple-access channel. From this basis, we introduce in Section 3.2.2 the capacity region, which is the multiple-user analog of the channel capacity. In Section 3.3 we consider some simple binary-input channels, namely the binary adder channel and the binary multiplier channel. These serve to illustrate some of the main ideas regarding the computation of the capacity region for discrete channels. Multiple-access channels with additive Gaussian noise are developed in Section 3.4. Of particular interest is the direct-sequence code-division multipleaccess channel, which is the subject of Section 3.4.2. We discuss the relationship between the selection of signature sequences and the channel capacity,

46

3 Multiuser Information Theory

and give an overview of large systems analysis, applicable to random signature sequences. This section also serves as an introduction to Chapters 4 – 6, which discuss the design of multiple-user receivers for DS-CDMA. In Sections 3.5.1 and 3.5.2 , we discuss the design of uniquely decodeable block and trellis codes, which are error-control codes speciﬁcally designed for the multiple-access channel. Another coding strategy for the multiple-access channel is superposition coding, discussed in Section 3.6. This approach allows the multiple-access channel to be viewed as a series of single-user channels, with no theoretical loss in channel capacity. Various modiﬁcations of the channel assumptions have diﬀerent eﬀects upon the size and shape of the capacity region. In Section 3.7, we discuss the eﬀect of feedback, and in Section 3.8 we discuss asynchronous transmission.

3.2 The Multiple-Access Channel 3.2.1 Probabilistic Channel Model In Chapter 2, we developed signal-based mathematical models for the linear multiple-access channel. In order to obtain an information theoretic understanding of the multiple-access channel, we must ﬁrst deﬁne an appropriate probabilistic model (the underlying probabilistic models were implicit in Chapter 2). Let us begin by introducing some concise notation to describe many user systems. For a K user system, let K = {1, 2, . . . , K} be the set of user index numbers. For an arbitrary subset of users, S ⊆ K, the subscript S denotes objects indexed by the elements of S. For example, XS = {Xk , k ∈ S} are those users indexed by the elements of S. Likewise, letting Rk be the information rate of user k, RS = {Rk , k ∈ S} . For objects with a suitably deﬁned addition function (rates, powers), let the functional notation

R (S) = Rk k∈S

be the sum over those users indexed by S. We are now ready to deﬁne the discrete memoryless multiple-access channel.

3.2 The Multiple-Access Channel

47

Deﬁnition 3.1 (Discrete Memoryless Multiple-Access Channel). A discrete memoryless multiple-access channel (DMMAC) consists of K ﬁnite input alphabets, X1 , X2 , . . . , XK (denoted XK in the shorthand notation introduced above), a set of transition probabilities, p(y | xK ) = p(y | x1 , x2 , . . . , xK ) and an output alphabet Y. We shall refer to such a channel by the triple (XK ; p(y | xK ) ; Y), which emphasizes the entities that completely deﬁne the channel. Figure 3.1 shows a schematic representation of a two-user multiple-access channel. Transmission over the DMMAC takes place as follows. Time is discrete, requiring symbol transmissions to be aligned to common boundaries. At each symbol interval, every user k = 1, 2, . . . , K transmits a symbol xk , drawn from Xk according to a source probability distribution pk (xk ). The received symbol, y ∈ Y is observed according to the transition probabilities p(y | xK ).

x1 ∈ X1 p(y | x1 , x2 )

y∈Y

x2 ∈ X2 Fig. 3.1. Two-user multiple-access channel.

Although we have only formally deﬁned a discrete memoryless multiple access channel, it is straightforward to extend this deﬁnition to some other types of channel. For example, we can model the discrete-input continuousoutput MAC by allowing the received symbols to be real-valued, Y ⊆ R. Accordingly, the conditional transition probabilities must now be density, rather than mass functions. Similarly, continuous-input continuous-output channels (usually subject to input power constraints), allow the sources to take values from the reals, XK ⊆ RK according to some density function pk (xk ), yielding now a joint probability density, p(y | xK ). Although we have deﬁned the individual user’s (marginal) source probabilities, we have not mentioned any statistical dependence between users. The form of the joint source distribution p(xK ) depends on the degree of cooperation allowed between the users. If the users cannot cooperate, p(xK ) is restricted to be a product distribution,

48

3 Multiuser Information Theory

p(xK ) =

K #

pk (xk ) .

k=1

Such absence of cooperation between users can be caused for example by physical separation of the users, who have no direct communication links with each other. This is the case in most mobile radio networks. Alternatively, if full cooperation between users is allowed (which implies the existence of a communication link between the users), there is no restriction on the joint distribution p(xK ). Such full cooperation of sources allows the complete channel resource to be used as if by a single “super-source” with alphabet X1 × X2 × · · · × XK , and can therefore be used as a benchmark for systems with independent users. Note that our deﬁnition of a DMMAC did not include the source distributions. The system designer is typically free to choose the most advantageous set of source distributions (for example to maximize the rate of transmission). A discrete memoryless multiple-access channel may be schematically represented in a fashion similar to single-user channels. Example 3.1. Figure 3.2 shows the transition probability diagram for an example two-user channel with X1 = X2 = {0, 1}. Each user transmits independently from uniformly distributed binary alphabets. The arrows on the graph show the probability of receiving a particular y ∈ Y = {0, 1, 2}, corresponding to a matrix of transition probabilities (the outputs index the columns) ⎤ ⎡ 0.9 0.1 0 ⎢0.05 0.8 0.15⎥ ⎥ ⎢ ⎣ 0 0.3 0.7 ⎦ . 0 0 1 Note that when the users transmit independently, the resulting multipleaccess channel model is very similar to that for a single-user discrete multipleaccess channel with memory, e.g. an inter-symbol interference (ISI) channel. Instead of the source labels referring to the current, and previous transmitted bit (in an ISI channel with memory one), they now refer to user 1 and user 2. 3.2.2 The Capacity Region The capacity region is the multiple-user equivalent of the channel capacity for a single-user system. In a single-user system, there is a single number, C, which is the channel capacity.1 The channel capacity is the rate of transmission below which there exist encoders and decoders that allow arbitrarily low probability of decoding error. In a multiple-user system with K users, there are K possibly diﬀerent rates, one for each user and the capacity region is the set of allowable rate 1

Provided the channel has a capacity.

3.2 The Multiple-Access Channel p(x1 , x2 ) (x1 , x2 ) 1 4

(0, 0)

Y 0.9

49

p(Y )

0 0.2375

0.1 1 4

(0, 1)

0.05 0.8 0.15

1 4

(1, 0)

1

0.3

0.3 0.7

1 4

(1, 1)

1.0

2 0.4625

Fig. 3.2. Example of a discrete memoryless multiple-access channel.

combinations, which we represent geometrically in K-dimensional Euclidean space as rate vectors, R = (R1 , R2 , . . . , RK ). These rate combinations may be used such that arbitrarily low error probabilities can be theoretically achieved. First, we must deﬁne the concept of a channel code, and the probability of error, before any capacity theorem can make sense. With reference to Figure 3.3, a code for a multiple-access channel consists of K encoding functions, one for each source. Since the sources are independent, we can describe the multiple-access code in terms of the individual codes for each user (rather than a single joint encoder). Let Mk = {1, 2, . . . , Mk } represent the set of possible messages that user k may wish to transmit. Thus the message for user k is a random variable Uk ∈ Mk , and the encoding function fk for user k maps from this set of Mk messages onto a n dimensional code vector, whose elements are drawn from the source alphabet Xk : fk : Mk → Xkn . The parameter n is called the codeword length. Assuming symbol synchronous transmission of the users, we can represent the entire process as a multipleaccess encoder fK mapping as follows n fK : M1 × M2 × · · · × MK → X1n × X2n × · · · × XK ,

where in this instance, × denotes the Cartesian product. We shall abbreviate this mapping by fK : MK → XKn . For user k, deﬁne 1 Rk = log2 Mk n to be the code rate, in bits per symbol.

50

3 Multiuser Information Theory

The decoding function for the multiple-access code is a mapping g : Y n → MK . from the set of possible received vectors onto the message sets of the individual users. This deﬁnition implies the use of receiver cooperation, i.e. joint ˆk . decoding. The K messages output by the decoder are random variables, U

U1

Encoder

X1n

ˆ1 U

X2n

ˆ2 U

f1 U2

Encoder

f2 Channel

Yn

Decoder

p(y|xK )

UK

Encoder

g

ˆK U

n XK

fK

Fig. 3.3. Coded multiple-access system.

A decoding error is said to have occurred if the message set at the output of the decoder does not agree completely with the input message set. In other words an error occurs if the decoded message for any of the users is in error. The error probability conditioned on a particular set of transmitted messages mK = m1 , m2 , . . . , mK is therefore Pr (Error | UK = mK ) = Pr (g (Y n ) = mK ) Assuming equi-probable transmission of messages, the average error probability for a multiple-access code is given by P¯e =

1 2nR(K)

Pr (g (Y n ) = mK ) .

mK ∈MK

Deﬁnition 3.2 (Achievable Rate Vector). A particular rate vector R = (R1 , . . . , RK ) is achievable if, for any > 0 there exists a multiple-access code with rate vector R such that P¯e ≤ . Otherwise it is said to be not achievable.

3.2 The Multiple-Access Channel

51

Now we have formalized our description of the multiple-access system, we can present the theorem which describes the entire set of achievable rates. We describe the capacity region in terms of the achievable rate regions for ﬁxed source distributions. Theorem 3.1 (Achievable Rate Region). multiple-access channel

Consider the

(XK ; p(y | xK ) ; Y) . For a ﬁxed product distribution on the channel input symbols, # π (xK ) = pk (xk ) k

every rate vector R in the set * + R [π (xK ) , p(y | xK )] R : R (S) ≤ I XS ; Y | XK\S , ∀S ⊆ K (3.1) is achievable in the sense of Deﬁnition 3.2. Polytopes which are deﬁned by a system of constraints such as (3.1) are known as bounded polymatroids [138]. The following example writes out the achievable rate region for the two-user case. Example 3.2 (Two-user achievable rate region). Consider a two-user DMMAC. For a ﬁxed input distribution π (x1 , x2 ) the following region is achievable. ⎫ ⎧ R1 ≤ I (X1 ; Y | X2 )⎬ ⎨ R2 ≤ I (X2 ; Y | X1 ) R [p1 (x1 ) p2 (x2 ) , p(y | xK )] = (R1 , R2 ) : ⎭ ⎩ R1 + R2 ≤ I (X1 , X2 ; Y ) This region is shown in Figure 3.4. Example 3.3 (Three-user achievable rate region). Consider a three-user DMMAC. For a ﬁxed input distribution π (x1 , x2 , x3 ) the following region is achievable. ⎫ ⎧ R1 ≤ I (X1 ; Y | X2 , X3 )⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ R2 ≤ I (X2 ; Y | X1 , X3 )⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ R3 ≤ I (X3 ; Y | X1 , X2 )⎪ ⎨ ⎬ ≤ I (X1 , X2 ; Y | X3 ) R [π (xK ) , p(y | xK )] = (R1 , R2 , R3 ) : R1 + R2 ⎪ ⎪ ⎪ R1 + R 3 ≤ I (X1 , X3 ; Y | X2 )⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ≤ I (X2 , X3 ; Y | X1 )⎪ R2 + R 3 ⎪ ⎪ ⎪ ⎪ ⎭ ⎩ R1 + R2 + R3 ≤ I (X1 , X2 , X3 ; Y ) This region is shown in Figure 3.5.

52

3 Multiuser Information Theory R2 R1

I (X2 ; Y | X1 )

+ R2 = X I( ,X

1

;Y

2

)

I (X2 ; Y )

R1 I (X1 ; Y | X2 )

I (X1 ; Y )

Fig. 3.4. Two-user achievable rate region. R2 I(X2 ;Y |X1 ,X3 )

R1 +R2 =I(X1 ,X2 ;Y |X3 ) R2 +R3 =I(X2 ,X3 ;Y |X1 ) R1 +R2 +R3 =I(X1 ,X2 ,X3 ;Y )

I(X3 ;Y |X1 ,X2 )

I(X1 ;Y |X2 ,X3 ) R1 +R3 =I(X1 ,X3 ;Y |X2 )

R3

R1 Fig. 3.5. Three-user achievable rate region.

3.2 The Multiple-Access Channel

53

In general there are 2K − 1 rate constraints, one for each non-empty subset of users. Note that in some cases, there may be a smaller number of eﬀective constraints, since some inequalities may be implied by others. The system designer can arbitrarily select the source product distribution π (·) and hence the union of achievable regions (over the set of all source product distributions) is achievable. Furthermore, suppose R1 and R2 are both achievable rate vectors. Every point on the line connecting R1 and R2 is also achievable, by using the codebook corresponding to R1 for λn symbols and that corresponding to R2 for the remaining (1 − λ)n, 0 ≤ λ ≤ 1 symbols. This process is known as time-sharing and implies that the convex hull 2 of an achievable region is achievable. A common time reference3 is however required so the users know when to change codebooks. Theorem 3.2 (Multiple-Access Capacity Region). Assuming the availability of a common time reference to all users, the capacity region is given by the closure of the convex hull (denoted co) of the union of all achievable rate regions over the entire family of product distributions on the sources: / C [p(y | xK )] = co R [π (xK ) , p(y | xK )] . (3.2) π(xK )

The capacity region was ﬁrst determined by [2], [129] and [76]. More accessible proofs of the multiple-access channel coding theorem may be found in [51] using the concept of jointly typical sequences, see also [24] and in [50], using random coding arguments. The DMMAC is rather special in that it admits a single-letter description of its capacity region, i.e. can be expressed in terms of quantities such as I (X; Y ), rather than limiting expressions4 such as lim I (X[1], X[2], . . . , X[n]; Y [1], Y [2], . . . , Y [n]) .

n→∞

There are not many other examples of multi-terminal channels in which the general solution is single-letter. Although single-letter achievable regions may be easily constructed for most multi-terminal channels, it appears diﬃcult in general to obtain single-letter converse theorems. 2 3 4

The convex hull (convex cover) of a set of points, A is deﬁned in [32] as the set of points which is the intersection of all the convex sets that contain A. This qualiﬁcation shall be examined in more detail in Section 3.8 It is possible to obtain “single-letter” expressions at the expense of unbounded increase in the source cardinality, but this is cheating.

54

3 Multiuser Information Theory

3.3 Binary-Input Channels In order to illustrate the concepts introduced above, let us now consider some simple examples, concentrating on binary-input channels. In particular, we shall consider the binary adder channel and the binary multiplier channel. 3.3.1 Binary Adder Channel The binary adder channel is a K-user DMMAC in which each user transmits from a binary alphabet Xk = {0, 1}, k = 1, . . . , K and the output is given by the the real addition K

Y = Xk k=1

The channel output will therefore be a member of Y = {0, 1, . . . , K}. What are the allowable rates of transmission for the binary adder channel? A simple upper bound based on the cardinality of Y is R(K) ≤ log(K + 1). We can however say a lot more than this. Two Users First, we shall ﬁnd the capacity region for the two-user case. Let user 1 and user 2 transmit a 0 with probabilities p1 (0) = p and p2 (0) = q respectively. The transition diagram of the channel is shown in Figure 3.6. By Theorem 3.1, the p1 (x1 ) p2 (x2 ) (x1 , x2 ) pq

(0, 0)

p(1 − q)

(0, 1)

(1 − p)q

y

pY (y)

0

pq

1

p + q − 2pq

2

(1 − p)(1 − q)

(1, 0)

(1 − p)(1 − q) (1, 1)

Fig. 3.6. Two-user binary adder channel.

3.3 Binary-Input Channels

55

achievable rate region for a given p and q is given by the rate vectors, (R1 , R2 ), which satisfy R1 ≤ I (X1 ; Y | X2 ) R2 ≤ I (X2 ; Y | X1 ) R1 + R2 ≤ I (X1 , X2 ; Y )

(3.3) (3.4) (3.5)

Consider the ﬁrst inequality (3.3). I (X1 ; Y | X2 ) = H (X1 | X2 ) − H (X1 | X2 , Y ) = H (X1 ) = h (p) where h (p) is the binary entropy function. The ﬁrst step follows from the independence of the two users (no transmitter cooperation), and the fact that if we know both Y and X2 , we can always determine X1 . Similarly, (3.4) is found to be I (X2 ; Y | X1 ) = h (q) The sum constraint (3.5) is determined as follows: I (X1 , X2 ; Y ) = H (Y ) − H (Y | X1 , X2 ) = H (Y ) = −pq log pq − (1 − p)(1 − q) log(1 − p)(1 − q) − (p + q − 2pq) log(p + q − 2pq) since the two users’ inputs uniquely determine the output. It is now necessary to take the closure of the convex hull over the union of all such regions, where 0 ≤ p, q ≤ 1. Fortunately, there is a trick to be employed, but for the moment, let us see what this statement means. Figure 3.7 shows an example. The region bounded by the thick line is due to p = 0.1, q = 0.6. The second region, bounded with a thin line is due to p = 0.5, q = 0.8. The shaded region shows the additional rates found by taking the convex hull over these two regions. In principle, one continues this way for all p, q combinations. It is not hard to see that this processes would be rather tedious, since p and q have uncountable support. Instead, note that for p = q = 0.5, we obtain the maxima h (p) = h (q) = 1.0. Fortunately, it is precisely these distributions which also maximize the third constraint, giving maxp,q H (Y ) = 1.5. Since it is possible to simultaneously maximize all constraints, the resulting region must contain all other regions. The resulting capacity region is therefore R1 < 1 R2 < 1 R1 + R2 < 1.5.

56

3 Multiuser Information Theory R2

1.0

(p, q) = (0.1, 0.6)

0.5

(p, q) = (0.5, 0.8)

R1 0.5 1.0 Fig. 3.7. Convex hull of two achievable rate regions for the two-user binary adder channel.

This region is a pentagon and is shown bounded by the thick line in Figure 3.8.

Single-user Decoding and Time Sharing Two other regions are shown on Figure 3.8, corresponding to diﬀerent transmission and decoding strategies. The triangle deﬁned by R1 + R 2 < 1 and labeled no cooperation is the set of rates achievable with single-user decoding, in which the decoder for each user operates independently and treats the unwanted user as noise. This region is sometimes referred to as the time-sharing region, since it may be achieved by time sharing the channel between the two users. To achieve a rate point R1 = λ, R2 = 1−λ via time sharing, the input distribution p = 1/2, q = 0 is used λ of the time, and the remaining 1 − λ of the time the input distribution p = 0, q = 1/2 is used. Note that such time-sharing implies a degree of cooperation between the users, namely that they have access to a common time reference. It is perhaps less well-known that time sharing is not really necessary to achieve points with R1 + R2 = 1. If each user is decoded independently and treats the other user as noise, the rate constraints are

3.3 Binary-Input Channels

57

R2

1.5

Full cooperation B

1.0

Joint decoding

0.5

A

No cooperation

R1 0.5 1.0 1.5 Fig. 3.8. Capacity region of the two-user binary adder channel.

R1 ≤ H (Y ) − H (Y | X1 ) = H (Y ) − h (q) R2 ≤ H (Y ) − H (Y | X2 ) = H (Y ) − h (p) . Setting q = 1/2 results in 1 1 h (p) ≤ 2 2 1 1 R2 = 1 − h (p) ≥ . 2 2 R1 =

and the portion of the line R1 + R2 = 1 from R = (0, 1) to R = (1/2, 1/2) may be achieved by varying p. Alternatively, setting p = 1/2 results in 1 1 R1 = 1 − h (q) ≥ 2 2 1 1 R2 = h (q) ≤ . 2 2 which yields the remaining portion of the line R1 +R2 = 1 from R = (1/2, 1/2) to R = (1, 0). The full cooperation region is bounded by the sum rate R1 + R2 = log2 3, which is achieved if the users cooperate in their transmissions to ensure a

58

3 Multiuser Information Theory

uniform distribution over the three output symbols. This region is outside the multiple-access channel capacity region, because without cooperating in their transmissions, the required uniform distribution of output symbols cannot be obtained. Cancellation and the Corner Points It is worthwhile discussing the corner points of the joint decoding capacity region, marked A and B on the ﬁgure. Consider the ﬁrst of these points, A which is at (I (X1 ; Y | X2 ) , I (X2 ; Y )) = (1, 0.5). The form of the expression for this point is suggestive of a method for transmission on the channel, namely interference cancellation. Let user 2 transmit at rate R2 = I (X2 ; Y ), which is the mutual information between user 2 and the output, treating user 1 as noise. Recall that p = q = 0.5, hence user 2 sees the single-user channel shown in Figure 3.9. This is the binary erasure channel, with erasure probability 1 2

0Q

1

Q 1Q

-0

2

Q

1 2

1 2

Q s 1 Q 3 - 2

Fig. 3.9. Channel as seen by user two.

δ = 0.5. The capacity of this channel is known to be 1 − δ = 0.5 [24, p. 187], hence user 2 may indeed transmit with arbitrarily low error probability (using a separate single-user decoder) at 0.5 bits/channel use. Assuming that we have decoded user 2 error free, user 1 may transmit at R1 = I (X1 ; Y | X2 ) = 1 bit per channel use, which is the maximal mutual information between user 1 and the output, given complete knowledge of user 2. Thus we can achieve the point A, using only single-user coding techniques, combined with interference cancellation. Point B may be achieved in a similar way, decoding user 1 ﬁrst. Any point on the line connecting A and B may be achieved by time-sharing between these two schemes. K > 2 Users For the general case of K > 2 users, the capacity region is given [18, 19] as the closure of the convex hull of the union of the following regions: ⎫ ⎧ |S| |S| ⎬ ⎨ |S|

2 k , ∀S ⊆ K . log (R1 , R2 , . . . , RK ) : 0 ≤ R (S) ≤ 2 |S| ⎭ ⎩ 2|S| k=1

k

3.4 Gaussian Multiple-Access Channels

59

Often, we are interested in the maximum total transmission rate, or sum capacity. A good approximation to the sum rate constraint, R (K) ≤ Csum for this channel is [19, 165] Csum ≈

πeK 1 log2 bits. 2 2

(3.6)

In fact the following bounds show the asymptotic tightness of this approximation [18] 1 log2 πeK Keven, πK 1 2 log2 ≤ max R (K) ≤ 21 πe(K+1) 2 2 log Kodd. 2 2 2 From (3.6), the total throughput is proportional to log K. As the system grows larger, the sum capacity increases, with individual users transmitting less. It is interesting to note that log2 πeK 1 2 = . k→∞ log2 (K + 1) 2 lim

1 2

In the limit, the lack of transmitter cooperation reduces the available sum capacity by a factor two. As the number of users grows, the distribution of the channel output is becoming Gaussian (for independent users), rather than uniform (with transmitter cooperation). 3.3.2 Binary Multiplier Channel The previous example showed how joint detection could be used to increase the region of achievable rates (and how full cooperation increases this region even further). In the next example, we see that this is not always the case. The two-user binary multiplier channel performs the real multiplication Y = X1 X2 , where X1 , X2 ∈ {0, 1}. Hence Y ∈ {0, 1}. The cardinality of the output alphabet places a restriction on the maximum sum rate, R1 + R2 ≤ 1. The capacity region is shown in Figure 3.10. In this case, there is nothing to be gained over time-sharing of the channel, or even by complete transmitter cooperation.

3.4 Gaussian Multiple-Access Channels 3.4.1 Scalar Gaussian Multiple-Access Channel The Gaussian multiple-access channel (GMAC) diﬀers from the channels examined so far, in that each user transmits from an inﬁnite alphabet.

60

3 Multiuser Information Theory R2 1.0

0.5

R1 0.5 1.0 Fig. 3.10. Capacity region of the two-user binary multiplier channel.

Deﬁnition 3.3 (Gaussian Multiple-Access Channel). The Gaussian multiple-access channel is a K-user multiple-access channel in which each user transmits from an inﬁnite 0 1alphabet, Xk ∈ R, subject to an average power constraint E |Xk |2 ≤ Pk . The channel output Y ∈ R is given by the real addition Y =

K

Xk + z,

k=1

where z is a zero mean, σ 2 , white Gaussian random vari variance 2 able, denoted z ∼ N 0, σ . The above deﬁnition is easily extended to a complex channel, in which the users transmit complex numbers and the additive noise is a zero mean circularly symmetric complex Gaussian random variable (i.e. independent real and imaginary parts, each with variance σ 2 /2). The capacity region of the Gaussian multiple-access channel is found by extending the work of [2], [129] and [76], which was for ﬁnite alphabets, to the case of inﬁnite alphabets. This was done independently by [22] and [169]. The resulting capacity region (in the real case) is given by the following constraints 1 P (S) R (S) ≤ log2 1 + bits, ∀S ⊆ K. 2 σ2 For complex channels, the factor of 1/2 is omitted. From this we can see that although the users transmit independently, the total rate is equal to that obtained by a single user, with average power

3.4 Gaussian Multiple-Access Channels

61

P (K), with exclusive access to the channel. This implies that no rate loss is suﬀered due to the distributed nature of the various users. As was the case for the binary adder channel, the total sum rate is proportional to log K, with ever increasing capacity as the numbers of users increases, with each user transmitting a smaller and smaller amount. It is convenient to introduce the following notation for Gaussian channel capacity, 1 log 1 + P/σ 2 real channel 2 2 C P, σ = (3.7) 2 log 1 + P/σ complex channel Using this notation, Figure 3.11 shows an example of a capacity region, in which P1 > P2 . R2 R R +

1

` ´ C P2 , σ 2

C =

2

(P 2

,σ P2 +

1

)

` ´ C P2 , P 1 + σ 2

` ` ´ ´ C P 1 , P 2 + σ 2 C P1 , σ 2

R1

Fig. 3.11. Example of Gaussian multiple-access channel capacity region.

Orthogonal Multiple-Access What rate vectors are achievable with only single-user decoding on the GMAC? Consider the use of an orthogonal multiple-access technique, such as time sharing (i.e. time-division multiple-access). For the purposes of illustration, consider a two-user channel and suppose user 1 has exclusive access to the channel λn symbols out of n. Correspondingly, user 2 has exclusive access for (1 − λ)n symbols. User 1 transmits only a fraction λ of the time, and hence when it does transmit it may do so with power P1 /λ while still maintaining the long-term rate of transmission during the active periods average powerP1 . The resulting is therefore C P1 /λ, σ 2 (noting that there is no interference since user 2 does not transmit at the same time). The overall transmission rate for user 1 is

62

3 Multiuser Information Theory

reduced by the factor λ. Similar arguments apply to user 2 and the achievable rate pair is P1 2 ,σ R1 = λ C λ P2 , σ2 . R1 = (1 − λ) C 1−λ Figure 3.12 shows the region obtained by varying 0 ≤ λ ≤ 1 compared to the capacity region (in this example P1 = P2 = σ 2 = 1). Orthogonal multiple-access touches the outer boundary of the capacity region at the trivial points R = (0, C P2 , σ 2 ) and R = (C P1 , σ 2 , 0), in which case either user 1 or user 2 has exclusive access to the channel. The more interesting case is achieved by setting λ=

P1 , P1 + P2

which results in the non-trivial boundary point P1 C P1 + P2 , σ 2 P1 + P2 P2 C P1 + P2 , σ 2 . R2 = P1 + P2 R1 =

Figure 3.12 shows how almost the entire capacity region of the two-user GMAC can be achieved, with the only cooperation required being a common time reference to schedule the transmissions. Similarly, a point on the boundary of the K-user capacity region of a GMAC with user powers P1 , P2 , . . . , PK can be achieved with user k having exclusive channel access a proportion λk =

Pk P (K)

of the time. The resulting rate for user k is Pk C P (K) , σ 2 , P (K) and the sum of these rates is clearly C P (K) , σ 2 . There is of course nothing special about the use of time-division as the orthogonal accessing method. The same rates result from the use of any orthogonal access method, where the variable λ accounts for the proportion of degrees of freedom used by user 1. For example, in a band-limited system with bandwidth W , we could consider allocating orthogonal frequency bands. Allocating λW Hz to user 1 and (1 − λ)W Hz to user 2 results in Rk =

3.4 Gaussian Multiple-Access Channels

R1 = λW C (P1 , λW N0 ) = λW C

P1 , W N0 λ

63

R1 = (1 − λ)W C (P2 , (1 − λ)W N0 ) = (1 − λ)W C

P2 , W N0 . 1−λ

R2 0.5 0.4 0.3 0.2 0.1

0.1

0.2

0.3

0.4

0.5 R 1

Fig. 3.12. Rates achievable with orthogonal multiple-access.

3.4.2 Code-Division Multiple-Access The remaining chapters of this book are devoted to the the design of multiuser receivers for linear multiple-access channels, typically exempliﬁed by direct-sequence code-division multiple-access. Therefore let us now consider the CDMA channel, from an information theoretic point of view. The approach that we take is to model the spreading as part of the channel speciﬁcation, and to calculate the resulting capacity. Using this approach we can gain insight into several important questions concerning CDMA channels: • What are the fundamental limitations of such systems? • We know from the data processing theorem that spreading cannot increase capacity. However spreading is useful for reasons other than capacity. Are there cases when spreading does not decrease capacity? • How much spectrum spreading should be used? • What sort of performance improvement can be expected through the use of receiver cooperation, i.e. joint decoding? • How do we design sequences to maximize capacity? There are two main cases of interest: Firstly, time-invariant spreading, according to Deﬁnition 2.1. Secondly, we can consider random spreading,

64

3 Multiuser Information Theory

according to Deﬁnition 2.2, which models aperiodic sequences (or sequences with period much longer than the symbol interval). Time-invariant spreading sequences S[i] = S, model periodic, synchronous spreading sequences. The following theorem gives the capacity of the timeinvariant CDMA channel (normalized by the number of signaling dimensions) as a function of the spreading sequences [112, 148]. We will concentrate on the complex channel (with complex spreading). Real channel results are obtained via a factor 1/2. Theorem 3.3 (Synchronous CDMA). The capacity region of the time-invariant CDMA channel with spreading sequence matrix S, average energy constraints W and noise variance σ 2 is the closure of the region 4 2 3 1 1 ∗ C [S] = R : R (T ) ≤ log det I + 2 ST WT ST . L σ T ∈K T =∅

ST is the matrix S, with the columns whose indices do not belong to T omitted. WT is formed from W by removing rows and columns whose indices do not belong to T . Capacity is achieved for Gaussian inputs X. Using our notation (3.7), we can also write the sum rate constraint Csum (S) for real or complex channels in terms of λ1 , λ2 , . . . , λL , the eigenvalues of SWS∗ L 1 R (K) ≤ Csum (S) = C λl , σ 2 . (3.8) L l=1

We shall see that the latter expression is useful for calculating the capacity for randomly spread systems. An upper bound on Csum is the capacity of the channel with L = 1, namely the K user Gaussian multiple-access channel with average energy constraint wtot = P (K). It turns out that there exist choices for the sequences S that achieve this bound [111, 112]. Theorem 3.4 (Optimal Sequences for Time Invariant CDMA). Let W = P I. The Gaussian multiple-access channel upper bound Csum ≤ CGMAC = C wtot , σ 2 nats per chip is achieved by use of sequences that satisfy SSt = IL .

3.4 Gaussian Multiple-Access Channels

65

Now the spreading sequences are the columns of S. The condition for equality in the above theorem requires the rows to be orthogonal. It is not required that the sequences themselves be orthogonal. A requirement for row orthogonality is that K ≥ L, otherwise S will be rank deﬁcient. Sequences satisfying this condition also satisfy the Welch lower bound on total squared correlation [159]. Hence, by restricting the amount of spreading to L ≤ K, no capacity penalty need be suﬀered. Note that these are the same sequences that are optimal for the correlation detector, as described in Section 2.5.3. Let us now consider the eﬀect of using randomly selected sequences, according to Deﬁnition 2.2. Now, according to (3.8), for any particular set of spreading sequences S, we can write the sum capacity Csum (S) in terms of the eigenvalues of SSt (we shall for the moment consider systems with equal powers for all the users, W = I). This motivates interest in the eigenvalue distribution of random matrices of the form SSt . Now since the spreading sequences are being chosen randomly, the eigenvalues corresponding to any particular S[i] (and the corresponding capacity) are also random. The random nature of these spreading sequences therefore makes it diﬃcult to make statements about the capacity. However, it turns out that if we instead consider a large systems limit, we can calculate the asymptotic capacity. By large systems we mean that we take the limit K → ∞ such that K/L → β, a constant. In this scenario, results from random matrix theory state that the distribution of eigenvalues tends to a non-random, known limit. This in turn enables us to compute the limiting capacity to which the random capacity for a ﬁnite system converges, as we increase the system dimensions. The following theorem shows how the eigenvalue spectrum for a large random matrix converges to a non-random limit distribution, see [64, 127, 156, 172] Theorem 3.5 (Asymptotic Spectral Distribution). Let Fˆ (x) be the empirical cumulative distribution of a randomly se1 lected eigenvalue of K SSt , where S is selected according to Deﬁnition 2.2. For large systems Fˆ (x) → F(x), where √ (x−a(β))(b(β)−x) a(β) ≤ x ≤ b(β) 2πβx F (x) = f(x) = 0 otherwise 2 a(β) = ( β − 1) b(β) = ( β + 1)2 . Example 3.4 (Convergence of Eigenvalue Distribution). Figure 3.13 shows how the eigenvalue distribution of SSt converges to the theoretical limit. The ﬁgure compares simulated eigenvalue histograms to the limit distribution f(x) for K = L = 5, 10 and 50. We see that the convergence is quite fast. Although

66

3 Multiuser Information Theory

the histogram and density do not match very well at K = L = 5, they do match well already at K = L = 10.

K=L=5

K = L = 10

K = L = 50 Fig. 3.13. Convergence of spectral density.

Using Theorem 3.5, it is possible to ﬁnd the large-systems sum capacity Cr of the randomly spread CDMA channel [150]. Theorem 3.6 (Spectral Eﬃciency of Random CDMA). For large systems, the sum capacity of random CDMA where the users have identical powers wtot /K is Cr = log (1 + γx) f (x) dx 1 1 1 β = log 1 + γ − F(γ, β) + log 1 + γβ − F(γ, β) 2 4 2 4 log e F(γ, β) − 8γ bits per chip, where γ = wtot /(Lσ 2 ) and F(x, z) =

x(1 +

√

z)2 + 1 −

x(1 −

√

2 z)2 + 1

3.4 Gaussian Multiple-Access Channels

67

The convergence is in probability, i.e. as we increase the system dimensions, Csum (S)/L → Cr . A direct solution for a closed-form expression for this capacity is also given in [106]. Using similar techniques, one may also ﬁnd the capacity of the randomly spread CDMA channel with no cooperation (i.e. each user independently decodes based only the output of its own matched ﬁlter) [150]. Theorem 3.7 (No Cooperation Spectral Eﬃciency). For large systems, the sum capacity of match-ﬁltered Random CDMA with no cooperation is γ β CMF = log 1 + 2 1 + γβ

Finally, if no spreading (or Welch Bound Equality sequences) is used, the spectral eﬃciency is the solution to the following equation. 1 Eb COPT = log 1 + 2COPT 2 N0 If orthogonal sequences are used (K ≤ L), we have C⊥ = βCOPT . These results are illustrated on Figure 3.14 which is plotted for Eb /N0 = 10 dB. For a ﬁxed E b/N0 , the spectral eﬃciency of either the optimal detector or the correlation detector is maximized by letting β → ∞. In the case of correlation detection, −1 log2 e 1 Eb − , lim CMF = K→∞ 2 2 N0 which converges to log2 e/2 ≈ 0.72 as Eb /N0 → ∞ [62, 150]. Figure 3.15 shows the spectral eﬃciency for both optimal detection and correlation detection as a function of Eb /N0 for K → ∞. The spectral efﬁciency of the optimal system grows without bound as the SNR increases. In fact, comparing to (3.6), we see that for L = 1 and a ﬁxed, large K, the spectral eﬃciency behaves like 1 πeK log2 2 2 with increasing SNR. This is the same limit obtained for the K-user binary adder channel in Section 3.3.1. In the limiting case of many users or high signal-to-noise ratios, it it possible to write a closed-form expression for the sum capacity of the randomly spread channel [56].

68

3 Multiuser Information Theory Spectral Eﬃciency bits/chip No spreading/WBE Spreading 3 Random Spreading Joint Decoding

Orthogonal 2

Random Spreading Matched Filter

1

0

1

2

β

Fig. 3.14. Spectral eﬃciency of DS-CDMA with optimal, orthogonal and random spreading. Eb /N0 = 10 dB. Spectral Eﬃciency bits/chip

2

Joint Decoding

1

Matched Filter

0

Eb /N0 (dB)

0

2

4

6

8

10

Fig. 3.15. Spectral eﬃciency of DS-CDMA with random spreading.

3.4 Gaussian Multiple-Access Channels

69

Theorem 3.8 (Limiting Capacity). For large systems (many users) or for high signal-to-noise ratios, the average sum capacity of a direct-sequence CDMA channel with equal powers is given by K K −L ln − 1 nats, K ≥ L. lim Cr = log γ + γ→∞ L K −L Furthermore 0 lim CGMAC − Cr = γ→∞ 1

L K L K

→ 0, → 1.

From this limiting expression we can see that the penalty for using randomly selected, rather than optimal spreading sequence in large systems is at most 1 nat. As L/K decreases, random spreading becomes optimal. All of the large systems results that we have presented have been for the case when all the users transmit with equal powers. If unequal powers are used, the analysis may be extended by using appropriate results from random matrix theory. The results end up being expressed in terms of Stieltjes transforms. Deﬁnition 3.4 (Stieltjes Transform). The Stieltjes transform m(z), z ∈ C+ of a cumulative distribution F (x) with probability density function F (x) = f (x) is given by 1 m(z) = f (λ) dλ, λ−z and possesses the inversion formula

b

f (x) dx = a

1 lim π η→0+

b

[m(ξ + iη)] dξ

(3.9)

a

We shall continue using the convention of using upper-case letters for cumulative distributions and the corresponding lower case letter for densities.5 As a special case of the result presented in [128] we have the following. 5

Most of the following results are usually presented in measure-theoretic terms. We assume the existence of probability density functions in order to avoid measuretheoretic concepts.

70

3 Multiuser Information Theory

Theorem 3.9 (Limiting Spectrum for Unequal Powers). Let the elements of S ∈ CL×K be chosen according to Deﬁnition 2.2. Let W = diag(w1 , w2 , . . . , wK ), wi ∈ R be independent of S and let the empirical distribution of {w1 , w2 , . . . , wK } converge almost surely to a cumulative distribution function H with density H = h as K → ∞. Then for large systems, the empirical distribution function of the eigenvalues of the matrix SWSt /spreadlength L converges to a non-random probability distribution whose Stieltjes transform m(z) ∈ C+ is the unique solution to −1 τ h(τ ) 1 dτ . m(z) = − z − β 1 + τ m(z)

(3.10)

We can now describe the large-systems capacity result for the unequal power case. Theorem 3.10. At each symbol interval, let the matrix of spreading sequences S be randomly selected as in Theorem 3.9 and let the empirical distribution function of the users’ energies converge to a non-random cumulative distribution function H. Then xL Cr = log 1 + 2 f (x) dx, (3.11) σ where the Stieltjes transform of the cumulative distribution function F (x) satisﬁes (3.10). This theorem shows that for ﬁxed β, the sum capacity depends only upon the noise variance and the limiting distribution of the users’ energies, albeit through the rather awkward (3.10), combined with the inverse Stieltjes transform. Example 3.5. Let the power distribution be discrete, consisting of J point masses at powers 0 < P1 < P2 < · · · < PJ , h(τ ) =

J

j=1

αj δ(τ − Pj )

5 where αj = 1. Substituting into (3.10) and using the sifting properties of the Dirac delta function results in ⎛ ⎞−1 J

Pj αj 1 ⎠ . m(z) = − ⎝z − (3.12) β j=1 1 + Pj m(z)

3.4 Gaussian Multiple-Access Channels

71

The resulting equation for m(z) is a degree J + 1 polynomial. For a single point mass (the equal power case), solution of the corresponding quadratic, together with the inversion formula (3.9), results in the expression for F given in Theorem 3.5. Capacity Computation for Arbitrary Power Distributions In all but the simplest of cases, direct computation of the Stieltjes transform m(z) of the limiting eigenvalue distribution F (λ) using (3.10) is diﬃcult, if not intractable (for example four point masses results in a quintic for m). It is possible however to ﬁnd a parametric relation for the random sequence capacity that side-steps the problem of solving (3.10). This parametric equation is in terms of the capacity of a parallel (orthogonal modulation) channel with the same power distribution. Deﬁnition 3.5 (Shannon Transform). Let h(x) be a probability density on the positive real numbers. Deﬁne 1 ∞ log (1 + γx) h(x) dx. ηH (γ) = 2 0

Theorem 3.11. Retaining the deﬁnitions and assumptions of Theorem 3.10, the spectral eﬃciency of the randomly spread multiple-access channel is given parametrically via s 1 s ηH (s) − ln + − 1 β γ γ −1 s dηH (s) γ =s 1− β ds

Cr = ηF (γ) =

(3.13) (3.14)

where for convenience γ = L/σ 2 . Example 3.6 (Equal Powers). Let h(x) = δ(x − 1) be a single point mass, corresponding to the unit equal power case. Consider β = 1. Then ηH (s) = ln(1 + s) and hence γ = s(1 + s), from (3.14). Solving for s in terms of γ gives s=

1 −1 + 1 + 4ρ , 2

which when substituted into (3.13) gives √ 4ρ + 1 − 1 ηF (ρ) + 2 ln 4ρ + 1 + 1 − 1 − 2 ln 2 2ρ

72

3 Multiuser Information Theory

which is the same result obtained in [150, (9)], which was given in Theorem 3.6 above. Although this parametric form appears to simplify the problem of ﬁnding closed-form capacity results, it suﬀers from the same basic problem as the ﬁxed-point equation (3.10). For instance, the J-level discrete distribution described in Example 3.5 results again in a degree J + 1 polynomial. Example 3.7 (Two Point Masses). Consider a two-level power distribution with point masses at powers P1 > 1 > P2 , normalized such that the average power over all users is 1, h(x) =

P1 − 1 1 − P2 δ(x − P1 ) + δ(x − P2 ). P1 − P2 P1 − P2

Then ηH (s) and γ(s) are given by (P1 − 1) log (sP2 + 1) + (1 − P2 ) log (sP1 + 1) P1 − P2 s (sP1 + 1) (sP2 + 1) γ(s) = . (P1 + P2 ) s − s + 1

ηH (s) =

(3.15) (3.16)

From (3.16) it can be seen that solution for s in terms of γ (which would yield a closed-form expression for ηF (γ)) involves solution of the same cubic equation arising in (3.12). Nevertheless, it is straightforward to parametrically obtain plots of ηF (γ) using (3.15) and (3.16) in Theorem 3.11. Thus the polynomial is still present, there is just no need to solve it. Example 3.8 (Rayleigh Fading). Let each user’s transmission experience independent Rayleigh fading, namely e−x/2 h(x) = √ √ . 2π x Then ηH (x) may be easily found via numerical integration and 1 e− 2s π2 s3/2 . γ(s) = 1 − erf √21√s The resulting random sequence capacity can therefore be numerically determined. The result is shown in Figure 3.16, as a function of γ. Furthermore, it is possible to ﬁnd ηF (γ) directly from ηH (s), in the following way. For each point (s, ηH (s)) there is a single corresponding point (γ, ηF (γ)), which can be obtained directly from the plot of ηH (s) by a simple point-wise geometric construction. Plot the curve β −1 ηH (s) and extend a tangent line

3.5 Multiple-Access Codes

73

0.8

Equal Power 0.6

ηF (γ)

0.4

Rayleigh

0.2 0 0

2

4

6

8

10

12

γ Fig. 3.16. Random sequence capacity with Rayleigh fading.

back from s to the s = 0 intercept, say η0 . Denote the vertical drop of this line by α = β −1 ηH (s) − η0 , which is always less than 1. The point (s, ηH (s)) corresponds to a point (γ, ηF (γ)) = (s + Δs, ηH (s) + Δη) above and to the right, according to the following theorem, illustrated in Figure 3.17. Theorem 3.12. The following point-wise geometrical construction obtains ηF (γ) from ηH (s). 1 ηH (s) + Δη β γ = s + Δs,

ηF (γ) =

where the coordinate increments Δη and Δs are given in terms of α=s

dηH (s) ds

(3.17)

as Δη(α) = − ln(1 − α) − α αs Δs(α) = . 1−α The utility of this theorem is due to the fact that typically, ηH (s) and its derivative is easily computed and in some cases may be found in closed form.

3.5 Multiple-Access Codes Although it is important to know the ultimate information rate limits for the multiple-access channel, practical systems require channel coding schemes

74

3 Multiuser Information Theory

Δs

ηF (γ) Δη ηH (s) /β

α

s γ Fig. 3.17. Finding the asymptotic spectral eﬃciency via a geometric construction.

to achieve these rates. In addition to providing error detection/correction capability in the presence of noise, multiple access channel codes should also have the property that they separate the users’ transmissions (reduce MAI). A code that can perform this task without error is called uniquely decodeable. Deﬁnition 3.6 (Uniquely Decodeable). A code for the multiple-access channel is uniquely decodeable (UD) if, in the absence of noise, the mapping MK → Y n is one-to-one. Codes that fulﬁll this deﬁnition have the property that the combination of the users’ encoding function and the channel function is uniquely invertible. Both block and trellis uniquely decodeable codes have been constructed for a variety of multiple-access channels. In particular, much eﬀort has been spent on the binary adder channel. In the following sections, we shall summarize some of the currently available techniques. For a survey of channel codes up to 1980, one is directed to Farrell [40]. Note that the requirement of unique decodeability is related to the concept of the zero-error capacity region. If we require codes that can be decoded with zero probability of error, then we should compare their performance to the zero-error capacity region, rather than the -error region. In general, the zeroerror capacity region for the multiple-access channel is unknown (apart from special cases such as [84]. It has in fact been shown for the two-user binary

3.5 Multiple-Access Codes

75

adder channel, the zero error capacity region is strictly smaller than the -error region [141]. Much of the literature on coding is for the binary adder channel, and as such, is a useful basis for comparison of various schemes. Figure 3.18 summarizes graphically various codes of historical interest, including the current best known code (in terms of sum rate).

R2

6 1.0

•7•8•9•10

••1112 6

•

@ ••1314 15 1 ••3 •5

0.5

•4

0

0.5

@ @

•2

@

@ @

@

@ @ @

1.0

- R1

Fig. 3.18. Rates achieved by some existing codes for the BAC

3.5.1 Block Codes Let us begin by introducing some notation for block codes. Denote the codebook for user k by Ck , the number of codewords by Mk = |Ck |, and the rate Rk = n1 log2 Mk . Block Codes for the Binary Adder Channel Let us take a look at our ﬁrst multiuser code for the two-user BAC, and in doing so, we shall illustrate the principle of unique decodeability. Example 3.9 (Uniquely Decodeable Code for the Two-User BAC). Until 1985, the best known code for the 2-BAC was also one of the simplest. It assigns

76

3 Multiuser Information Theory Point number R1 1 0.5 2 0.571 3 0.512 4 0.5 5 0.75 6 0.33 7 0.100 8 0.143 9 0.178 10 0.222 11 0.288 12 0.307 13 0.414 14 0.434 15 0.512

R2 0.792 0.558 0.793 0.5 0.5 0.878 0.999 0.999 0.998 0.994 0.972 0.961 0.885 0.869 0.798

R1 + R2 1.292 1.129 1.306 1.0 1.25 1.208 1.099 1.143 1.177 1.216 1.260 1.268 1.300 1.303 1.310

Reference [67] [67] [14] [96] [20] [27] [14] [14] [14] [73] [14] [14] [14] [14] [141]

Type Block Block Block Convolutional Trellis Block asynch. Block Block Block Block Block Block Block Block Block

Table 3.1. Coding schemes shown in Figure 3.18.

to user X1 the codebook C1 = {00, 11}, and to user X2 the words C2 = {00, 01, 10}. This output words resulting from C1 × C2 are shown in Table 3.2. This code was ﬁrst given in [67]. For this code, R1 = 0.5, and R2 = 0.792. The sum rate R1 + R2 = 1.29. This code is represented by point 1 in Figure 3.18. From the table, one can verify that all output codewords are distinct, hence the code is uniquely decodeable. Let us now assume that we want to increase the C2 C1 00 01 10 00 00 01 10 11 11 12 21 Table 3.2. Uniquely decodeable rate 1.29 code for the two-user BAC.

rate of the code, by including another codeword for user 1, say 01. Table 3.3 shows the new codebook pair, and the corresponding channel outputs. We now have the problem that two channel outputs are ambiguous: the output 01 could have been caused by the transmission (user 1, user 2) of either (01, 00) or (01, 00). Similarly, the output 11 is ambiguous. The best that the decoder could do in such cases would be to choose the message pair at random, which would of course result in an irreduceable error rate. It is interesting to compare the sum rate of the code in the previous example, R1 + R2 = 1.2925, this was the best code in terms of sum rate, until 1985, when it was beaten by a R1 + R2 = 1.306 code [14]. More recently, this record was pushed to R1 + R2 = 1.30999 [141]. The diﬃculty in improving the sum

3.5 Multiple-Access Codes

C1 00 11 01

00 00 11 01

C2 01 01 12 02

77

10 10 21 11

Table 3.3. Non uniquely decodeable code for the two-user BAC.

rate of such codes may turn out to be a limitation of the zero-error capacity of the channel. Kasami and Lin were the ﬁrst to consider code construction for the twouser BAC [67]. They considered code constructions in which C1 is taken to be a linear (n, k) block code [77], which contains the all ones codeword 11 . . . 1. The members of C2 are then chosen from the cosets of C1 . One particular UD construction is to select the members of C2 as the coset leaders of the cosets of C1 which are not equal to C1 , their one’s complements and the all zeros codeword 00 . . . 0. Using this technique one can construct codes with rate vector k 1 R= , log2 2n−k+1 − 2 . n n They also provide bounds on the number of vectors that can be taken from the cosets of C1 . Example 3.10 (Kasami-Lin Construction). Let us now use this method to construct a two-user code. Let the codebook for user 1, C1 be the (7, 4) Hamming code. This code is perfect [77], which means that it has as coset leaders every error pattern of weight 0 or 1. We include these eight coset leaders in the codebook for user 2. According to the construction, we also include the one’s complement of each of these codewords, except for the all-one word, which is already in C1 . This gives |C1 | = 15. Therefore this code has rate vector R = (0.571, 0.558) and is point 2 on Figure 3.18. The two codebooks are shown in Table 3.4. Coebergh van den Braak and van Tilborg have described another construction for the 2-BAC [14]. They construct code pairs at sum rates above 1.29, but only marginally. The highest rate code presented has n = 7, R = (0.512, 0.793), giving sum rate 1.30565. This is shown as point 3 in Figure 3.18. Apart from codes with high sum rate, they also construct many codes achieving point very close to the single-user constraint (i.e. with rate for one of the users very close to 1, and a non zero rate for the other user). Kasami and Lin also investigated codes for the noisy BAC, and in [68], give bounds on the achievable rates for certain block coding schemes for the noisy channel. In [69] they present a reduced complexity decoding scheme, which however is still exponentially complex as the code length increases. Van Tilborg gives a further upper bound on the sum rate for the two-user

78

3 Multiuser Information Theory C1 0000000 1101000 0110100 1011100 1110010 0011010 1000110 0101110 1010001 0111001 1100101 0001101 0100011 1001011 0010111 1111111

C2 0000000 0000001 0000010 0000100 0001000 0010000 0100000 1000000 1111110 1111101 1111011 1110111 1101111 1011111 0111111

Table 3.4. Rate R = (0.571, 0.558) uniquely decodeable code for the two-user binary adder channel.

BAC, where C1 is linear [136]. Kasami et al. [70] have used graph theoretic approaches to improve upon lower bounds for the Kasami–Lin codes. They relate the code design problem to the independent set problem of graph theory, and use the Tu´ran theorem, which gives a lower bound on the independence number, in terms of the numbers of vertices and edges of the graph. It is interesting however to note that if R1 > 0.5, the best code pairs require both codes to be non-linear [160]. K-user binary adder channel Chang and Weldon [18] have constructed codes using an iterative method for the K-BAC. Their construction for the noiseless case (they also present a similar construction for the noisy channel) is based on a linearly independent diﬀerence matrix d ∈ {−1, 0, 1}K×n . User k is assigned two codewords, ck,1 and ck,2 obtained from dk , the kth row of d in the following way. ⎧ ⎪(0, 0) if (dk )l = 0 ⎨ (ck,1 )l , (ck,2 )l = (1, 0) if (dk )l = 1 ⎪ ⎩ (0, 1) if (dk )l = −1 The iteration is on the matrix d. Let d0 = [1]. Then ⎡ ⎤ dj−1 dj−1 dj = ⎣dj−1 −dj−1 ⎦ I2j−1 02j−1

3.5 Multiple-Access Codes

79

deﬁnes a K = (j + 2)2j−1 user code of length 2j , where I2j−1 and 02j−1 are the 2j−1 × 2j−1 identity and all-zero matrices respectively. Example 3.11 (Chang-Weldon Construction). Let us consider the ChangWeldon construction for 3 users. We generate the diﬀerence matrix ⎡ ⎤ 1 1 d1 = ⎣1 −1⎦ 1 0 This gives the codebooks C1 = {11, 00}, C2 = {10, 01}, C3 = {10, 00}. The code has rate vector R = ( 12 , 12 , 12 ). This construction is interesting because of the following theorem, which says that these codes are asymptotically optimal in a certain sense. Theorem 3.13. The Chang-Weldon construction is asymptotically good, in the sense that as the number of users increases, the ratio of the sum code rate to the sum capacity approaches unity. R (K) lim = 1. K→∞ Csum Although the ratio of the code rate to the sum capacity approaches one, the absolute diﬀerence increases without bound, with the logarithm of j Ferguson [41] generalizes these codes, by replacing the identity and all-zero matrices in the iteration of (3.5.1) with arbitrary matrices A, B with entries from {−1, 0, 1} such that the modulo 2 reduction of the sum A + B is an invertible matrix. Ferguson also determines the size of the equivalence classes of the Chang–Weldon codes. Hughes and Cooper [61] have investigated codes for the K-user binary adder channel, where the users have diﬀerent rates. They modify the ChangWeldon construction [18] to distribute the codewords among as few users as possible. The main result is that K-user codes may be constructed for any rate point within the polytope derived from the capacity region by subtracting 1.09 bits/channel use from each constraint. In particular they construct family of K-user code with sum rate R (K) ≥ Csum − 0.547. Frame-Asynchronous Binary Adder Channel Block codes for the frame-asynchronous two-user BAC have been considered by Deaett and Wolf [27, 165]. One such simple code assigns two codewords of length n1 to user 1: the all one word 11 . . . 1 and the all zero word, 00 . . . 0. The codebook for the second user is all binary n2 -tuples such that the ﬁrst symbol is 0, and the word does not contain n1 consecutive ones. The maximum

80

3 Multiuser Information Theory

rate pair is achieved for n1 = 3: R = (0.33, 0.878). This is point 6 on the graph. Plotnik [98] considers the code construction problem for the frameasynchronous channel, with random access to the channel. Example 3.12 (Deaett-Wolf Construction). Let use now construct a code for the two-user frame-asynchronous channel, with n1 = 2 and n2 = 4. To user 1 we assign the codebook C1 = {00, 11}. According to the construction, we assign to user 2 the codebook {0000, 0001, 0010, 0100, 0101}. K Users, N -ary Orthogonal Signal Channel Chang and Wolf [19] have constructed codes for a generalization of the KBAC, to larger input alphabets. In particular they consider a channel in which each user may at each symbol interval, activate one of N orthogonal frequencies, {f1 , f2 , . . . , fN }. Of course, we can consider the use of any set of orthogonal signals, but for convenience, we shall refer to frequencies. The receiver observes the output of each frequency. Depending upon what type of observation is made of each frequency, two channels are deﬁned: the channel with intensity information, and the channel without intensity information. For the former case, the number of users transmitting on each frequency is available to the receiver. The latter case provides only a binary output of active/not active for each frequency. Three code constructions are given for the channel without intensity information. All of these constructions are characterized by the use of unique frequencies, or frequency patterns as markers for each user. This has the eﬀect of transforming the system into a frequency division multi-access system. The ﬁrst construction, for K < N assigns two codewords of length one to each user, Ck = {f1 , fk+1 }. This construction has sum rate N − 1. The remaining constructions are for the K = 2 channel, for which the reader is referred to the original work. Wilhelmsson and Zigangirov construct codes for this channel with polynomial decoding complexity [162]. The construction for the channel with intensity information is a generalization of the approach of Chang and Weldon [18]. As for the Chang–Weldon codes, their construction is asymptotically optimal, if one increases K → ∞, while ﬁxing N . For further results concerning coding for the K-BAC, the interested reader is referred to [17]. Mathys considers the case when random access to the channel is allowed [86]. Binary Switching Channel Vanroose [144] considers coding for the two-user binary switching MAC, which is a counterpart to the two-user BAC. Each user transmits symbols from {0, 1}, but the output is Y = x1 /x2 . Division by 0 results in the ∞ symbol. This is of interest, as it is the only other ternary output, binary input MAC form. The capacity region is determined, and is found to touch the total

3.6 Superposition and Layering

81

cooperation line. In addition, the capacity region is also shown to be the zero-error capacity region. 3.5.2 Convolutional and Trellis Codes The subject of designing trellis codes for multiple-access channels has received considerably less attention than for block codes. Work has focused entirely on the binary adder channel. Peterson and Costello have investigated convolutional codes for the twouser binary adder channel [96]. They introduce the concept of a combined twouser trellis (see Figure 3.19), and deﬁne a distance measure, the L-distance between any two channel output sequences. They go on to prove several results. Among these are conditions for unique decodeability and conditions for catastrophicity. In a rather striking theorem, they show that no convolutional code pair for the two-user BAC exists at a sum rate greater than 1, which could have been achieved anyway with no cooperation. Figure 3.19 shows a combined trellis for a two-user uniquely decodeable code. The branch labels on the trellis are of the form u1 u2 | v1 v2 . If user 1 and user 2 input u1 and u2 respectively to their separate convolutional encoder inputs, the channel output is the symbol v1 followed by the symbol v2 . The state labels on the combined trellis are of the form s1 s2 , where sk is the state of user k’s individual code. Decoding takes place by simply applying the Viterbi decoding algorithm [152] to the combined trellis, using L-distance as the metric. This code achieves the upper bound set for convolutional codes on this channel, and is shown as point 4 in Figure 3.18. Chevillat has investigated trellis coding for the K-BAC [20]. He ﬁnds convolutional code pairs with large dL,free and gives a two-user non-linear trellis code for the BAC, found by computer search. This code is shown in Figure 3.20. It possesses sum rate 1.25, and is point 5 on the comparison ﬁgure. Peterson and Costello compute bounds on error probability and free distance for arbitrary two-user multiple-access channels [95]. Sorace gives an algebraic random coding performance bound [130].

3.6 Superposition and Layering We shall now see that the vertices of achievable rate regions possess special properties that provide a “single-user” interpretation of the capacity region. First, let us formally describe these vertices, or extreme points of the achievable rate region. Vertices are rate points R that have (after a possible re-indexing of the users) elements with the following form6 : 6

See [57] for the one-to-one correspondence between such points and vertices of the rate region

82

3 Multiuser Information Theory 0 | 00 0

0

@

00 | 00

1 | 11@

@

1

@ @ 1

1 | 10 Single user trellis for user 1

0 | 00 0

0

@

1 | 11@

@

0 | 10 1

@

00

11

11

@HH01 | 11 J

H J @

11 | 22 H J@10 | 11 H

H HH

J@ 00 | 10 J @

HH H 01 01 | 01J @

01 H @HH 10 | 21 J @ 11 | 12 @ HHJ @ H @ J

H @ @ J HH@ 00 | 01 | 12 H@ @ 1001| 10 J

H H @ 10 @ J 10 H

11 | 21H @ J H H J @

HH | 02 HH@ J

01 00 | 11 H@ J

10 | 20 HH @ J

H @

@

0 | 01

00 H

@ @ 1

11 | 11

Combined trellis 1 | 01 Single user trellis for user 2 Fig. 3.19. Combined 2 user trellis for the BAC 0001,0010,0011,0100

0 @ @

0001,1110

0

@ @@ @ 0000,1011 @ 0100,1000 @@ @ @ @@ @ 1001,1011 @ @@ @ 0101,0110 @ @@ @ @ @ @ @ @ 1 @ 1 0101,1001,1010,1101

User 1

0 @

0

@@ 0000,1111 @@ @@ @@ 0011,1100 @@ @ @ 1 @ 1 0011,1100

User 2

Fig. 3.20. Two user nonlinear trellis code for the BAC

Ri = I Xi ; Y | X{1,2,...,i−1} ,

(3.18)

that is R looks something like R = (I (X1 ; Y ) , I (X2 ; Y | X1 ) , . . . , I (XK ; Y | X1 , X2 , . . . , XK−1 )) . Note that by deﬁnition of the achievable rate region (3.1), such points are on the boundary of the region. In Section 3.3.1, we saw that the vertices of the achievable rate region (in that case coincident with the capacity region) were

3.6 Superposition and Layering

83

achievable by interference cancellation. This property holds for all vertices, and is in fact due to the form of the vertices (3.18). At a vertex, the users may be ordered, such that the maximum rate is given by the mutual information between each user and the output, conditioned only on knowledge of “previous” users, where previous is deﬁned by the ordering.

X1

Source 1

Encoder 1

Source 2

Encoder 2

Source 3

Encoder 3

Linear Multiple-Access Channel

Y

X2

+

X3 Noise

X1

Decoder 1 I(X1 ;Y )

X2 X3 interference

X2

Decoder 2

+

I(X2 ;Y |X1 )

X3

X3

Decoder 3

interference

+

I(X3 ;Y |X1 , X2 )

no interference

Fig. 3.21. Successive cancellation approach to achieve vertex of capacity region.

Wither reference to Figure 3.21, the achievability of the vertices is proved using a successive decoding argument as follows. The ﬁrst user treats all other users as noise, and sees a discrete memoryless channel. By the standard random-coding argument, this user may transmit at rate R1 ≤ I (X1 ; Y ) with arbitrarily low error probability, which is the rate point required by (3.18). We now assume that we have used such a coding scheme for this user, and that we know perfectly the transmitted data, which is now available for subsequent users. The second user now treats the remaining users, k > 2 as noise,

84

3 Multiuser Information Theory

and once again from the single-user random coding argument, may transmit at R2 ≤ I (X2 ; Y | X1 ), recalling that the users transmit independently. We continue this argument down the chain for all users, each transmitting using single user codes which are decoded one by one, using knowledge of all previous users. For linear channels, such as the binary adder channel, or the Gaussian multiple-access channel, the previous users’ data may be incorporated into the decoding process by subtracting it from the channel output, hence the name interference cancellation. A common objection to such schemes is the “error propagation” argument, whereby errors in one decoding step lead to even more errors in the next. However in proving the achievability of these rate points by successive cancellation, we do not suﬀer from this problem, as the single-user coding theorem guarantees the existence of codes with vanishing error probabilities. In fact it is possible to bound the probability that an error is made anywhere in the step-by-step process by a function which tends to zero exponentially with the codeword length. The error propagation is only a concern for a practical implementation of the scheme. In order properly understand the eﬀect of asynchronism in the next section, we ﬁrst need to better understand the role of the convex hull operation in Theorem 3.1. This convex hull operation is due to the idea of time-sharing. Given that the two rate vectors, R1 and R2 are achievable, then every point on the line connecting R1 and R2 is also achievable, simply by using the codebook corresponding to the the point R1 for λn symbols and that corresponding to R2 for the remaining (1 − λ)n, 0 ≤ λ ≤ 1. In general, we can achieve any additional point that is the convex combination of any number of achievable points. Note that in order to implement this time-sharing scheme, a common time reference must be available, in order for the users to agree upon when to change transmission strategies. Carath´eodry’s theorem [32] states that every point in the convex closure of a connected compact set A is contained within at least one simplex7 which takes its vertices from A. This implies that every point within the capacity region for a K user channel may be achieved by time-sharing between at most K + 1 vertices, requiring each user to have access to K + 1 codebooks. We now see that we may achieve any point in the capacity region by timesharing between at most K + 1 successive cancellation schemes, each with K cancellation steps.

3.7 Feedback In this section, we will consider some results for the multiple-access channel under the assumption of perfect feedback, by which we mean that the encoder for each user has available all previous channel outputs. Figure 3.22 7

A simplex in d-dimensional space is a polytope with d + 1 aﬃnely independent vertices, e.g. a tetrahedron for 3-dimensional Euclidean space.

3.7 Feedback

85

shows a two-user channel with feedback. Is is a somewhat surprising result

Source 1

Encoder 1

Channel

Source 2

Encoder 2

Fig. 3.22. Two-user MAC with perfect feedback.

of single-user information theory, that noiseless feedback from the receiver to the transmitter does not increase the capacity of a memoryless channel, a fact proved by Shannon in 1956 [124]. Feedback does however increase the capacity of channels with memory, by aiding the transmitter in predicting future noise, using the time-correlation properties of the channel. It is therefore not altogether surprising that feedback can increase the set of achievable rates for the multiple-access channel. Such feedback essentially enables the users to cooperate in their transmissions to some degree. We shall also see that the use of feedback can simplify the coding schemes required for transmission. As a motivation, let us see how we can use feedback in a simple way to increase the set of achievable rates. The following example is due to Gaarder and Wolf [48], which was historically the ﬁrst example of feedback increasing the set of achievable rates. Example 3.13 (Gaarder-Wolf Feedback Scheme). Consider the two-user binary adder channel, X1 , X2 ∈ {0, 1}, Y = X1 +X2 ∈ {0, 1, 2}. At the output, Y = 1 is the only ambiguous symbol. Call this output an erasure. Transmission will take place in two stages. Stage 1: In the ﬁrst stage, each user transmits at rate 1. At the output of the channel, the decoder will not be able to successfully decode, since there are erasures. However each user observes the channel output, which combined with the knowledge of its own transmitted data, allows the deduction of the transmitted data of the other user. Stage 2: The users can now cooperate to re-transmit those bits of one user, say user 1 which suﬀered erasure. This can be done at a rate of log2 3, since both users know from stage 1 what must be transmitted. The output can easily determine the second user’s erased bits from those of the ﬁrst.

86

3 Multiuser Information Theory

What rates can be achieved using this method? Consider a block of n transmissions. Let the ﬁrst stage use λn symbols. With probability exponentially approaching 1 with n, there will be 12 λn erasures in the ﬁrst stage. For each erasure, we must provide 1 bit of information in stage 2, i.e. we must have (1 − λ)n log2 3 =

1 λn. 2

Solving for λ, which is the rate for each user, we ﬁnd R1 = R 2 = λ =

2 log2 3 . 1 + 2 log2 3

The resulting sum rate 1.52 exceeds the sum constraint of 1.5 for the channel with no feedback. This two-stage approach can be thought of in the following way. In the ﬁrst stage, the users transmit at the highest possible rate to each other via the feedback link. The output however cannot fully decode, but may be able to partially decode, e.g. with a list decoder. During the second stage, the users cooperate to resolve any ambiguity at the output. For the case of list decoding, they would transmit the index of the correct codeword in the list. This listdecoding method was in fact used to show the following achievable rate region for the two user discrete memoryless MAC, where either one, or both users can observe the channel output. Theorem 3.14 (Cover-Leung Achievable Rate Region). The following is achievable rate region for the K = 2 discrete memoryless multiple-access channel (XK ; p(y | xK ) ; Y) with perfect feedback to one, or both users. R [pU,X1 ,X2 ,Y (u, x1 , x2 , y)] = {R : 0 ≤R1 ≤ I (X1 ; Y | X2 , U ) 0 ≤R2 ≤ I (X2 ; Y | X1 , U ) 0 ≤R1 + R2 ≤ I (X1 , X2 ; Y ) , where the joint distribution pU,X1 ,X2 ,Y (u, x1 , x2 , y) is of the form pU,X1 ,X2 ,Y (u, x1 , x2 , y) = pU (u) pX1 |U (x1 | u) pX2 |U (x2 | u) pY |X1 ,X2 (y | x1 , x2 )

(3.19)

and U ∈ U is an arbitrary random variable, with |U| ≤ min{|X1 | |X2 | + 1, |Y| + 2}. The achievability of this region was originally shown in [23], for the case of feedback to both users. It was shown to also be achievable for feedback to only one user in [164].

3.7 Feedback

87

For a certain class of channels, the achievable rate region of Theorem 3.14 coincides with the capacity region. The converse required to show the optimality of the Cover-Leung region for these channels was proved by Willems in [163]. Theorem 3.15 (Optimality of Cover-Leung Region). Consider the class of two-user discrete memoryless multiple-access channels in which at least one input is a deterministic function of the output and the other input, i.e. either H (X1 | Y, X2 ) = 0 or H (X2 | Y, X1 ) = 0. For channels within this class, the rate region of Theorem 3.14 is the capacity region, for the case of perfect feedback to one or both users. The binary adder channel is a member of the class of channels described in Theorem 3.15. Example 3.14 (Binary Adder Channel with Feedback). The capacity region for the binary adder channel with feedback is shown in Figure 3.23. Also shown is the total cooperation line, R1 + R2 = log2 3 = 1.58496. The maximum sum rate achievable with feedback is at the equal rate point, (R1 , R2 ) = (0.7911, 0.7911), giving a sum value of 1.5822, slightly less than that for total cooperation. For the binary adder channel, as the number of users increases, this diﬀerence vanishes, and with feedback, the total cooperation rate may be achieved at the equal rate point. We now describe a simple scheme due to Kraemer [74], which achieves the capacity region for the two-user BAC (and in fact any channel for which H (X1 | X2 , Y )=0). The system diagram is shown in Figure 3.24. X1 , X2 generate new information such that Pr(X1 = 0) = Pr(X2 = 0) = p. The random variable V ∈ {0, 1}, which is common to both transmitters is generated from the feedback signal in the following way. The encoder for V takes as its input the feedback output Y , via a mapping, which outputs 1 if Y = 1, and 0 otherwise (i.e. the mapping is an erasure detector). This feedback signal is encoded, using an identical random block code for each user at a rate RV . Each user transmits the modulo-2 sum Xk ⊕ V . The cooperative random variable V serves to resolve the ambiguity about the erased Y = 1 channel outputs. In order for the scheme to work, note that for each Y = 1 erasure, V must successfully transmit one bit of information to the receiver, i.e. we must have RV ≥ Pr(Y = 1),

(3.20)

with arbitrarily low error probability for V . Under these conditions, the rate point achieved will be (R1 , R2 ) = (h (p) , h (p)). Let us now calculate these rates. V sees the binary symmetric erasure channel shown in Figure 3.25. The capacity of this channel can be found as follows, 0 1 p2 2 2 , (3.21) RV ≤ max I (V ; Y ) = (1 − p) + p · 1 − h (1 − p)2 + p2 Pr(V =0)

88

3 Multiuser Information Theory R2

R1 Fig. 3.23. Capacity region for the two-user binary adder channel with feedback.

for Pr(V = 0) = 12 . Note also that Pr(Y = 1) = 2p(1 − p).

(3.22)

Substituting the maximum value for RV (3.21) and the erasure probability (3.22) into (3.20) and solving for p, we ﬁnd p = 0.23766, hence (R1 , R2 ) = (0.7911, 0.7911), which is on the boundary of the capacity region. This scheme is a special case of the Cover-Leung list decoding scheme. For each received codeword of length n, we have a certain number of erasures, e. The decoder for Y generates a list with 2e entries, consisting of each possible codeword, assuming all combinations of values for the erased symbols. However superimposed upon this “fresh” information is the resolution information V , which can be thought of as the index into the list (for the previous codeword). Example 3.15 (Gaussian Multiple-Access Channel with Feedback). The capacity region for the white Gaussian noise multiple-access channel with feedback was found by Ozarow [93]. Consider the two-user white Gaussian multiple-access channel, in which for k = 1, 2, each source Xk ∈ R, is subject to an average power constraint E Xk2 ≤ Pk . The channel output is given by

3.7 Feedback

89

Encoder 1

V Souce 1

map

+

X1

+

X2 Source 2

map

+

V Encoder 2

Fig. 3.24. Simple feedback scheme for the binary adder channel. V

(1 − p)2

0 H

Y

- 0

@HH 2p(1 − p) @ HH HH p2 @ HH @ j 1 H @ * @ 2 @ p @ @ 2p(1 − p) - 2 R @ 1

Pr(Y = 1) = 2p(1 − p)

(1 − p)2

Fig. 3.25. Channel seen by V .

Y = X1 + X2 + z, where z ∼ N 0, σ 2 . The capacity region for this channel under the assumption of perfect feedback to both users is given by

90

3 Multiuser Information Theory

C=

/

P1 2 (1 − ρ ) , σ2 P2 2 (1 − ρ ) , σ2 √ 6 P1 + P2 + 2ρ P1 P2 1 . 0 ≤ R1 + R2 ≤ log 1 + 2 σ2 1 log 1 + 2 1 0 ≤ R2 ≤ log 1 + 2

(R1 , R2 ) : 0 ≤ R1 ≤

0≤ρ≤1

The feedback capacity region (for the case P1 = P2 = σ 2 = 1) is shown in Figure 3.26. Also shown for reference is the non-feedback region. In the case R2

R1 Fig. 3.26. Capacity region for the two-user GMAC channel with feedback.

of non-white noise, the capacity region is upper-bounded by the non-feedback region, scaled by a factor of two [92].

3.8 Asynchronous Channels In the preceding discussion of the capacity region, two types of synchronism were assumed. The ﬁrst was symbol synchronism - the users strictly align their symbol epochs to common time boundaries. The second type of synchronism was frame synchronism, in which the users align their codewords to common time boundaries. We shall see that loss of either type of synchronism changes the region of achievable rates.

3.8 Asynchronous Channels

91

Frame-Asynchronism First, we shall consider the case in which symbol synchronism is maintained, but frame synchronism is not. It has been shown [100] and [63] that the capacity region for this channel is given by the following theorem. Theorem 3.16. The capacity region of the symbol synchronous, frame asynchronous memoryless multiple-access channel (XK ; p(y | xK ) ; Y) is the closure of / R [π (xK ) , p(y | xK )] , C [p(y | xK )] = π(xK )

where R [·] is deﬁned in (3.1) and π (xK ) is the family of product distributions on the sources. In other words, the loss of frame synchronism removes only those extra rate points included by the closure of the convex hull operation which are not already included in the union of achievable rate regions. Intuitively, we can see that if frame synchronism is lost, the users cannot coordinate their transmissions to time-share between two achievable points, and thus we cannot apply the convex hull operation. The proof of Theorem 3.16 however still requires that the remaining “union” points are shown to be achievable without frame synchronism. For many channels, the union region is already convex, for example the binary adder channel and the Gaussian multiple-access channel. For such channels, the removal of frame synchronism has no eﬀect upon the capacity region. We shall see however that there are channels where this is not true. An interesting question is: Does the loss of frame-synchronization destroy our single-user interpretation of the capacity region, which relied on a globally known time reference, as discussed in Section 3.6? The answer is no, but we must slightly change our single-user strategy [57]. Complete Asynchronism In modeling the completely asynchronous case, a new concept must be introduced. Whereas the synchronous and frame-asynchronous channels were discrete time, the completely asynchronous channel must be modeled as a continuous time multiple-access channel8 . The general form of the capacity region for the completely asynchronous channel is at present unknown. We now present two examples, for which the capacity region is known, namely the completely asynchronous Gaussian MAC [148] and the collision channel without feedback [84] 8

It is tempting to try to model the channel as discrete time, with smaller time increments, but this is really the same as the symbol synchronous case.

92

3 Multiuser Information Theory

Example 3.16 (Completely Asynchronous Gaussian MAC). The completely asynchronous two-user Gaussian multiple-access channel has been studied by Verd´ u [148]. The model diﬀers a little from the synchronous GMAC model, since we now have a continuous time waveform channel. Each user k transmits a length n codeword (xk [1], xk [2], . . . , xk [n]) ∈ Rnk , at a rate of one codeword symbol every T seconds, by sending the linear modulation of a ﬁxed “signature” waveform sk (t). The signature waveforms are zero outside the interval [0, T ]. The channel output is therefore represented as n n

y(t) = x1 [i]s1 (t − iT − τ1 ) + x2 [i]s2 (t − iT − τ2 ) + z(t), i=1

i=1

where z(t) is white Gaussian noise with power spectral density σ 2 , and the delays τ1 , τ2 ∈ [0, T ) introduce the symbol asynchronism. It is necessary to assume that the receiver knows these delays, but they are unknown to the transmitters. We must also apply an energy constraint, 1 2 x [i] ≤ wk . n i=1 k n

The capacity region is found by considering a “equivalent” discrete time channel, which has the same capacity as the continuous time one. This can be done if the outputs of the discrete time channel are suﬃcient statistics for the original channel. It can be shown that the output of ﬁlters matched to s1 (t) and s2 (t), sampled at iT + τ1 and iT + τ2 respectively are indeed such suﬃcient statistics. The capacity region is given by C=

[

(

(R1 , R2 ) : „ « S1 (ω) log 1 + dω σ2 −π „ « Z π S2 (ω) 1 log 1 + dω R2 ≤ 4π −π σ2 „ Z π S1 (ω) + S2 (ω) S1 (ω)S2 (ω) 1 inf log 1 + + · R1 + R2 ≤ ρ12 ,ρ21 ∈Γ 4π −π σ2 σ4 « ) ˜ ˆ 1 − ρ212 − ρ221 − 2ρ12 ρ21 cos ω dω , R1 ≤

1 4π

Z

π

where inputs to the channel are stationary Gaussian processes with power spectral densities S1 (ω) and S2 (ω), with Sk (ω) ≥ 0, ω ∈ [−π, π] and the union is over all processes conforming to the energy constraint π 1 Sk (ω) dω = wk 2π −π The ρij terms are the cross correlations between the signature waveforms (assuming τ1 ≤ τ2 ):

3.8 Asynchronous Channels

T

ρ12 =

93

s1 (t)s2 (t + τ1 − τ2 ) dt.

0

ρ21 =

T

s1 (t)s2 (t + T + τ1 − τ2 ) dt.

0

The inﬁmum in the last constraint is over Γ = {(ρ12 , ρ21 )}, which is the set of possible cross correlations, given the signature waveforms. This represents the fact that the users do not know their relative time oﬀsets, and hence do not know their cross correlations. The capacity region therefore depends upon the cross-correlation between the users’ signature waveforms. It is interesting to see what happens if the users are assigned the same waveform. In this case, Γ contains (0, 1) and 1 − ρ212 − ρ221 − 2ρ12 ρ21 cos ω = 0, which is the minimum value that it can take. It is now easy to see that the resulting capacity region is exactly the same as that of the synchronous channel of example 3.49 . If the waveforms are however diﬀerent, the resulting capacity region is a larger polytope, with rounded corners. The rounded corners are due to the fact that there is no combination of S1 (ω) and S2 (ω) which simultaneously maximizes all constraints. Figure 3.27 shows the capacity regions for an equal power asynchronous channel. Region A is for the completely correlated waveform case. Region B is for waveforms that are orthogonal when τ1 = τ2 . Note that in either case, the capacity region is still convex. Example 3.17 (Collision Channel without Feedback). The collision channel without feedback was proposed by Massey and Mathys [84]. This channel has a number of distinct features, which makes it an interesting comparison to the channels already described. In particular, it attempts to model random accessing to the channel. The channel is described as follows. User k sends a packet of ﬁxed duration T , with some probability, pk . Users’ transmissions are not synchronized in any way, and there exists no feedback path to the users, so they can never determine the degree of asynchronism, whereas the receiver can. At the receiver, a collision is said to have occurred if two or more packets overlap by any nonzero time duration. Any packet involved in such a collision is destroyed, and its information lost. In the absence of a collision, packets are assumed to be received successfully, and all the information contained is retrieved. It is shown in [84] that the capacity region, and zero-error10 capacity region coincide, and this region further coincides with the corresponding regions obtained if slot (frame) synchronism is allowed. The outer boundary of the capacity region is given by 9 10

If it is not so easy to see, consider each user as a white Gaussian process, which together with (3.23) gives Sk (ω) = wk . By zero-error it is meant that there exists a coding scheme such that P¯e = 0. See [124].

94

3 Multiuser Information Theory R2

B A

R1 Fig. 3.27. Capacity region for symbol-asynchronous two-user Gaussian multipleaccess channel.

Rk ≤ p k

K #

(1 − pj ) ,

j=1 j=k

5K where pj ≥ 0 and j=1 pj = 1. This is shown for two users in Figure 3.28. This region is not convex, and in fact the convexity of its ﬁrst orthant complement was proved by Post in [103]. The symmetric capacity (the maximum sum rate achievable with every user at the same rate) of this channel approaches 1/e for large systems, which is equal to the maximum throughput of slotted ALOHA for an inﬁnite number of identical users [13].

3.8 Asynchronous Channels R2

R1 Fig. 3.28. Capacity region for two-user collision channel without feedback.

95

4 Multiuser Detection

4.1 Introduction In this chapter we explore basic detection techniques for linear multiple-access channels. These techniques exploit the structure of the interference signals and do not assume the interference to be uncorrelated Gaussian noise as is the case in conventional correlation detection. They make use of the fact that the interference signal is information bearing. We will make a distinction between multiuser detection which deals with (uncoded) systems and essentially demodulates the transmitted signals without regard to any time correlation among the data symbols (coding), and multiuser decoding, which explicitly includes data dependencies in the form of forward error control coding (FEC). Multiuser decoding is treated in detail in Chapter 6, but these decoding methods are built on the detection techniques discussed here. Multiuser detection was ﬁrst proposed for CDMA system in [121, 142, 147]. In [147] multiuser detection was shown to be able to handle the debilitating eﬀect of diﬀerent received power levels, the so-called near-far problem. This problem occurs when the signal of a close-by user drowns the signals of users further away which are received with less power. With conventional correlation reception, CDMA signals are very susceptible to the near-far problem, see Section 4.5.1. In cellular networks [134, 135], power control assures that all the users’ received powers are kept approximately equal. Since no joint detection is attempted, the near-far problem can only be avoided by carefully monitoring and adjusting the transmission power levels of the transmitting users. The complexity of power control resides in the network. Multiuser detection, however, can alleviate or eliminate this network complexity by translating it into computational receiver complexity. Apart from the promise of higher spectral and power system eﬃciencies, it is this capability to eliminate the near-far problem which makes multiuser detection so interesting. Given the inexorable tendencies of Moore’s law, however, multiuser detection and decoding have come close to being practical and economically interesting. Avoiding

98

4 Multiuser Detection

unnecessary network complexity and locating it in the receiver instead has the potential to make future networks far more resource eﬃcient. Figure 4.1 lists some the classes of multiuser detectors (and decoders) discussed in this book loosely ordered by computational complexity and performance (not to scale). At the bottom of the diagram we ﬁnd the conventional correlation detector, which does not use explicit joint detection principles and treats the signals of the interfering users as additive uncorrelated noise. It is a most simple receiver, based on conventional point-to-point transmission techniques, and aﬀords adequate performance as long as the number of users is limited, usually to a fraction of the processing gain of the system [97, 154]. This kind of receiver is based on low-complexity terminals, but requires a sophisticated network system with provides control functions such as power control.

Uncoded Systems

Coded Systems Optimal Decoding Viterbi Algorithm APP Algorithm

Complexity

Optimal Detection Viterbi Algorithm APP Algorithm

Exponential Complexity Iterative Decoders Turbo Decoders

Polynomial Complexity

Linear Front End Decoders

Statistical Interference Cancellation Decorrelator, LMMSE, Multistage

Cancellation/ Bounded Search Tree Search Sphere Detector List Detection

Near-Far Resistant NOT Near-Far Resistant

Correlation Detection IS-95, cdma2000 3GPP

Performance

Fig. 4.1. Classiﬁcation of multiuser detection and decoding methods.

At the top of the diagram of Figure 4.1, the most complex receivers perform a maximum-likelihood (ML) estimate of the data sequences of all the users. This receiver is near-far resistant, and forms a benchmark for ideal performance. Its computational complexity makes it largely uninteresting as a practical detector. We will discuss optimal detection in detail in Section 4.2. We also note that optimal, or ML-decoding, is not necessary to approach

4.1 Introduction

99

the capacity limit of the multiuser channel, since in the proof of Shannon’s capacity theorems (Chapter 3) an ML detector does not have to be assumed. Between the extremes, we ﬁnd a number of detectors, which all share the property that they are near-far resistant, or nearly so. We have somewhat arbitrarily grouped them into three classes. The ﬁrst class are the statistical interference cancelers. The decorrelator and minimum mean-square error detectors discussed in Sections 4.4.2 and 4.4.5, are the main representatives of this class. The term statistical interference cancellation refers to the fact that these receivers do not attempt to perform actual interference signal cancellation, but use statistical properties of the received and interfering signals in order to provide improved performance. These methods work surprisingly well with complexities much lower than that for optimal detection, primarily due to the fact that for the purpose of statistical cancellation, complete receivers for the interfering users are not needed. The next class of multiuser detectors are the actual interference cancelers. Many of these detectors were originally of ad-hoc design. Typically they either decode users via some form of successive cancellation, or they are approximations to the optimum detector. Systems which successively cancel users’ signals start by power-ordering the users, then subtract the inﬂuence of stronger users from the received signal and proceed recursively to decode subsequent users. This works well as long as the detection of the diﬀerent users’ transmitted symbols is correct. In fact, assuming error-free detection at each stage, we show in Section 5.2.2 that such successive cancellation receivers can achieve the capacity of the multiple-access channel. Without accurate decoding, however, such receivers suﬀer from potentially debilitating error propagation. In Chapter 5 on implementation aspects of multiuser detectors, we show that such interference cancelers can be viewed as variants of iterative implementations of the statistical interference cancelers, making cancellation techniques the most important practical detection and decoding concept for multiuser systems. The interference cancelers approximate the optimal detector. Among other approximations to the optimal detector are limited search algorithms such as branch-and-bound tree search algorithms as well as the sphere decoder. They typically suﬀer from loosing the correct signal hypothesis, but provide excellent performance for a given computational eﬀort. More recently, principles of iterative decoding have successfully been applied to the problem of joint decoding of CDMA systems. Iterative decoding has become popular with the comet-like ascent of turbo and low-density parity-check codes [12, 81, 120] in recent years. Their application to linear multiple-access channels has lead to very eﬃcient iterative decoders. We will treat such iterative decoding systems in Chapter 6. All of the CDMA multiuser detection methods are based on the particular channel model which arises from CDMA transmission, i.e. on the canonical linear-algebraic channel from Section 2.3 r = SAd + z.

(4.1)

100

4 Multiuser Detection

Recall that S is the modulation matrix whose columns are the spreading sequences of the diﬀerent users and diﬀerent transmission time intervals. In the sequel we will set the amplitude matrix A = I in some of the derivations for convenience. We do this only where A does not play an important role and its inclusion is a straightforward exercise. Note that the linearity of (4.1) makes many of the multiuser detectors possible, and indeed feasible complexity-wise.

4.2 Optimal Detection A joint detector considers all available information, either in an optimal or an approximate way. It therefore must jointly decode all the accessing users’ data streams as shown in Figure 4.2, which is a simpliﬁed rendition of Figure 2.9. Brute-force optimal joint detection typically translates into a large complexity of the joint detector as we will show below. If the diﬀerent users employ FEC codes, the complexity of the decoder sharply increases unless cancellation or iterative decoders are used. At any rate, for full exploitation of the CDMA channel capacity the view taken in Figure 4.2 needs to be adopted. u

ˆ ˆ or d u

d

Source

Encoder / Modulator

Source

Encoder / Modulator

Joint Detector for all users

Noise Source

Encoder / Modulator

Fig. 4.2. A joint detector considers all available information.

4.2.1 Jointly Optimal Detection We will now turn our attention to the derivation of the optimal detector for the CDMA channel [121, 142, 147], which is applicable in general to channels

4.2 Optimal Detection

101

of the form (4.1). By “jointly optimal” we mean the detector which produces the maximum-likelihood estimate of the transmitted uncoded symbols d, and, ˆ = d) for the unconsequently, minimizes the (vector) error probability Pr(d coded symbols. The received vector of noise samples z is (mostly) caused by thermal receiver noise, and hence is well described by independent Gaussian noise samples with variance σ 2 = N0 /2, where N0 is the one-side noise power spectral density1 . Armed with these assumptions, we expressed the conditional probability of r given d by a multi-variate Gaussian distribution in Section 2.5.4, and derived the ML estimate as ˆ ML = arg min r − Sd2 . d 2 d∈D Kn

(4.2)

Note that in the synchronous case this minimization can be carried out independently over each symbol period, which, ironically, does not reduce the complexity of the algorithm, as we will see. We may furthermore neglect the term r∗ r in (4.2) and write ˆ ML = arg max (2d∗ y − d∗ S∗ Sd) . d d∈D Kn

(4.3)

Note that the vector y = S∗ r has dimension Kn and its j-th entry, given by (S∗ r)[j] = s∗j r, is the correlation of the received vector with the spreading sequence of the k-th users at symbol interval i, where i = j/K and k = j − (i − 1)K. (This correlation can also be computed as the output of a (k) ﬁlter matched to sj and sampled at time Ti = jT + τk . That is, S∗ r is the output of a bank of ﬁlters matched to the spreading sequences of the K users). The receiver of (4.3) is shown in Figure 4.3, reproduced from Figure 2.6 for convenience. If the spreading sequences which make up the channel matrix S are orthogonal, S∗ S = I, and the channel symbols d are uncorrelated and have constant energy, (4.3) reduces to ˆ ML = arg max (d∗ y) , d d

(4.4)

ˆ ML = sgn(y). This is the correlation receiver which which is easily solved by d performs no joint detection. In general, however, the term d∗ S∗ Sd needs to be considered. This term is the source of the complexity of the joint detector, but also of its performance advantage. The spreading sequences sj = sk [i] may be time-varying, i.e. depend not only on k but also on the index i. This is the case for time-varying CDMA which uses sequences for each of the users with periods much longer than L [135]. This is also referred to as random spreading. The other alternative is to use time-invariant CDMA, i.e. sk [i] = sk , where identical, length-L, spreading sequences are used for all symbol transmissions. There are advantages 1

For an introductory discussion of Gaussian noise see e.g. [49, 104, 166]

102

4 Multiuser Detection Matched Filter Front End

√ Baseband Modulator

Baseband Modulator

s1 [i]

P1 s1 [i]

d1 [i] √

Reset

P

y1 [i]

P

y2 [i]

s2 [i]

P2 s2 [i]

d2 [i] r[i]

√ Baseband Modulator

Optimal Detector

z[i] PK sK [i]

sK [i] Noise

P

dK [i]

yK [i]

Fig. 4.3. Matched ﬁlter bank serving as a front-end for an optimal multiuser CDMA detector.

and disadvantages to both systems, however, random CDMA is becoming the more popular variant in practical applications [134, 135]. The system model of Figure 4.3 and equation (4.3) is equally applicable to both alternatives as well as synchronous and asynchronous transmission, as discussed in Chapter 2. Since optimal detection based on y is possible, the outputs of the correlation detector provide what is known as a suﬃcient statistic (see Deﬁnition A.5), and, while making hard decisions on y is suboptimal in general, the bank of correlators or matched ﬁlters serves as a front-end of the optimal detector in Figure 4.3. No information is lost due to the linear correlation operations. We now show that optimal detection can be performed by a ﬁnite-state machine with 2K − 1 states [147], more speciﬁcally by a trellis decoder [120]. The observation is based on realizing that the term d∗ S∗ Sd in (4.3) has banddiagonal form with K − 1 oﬀ-diagonal terms and can be recursively evaluated by formulating the problem as a ﬁnite-state system that evolves through the row index. The algorithm is identical to the trellis decoding algorithms discussed in [120, Chapter 6], with only minor diﬀerences in how the branch metrics are generated. The operation that causes the computational complexity of the optimal detector is the evaluation of the quadratic form d∗ S∗ Sd. The correlation matrix R = S∗ S of the spreading sequences has dimensions Kn × Kn and its n, m-th entry is given by (Section 2.5.2) Rlm = s∗l sm = s∗k [i]sk [i ].

(4.5)

4.2 Optimal Detection

103

Recall that the subscripts k and k refer to users, and the arguments i and i to time intervals. Note that the indexing l, m of the spreading sequences is in the order in which they appear in S, since the distinction between users and time units is irrelevant to the optimal detector, but technically sl = sk [l/K ]; k = l mod K. Figure 4.4 shows the structure of the cross-correlation matrix R for three asynchronous users. The fact that R is band-diagonal with width 2K − 1 allows us to evaluate d∗ Rd with a trellis decoder with 2K−1 states as follows. We must evaluate (4.3) for every possible sequence d, that is, we need to calculate a sequence metric λ(d) = 2d∗ y − d∗ S∗ Sd = 2

Kn

d i yi −

i=1

Kn

Kn

di Rij dj

(4.6)

i=1 j=1

and then select ˆ ML = arg max (λ(d)) . d

(4.7)

d∈D Kn

i k i'

1

2

1

2

2 3

1

2

3 3

1

2

3

k' 1

R11 R12 R13

2

R12 R22 R23

R24

3

R13 R23 R33

R34

1

R24 R34

R44

2

R(l−2)l

3

R(l−1)l

1 3

1

R(l−2)l R(l−1)l Rll

2 3

Fig. 4.4. Illustration of the correlation matrix R for three asynchronous users.

To do this, let us deﬁne the partial metric at time l − 1 as

104

4 Multiuser Detection

λl−1 (d) = 2

l−1

d i yi −

i=1

l−1

l−1

di Rij dj ,

(4.8)

i=1 j=1

and write the entire sequence metric in recursive form as λl (d) = λl−1 (d) + 2dl yl −

l−1

7 72 2di Ril dl − Rll 7dl 7 ,

(4.9)

i=1

where we have used the crucial fact that R is symmetric. The key observation, ﬁrst made by Ungerb¨ ock in [140] in the context of inter-symbol interference channel equalization, is to note that λl (d) depends only on di , i ≤ l, and not on “future” symbols di , i > l – see Figure 4.4. Furthermore, since we7 are 7 assuming binary modulation, dk ∈ {+1, −1} we may neglect all terms 7dl 72 in the metrics since their contributions are identical for all metrics. We now may modify our partial metrics to t0

term 1

l−1

λl (d) = λl−1 (d) + 2dl yl − 2di Ril dl

(4.10)

i=1

= λl−1 (d) + bl ,

(4.11)

where bl , implicitly deﬁned above, plays the role of the branch metric in a trellis decoder [120]. Figure 4.5 illustrates the recursive nature of the computation of λl (d). Ateach time index l, the previous metric λl−1 (d) is updated by two terms, t0 and term 1. The ﬁrst term t0 depends only on the received signal at time l, i.e., yl , and the data symbol at time l. Term 1 depends on K −1 previous data symbols di ; i = l − K, · · · , l, which will need to be stored. The incremental terms in (4.11) are represented by the wedge-like slice highlighted in Figure 4.5. At the next time interval, a new such slice is added, until the entire quadratic form is computed. Note that it makes no diﬀerence if the system is synchronous or asynchronous. In a synchronous system, the triangular components are simply zero, as discussed in Section 2.5.2, however, at its widest point in the matrix, there are still K − 1 symbols which need to be stored to compute the increment. From Figure 4.5 it is evident that the branch metric bl can be calculated requiring only the K − 1 most recent values (dl−1 , . . . , dl−K ) and the present value dl from the symbol vector d. We therefore deﬁne a decoder state sl−1 = (dl−1 , . . . , dl−K ) and note that there are a total of 2K−1 such states. The sequence metrics λl (d) can now be calculated for all d by a trellis decoder as illustrated in Figure 4.6 for three users, where each state has two branches leaving, corresponding to the two possible values of dl , and two branches entering, corresponding to the two possible values of dl−K .

105

term 1

time l

4.2 Optimal Detection

time l

term 1

t0

Fig. 4.5. Illustration of the recursive computation of the quadratic form in (4.11).

q1 State at l: (dl , dl−1 , dl−2 ) s

q2

Fig. 4.6. Illustration of a section of the CDMA trellis used by the optimal decoder, shown for three interfering users, i.e. K = 3, causing 8 states. Illustrated is the merger at state s, where each of the path arrives with the metric (4.12).

106

4 Multiuser Detection

Since we will be working with state metrics we rewrite (4.11) as λl (sl ) = λl−1 (ql−1 ) + bl (ql−1 → sl ),

(4.12)

where sl ranges over all possible 2K−1 states at time l, and ql−1 is a state one time unit earlier in the trellis for which a connection to sl exists. There are two such states, since there are two paths merging at each state sl . Of these two paths we may eliminate the partial path with the lesser partial metric without compromising (4.7). The resulting optimal detection algorithm has long been known in the error control community as the “Viterbi” algorithm [45, 77, 120, 152]. The ML algorithm, or Viterbi algorithm which computes the complete sequence metrics, for optimal decoding of correlated signal sets is described in Algorithm 4.1. Algorithm 4.1 (Viterbi decoder for optimal detection). Step 1: Initialize each of the S = 2K−1 states s of the detector ˆ (s) = (). Initialize with a metric m0 (s) = −∞ and survivor d the starting state of the encoder, state s = (0, · · · , 0), with the metric m0 (0) = 0. Let l = 1. Step 2: Calculate the branch metric b l = d l yl −

K−1

dl−j R(l−j)l dl ,

(4.13)

j=1

for each branch stemming from each state s at time l − 1 for each extension dl . Step 3: (Add-Compare-Select) For each state s at time l form the sum ml−1 (q) + bl for both previous states q which connect to l, and select the larger to become the new state metric, i.e. ml (s) = maxq (ml−1 (q) + bl ). Retain the path with the largest metric as survivor for state s, and append the symbol dl ˆ (s) = (d ˆ (q) , dˆl ). on the surviving branch to the state survivor d Step 4: If l < Kn, let l = l + 1 and go to Step 2. ˆ (s ) = (d1 , . . . , dKn )(s ) correStep 5: Output the survivor d m m sponding to the state sm = arg maxs (mKn (s)) which maximizes mKn (sm ) as the maximum-likelihood estimate of the transmitted sequence. The proof in Step 3 above that the merging partial path with the lesser metric can be eliminated without discarding the maximum likelihood solution is standard and can be found, e.g. in [77, 120]. This fact is referred to as the Theorem of Irrelevance.

4.2 Optimal Detection

107

In an asynchronous system with large frame length n it is not necessary to wait until l = Kn before the decisions on the decoded sequence can be made. We may modify the algorithm and obtain a ﬁxed-delay decoder by adding Step 4b and changing Step 5 above as follows: Step 4b: If l ≥ nt , where nt is a delay taken large enough, typically around 5K, called the truncation length, output dl−nt as the estimated symbol at ˆ (s) = (d1 , . . . , dl )(s) with the largest partial time l − nt from the survivor d metric ml+1 (s). If l < Kn, let l = l + 1 and go to Step 2. Step 5: Output the remaining estimated symbols dl ; Kn − nt < l ≤ Kn from ˆ (s ) , sm = arg maxs (mKn (s)). the survivor d m 4.2.2 Individually Optimal Detection: APP Detection While we derived an optimal decoder for the entire set of transmitted symbols, there is no guarantee that the output symbols dˆj in (4.3) are optimal for any given user k. They may even be poor estimates for some of the users. Individually optimal detection calculates dˆk [i] = arg max Pr(dk [i]|y) d∈D Kn

(4.14)

as the marginalization dˆk [i] = arg max

d∈D Kn

Pr(y|d)Pr(d)

(4.15)

d:dk [i]=d

as shown in Section 2.5.5. The a priori probabilities Pr(d) are identically and uniformly distributed, meaning that no prior information about d is available, unless iterative decoding systems make repeated use of (4.15), in which case Pr(d) is computed by external soft-decision decoders, as is the practice with turbo decoding systems [120]. The marginalization sum in (4.15) grows exponentially with the number of users. It can, however, be calculated relatively eﬃciently by a bi-directional trellis search algorithm, the BCJR or APP algorithm [9, 120], which operates in the same trellis as the Viterbi algorithm discussed. The state space complexity is still exponential in K − 1, and exact calculation of (4.15), however, is rarely an option. The purpose of the bi-directional a posteriori probability (APP) algorithm is the calculation of (4.14), by carrying out the marginalization (4.15) in an eﬃcient manner using the trellis description of the correlation between symbols. To accomplish this, the algorithm ﬁrst calculates the probability that the trellis model of the CDMA system traversed a speciﬁc transition, i.e. the algorithm computes Pr[sl−1 = q, sl = s|y], where sl is the state at time l, and sl−1 is the state at time l − 1. The algorithm computes this probability as the product of the three terms, i.e.

108

4 Multiuser Detection

Pr[sl−1 = q, sl = s|y] =

1 Pr[sl−1 = q, sl = s, y] Pr(y)

∝ αl−1 (q)γl (s, q)βl (s).

(4.16)

The α-values are internal variables of the algorithm and are computed by a forward recursion through the CDMA trellis

αl−2 (p)γl−1 (q, p). (4.17) αl−1 (q) = states p

This forward recursion evaluates α-values at time l − 1 from previously calculated α-values at time l − 2, and the sum is over all states p at time l − 2 that connect with state q at time l − 1. The α values are initiated as α0 (0) = 1, α0 (s = 0) = 0. This enforces the boundary condition that the encoder starts in state s = (0, · · · , 0). The β-values are calculated by an analogous procedure, called the backward recursion

βl (s) = βl+1 (t)γl+1 (t, s) (4.18) states t

which is initialized as βKn (0) = 1, βKn (s = 0)) = 0 to enforce the terminating condition of the trellis representing the CDMA system. The sum is over all states t at time l + 1 to which state s at time l connects. The forward and backward recursions are illustrated in Figure 4.7.

time l Backward Recursion ←− Forward Recursion −→

p2 s

q p1

t2 t1

Fig. 4.7. Illustration of the forward and backward recursion of the APP algorithm for individually optimal detection.

4.2 Optimal Detection

109

The γ values are conditional transition probabilities, and are the inputs to the algorithm. In order to compute the γl (s, q) values, recall that the algorithm needs to compute (4.15), where we now focus on the sequence d that carries the trellis path through the states q and s as illustrated in Figure 4.7. The probability Pr(d) breaks into a product of individual probabilities, and the one aﬀecting the transition in question is Pr[sl = q|sl+1 = s] = Pr(dl ). The term Pr(dl |y) ∝ exp (2d∗ y − d∗ S∗ Sd) (4.19) can be broken into three terms for the path through (q, s), and decomposed using the partial sequence metric formulation (4.12) from the preceding section into ⎛ ⎞ Kn

bj ⎠ (4.20) Pr(sl−1 = q, sl = s|y) ∝ exp (λl−1 (q)) exp (bl (q → s)) exp ⎝ j=l+1 αl−1 (q) γl (s,q) βl (s)

which are the three factors from (4.16). From (4.20) we see that γl (s, q) = exp (bl (q → s)) Pr(dl ) ⎛ = exp ⎝dl yl −

K−1

(4.21) ⎞

dl−j R(l−j)l dl ⎠ Pr(dl ).

(4.22)

j=1

This factor Pr(dl ) can be used to account for a priori probability information on the user data d in iterative systems. The a posteriori symbol probabilities Pr(dk [i] = d|y) can now be calculated from the a posteriori transition probabilities (4.16) or (4.20) by summing over all transitions corresponding to dl = 1, and, separately, by summing over all transitions corresponding to dl = −1 (with dl = dk [i]). A formal description of this algorithm can be found in [120], where a rigorous derivation is presented. This derivation was ﬁrst given by Bahl et. al. [9]. 4.2.3 Performance Bounds – The Minimum Distance Determining the performance of a general trellis decoder analytically is a known diﬃcult problem, and one is often satisﬁed with ﬁnding bounds or approximations. The minimum squared Euclidean distance between two possible sequences (signal points) can serve as a measure of performance for an optimal detector. It is deﬁned as d2min =

min

d ,d∈D Kn d=d

S(d − d )2 . 2

(4.23)

110

4 Multiuser Detection

A coarse approximation of the error performance for small noise values can be obtained as (see [104, Page 256]) ⎛8 ⎞ 2 dmin ⎠ , (4.24) Pe ≈ Admin Q ⎝ 2N0 where Q

d2min /(2N0 ) is simply the probability of error between two

equally likely signals with squared Euclidean distance (energy) d2min in additive white Gaussian noise of complex variance N0 , and Admin is the number of such minimum distance neighbors, which is known as the multiplicity of the minimum distance signal pair. Unfortunately, the calculation of d2min is computationally intensive via (4.23) except for small values of K, since the search space grows exponentially with K. In fact, the calculation of (4.23) is known to be NP-complete [149]. However, this does not necessarily mean that the calculation of d2min is impossible [118]. We need to calculate d2min =

∗

min

d ,d∈D Kn d=d

(d − d ) AS∗ SA(d − d )

(4.25)

where we have re-introduced the matrix A of amplitudes. If S is not rank deﬁcient, S∗ S is positive deﬁnite, and there exists a unique lower-triangular2 , non-singular matrix F of dimension Kn × Kn, such that S∗ S = F∗ F. This is known as the Cholesky decomposition [55]. Using this decomposition and (4.25) we obtain d2min =

min

FA(d − d )2 . 2

d ,d∈D Kn d=d

(4.26)

Now let εj = dj − dj for j = 1, . . . , Kn. Then equation (4.26) can be rewritten as the sum of squares given by d2min =

min

Kn

d ,d∈D Kn l=1 d=d

δl2 ,

(4.27)

where, due to the lower triangular nature of F, 2

Sometimes the Cholesky factorization 3 yields an upper-triangular matrix F, but 2 1 6 1 7 7 is a permutation matrix, is lower-triangular the matrix PF, where P = 6 5 4 · 1 and also complies with the decomposition, i.e. R = (PF)∗ PF = F∗ F.

4.2 Optimal Detection

⎛ δl2 = ⎝

l

111

⎞2 Pj Flj εj ⎠ .

(4.28)

j=1

Since δl2 depends only on ε1 , . . . , εl , we can use a bounded tree search to evaluate the minimum value of (4.27). This branch-and-bound algorithm starts at a root node and branches into two new nodes at each level. The nodes at level l are labeled with the symbol diﬀerences (ε1 , . . . , εl ) leading to them, 5l and the node weight is Δ2l = j=1 δl2 . The branch connecting the two nodes 2 (ε1 , . . . , εl ) and (ε1 , . . . , εl+1 ) is labeled by δl+1 . The key observation now is that only a small part of this tree needs to be explored. Due to the fact that δl2 is positive, the node weights can only increase, and most nodes can quickly be discarded from future consideration if their weight exceeds some threshold. This threshold can initially be chosen to be an estimate of the minimum distance. For instance, from the single-user bound, achieved by orthogonal spreading sequences, we know that d2min ≥ 2 min(Pj ). If we are interested in the minimum distance of a speciﬁc symbol k, we modify (4.27) to Kn

d2min,k = min δl2 . (4.29) d ,d l=1 dj =dk

Algorithm 4.2, illustrated in Figure 4.8, ﬁnds the minimum distance of user k. The algorithm is an adaptation of the T -algorithm [120] to this search problem. This basic concept of a tree search, combined with branch and bounding methods, forms the basic for a large number of decoding algorithms, such as the sphere decoders, to be discussed in Chapter 5. Algorithm 4.2 (Finding d2min of a CDMA Signal Set). Step 1: Initialize l = 1 and activate the root node, denoted by (), at level 0 of the search tree with Δ20 = 0. Step 2: Compute the node weight Δ2l = Δ2l−1 + δl2 for all extensions from active nodes at level l − 1. Step 3: Deactivate nodes whose weight exceeds the preset threshold. Step 4: Let l = l + 1, and if l < Kn go to Step 2, otherwise stop min = Δ2Kn . and output d2min = active nodes

Note that since nodes whose weight increases above the threshold are dropped, the number of branches searched by this algorithm is signiﬁcantly smaller than the total number of tree branches of the full tree (3Kn ).

112

4 Multiuser Detection δ12 δ22 depth of tree

δ32 δ42 δ52 δ62 width of tree

Fig. 4.8. Bounded tree search.

However, the problem (4.28) is still NP-complete, as shown in [118]. The key lies in the fact that the above algorithm ﬁnds d2min very eﬃciently in most cases. In fact, Schlegel and Lei [118] performed searches for a synchronous (n = 1) CDMA system with length-31 random spreading sequences, and found the following empirical distribution for the minimum distances, shown in Figure 4.9. The ﬁgure also shows the width of the search tree at each depth for the worst cases found for both search experiments. The maximum width of the active tree of 48,479 for 31 users is signiﬁcantly less than the total number of possible tree branches, which is 331 = 6.2 × 1014 .

4.3 Sub-Exponential Complexity Signature Sequences While optimal detection is, in general, NP-hard, meaning that an exact optimal detector, whether jointly or individually optimal, must expend a computational eﬀort which is exponential in the number of users, we have seen that there exist clever search algorithms which obtain “near-optimal” results with much reduced complexity. It is diﬃcult to precisely gauge the complexity for such algorithms as a function of the number of users K, and many diﬀerent claims can be found in the literature. However, there exist speciﬁc signature sets which have a provably lower complexity, no matter how large the set. For example, if the cross-correlation of the signature sets are all non-positive, then there exist an optimal detection algorithm with a complexity of order O(K 3 ) [113]. Another set for which sub-exponential optimal detection is possible, will be presented here. We assume that the cross-correlation values are all equal (or

50

10

5

40

10

4

Width of search tree

Relative frequency of signature waveforms

4.3 Sub-Exponential Complexity Signature Sequences

30

20

3

10 2

31 Users 20 Users

10

10

0

10

113

| 0.4

|

| 0.6

|

| 0.8

|

|

|

dmin

1

1

|

|

|

|

|

|

|

|

0

5

10

15

20

25

30

35

l

Fig. 4.9. Histograms of the distribution of the minimum distances of a CDMA system with length-31 random spreading sequences, for K = 31 (dashed lines), and K = 20 users (solid lines), and maximum width of the search tree.

maybe nearly so), and given by ρ. Such a class of signature sequences has been used as a benchmark [88] for general CDMA systems, and includes signature sequence sets of practical interest, such as synchronous CDMA systems using cyclically shifted m-sequences [139]. Given our assumption of equal cross-correlations, i.e. 3 1, i = j , (4.30) Rij = ρ, i = j we can rewrite (4.3) as (note we are focusing on a synchronous example with n = 1.) 2d∗ y − d∗ S∗ Sd = 2

K

i=1

=2

K

i=1

d i yi −

K

K

di dj Rij

(4.31)

i=1 j=1

d i yi − ρ

K

K

di dj − K(1 − ρ).

(4.32)

i=1 j=1

5K 5K The key observation is that i=1 j=1 di dj depends only on the number of negative (or positive) elements in d, and not their arrangement. Therefore, let N (d) be the number of elements in d which are negative, i.e. 9 : K

1 N (d) = K− (4.33) di . 2 i=1 With this deﬁnition, further deﬁne the functions

114

4 Multiuser Detection K

K

di dj = T1 (d) = ρ(2d − K)2

(4.34)

i=1 j=1

T2 (d) = −2

K

d i yi .

(4.35)

i=1

Now, since we can ignore the constant term K(1 − ρ), we see that maximizing (4.31) is equivalent to ˆ = arg min (T1 (N (d)) + T2 (d)) . d d∈D K

(4.36)

Upon inspection, the following observations about the functions T1 and T2 can be made. T1 (N (d)) is convex and possesses a unique extreme point at N (d) = K/2, which is a minimum for ρ > 0, and a maximum for ρ < 0. The second term, T2 (d), is minimized by d = sgn(y), and although it depends on the arrangement of the negative elements in d, we may form a lower bound that depends only on the number of negative terms that appear. For N (d) = 0, 1, · · · , K deﬁne ⎧ K

⎪ ⎪ ⎪ yi sgn(yi ), N (d) = η −2 ⎪ ⎪ ⎪ ⎪ i=1 ⎪ ⎪ ⎪ N (d) ⎨

|yπ(i) |, N (d) > η σ(η) + 4 (4.37) σ(N (d)) = ⎪ ⎪ i=η+1 ⎪ ⎪ η ⎪

⎪ ⎪ ⎪ ⎪ σ(η) + 4 |yπ(i) |, N (d) < η ⎪ ⎩ i=N (d)+1

where we have introduced the notational simpliﬁcation η = N (sgn(yi )), and π is a permutation among the yi , such that yπ(1) ≤ yπ(2) , · · · ≤ yπ(K) . Now, we conclude that σ(N (d)) is convex with a minimum at η, and it is furthermore clear that T2 (d) ≥ σ(N (d)). (4.38) Equality is achieved in (4.38) if the elements of d are arranged such that dπ(i) = −1for i −1, 2, · · · , N (d), and the remaining K − η elements are +1.

4.4 Signal Layering

Algorithm 4.3 (Optimal Correlations).

Decoding

for

Equal

115

Cross-

Step 1: Let η = N (sgn(y)) Step 2: Let π be the permutation of the indices 1, 2, · · · , K such that yπ(i) is non-decreasing with i. Step 3: Calculate the functions T1 and σ according to (4.34) and (4.37). Also, ﬁnd j = arg

min

m=1,··· ,K

(T1 (m) + σ(m)) .

(4.39)

j is the number of negative elements in the minimizing vector ˆ d. ˆ given by Step 4: Output the vector d −1, i = 1, · · · , j ˆ dπ(i) = (4.40) +1, i = j + 1, · · · , K. The optimality of Algorithm 4.3 is a direct consequence of minimizing the lower bound (4.38) by the permutation π, and the fact that the output vector meets this bound with equality. The complexity of this detection algorithm is dominated by the operation in Step 2, which, in the worst case is of order O(K log K). However, in many cases a full sort and search will not be required. Schlegel and Grant [117] generalize this decoding algorithm to the case where blocks of signature sequences have a ﬁxed cross-correlation, and show that the algorithm is exponential only in the number of unique cross correlation values. We have seen that in many cases optimal detection algorithms with subexponential complexity can be found, or that approximate algorithms can obtain solutions with error probabilities close to those of optimal detectors. Nonetheless, a general practical solution to optimal decoding of CDMA is both elusive, and not necessary, as we will see in subsequent sections and chapters.

4.4 Signal Layering Preprocessing of the received signal r has two functions. A preprocessor can act as a multiuser detector and suppress multiple access interference as is the case in the linear ﬁlter receivers we discuss in subsequent sections. A preprocessor, however, also acts by conditioning the channel of a single user to generate an “improved” channel for that user. This viewpoint is illustrated

116

4 Multiuser Detection

in Figure 4.10, where a linear preprocessor is used to create a single-user channel for a given user.

Source

Encoder / Modulator

Source

Encoder / Modulator

Multiple Access Channel

Single User Channel

Linear Preprocessing

Decoder

Noise Source

Encoder / Modulator

Fig. 4.10. Linear preprocessing used to condition the channel for a given user (shaded).

This single-user channel has an information theoretic capacity which we will calculate for a number of popular preprocessors. From the results of conventional error control coding it is well known that this single-user capacity can be approached closely by powerful forward error control coding systems [120], that is, single-user capacity achieving codes can be used to approach the “layered capacity”. Calculation of these layered capacities allows us therefore to determine how much capacity is lost by converting the multiple-access channel into parallel single-user channels. This process shall be called channel layering. It can be very eﬀective in reducing the complexity of the joint decoding problem, and, as we will see, in many cases can achieve a performance which rivals or equals that of a full joint detector. Figure 4.11 shows the layered information theoretic capacities for several cases as a function of the signal-to-noise ratio Eb /N0 . More precisely, the curves are Shannon bounds which relate spectral eﬃciency in bits/Hz (y-axis) to power eﬃciency (x-axis). The Shannon bound for an AWGN channel is quickly computed from the capacity formula, i.e. from CAWGN = 1/2 log2 (1 + 2P/N0 ), we obtain 22CAWGN − 1 Eb ≥ , (4.41) N0 2CAWGN by realizing that at the maximum rate Eb CAWGN = P . Equation (4.41) is the well-known Shannon bound for the AWGN channel.

4.4 Signal Layering

117

The Shannon bounds calculated in Figure 4.11 are for random CDMA systems. The CDMA channel capacity was calculated in Chapter 3, where we showed that the loss w.r.t. the AWGN capacity is minimal and vanishing for large loads β → ∞, or large P/N0 → ∞. The two ﬁlters used are the minimumsquare error (MMSE) ﬁlter discussed in Section 4.4.5, which produces the best linear layered channel, and matched ﬁltering, which represents conventional correlation detection. It is important to note that the capacity bounds are calculated for equal received powers of all the diﬀerent users.

Capacity bits/dimension

10

city Capa N G AW =2 A: β CDM = 0.5 A: β M = 0.5 D C E: β MMS

1

MMSE: β = 2 Matched Filter: β = 2 Matched Filter: β = 0.5

0.1

-2

0

2

4

6

8

10

12

14

Eb /N0 16 [dB]

Fig. 4.11. Information theoretic capacities of various preprocessing ﬁlters.

The ﬁgure3 distinguishes between two system loads, deﬁned as β = K/L, i.e. the ratio of (active) users to available dimensions, also known as the processing gain. For lower loads, β < 0.5, the layered capacities of linear ﬁlters with equal received power are virtually equivalent to the optimal system capacity, and very little is lost by layering. In contrast, for large loads β > 1, 3

The range of Eb /N0 is chosen to represent typical values in wireless communications systems. Note that Eb /N0 values in excess of 16dB are rare, since they would imply constellations with 256 points per complex symbol, or larger.

118

4 Multiuser Detection

layering via linear ﬁlters becomes ineﬃcient, and signiﬁcant extra capacity can be gained by going to more complex joint receivers. Some interesting conclusions can be drawn from these results. First, if the signal-to-noise ratio (SNR) is small, the system is mostly ﬁghting additive noise, and multiuser detection has only a limited eﬀect on capacity and its complexity may not be warranted. Also, in the low SNR regime, a loss of about 1dB is incurred by using random spreading codes versus Welsh-bound equivalent codes. Furthermore, it becomes evident that not only are linear ﬁlters not capable of extracting the channel’s capacity at higher loads, but their performance actually degrades below the capacity of a lighter loaded system. Both of these eﬀects are due to the fact that the linear ﬁlter is suppressing signal space dimensions in the absence of more detailed information about the transmitted signal. This will be explored later in this chapter, as well as the inﬂuence of unequal powers on the system capacity. 4.4.1 Correlation Detection – Matched Filtering We have encountered the correlation or matched ﬁlter bank as a front-end of the optimal detector in Section 4.2. If we make simple sign decisions on the received matched ﬁlter output vector y, we have the conventional correlation receiver. Interference in the correlation receiver is only suppressed by the processing gain of the spreading sequences used. This suppression is not ideal, unless the sequences are orthogonal, and, on average, each user in the system contributes an amount of interference which is approximately equal to its power divided by the processing gain [154]. More precisely, the detection process is given by ˆ = sgn (y) . d

(4.42)

For a given user k this means ⎛ dˆk = sign ⎝dk +

⎞ dj Rkj + zk ⎠ .

(4.43)

j=k

Since Rkj is the product of two spreading sequences according to (4.5), its variance is straightforward to calculate if we assume that the chips of these spreading sequences are chosen randomly and with equal probability to be √ ±1/ L. The interference term in (4.43) is then the sum of a (large) number of random variables with bounded variance, which allows us to apply the central limit theorem and replace

dj Rkj + zk (4.44) Ik = j=k

by a Gaussian noise source with zero-mean and variance

4.4 Signal Layering

var(Ik ) =

K −1 P + σ2 L

K,L→∞

−→

βP + σ 2 .

119

(4.45)

The channel for a single user now simply looks like an AWGN channel with noise variance (4.45), and therefore with capacity [166] P 1 CMF = log2 1 + bits/use. (4.46) 2 βP + σ 2 The capacity per dimension of the overall system is K times this, divided by the processing gain, i.e. β P bits/dimension. (4.47) CMF = log2 1 + 2 βP + σ 2 Substituting the limiting rate CMF P/β = Eb into the equation above gives the Shannon bound curve shown in Figure 4.11. The deleterious eﬀect of unequal received power can also easily be seen from (4.43). A user whose signal arrives at the receiver κ times stronger, will show up as κ virtual users. This can quickly lead to a signiﬁcant loss in system capacity. 4.4.2 Decorrelation Decorrelation has intuitive appeal since it is a linear ﬁltering operation which completely eliminates the multiple-access interference, and one is tempted to conclude that its performance should therefore be close to optimal. We will however see that this is not the case, since in the process of eliminating the multiple access interference signiﬁcant and detrimental enhancement of the additive channel noise can occur. This limits what is theoretically possible by decorrelation, and this limitation is especially severe for large system loads. We start by considering the output signals of the matched ﬁlter outputs, which were calculated as y = S∗ r = S∗ SAd + S∗ z.

(4.48)

If the spreading sequences in S are linearly independent, the correlation matrix R = S∗ S has an inverse, which can be applied to y to obtain ˆ = R−1 y d = Ad + R−1 S∗ z.

(4.49)

ˆ is an interference-free estimate of d. The matrix From (4.49) we see that d H† = R−1 S∗ is the pseudo-inverse of S [55, 131] and we deﬁne the decorrelator as

120

4 Multiuser Detection

Deﬁnition 4.1 (Decorrelator). The decorrelator detector outputs dDEC = sgn R−1 S∗ r

The decorrelating detector, or the decorrelator for short, has been studied in the literature in [65, 79, 80]. While the pseudo-inverse H† always exists, a complete separation of the interfering users is only possible whenever R−1 exists, which is the case whenever S has full column rank. That is, all the users employ linearly independent spreading sequences. This assumption is reasonable, since if the signals of two or more users form a linearly dependent set, they become inseparable by linear operation, and the detection of the users’ data sequences is much complicated. In fact, sets of linearly dependent users would remain correlated after decorrelation and joint detection would have to be applied to these subsets. An interesting derivation of the decorrelator proceeds as follows. Let 2 dDEC = sgn arg min r − SAd , (4.50) d

where d ∈ RKn , in contrast to (4.2) where d was restricted to be from the set {+1, −1}Kn . This simpliﬁcation in the search space turns an NP-complete discrete minimization problem into a continuous quadratic minimization problem. We deﬁne v = Ad and minimize (4.50) by taking partial derivatives, i.e. ∂ 2 (r − Sv) = 2S∗ r − 2S∗ Sv = 0 ∂v ˆ = (S∗ S)−1 S∗ r, ⇒v

(4.51)

obtaining the same solution as (4.49). If the amplitudes A of the transmitˆ is the maximum ted symbols are unknown, then we have just proven that v likelihood estimate of v. 4.4.3 Error Probabilities and Geometry Let us calculate the probability of error for the decorrelator. Assume that dk = −1. For binary modulation, an error occurs if dDEC,k = 1, and is given by 7 Pr dˆDEC,k = 17dk = −1 = Pr (R−1 S∗ z)k > Pk . (4.52) The quantity ηk = (R−1 S∗ z)k is a Gaussian random variable since it is the sum of weighted Gaussian components. Its expectation and variance are easily computed as

4.4 Signal Layering

E [ηk ] = 0; 0 1 0 1 var(ηk ) = E R−1 S∗ zz∗ SR−1 kk = σ 2 R−1 kk .

121

(4.53) (4.54)

With this, the error probability is given by the standard Gaussian integral 98 : P k ˆ DEC,k = 1|dk = −1 = Q Pr d , (4.55) σ 2 [R−1 ]kk where Pk is the energy per symbol of user k. As can be seen from (4.55), the only diﬀerence between antipodal signaling in AWGN and the error probability of the decorrelator is the noise enhancement factor [R−1 ]kk , which can be shown to be always larger or equal to unity, that is, the decorrelator will always increase the noise and the bit error probability. Furthermore, the performance of user k is independent of the powers of any of the other users, since multiple-access interference is completely eliminated. The actual error rate depends very strongly on the signature sequences, but as P/σ 2 → ∞, the error probability P¯e → 0. This has been referred to as near-far resistance. Furthermore, as the minimum eigenvalue of R, λmin → 0, the noise enhancement coeﬃcient [R−1 ]kk → ∞, as can be seen by writing the inverse in terms of the spectral decomposition of R. Hence, any two spreading sequences with a high cross-correlation will degrade the performance of the decorrelator substantially. Since σ 2 [R−1 ]kk ≥ σ 2 , there is an inherent loss of performance suﬀered by decorrelation w.r.t. interference-free transmission. For example, consider a synchronous two-user system with ⎤ ⎡ 1ρ ⎥ ⎢ρ 1 ⎥ ⎢ ⎥ ⎢ .. (4.56) R=⎢ ⎥, . ⎥ ⎢ ⎣ 1 ρ⎦ ρ1 for which the noise variances can be calculated as follows: E [η ∗ η] = σ 2 R−1 , and from there σ 2 [R−1 ]kk = σ 2 /(1 − ρ2 ). Hence, a correlation of 50% (ρ = 0.5) implies a loss of 1.25dB in signal-to-noise ratio. If we do not ignore the noise correlation, the users could share information about the noise and better performance would be possible using an optimal detector. Figure 4.12 shows a geometrical perspective of the decorrelator. Let us start with the decorrelator output as the pseudo-inverse of the spreading matrix (using a synchronous system for illustration) multiplying the received chip-sampled vector ˆ DEC = R−1 S∗ r = H† r. (4.57) d The rows j of the pseudo-inverse H† are orthogonal to the spreading sequences i = j in S, as can be seen from

122

4 Multiuser Detection

⎡

⎤ h∗1 1 0 ∗ −1 ∗ 1 ⎢ .. ⎥ 0 ⎣ . ⎦ s1 , · · · , sK = (S S) S S = I.

(4.58)

h∗K

The vector vj in Figure 4.12 is the projection of sj onto hj , which is calculated as hj hj ˆ vj = (h∗j sj ) (4.59) 2 = dDEC,j 2. hj hj It must be orthogonal to all si , ∀i = j from (4.58). The length of the projection vector equals dˆDEC,j multiplied by 1/ hj . This dependence of the projected length of hj and dˆDEC,j is unimportant for symbol by symbol detection, but is very important for multiuser decoders using FEC coding. There the metrics need to be adjusted to this scaling factor (see Section 6.2.1).

span{sk }k=j

hj vj

vj sj

span{sk }⊥ k=j

Fig. 4.12. Geometry of the decorrelator.

4.4.4 The Decorrelator with Random Spreading Codes As discussed above, the actual performance of the decorrelator depends heavily on the set of spreading sequences used, and few general conclusions can be drawn. If we consider the use of random spreading sequences, however, very precise results exist. As derived in (4.57) ﬀ., the output of the decorrelator for a given user 2 k consists of a signal power component: Pk (h∗k sk ) and a noise component σ 2 s∗k sk , giving a signal-to-noise ratio of the k-th user of Pk (h∗k sk ) . σ 2 s∗k sk 2

SNRk =

We will show that for large systems the following theorem holds:

(4.60)

4.4 Signal Layering

123

Theorem 4.1 (Decorrelator Performance in Random CDMA). If K → ∞, and L → ∞, such that the load β = K/L is constant, then SNRk

K,L−→∞

−→

Pk L − K + 1 σ2 L

(4.61)

Proof. Assume that hk = 1 and let Q = [q1 , · · · , qK−1 ] be a basis for span{sk }k=j of dimension K − 1. Since sk is random, i.e. each component is randomly chosen, E [q∗l sk ] = 0 (4.62) and

1 ql Iq∗l = . (4.63) L L That is, the length of the average projection onto an arbitrary basis vector is 1/L, irrespective of whether the system is synchronous or not. The average total length of the projection onto span{sk }k=j is therefore (see Figure 4.12) ⎤ ⎡ K

K−1 K

0 ∗ 1 K −1 ⎢ ⎥

¯j = E ⎣ ¯j v (4.64) q∗l sk s∗k qm ⎦ = E [ql Iq∗l ] = E v L m=1 E [q∗l sk s∗k ql ] =

l=1 l=k m=k

l=1

¯ j∗ v ¯ j + vj∗ vj = si s∗i = 1, and, since, v 0 1 L−K +1 E vj∗ vj = . (4.65) L L→∞ It remains to show that var vj∗ vj −→ 0, which is accomplished in a similar fashion. From this theorem we can calculate the information theoretic capacity of the decorrelator layered single-user channel using random CDMA. Equation (4.61) states that the symbol energy, or the signal-to-noise ratio, of an arbitrary user k is reduced by the factor 1−β+1/L with respect to interference-free transmission. At the same time, the system has K channels in L dimensions. Therefore, the decorrelator (system) capacity is given as Eb β 1 CDEC = log 1 + 2 bits/dimension. (4.66) 1−β+ 2 N0 L We recall the Shannon bound for an AWGN channel (4.41), and, from (4.66), we can analogously derive the Shannon Bound for the decorrelator layered channel as

124

4 Multiuser Detection

22C/β − 1 Eb , ≥ N0 2C(1 − β)

(4.67)

which is shown in comparison to the AWGN Shannon bound of (4.41) in Figure 4.13 for a few diﬀerent system loads β.

Decorrelator per-user Capacity CDEC

10

AWGN Capacity β = 0.75 Decorrelator Capacities 1 β = 0.25 β = 0.5

β = 0.9 0.1

-2

0

2

4

6

8

10

12

14 Eb /N0 [dB]

Fig. 4.13. Shannon Bounds for the AWGN channel and the random CDMA decorrelator-layered channel. Compare with Figure 4.11.

As can be seen from Figure 4.13, system loads in the range of 0.5 ≤ β ≤ 0.75 are most eﬃcient. For larger loads the energy loss of the decorrelator is too big, and for smaller loads the spectrum utilization is ineﬃcient. 4.4.5 Minimum-Mean Square Error (MMSE) Filter The MMSE detector is an estimation theory concept transplanted to the ﬁeld of (multiuser) detection. The MMSE ﬁlter minimizes the variance at the output of the ﬁlter taking into account both channel noise and interference, but ignoring the data structure. As long as this structure is disregarded,

4.4 Signal Layering

125

the MMSE ﬁlter provides the best possible preprocessing. Since the residual noise is asymptotically Gaussian [101], the minimum variance ﬁlter is also the best overall minimum variance estimator, and maximizes the capacity of the resulting layered channels. The MMSE ﬁlter minimizes the squared error between the transmitted signals and the output of the ﬁlter. It is found as < ; 2 M = arg min E d − Mr . (4.68) M

Carrying out this minimization is standard [60], and makes use of the orthogonality principle: ; < ∂ 2 E d − Mr = 2 E [(d − Mr)r∗ ] = 0. (4.69) ∂M Evaluating the various terms we obtain E [dr∗ ] = M E [rr∗ ] AS∗ = M SAA∗ S∗ + σ 2 I −1 . M = AS∗ SWS∗ + σ 2 I

(4.70)

See also Section A.4.3. Using the matrix inversion lemma4 an alternate form of the MMSE ﬁlter (4.70) can be found as follows: −1 W S∗ + σ2 I S M = AS∗ B

C

D

A

−1 1 1 1 ∗ ∗ ∗ 1 −1 = 2 AS − 2 AS S S 2 S + W σ σ σ σ2 −1 ∗ 1 S = 2 A I − R R + σ 2 W−1 σ − 1 −1 = A∗ R + σ 2 W−1 S∗ .

(4.71)

Scaling the ﬁlter output by A∗ −1 does not aﬀect hard decision, hence we propose the following Deﬁnition 4.2 (Minimum Mean-Square Error Detector). The Minimum Mean-Square Error Filter (MMSE) Detector outputs −1 ∗ dMMSE = sgn R + σ 2 W−1 S r 4

Given as

−1

(A + BCD)−1 = A−1 − A−1 B(DA−1 B + C −1 ) DA−1

126

4 Multiuser Detection

The MMSE detector ignores the data structure of the interference, and takes into account only its signal structure. In this class of receivers, the MMSE is optimal, and the capacity of the layered channel is maximized. The MMSE ﬁlter application to CDMA joint detection was ﬁrst proposed in [171], and then further developed in [107] and [82]. 4.4.6 Error Performance of the MMSE The error probability of the MMSE receiver can be calculated analogously to the decorrelator. Again, assume dk = −1. An error occurs if dMMSE,k = 1 (for BPSK), and the error probability is calculated as −1 R + σ 2 W−1 y > 0 . (4.72) Pr (dMMSE,k = 1|dk = −1) = Pr k

−1 Introducing the abbreviation T = R + σ 2 W−1 , we continue as follows: Pr (dMMSE,k = 1|dk = −1) = Pr ((T (S∗ Rd + S∗ z))k > 0) = Pr ((TS∗ Rd)k +(TS∗ z)k > 0) .

(4.73)

It is straightforward to show that E [(TS∗ z)k ] = 0;

var((TS∗ z)k ) = E [TS∗ zz∗ ST] = σ 2 [TRT]kk

and that the error probability of user k is therefore given by 9 : ∗

Rd) (TS k Pr (dMMSE,k = 1|dk = −1) = 21−K . Q 2 [TRT] σ kk d

(4.74)

(4.75)

(dk =−1)

The error formula assuming dk = 1 is completely analogous. From (4.75) we can draw a few conclusions. First, the error probability depends on the powers of the interfering users, unlike for the decorrelator. The error probability also depends strongly on the spreading sequences S which are used, but, as σ 2 → 0, the error probability Pe → 0. It is very informative to express the error of the MMSE in terms of the spectral decomposition of the correlation matrix. Evaluating the mean square error of the MMSE receiver we obtain ; < −1 2 E d − Mr = tr I − A∗ S∗ SWS∗ + σ 2 I SA −1 = K − tr SWS∗ SWS∗ + σ 2 I =K−

L

i=1

σ2 λi = , λi + σ 2 λ + σ2 i=1 i K

(4.76)

where the eigenvalues λi are the K eigenvalues of SWS∗ . This particular expression will be useful in the next section.

4.4 Signal Layering

127

4.4.7 The MMSE Receiver with Random Spreading Codes As for the decorrelator, little can be said in general for any given speciﬁc set of spreading sequences. However, for the class of random spreading sequences, the results that are obtained are very precise, and in the limit of large system, these results are no longer random, but become deterministic, as the variances of the random quantities involved vanish. It is this property, and the widespread acceptance of random spreading codes as practical alternatives, which make them popular both for theory and application. First we note from (4.76) that the sum of the mean square errors of all K users equals K K

σ2 MMSEk = . (4.77) λ + σ2 i=1 i k=1

In a random system MMSEk will be equal for all k if Pk = Pk , ∀k, k , and K

1 1 1 K,L−→∞ ∞ MMSEk = −→ (4.78) fλ (λ) dλ, P λi K λ + 1 0 σ2 + 1 σ2 i=1 where fλ (λ) is the limiting eigenvalue distribution of SS∗ /K. We have come across this distribution in Chapter 3, Theorem 3.5, It was calculated by Bai and Yin [10] as [x − a(β)]+ [b(β) − 1]+ −1 fλ (x) = [1 − β ]δ(x) + 2πβx 2 2 a(β) = β − 1 ; b(β) = β + 1 ; [z]+ = max(0, z). Verd´ u and Shamai [150] have shown that there exists the following closedform formula for the mean square error of the arbitrary user k in the case of synchronous CDMA: −1 P P 1 ,β , (4.79) MMSEk = 1 + 2 − F σ 4 σ2 where the function F(x, z) is deﬁned as 2 √ 2 √ 2 x 1+ z +1− x 1− z +1 . F(x, z) =

(4.80)

The signal-to-noise ratio and the mean square error of the MMSE receiver are related by 1 MMSEk = (4.81) 1 + SNRk and, conversely, 1 SNRk = − 1. (4.82) MMSEk Combining (4.82) and (4.79) we arrive at the following theorem.

128

4 Multiuser Detection

Theorem 4.2 (MMSE Performance in Random CDMA). If K → ∞, and L → ∞, such that the load β = K/L is constant, then, for synchronous CDMA P 1 K,L−→∞ P F − , β . (4.83) SNRk −→ σ2 4 σ2

Since the residual noise of the MMSE receiver is Gaussian, the capacity of the corresponding single-user channel is given as straightforward application of the capacity formula (4.41), where the Gaussian noise SNR is replaced by (4.83), and, using P = REb in order to normalize equations, we obtain REb REb β 1 CMMSE = log 1 + 2 − F 2 ,β . (4.84) 2 N0 4 N0 Setting βR = CMMSE leads to an implicit equation for the maximum spectral eﬃciency of the MMSE receiver, which is plotted in Figure 4.14 below for a few load values. Observation of these capacity curves reveals some interesting properties of the respective ﬁlters. The MMSE receiver is most eﬃcient in the low signal-tonoise ratio regime, where thermal noise dominates the channel impairments. Here, however, the simple matched ﬁlter performs almost as well. As the signal-to-noise ratio improves, the diﬀerences between decorrelator and MMSE receivers diminish. Like the decorrelator, albeit not as severely, the eﬃciency of the MMSE receiver diminishes as the system load is increased in the high signal-to-noise ratio regime. Like the decorrelator, the MMSE receiver requires the inversion of a K × K correlation matrix at each symbol period. This represents an enormous complexity burden and is not desirable for actual systems implementations. We will present lower complexity iterative approximations for both of these preprocessors in the next chapter. 4.4.8 Whitening Filters The whitening ﬁlter approach represents an intermediate stage between ﬁltering and cancellation receivers. It is based on a matrix decomposition method, namely the Cholesky decomposition [55], which leads to this detector structure. The symmetric matrix R allows the decomposition R = F∗ F, where F∗ is lower-triangular, as already encountered in Section 4.2.3. The Cholesky decomposition can be accomplished in O(n3 /6) operations, where n is the size of the matrix [55], and has therefore a complexity which is comparable to that of inverting the matrix. Furthermore, if R is invertible, so are F and F∗ . We now pre-multiply the correlator output (4.48) by F∗ −1 and obtain

4.4 Signal Layering

129

10

Capacity bits/dimension

AWGN Capacity

β=1 MMSE Capacities 1 β=2

β = 0.5

Decorrelator

β = 0.5

0.1

-2

0

2

4

6

8

10

12

14

16

Eb /N0 [dB]

Fig. 4.14. Shannon bounds for the AWGN channel and the MMSE layered singleuser channel for random CDMA. Compare with Figures 4.11 and 4.13.

yWF = F∗ −1 y = FAd + zWF .

(4.85)

The correlation of the ﬁltered noise vector zWF is easily calculated as E [zWF z∗WF ] = F∗ −1 S∗ E [zz∗ ]SF−1 = σ 2 I,

(4.86)

that is, the noise samples zWF are white (uncorrelated) with common variance σ 2 per dimension. This property has also given the ﬁlter F∗ −1 the name noise whitening ﬁlter. In a sense F∗ −1 accomplishes the complement of the decorrelator, which whitens the data. The beneﬁt of yWF is that, since F is lower triangular, ⎤ ⎡ ⎤ ⎡ F √P d 11 1 1 yWF,1 √ √ ⎥ ⎢ yWF,2 ⎥ ⎢ F21 P1 d1 + F22 P2 d2 ⎥ ⎢ ⎥ ⎢ ⎥ + zWF ⎢ yWF = ⎢ (4.87) ⎥=⎢ .. .. ⎥ ⎣ ⎦ ⎣ . ⎦ . √ 5Kn yWF,Kn F Pd i=1

Kn,i

i i

130

4 Multiuser Detection

and a successive interference cancellation detector suggests itself which starts with dˆ1 = sgn(yWF,1 ) and proceeds to decode successively ⎛ ⎞ i−1

dˆi = sgn ⎝yWF,i − Fij Pj dˆj ⎠ , (4.88) j=1

as shown in Figure 4.15. Since F is lower triangular with width K (see Section 4.4.9) even in the asynchronous case ⎛ ⎞ K

Fi(i−j) Pi−j dˆi−j ⎠ , (4.89) dˆi = sgn ⎝yWF,i − j=1

and we never need to cancel more than K − 1 symbols. However, in contrast to the decorrelator, and as for the MMSE receiver, this decoder requires the amplitudes A of all the users in order to function properly, a complication which must be handled by the estimation part of the receiver. Since F is lower triangular, only previous decisions are required at each level, forming the input to the i-th decision device. If all previous i−1 decisions are correct, i.e. dˆj = dj , j = 1, . . . , i − 1, the input to the i-th decision device is given by i−1

Fij Pj dj = Fii Pi di + zWF,i , (4.90) yWF,i − j=1

and yWF,i is free of interference from the other users’ symbols. The discrete signal-to-noise ratio at the input of the decision device can now be calculated as F 2 Pi SNRi = ii2 . (4.91) σ Since we are starting with i = 1, it is important to rearrange the users such that they are ordered with decreasing SNRi , or equivalently with decreasing Fii2 Pi . This is to minimize error propagation, since, if one of the decisions dˆi is erroneous, equation (4.91) no longer applies and we have an instance of error propagation typical for decision-feedback systems. It is interesting to note that the ﬁlter F∗ −1 maximizes SNRi for each symbol d[i], given correct previous decisions, as originally shown by [30, 31], and restated in Theorem 4.3 (Local Optimality of the Whitening Filter). Among all causal decision feedback detectors, F∗ −1 maximizes SNRi for all i, given correct previous decisions dˆj = dj , j = 1, . . . , i − 1.

4.4 Signal Layering

131

dˆ1

sgn( )

F21 Partial Decorrelator

-

dˆ2

sgn( )

F∗−1 FK1

FK2

-

sgn( )

dˆK

Fig. 4.15. The partial decorrelating feedback detector uses a whitened matrix ﬁlter as a linear processor, followed by successive cancellation.

Proof. Assume M is any ﬁlter such My = T d + z , where T is lower triangular. We may write T = TF, where T must necessarily also be lower trian 5 ∗ i gular. But E[z z ] = σ 2 TT∗ , and SNRi = t2ii Fii2 Pk / σ 2 j=1 t2ij , which is maximized for all i by setting T = I. Using optimal error control codes at each stage before cancellation user #1 could operate at a at a maximum error free rate per dimension of R1 = 2 F11 P1 1 , and after cancellation the k-th decoder could operate at 2 log 1 + σ 2 2 Fkk P 1 Rk = 2 log 1 + σ2 k , yielding the maximum sum rate Rsum =

K

k=1

Rk =

F 2 Pk log 1 + kk2 . 2 σ

K

1 k=1

(4.92)

=K ∗ Since F is triangular det(F) = k=1 Fkk , and hence det(F WF) = =K 2 ∗ ∗ k=1 Fkk Pk . We conclude further that, since (F WF) = (S WS). We obtain from (4.92) S∗ WS 1 log det I + = Csum , (4.93) 2 σ2 which is the information theoretic capacity of the CDMA channel as derived in Chapter 3. Whitened ﬁltering with cancellation therefore theoretically suﬀers no performance loss. It is a capacity achieving serial cancellation method. Note, furthermore, that power ordering is no longer required if each level is using capacity achieving coding. We will later show in Chapter 6 that a bank

132

4 Multiuser Detection

of MMSE ﬁlters with a successive cancellation structure like that in Figure 4.15 can also achieve the capacity of the CDMA channel. Despite this, there are a number of diﬃculties associated with partial decorrelation. Not only is exact knowledge of the power levels Pk required, but the transmission rates have to be tailored according to (4.92), and that requires accurate backward communication with the transmitters, i.e. feedback. The decoding delay may also become a problem since each stage has to wait until the previous stage has completed decoding of a particular symbol before cancellation can occur. 4.4.9 Whitening Filter for the Asynchronous Channel For asynchronous CDMA and continuous transmission the frame length n may become quite large. As a consequence the ﬁlter decomposition R = F∗ F via a straightforward Cholesky decomposition becomes too complex. Since the diﬀerent symbol periods do not decouple like in the synchronous case, we need some way of calculating this decomposition eﬃciently, exploiting the fact that most entries in the asynchronous correlation matrix R are zero. We will make use of the fact that the matrix R has the special form of a band-diagonal matrix of width 2K − 1, i.e. ⎤ ⎡ R0 [0] R∗1 [1] ⎥ ⎢R1 [1] R1 [0] R∗2 [1] ⎥ ⎢ ∗ ⎥ ⎢ R2 [1] R2 [0] R3 [0] (4.94) R=⎢ ⎥. ⎥ ⎢ R2 [1] R3 [0] ⎦ ⎣ .. .. . . It can be shown then ([55], or by considering F∗ F = R) that F is lower triangular with width K, or in block form ⎤ ⎡ F0 [0] ⎥ ⎢F0 [1] F1 [0] ⎥ ⎢ ⎥ ⎢ F1 [1] F2 [0] (4.95) F=⎢ ⎥, ⎥ ⎢ F2 [1] F3 [0] ⎦ ⎣ .. . where (i) Fi [0] is lower triangular, and (ii) Fi [1] is strictly upper triangular. Expanding R = F∗ F we obtain the following set of equations for the individual blocks of F: Rn−1 [0] = F∗n−1 [0]Fn−1 [0] 0
(4.96) (4.97) (4.98)

4.4 Signal Layering

133

These equations can be calculated recursively starting with (4.96) which is a K × K Cholesky factorization. Substituting into (4.97) we ﬁnd Fn−2 [1] = −1 ∗ Rn−1 [1], and, Fn−2 [0] can be found by a Cholesky factorization of Fn−1 [0] Rn−2 [0] − F∗n−1 [1]Fn−1 [0]. At this point the process becomes recursive. Note, however, this decomposition starts at the end of the frame with i = n − 1, and we hence need to wait until the entire frame is received before processing can commence – an undesirable restriction. For large values of n, we cannot wait for the entire frame to be received, and we need to use another approach. The idea is to perform the Cholesky factorization “locally”. This will lead to a sliding window approximation of the algorithm above. Let us substitute the solution of (4.97), i.e. −1 Ri+1 [1] Fi [1] = F∗i+1 [0]

(4.99)

into (4.98), and we obtain −1 Ri+1 [1] Ri [0] = F∗i [0]Fi [0] + R∗i+1 [1] F∗i+1 [0]Fi+1 [0] −1 ∗ = Yi + Ri+1 [1]Yi+1 Ri+1 [1],

(4.100)

where we have deﬁned Yi = F∗i+1 [0]Fi+1 [0] for convenience. Again, if we start out at the end of the frame, i.e. i = n − 1, (4.100) can be solved recursively. However, in order to obtain a continuous algorithm, we apply a sliding window to the equation above and start the algorithm at i = r + j, rather than at the end of the frame. This then gives us the following sliding window factorization algorithm [4, 6]: Algorithm 4.4 (Asynchronous Cholesky Factorization). Step 1: Set Yr+j−1 = Rr+j−1 [0]. Let i = 2 and m = r + j − 2. Step 2: Compute −1 Rm+1 [1]. Ym = Rm [0] − R∗m+1 [1]Ym+1

(4.101)

Step 3: If i < j, let i = i + 1, m = r + j − i and go to Step 2. Step 4: Ym=r is the j th -step approximation of the solution to (4.100). Find Fr [0] via the Cholesky factorization of Yr = F∗r [0]Fr [0], and −1

Fr−1 [0] = (F∗r [0])

Rr [1].

(4.102)

Furthermore it can be shown that the solutions to the above algorithm monotonically converge towards the correct solutions of (4.100) [4, 6]. The parameter j determines the degree of accuracy of the approximative algorithm.

134

4 Multiuser Detection

4.5 Diﬀerent Received Power Levels A major complication with linear layering ﬁlters is their sensitivity to large power ﬂuctuations. Proper signal cancellation cannot be accomplished without data estimation of the interfering users, and hence, linear preprocessors all suppress the interfering signal spaces in one form or another. This renders them ineﬃcient in scenarios with unequal received power distributions, as we will show in this section. 4.5.1 The Matched Filter Detector The situation of diﬀerent received power levels for the correlation receiver is fairly easy to analyze. We have already mentioned that for the correlation, or matched ﬁlter receiver, users with larger received power levels appear implicitly as “additional” users, reducing the capacity for the low power users. More precisely, if we assume a large user population, and impose the Lindeberg condition on asymptotic negligibility on the powers of the users, given formally as Pk K→∞ −→ 0; ∀k, (4.103) K

Pj m=1 j=k

the interference variance from (4.45) approaches for all users the value 1

2 Pj + σ 2 = σMF , L j=1 K

var(Ik ) =

(4.104)

as K, n → ∞. Analogously to (4.47) we can compute the system capacity as CMF

K Pk 1

= log2 1 + 2 2L σMF

bits/dimension.

(4.105)

k

Furthermore, applying Jensen’s inequality to bring the sum inside the logarithm in (4.105) we obtain K Pk β 1

P¯ log2 1 + 2 ≤ log2 1 + 2 , 2L σMF 2 σMF

(4.106)

k

5K 1 where P¯ = K k Pk is the average energy of the users. We now see from (4.106), comparing it to (4.47), that unequal received power levels hurt the overall system eﬃciency, and that this loss can be dramatic. It is therefore not surprising that power control is such a vitally important aspect of current CDMA systems [154].

4.5 Diﬀerent Received Power Levels

135

The decorrelator shares the same fate as the matched ﬁlter receiver. In the decorrelator the signals are whitened, and the capacity of each layered channel is given by Pk 1 1 bits/use. (4.107) CDEC,k = log 1 + 2 1 − β + 2 σ L Applying Jensen’s inequality in exactly the same manner as above shows that the system capacity of the decorrelator is maximized by the equal received power distribution, and that substantial loss can occur if large power variations are present. Note that the result of the decorrelator does not depend on whether the Lindeberg condition (4.103) is met or not, since interferers are ideally suppressed. 4.5.2 The MMSE Filter Detector The MMSE ﬁlter does not fare any better, it too has maximum system capacity for an equal power distribution. Let us start out with the interference term. Similar to the matched ﬁlter receiver, the MMSE ﬁlter leaves a residual interference term that has noise and signal components. Following [138], it can be shown that this term is zero-mean and has a variance that can be calculated as K 1

Pi Pk var(Ik ) = σ 2 + . (4.108) P i Pk L i=1 Pk + var(I k) i=k

We see that, contrary to the matched ﬁlter equation, the interference variance is only available as an implicit equation. If we let γk = Pk /var(Ik ), then the above equation leads to an implicit equation for the signal-to-noise ratios of the diﬀerent layered channels, given by Pk 1 Pi Pk + . 2 σ L i=1 Pk + Pi γk K

γk =

(4.109)

i=k

We now apply the same arguments as above, i.e. the total system capacity is calculated as 1

β = log2 (1 + γk ) ≤ log2 (1 + γ¯ ), 2L 2 K

CMMSE

(4.110)

k=1

5K 1 where γ¯ = K k γk is the average signal-to-noise ratio for the diﬀerent users. Now, showing that γ is maximized if all γk are equal establishes equality in the upper bound (4.110) and that unequal receiver signal-to-noise ratios lead to an overall system capacity loss. Tse and Hanly have established the related result that if we require all receivers to have the same γk , then the equal received

136

4 Multiuser Detection

power distribution minimizes the total power. Again, the MMSE performance suﬀers in an unequal power environment. The above analytical derivations are diﬃcult to visualize, and we therefore present a couple of example scenarios. Four such power scenarios will be discussed, the ﬁrst is the equal power optimal situation, the second is the the strong user case, where a single user has a signiﬁcantly larger power than the rest of the user, which have equal received power. The situation is reversed in the weak user case, where one user has substantially less power than the remaining users, and the fourth case is that of two equal-sized groups with diﬀerent powers each. The equal power case will be used as reference, and the weak user case is easily dismissed, since the weak user has no eﬀect on the remaining users, the situation is simply that the weak user has a very poor channel. The remaining two cases are more interesting. Starting from the ﬁrst equality in (4.110), we compute the average bit en5K ergy to noise power spectral ratio as Eb /N0 = k=1 Pk /(σ 2 CMMSE ). Equation (4.110) is the spectral eﬃciency. Plotting it against Eb /N0 gives the Shannon bounds shown in Figure 4.16 and 4.17. The examples are calculated for an eight-user system and the two cases of the strong user (Figure 4.16), and the two equal-sized power groups (Figure 4.17). The resulting Shannon bounds are drawn for four levels of power diﬀerence, 10dB, 15dB, 20dB, and 30dB and for a system load of β = 8/15. It becomes evident that the single powerful user has the most detrimental eﬀect on the overall system eﬃciency, since, in essence, everybody else is trying to suppress the strong user and thus decreases their layered capacities. The case of a single strong user is, in a certain sense, the worst case situation. If we require a minimum power level of the lowest power user, the speciﬁc power distribution with a single strong user cannot be written as the weighted combination of other power distributions. Furthermore, it can be shown that both the decorrelator and MMSE capacities are convex in the powers W. Consequently, any distribution other than the ones with a single strong user, which can be written as a linear combinations of the former, has a larger capacity. We will see in Chapter 6 that interference cancellation where the signals of these strong users are detected or estimated will fare much better and will not suﬀer from this caveat of unequal received powers. In fact, there we will show that the equal received power situation is the worst-case scenario, not the best case as it is here with linear preprocessing. One can, of course, argue that the average energy is not a appropriate measure of the system energy load, but similar conclusions can be drawn with other deﬁnitions of power eﬃciency.

4.5 Diﬀerent Received Power Levels

137

10

Capacity bits/dimension

1

$:*1FDSDFLW\LGHDOFDVH 006(HTXDOSRZHU 006(\ G% 006(\ G% 006(\ G%

0.1

-2

006(\ G%

0

2

4

6

8

10

12

14

Eb /N0 16 [dB]

Fig. 4.16. Shannon Bounds for an MMSE joint detector for the unequal received power scenarios of one strong user, and equal power for the remaining users.

138

4 Multiuser Detection

10

Capacity bits/dimension

1

$:*1FDSDFLW\LGHDOFDVH 006(HTXDOSRZHU 006(\ G% 006(\ G% 006(\ G%

0.1

-2

006(\ G%

0

2

4

6

8

10

12

14

Eb /N0 16 [dB]

Fig. 4.17. Shannon Bounds for an MMSE joint detector the case of two power classes with equal numbers of users in each group.

5 Implementation of Multiuser Detectors

5.1 Iterative Filter Implementation We have seen in the previous chapter that linear ﬁltering is a very powerful joint detection method. Unfortunately, most of the eﬃcient linear multiuser receivers require a matrix inversion, such as the MMSE detector of Section 4.4.5, and the decorrelator of Section 4.4.2. Each requires a matrix inverse of the size of the system, that is, the maximum number of users. For high speed implementations, the required complexity may be prohibitive. Furthermore, if random spreading sequences are used as in IS95 [135], cdma2000 [134], and 3GPP [38], these inverses cannot be found via adaptive signal processing methods, and have to be computed for each symbol interval. The situation with a straightforward matrix inversion is further complicated if the system is asynchronous, see Figure 2.7, where the correlation matrix becomes band-diagonal. Recursive ways to invert such a matrix are known, however they require a windowed procedure analogous to the one discussed in Section 4.4.9. In the remainder of this chapter we discuss iterative matrix inversion techniques [8, 34, 35] applied to linear multiuser detectors which require a matrix inverse. These techniques have been developed in the linear algebra community to reduce the complexity of solving linear algebraic equations. Well-known methods such as the Jacobi, ﬁrst and second-order stationary, Chebyshev, and Gauss-Seidel iterative methods can all be applied to multiuser detection. In subsequent sections we explore these methods and their convergence behavior as applied to random CDMA systems. 5.1.1 Multistage Receivers We start this chapter with the concept of a multi-stage receiver, and then show that this receiver is, in fact, the realization of an iterative matrix inversion method. Multistage receivers were developed as a response to the complexity problem of multiuser receivers, initially without reference to iterative matrix solution methods.

140

5 Implementation of Multiuser Detectors

Let us start with the outputs of the bank of matched ﬁlters i.e. with (4.48) y = S∗ r = RAd + z .

(4.48)

Let y[i] be the sub-vector corresponding to the i-th time interval, i.e. y[i] = (y[iL + 1], · · · , y[(i + 1)L]), and therefore y = (y[1], · · · , y[n]). Decomposing (4.48) into equations for each time interval as shown in Figure 5.1, we obtain ˜[j], d[i] = R1 [j −1]Ad[j −1] + R0 [j]Ad[j] + R∗1 [j +1]Ad[j +1] + z (5.1) √ √ where A = diag P1 , · · · , PK is the synchronous amplitude matrix (see Section 2.4). Rearranging terms in (5.1) and ignoring the noise we obtain y[j] = A−1 (y[i] − R1 [j −1]Ad[j −1] − (R0 [j] − I) Ad[j] − R∗1 [j +1]Ad[j +1]) . (5.2)

R∗1 [j]

R1 [j]

zeros

R0 [j] R∗1 [j +1]

zeros

R1 [j +1]

Fig. 5.1. Illustration of the asynchronous blocks in the correlation matrix R.

If we have available an initial estimate of the symbols d[j−1], d[j], d[j + 1], denoted by d(0) [j −1], d(0) [j], d(0) [j + 1], we may use the above equation to compute iteratively

5.1 Iterative Filter Implementation

141

d(m+1) [j] = A−1 y[j] − R1 [j]Ad(m) [j −1] − R0,lower [j]Ad(m) [j] −R0,upper [j]Ad(m) [j] − R∗ [j +1]Ad(m) [j +1] , (5.3) from a previous estimate d(m) [j −1], d(m) [j], d(m) [j + 1], where R0,lower [j] is the lower triangular part of R0 [j], and R0,upper [j] is the upper triangular part. The initial estimate may be taken from (4.48), i.e. Ad[j] = y[j]. Of course multiplication and division by A does not need to be carried out in (5.3), since we can iterate over Ad[j] rather than d[j]. This is in essence the basic idea behind the multistage detectors which were originally proposed and considered for implementation in [29, 94, 145, 173]. The block diagram of such a multistage receiver is shown in Figure 5.2.

Unit Delay

Ad

Unit Delay

R∗0 [j]−I

[j] (1)

y[j −1]

R∗1 [j +1]

Ad(0) [j]

R∗1 [j]

Ad(1) [j−1]

R∗0 [j −1] −I

Ad(0) [j −1]

R1 [j]

Unit Delay

Stage 1

Unit Delay

Ad(1) [j−2]

R1 [j −1]

Stage 2

y[j +1]

Ad(0) [j +1] Correlators

Ad(2) [j −1] Fig. 5.2. The multistage receiver for synchronous and asynchronous CDMA systems.

If we study this detector carefully we see that we may replace the previous estimates as soon as the new ones become available, rather than wait until an entire iteration has been carried out. We then obtain a modiﬁed iterative update equation, given by

142

5 Implementation of Multiuser Detectors

d(m+1) [j] = A−1 y[j] − R1 [j]Ad(m+1) [j −1] − R0,lower [j]Ad(m+1) [j] −R0,upper [j]Ad(m) [j] − R∗ [j +1]Ad(m) [j +1] , (5.4) It turns out that this version of the multistage receiver converges much faster to a good solution, and we will see the reasons why in the next subsection. In the implementations presented here, the intermediate estimates d(m) [j] are real-number estimates. The original multistage receivers used hard decisions at each stage, arguing that the symbols of d ∈ {±1}. It turns out that the hard decision multistage receivers show a slightly better performance (see Section 5.1.2), but are signiﬁcantly more diﬃcult to analyze since the linear analysis techniques discussed below does not apply. 5.1.2 Iterative Matrix Solution Methods We will now show that (5.4) is in fact an iterative version of the decorrelator equation (4.49), and equation (5.4) is the well-known Gauss-Seidel method [55, 131] for iteratively calculating the solution of a linear algebraic equation. Borrowing from the ample mathematical literature devoted to the problem of iterative equation solving, see e.g. [8, 55, 60, 131], we now start by considering an arbitrary linear systems equation, given by Mx = b, and let M = S − T be a splitting of M. Now Sx = Tx + b =⇒ Sx(m+1) = Tx(m) + b.

(5.5)

We have now generated an iterative solution approach for x: x(m+1) = S−1 Tx(m) + b .

(5.6)

The question is whether repeated iterations of (5.6) will lead to a solution of Mx = b. This question is quickly settled by studying the error e(m) at iteration j. Let x(m) = x + e(m) , then S(x + e(m+1) ) = T(x + e(m) ) + b Se(m+1) = Te(m) m e(m+1) = S−1 Te(m) = (S−1 T) e(0) .

(5.7) ∗

Using the spectral decomposition theorem to write S T = QΛQ , where Λ is a diagonal matrix of the eigenvalues of S−1 T, and Q is unitary, the iteration error is expressed by e(m+1) = QΛm Q∗ e(0) . (5.8) −1

The matrix Λm contains the powers of the eigenvalues λk of S−1 T on its diagonal, and therefore e(m+1) is dominated by the largest eigenvalue of S−1 T, also known as the spectral radius of S−1 T, i.e. λmax = ρ(S−1 T). As long as ρ(S−1 T) < 1, the error e(m+1) will decrease at each iteration, and e(m+1) → 0, irrespective of the starting vector x(0) . Diﬀerent iterative solution methods are now obtained by choosing diﬀerent splittings of the original matrix (for an in-depth discussion, see [8]).

5.1 Iterative Filter Implementation

143

5.1.3 Jacobi Iteration and Parallel Cancellation Methods If we let M = R and solve Md = r, then d(m+1) = A−1 (I − R) Ad(m) + y

(5.9)

is an iterative solution for the decorrelator output A−1 R−1 y. The particular splitting used is S = D = I and yields the iterative solution known as Jacobi’s method. Writing (5.9) out in block matrix form for an asynchronous system we obtain the exact form of the multistage equation (5.3). The iterations may be started with an arbitrary initial vector, e.g. d(0) = y. Of course the initial error has an eﬀect on the number of iteration steps required to achieve a given accuracy. Furthermore, this initial estimate should be found with low complexity, which is the case if we use rMF . Furthermore, it has been noted that using the matched ﬁlter outputs as a starting point is faster than initializing the iterations with the all-zero estimate [66]. Equation (5.3) corresponds to parallel interference cancellation, whose circuit implementation is shown in Figure 5.2, i.e. if the soft decisions from the previous stage are used in the cancellation procedure, the Jacobi iterative decorrelator is identical to the multistage receiver presented in the last section. This connection of iterative matrix inversion with parallel and serial interference cancellation has been explored by a number of authors [34, 35, 58]. Figure 5.3 shows the experimental simulated performance of an iterative Jacobi decorrelator as a function of the number of iterations for a system load of β = 0.16, and Es /N0 = 7dB. The performance is compared to a ﬁrst-, and second-order stationary iteration method discussed below. The performance approaches that of the decorrelator within about 5 iterations, then dips below the decorrelator error probability, and ﬁnally, asymptotically approaches its performance again, as predicted by the theory. From the convergence criterion it is clear that the Jacobi decorrelator converges if ρ(R − I) < 1 ⇒ ρ(R) < 2, for a given ﬁxed set of signature sequences S, and irrespective of the amplitudes A. If the spreading sequences are random, we have the following Theorem 5.1. For large systems with random spreading codes, the Jacobi iterative receiver converges to the decorrelator using √ 2 the splitting S = D = I if and only if K < L 2 − 1 , and the asymptotic convergence factor1 is given by (5.10) ρJ = β + 2 β.

144

5 Implementation of Multiuser Detectors 10−1

Average Bit Error Rate

)LUVWRUGHUVWDWLRQDU\ 6HFRQGRUGHUVWDWLRQDU\ -DFREL

10−2

GHFRUUHODWRU 10−3

006( 2

4

6

8

10

12

14

16

Iterations

Fig. 5.3. Example performance of a parallel cancellation implementation of the decorrelator as a function of the number of iteration steps.

Proof. The Jacobi iteration converges if and only if ρ(I − R) < 1. Now for any eigenvalue λ of R, 1 − λ is an eigenvalue of I − R [60]. The iteration converges therefore if and only if maxλ(R) |1 − λ| < 1. According to Theorem √ 2 3.5 in Chapter 3, ρ(R) converges in probability to β + 1 if K, L → ∞, 2 √ and K/L = β is ﬁxed. The minimum eigenvalue of R converges to √ √ β−1 . For β ∈ [0, 1] therefore |1 − λmin | = 2 β − β ≤ |1 − λmax | = 2 β + β, and √ 2 √ hence the condition 2 β + β < 1 implies that K < L 2 − 1 . The above proof is strictly true only for synchronous systems due to Theorem 3.5, for which extensions to asynchronous matrices are not readily available. However, numerical evidence indicates that asynchronous systems follow the same performance and convergence laws. Since the decorrelator and the MMSE ﬁlter (4.71) diﬀer only in the diagonal components of the ﬁlter matrix, we can equally well implement an iterative Jacobi MMSE ﬁlter. From Chapter 4 the MMSE solution is given as −1 y (4.71) AdMMSE = R + σ 2 W−1 by using the splitting S = I + σ 2 W−1 = W−1 W + σ 2 I which leads to the iterative implementation 1

For a given norm of an iteration matrix B, the k-step convergence factor is given ‚ ‚1/k ‚ ‚ = ρ(B). Thus the by ‚Bk ‚. However, for any matrix norm limk→∞ ‚Bk ‚ average convergence factor per step approaches the spectral radius [60].

5.1 Iterative Filter Implementation

−1 d(m+1) = A−1 W + σ 2 I W (I − R) Ad(m) + y .

145

(5.11)

For this iterative receiver the following convergence result is proven in [58]: Theorem 5.2. For large systems and random spreading sequences the Jacobi iterative receiver converges to the MMSE receiver if 2 −1 2 + γmax − 1 where γmax = max(wk /σ 2 ). K
(5.12) Moreover, for equal power users (5.12) is also tight, and therefore necessary. Proof. Note that

−1 W + σ2 I W (I − R) −1 ≤ ρ W + σ2 I W ρ (I − R) wi = (λmax − 1) max i=1···K wi + σ 2 1 = (λmax − 1) −1 , 1 + γmax

ρ (S−1 T) = ρ

(5.13) (5.14) (5.15) (5.16)

since the spectral radius is a metric measure. After some straightforward algebra (5.12) follows. For equal powers W + σ 2 I is just a scaled identity matrix and (5.14) becomes an equality. The iterations (5.11) can be made to converge for all −1 β ≤ 1 if γmax ≥ 2 ⇒ γmax ≤ −3dB. However, in this uninteresting case performance for all users would be quite poor. Figure 5.4 shows simulation results for K = 8 to illustrate that the asymptotic results are achieved quite rapidly. The curve labeled “hard” uses hard decisions at each iteration, i.e. d(j) → sgn d(j) before the next iteration. This was the procedure suggested in the original papers on multistage decoding [145, 146]. Clearly observable is the failure of the Jacobi decorrelator for β > 0.16, even though K ∞. It is worth noting that the hard-iteration Jacobi receiver outperforms the decorrelator for small loads β, a phenomenon which we have seen before in Figure 5.3, which derives from the facts that the decorrelator does not achieve the minimum mean-square error, and that we are measuring hard decision error rates obtained from the soft estimates d(j) . From iterative matrix solution theory, several more advanced methods exist, such as higher order stationary methods using a ﬁxed iteration scheme,

146

5 Implementation of Multiuser Detectors 1

√ β = ( 2 − 1)2

Average Bit Error Rate

Jacobi 10

−1

Hard Decorrelator

10−2

Eb /N0 = Es /N0 = 7dB

10−3

10−4 0

0.2

0.4

0.6

0.8

1 β

Fig. 5.4. BER Performance of Jacobi Receivers versus system load for an equal power system, i.e. A = I.

or non-stationary method such as the Chebyshev method [8, 60]. All of these methods lead to variations of parallel interference cancellation, the details of which are discussed in [58]. Furthermore, the asymptotic convergence factors diﬀer slightly for diﬀerent methods. They are tabulated for comparison at the end of the next section. Note also that any invertible K × K matrix M has a series expression of the inverse M−1 with at most K terms, since each matrix fulﬁlls its own characteristic equation [60, 131]. This is the celebrated Cayley Hamilton Theorem: Theorem 5.3. A matrix M fulﬁlls its own characteristic equation c(λ) = (M − λI) v = 0, i.e. c(M) = cK MK + cK−1 MK−1 + · · · + c1 M = −c0 I.

From Theorem 5.3 we can ﬁnd an exact inverse for M as 1 cK MK−1 + · · · + c1 I . M−1 = − c0

(5.17)

(5.18)

However, computation of the series coeﬃcients ck is tantamount to inverting the matrix and thus not practical.

5.1 Iterative Filter Implementation

147

5.1.4 Stationary Iterative Methods The limitations of the Jacobi iterative method can be avoided by generalizing the iteration equation (5.9). This is accomplished by the ﬁrst-order stationary iteration solution, given by Ad(m+1) = Ad(m) − τ (m) MAd(m) − y , (5.19) where τ (m) is a sequence of constants [8]. If τ (m) = τ for all m, the method is called stationary. Clearly, we now want to choose τ such that the spectral radius of the iteration matrix S−1 T is minimized. This is accomplished by the following Theorem 5.4. For large systems with β < 1, the ﬁrst-order stationary iteration 1 1 (m+1) Ad y− R − I Ad(m) = (5.20) 1+β 1+β converges to the decorrelator for any initial vector d(0) , and the constant τ = 1/(1 + β) maximizes the convergence speed. The asymptotic convergence factor is given by √ 2 β . (5.21) ρS1 = 1+β

Proof. According to [8, Theorem 5.6], for M symmetric positive deﬁnite, the parameter which results in the fastest convergence of the ﬁrst-order stationary iteration is τopt = 2/(λmin + λmax ). Applying the limiting distribution of the eigenvalues of R from Theorem 3.5 completes the proof. A second-order iteration is a generalization of (5.19) using the two most recent estimates, given by Ad(m+1) = a(m) Ad(m) − (1 − a(m) )Ad(m−1) − b(m) MAd(j) − y . (5.22) For a stationary iteration we set a(j) = a and b(j) = b, as well as a(0) = 1 and b(0) = τopt . In [58] the following theorem is proven:

148

5 Implementation of Multiuser Detectors

Theorem 5.5. For large systems with β < 1, the ﬁrst-order stationary iteration Ad(m+1) = y − (R − (1 + β)I) Ad(m) − βd(m−1) with Ad(1) = y −

1 (R − I) Ad(0) 1+β

(5.23)

(5.24)

converges to the decorrelator for any initial vector d(0) , and the chosen parameters are optimal for second-order stationary methods. The asymptotic convergence factor is given by ρS2 = β. (5.25)

Letting M = R + σ 2 W−1 , both ﬁrst-, and second-order stationary method can also be used to implement the MMSE ﬁlter. 5.1.5 Successive Relaxation and Serial Cancellation Methods In this section we explore a very eﬃcient splitting, which leads to the socalled successive relaxation technique known in linear algebra [60]. It will also lead to a modiﬁed cancellation method, namely serial cancellation. We start with the matrix splitting M = D − ωL − (1 − ω)L − L∗ where −L is the lower triangular part of M. The parameter ω is the relaxation parameter. The resulting iterative solution can be obtained after a few algebraic manipulations as 1 1−ω D − L Ad(m+1) = y + D + L∗ Ad(m) . (5.26) ω ω As long as ω ∈ (0, 2) (5.26) alwaysconverges [58]. In fact, what is required is −1 1−ω ∗ that ρ ω1 D − L < 1. It can be shown that this is always ω D+L the case for symmetric matrices M with positive diagonal elements, and hence for the two matrices M = R, and M = R+σ 2 W−1 , which are of most interest to joint linear detection. The relaxation parameter can be designed to optimize convergence speed and lies between 1 and 1.8 for β ∈ [0, 1]. Note that we have again assumed the starting vector b = y. If we choose ω = 1, a particularly simple successive iteration scheme results, known as the Gauss-Seidel method, and given by (D − L) Ad(m+1) = y + L∗ d(m) .

(5.27)

5.1 Iterative Filter Implementation

149

Since D − L is lower diagonal, and L∗ is upper diagonal, the iteration solution d(m+1) can calculated successively by starting with its ﬁrst entry d(m+1) [1], which depends only on previous iteration results d(m) , and continues as K K

Pk d(m+1) [k] = y[k] − Rki Pi d(m+1) [i] − Rki Pi d(m) [i].

(5.28)

i=k +1 i>k

i=1 i
Inspection of (5.28) and (5.4) reveals that multistage decoding according to (5.4) is an instance of the Gauss-Seidel iterative matrix solution method for an asynchronous system. Let us consider an alternate derivation [39] of the Gauss-Seidel method which will allow us to gain further intuitive understanding of its operation. Let us consider the quadratic function J(Ad) = r − SAd

(5.29)

= d∗ ARAd2− d∗ AS∗ r − r∗ SAd + c J(x) = x∗ Rx − b∗ x − x∗ b + c,

(5.30) (5.31)

with x = Ad and b = SAd. Since R is positive deﬁnite, the function J(x) is convex and has a single minimum. This minimum can be computed exactly by setting the gradient of J(x) equal to zero, and will lead to the closed form MMSE solution discussed in Section 4.4.5. However, in an iterative fashion the Gauss-Seidel method approaches this minimum with successive minimization steps over each element x, assuming the rest of the elements of x to be ﬁxed. Algorithmically, if the i-th element of x is varied to minimize J(x), its partial derivative

∂J(x) =2 Ril x[l] − 2b[i] ∂x[i] K

(5.32)

l=1

is set to zero, leading to : 9 i−1 K

1 b[i] − Ril x(m+1) [l] − Ril x(m) [l] x(m+1) [i] = Rii l=1 l=i+1 ⎞ ⎛ i−1 K

1 ⎝b[i] − Ril x(m+1) [l] − Ril x(m) [l]⎠ . = x(m) [i] + Rii l=1

l=i)

(5.33) Executing the Gauss-Seidel updates according to (5.33) (second line) not only is numerically robust, but also allows for a simple visualization of the update procedure, shown in Figure 5.5 for a system with two variables, where the

150

5 Implementation of Multiuser Detectors

ellipses are the contour lines for a constant value of J(x). Note that the Gauss-Seidel method optimizes each variable in turn, leading to the indicated zig-zag trajectory towards the global minimum, which represents the MMSE solution.

$

%

&

'

) ( +

* ,

Fig. 5.5. Visualization of the Gauss-Seidel update method as iterative minimization procedure, minimizing one variable at a time. The algorithm starts at point A.

Figure 5.6 shows the simulated performance of some of the advanced iterative receiver systems, including the Gauss-Seidel method for a load of β = 8/32 = 0.25 at a signal-to-noise ratio of Es /N0 = 10dB. The target ﬁlter was the MMSE ﬁlter. As can be seen, the stationary methods have roughly the same performance, while the advanced methods are slightly faster. The diﬀerence between diﬀerent over-relaxation methods is minor, which means that the simple Gauss-Seidel method is preferable in practical applications due to its simplicity. The conjugate gradient method is in general the fastest, but it is more complex than the simpler parallel and serial cancellation methods, which have the additional advantage that their implementations can be pipelined to reduce latency. Using the results on the asymptotic behavior of the eigenvalues of random matrices as discussed in Chapter 3, Grant and Schlegel [58] have derived the following table of convergence factors for the diﬀerent iterative methods:

5.1 Iterative Filter Implementation

151

Average Bit Error Rate

10−1

10−2

10−3

10−4 MMSE Performance

Eb /N0 = 10dB N = 32, K = 8

10−5 0 1 2 3 4 5 6 7 Iterations Fig. 5.6. Iterative MMSE ﬁlter implementations for random CDMA systems. Method

Optimal Parameters

Jacobi Stationary 1 Stationary 2 Chebyshev

τopt = 1 + β −1 aopt = 1 + β, bopt = 1 h “ ”i−1 √ τi = 1+β +2 β cos kk−1/2 π +1 max

Converges Factor √ β < 0.17 β√ +2 β β<1 2 β/(1+β) √ β<1 β √ β<1 β

Succ. Relax.

ωopt = 2/(2 − β)

β<1

β 1/4

All but the successive relaxation method are parallel interference cancellation schemes. 5.1.6 Performance of Iterative Multistage Filters In the previous sections we have seen how linear ﬁlters requiring matrix inversions can be implemented via multistage ﬁltering structures which realize iterative solutions to the linear algebraic matrix problem. The performances of these ﬁlters will, in the limit of large numbers of stages, exactly approach that of the original ﬁlters, and the performance limits discussed in the previous chapter therefore apply. In this section we again look at the ultimate performance potential of a system using capacity arguments. We are interested in the performance for a ﬁnite number of stages, since low complexity implementations will attempt to minimize the number of iterations stages. An understanding of the complexity/performance tradeoﬀs involved are thus important.

152

5 Implementation of Multiuser Detectors

The ﬁrst-order stationary iterative ﬁlter is probably the structure most analyzed and best understood. It is both simple and eﬃcient, and allows a fair amount of analytical treatment. We will concentrate our treatment on ﬁrstorder iterative ﬁlters in this section. First note that the ﬁrst-order iterative approximation of the MMSE ﬁlter2 , given by σ2 (m+1) I d(m) = τy − τR − 1 − τ (5.34) d P can be rewritten as a linear ﬁlter equivalent in the following way:

m m

σ2 (m+1) d I − τR = τ 1−τ P i=0 m+1 σ2 I − τR y. + 1−τ P

(5.35)

Using the spectral decomposition of the hermitian matrix R = QΛQ∗ we can express (5.35) as

m i

σ2 (m+1) I − τΛ =Q τ 1−τ d P i=0 m+1 σ2 I − τΛ Q∗ y + 1−τ P = QΓ Q∗ y.

(5.36)

The composite matrix Γ is diagonal with l-th entry given as 2 i 2 m+1 m

σ σ + λl + λl + 1−τ 1−τ γl = τ P P i=0 2 m+1 2 m+1 1 − 1 − τ σP + λl σ + λl = + 1−τ σ2 P P + λl =

1 m+1 1 − (1 − gl ) (1 − τ gl ) , gl

(5.37)

where gl = σ 2 + λl . For random spreading sequences and equal powers the eigenvalues λl of R are known and given by Theorem 3.5. The values gl therefore have a known distribution, which is that for λl shifted by the noise power σ 2 . 2

We choose the implementation for the MMSE ﬁlter here, but an equivalent derivation holds for any other ﬁlter involving an inverse. Furthermore, √ we assume for the remainder of this section an equal-power system with A = P I.

5.1 Iterative Filter Implementation

153

Furthermore, using the formula for optimal convergence given by τopt =

2 λmin + λmax

(5.38)

as in Theorem 5.4, together with the limiting eigenvalues of the random matrix, we ﬁnally ﬁnd 1 τopt = . (5.39) 1 + β + 2σ 2 /P It is important to note that for τ > τopt a severe degradation of the performance can occur. This point was stressed in [137] who have done extensive numerical simulations. This necessitates that, for ﬁnite stages, the iteration constant τ may have to be backed oﬀ from its theoretical value (5.39). The output d(m+1) of the multistage ﬁlter will contain signal components, noise components, and incompletely canceled signal components. The noise components are given by zMS = QΓ Q∗ S∗ z,

(5.40)

which has a correlation matrix given by Rz = E [zMS ∗ zMS ] = σ 2 QΓ ΛΓ Q∗ .

(5.41)

2 = σ 2 tr(QΓ ΛΓ Q∗ ), The total noise power over all components equals σtot and therefore, by symmetry, the noise component in each component of the multistage ﬁlter equals 2 0 1 λl m+1 2 σMS = σ 2 E γl2 λl = σ 2 E 2 1 − (1 − gl ) (1 − τ gl ) , (5.42) gl

and the expectation is over the distribution of the eigenvalues λl of R given by the density in Theorem 3.5. Likewise, the signal components in the output d(m+1) are given by dMS = QΓ Q∗ S∗ SAd.

(5.43)

Assuming the data to be independently distributed, i.e. E [d∗ d] = P I we calculate the total signal and interference power as (5.44) tr(d∗MS dMS ) = P tr Γ 2 Λ2 , which contains signal as well as interference components. Again, due to symmetry, the signal and interference value of any given dimension is given by 2 2 λ m+1 . (5.45) PS+I = P E 2l 1 − (1 − gl ) (1 − τ gl ) gl Since the received vector is given by (5.36), the signal part of the K diﬀerent layered channels is given by the diagonal values

154

5 Implementation of Multiuser Detectors

dS = diag {QΓ ΛQ∗ } d = diag {B} Ad.

(5.46)

The signal powers are the squared diagonal entries of B. We can bound the sum of their values using Jensen’s inequality as follows: K

i

1 2 Bii ≤ K

9

K

:2 Bii

.

(5.47)

i

Furthermore, for asymptotically large systems K

2 Bii

i

K,L→∞

−→

1 K

9

K

:2 Bii

,

(5.48)

i

and the bound quickly becomes tight. Assuming independence between the signal and interference terms, and using tr (B) = tr (Γ Λ) which equals the sum of the eigenvalues, the user power of any given layered signal stream converges to ⎡ 9 : ⎤ K 2

P ⎣ λl K,L→∞ m+1 ⎦, PS E 1 − (1 − gl ) (1 − τ gl ) −→ (5.49) K g l i and the interference power is 2 σIP = PS+I − PS .

(5.50)

From the above we can compute the signal-to-noise ratio of the iterative ﬁlter as PS (5.51) γMS = 2 2 , σMS + σIP from which the capacity of an individual layered channel can be computed. Trichard et. al. [137] use a recursive moment generating function for the moments of the eigenvalues of S∗ S which allows the evaluation of the above integrals as recursive algebraic equations. Following the same procedure as in Chapter 4 an implicit equation for the maximum spectral eﬃciency can be computed from the per-dimension capacity formula β CMS = log (1 + γMS ) (5.52) 2 as follows. The overall (interference-free) signal to noise ratio KP CMS Eb 2CMS Eb = = σ2 βσ 2 βN0 for one-dimensional systems, and therefore, in terms of Eb /N0

(5.53)

5.1 Iterative Filter Implementation

CMS

9 : Eb 2CMS PS N β 0 . = log 1 + 2 Eb + P βσ 2 2 2CMS σIP MS N0 σ2

155

(5.54)

Figure 5.7 shows numerical evaluations of the Shannon bound equation (5.54) for an example load of β = 0.5 and equal power of all users – other loads produce quite similar results. What is remarkable is that in the range of moderate values of Eb /N0 of about 0–4dB, simple iterative ﬁlters with as few as two stages are suﬃcient to approach the capacity of the complete MMSE ﬁlter. Such values of Eb /N0 are not uncommon in typical coded wireless links, suggesting that the complexity of linear multiuser detection could be as little as a few times that of simple matched ﬁltering. Furthermore, note that the number of stages does not depend on the system size, i.e. even if K, L → ∞, the same small number of stages is required as for a small system to achieve a certain per-user capacity.

MMSE per-user Capacity CMS

10

AWGN Capacity

β = 0.5 Multistage Capacities

m = 10 m=5

1

m=2 m=1 Matched Filter: m = 0

0.1

-2

0

2

4

6

8

10

12

14 Eb /N0 [dB]

Fig. 5.7. Shannon bounds for the AWGN channel and multistage ﬁlter approximations of the MMSE ﬁlter for a random CDMA system with load β = 0.5. The iteration constant τ was chosen according to (5.39).

Note that thus far, including Figure 5.7, the iterative ﬁlter has been based on the minimum-square error ﬁlter R + σ 2 W−1 . If we are to use an iterative

156

5 Implementation of Multiuser Detectors

approximation to the decorrelator ﬁlter R, some of the equations simplify, and the noise variance (5.42) becomes 2 1 m+1 2 2 , (5.55) 1 − (1 − λl ) (1 − τ λl ) σDEC = σ E λl while the signal and interference power (5.45) is given by 2 m+1 , PS+I = P E 1 − (1 − λl ) (1 − τ λl ) and the signal power (5.49) by ⎡ 9 :2 ⎤ K

P ⎣ K,L→∞ m+1 ⎦. E −→ 1 − (1 − λl ) (1 − τ λl ) PS K i

(5.56)

(5.57)

Equation (5.39) for the optimal iteration constant still applies, but becomes now independent of the signal-to-noise ratio P/σ 2 . Using (5.55), (5.56), and (5.57) in (5.54) produces the diﬀerent Shannon bounds of Figure 5.8 for the ﬁnite-stage approximation of the decorrelator. The major diﬀerence is that higher-order ﬁlters show a poorer performance at low values of Eb /N0 , that is, a higher number of stages is counter productive for Eb /N0 less than approximately 2dB. This is not surprising, since the iterative ﬁlter approaches the performance of the decorrelator, which is signiﬁcantly poorer at low signal-to-noise ratios than that of the MMSE ﬁlter. Somewhat surprisingly, however, at lower values of the Eb /N0 a low-stage iterative ﬁlter approximation of the decorrelator approaches the performance of the MMSE ﬁlter. A phenomenologically similar behavior of the iterative approximation to the decorrelator we have already encountered in Figure 5.3. The performance of the MMSE ﬁlter constitutes an upper bound on the capacities of these iterative ﬁlters, while the lower of the two values of either the decorrelator or the matched ﬁlter receiver forms a lower bound.

5.1 Iterative Filter Implementation

157

Decorrelator per-user Capacity CDEC

10

AWGN Capacity

β = 0.5 Multistage Capacities

m = 10 m=5

1

m=2 m=1 Matched Filter: m = 0 Decorrelator 0.1

-2

0

2

4

6

8

10

12

14 Eb /N0 [dB]

Fig. 5.8. Similar Shannon bounds for multistage ﬁlter approximations of the decorrelator with load β = 0.5.

158

5 Implementation of Multiuser Detectors

5.2 Approximate Maximum Likelihood As discussed in Section 4.2.1, jointly optimal detection minimizes the vector ˆ = d). Recalling (4.2), this is achieved for uniform prior error probability Pr(d probabilities by ˆ ML = arg min r − SAd2 . (5.58) d 2 d∈D nK

The main impediment to low-cost implementation of jointly optimal detection is the cardinality of the set DK . Brute-force implementation of (5.58) requires computation of |D|K metrics. This appears to be a fundamental limitation in light of the NP-hard nature of the problem [149]. The Viterbi algorithm implementation of optimal detection described in 2 Section 4.2.1 computes the metric r − SAd2 for every possible d ∈ DnK and operates on a trellis with maximum width of 2K−1 states. Despite the apparent diﬃculty of the jointly optimal detection problem, in practice only a relatively small subset of DnK has any signiﬁcant posterior probability (depending upon the noise level in the system). The goal of low-complexity sub-optimal approaches is to make eﬀective use of this phenomenon. If it were possible to restrict attention to only those (hopefully few) sequences with signiﬁcant posterior probability, detector complexity would be signiﬁcantly reduced. The main design problem is therefore to determine which sequences to consider. In the context of the trellis representation introduced in Section 4.2.1, the problem is to design an eﬀective tree search that does not visit every node. The idea is to avoid following paths through the tree that can be determined early on to have little chance of being the optimal one. There are many such search methods in the literature, and this basic idea has been used widely in the area of error control coding. The eﬀectiveness of restricted search algorithms therefore depends heavily upon the ability to prune sections of the tree as early as possible. In Section 5.2.2 we will describe reduced-complexity tree search methods. Other search methods motivated by lattice structures will be developed in Section 5.2.3. First however, we need to develop a more suitable recursive metric. In Section 4.2.1, we developed a recursive metric for use in a Viterbi implementation of the jointly optimal detector. Let us review the development of this metric. The relevant terms of the Euclidean distance 2

r − Sd2

(5.59)

for the purposes of optimal detection are −2y∗ d + d∗ Rd

(5.60)

which (using the symmetry of R) led directly to the recursive metric (4.11)

5.2 Approximate Maximum Likelihood

λl (d) = λl−1 (d) − 2dl yl +

l−1

2di dl Ril .

i=1

159

(5.61)

Branch Metric

An important observation from (5.61) is that the branch metric bl (d1 , d2 , . . . , dl ) = 2dl yl +

l−1

2di dl Ril

(5.62)

i=1

can be either positive or negative. This is not a problem for an algorithm that searches the entire tree, but it does cause diﬃculties for restricted tree search algorithms. This is due to the fact that a partial metric along a given path may in fact decrease further along the path. This means that some paths that appear very unlikely part of the way the tree may later become more likely through the addition of negative branch metrics. This possible future reduction in metric makes it very hard to compare two partial paths. It would be preferable for the partial metric to be monotonic. This would mean that unlikely paths cannot become any more likely based on future observations, and would motivate early pruning of unlikely portions of the tree. 5.2.1 Monotonic Metrics via the QR-Decomposition It is possible to obtain just such a monotonic metric, via use of the wellknown QR-decomposition. This is a matrix decomposition of the form QU, where Q is unitary, Q∗ Q = I and U is upper triangular. (we use U rather than the more traditional R to emphasize the upper triangular structure and to avoid confusion with the correlation matrix). For our purposes, it will be more convenient to work with a modiﬁed QR-decomposition that yields a lower triangular factor. This can be accomplished as follows. Let ⎞ ⎛ 0 ... 0 1 ⎟ ⎜ .. ⎜. 1 0⎟ ⎟ ⎜ ⎟ J=⎜ ⎟ ⎜ ⎜ .. ⎟ ⎝0 1 .⎠ 1 0 ... 0 be the identity matrix with its columns in reverse order. Let ˜ = SJ QU be the regular QR-decomposition of the re-ordered modulation matrix. Then, since J2 = I, ˜ S = QUJ ˜ = (QJ)(JUJ) = QL

160

5 Implementation of Multiuser Detectors

˜ is unitary and L = JUJ is lower triangular. where Q = QJ Considering again the Euclidean distance metric (5.59), we can use the modiﬁed QR-decomposition of S to write 2

2

r − Sd2 = r − QLd2 = =

∗

2 Q (Q r − Ld)2 2 Q∗ r − Ld2 ,

(5.63) (5.64) (5.65)

where the last line is due to the invariance of the 2-norm under unitary transformations. Noting that R = S∗ S = L∗ Q∗ QL∗ = L∗ L, we obtain the Cholesky decomposition of R, i.e. the lower triangular matrix L = F is in fact the Cholesky factor of R described in Section 4.4.8. In fact, Q∗ r = L−∗ L∗ Q∗ r = L−∗ S∗ r = L−∗ y = rWF according to (4.85). Thus the above manipulations using the QR decomposition are identical to the application of the noise whitening ﬁlter in Section 4.4.8. Recall that the noise whitening ﬁlter is a kind of dual to the decorrelator. The decorrelator whitens the data, at the expense of correlated noise, whereas the application of L−∗ to the matched ﬁlter output results in white noise, at the expense of correlated data. Equations (5.63)-(5.65) demonstrate that computing the Euclidean distance (5.65) based on the output of the noise whitened matched ﬁlter output results in no loss of optimality. The form of the metric (5.65) oﬀers a particular advantage, namely the lower triangular structure of F = L. Rather than performing similar manipulations to (5.60) above (where the 2-norm was expanded, and certain terms dropped), a recursive partial metric can be written directly in terms of the Euclidean metric as follows. The entire metric for a given data hypothesis d is 2

μnK (d, F) = rWF − Fd2 ⎛ ⎞2 nK l

⎝rWFl − = Flj dj ⎠ , l=1

j=1

5.2 Approximate Maximum Likelihood

161

which can be written recursively as ⎛ μl (d, F) = μl−1 (d, F) + ⎝rWFl −

l

⎞2 Flj dj ⎠ .

j=1

(5.66)

Branch Metric

Just like the Ungerb¨ ock metric λl (d), this new metric depends only on previous symbols, μl (d) = μl (d1 , d2 , . . . , dl ), and is therefore suitable for use in a tree or trellis search. However, unlike (5.62), the branch metrics ⎛ ⎞2 l

Flj dj ⎠ (5.67) b(d1 , d2 , . . . , dl ) = ⎝rWFl − j=1

are always non-negative. The complexity of noise whitening lies in the computation of the QRdecomposition, or Cholesky factorization, which is of order O(K 3 ) both in the synchronous and asynchronous cases. However, the resulting monotonic branch metrics allows for relatively low complexity extensions to the detection algorithm that can yield dramatic performance improvements. 5.2.2 Tree-Search Methods Optimal detection may be viewed as operating on a full D-ary tree of depth nK (assuming without loss of generality that each user transmits symbols from an identical modulation alphabet D). Starting from the root node, each node branches into |D| children, one for each possible transmitted symbol. Each node at depth l corresponds to a particular choice of data symbol hypothesis dˆl ∈ D. Let us label each node at depth l with the parˆ l = dˆ1 , dˆ2 , . . . , dˆl and the corresponding partial metric tial data sequence d ˆ ˆ ˆ μl (d1 , d2 , . . . , dl ). According to the recursive formulation (5.66), this can be computed by ˆ l ) = μl−1 (d ˆ l−1 ) + ˆl) μl (d b(d . Parent node metric

Branch metric

Thus the node metric is the sum of the parent node’s metric and the branch metric. The path from the root node to any of the |D|nK leaves corresponds to one particular hypothesis for the entire vector d. Optimal detection consists therefore of searching the entire tree and choosing the best sequence by comparing the corresponding metrics ˆ = μnK (d)

nK

l=1

ˆ l ). bl (d

162

5 Implementation of Multiuser Detectors

Example 5.1. Figure 5.9 shows a three-user symbol synchronous example, where each user transmits binary symbol. One path corresponding to ˆ = (−1, +1, +1) is highlighted. d

μ0 dˆ1

b1 (−1) μ1 (−1)

dˆ2

b2 (−1, +1) μ2 (−1, +1)

dˆ3

b3 (−1, +1, +1)

μ3 (−1, +1, +1) Fig. 5.9. Three-user binary tree.

One approach to complexity reduction is to limit the search space using a bounded tree, retaining a maximum number M (l) of partial sequences (and ˆ l at each step l. The partial sequences retained for the next hence nodes) d ˆ l ). The larger we choose step are those which minimize the partial metrics μl (d M (l), the larger the likelihood that the set of retained sequences will contain ˆ ML at the end of the frame, in which case the algorithm will deliver the same d decision as the optimal decoder. Since the tree grows starting from the root node, M (i) = min(2l , M ), where M is the search-breadth parameter. This algorithm is known in the coding community as the M -algorithm [120] and was ﬁrst applied to the multiuser detection problem in [158], where it was called the improved decorrelating decision feedback detector (IDDFD). This is summarized in Algorithm 5.1. The tree search algorithm is very similar to the algorithm we used to ﬁnd the minimum distance of a sequence set in Section 4.2.3.

5.2 Approximate Maximum Likelihood

163

Algorithm 5.1 (Improved decorrelating decision feedback detector). ˆ 0 = () of the Step 1: Initialize l = 1 and activate the root node d decoding tree and its metric μ0 = 0. ˆ l ) = μl−1 (d ˆ l−1 )+b(d ˆ l ) for Step 2: Compute the node metrics μl (d ˆ all active nodes, where b(dl ) is calculated according to (5.67). Step 3: Select the M (l) nodes with the best metrics and deactivate other nodes. Step 4: If l ≥ nt , where nt is some convenient truncation length as ˆ i−n from the node in the set of active in Section 4.2, output d t nodes at depth l which possess the smallest node metric λl as the estimated symbol at time l − nt . Let l = l + 1, and, if l ≤ nK go to Step 2. ˆ l ; nK − nt < Step 5: Output the remaining estimated symbols d l ≤ nK from the node with smallest node metric. Note that if we set M = 1, the M -algorithm is identical to the partial decorrelating decision feedback receiver from Section 4.4.8, and if we choose M = 2Kn , we have optimal decoding. Between these two extremes, the Malgorithm provides a complexity versus performance trade-oﬀ. Figure 5.10 shows the performance of a synchronous CDMA system with 20 users employing random spreading sequences of length 31. As can be seen in this case, the decoder approaches the single user bound with moderate values of M . Figure 5.11 also illustrates the near-far resistance of the IDDFD. This simulation was performed for the situation which presents the worst case for the original DDFD, i.e. , the ﬁrst user has the least power. Users 2–31 have equal power (x-axis of ﬁgure), and user 1 has 3dB less power. All users achieve their respective single-user bounds with use of the IDDFD. The codes used in this experiment were Gold codes [54] which possess low cross-correlation properties, and the system was fully loaded, i.e. β = 1 ⇒ K = L = 31. Finally, Figure 5.12 shows a comparison of the error performance of these diﬀerent multiuser decoding methods, plotted as a function of the system load for a CDMA system using random spreading codes for a value of Eb /N0 = 8dB. The IDDFD, as the most complex decoding algorithm, achieves the best performance and operates very close to the single-user bound for load β ≤ 2/3. The correlation receiver fails for system loads larger than about β = 0.1, while the DDFD shows an almost linear degradation of the bit error rate with system load. The results in Figure 5.12 were obtained for an equal received power scenario.

164

5 Implementation of Multiuser Detectors 1

10−1

Bit Error Rate

10−2

10−3

10−4

10−5 3

4

5

6

7

8

9

Ebb/N00

Fig. 5.10. Performance of the IDDFD for a system with 20 active users and random spreading sequences of length 31.

5.2.3 Lattice Methods An alternative framework to tree search methods is a class of search methods that operate within the framework of a lattice. Given a lattice basis L, an integer lattice is the set of all points {Lq : qi ∈ Z}. For simplicity, assume that the signal constellations D for all users are identical, with |D| = Q points such that they may be linearly transformed onto a set of consecutive integers, Q = αD + β = {0, 1, . . . , Q − 1}, and deﬁne q = αd + β1 We can now transform the jointly optimal detection problem into a problem of ﬁnding the lattice point closest to a given point. Starting with the output of the noise whitened matched ﬁlter, rWF = Fd + z β 1 = F (αd + β1) − F1 + z, α α

5.2 Approximate Maximum Likelihood

165

1

10−1

Bit Error Rate

10−2

10−3

10−4

10−5 3

4

5

6

7

8

9

Eb /N0

Fig. 5.11. Performance of the IDDFD under an unequal received power situation with one strong user.

and hence rWF +

1 β F1 = Fq + z. α α

β Deﬁning the translated whitened matched ﬁlter output, y = rWF + α F1 and the scaled lower triangular matrix H = F/α, results in

y = Hq + z. Thus the optimal detection problem (5.58) is equivalent to the constrained lattice search problem 2 min y − Hq2 . (5.68) q∈QnK

The Sphere Detector The objective of the sphere detector [42, 99, 122] is to ﬁnd all lattice points q ∈ ZnK within a given radius r of the received point y, namely > ? 2 Sr (y) = q ∈ ZnK : μ(q, H) = y − Hq2 < r2 .

166

5 Implementation of Multiuser Detectors 1

Average Bit Error Rate

10−1

10−2

10−3

10−4 11

66

11 11

16 16

21 21

26 26

31 31

Users Users

Fig. 5.12. Performance of diﬀerent multiuser decoding algorithms as a function of the number of active users at Eb /σ 2 = 7 dB.

Provided Sr (y)∩QnK is non-empty, this set contains the maximum likelihood point. Just like the tree search methods described above, the sphere detector makes use of the monotonic property of the μl (q, H) metric. Speciﬁcally, the condition μ(q, H) < r2 implies μl (q, H) < r2 for all l = 1, 2, . . . , nK. Considering user l, and using the recursive property of μl , this translates into μl (q) = μl−1 (q) + bl (q) ≤ r2 . Substituting for the branch metric,

5.2 Approximate Maximum Likelihood

⎛ μl−1 (q) + ⎝yl −

nK

167

⎞2 Hlj qj ⎠ ≤ r2

(5.69)

j=1

yl −

nK

Hlj qj ≤

r2 − μl−1 (q)

(5.70)

j=1

−Hll ql ≤ −yl +

nK

Hlj qj +

r2 − μl−1 (q)

(5.71)

j=1

⎞ ⎛ nK

1 ⎝ Hlj qj − r2 − μl−1 (q)⎠ yl − ql ≥ Hll j=1 (5.72) which gives a lower bound on ql in terms of the radius. Taking the negative square root in (5.70) results in a similar upper bound, and we have Ll (q1 , . . . , ql−1 ) ≤ ql ≤ Ul (q1 , . . . , ql−1 ) where ⎡

⎞⎤ ⎛ l−1

1 ⎝ Hlj qj − r2 − μl−1 (q)⎠⎥ Ll (q1 , . . . , ql−1 ) = ⎢ ⎢ Hll yl − ⎥ ⎢ ⎥ j=1 ⎢ ⎞⎥ ⎛ ⎢ ⎥ l−1

⎢ 1 ⎥ 2 ⎣ ⎝ ⎠ Hlj qj + r − μl−1 (q) ⎦ . Ul (q1 , . . . , ql−1 ) = yl − Hll j=1

(5.73)

(5.74)

Algorithm 5.2 describes the sphere detector algorithm, which is a depthﬁrst search of the full Q-ary tree of depth nK. Starting from the root node, the algorithm performs a “one-step look ahead” and discards (prunes) portions of the tree for which the bounds Ll or Ul are not met. The algorithm is parameterized by an ordering, or permutation function π : {0, 1, . . . , Q−1} → {0, 1, . . . , Q− 1}, which determines the order in which the branches of the tree are searched. Various choices of π have been suggested in the literature. The algorithm descends the tree until it either reaches a dead end, meaning all child nodes have been pruned (upon which it ascends one level), or it reaches a leaf node. Upon reaching a leaf node, the corresponding path is compared to the current best path. If it is better (has a smaller metric), the current best path is reset and the sphere radius is reset to the metric of the new best path. Thus the algorithm successively contracts the radius each time it ﬁnds a complete path, until there is only one full path inside the sphere, namely the maximum likelihood estimate.

168

5 Implementation of Multiuser Detectors

Algorithm 5.2 (Sphere Detector). Require: r2 > 0 1: ν ← ROOT 2: l ← 0 3: μbest ← ∞ 4: while l ≥ 0 do 5: if ν is a LEAF then 6: if μ < r2 then 7: r2 ← μ ˆ ← d associated with ν 8: d 9: end if 10: ν ← PARENT of ν 11: l ←l−1 12: end if 13: Compute node metric μl 14: Compute L and U according to (5.73) and (5.74). 15: Compute permutation π 16: Prune all CHILD nodes of ν with q < L or q > U . 17: if ν has remaining CHILD nodes then 18: ν ← CHILD of ν with smallest π(q). 19: l ←l+1 20: else 21: ν ← PARENT of ν 22: l ←l−1 23: end if 24: end while The choice of branch ordering π and initial radius r is critical to the computational complexity of the algorithm. If the radius is too large, or the ordering poorly chosen, the search sphere will be very slow to contract, and the algorithm may end up searching the entire tree. Conversely, the better the ﬁrst point, the faster the algorithm will converge. Fincke-Pohst enumeration [42, 99] uses the natural ordering π(q) = q. Fincke-Pohst enumeration can however be computationally ineﬃcient [1]. An improved ordering was given by [122], alternating about the midpoint between L and U . The Schnorr-Euchner enumeration has the useful property that the ﬁrst path found by the algorithm is the zero forcing decision feedback estimate (ZF-DFE) [1, 26]. Other improved orderings can be found in [47] and [168]. Since the Schnorr-Euchner enumeration ﬁnds the ZF-DFE point ﬁrst, the initial radius may be set to a maximum value r = ∞. In the case of FinckePohst, more care must be taken. Selection of a suitable radius is somewhat of an art, and is typically found in various heuristic ways.

5.2 Approximate Maximum Likelihood

169

The look-ahead bounds Ll and Ul are most eﬀective with high-order modulation. For small constellations, it may be more eﬃcient to not use these bounds, and to simply prune the tree based on the metrics μl alone. In the absence of these bounds, the resulting algorithm corresponds to a depth-ﬁrst T -algorithm, where the threshold T is reset each time a better complete path is found. Figure 5.13 shows the average bit error rate performance of the optimal detector (implemented using the Schnorr-Euchner sphere detector). We emphasize that there is no loss from optimality in these results, since the sphere detector is simply an implementation of the optimal detector. In this result, antipodal modulation and random spreading of length L = 32 is used. Figure 5.14 shows the corresponding average number of leaves (terminal nodes), i.e. candidate data vectors found by the Schnorr-Euchner sphere detector. Typically less than ten candidates are found on overage, compared to 23 1 for a K = 31 user system. In general, the complexity of the sphere detector is O |Q|K , since in the worst case, it may have to search the entire tree. In practice however, the portion of the tree searched depends upon the operating conditions, e.g. the signal-to-noise ratio. In practice, a limited search width may be enforced (similar to the M -algorithm). This would ensure a manageable upper bound on complexity, at the cost of a performance degradation.

10

10

0

-1

31 U

Average Bit Error Rate

Single 10

10

10

10

10

User

sers 24 U sers

-2

-3

-4

-5

-6

0

1

2

3

4

5

6

7

8

Eb /N0 [dB] Fig. 5.13. Sphere detector performance.

9

10

170

5 Implementation of Multiuser Detectors 11

Average number of leaves found

10 9 8 7

31 Users

6 5 4

24 Users

3 2 1 0

1

2

3

4

5

6

7

8

9

10

Eb /N0 [dB] Fig. 5.14. Sphere detector average complexity.

5.3 Approximate APP Computation In Section 4.2.2, the individually optimal detector computes the marginal a-posterior probability p(dl = d|r) that a particular symbol d ∈ D was transmitted at time i by user k according to

p(r|d)p(d), (5.75) p(dk [i] = d|r) ∝ d∈D nK :dk [i]=d

where p(r|d) is the conditional probability of the observation r, conditioned on the candidate transmit vector d, and p(d) is the prior probability. In most cases of interest, it is assumed that the individual transmit dimensions are independent, namely that p(d) =

nK #

pl (dl ) .

l=1

The sum (5.75) has |D|nK−1 terms, therefore brute force computation of the marginal posterior is impractical for all but the smallest of systems. Typically however, only a relatively small number of candidate vectors need be considered in order to generate a good estimate of the true APP. The problem is how to identify a suitable (hopefully small) list of vectors L(r, S) ⊂ DnK to marginalize over in order to produce an accurate APP

5.4 List Sphere Detector

171

estimate with reduced complexity. Thus the basic approach is similar to that taken in Section 5.2. In an additive Gaussian channel, the APP approximation (up to a normalization factor) is

e−Λ(r,d) (5.76) p(dl = d|r) ∝ d∈L dl =d

where 2

Λ (r, d) = r − Sd2 −

nK

ln pl (dl )

(5.77)

ln pl (dl )

(5.78)

l=1

= μnK (d, S) −

nK

l=1

In the log domain, the metric consists of the Euclidean distance term already encountered in joint ML detection 5.58, with a correction factor representing the prior probability.

5.4 List Sphere Detector In the absence of a-priori information (uniform priors), one approach is to only marginalize over the most likely symbol vectors. These are the vectors d falling within a sphere of radius r centered on the receive point r. A list sphere detector ﬁnds all vectors which satisfy 2

y − Hq2 ≤ r2 . In fact Algorithm 5.2 already performs this task. Instead of lines 6–9, which update the best path and contract the radius, the list sphere detector simply adds every complete path that it ﬁnds to a list L (and leaves the radius unchanged). At the termination of the algorithm, the list L consists of all paths corresponding to leaves reached by the algorithm. The radius r must be set large enough such that a good estimate of the posterior probabilities can be obtained from (5.76). Determining this radius is non-trivial, since even the desired list size may be unknown. However if the desired list size |L| is known, and is suﬃciently large, the volume of the desired sphere is approximately |L| times the fundamental lattice volume, and the corresponding radius is r= where

|L|| det H| VnK

1/nK

172

5 Implementation of Multiuser Detectors

VnK =

π nK/2 Γ (nK/2 + 1)

is the volume of a unit sphere of dimension nK. Since the goal of the list detector is to ﬁnd all points inside the sphere, the branch search order π is irrelevant. Any choice of π will result in the same list. In this case, the Fincke-Pohst natural order is a good choice, since it does not require any additional computations. The branch ordering is only important if it is only the joint ML vector that is required. 5.4.1 Modiﬁed Geometry List Sphere Detector The assumed independence of the prior probabilities in (5.78) allows a direct modiﬁcation of the branch metrics used in the recursive metric computation (5.66). Furthermore, since − ln pl (dl ) ≥ 0, the resulting recursive metric is still monotonic, μl (d, F) = μl−1 (d, F) + b(d1 , . . . , dl )

(5.79)

where ⎛ b(d1 , . . . , dl ) = ⎝rWFl −

l

⎞2 Flj dj ⎠ − ln pl (dl ) ≥ 0.

(5.80)

j=1

This new metric can be used directly in any tree-search algorithm, or it can be used to ﬁnd modiﬁed bounds Ll and Ul for use in a list sphere detector. It should be noted that the resulting sphere detector no longer ﬁnds all point within a Euclidean sphere. With non-trivial priors, the prior probabilities deﬁne a non-Euclidean geometry where the relevant sphere is deﬁned by Λ(r, d) ≤ r2 . This is the approach used by [151]. 5.4.2 Other Approaches There are several other approaches for APP approximation described in the literature. A dual-list approach has been proposed by [109, 110]. This maintains computes two lists. The ﬁrst list L1 is simply a list of the most likely sequences according to the Euclidean distance metric, which can be found using the list sphere detector of Section 5.4, or approximated using any restricted treesearch algorithm. The second list L2 is computed based only on the prior probabilities. The P -shortest paths algorithm (e.g. [36]) is used to ﬁnd the P vectors with greatest a-priori probability. This can be accomplished in

5.4 List Sphere Detector

173

O (K + P + (K + 1) log K). The ﬁnal list L = L1 ∪ L2 is found by taking the union of the two lists. Unlike the modiﬁed list sphere detector, this approach is not guaranteed to return the most likely vectors. The main advantage of this approach is found when used in an iterative decoding framework. The list L1 (which typically dominates the overall complexity via the QR-decomposition and list sphere detector/tree search) only needs to be computed once for each symbol, rather than for every iteration of the decoder. The second list L2 is re-computed each iteration, but this computation has very low complexity. An alternative approach is the LISS detector of [11, 59, 75]. In this approach a straightforward depth-ﬁrst (sequential) tree search is applied with a restriction on the total eﬀort, e.g. the total number of branches searched. With any such ﬁnite-memory depth-ﬁrst search there is no guarantee that even a single full path will be found. The idea of the LISS detector is to complete each partial path with soft decisions computed from the prior probabilities.

6 Joint Multiuser Decoding

6.1 Introduction In order to achieve rates that approach the information theoretic capacity of any communications channel, we need to use redundant signaling, referred to as forward error control (FEC) coding. Dating back to Shannon’s groundbreaking theory [123], it has been known that large FEC codes can approach the capacity of a channel virtually arbitrarily, and families of very powerful, capacity-approaching codes have been discovered (or rediscovered) over the last two decades. The most well-known code families are trellis codes [78, 120], turbo codes [120], and LDPC codes [120]. FEC thus forms an indispensable component of modern communications systems, and therefore also plays a fundamental role in multiple access communications. In this chapter we add FEC to the problem of multiuser communications and study how methods and designs are impacted. Our purpose is to illustrate the design and analysis of coordinated multiple-user systems which incorporate FEC, and gain understanding of the role of FEC in such systems. We concentrate our discussion on code-division multiple-access (CDMA) due to its practical importance and widespread use. Many of the results, however, apply to other systems as well, in particular to any linear multiple-access systems. Among these are multiple-input multiple-output (MIMO) channels which have recently gained prominence as multiple-antenna channels due to their potential to dramatically increase the information theoretic capacity of a wireless link [46, 132]. Two basic approaches are studied in this chapter. First, it is often the case that the single-user channel created by linear pre-processing exhibits a capacity which is close to the per-user capacity of the original multipleaccess channel, and therefore linear pre-processing is completely suﬃcient. This methodology has been studied in detail in Chapter 4. Decomposing the multiple-access channel into single-user channels via linear pre-processing has the advantage that basic error control methods designed for single-user AWGN

176

6 Joint Multiuser Decoding

channels can be used “oﬀ the shelf” to build a multiple-access system with FEC. One situation where pre-processing is suﬃcient is when the CDMA channel is only “lightly loaded”, i.e. when the ratio of users to signal dimensions is less than unity. On the other hand, in cases where optimal processing exhibits a signiﬁcant increase in capacity over linear pre-processing, more complex joint detectors will have to be used to harness the channel’s capacity. While optimal decoders can be formulated in many cases, they do not have much practical appeal since they usually require a prohibitively large complexity (for CDMA, see [52, 53]); nor do they hold much appeal to the theorist since their analysis is typically intractable. In the cases where joint detection is indicated, we focus on iterative receiver systems, whose genesis can be traced to the turbo principle invented for decoding of turbo codes [12]. We will see that these iterative receivers are very eﬀective and that their low complexity requirements are no match for other, more complex receiver structures. Figure 6.1 shows the information theoretic capacities for random CDMA systems using both a linear minimum mean-square error ﬁlter and optimal processing for two system loads. The two capacities for β = 0.5 show very little diﬀerence, and it is therefore “nearly optimal” to use a minimum mean square error ﬁlter as linear preprocessor which converts the CDMA channel into parallel single-user AWGN channels where well-known FEC methods can be used. This is the case for a system load β of approximately less than 0.5 in random CDMA channels. As the system load increases, linear preprocessing becomes increasingly less eﬃcient, even for equal power channels, and for highly loaded systems joint detection can provide signiﬁcant extra gains. When and where joint detection provides a capacity advantage depends on the particular system, but the capacity formulas discussed in Chapter 4 can be used to determine this. In general, systems with a small number of users per signal dimension do not proﬁt signiﬁcantly from joint processing. Figure 6.2 shows the system diagram of CDMA using FEC. We also assume receiver cooperation, i.e. the sources independently encode their information. Since the diﬀerent encoders in this model are assumed to be located at diﬀerent physical locations, transmitter cooperation in a one-way communication system like the one we consider here is not possible. The receiver, however, makes maximum use of the received signal by joint decoding. Figure 6.2 is an extension of Figure 2.1 where the incoming information bits have been labeled by uk [j] at time j for user k, and the interleavers (π1 , · · · , πK ) have been added between the FEC encoders and the channel. An interleaver is nothing more than a large permutation operation, where the symbols in the output blocks of length n are reordered according to some rule. Here, as is customary with turbo coding, this relabeling is assumed to be completely random. Some sophisticated rules in the construction of these interleavers have been successfully applied to turbo coding [120] to improve performance. However, in the context of the system in ﬁgure 6.2, the structure of the interleaver is

6.1 Introduction

177

10

AWGN Capacity

Capacity bits/dimension

optimal β=1

optimal β = 0.5

1 Linear β = 0.5 Linear β=1

0.1

-2

0

2

4

6

8

10

12

14

Eb /N0 [dB]

Fig. 6.1. Comparison of the per-dimension capacity of optimal and linearly processed random CDMA channels. The solid lines are for β = 0.5 for both linear and optimal processing, the dashed lines are for a full load β = 1.

less important, and has not been researched either. In fact, we will show that the performance of the system is determined primarily by the error control code. Random interleavers make achieving this performance possible and also allow for a precise analysis. If the FEC code already contains an interleaver as part of its internal structure, as is the case with turbo codes or LDPC codes, no additional interleaver is required between the FEC encoder and the channel. For reasons of analysis, we will however assume that the interleavers (π1 , · · · , πK ) are always there, and are of large size. We will see that depending on the system load, the role of the error control codes can be quite diﬀerent. For low loads, the property of the error control control codes to operate in noisy environments dominates, while for large load the dominant role of the error control codes is to resolve the multiple-access interference. We will see that in an iterative joint decoder the FEC codes have these two major functions of overcoming the multiple-access resolution

178

6 Joint Multiuser Decoding √

u1 [j]

u2 [j]

Error Control Encoder

Error Control Encoder

Interleaver

d1 [i]

π1

Interleaver

d2 [i]

π2

Baseband Modulator

√

P1 s1 [i]

P2 s2 [i]

Baseband Modulator

r[i] z[i] √

uK [j]

Error Control Encoder

Interleaver

πK

dK [i]

PK sK [i]

Noise

Baseband Modulator

Fig. 6.2. Diagram of a “coded CDMA” system, i.e. a CDMA system using FEC coding.

by assuring convergence of the iterations, and providing low error probabilities by overcoming the channel noise after cancellation. The following main strategies have been proposed for decoding coded multiple-access signals: 1. Optimal Decoding is an extension of the maximum likelihood sequence detection principle discussed in Chapter 4 to include the constraints imposed by the error control codes. The immediate and debilitating disadvantage of this approach is that the decoder state space, already large for uncoded optimal detection, is further increased by the product of the code state spaces of the diﬀerent users. Optimal decoding has been proposed and studied for convolutionally coded CDMA systems in [52, 53]. We will not further discuss optimal decoding for two reasons: Its practical importance is very limited due to its excessive complexity, and its derivation is a direct extension of the optimal uncoded detector (Chapter 4). 2. Linear Pre-Processing and Single-User Decoding is appropriate whenever the user density is relatively low. Diﬀerent ﬁlters can be used as pre-processors and are combined with conventional error control codes in single-pass decoding systems. For random CDMA systems, where the signal geometry changes at each symbol interval, the generation of a proper metric for the error control decoder is important [7, 115, 116]. This is discussed in the next section. 3. Iterative Decoders arise from the perspective of viewing a coded CDMA system as the concatenation of FEC codes with the CDMA channel. The idea is to use soft-output decoders for both the CDMA channel and the

6.2 Single-User Decoding

179

FEC codes in an iterative decoding arrangement. This methodology was introduced in [5, 88, 108] and further developed in [3, 33, 126, 157]. Moher [88] proposed the use of a channel a-posteriori probability detector, i.e. the channel “decoding” was performed optimally at the cost of a complexity of O(2K ). All other proposals used some form of reduced-complexity channel decoding, with Alexander et. al. [5] proposing simple, yet eﬃcient interference cancellation achieving excellent results. Wang and Poor [157] introduced the per-user MMSE ﬁlter, and Shi and Schlegel [125] provide an analysis of using linear cancellation techniques. We study these methods in subsequent sections of this chapter.

6.2 Single-User Decoding As discussed in Chapter 4, linear pre-processing, such as decorrelation and MMSE ﬁltering, convert the multiple-access channel into parallel channels with additive noise. Using central limit theorem arguments [101], it can be shown that the residual noise is well approximated by uncorrelated Gaussian statistics. These results formed the basis for the capacity calculations in Chapter 4. The use of error control codes now seems pretty straight forward. Since the two problems of decoding and multiple-access interference resolution have been decoupled. However, due to the fact the multiple-access channel may be time-varying, in particular for random CDMA, care must be taken in how the metrics for the error control decoders are derived. The simple optimal AWGN channel squared-distance metric λ = r − d

2

(6.1)

is not optimal in all situations. This will become clearer when we discuss the relationships between the decorrelator pre-processor and the projection receiver. 6.2.1 The Projection Receiver (PR) The projection receiver (PR) is a single-pass detection method which combines FEC and multiuser interference cancellation. The PR makes no hard decision on the transmitted symbols d, but instead works with sequence metrics λ(d) for the encoded bit streams. The PR does not actually perform joint decoding, but decoding with linear interference suppression. We start our derivation with the formulation of the maximum-likelihood (ML) detector, given in (4.2), i.e. ˆ ML = arg min r − SAd2 , d 2 d∈D Kn

(6.2)

180

6 Joint Multiuser Decoding

where we recall that DKn is the set of sequence candidates d taken from {−1, +1}, and constrained by the error control codes. The projection receiver reduces the complexity of this ML search by partitioning the symbols in d into two sets, d → {dc , du }. (6.3) The complexity reduction is now achieved by estimating du over the real (unconstrained) domain, and detecting only dc over the constrained domain, now denoted by Dc . This leads to the partitioned minimization 2 ˆ PR = arg min min r − SAd . (6.4) d dc ∈Dc du ∈ Kn

The constrained portion dc of the data can be any combination of transmitted symbols, but typically it would be the sequence of a single user, i.e. dc = dk = [dk [1], . . . , dk [n]], or that of a few “desired” users. The complexity savings of (6.4) over (6.2) are substantial. In the fully projected case, where dc contains the symbols of only one user, the savings over maximum-likelihood decoding are a factor of 2K−1 ! The minimization over du is quadratic and can be accomplished in closed form. Let us rewrite the inner minimization in (6.4) as min r − du

2 SAd2

@ @2 @ Ac dc @ @ @ = min @r − [Sc Su ] Au du @ du 2

2

= min r − Sc Ac dc − Su v2 , du

(6.5)

where we have partitioned the matrix S into the component Sc with the columns corresponding to the symbols in du removed, and Su which contains these columns. The diagonal matrix A is partitioned likewise. Furthermore we deﬁne v = Au du as the product of the unconstrained symbols and their amplitudes. We now take the partial derivative ∂ 2 r − Sc Ac dc − Su v = −2S∗u (r − Sc Ac dc − Su v) ∂v

(6.6)

which we set to zero to obtain ˆ = (S∗u Su )−1 S∗u (r − Sc Ac dc ) v

(6.7)

for the joint estimate of interfering symbols and amplitudes. Note that this is the decorrelator ﬁlter output of the unconstrained symbols with the hypothesis dc removed or canceled. Substituting the unconstrained estimate (6.7) into (6.5), the detected constrained symbols are now calculated as 2 dˆcPR = arg min MPR (r − Sc Ac dc )2 , dc ∈Dc

(6.8)

6.2 Single-User Decoding

where

−1

MPR = I − Su (Su ∗ Su )

Su ∗ = I − Pu .

181

(6.9)

The matrix MPR is the projection matrix onto the null space of Su , and Pu is the projection on the subspace spanned by Su hence the name “projection receiver”. Note that the amplitudes Au of the unconstrained users are not required for detection. This also makes the receiver near-far resistant, that is, its performance is independent of power levels of the interfering users. Furthermore, since the amplitudes can be complex, given diﬀerent phases of the diﬀerent user, the receiver also does not need phase knowledge of the interfering users in order to generate the metrics in (6.8), that is, the receiver is non-coherent as far as the interferers are concerned. These properties are shared with the linear pre-processing ﬁlters discussed in Chapter 4. The minimization in (6.8) can be further simpliﬁed by observing that MPR x ⊥ Pu x. This leads to 2 dˆcPR = arg min MPR r − Sc Ac dc + Pu Sc Ac dc dc ∈Dc > ? 2 2 = arg min MPR r − Sc Ac dc + Pu Sc Ac dc + −Pu Sc Ac dc dc ∈Dc

2

= arg min MPR r − Sc Ac dc ,

(6.10)

dc ∈Dc

where the addition of the second term in the second step does not change the result of the arg(·) function for binary modulations with dk [i] ∈ {−1, +1}, since this term is orthogonal to the ﬁrst and independent of the signs in dc . The projection front-end of the receiver now produces a metric 2

λ(dc ) = MPR (r − Sc Ac dc ) ≡ MPR r − Sc Ac dc

2

(6.11)

for a given codeword dc , and the detector will select as ﬁnal estimate the codeword dˆc for each user with the smallest metric λ(dc ), i.e. dˆcPR = arg min λ(dc ). dc ∈Dc

(6.12)

ˆ the bit estimates uc.PR ˆ are easily computed via the error control From dc,PR encoder inverse. If the projection metrics have to be computed for all users, it is more economical to approximate −1

MPR r ≈ r − Su (S∗ S)

S∗ r = r − Su R−1 rMF .

(6.13)

The error thus introduced is negligible, but this formulation nicely illustrates the method’s relationship to cancellation techniques and decorrelation as shown in Figure 6.3. Figure 6.3 illustrates that the PR does not need to obtain phase estimates of the interfering signals, the metric calculations and cancellations according

182

6 Joint Multiuser Decoding

4H[JOLK-PS[LY-YVU[,UK

s1 [i]

9LZL[

s1 [i]

P

v1

s2 [i]

s2 [i] P

v2 R−1

r[i]

sK [i]

sK [i] P

vK

-

7OHZLHUK-YLX\LUJ` :`UJOYVUPaH[PVU

:PUNSL
dk

Fig. 6.3. Projection Receiver block diagram using an embedded decorrelator.

to (6.11) and (6.13) can be performed without that knowledge, that is, noncoherently. Phase (and possibly frequency) synchronization only needs to be performed for the user of interest. Note that using the output vk directly as input to the single-user error control decoder corresponds to decorrelator pre-processing discussed in Chapter 4. We show later, that this shortcut can cause a loss in performance of random CDMA, and that the metric generation according to the projection principle is the “correct” way. 6.2.2 PR Receiver Geometry and Metric Generation Let us spend some time on developing a geometric view of the metric generation (6.11). The matrix (6.9) is the projection onto the space orthogonal to the space spanned by the columns in Su , i.e. onto Span{Su }⊥ . This is illustrated in Figure 6.4. The metric for a candidate sequence dc is produced by ﬁrst subtracting its eﬀect from the received sequence r, i.e. we generate r − Sc Ac dc . This requires knowledge of the amplitudes of the constrained user(s). The

6.2 Single-User Decoding

183

resulting vector is then projected by the matrix MPR . The length of the resulting projected vector is the metric λ(dc ). The receiver, in eﬀect, ignores the dimensions in signal space which are “contaminated” by the interference. This results in a shortening of the received signal vector (since MPR x ≤ x), which is equivalent to an energy loss. In Section 6.2.3 we will calculate this energy loss and learn that it is not severe, as long as the load β < 1 and not close to unity. It also becomes obvious now why the PR ideally is near-far resistant. Since it operates in the space orthogonal to the interference, the power levels of the interferers are irrelevant. Computing the metrics (6.11) for all possible sequences dc is a formidable task for asynchronous CDMA systems. In practical implementation, an iterative implementation of the inversion in Figure 6.3 would be preferred as discussed in Chapter 5. This iterative implementation is identical for both synchronous or asynchronous system (see Chapter 5). Orthogonal Complement r

Sc Ac dc r − Sc Ac dc MPR (r − Sc Ac dc ) Origin Span{Su }

Fig. 6.4. Projection Receiver diagram using an embedded decorrelator.

Alternately, the metric calculation can be performed in a recursive fashion. To this end let us partition MPR into L × L blocks. MPR is, in general, a full matrix, but its oﬀ-diagonal terms decrease rapidly with distance from the diagonal. This allows us to work with a block-diagonal approximation MPR,s retaining s of the oﬀ-diagonal blocks on either side of the diagonal. So, for example

184

6 Joint Multiuser Decoding

⎡

MPR,1

M11 M12

0

···

0 .. . .. . .. .

⎤

⎢ ⎥ ⎢M21 M22 M23 ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ 0 M32 M33 M34 ⎥. ⎢ ⎥ ⎢ . ⎥ .. ⎣ .. ⎦ . 0 ··· Mnn

(6.14)

Note that due to (6.9), Mij = M∗ji . In what follows we shall concentrate on the case where the constrained symbols are those of a single user, i.e. dc = dk . Writing out the sequence metric (6.10) in terms of individual symbol components we obtain @ @2 n @ @

@ @ Pk sk [i]dk [i]@ λ(dk ) = @MPR r − @ @ i=1 @ @2 n n @

@

@ @ = −2r∗ MPR Pk sk [i]dk [i] + @ Pk sk [i]dk [i]@ , (6.15) @ @ i=1

i=1

where the sequence energy term has MPR r has been discarded since it does not aﬀect the decoding decision. Since the spreading sequences sk [i] of user k are not overlapping in time, the second term in (6.15) is equal for all sequences as long as binary modulation is used, and can therefore be omitted. This simpliﬁes (6.15) further and we obtain an equivalent additive sequence metric n n

def λ(dk ) = λ(dk )[i] = −r∗ MPR Pk sk [i]dk [i]. (6.16) i=1

i=1

We now decompose r = (r[1], . . . , r[n]) into n partial vectors of length L, associated with each symbol transmission, and obtain λ(dk )[i] = −

i−1

j=i−s

i

r∗ [j] Pk Mji sk [j]dk [j] − r∗ [i] Mij Pk sk [j]dk [j]. j=i−s

(6.17) The decomposition of (6.16) into (6.17) follows analogous steps as those used in Section 4.2.1 to decompose the optimal metric. The above metric is reminiscent of the case of a single input single output with inter-symbol interference [104]. But in our case the interference between adjacent symbols is created by the asynchronicity of the CDMA channel in conjunction with the projection ﬁltering. For synchronous CDMA [7, 119] the oﬀ-diagonal elements of MPR vanish, i.e. Mij = 0 for i = j and the metrics in (6.17) simplify to (6.18) −λ(dk )[i] = r∗ [i]Mii Pk sk [j]dk [i]. Detection via (6.17) requires an extended sequence detector, except for synchronous CDMA. In the case of using an error control code with a trellis

6.2 Single-User Decoding

185

representation1 , this results in an augmentation of the code state space, similar to the case of coding for inter-symbol interference (ISI) channels [155]. The augmented code space will contain 2s times as many states as the original code state space. This is the price to be paid by the asynchronous nature of the channel. The augmented code state space can now be searched by any type of trellis search algorithm, for example the Viterbi algorithm. 6.2.3 Performance of the Projection Receiver In Section 4.4.4 we proved that the signal-to-noise ratio of the decorrelator output for random spreading sequences converges to the limiting value Pk L − K + 1 , (6.19) σ2 L as both K, L → ∞. For ﬁnite values of K and L, the the signal-to-noise ratio in (6.19) is not constant, which means that the decorrelator outputs cannot directly be fed into a standard error control decoder without incurring a loss, since in that case the decorrelator outputs are no longer ML metrics. This loss is caused by “surges” in the noise value, which distorts the decision statistics, i.e. diﬀerent symbols are not weighed equally. These surges can be thought of as caused by spreading sequence combinations during certain symbol intervals which cause particularly large noise enhancement by the decorrelator. This eﬀect is particularly noticeable for codes with small to moderate free distance. We will show in this section that the PR correctly compensates for this distortion, and that its performance can be superior to that of decorrelation. We approach this development by ﬁrst showing that the performance of the PR is lower bounded by that of a channel with the asymptotic signal-to-noise ratio (6.19), with equality as K, L → ∞. In a second step we then show that the performance of decorrelation is lower bounded by the performance of the PR. Simulation results will give quantitative examples of this eﬀect. In the sequel we prove the following theorem: SNRk =

Theorem 6.1. The performance of the PR with random spreading codes (synchronous or asynchronous) using the metric (6.11) suﬀers an energy degradation of L with respect to interference-free single-user performance, and L L ≥ 10 log10 [dB]. (6.20) L−K +1 This loss L is plotted in Figure 6.5 as a function of the system load β = K/L. We see that below 50% system load, the loss is less than 3dB. 1

We assume that an error control code which can be represented by a trellis is used. The most obvious such choice would be a convolutional code. However, block codes, too, have a code trellis [120] which can be used.

186

6 Joint Multiuser Decoding 16 14

Loss in dB

12 10 8 6 4 2 β

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 6.5. Lower bound on the performance loss of the PR.

The bound in Theorem 6.1 on performance loss is very tight, even for certain non-random spreading sequences, as shown by the simulation results presented Figure 6.6. The FEC code was chosen to be a rate R = 1/2 convolutional code with 4 states, and generator polynomials (5,7) [104, 120]. Simulation results are shown for K/L = 5/15, 8/15, and 10/15, leading to bounded loss factors of 1.3dB, 2.7dB, and 4dB, illustrated by the dashed lines in the ﬁgure. The error ﬂares that occur for the higher loads at lower bit error rates are mainly due to the fact that the inter-symbol interference was neglected in the decoder. For other classes of spreading codes, similar results can be obtained. For example, if Gold codes [54] are used as spreading sequences in a synchronous system, a loss of (L − K + 2)L [dB] (6.21) L = 10 log10 (L − K + 1)(L + 1) can be calculated [7]. Gold codes are nearly orthogonal, causing, i.e. a loss of only 0.39dB with K = 10, L = 15. Before we proceed with the proof of Theorem 6.1, we outline the results in this section. The reader not interested in the mathematical details of the proofs can thus skip these details. First, the bound in (6.20) becomes tight, not only as K, L → ∞, but also as the minimum Hamming distance of the code becomes large. The proof of this is analogous to the proof in Section 4.4.4 and is omitted. Next, we show in Lemma 6.1 that the performance of asynchronous CDMA is strictly poorer than that of synchronous CDMA, even though the

6.2 Single-User Decoding

187

1

Average Bit Error Rate

10−1

10−2

$V\QFKURQRXV35 XVHUV

10−3 6LQJOH8VHU 3HUIRUPDQFH

$V\QFKURQRXV 35XVHUV

−4

10

$V\QFKURQRXV 35XVHUV

10−5 2

3

4

5

6

7

8

Eb /N0

Fig. 6.6. Performance examples of the PR for random CDMA. The dashed lines are from applying the bound from Theorem 6.1. The values of Eb /N0 is in dB.

diﬀerence is negligible in most cases. Finally, in Lemma 6.2 we prove that decorrelation produces a larger error rate than using the PR metrics. This lemma holds both for asynchronous and synchronous CDMA. These results then show that the PR metrics (6.17) are the correct way of interfacing the multiuser pre-processor with the FEC decoder. These conclusions are valid also if other linear pre-processors are used, such as the MMSE ﬁlter. We now proceed with the somewhat tedious proof of Theorem 6.1. We approach the performance of the PR in terms of decoded bit error probability with random spreading codes. We ﬁrst address the degradation of the PR system relative to the interference free single-user performance in general terms, and then proceed to develop the bound (6.20). For mathematical convenience we assume unit received powers for all users, i.e. A = I, but note that the result is independent of the power distribution of the users. We follow the standard approach for calculating error probabilities in coded systems by considering the probability that an incorrect codeword dk of user k’s codebook is decoded. The resulting two-codeword error probability can then be used in standard distance spectrum analyses to ﬁnd bounds for convolutional or block codes [120].

188

6 Joint Multiuser Decoding

We must derive an expression for the probability Pw that the correct codeword dk , which shall be assumed to have Hamming distance w with respect to dk , has a larger metric. The Hamming-weight w error vector = dk − dk

(6.22)

has therefore w elements equal to ±2. We place the symbol indexes of these error positions into the set G. The two codeword error probability for such a pair-wise error event based on the PR metrics of (6.11) is given by Pw = P (λ(dk ) > λ(dk )) 2 2 = P MPR (r − Sk dk ) > MPR (r − Sk dk ) 2 2 = P MPR Sk dk −MPR Sk dk −2r∗ MPR Sk (dk −dk ) > 0 .(6.23) Using r∗ = d∗k S∗ + z∗ , and using (6.22) we obtain “ ” ‚2 ‚ Pw = P MPR Sk dk 2 − ‚MPR Sk dk ‚ − 2d∗ S∗ MPR Sk − 2z∗ MPR Sk > 0 ` ´ = P − MPR Sk 2 + 2z∗ MPR Sk > 0 = P (ηw > 0) .

(6.24)

The random variable ηw is Gaussian distributed as follows 2 2 ηw ∼ N − MPR Sk , 4σ 2 MPR Sk , which leads to the two codeword error probability ⎛8 ⎞ 2 MPR Sk ⎠ Pw = Q ⎝ . 4σ 2

(6.25)

(6.26)

This equation is analogous to the error equation for an AWGN channel, the only diﬀerence is the energy reducing projection MPR , and therefore, the projection receiver suﬀers an energy loss which depends on the spreading sequences used, which is quantiﬁed by (6.26). We will now break the argument of this error probability into a sum over individual symbol metrics. In order to obtain a bound, we will average Pw over all possible combinations of the error vector . This leads to the bound: ⎛A ; <⎞ B B E M S 2 PR k ⎜C ⎟ ⎟ E [Pw ] ≥ Q ⎜ (6.27) ⎝ ⎠ 4σ 2 ⎛8 = Q⎝

⎞ 2 M s [i] PR k i∈G ⎠. σ2

5

(6.28)

6.2 Single-User Decoding

189

√ The inequality is established by the fact that Q( x) is a ∪-convex function of x. Equality in (6.28) occurs if and only if MPR sk [i] ⊥ MPR sk [j];

∀i, j ∈ G.

(6.29)

This occurs, for example, if the CDMA channel is synchronous. We have thus established the following Lemma 6.1. The error probability of the PR with asynchronous CDMA is strictly larger than the error probability of the PR with synchronous CDMA. For the purpose of this lemma a CDMA system is called asynchronous if (6.29) does not hold for some i, j. Note, however, that with random CDMA the diﬀerence of error performance is typically very small. We now study the behavior of Pw for random CDMA. We will proceed analogously to the proof of Theorem 4.1 in Section 4.4.4. Let Q = [q1 , · · · , qn(L−K+1) ] be an orthonormal basis for span S⊥ ¯ = u of dimension u n(L − K + 1). In terms of Q, the projection matrix MPR is given by ⎡ ⎤ M11 · · · Mn1 ⎢ .. ⎥ = QQ∗ , (6.30) MPR = ⎣ ... . ⎦ M1n · · · Mnn where Q has dimension nL × u ¯, and the L × L sub-matrix Mij of MPR is given by Mij = Qi Q∗j , (6.31) where Qi has dimension L × u ¯ and is deﬁned as ⎡ row iL of Q ⎢ row iL + 1 of Q ⎢ Qi = ⎢ . ⎣ ..

⎤ ⎥ ⎥ ⎥. ⎦

(6.32)

row iL + L − 1 of Q Since Q∗ Q = Iu¯ tr(Q∗ Q) = u ¯

9

= tr

n

: Q∗i Qi

i=1

=

n

tr(Q∗i Qi ) .

(6.33)

i=1

The term tr(Q∗i Qi ) is a random variable, but from (6.33) we conclude that E [tr(Q∗i Qi )] =

u ¯ = L − K + 1. n

(6.34)

190

6 Joint Multiuser Decoding

In the synchronous channel Qi will have only L columns that are non-zero, and therefore tr(Q∗i Qi ) = L − K + 1 and (6.34) holds trivially. However, in the asynchronous channel the number of non-zero columns is increased due to the dispersion of the signal that is caused by the projection ﬁlter. Now from (6.26) and (6.30) ⎛ ⎞ A ⎜B 1

⎟ 1

∗ ⎜B ⎟ s∗k [i]Mii sk [i] + 2 i sk [i]Mij j sk [j]⎟ Pw = Q ⎜B 2 2σ ⎝C σ ⎠ i∈G

i∈G j∈G j=i

⎛ ⎞ A ⎜B 1

⎟ 1

∗ ⎜B ⎟ = Q ⎜B 2 s∗k [i]Qi Q∗i sk [i] + 2 i sk [i]Qi Q∗j j si,k⎟ , 2σ ⎝C σ ⎠ i∈G

i∈G j∈G i=j

(6.35) and using x∗ x = tr(xx∗ ), ⎛ ⎞ A ⎜B 1

⎟ 1

⎜B ⎟ tr(Q∗i sk [i]s∗k [i]Qi )+ 2 tr(Q∗i i sk [i]j s∗k [i]Qj )⎟ . Pw=Q ⎜B 2 2σ ⎝C σ ⎠ i∈G

i∈G j∈G i=j

(6.36) We can now proceed by evaluating the expectation over the channel, where √ we introduce the lower bound by moving the expectation inside the Q( ·) function:

6.2 Single-User Decoding

191

⎞ ⎛A

B

B 1 ⎟ ⎜B tr(Q∗i sk [i]s∗k [i]Qi ) ⎟ ⎜B σ 2 E ⎜B i∈G ⎡ ⎤⎟ ⎟ ⎜B B ⎟ E [Pw ] ≥ Q ⎜ ⎟ ⎜B

⎢ ⎥ 1 ⎜B ∗ ∗ ⎥⎟ ⎜B tr(Q s [i] s [j]Q ) + 2 E⎢ i k j j i k ⎣ ⎦⎟ ⎠ ⎝C 2σ i∈G j∈G i=j

⎛ ⎞ A ⎜B 1 u ⎟ 1

¯ (1) ⎜B ⎟ + 2 = Q ⎜B 2 E [tr(Q∗i i sk [i]j sk [i]Qj )]⎟ n 2σ ⎝C σ ⎠ i∈G

i∈G j∈G i=j

⎞ ⎛8

u 1 ¯ ⎠ = Q⎝ σ2 n i∈G

9D = Q

P (L − K + 1) σ2

: (6.37)

√ where the inequality follows from the convexity of Q( x) in x and an application of Jensen’s inequality. Equality above in (1) results from the fact that E [sk [i]s∗k [i]] = I, that Qi and sk [i] are statistically independent with E [sk [i]] = 0, and from E [sk [i]s∗k [j]] = 0 for i = j. We note that the probability of an error depends only on the Hamming weight w of the error path of the constrained user, and, furthermore, that the degradation is identical for the synchronous and the asynchronous channel. Together with the fact that 2 Pw = Q P/σ , Theorem 6.1 is established. This last result justiﬁes our extensive treatment of the PR receiver: Theorem 6.2. The performance using the PR metrics (6.15) is strictly better than the performance using the distance metrics of the output of the decorrelator, for both synchronous and asynchronous random CDMA. Proof. First we introduce an upper bound on bit error probability (6.26) of the√PR by lower bounding the numerator in the argument of the function Q( ·), given by @ @2 @ @2 @ @

@ @

@ @ @ @ λ(dk ) = @MPR sk [i]i @ ≥ @ Mi sk [i]i @ , (6.38) @ @ @ @ i∈G

i∈G

⊥

where Mi is the projection onto span{S/sk [i]} , i.e. onto the subspace which is the complement of the space of all the spreading sequences without sk [i].

192

6 Joint Multiuser Decoding

This is true because MPR sk [i] ≥ Mi sk [i] ,

(6.39)

since the space onto which Mi projects has fewer dimensions, and ⎛8 ⎞ 5 2 i∈G Mi sk [i] ⎠ Pw ≤ Q ⎝ . σ2 On the other hand, using the following block matrix description of ∗ s s s∗ S R = c∗ c c∗ u , Su sc Su Su

(6.40)

(6.41)

letting sc = sk [i], and applying the partitioned matrix inversion lemma, we obtain for the decorrelator output at time i vk [i] =

s∗k [i]Mi r Mi sk [i]

2,

(6.42)

which is Gaussian distributed with mean and variance 2 vk [i] ∼ N 1, σ 2 Mi sk [i] .

(6.43)

This leads to the following two-codeword error probability if symbol-wise decorrelation is used: 98 : P2 (6.44) Pw,DEC = Q 5 2 σ 2 i∈G Mi sk [i] Due to the fact that the function 1/x is ∪-convex, 1

2 Mi sk [i] ≥ P i∈G

and therefore

⎛8 Pw,DEC ≥ Q ⎝

1 P

5 i∈G

1

2,

Mi sk [i]

⎞ 2 M s [i] i k i∈G ⎠ ≥ Pw σ2

(6.45)

5

(6.46)

for any two codewords. This proves our theorem.

6.3 Iterative Decoding Iterative decoding results from interpreting the joint multiple-access system with error control coding in Figure 6.2 as a serially concatenated coding system [120], in which the error control codes take the role of outer codes and the

6.3 Iterative Decoding

r1

CDMA

r

r2

Filter

Deinterleaver

w1∗

π1−1

Filter

Deinterleaver

w2∗

Interference

π2−1

Soft Output FEC Decoder

Interleaver

Soft Output FEC Decoder

Interleaver

Soft Ouput FEC Decoder

Interleaver

193

π1

π2

Resolution Function

rK

Filter

Deinterleaver

∗ wK

−1 πK

πK λ(dK )

tanh( //2)

λ(d2 ) tanh( /2)

λ(d1 ) tanh( /2)

Fig. 6.7. Iterative multiuser decoder with soft information exchange.

multiple-access channel takes the role of the inner code. With this interpretation, a soft information exchange decoding algorithm, similar to the turbo decoding algorithm, suggests itself, and has proven very eﬀective. A diagram of such an iterative multiuser decoder is shown in Figure 6.7.

The error control decoders are soft-output a posteriori probability (APP) decoders [120] which generate the a posteriori probabilities of the individual transmitted coded bits, i.e. Pr(dk [i] = 1|rk ); Pr(dk [i] = −1|rk ),

(6.47)

which are typically calculated as the symbol-wise log-likelihood ratio (LLR) Pr(dk [i] = 1|rk ) . (6.48) λ(dk [i]) = log Pr(dk [i] = −1|rk ) The interleavers π1 , · · · , πK are assumed to be large enough to destroy any correlation between diﬀerent LLR values. This independence assumption is

194

6 Joint Multiuser Decoding

very important for the statistical analysis of the detector, as well as for its successful operation. Figure 6.7 shows the canonical form of an iterative multiuser decoder. Different receivers mainly diﬀer in the type of FEC codes used, in the soft-output decoding algorithms applied, in the CDMA interference resolution function employed, and, in the loop ﬁlters wk∗ used. We will see in the next sections that a cancellation front-end is not only an attractive low-complexity solution, but, in conjunction with diﬀerent received power levels or code rates, can approach the capacity of the multiple access channel. 6.3.1 Signal Cancellation The interference resolution block has the function of extracting soft estimates from the received vector r using as input soft estimates of the coded symbols from the soft-output error control decoders. A possible way of doing this is by calculating the APP probability of the coded symbols, this time using the channel constraints, i.e. from Chapter 4, equation (4.15), we calculate

dˆk [i] = arg max Pr(r|d)Pr(d), (6.49) d∈D Kn

d:dk [i]=d

where Pr(r|d) is the multiple-access channel transition probability and is Gaussian for CDMA, and Pr(d) is the vector of a priori probabilities of the transmitted symbols learned from a previous iteration. Equation (6.49) represents the “optimal” APP detector proposed by Moher [88]. Unfortunately, even a cursory analysis shows that the sum has an exponentially growing number of terms in K and n, and becomes thus quickly unmanageable, and is therefore not of much practical interest. Guided by information theoretic thinking, or simple intuition, one can argue that the next best thing to do is to cancel the interference √ for each of the individual users by subtracting an appropriate estimate Pl d˜l [j]sl [j] of the interference caused by other users. This operation generates a new received (2) sequence rk for user k as (1)

Ik

(2)

rk

n n

K

= Pk dk [j]sk [j] + Pl dl [j] − d˜l [j] sl [j] +z j=1

=

j=1 l=1 (l=k)

n

(1) Pk dk [j]sk [j] + Ik + z.

(6.50)

j=1 (i)

(i−1)

Note that the received sequence rk as well as the interference Ik change with each iteration i, denoted by the superscripts. We will usually omit these

6.3 Iterative Decoding

195

superscript unless they are speciﬁcally needed. The received signal of the k-th user rk is distorted by additive noise and the interference Ik . Central limit theorem arguments quickly establish that this interference, which results from incomplete cancellation of the interfering users’ signals is Gaussian – see below. ∗ The ﬁlters w1∗ , · · · , wK are receiver loop ﬁlters inserted in the processing chain of each of the users. In the absence of multiple-access interference these ﬁlters would simply be discrete signal matched ﬁlters. The question now turns to how to generate the estimate of interference. Clearly, if we know precisely all of the interfering symbols, perfect cancellation is theoretically possible. However, the other APP decoders are not, initially, capable of generating exact estimates. We therefore attempt to minimize the error of the estimate dl [j] − d˜l [j] between actual and estimated symbols in the mean square sense. This is accomplished by using as estimated symbols the conditional expectations 7 7 λ(dl [j]) , (6.51) d˜l [j] = E dl [j]77λ(dl [j]) = tanh 2 which leads to the tanh(·) functions to generate the soft-bits d˜l [j]. These softbits will minimize the variance of Ik given the locally optimal probability estimates from the code APP decoders. The received signals rk are now considered to be distorted only by Gaussian noise, and matched ﬁlters wk∗ [j] = s∗l [j] are therefore the optimal receiver ﬁlters. 6.3.2 Convergence – Variance Transfer Analysis Using random CDMA, the interference is proportional to the ratio of users to processing gain as shown in Chapter 4. Furthermore, the interference is proportional to the variance of the estimation error 2 vd,l = E dl [j] − d˜l [j] . (6.52) Assuming equal power for all users Pk = P , all estimation error will be equal vd,l = vd . Further, using matched receiver ﬁlters, i.e. wk∗ [j] = s∗k [j], leads to a particularly simple expression for the variance of the residual interference plus noise at the input to the error control decoder, given by vIC (vd ) = σ 2 +

K −1 P vd L

K,L→∞

=⇒

σ 2 + βP vd ,

(6.53)

which is intuitive: there are K − 1 interferers for user k, each being suppressed by the processing gain L. The APP decoder estimation error variance vd is the average power of the residual interference. Therefore, given an average residual symbol interference variance vd , equation (6.53) describes the noise variance in the signal rk for the next iteration

196

6 Joint Multiuser Decoding

Single-user noise level vIC

through the FEC decoders. For matched ﬁlters in the loop this variance transfer curve (VT) is a linear function in the system load and the residual symbol variance, and is conveniently visualized by the following Figure 6.8.

slope: system load

channel noise level

vd = 0

vd : cancelled symbol variance

vd = 1

Fig. 6.8. Soft cancellation variance transfer curve.

The more accurate the symbol estimates of the interferers, the smaller the residual noise that the error control decoder has to overcome. Note that even if the interfering symbols are exactly known, vd = 0, there is still the AWGN noise which the error control decoder has to overcome. The idea of iterative decoding is, that through iterations, the variance on all the single-user channels can be improved until complete cancellation of all interference is achieved. In order to understand this process, we need to gain understanding of the operation of the soft-output FEC APP decoders. The input sequence rk to the k-th APP decoder has an additive Gaussian noise distortion with associated variance vIC per symbol at the input of the decoder, and the decoder analysis can be handled analogously to an AWGN channel. The output of the decoder are the soft-bits (6.51), and the primary measure of their reliability is the variance vd . While much numerical analysis [28, 133] seems to indicate that the LLRs (6.48) are Gaussian distributed with good accuracy, the soft-bits are not Gaussian distributed. We will calculate their distribution later assuming that the LLRs are Gaussian. However, since in the limit of many users, the number of terms in Ik is large, Ik is still

6.3 Iterative Decoding

197

Input Noise Variance vIC

asymptotically Gaussian by central limit theorem arguments, and therefore vd is a suﬃcient measure of the reliability of d˜l [j]. Unfortunately, there exists no closed form expression for vd as a function of vIC other than for very simple codes, such as the rate [2,1,2] parity check code. The variance transfer (VT) curves vd = g (vIC ) must therefore be found numerically for all codes of interest. Figure 6.9 shows such VT curves for a selection of low-complexity FEC codes. Note that for reason which will soon become clear, the input vIC is plotted along the vertical axis.

Soft-bit variance vd = g (vIC ) Fig. 6.9. Code VT curves for a selection of low-complexity FEC codes.

The operation of an iterative receiver now carries out in the following way: Canceled signals rk for each user are generated according to (6.50) and passed to the FEC APP decoders. They, in turn, generate soft estimates of the coded signals which are used to generate interference estimates to be used in a new cancellation iteration. If the interleavers used in the system are large enough to ensure that the implicit assumption of independence between signals is accurate, the process of iterative cancellation and decoding can be captured by a variance transfer chart shown in Figure 6.10 for the rate R = 1/3, ν = 3 convolutional code and a system load of β = 3. Decoding starts at point (0) where the FEC decoders have to work with the full interference and noise. Nonetheless, after one iteration, the system manages to reduce the interference by subtracting estimates of the interfering signals, this leads to point (1), whose vertical component is the now reduced

vIC

6 Joint Multiuser Decoding

vIC

198

3

Rate 1/3 CC

Interference Limitation

2.5 2

(0)

K = 30

L = 10 Eb /N0 = 10dB

(1) (2)

1.5 1 0.5 Noise Limitation 0 0

0.2

β=3

0.4 0.6 0.8 Soft-bit variance vd

1 vd

Fig. 6.10. VT chart and iteration example for a highly loaded CDMA system. FEC code VT is dashed, the cancellation VT curve is the solid curve.

noise variance at the input of the soft-output error control decoder. In the next iteration, the noise variance can be further reduced to point (2), and so on, until the iterations reach the intersection point between the two curves, the solid point in the enlargement. At this point virtually all the interference has been canceled, and only channel noise is left. This intersection point is a ﬁx point of the one-dimensional iterated variance transfer map, which models the behavior of the iterative decoder. The performance of the individual decoders at that point is essentially that of the decoders in Gaussian noise, and we call it the noise limitation of the iterative decoder. Note that along the iterative trajectory there is an section which forms a narrow channel through which the trajectory progresses in small steps. This area is the interference limitation. In this example, a small increase in the system load will create another intersection point between the two VT curves, and the variance will stop improving at that new ﬁx point, rather than at the noise limitation ﬁx point. The decoder does not function due to an excessive system load. As the load decreases, or the signal-to-noise ratio increases, the channel opens up and convergence to the noise limitation ﬁx point is suddenly enabled. This eﬀect happens at a very sharp signal-to-noise threshold, and gives rise to the abrupt, cliﬀ-like behavior of the error rates in such systems. Figure 6.11 shows the bit error curves for the system whose VT curves are shown in Figure 6.10. Clearly visible is the sharp onset of the drop in

6.3 Iterative Decoding

199

error rates as the signal-to-noise ratio is increased suﬃciently to open the convergence channel and allow the decoder system to progress beyond the interference limitation. At suﬃcient signal-to-noise ratios the bit error rates closely follow those of the FEC code operated in additive noise alone, this is the noise limitation and is only a function of the FEC code used.

1 1 Iteration

5 Iterations

10−1 I 10 ter

s

ration

Bit Error Rate

s on

ati s

ation

10−3

20 Ite

Sin gle Use r

er 15 It

30 Iterations

10−2

10−4

10−5 2.5

3

3.5

4

4.5

5

Eb /N0

Fig. 6.11. Illustration of the turbo eﬀect of iterative joint CDMA detection.

The question now poses itself as to what FEC code should be used for eﬃcient communication. Examining stronger error control codes, such as turbo codes or LDPC codes will be beneﬁcial in the noise limitation area, where the single-user performance of the codes comes to bear. Figure 6.12 shows the VT curves of a selection of powerful error control codes, several rate R = 0.5 LDPC codes at their ﬁrst iteration and at iteration 20, as well as two rate R = 1/3 serially concatenated turbo codes, whose rates and component codes are given in the following table. All of these codes have good to excellent performance on AWGN channels, and their VT transfer curves all show a general trend. The more powerful the code, the more abrupt the transition between unreliable and reliable code output behavior.

200

6 Joint Multiuser Decoding

Inner Code Code 1

g1 = "

Code 2 g1 =

10 01

Outer Code

1 1+D 1+D 2 1+D+D 2 1+D 1+D+D 2

g2 =

h i 1+D 2 , 1+D+D 2+D 3 , 1+D+D 2+D 3

Ri Ro 1 1/3

# rate 1/2 repetition code

2 3

1/2

Table 6.1. Serially Concatenated Codes of total rate 1/3 whose VT curves are shown in Figure 6.12. For details on serial concatenated turbo codes, see [120, Chapter 11].

In fact, for very large LDPC or turbo codes, which exhibit a sharp threshold signal-to-noise ratio, the following upper bound on the code variance transfer function is quite tight: 1 if vIC ≥ τ Pk vIC 2 (6.54) ≥ σd ≥ g Pk 0 if vIC < τ Pk , where 1/τ is the signal-to-noise ratio at which the original code has its convergence threshold for an additive-white Gaussian noise channel. This bound on the VT curve will allow us to derive optimal rate and power distributions in multiple-access systems where either rates or power levels can be chosen to optimize system eﬃciency. This will be done in Section 6.5.2. Figure 6.13 shows an iteration example of the serially concatenated code SCC 2 for a load of β = K/L = 23/10 = 2.3 at a signal-to-noise ratio of Eb /N0 = 10dB. The achieved performance in terms of spectral eﬃciency is inferior to that achieved by the weaker convolutional code (Figure 6.10). In both cases convergence of the iterations to low error rates is guaranteed as long the VT curves of the cancellation function and the FEC APP decoder do not intersect at the interference limited point. In order to ﬁnd the limiting performance of a given system in terms of system load and spectral eﬃciency, one increases either the noise level or the system load β until the cancellation VT line becomes tangential to the FEC decoder VT curve. This is illustrated in Figure 6.14 for the two FEC codes discussed and for Eb /N0 → ∞. From Figure 6.14 one can read oﬀ the limiting performance values of the supportable loads of β = 2.7 for the concatenated system, and β = 3.5 for the convolutional code. In order for the system to overcome the convergence bottleneck created at the interference limitation intersection point, it is desirable that the FEC code VT curve has a steep slope throughout to closely match the steep slope of the cancellation VT curve caused by a high β. This puts the stronger code, such as the LDPC and turbo code at a disadvantage to handle high system loads, since their VT curves have shallow slopes right at their turbo

201

vIC

6.3 Iterative Decoding

3 (2,4) regular LDPC Serial Concatenated Code 2

2.5

Serial Concatened Code 1

2

1.5

(3,6) regular LDPC (4,8) regular LDPC

iteration 20

1 0.5 irregular LDPCs iteration 1

0 0

0.2

0.4

0.6

0.8

1 vd

Fig. 6.12. Illustration of variance transfer curves of various powerful error control codes, such as practical-sized LDPC codes of length N = 5000 and code rate R = 0.5, as well as two serially concatenated turbo codes.

cliﬀ noise variance level. Granted, for low loads, these same turbo codes can overcome interference and high noise, however, we have seen in Chapter 4 that for low loads (β ≤ 0.5), no iterative detection is needed, since linear ﬁlters preserve most of the channel capacity. The use of strong error control codes turbo in iterative receivers for highly loaded CDMA systems may be counter-productive. The error behavior of such iterative decoders follows the typical pattern of turbo codes, with a sharp error cliﬀ which happens at the threshold signalto-noise ratio, above which error rates rapidly drop to very low values, before approaching the single-user error curve. This error ﬂoor domain is completely determined by the error behavior of the error control code in additive white Gaussian noise. While an individual iteration has a very low complexity, the number of iterations increases rapidly as the cutoﬀ load, or signal-to-noise ratio, is approached. Figure 6.15 shows the performance of the receiver for the code SCC2 from Table 6.1 as a function of the number of iterations. This code has a cutoﬀ signal-to-noise ratio of Eb /N0 = 1.3dB where the VT curves become tangential for a load of β = 1.

202

6 Joint Multiuser Decoding

Rate 1/3 Serially Concatenated Code (SCC 2)

Cancelled MU Variance vIC

K = 23, L = 10 Eb /N0 = 10dB

β = 2.3

Soft-bit variance vd Fig. 6.13. VT transfer chart and iteration example for a highly loaded CDMA system using a strong serially concatenated turbo code (SCC 2).

6.3.3 Simple FEC Codes – Good Codeword Estimators The diﬃculty faced by the FEC codes in an iterative system, and the reason why weak codes perform better is related to how well the soft-output decoder can estimate the transmitted codewords, and not to how well the decoder can decode the information sequences contained in these codewords. In order to overcome the multiple-access interference, accurate estimation of the transmitted codewords is required, such that the mutual interference can be reduced eﬀectively, while for low-error detection, good error control properties are required, such as large distances between codewords. These two requirements are largely contradictory, as can be seen by the following intuitive arguments. It is easy to construct a code which allows for perfect codeword estimation, this code would simply map each message into the same codeword, or into a set of codewords with very small Euclidean distances. The decoder will then always be able to estimate the transmitted codeword with essentially arbitrary accuracy, and thus cancellation will be very accurate. That is, the VT transfer characteristics of such a code would be a horizontal line that coincides with the y-axis. The problem obviously is that such a code cannot produce a low decoded bit error rate, even after perfect cancellation of all interference; for that to be possible we need codes that map the the information sequences

6.3 Iterative Decoding

203

5DWHVWDWH &RQYROXWLRQDO&RGH

Cancelled MU Variance vIC

&DQFHOODWLRQ97 ORDG

&DQFHOODWLRQ97 ORDG

6&&

Soft-bit variance vd Fig. 6.14. Determination of the limiting performance of FEC coded CDMA systems via VT curve matching for Eb /N0 → ∞.

into codewords as far apart from each other as possible, such that the noise spheres around each codeword do not overlap as required by Shannon’s theory (see i.e. [167] for geometrical arguments and error bounds). The practical codes whose VT curves we have presented in Figures 6.10 and 6.12 are representatives of these two groups of error control codes, and the weakest code, a simple parity check code, achieves the highest system load, but requires substantial signal-to-noise ratio for acceptable bit error rates. As we will see later, contrary to linear ﬁlter detection, the equal power, equal rate situation represents a worst case scenario for an iterative cancellation receiver, since neither power nor rate distributions can be used to maximize spectral eﬃciency, as will be done in Section 6.5.4. As discussed, weaker error control codes, which exhibit a steeper VT curve than stronger codes, achieve a signiﬁcantly higher system load β. In a “weak” code, however, the distinction between interference limitation and noise limitation is blurred, making it more diﬃcult to determine the maximal supportable system load. Consider Figure 6.16, which shows the VT curve of a R = 1/3 repetition code. The distinction between interference and noise limitation has vanished, and low BER performance is not possible due to the weak nature of the repetition code, quite independently of the load. This leads to a new view, which is to

204

6 Joint Multiuser Decoding 1

Average Bit Error Rate

Eb /N0 = 1.3 dB 10−1 Eb /N0 = 1.4 dB 10−2 Eb /N0 = 1.45 dB

Eb /N0 = 1.5 dB

10−3

10−4 1

5

10

15

20

25

30

Iterations

Fig. 6.15. Bit error performance of SCC2 from Table 6.1 as a function of the number of iterations.

consider the iterative cancellation system using a weak error control code as a non-linear separation ﬁlter, rather than a detector. The intersection point which corresponds to the ﬁx point of the iterative system equation, corresponds to a certain variance in the log-likelihood ratio of the decoded information bit. (This is not the value of the x-coordinate of the intersection point, which is the variance of the coded bit). The LLR of the information bit, furthermore, is Gaussian distributed, and we therefore have a binary-input, Gaussian-output channel between input information bits and their corresponding output information bit LLR values. The channel is binary since we assume a BPSK modulation of our CDMA system, for other modulation formats, we would obtain the corresponding discrete-input, Gaussian-output channel. The iterative decoder now acts as a non-linear ﬁlter, suppressing interference, but not completing the decoding operation due to the weakness of the repetition code. The system can also be viewed as a non-linear signal layering system (compare Chapter 4). Its spectral eﬃciency is then given by P C = βRCB (6.55) 2 (β, C) , σllr where CB (SNR) is the capacity of the binary-input channel with signal-tonoise ratio SNR. This is so since these output binary LLR can be understood as noisy versions of the transmitted data bits, and by Shannon’s theorem, 2 they can carry at most a rate given by (6.55). The ﬁnal output variance σllr

6.3 Iterative Decoding

205

3.5 Rate R = 1/3 Repetition Code 3

2.5

vIC

2

1.5

1

2 , vd ) Intersection Point: (v∞

0.5

0 0

0.2

0.4

0.6

0.8

1 vd

Fig. 6.16. Interference cancellation with a weak rate R = 1/3 repetition code, acting as non-linear layering ﬁlter.

of the (Gaussian) log-likelihood ratio of the data bits at the decoder output corresponds to that achieved at the iteration ﬁx point. The (ﬁnal) output 2 signal-to-noise ratio P/σllr does depend on the system load β as well as on the particular code C used, as discussed above. In general it is very diﬃcult to calculate the variance transfer function of a given code, and this makes optimization over the codes C in (6.55) a diﬃcult task as well. The VT functions of typical error control codes, like the ones shown in Figure 6.12, are found via numerical methods by simulating the input-output behavior of the FEC code. However, the output extrinsic LLR of any one of the coded bits in a length-N repetition code at iteration m is simply

(m) 2 (m) λ(E) (dk [j]) = λ yk [i] = rk [i], (6.56) vd i=j

i=j

where yk [j] = sk [j]r∗ k [j] is the loop matched ﬁlter output, and the information bit LLR can be calculated as (m)

(m)

206

6 Joint Multiuser Decoding

λ(u) =

N

(m) λ yk [i] .

(6.57)

i=1

From the extrinsic LLR values λ(E) (dk [j]) of the coded bits the decoder computes the soft bits d˜k [j] = tanh(λ(E) (dj [i]/2)) used for cancellation. Decoding is very simple since most of the complexity lies in the tanh(·) function. The PDFs of both the a priori and extrinsic LLRs are Gaussian due to (6.56) and (6.57) with variance 2 vE =

vd P (N − 1)

and

2 σllr vd = P PN

(6.58)

respectively. However, the PDF of the soft-bits d˜k [j] is not Gaussian. It can be calculated as 2 1 1+d˜ 2 exp 2v2 − log 1−d˜ − v2 E E ˜ k [j] = 1) = 2 (6.59) p(d|d 2πv 2 1 − d˜2 E

via standard random variable transformation. Applying the central limit theorem to the large number of terms in (6.50) as K → ∞ and the fact that ˜ var dk [j] ≤ 1, the residual interference term Ik is converging to a Gaussian distribution with variance βvd , where 1 2 ˜ k [j] = 1) 1 − d˜ = gd (v 2 ). vd = p(d|d (6.60) E −1

Equation (6.60) needs to be evaluated by numerical methods. Using (6.60) and (6.58) leads to the following lemma: Lemma 6.2. The system spectral eﬃciency in (6.55) using repetition codes is maximized for N → ∞. 2 2 Proof. Let the point (x, y) = (g(v∞ ), v∞ ) be the intersection between the load 2 line σ + βvd and the variance transfer curve g(vd /P ) of the repetition code, i.e. (−1) βx − σ 2 = gd (x). (6.61)

The spectral eﬃciency of the layered channels is then given from (6.55) as 1 2 CB (N P/v∞ ) N g (−1) (x) − σ 2 1 NP = CB x N vd 2 N − σ2 1 (N − 1)vE 2 = CB v , 2) gd (vE N N −1 E

C=β

which can easily be shown to be monotonically increasing with N .

(6.62)

6.3 Iterative Decoding

For the repetition code we obtain from (6.56) ⎞ ⎛ v

P d = tanh ⎝ rk [i]⎠ g P vd i=j ⎞ ⎛ 8 (N − 1)P (N − 1)P + ξ⎠ , = tanh ⎝ vd vd

207

(6.63)

where ξ ∼ N (0, 1). Now 8

1 0 g(b) = E 1 − tanh b2 + bξ ;

b=

(N − 1)P . vd

(6.64)

For our further development, we will make use of some fundamental approximations, given in the following Lemma 6.3. This lemma consists of two parts: i) The function g(b), b ≥ 0 is monotonically decreasing, and for any b > 0 the following relation holds ∞

0 12 2 2 2 = 4e−b /2 (−1)n eb (2n+1) /2 Q [b(2n + 1)] , E 1 − tanh b2 + bξ n=0

(6.65) √ E ∞ −u2 /2 du is the Gaussian error function. where Q(x) = 1/( 2π) x e ii) For any b ≥ 0 the following inequality is true 3 4 3 1 0 12 1 1+b2 , b < 1 E 1 − tanh b2 + bξ ≤ min , πQ(b) = πQ(b), b ≥ 1. 1 + b2 (6.66) Remarks. Equation (6.66) taken as an approximation has an accuracy better than 90% for all b ≥ 0. Note also that limb→∞ g(b)/(πQ(b)) = 1. Proof. See [15]. 2 , and using matched-ﬁlter Now, let vIC at iteration m be denoted by vm interference cancellation, and equal unit powers Pk = 1 for simplicity, we ﬁnd 2 2 = βg(vm ) + σ2 . vm+1

(6.67)

Using 2 g(vm )≤

1 1+

N −1 2 vm

(6.68)

by applying the ﬁrst upper bound (6.66) for a repetition code of length N , we obtain

208

6 Joint Multiuser Decoding 2 vm+1 ≤

2 vm

2 βvm + σ2 . +N −1

(6.69)

For the unique ﬁxed point variance we obtain the simple quadratic boundary equation 2 βv∞ 2 σ∞ + σ2 = 2 (6.70) v∞ + N − 1 which has the solution 1 2 2 2 2 2 v∞ = β + σ − (N − 1) + (β + σ − (N − 1)) + 4(N − 1)σ . 2 (6.71) Equation (6.71) is due to using the ﬁrst term in (6.66) as the biting constraint, 2 > 1. which is achieved for v∞ Deﬁning the uncoded load β = β/(N − 1), i.e. the load with respect to the information bit rate, we obtain ⎛ ⎞ 8 2 2 2 2 N −1⎝ σ σ σ ⎠ 2 β −1+ + . ≤ +4 β − 1 + v∞ 2 N −1 N −1 N −1 (6.72) Recall that in Chapter 4 we calculated the single-user signal-to-noise ratio of the MMSE layering ﬁlter under equal received power levels, given by equation (4.83). Solving it for the residual noise and interference variance, using (4.80), we obtain 1 2 2 vmmse β − 1 + σ 2 + (β − 1 + σ 2 ) + 4σ 2 . = (6.73) 2 From (6.72) and (6.73) we see that the MMSE ﬁlter with load β and noise variance σ 2 can directly be related to the performance of the iterative decoder with load β(N − 1) and noise variance σ 2 (N − 1). We express these ﬁnding in the following Theorem 6.3. Iterative cancellation of equal power random CDMA users using rate 1/N repetition codes with N → ∞, has a per-chip capacity at least as large as linear minimum-mean square error ﬁltering. The capacity of iterative cancellation is strictly larger for β < 2 − 2σ 2 /P . Proof. The variance (6.72) of an iterative receiver is at most as large as that of an MMSE ﬁlter with load β/(N −1) and noise variance σ 2 /(N −1). Assuming BPSK modulation for both, we obtain NP β (6.74) C = CB 2 N v∞

6.4 Filters in the Loop

209

for the iterative ﬁlter, and Cmmse

β = CB N −1

(N − 1)P 2 v∞

(6.75)

for the MMSE ﬁlter. Furthermore, equality in (6.71) holds for iterative cancellation if equality holds in the ﬁrst inequality of (6.66), i.e. if 2 < N − 1. v∞

(6.76)

From this we easily derive the condition of the theorem. Figure 6.17 shows the spectral eﬃciencies in bits/dimension achievable with MMSE ﬁltering and non-linear iterative ﬁltering. The iterative systems operates at an N − 1 times higher system load than the MMSE ﬁlter, which maximizes spectral eﬃciency for load values 0.5 ≤ β ≤ 1, and rapidly degrades with higher loads. Furthermore, iterative ﬁltering signiﬁcantly outperforms linear MMSE ﬁltering in the regime where Eb /N0 > 6dB. The curious curvature of the iterative capacity curves in Figure 6.17 is related to the change in curvature of g(vd ), and occurs as the intersection point traverses the section of zero curvature of the repetition code VT function – see Figure 6.16. This happens where the biting bound in (6.66) switches.

6.4 Filters in the Loop 6.4.1 Per-User MMSE Filters In Section 6.3 the processing ﬁlters wk∗ [j] in Figure 6.7 are ﬁlters matched to the spreading waveform of user k. Since the function of the CDMA interference canceler and the ﬁlters is the eﬃcient suppression of the interference, Wang and Poor [157] concluded that local minimum-mean square error (MMSE) ﬁlter could give better performance. In fact, from a variance point of view the MMSE ﬁlter is the optimal choice. We therefore consider the ﬁlter < ; 2 wk∗ [j] = arg min E dk [j] − w∗ rk [j] (6.77) w∈RL

which is the best linear ﬁlter in the receiver chain. The advantage of the MMSE ﬁlter over the matched ﬁlter comes from its capability to include the correlation inherent in the residual interference signal. This is evident from its derivation which we develop now. From the orthogonality principle of estimation theory (see Appendix A) the optimal ﬁlter in (6.77) obeys the following relation E [dk [j] − wk∗ [j]rk [j]] r∗k [j] = 0 Pk sk [j] = E [rk [j]r∗k [j]] wk [j].

(6.78)

210

6 Joint Multiuser Decoding

Capacity bits/dimension

10

DFLW\ &DS *1 : $

1

006()LOWHU /RDG

,WHUDWLYH6\VWHP 5DWH&RGHV

,WHUDWLYH6\VWHP 5DWH&RGHV

006()LOWHU /RDG

0.1

-2

0

2

4

6

8

10

12

14

16 Eb /N0

Fig. 6.17. Achievable spectral eﬃciencies using linear and nonlinear layering processing in equal power CDMA systems.

If we assume for the moment that there is no interference – or that the inter√ ference is assumed to be white – then rk [j] = Pk dk [i]sk [j] + z, and √ Pk sk [j] wk [j] = (6.79) Pk + σ 2 that is, apart from an unimportant scaling factor, the required ﬁlter is a spreading sequence matched ﬁlter as used in previous sections. Next we will take the correlation of the interference into account. In order to do this, it is advantageous to slightly rewrite equation (6.78) as E [dk [j]r∗k [j]] = wk∗ [j] E [rk [j]r∗k [j]].

(6.80)

We now conﬁne our treatment to the synchronous case, noting that asynchronous CDMA can be handled in completely analogous manner, but with a much more complicated notational apparatus. Furthermore, as we have seen in Chapter 5 and will see in the next section, iterative ﬁlter implementations are

6.4 Filters in the Loop

211

identical for both synchronous and asynchronous systems, and do not suﬀer the complications of directly inverting an asynchronous correlation matrix. We now proceed by evaluating M[j] = E [rk [j]r∗k [j]] = S[j]WS∗ [j] + σ 2 I (6.81) = sk [j]Pk s∗k [j] + Sk [j]Wk S∗k [j] + σ 2 I as the interference correlation matrix calculated in Chapter 4, and we have separated the term into an “interference and noise” term and the signal part. The inverse of M is the minimum-mean square error matrix ﬁlter. We now apply the matrix inversion lemma2 , and drop the time index j for this calculation to obtain ⎛ ⎞−1 ⎝sk Pk s∗k + Sk Wk S∗k + σ 2 I⎠ BCD

=

A

−1 −1 1 αI − Sk Wk S∗k + σ 2 I sk Pk s∗k Sk Wk S∗k + σ 2 I α −1 α = Pk + s∗k Sk Wk S∗k + σ 2 I sk . Furthermore, using

E [dk r∗k ] =

(6.82)

Pk s∗k

(6.83)

the per-user, per-iteration MMSE ﬁlter in (6.80) can be calculated as −1 √ Pk s∗k Sk Wk S∗k + σ 2 I ∗ , (6.84) wk = −1 Pk + s∗k (Sk Wk S∗k + σ 2 I) sk where the time index j is implicit for a synchronous system, while for asynchronous CDMA the matrices in (6.84) will have to be properly redeﬁned. We see that this ﬁlter now depends not only on the spreading sequence of the interfering users, but also on their powers Wk . Furthermore, since these are residual powers, the matrix inverse in (6.84) has to be recomputed for every iteration√ – a truly tremendous eﬀort. The factor Pk /α is a metric adjustment factor and has the same corrective role as for projection receiver discussed in Section 6.2.3. It’s importance diminishes as the system size grows and the variance of α diminishes. Evidently, it also has no eﬀect on calculations of signal-to-noise ratios values. In order to proceed with our analysis we make use of a result from Tse and Hanly [138] which relates the asymptotic signal-to-interference ratio (SIR) for 2

Given as −1

(A + BCD)−1 = A−1 − A−1 B(DA−1 B + C−1 ) DA−1

212

6 Joint Multiuser Decoding

an arbitrary user in a CDMA system with random spreading codes where the receiver uses a joint MMSE ﬁlter. The diﬀerent users are allowed to have diﬀerent power levels, and the result is asymptotic as K, L → ∞. It is given by by the implicit formulas of the following theorem, proven for synchronous CDMA: Theorem 6.4 (Tse and Hanly [138]). The asymptotic signal to interference ratio γk for user k as K, L → ∞, K/L = β, in a random CDMA system with a MMSE receiver ﬁltering obeys the implicit equation γk =

Pk , σ 2 + β E [I(P, Pk , γk )]

(6.85)

where the eﬀective interference is I(P, Pk , γk ) =

P Pk . Pk + P γk

(6.86)

The expectation in (6.85) is over the distribution of power Pk . Equation (6.85) is an implicit equation which has no closed-form solution in general. The signal-to-interference ratio of user k depends on the powers of all the other users, or more precisely, on the power distribution of the inﬁnite population of interfering users. For our purposes we are interested in the equal power case, where Pk = P for all users k. In this case a closed-form expression for γk can be found as 8 P (β − 1) − σ 2 P (β + 1) 1 P 2 (β − 1)2 γk = (6.87) + + + . 2 2 2 2σ 2σ 2 4 4σ Let us consider user k in the case of a joint iterative decoder. During iterations, the eﬀective interference power of the other users will decrease according to (m) 2 → vm Pl (6.88) Pl at iteration m. Now note that the per-user MMSE ﬁlter (6.84) does not depend on Pk other than via a trivial scaling factor, and depends only on the common equal power levels of (6.88). We can therefore apply (6.87) for the signal-tonoise ratio, which we simply need to adjust to the power Pk , i.e. 8 2 2 4 P 2 (β − 1)2 v P (β − 1) − σ vm v 2 P (β + 1) 1 2 (m) + + m + . (6.89) vm γk = m 2 2 2 2σ 2σ 2 4 4σ Assuming that all use the same coding system, let us study the iteration behavior of a per-user MMSE ﬁltered cancellation receiver analogously to the

6.4 Filters in the Loop

213

simple matched ﬁlter cancellation receiver. Normalizing Pk = 1, the residual variance vIC = 1/γk is plotted in Figure 6.18 for a load of β = 2 and diﬀerent signal-to-noise ratios. 2.5

Residual MU Variance vIC

2

P/N0 = 0

1.5

1

P/N0 → ∞

0.5

0

0

0.2

0.4

0.6

0.8

1

Soft-bit variance vd Fig. 6.18. Variance transfer curves for matched ﬁlter (simple) cancellation and per-user MMSE ﬁlter cancellation (dashed lines) for β = 2 and Es /N0 = 0dB and Es /N0 → ∞.

What becomes evident in Figure 6.18 is substantiated by the following lemma, namely that the variance transfer curve of the per-user MMSE ﬁlter is nearly linear, and thus aﬀords us a tool to compare its performance directly with that of simple matched ﬁlter cancellation. Theorem 6.5. The variance transfer function (6.87) of the peruser MMSE ﬁlter canceler with P → vd P is a linear function of vd with slope β − 1 as P/σ 2 → ∞. Proof. The slope is given by lim

P/σ 2 →∞

vIC 1 = lim vd P/σ 2 →∞ γk vd =

lim

P/σ 2 →∞ β+1 β−1

2 +

σ2 2vd (β−1)P

−1

= β − 1,

(6.90)

214

6 Joint Multiuser Decoding

where we have used (6.89) and a number of detailed but simple algebraic steps. Theorem 6.5 has the following corollary as immediate consequence: Corollary 6.1. If matched ﬁlter cancellation can support a load of β, the use of per-user MMSE ﬁlters instead of the matched ﬁlters allows a load of β + 1 for high signal-to-noise ratios. Proof. Choose the signal-to-noise ratio large enough such that the variance transfer function of the per-user MMSE ﬁltered canceler is linear in vd with slope β − 1. The limiting load is found where the canceler VT curve and that of the error control code intersect. If this happens for a slope β = βIC for matched ﬁlter cancellation, the load for per-user MMSE ﬁltering can be increased until β − 1 = βIC . 6.4.2 Low-Complexity Iterative Loop Filters While a highly specialized loop ﬁlter like the MMSE ﬁlter can produce a marked advantage for a certain range of channel loads, its implementation is signiﬁcantly more complex than that of a matched ﬁlter due to the inverse in (6.84). However, if we replace −1 Kk = Sk Wk S∗k + σ 2 I

(6.91)

in (6.84) by the ﬁrst-order stationary multi-stage approximation ≈

n

Kk = (I − τ Kk ) + −1

s−1

p

τ (I − τ Kk )

p=0

= φn (Kk )

(6.92)

according to the principles explored in Chapter 5, the resulting system complexity is much reduced, depending only on the number of stages s. As in the previous section, the user may interpret (6.92) as a symbol-wise equation for a synchronous system where we have omitted the symbol index j, or a full asynchronous equation using the appropriate extensions of the matrices and vectors. The point is that the key conclusions are the same and that the implementation of (6.92) via multistage ﬁltering methods also does not depend on whether the system is synchronous or not. We will show now that the number of stages s does not depend on L or K, and that very few stages recover almost all of the diﬀerence between a matched-ﬁlter iterative decoder and one using per-user MMSE ﬁlters. Instead of (6.84), the loop ﬁlter is now wk = φs (Kk ) sk

(6.93)

6.4 Filters in the Loop

215

where we have dropped the normalization factor in (6.84) since its variance goes to zero for large systems with K, L → ∞. The output of the multi-stage ﬁlter contains (desired) signal components and noise components, more precisely ⎛ ⎞

⎜ ⎟ ˜j sj + zk ⎟ wk∗ rk = s∗k φs (Kk ) ⎜ d P d s + P − d k k k k j ⎝ ⎠

(6.94)

j=1 (j=k)

Under the assumption that dj − d˜j has zero mean, the interference in (6.94) is also zero mean, and the ﬁltered signal has a signal-to-noise ratio of SPk (s∗k φs (Kk ) sk ) = ∗ IPk sk φs (Kk ) Kk φs (Kk ) sk 2

γk =

(6.95)

where we have simply averaged of the noise z. For large systems, as K, L → ∞, the variances of both SPk and IPk vanish, and (6.95) becomes a ﬁxed system signal-to-noise ratio, independent of any speciﬁc time and k. From random matrix theory, Section 3.4.2 and [128] it is known that the p random s∗k (Sk S∗k ) sk converges in probability to the deterministic E ∞variable value 0 λp fλ (λ)dλ, where λ is an eigenvalue of Sk S∗k , and fλ (λ) is the distribution of these eigenvalues. Since both Kk and Φs (Kk ) are polynomial expressions of the random matrix Sk S∗k , these random matrix theory results can be used to compute the average signal-to-noise ratio. In particular, since the eigenvalues of a polynomial function of a matrix are the polynomial functions of the original eigenvalues of that matrix, we obtain SPk −→

2

∞

φs (λ)fλ (λ)dλ

(6.96)

0

where φs (λ) is the distribution of the eigenvalues of φs (Kk ). The convergence in (6.96) is in probability. Similarly, we can compute ∞ φ2s (λ)κ(λ)fλ (λ)dλ (6.97) IPk −→ 0

where κ(λ) = λ + σ 2 is an eigenvalue of Kk . Equations (6.96) and (6.97) can now be used to ﬁnd multiuser front-end VT curves by numerical integration. Figure 6.19 below shows such VT curves for a number of stages and two values of the signal-to-noise ratio. It is remarkable that one or two stages are almost completely suﬃcient to achieve the VT characteristics of the MMSE loop ﬁlter over a wide range of input soft bit variance vd and channel signal-to-noise ratios.

216

6 Joint Multiuser Decoding

Residual MU Variance vIC

2.5

Matched Filter Interference Cancellation

2

per-user MMSE Filter 4 stages 2 stages 1 stage

P/σ 2 = 3dB

1.5

1

10 stages

0.5 5 stages 2 stages

P/σ 2 = 23dB

1 stage 0

0

0.2

0.4

0.6

0.8

1

Soft-bit variance vd Fig. 6.19. Variance transfer curves for various multi-stage loop ﬁlters for a β = 2 and two values of the signal-to-noise ratio: P/σ 2 = 3dB and P/σ 2 = 23dB.

We use the following numerical example to illustrate the accuracy of the analysis as well as the power of the multi-stage loop ﬁlters. The test example is made up of a convolutional rate R = 1/3 four-state code with G(D) = [1 + D2 , 1 + D + D2 , 1 + D + D2 ] and a two-stage ﬁrst order station 2 K−1 2 ary loop ﬁlter with stationarity parameter τ = 2/ 2σ + + 1 to L guarantee convergence. Figure 6.20 shows the matching between the two-stage loop ﬁlter canceler and the 4-state convolutional codes used as FEC. At a signal-to-noise ratio of Eb /N0 = 4.5dB, both the two-state loop ﬁlter and a full MMSE loop ﬁlter for the load of β = 3 converge, while a matched ﬁlter canceler would no longer achieve low error rate since its VT curve intersects with that of the FEC codes at high values of vd , that is, before the turbo cliﬀ. Figure 6.21 shows bit error rates for this example system, showing the typical turbo cliﬀ occur as soon as the convergence channel opens at around 4.5dB. The number of users is K = 45, and the spreading gain is L = 15, creating system load β = 3. Note that a full MMSE canceler would gain only about 0.3dB in signal-to-noise ratio over the two-stage loop ﬁlter – hardly worth the additional complexity.

6.4 Filters in the Loop

217

97IRUUDWH&&Y 97IRULQWHUIHUHQFHFDQFHOOHU VLPXODWLRQWUDMHFWRU\

Residual MU Variance vIC

n = 5000 K = 45, L = 15

97IRUVWRUGHUILOWHUVWDJHV 97IRUSHUXVHU006(

Soft-bit variance vd Fig. 6.20. Variance transfer chart for an iterative decoder using convolutional error control codes and a two-stage loop ﬁlter, showing an open channel at Eb /N0 = 4.5dB.

6.4.3 Examples and Comparisons Let us explore the potential of these iterative cancellation schemes. Matched ﬁlter cancellation has the lowest complexity, especially in conjunction with a simple error control decoder such as a repetition code whose APP decoder is rather trivial. Per-user MMSE ﬁltering, on the other hand, provides somewhat better performance in the context of iterative cancellation decoding, but its complexity is large. Other cancellation methods will lie in between these two extremes. A comparison in terms of performance is therefore a valuable tool. We ﬁrst consider the ultimate performance capacity limits as computed in Chapter 4. Figure 6.22 shows the capacities of optimal decoding and linear MMSE ﬁltering (without iterations) for three diﬀerent system loads β = 2, 1, and β = 0.5. From these ﬁgures we can deduce the following proposition. The use of complex joint detection methods, such as the iterative schemes discussed in this section, is eﬃcient only for large system loads β > 1. For smaller loads linear ﬁltering followed by FEC decoding is nearly as eﬃcient, and would thus constitute a simpler system alternative, especially if iterative ﬁlter implementations are used. Let us then focus on large system loads for the application of iterative cancellation receivers. As Lemma 6.1 asserts, the advantage of per-user MMSE ﬁltering diminishes as the system load increases. In Figure 6.23 we plot the

218

6 Joint Multiuser Decoding 1 ,WHUDWLRQ

,WHUDWLRQV

10−1

Bit Error Rate

10

V RQ

Sin gle Use r

−3

L UDW ,WH

,WHUDWLRQV

10

QV HUDWLR ,W V UDWLRQ ,WH

006()LOWHU ,WHUDWLRQV

−2

10−4

10−5 2.5

3

3.5

4

4.5

5

Eb /N0

Fig. 6.21. Bit error rate performance of an iterative canceler with a two-stage loop ﬁlter for 1,10,20, and 30 iterations, compared to the performance of an MMSE loop ﬁlter. Load: β = 2 Capacity bits/dimension

Load: β = 1 Capacity

Load: β = 0.5 Capacity

10 Orthogonal

Orthogonal

Orthogonal Optimal

Optimal

Optimal

MMSE

MMSE

1 MMSE

0.1

Eb N0

0

2.5

5

7.5

10

12.5

16

5

7.5

10

12.5

15

5

7.5

10

12.5

15

Fig. 6.22. Optimal and linear preprocessing capacities for various system loads for random CDMA signaling.

6.5 Asymmetric Operating Conditions

219

performance of repetition codes versus diﬀerent capacity curves of random CDMA. As the rate of the repetition code is decreased, the number of simultaneous transmissions that can be supported increases, i.e. the system load increases. We observe two noteworthy eﬀects: Firstly, the overall eﬃciency of the system monotonically increases as the code rate decreases due to a higher supportable system load. The diﬀerent rate curves are drawn for repetition codes of rate 1/2 to 1/12 using loop matched ﬁlters. The range of performances achieved by a per-user MMSE ﬁlter lies between the thin dashed lines, which delimit the performance of rate-1/2 codes at the lower end, and rate-1/12 codes at the upper end. A rate-1/2 system with MMSE ﬁltering performs about as well as a rate-1/4 system with loop matched ﬁlters, while for rates 1/12 there is virtually no diﬀerence. This is in agreement with Theorem 6.5, and conﬁrms that, for large system loads, the advantage of per-user MMSE ﬁltering vanishes and the performance of simple cancellation becomes practically identical to that of the more complex MMSE ﬁltering system. The capacity curves of optimal detection and the two preprocessing ﬁlters are drawn for rate R = 1/3 coding, which serves as our baseline rate. The system load is adjusted such that the respective capacities are maximized. Compare this to the Figure (4.11), where the system load β is kept constant. The ﬁgure shows performance points for simple cancellation, that is, limiting system loads where the VT curves intersect. The dashed lines show the performance limits of per-user MMSE ﬁltering for both the rate R = 1/2 and the rate R = 1/12 repetition coded systems, illustrating the vanishing advantage of the MMSE ﬁlter. At the left ends of the performance curves, a bit error rates of 10−4 are reached. For lower signal-to-noise ratios the system does not act as a decoder anymore, but as the non-linear ﬁlter discussed in Section 6.3.3. The performance curves are virtually identical to the nonlinear ﬁltering capacities shown in Figure 6.17. Note that at the closest point these simple cancellation schemes achieve about 70% of the equal-power joint processing Shannon capacity of the system.

6.5 Asymmetric Operating Conditions What we have considered thus far are symmetric operating condition like they are aspired to in conventional CDMA system with correlation reception [21, 135, 161]. The conditions are symmetric in that the received power levels are kept constant to within a small tolerable level using open and closed loop power control mechanisms. The systems are also symmetric in the sense that all users transmit at equal rates and use the same error control coding. Guaranteeing power control, in particular, might be unnecessarily complex, and multiuser decoders typically beneﬁt from asymmetric operating conditions, as we will see below. This is in marked contrast to the linear joint detectors, which achieve the largest power and bandwidth eﬃciency at symmetric operating conditions – see Section 4.5. We treat the unequal power case ﬁrst, and

220

6 Joint Multiuser Decoding

Capacity bits/dimension

10

LW\

OFDSDF

RSWLPD

DFLW\ &DS *1

$:

1

5DWH 5DWH 5HSHWLWLRQ 5HSHWLWLRQ &RGH &RGH 5DWH 006()LOWHU 5HSHWLWLRQ 5DWH &DSDFLW\ &RGH 5HSHWLWLRQ 0DWFKHG)LOWHU &RGH 5DWH &DSDFLW\ 5HSHWLWLRQ &RGH

0.1

-2

0

2

4

6

8

10

12

14

16 Eb /N0

Fig. 6.23. Performance of low-rate repetition codes in high-load CDMA systems, compared to single-user layered capacities for matched and MMSE ﬁlter systems.

show that there exist optimal power distributions which achieve the Shannon limits of the channel. 6.5.1 Unequal Received Power Levels Restricting attention to matched ﬁlters wk [j] = sk [j] in the loop the residual interference and noise signal after matched ﬁltering is generalized from (6.50) to n

K

ηk [j] = Pl dl [i] − d˜l [i] (s∗k [j]sl [i]) + zk [j], (6.98) i=1 l=1 (l=k)

where ηk [j] is the combined noise and interference sample at the output of user k at time j. The diﬀerent users’ powers are now assumed variable and unequal. Assuming unbiased estimates of the i.i.d. distributed coded symbols dk [j] allows us to calculate the mean and variance of ηk [j] as

6.5 Asymmetric Operating Conditions

221

E [ηk [j]] = 0 n

K ; <2 0 1

2 E ηk2 [j] = Pl E dl [j] − d˜l [j] E [s∗k [j]sl [j]] + σ 2 .

(6.99)

i=1 l=1 (l=k)

For random spreading, where each chip in each at each time is chosen independently with equal probability, it follows, even in the asynchronous case, 2 that E [s∗k [j]sl [i]] = 1/L and K ; <2 0 1 1

σk2 = E ηk2 [l] = Pl E dl [j] − d˜l [j] + σ 2 , L

(6.100)

l=1 (l=k)

which is independent of the time index j, and we can therefore drop it in equation (6.100). For equal powers this equation obviously reduces to (6.53). The output symbol estimation variance 2 <2 ; σl (6.101) E dl − d˜l = g Pl is a function of the code VT characteristics and the power of user l. We have seen that non-pathological VT functions g(x) are monotonically increasing with range [0, 1]. With iterations, the residual noise and interference variance σk2 will change from iteration to iteration. If we denote this variance for user k at iteration v 2 by σk,v , then we can formulate the following recursive formula: 9 : K 2 σj,v−1 1

2 σk,v = + σ2 . Pl g (6.102) L Pl l=1 (l=k)

2 Fortunately, the sequence σk,v , v = 0, 1, 2, · · · is monotonically decreasing, 2 which we show as follows. Formally we may set σk,0 = ∞, and assume that 2 2 2 σk,0 > σk,1 ≥ σk,v−1 . Using the monotonicity property of the function g(x) we now show that 9 : K 2 σl,v−1 1

2 + σ2 σk,v = Pl g L Pl l=1 (l=k)

K 1

≤ Pl g L l=1 (l=k)

9

2 σl,v−2 Pl

: 2 + σ 2 = σk,v−1

(6.103)

2 which shows, by standard induction, that the sequence σk,v is monotonically 2 decreasing ∀k, and, since σk,v ≥ 0, the sequence must converge to a limit which satisﬁes

222

6 Joint Multiuser Decoding

9

1

= Pl g L K

2 σk,∞

l=1

2 σl,∞ Pl

:

1 − g L

9

2 σk,∞ Pk

: + σ2

(6.104)

This is good news, since it tells us that the iterations improve the performance monotonically for all users, and that there exists a ﬁx point towards which these iterations converge. Furthermore, in cases where the second term in (6.104) can be neglected, i.e. if K and L are large, the residual interference in fact becomes independent of the user considered. Note that this does not mean the signal-to-noise ratio becomes user independent, that is the case only in equal power systems. To formalize this situation, we impose what is known as the Lindeberg condition on asymptotic negligibility on the powers of the users, given as Pk 5K

K→∞

l=1 l=k

Pl

−→ 0;

∀k.

(6.105)

Given (6.105), the v-th iteration residual interference and noise variance (6.102) as well as the limit variance (6.104) of the iterative process become independent of the user index and approaches for all users the value 1

= Pl g L K

σv2

l=1

2 σv−1 Pl

+ σ2 .

(6.106)

This decisive observation is quite independent on which loop ﬁltering method is used. It enables us to describe the behavior of iterative decoding systems in the case of unequal powers by a single-parameter dynamical system, as we have done in the equal power case. Before we look at speciﬁc cases, we will study some limiting optimal asymmetric operating conditions. 6.5.2 Optimal Power Proﬁles We concentrate on FEC codes with an ideal VT characteristics, given by equation (6.54), i.e. by 2 1 if σ 2 ≥ τ P j v σv vd ≥ g ≥ Pj 0 if σv2 < τ Pj Recall that 1/τ is the threshold signal-to-noise ratio of a powerful turbo or LDPC code (or any other code with such a threshold characteristics). Now assume that we have separated the K users into J groups of powers P1 , · · · , PJ , and Pj is the received power of the users in group j. Also, let the powers be ordered as P1 ≤ P2 ≤ · · · ≤ PJ , even though this is irrelevant in practice due to the parallel nature of the decoder. 5 5J βj as the sum of the partial Furthermore deﬁne β = j=1 Kj /L = loads βj of the j-th group. Due to (6.54), a given user group will completely

6.5 Asymmetric Operating Conditions

223

decode, before the next lower-power group can be decoded, and we have effectively a group-serial decoder even though the process is run as a parallel algorithm. Assuming then that Kj 1, in fact ensuring the non-dominance condition, and beginning with j = 1 we observe that P1 must obey the following condition for successful decoding of the last user group #1, after all other J − 1 groups have been decoded and successfully canceled: τ P 1 ≥ β1 P1 + σ 2 and τ Pj ≥

j

(6.107)

βl Pl + σ 2 .

(6.108)

l=1

From the above we recursively ﬁnd 5j−1 βl Pl + σ 2 Pj ≥ l=1 ; τ − βj

1 ≤ l ≤ J.

(6.109)

The minimum powers are achieved in (6.109) when the inequalities are met with equality, in which case the power levels are given by Pj = σ 2 τ j−1

j # l=1

1 ; τ − βl

1 ≤ j ≤ J.

(6.110)

We now consider optimization of the partial loads βj in order to maximize the spectral eﬃciency, or equivalently, minimize the average power per user (assuming that all users use identical code rates), that is, we want to minimize P given by 5J j=1 Pj βj P = min 5J (6.111) {βj } j=1 βj The solution to (6.111) is given by the following Lemma 6.4. The average power in (6.111) is minimized by the uniform load distribution βj = β/J; ∀j. Proof. We wish to minimize the average power P =

J σ2

βj τ i−1 =i β i=1 l=1 (τ − βl )

given the constraint

J

βi = β.

i=1

Letting xi = βi /τ for i = 1, . . . , J, we minimize P min Note that

J σ2

xi = min =i x1 ,...,xJ β l=1 (1 − xl ) i=1

given

J

i=1

xi =

β . τ

224

6 Joint Multiuser Decoding J

=i

l=1 (1

i=1 J

xi − xl )

1

=

J

=i

i=1

−1−

(xi − 1)

J

l=1 (1

− xl ) 1

+

J

=i

1

l=1 (1

i=1

− xl )

=

1

−1= − xl ) 6 :6 J J

1

1 ln(1 − xl ) − 1 ≥ exp −J ln 1 − −1= xl = exp −J J J =

i=1

=i

l=1 (1

l=1

− xl )

=i−1

l=1 (1

i=2

= =J

− xl ) 9

l=1 (1

l=1

1 α ? −1= − 1, = exp −J ln 1 − J Jτ (1 − α/(Jτ )) >

where we have used the standard inequality E [ln ξ] ≤ ln(E [ξ]), and equality is achieved for β1 = . . . = βJ . Lemma 6.4 shows that equal-sized groups are in fact optimal, and under these conditions, the group powers (6.110) are given by σ2 Pj = τ − Jβ

1−j −j β σ2 β 1− = ; 1− τJ τ τJ

1 ≤ j ≤ J,

(6.112)

that is, the powers follow an exponential distribution with optimal average power (6.111) given by P opt =

J

Pj βj j=1

β

=

J 1

Pj J j=1

−j J σ2

β 1− τ J j=1 τJ

J τ σ2 = −1 . β τ − β/J =

(6.113)

In (6.113), the optimal power is a function of σ 2 , β and τ . The corresponding bit normalized signal-to-noise-ratio is then

J Eb τ 1 P opt = = −1 , (6.114) N0 opt 2Rσ 2 2Rβ τ − β/J where R is the code rate common to all the FEC codes. Figure 6.24 shows the spectral eﬃciency achievable with rate R = 1/3 Shannon-type strong codes, for both matched ﬁlter cancellation (solid curves) and MMSE loop ﬁlter cancellation (dashed). Curves are shown for J = 1, 5, and 10 optimized power groups, using an ideal Shannon-capacity achieving

6.5 Asymmetric Operating Conditions

225

code whose (iterative) detection threshold τ = σ 2 /Es is related to its rate via the well-known Shannon bound [120] τ=

1 . 22R − 1

(6.115)

The following observations can be made. The achievable capacity is close to the Shannon limit for AWGN channels for all values of Eb /N0 . This is contrary to the situation in the equal power case, where we were unable to ﬁnd capacity rates that grew with increasing Eb /N0 – see Figures 6.17 and 6.23. 10

J = 10

Sum Capacity [bits/dimension]

J =5 FLW\ DSD

J =1

$:

& *1

1

0.1

−2

0

2

4

6

8

10

12

14

16 Eb /N0

Fig. 6.24. CDMA spectral eﬃciencies achievable with iterative decoding with different power groups assuming ideal FEC coding with rates R = 1/3 for simple cancellation as well as MMSE cancellation.

There is a caveat in such power optimized distributions, and this is that the initial parallel iterative interference cancellation receiver is transformed into a successive canceler in the sense that the diﬀerent power groups are defacto decoded successively. However, the novelty of parallel iterative detection over classic serial cancellation is that cancellation does not need to be error free in order for the entire system to converge as we have shown in general. On

226

6 Joint Multiuser Decoding

the contrary, if weak error control codes are used, convergence is achieved in a manner that one could call partial parallel cancellation, whereby convergence in any given group of users may not yet be complete while convergence in the next lower power groups has already begun. The attainable spectral eﬃciency is evidently improved as the number of power groups is increased. This following lemma formalizes this observation: −J Lemma 6.5. The sequence 1 − τβJ is monotonically decreasing with J and converges to eβ/τ in the range [J ∗ , ∞), where J ∗ denotes the smallest integer such that β/(J ∗ τ ) < 1. Proof. Since

τ τ − β/J

J

3 = exp J ln

τ τ − β/J

4

we consider the function f (J) = J ln

τ . τ − β/J

Using the standard inequality ln x ≤ x − 1, we have f (J) = ln

β τ − ≤0 τ − β/J Jτ − β

from which the lemma follows. In the limit as J → ∞ the required threshold Eb /N0 obeys

J τ Eb eβ/τ − 1 1 , = lim −1 = N0 lim J→∞ 2Rβ τ − β/J 2Rβ

(6.116)

x1 1 → e as x → 0 in the third step. The optimal where we have used 1−x signal-to-noise ratio follows approximately an exponential function of the total system load β. The minimum signal-to-noise ratio for CDMA can be obtained by letting β → 0 in (6.116), and equals 1/(2Rτ ), which is the threshold of the FEC code in single-user channels. Alternately, for R → 0 we obtain the following Theorem 6.6. Single-user Shannon-capacity achieving codes obeying (6.115) achieve the multiple-access channel Shannon bound as R → 0 given the optimal received power distribution of (6.110) , i.e. their signal-to-noise ratio is given by 22C − 1 Eb ; C = Rβ. (6.117) = N0 lim 2C

6.5 Asymmetric Operating Conditions

227

Proof. Using (6.115) in (6.116) we obtain

Eb N0

= lim

eβ(2

2R

−1)

−1

2Rβ

.

(6.118)

Now let C = Rβ be the spectral eﬃciency of the system and apply the rule of l’Hˆ opital to the 0/0 limit in the exponent of the exponential to obtain the Shannon bound (6.117). Equation (6.117) is identical to the Shannon limit for an additive white Gaussian noise channel [120]. It states that in order to approach the multipleaccess channel’s capacity limit, low-rate coding needs to be used. For a general code threshold τb < τ , it is easy to see that Eb C = τb ln 2 log2 1 + 2C , (6.119) N0 that is, the AWGN Shannon limit is multiplied by the scaling factor 2τb ln 2 < 1. Codes that have a threshold τb = 12 log2 e make the overall system eﬃciency approach that of an interference free AWGN channel. This again necessitates R → 0. Furthermore, (6.119) demonstrates that, in conjunction with the optimal power distribution (6.112), matched ﬁlter cancellation is completely suﬃcient to achieve the capacity of the multiple-access channel. The exponential power distribution is well-known to achieve the capacity of a (Gaussian) multiple access channel if successive cancellation is performed, where each user is completely decoded and its signal canceled from the received signal before the next lower-power user is decoded. That is, if βj → 0 in (6.109), the load in each group is small, then Pj =

j−1 1

βm Pj + σ 2 , τ m=1

(6.120)

that is, the threshold signal-to-noise ratio is given by Pj 1 = 5j−1 , 2 τ m=1 βm Pm + σ which implies that the code rate is bounded by 9 : Pj 1 R ≤ log 1 + 5j−1 , 2 2 m=1 βm Pm + σ

(6.121)

(6.122)

which in turn equals the rate constraint for the Gaussian multiple-access channel if the signals of the users in groups j +1, · · · , J are known [24]. The optimal power assignment therefore adjusts the group powers such that (6.122) is an equality for all k. The decoder then progresses by following a sequence of

228

6 Joint Multiuser Decoding

corner points in the capacity polytope as predicted by the successive cancellation principle. Such a power distribution was suggested by Viterbi for low-rate convolutional coding in CDMA systems in [153]. The process is illustrated by the three-dimensional example capacity polytope in Figure 6.25. Equal rates are achieved by adjusting the power such that the active corner point lies on the equal rate line.

Rate R2

Rate R1

This user decodes first

Rate R3 Fig. 6.25. Capacity polytope illustrated for a three-dimensional multiple-access channels. User 2 is decoded ﬁrst, than user 1, and ﬁnally user 3.

Theorem 6.6 seems at odds with the result in Chapter 3 which stated that the equal power distribution achieves capacity of the random CDMA channel. This situation is clariﬁed by realizing that the maximum of the CDMA channel for large systems is not unique. If fact, for K, L → ∞, the exponential distribution (6.112), as well as all linear combinations of the permuted exponential distributions also achieve the AWGN capacity. In practice, the number of power groups J does not have to be very large to achieve high spectral eﬃciencies (see Figure 6.24). 6.5.3 Unequal Rate Distributions What can be achieved with diﬀerent powers, can also be achieved by using codes of diﬀerent coding rates. This has a similar eﬀect of moving the code VT curves, and causes users with low-rate codes to be able to decode ﬁrst. We again assume J groups of users, who all transmit at the same symbol power level P . We assign rates Rj : R1 > R2 , . . . , > RJ to the diﬀerent groups, and assume Shannon-capacity achieving codes with a VT characteristics bounded

6.5 Asymmetric Operating Conditions

229

by (6.54). Since the maximum tolerable noise variance for rate Rj codes equals τj = 1/(4Rj − 1), which is monotonically decreasing with Rj , lower rate users converge before higher rate users, as suggested above. The power constraint equation for group j, in the case of matched-ﬁlter interference cancellation, is given by τj P ≥

j

βm P + σ 2 ,

(6.123)

m=1

which implies the rate constraints : 9 1 1 Rj ≤ log 1 + 5j . 2 2 m=1 βm + σ /P

(6.124)

The variational problem to be solved is that of maximizing the sum of the rates Rj in (6.124) for a given system load, i.e. maximize : 9 J 1

1 g({βj }, J, b) = βj log 1 + 5j 2 2 j=1 m=1 βm + σ /P (6.125) J

given that βm = β j=1

over all possible choices {βj }. The ﬁrst lemma establishes that the sequences of partial loads βj is a monotonically increasing sequence: Lemma 6.6. The maximizing solution of (6.125) is a monotonically increasing sequence of partial loads, i.e. β1 < . . . < βJ . Proof. In (6.125) take any two adjacent partial loads, say, βj and βj+1 , and consider maximizing (6.125) with respect to βj , βj+1 , constraining βj +βj+1 = a and keeping all other {βj } ﬁxed. This is equivalent to considering the case of only two variable groups, lumping all other interference into B = σ 2 /P + 5j−1 m=1 βm , for which we have with βj = x 1 1 + (a − x) log 1 + . g(βj , βj+1 ) = g(x) = x log 1 + x+B a+B We have

g (x) = ln 1 +

1 x+B

g (x) < 0 ;

− ln 1 +

1 a+B

g (0) > 0 ;

−

x (x + B)(x + B + 1)

g (a/2) < 0.

Then the maximum of g(x) is attained for some 0 < x < a/2, from which the lemma follows.

230

6 Joint Multiuser Decoding

The next lemma gives upper and lower bounds on the achievable sum rate. Lemma 6.7. Let β0 = maxj βj , then the following inequalities hold: 1 2

β

log 1 +

0

1 u + β0 + b

1 du ≤ g({βj }, J, b) ≤ 2

β

log 1 +

1 u+b

du,

0

(6.126) where b = σ 2 /P . The latter integral can be evaluated as (β + b + 1) log(β + b + 1) − (β + b) log(β + b) − (b + 1) log(b + 1) + b log b. Proof. Due to its length the proof has been relegated to the end of the Chapter, in Section 6.6. We can draw a number of interesting conclusions from Lemma 6.7. First, as long as β0 σ 2 /P , i.e. the maximum partial load is small w.r.t. the inverse σ 2 /P of the SNR, any load distribution gives the same maximum achievable sum rate. A corollary to this is that the case of a single group, the lower limit in (6.126) is minimized, i.e. the equal rate case is the worst case for cancellation using Shannon capacity achieving codes. The achievable capacity is given by β +b b+1 b β +b+1 log(β +b+1) − log(β +b) − log(b+1) + log b 2 2 2 2 β 1 1 1 β+b b + + log 1 + log 1 + = log 1 + 2 b+1 2 β+b 2 b

C(β, b) =

Expressing the bit energy to noise power ratio in terms of capacity, signal power, and noise variance, we obtain β Eb ; = N0 2bC(β, )

b=

σ2 . P

(6.127)

If we consider large system loads where β → ∞, we necessarily have low total signal-to-noise ratios, that is, b → ∞. In this case β log e β 1 1 − +O 2 , b → ∞. (6.128) C(β, b) = log 1 + 2 b 4b(β + b) b Furthermore using β ≈ 22C(β,b) eβ/(2b(β+b)) − 1 b in (6.127) we ﬁnally obtain 2C 2C 2 −1 2 −1 Eb 1+ , β → ∞, ≈ N0 2C 2β

(6.129)

(6.130)

where we have used the abbreviation C = C(β, b). For large system loads β, (6.130) again assumes the familiar form of the Shannon bound, similar to (6.117), which we express in the following

6.5 Asymmetric Operating Conditions

231

Theorem 6.7. Shannon-type strong codes can achieve the AWGN Shannon bound as long as the number of user groups is suﬃciently large such that the maximum partial load β0 σ 2 /P . We then obtain 22C − 1 Eb ; β → ∞. (6.131) = N0 lim 2C Figure 6.26 shows the resulting spectral eﬃciencies, i.e. equation (6.127), for diﬀerent ﬁnite values of β. It becomes evident that fairly large system loads would be required to approach the Shannon bound, but that it is theoretically possible.

10

β = 50 β = 20 β = 10

Sum Capacity [bits/dimension]

β=5 β=1

&D

*1 $:

LW\ SDF

1

0.1

−2

0

2

4

6

8

10

12

14

16 Eb /N0

Fig. 6.26. CDMA spectral eﬃciencies achievable with iterative decoding with equal power groups assuming ideal FEC coding with optimized rates according to (6.124).

232

6 Joint Multiuser Decoding

6.5.4 Finite Numbers of Power Groups The asymptotic situations considered in the previous sections show that the ideal, maximal capacity of the multiple-access channel can be achieved if a (relatively) large number of received power groups and diﬀerent rate groups can be established. In general we learn that diﬀerences in received power levels and/or rates are beneﬁcial to the operation of the joint iterative decoder, which is the opposite for single-pass linear ﬁlter detectors. In fact, in practice not very many diﬀerent power levels would be needed to achieve good performance. Furthermore the advantage of weak codes in terms of achievable spectral eﬃciency also quickly disappears as more power groups are allowed. Let us ﬁrst proceed by developing our graphical variance transfer method to the case of unequal received power levels. Recall equation (6.106) for the residual interference and noise variance of an arbitrary user, given by 1

= Pj g L j=1 K

σv2

2 σv−1 Pj

. + σ2

(6.106)

Expressing (6.106) in terms of power groups we ﬁnd σv2

2 J

σv−1 K j Pj g , +σ 2 , = L P j j=1

(6.132)

where Kj is the number of users in group j. We need a notion of an average system load, since diﬀerent users 5Jnow weigh in with diﬀerent received power levels. This we deﬁne as βav = l=1 βl Pl . We now obtain σv2

2 J

σv−1 βj Pj + σ2 = βav g β K av j j=1 2 = βav gav σv−1 + σ2 ,

(6.133)

2 is an “average” variance transfer function. It is obtained where gav σv−1 by weighing the individual code VT functions by the weight factor βPj /βav composed of their loads and powers. Equation (6.133) is linear in the average load, and can be visualized and plotted analogously to the equal power case, where the FEC VT function of the code in the latter corresponds to the composite FEC VT function gav (·) in the case of unequal power levels. Figure 6.27 presents this combined VT dynamical equation (6.133) for two types of FEC codes and 4 equal-sized power groups. The “strong” code system all use the the serial concatenated code 2 from Table 6.1 discussed in Section 6.3.2, and the “weak” code is a 4-state convolutional codes. The power levels are P1 = P, P2 = 2P, P3 = 4P . The solid VT curve presents the average VT curve, the individual VT curves are dashed and reﬂect the diﬀerent power

6.5 Asymmetric Operating Conditions

233

levels. It is noteworthy that the average VT curve is fairly “diagonal” for both cases, and matches well with the cancellation VT curve. Consequently, there is little diﬀerence in the supportable loads of both systems. The exact numbers depend on the error criterion chosen. The weak-code system supports about 1.3bits/dim at around 8.5dB, which is quite close to the theoretical maximum. The convergence trajectories are single simulations illustrating the analysis technique. Strong Codes

10 9

9

8

8

5

7 3 H97

3

RG

)(&&

&R

3

RG

5

0.4

0.6

& )(

GH &R

3 97

3 7

9

GH

R G&

H

Q EL

P &R

3

73

3

H9 &RG

)(&

2

)(&&RGH973 3

1 0.2

6 4

3 3 GH97 )(&&R

2

0 0

7 H9

LQ

PE

4

& HG

2 σeﬀ w

2 σeﬀ /w

7 6

Weak Codes

10

0.8

1 σd2

0 0

73

RGH9

)(&&

1 0.2

0.4

0.6

3

0.8

1

Fig. 6.27. Illustration of diﬀerent power levels and average VT characteristics shown for both a serial turbo code (on the left) and a convolutional code (on the right). The system parameters are K1 = 22, K2 = 18, K3 = 16 for the SCC system at an Eb /N0 = 13.45dB, and K1 = K2 = K3 = 20 at and Eb /N0 = 8.88dB.

In order to ﬁnd a numerical technique to optimize the power levels, we now assume that there are J diﬀerent power levels, each with a partial load βj , as deﬁned in Section 6.5.2, however, this time the number of levels J is arbitrary, and simply determines the complexity of the numerical method and its accuracy. The key observation, ﬁrst made by Caire et. al. [16], is that optimizing the partial loads will solve the problem and turn the optimization into a well-known linear programming problem. Note that, given enough levels J, optimizing the partial loads is synonymous with optimizing a power distribution. Let us start with the residual recursive interference equation (6.132) σv2 =

J

j=1

βj Pj g

2 σv−1 Pj

2 + σ 2 = f P, β, x = σv−1 ,

where P = (P1 , · · · , PJ ), and β = (β1 , · · · , βJ ). The condition that he VT curves of the FEC decoders and the multiuser interference cancelers do not

234

6 Joint Multiuser Decoding

intersect can be reformulated as f (P, β, x) < x;

0 2 1 x ∈ σmin ,∞ .

(6.134)

2 where the lower limit σmin is an arbitrary limit dictated by some minimal performance criterion, such as the error probability of the lowest power user group. Given their power P1 and the error control codes used, we can cal2 culate the maximum tolerable error variance σmin , and (6.134) ensures that no intersection point exist for larger residual interference variances, therefore enforcing this minimal performance criterion. With these preliminaries, we now have the following optimization problem: ⎧ 1 0 2 ,∞ f (P, β, x) ≤ x−ε; x ∈ σmin ⎪ ⎪ ⎪ ⎪ J J ⎨

minimize βj Pj subject to: βj = β ⎪ ⎪ j=1 j=1 ⎪ ⎪ ⎩ βj ≥ 0. (6.135) The optimization criterion in (6.135) minimizes the average Eb /N0 , see (6.114), which is equivalent to optimizing the system load for a given average Eb /N0 . This seems a fair optimization criterion, and, above all, leads to (6.135), which is a linear optimization problem for which excellent numerical techniques exist. The parameter ε controls the width of the “convergence tunnel” in that it ensures a minimal opening through which the iterations have to proceed, see Figure 6.27. A wider channel opening allows for faster convergence, but at the cost of a decreased load of the system. The optimization strategy is very versatile, diﬀerent power groups can use diﬀerent error control codes, i.e. diﬀerent VT transfer functions in (6.132). Furthermore in [16] it is shown that similar linear optimization strategies can also be formulated for the more complex receivers using MMSE ﬁltering at the multiuser stage, or even a posteriori probability (fully complex) estimation.

6.6 Proof of Lemma 6.7 We consider the following variational problem min

β1 ,...,βJ

J

j=1

βj

given

g({βj }, J, b) =

J

9 βj ln 1 + 5j

j=1

1

i=1

βi + b

: = C, (6.136)

where b = σ 2 /P . The problem (6.136) is equivalent to : 9 J

1 max −→ provided βj ln 1 + 5j β1 ,...,βJ i=1 βi + b j=1

J

j=1

βj = A.

(6.137)

6.6 Proof of Lemma 6.7

235

We ﬁrst note that the solution to both problems, (6.136) and (6.137), is strictly monotonically increasing, i.e. β1 < . . . < βK . This is shown as follows: In (6.137) take any two adjacent variables, say, βj and βj+1 , and consider the the minimization with respect to βj , βj+1 , assuming that βj + βj+1 = A and all other {βi } remain ﬁxed. Then, letting β1 = β it is suﬃcient to consider the maximization over β of 1 1 g(β1 , β2 ) = g(β) = β ln 1 + + (A − β) ln 1 + , β+B A+B where B is constant. We observe that 1 1 β − ln 1 + − ; g (β) = ln 1 + β+B A+B (β + B)(β + B + 1) g (β) < 0 ,

g (0) > 0 ,

g (A/2) < 0.

Then maximum of g(β) is attained for some 0 < β < A/2, from which the statement follows. Now we obtain lower and upper bounds for g({βj }, J, b) that work well for large J and well-behaved {βj }. The following statement holds: With β0 = maxj βj the inequalities u1 0

1 ln 1 + u + β0 + b

u1 du ≤ g({βj }, J, b) ≤

ln 1 +

1 u+b

du

(6.138)

0

5J hold, where u1 = j=1 βj . This part is shown as follows: For a non-decreasing function β(x), x ≥ 0 with β(0) ≥ 0 deﬁne the functional J g(β(x), J, b) = 0

9

1 β(z) ln 1 + E z β(u) du + b 0

: dz.

Also, for a non-negative sequence β1 , . . . , βJ let β(x), 0 ≤ x ≤ J be the stepwise function such that β(j) = βj , j − 1 < x ≤ j, j = 1, . . . , J. Then for j = 1, . . . , J we have j j

βi = β(x) dx , i=1

0

and 9 βj ln 1 + 5j

1

i=1

βi + b

:

j

9 β(x) ln 1 + E j

= j−1

0

1 β(u) du + b

: dx ≤

236

6 Joint Multiuser Decoding

j ≤ j−1

9

1 β(x) ln 1 + E x β(u) du + b 0

: dx,

and therefore 9

J

1 β(x) ln 1 + E x β(u) du + b 0

g({βj }, J, b) ≤ 0

J = 0

9

1 ln 1 + E x β(y) dy + b 0

:

x

β(y) dy

d

u1 =

0

: dx =

ln 1 +

1 u+b

du =

0

= (u1 + b + 1) ln(u1 + b + 1) − (u1 + b) ln(u1 + b) − (b + 1) ln(b + 1) + b ln b, where

J u1 =

β(y) dy =

J

βj

j=1

0

from which the right-hand side of inequalities (6.138) follows. On the other hand, we have 9 βj ln 1 + 5j

1

i=1

βi + b

:

j ≥ j−1

9

1 β(x) ln 1 + E x β(u) du + β0 + b 0

: dx

from which the left-hand side of inequalities (6.138) follows. 5J For a given j=1 βj and large J both bounds (6.138) are close to each other if β0 b. Note that the function 1 ; B≥0 f (β) = β ln 1 + β+B increases monotonically with β ≥ 0.

A Estimation and Detection

A.1 Bayesian Estimation and Detection Throughout this book, we take the Bayesian approach to estimation and detection. The purpose of this Appendix is to provide a short refresher on the subject. It is by no means an exhaustive introduction to estimation theory, and it is limited in scope to the basic concepts that are used frequently throughout the book. There are many textbook introductions to estimation and detection theory, for example [71, 72, 102, 114, 143]. When we speak of estimation in the context of random signals, we generally mean the process whereby we attempt to determine one or more unknown parameters, x from an observation y. The Bayesian approach is to assign a known prior probability density (or mass function) fX (x) to the parameters, which means they are considered as random variables. This is in contrast to the “classical” approach, which treats the parameters as deterministic, but unknown. The Bayesian approach is particularly relevant to communications theory, since Shannon’s theory of information is ﬁrmly founded on the idea that unknown messages are represented as random variables. The underlying probabilistic structure is therefore the joint probability density of the observation, y ∈ Y and the parameter x ∈ X , namely fX,Y (x, y) = fX (x) fY |X (y | x) . The supports X and Y may be discrete or continuous. Typically there may be a sequence y1 , y2 , . . . , yn of samples and this may be accommodated by allowing Y to be a vector space Y n . Vector parameters can likewise be accommodated by allowing X to be a vector space. There are two main cases of interest, depending upon the form of the parameter support X . If X is discrete, meaning that the parameter is one of a countable number of alternatives, the estimation problem is a decision problem. In this case, the problem is usually referred to as detection, or hypothesis testing. The term estimation is usually reserved for the case when X is con-

238

A Estimation and Detection

tinuous. Despite diﬀerences of terminology, the underlying problem is still the same. Deﬁnition A.1 (Statistic). A statistic for x ∈ X is a deterministic function t : Y → X . t is also known as an estimator for continuous X and a detector for discrete X . A particular value x ˆ is referred to as an estimate of x. Note that t is in fact a well-deﬁned random variable. It is sometimes useful to also consider functions that can take values outside of the parameter set. The basic problem at hand is the design of estimators t with desirable properties. Suppose that there exists a certain cost involved with accepting x ˆ, rather than x as the true value of the parameter in question. Let this cost be deﬁned by a cost function, C(ˆ x, x). For example, the cost function could be the square diﬀerence (ˆ x − x)2 or absolute diﬀerence |ˆ x − x|. Deﬁnition A.2 (Risk). For a given cost function C, the risk incurred through use of some estimator t(y) : Y → X is R(t, x) = C(t(y), x) fY |X (y | x) dy, Y

which is the expected cost, given that x is the true value of the parameter. The Bayesian approach to estimation is based on minimization of the risk, averaged over the parameter prior. Deﬁnition A.3 (Bayes Criterion). Under the criterion of Bayes, the estimator should minimize the average risk, E [R(t, x)] = R(t, x)fX (x) dx. X

Such an estimator is called a Bayes estimator. We can use Bayes rule to write the Bayes estimate in terms of the average conditional risk, deﬁned as follows.

A.2 Suﬃciency

Deﬁnition A.4 (Conditional Risk). R(t | y) = fX|Y (x | y) C(t(y), x) dx X fX (x) fY |X (y | x) C(t(y), x) dx = fY (y) X

239

(A.1)

Using this conditional risk, re-write the average risk E [R(t, x)] as the average with respect to the prior distribution of the samples, E [R(t, x)] = R(t | y)fY (y) dy, (A.2) Y

which is minimized if R(ˆ x | y) is minimized for each value of y ∈ Y (since the conditional density is non-negative). This is summarized in the following Lemma. Lemma A.1 (Bayes Estimator). The Bayes estimate t(y) minimizes the conditional risk R(t(y) | y) for each value of y.

A.2 Suﬃciency The concept of suﬃciency is an important one in estimation theory. It is usually developed within the framework of classical estimation. There is however also a strong connection to Bayesian estimation. The following deﬁnition sets the scene. Deﬁnition A.5 (Suﬃcient Statistic). A statistic t is said to be suﬃcient for estimation of a parameter x, if fY |T (y | t) = fY |T ,X (y | t, x) .

(A.3)

In other words, conditioned on the statistic, the observation is independent of the parameter. This means that once we have formed the suﬃcient statistic, the observation gives us no further information about the parameter – the statistic serves as a summary of the observation. It is a consequence of Deﬁnition A.1 that the parameter, observation and statistic form a Markov chain, denoted x→y→t which means the joint density factors in the following way,

240

A Estimation and Detection

fX,Y,T (x, y, t) = fX (x) fY |X (y | x) fT |Y (t | y) ,

(A.4)

Conditioned on the observation, the statistic is independent of the parameter, fT |Y,X (t | y, x) = fT |Y (t | y) . It is a known property of Markov chains that the reverse of a Markov chain is also a Markov chain, [24]. Hence t→y→x is also a Markov chain. Conditioned upon the observation, the parameter is therefore independent of the statistic, fX|Y,T (x | y, t) = fX|Y (x | y) . Suﬃciency introduces the extra conditional independence (A.3) that is not implied by the Markov structure. We can use the Markov structure, together with suﬃciency to prove the following useful Lemma. Lemma A.2. Let t be a suﬃcient statistic. Then fX|Y (x | y) = fX|T (x | t)

(A.5)

Proof. From the Markov property, fX|Y,T (x | y, t) = fX|Y (x | y). The goal is to use suﬃciency to show that fX|Y,T (x | y, t) = fX|T (x | t). This may be accomplished using nothing more that the chain rule for probability and the deﬁnition of suﬃciency as follows. fX,Y,T (x, y, t) fY,T (y, t) fX|T (x | t) fY |X,T (y | x, t) = fY |T (y | t)

fX|Y,T (x | y, t) =

= fX|T (x | t) . Now in Lemma A.1 we expressed the Bayes estimator in terms of the minimal average conditional risk (A.1). In the case of a suﬃcient estimator however, Lemma A.2 allows us to write the conditional risk as fX|Y (x | y) C(t(y), x) dx = fX|T (Y ) (x | t(y)) C(t(y), x) dx. X

X

The consequence of this is the following Theorem. Theorem A.1. The Bayes estimator for any cost function C is a function of any suﬃcient statistic.

A.3 Linear Cost

241

This shows more directly how a suﬃcient statistic indeed summarizes an observation. Once we have a suﬃcient statistic, we no longer need the observation to construct a Bayes estimate. There are many suﬃcient statistics for any given parameter. It is known that if t is invertible then x → t → y is also a Markov chain, and in this case, the resulting Markov structure implies the suﬃciency condition (A.3). Lemma A.3. Any invertible transformation of the observation is suﬃcient. The main motivation for ﬁnding a suﬃcient statistic is however to replace a sequence of observations y1 , . . . , yn or a vector observation with a statistic of reduced dimension. We may even desire a suﬃcient statistic that is minimal, in the following sense. Deﬁnition A.6. A minimal suﬃcient statistic is a function of all other suﬃcient statistics. Of particular interest in this book are the Gaussian linear models arising in the context of multiuser detection. Theorem A.2. Let y ∼ N (Ax, Σ) be a conditionally Gaussian random vector with mean Ax (depending on the vector parameter x) and covariance Σ, where A and Σ are known. Then At Σ −1 y is a minimal suﬃcient statistic for x. Note that via the application of At , the dimensionality of the statistic may be reduced. The minimal suﬃcient statistic given in the above Theorem is not unique. For any invertible matrix B, the statistic BAt Σ −1 y is also minimal.

A.3 Linear Cost Perhaps the simplest measure of cost is the estimation error, C(t, x) = t − x. For this particular cost function, the average risk is called bias. Deﬁnition A.7 (Bias). The bias of an estimator is deﬁned to be the expected value of the error, E [t − θ] , where the expectation is over fY,X (y, x) An estimator is unbiased if its bias is zero.

242

A Estimation and Detection

Note that trying to deﬁne a Bayes estimator from C(t, x) = t − x leads to the meaningless estimate t(y) = −∞ (the problem here is that C is not a positive deﬁnite function). Alternatively, setting t(y) = E[x] results in zero average risk, which is again meaningless, since it does not depend on the observation. Rather than trying to deﬁne a Bayes estimate based on linear cost, bias is considered as a property of any given estimator. While it is desirable that an estimator be unbiased, the property itself says little about how good the estimator really is. It does not take too much imagination to construct a pathological unbiased estimator which, irrespective of how many samples are taken, has in probability as large an error as can be deﬁned on X . The following deﬁnition describes a desirable asymptotic property. Deﬁnition A.8 (Consistent Estimator). An estimator tn : Y n → X is consistent if tn → x in probability as n → ∞, by which we mean that for every δ > 0 and > 0, there exists an integer n such that Pr (|tn − x| > ) < δ.

A.4 Quadratic Cost Perhaps the simplest positive deﬁnite cost function is the squared error, C(t, x) = (t − x)2 . The resulting average risk is the mean-squared error, which for unbiased estimators is equal to var(t). In the case of vector parameters, the 2 t quadratic cost is deﬁned C(t, x) = t − x2 = (t − x) (t − x). The resulting Bayes estimator is known as the minimum mean-squared error estimate. A.4.1 Minimum Mean Squared Error Theorem A.3 (Bayes Estimator for Quadratic Cost). The minimum mean-squared error estimator is t(y) = E [x | y]

(A.6)

Proof. We want to minimize the conditional risk for each observation. Now the cost function is positive deﬁnite and convex, which means we may perform this minimization by equating the gradient of the conditional risk to zero. Now ∂ t (t − x) (t − x)fX|Y (x | y) dx = 2 (t − x)fX|Y (x | y) dx ∂t = 2 (t − E [x | y]) ,

A.4 Quadratic Cost

243

and clearly this is set to zero by choosing t(y) = E [x | y]. t

It is interesting to note that the more general quadratic cost (t − x) A(t − x), where A is positive deﬁnite symmetric, results in the same Bayes estimate. The matrix A has no eﬀect on the minimization of conditional risk (since t ∂ ∂t (t − x) A(t − x) = 2A(t − x) and expectation is linear). Although the Bayes estimator has a very compact description, the conditional expectation may be hard to compute in practice. A.4.2 Cram´ er-Rao Inequality Development of the minimum mean-squared error estimator was based on 2 the conditional risk, which is the expectation of t − x2 with respect to the conditional density fX|Y (x | y). One of the most famous results from classical estimation theory is the Cram´er-Rao inequality [25, Section 32.3] and [105]. This inequality provides a bound on the expectation of (t − x)2 with respect to the conditional density fY |X (y | x) (note that in classical estimation theory there is no fX (x)). It gives a bound on the mean-squared error as a function of the realization of the parameter. Theorem A.4 (Cram´ er-Rao Inequality). Every suﬃciently well-behaved estimator t : Y n → X , a function of n i.i.d. samples selected according to fY,X (y, x), with x ∈ X , satisﬁes

2 ∂ 1 E [t | x] , ∂x nJ(x)

(A.7)

2 ∂ ln fY |X (y | x) fY |X (y | x) dy. ∂x

(A.8)

1 0 E (T − x)2 | x ≥ where J(x) =

J(x) is also known as the Fisher information of x, see [24, Section 12.11]. Note that for unbiased estimators, (A.7) reduces to var [t | x] ≥

1 J(x)

(A.9)

There are two required conditions for equality in (A.7) or (A.9), (a). t must be suﬃcient. (b). It must be possible to write ∂ ln fY |X (y | x) = k(x) (t(y) − x) , ∂x where k(x) does not depend upon the observations, y1 , y2 , . . . , yn .

(A.10)

244

A Estimation and Detection

See [24] for this theorem in vector form. This theorem as it stands, has a certain “Shannon” ﬂavor to it, i.e. it says how good an estimator may be, but does not give clues on how to ﬁnd such estimators. Nevertheless, we make the following deﬁnition, due to [43, 44]. Deﬁnition A.9 (Eﬃcient Estimator). An unbiased estimator t is eﬃcient if it satisﬁes (A.9) with equality. Such an estimator is a minimum variance unbiased (MVU) estimator. Eﬃcient estimators do exist, but they are somewhat rare. In order for an estimator to be eﬃcient, it must be unbiased and satisfy conditions (a) and (b) of Theorem A.4. To summarize, under the mean-squared error criterion, the Bayes estimator minimizes the conditional risk (t − x)2 fX|Y (x | y) dx for each value of the observation, whereas an eﬃcient estimator minimizes a diﬀerent conditional risk, (t − x)2 fY |X (y | x) dy for each value of the parameter. A.4.3 Jointly Gaussian Model In the jointly Gaussian model, the minimum mean-squared error estimator has a particularly convenient form. Theorem A.5. Suppose y and x are jointly Gaussian vectors, y E[y] cov[y] cov[y, x] ∼N , x E[x] cov[x, y] cov[x] Then the minimum mean-squared estimate is −1

E [x | y] = E[x] + cov[x, y](cov[y]) (y − E[y])

(A.11)

Proof. It is straightforward to show that the parameter x is conditionally Gaussian with mean (A.11). Note that this is simply a linear transformation of the observation.

A.5 Hamming Cost

245

A.4.4 Linear MMSE Estimation The linear form of the minimum mean-squared estimator for the jointly Gaussian model is one of the gems of estimation theory. In general, however the conditional mean (A.6) will be some non-linear function of the observation. In the interests of saving computational eﬀort, one could be interested in ﬁnding the optimal linear estimator in the general case. This amounts to ﬁnding the best estimator of the form t(y) = Ay where A is a matrix of suitable dimension, i.e. 0 1 A = arg min E Ay − x22 A

Theorem A.6 (Linear Minimum Mean Squared Error Estimation). Let the parameter x and observation both be zeromean. Then the linear minimum mean-squared error estimator is −1

cov[x, y](cov[y]) y Example A.1. Suppose the observation vector y is related to the zero mean, identity covariance parameter x via y = Ax + z, where z is independent from x and A and cov[z] are known. Then the LMMSE estimate of x is −1 At AAt + cov[z] .

A.5 Hamming Cost In detection problems (discrete X ), we may be interested in the following cost 0 t=x C(t, x) = 1 t = x The average cost in this case is in fact the probability of error, E [C(t, x)] = Pr (t = x) .

246

A Estimation and Detection

A.5.1 Minimum probability of Error The Bayes estimator therefore minimizes the probability of error. Since we are considering a discrete parameter, fX|Y (x | y) is a probability mass function. Hence the conditional risk is

fX|Y (x | y) = 1 − fT |Y (t | y) , t(y)=x

which is minimized by choosing t(y) to maximize fT |Y (t | y). This is the maximum a-posteriori (MAP) estimator. Theorem A.7 (Bayes Estimator for Hamming Cost). The minimum probability of error estimator is t(y) = arg max fX|Y (x | y) . x

Using Bayes’ theorem, we can write the MAP estimator as x ˆMAP = arg max fY (y | x)fX (x), x∈X

where fX (x) is the prior distribution of the parameter, and we have removed a term representing the prior probability of y, since this contributes nothing to the maximization over x. A.5.2 Relation to the MMSE Estimator Suppose that the mode of the posterior density (or mass function) fX|Y (x | y) of the parameter coincides with the conditional mean E[x | y]. Then the MMSE and MAP estimators are the same. This is indeed the case for the jointly Gaussian model (A.5), and hence the MAP estimator for the Gaussian model is given by (A.11). A.5.3 Maximum Likelihood Estimation It often happens that the prior distribution of x is unknown, in which case, we must treat the prior distribution of x as uniform (which is the maximum entropy prior for a discrete set), and obtain the the maximum likelihood estimate, which minimizes error probability in the case of a uniform prior.

A.5 Hamming Cost

247

Deﬁnition A.10 (Maximum Likelihood Estimator). x ˆML = arg max fY (y | x), x∈X

equivalently, a solution to ∂ ln fY (y | x) = 0. ∂x

(A.12)

Maximum-likelihood estimators have some important properties, as we shall now show. The following theorem, is a corollary of the Cram´er-Rao lower bound. Theorem A.8. (1). If an eﬃcient estimator teﬀ exists, it is maximum-likelihood, i.e. teﬀ is the unique solution to (A.12). (2). If a suﬃcient estimator tsuﬀ exists, any maximum-likelihood estimate is a function of it, i.e. every solution to (A.12) is a function of tsuﬀ . Theorem A.8 makes statements about the form taken by eﬃcient and suﬃcient estimators, relating them to the ML estimate. The following theorem, proved in [25, p.500] describes the nature of the converse relationship. Theorem A.9 (Asymptotic Properties of ML Estimators). Every1 maximum-likelihood estimator is 1. Asymptotically eﬃcient. 2. Consistent. 3. Asymptotically normally distributed, with mean x, and variance given by (A.7). It may happen that a parameter is known to be an invertible function of some “underlying” parameter. In such cases, the maximum-likelihood estimates of the two parameters obey a simple, but very useful relationship.

248

A Estimation and Detection

Theorem A.10 (ML Estimation via Invertible Functions). Let x ∈ X be related to α ∈ A by a uniquely invertible function g : A → X . Let α ˆ M L and x ˆM L be the respective maximum likelihood estimators for α and x. Then α ˆ M L = g −1 (ˆ xM L ) . This is the so-called principle of invariance for maximum likelihood estimators [114, p. 217]. Proof. It is easy to see for any γ ∈ X , that x = γ and α = g −1 (γ) are the same event, and therefore result in the same value of the likelihood function for x, namely fY |X (y | x = γ) = fY |X (y | α = g −1 (γ)). It should be noted that this result does not hold for MAP estimation, or any method which incorporates the a-priori distribution on α, as the associated distribution on x will result from a transformation of variables under g. Example A.2. Let the observation vector y be related to a zero mean vector parameter x ∈ Rn via the noise linear transformation y = Ax + z where A is a known matrix and z is a zero-mean, zero-mode noise vector. Then the maximum likelihood estimate of x is A−1 y. This can be seen by considering the ML estimate of Ax, which is simply y (due to the zero-mode of z) and Theorem A.10. Example A.3. Let the observation vector y be related to a zero mean vector parameter x ∈ {−1, +1}n via the noise linear transformation y = Ax + z where A is a known matrix and z is a zero-mean, zero-mode noise vector. Unlike the previous example, in this case the ML estimate of x must be found by computing the posterior likelihood for each of the 2n possible vectors x. This shows how the form of the ML estimate can depend very strongly on the parameter support.

References

[1] E. Agrell, T. Eriksson, A. Vardy, and K. Zeger. Closest point search in lattices. IEEE Trans. Inform. Theory, 48:2201–2214, 2002. [2] R. Ahlswede. Multi-way communication channels. In 2nd Int. Symp. Inform. Theory, pages 23–52, Tsahkadsor, Armenian S.S.R, 1971. Hungarian Academy of Sciences. [3] P. Alexander, M. Reed, J. Asenstorfer, and C. Schlegel. Iterative multiuser interference reduction: turbo CDMA. IEEE Trans. Commun., 47 (7):1008–14, July 1999. [4] P. D. Alexander. Coded Multiuser CDMA. PhD thesis, University of South Australia, 1996. [5] P. D. Alexander, A. J. Grant, and M. C. Reed. Iterative detection on code-division multiple-access with error control coding. Europ. Trans. Telecommun., 9(5):419–426, 1998. [6] P. D. Alexander and L. K. Rasmussen. Processing ﬁlter derivation for asynchronous time–varying CDMA. In International Symposium on Information Theory and Its Applications, volume 1, pages 151–154, Sydney, Australia, 1994. [7] P. D. Alexander, L. K. Rasmussen, and C. B. Schlegel. A linear receiver for coded multiuser CDMA. IEEE Trans. Commun., 45(5):605–610, 1997. [8] O. Axelsson. Iterative Solution Methods. Cambridge University Press, 1994. [9] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv. Optimal decoding of linear codes for minimising symbol error rate. IEEE Trans. Inform. Theory, 20:284–287, 1974. [10] Z. D. Bai and Y. Q. Yin. Limit of the smallest eigenvalue of large dimensional sample covariance matrix. Ann. Probab., 21:1275–1294, 1993. [11] S. Baro, J. Hagenauer, and M. Witzke. Iterative detection of MIMO transmission using a list-sequential (LISS) detector. In IEEE Conf. Commun., pages 2653–2657, 2003.

250

References

[12] C. Berrou and A. Glavieux. Near optimum error correcting coding and decoding: turbo-codes. IEEE Trans. Commun., 44(10):1261–1271, 1996. [13] D. Bertsekas and R. Gallager. Data Networks. Prentice Hall, New Jersey, 1992. [14] P. A. B. M. C. v. d. Braak and H. C. A. v. Tilborg. A family of good uniquely decodeable code pairs for the two-access binary adder channel. IEEE Trans. Inform. Theory, IT-31(1):3–9, 1985. [15] M. V. Burnashev, C. Schlegel, W. A. Krzymien, and Z. Shi. Characteristics analysis of successive interference cancellation methods. Problemy Peredachi Informatsii, May 2004. Submitted. [16] G. Caire, R. R. Muller, and T. Tanaka. Iterative multiuser joint decoding: optimal power allocation and low-complexity implementation. IEEE Trans. Inform. Theory, 50(9):1950–1973, Sept. 2004. [17] S.-C. Chang. Further results on coding for T-user multiple access channels. IEEE Trans. Inform. Theory, IT-30(2):411–415, 1984. [18] S.-C. Chang and E. Weldon. Coding for T-user multiple access channels. IEEE Trans. Inform. Theory, IT-25(6):684–691, 1979. [19] S. C. Chang and J. K. Wolf. On the T-user M-frequency noiseless multiple access channel with and without intensity information. IEEE Trans. Inform. Theory, IT-27(1):41–48, 1981. [20] P. R. Chevillat. N-user trellis coding for a class of multiple-access channels. IEEE Trans. Inform. Theory, IT-27(1):114–120, 1981. [21] G. E. Corazza and A. Vanelli-Coralli. cdma2000. In Wiley Encyclopedia of Telecommunications, volume 1, pages 358–369. John Wiley & Sons, Hoboken, NJ, 2003. [22] T. M. Cover. Some advances in broadcast channels. In Advances in Communication Systems, volume 4, pages 229–260. Academic Press, New York, 1975. [23] T. M. Cover and S. K. Leung. An achievable rate region for the multiple access channel with feedback. IEEE Trans. Inform. Theory, IT-27(3): 292–298, 1981. [24] T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley, New York, 1991. [25] H. Cram´er. Mathematical Methods of Statistics. Princeton University Press, Princeton NJ, 1946. [26] M. O. Damen, H. El Gamal, and G. Caire. On maximum likelihood detection and the search for the closest lattice point. IEEE Trans. Inform. Theory, 49(10):2389–2402, Oct. 2003. [27] M. A. Deaett and J. K. Wolf. Some very simple codes for the nonsynchronized two-user multiple access adder channel with binary inputs. IEEE Trans. Inform. Theory, IT-24(5):635–636, 1978. [28] D. Divsalar, S. Dolinar, and F. Pollara. Iterative turbo decoder analysis based on density evolution. Technical Report 42-144, TMO Progress Report, Feb. 15 2001.

References

251

[29] D. Divsalar and M. K. Simon. CDMA with interference cancellation for multiprobe missions. Technical Report 42-120, JPL-TDA, Feb. 1995. Progress Report. [30] A. Duel-Hallen. Decorrelating decision–feedback multiuser detector for synchronous code–division multiple–access channel. IEEE Trans. Commun., 41(2):285–290, 1993. [31] A. Duel-Hallen. A family of multiuser decision–feedback detectors for asynchronous code–division multiple–access channels. IEEE Trans. Commun., 43(2):421–434, 1995. [32] H. G. Eggleston. Convexity. Cambridge University Press, London, 1958. [33] H. El-Gamal and E. Geraniotis. Iterative multiuser detection for coded CDMA signals in AWGN and fading channels. IEEE J. Select. Areas Commun., 18(1):30–41, 2000. [34] H. Elders-Boll, A. Busboom, and H. D. Schotten. Spreading sequences for zero-forcing DS-CDMA multiuser detectors. In 8th IEEE Int. Symp. PIMRC ’97, pages 53–57, Helsinki, Finland, Sept. 1997. [35] H. Elders-Boll, H.-D. Schotten, and A. Busboom. Eﬃcient implementation of linear multiuser detectors for asynchronous CDMA systems by linear interference cancellation. Europ. Trans. Telecommun., 9(5):427– 438, Sept-Oct 1998. Special Issue on Code Division Multiple Access Techniques for Wireless Communication Systems. [36] D. Eppstein. Finding the k shortest paths. Technical report, Dep. of Info. and Comp. Sci., University of California, 1994. [37] ETSI. Etsi/tc gsm recommendation 05.05: Transmission and reception, 1988. [38] ETSI/SMG/SMG2. The ETSI UMTS terrestrial radio access (UTRA) ITU-R RTT candidate submission, 1998. [39] B. Farhang-Boroujeny. MMSE linear processing for MIMO channel. Private communication, 2005. [40] P. G. Farrell. Survey of channel coding for multi-user systems. In J. Skwirynski, editor, New Concepts in Multi-User Communication, NATO Advanced Study Institute Series, pages 133–159. Sijthoﬀ and Noordhoﬀ, Alphen aan den Rijn, The Netherlands, 1981. [41] T. J. Ferguson. Generalised T-user codes for multiple-access channels. IEEE Trans. Inform. Theory, IT-28(5):775–779, 1982. [42] U. Fincke and M. Pohst. Improved methods for calculating vectors of short length in a lattice, including a complexity analysis. Mathematics of Computation, 44(170):463–471, 1985. [43] R. A. Fisher. On the mathematical foundations of theoretical statistics. Phil. Trans. Royal Soc., London, A, 222:309, 1921. [44] R. A. Fisher. Theory of statistical estimation. Proc. Cambridge Phil. Soc., 22:700, 1925. [45] G. D. Forney, Jr. The Viterbi algorithm. Proc. IEEE, 61:268–278, 1973.

252

References

[46] G. Foschini. Layered space-time architecture for wireless communication in a fading environment when using multi-element antennas. Bell Labs Tech. J., 1(2):41–159, Aug. 1996. [47] G. J. Foschnini, G. D. Golden, R. A. Valenzuela, and P. W. Wolniansky. Simpliﬁed processing for high spectral eﬃciency wireless communication employing multi-element arrays. IEEE J. Select. Areas Commun., 17 (11):1841–1852, Nov. 1999. [48] N. T. Gaarder and J. K. Wolf. The capacity of a multiple access discrete memoryless channel can increase with feedback. IEEE Trans. Inform. Theory, IT-21:100–102, 1975. [49] R. G. Gallager. Information Theory and Reliable Communication. John Wiley and Sons, New York, 1968. [50] R. G. Gallager. A perspective on multiaccess channels. IEEE Trans. Inform. Theory, IT-31(2):124–142, 1985. [51] A. E. Gamal and T. M. Cover. Multiple user information theory. Proc. IEEE, 68(12):1466–1483, 1980. [52] T. R. Giallorenzi and S. G. Wilson. Multiuser ML sequence estimator for convolutionally coded asynchronous DS-CDMA systems. IEEE Trans. Commun., 44(8):997–1008, 1996. [53] T. R. Giallorenzi and S. G. Wilson. Suboptimum multiuser receivers for convolutionally coded asynchronous DS-CDMA systems. IEEE Trans. Commun., 44(9):1183–1196, 1996. [54] R. Gold. Optimum binary sequences for spread spectrum multiplexing. IEEE Trans. Inform. Theory, 13:619–621, 1967. [55] G. Golub and C. V. Loan. Matrix Computations. The John Hopkins University Press, 2 edition, 1989. [56] A. Grant and P. Alexander. Random sequence multisets for synchronous code-division multiple-access channels. IEEE Trans. Inform. Theory, 44 (7):2832–2836, 1998. [57] A. Grant, B. Rimoldi, R. Urbanke, and P. Whiting. Rate-splitting multiple access for discrete memoryless channels. IEEE Trans. Inform. Theory, 47(3):873–90, 2001. [58] A. Grant and C. Schlegel. Iterative implementations for linear multiuser detectors. IEEE Trans. Commun., 49(10):1824–1834, Oct. 2001. [59] J. Hagenauer. A soft-in/soft-out list sequential (LISS) decoder for turbo schemes. In IEEE Int. Symp. Inform. Theory, page 382, 2003. [60] R. A. Horn and C. R. Johnson. Matrix Analysis. Oxford University Press, Cambridge, 1990. [61] B. Hughes and A. B. Cooper. Nearly optimal multiuser codes for the binary adder channel. IEEE Trans. Inform. Theory, 42(2):387–398, 1996. [62] J. Hui. Throughput analysis for code division multiple accessing of the spread spectrum channel. IEEE J. Select. Areas Commun., SAC-2(4): 482–486, 1984.

References

253

[63] J. Y. N. Hui and P. A. Humblet. The capacity region of the totally asynchronous multiple access channel. IEEE Trans. Inform. Theory, IT-31(2):207–216, 1985. [64] D. Jonsson. Some limit theorems for the eigenvalues of a sample covariance matrix. J. Mult. Anal., 12:1–38, 1982. [65] P. Jung and J. Blanz. Joint detection with coherent receiver antenna diversity in CDMA mobile radio systems. IEEE Trans. Veh. Technol., 44(1):76–88, 1995. [66] M. J. Juntti, B. Aazhang, and J. O. Lilleberg. Iterative implementations of linear multiuser detection for asynchronous CDMA systems. IEEE Trans. Commun., 46(4):503–508, April 1998. [67] T. Kasami and S. Lin. Coding for a multiple-access channel. IEEE Trans. Inform. Theory, IT-22(2):129–137, 1976. [68] T. Kasami and S. Lin. Bounds on the achievable rates of block coding for a memoryless multiple-access channel. IEEE Trans. Inform. Theory, IT-24(2):187–197, 1978. [69] T. Kasami and S. Lin. Decoding of linear δ-decodeable codes for a multiple-access channel. IEEE Trans. Inform. Theory, IT-24(5):633– 636, 1978. [70] T. Kasami, S. Lin, V. Wei, and S. Yamamura. Graph theoretic approaches to the code construction for the two-user multiple-access binary adder channel. IEEE Trans. Inform. Theory, IT-29(1):114–130, 1983. [71] S. M. Kay. Fundamentals of Statistical Signal Processing Volume I: Estimation Theory. Prentice Hall, Mew Jersey, 1993. [72] S. M. Kay. Fundamentals of Statistical Signal Processing Volume II: Detection Theory. Prentice Hall, Mew Jersey, 1998. [73] G. K. Khachatrian. On the construction of codes for noiseless synchronized 2-user channel. Prob. Contr. Inform. Theory, 11:319–324, 1987. [74] G. Kramer. Feedback strategies for a classof two-user multiple-access channels. IEEE Trans. Inform. Theory, 45(6):2054–2059, Sept. 1999. [75] C. Kuhn and J. Hagenauer. Iterative list-sequential (LISS) detector for fading multiple-access channels. In IEEE Global Commun. Conf., pages 330–335, 2004. [76] H. Liao. Multiple access channels. PhD thesis, Dept. of Electrical Eng., University of Hawaii, 1972. [77] S. Lin and D. J. Costello, Jr. Error Control Coding: Fundamentals and Applications. Prentice–Hall, New Jersey, 1983. [78] S. Lin and D. J. Costello, Jr. Error Control Coding. Prentice-Hall, Englewood Cliﬀs, 2 edition, 2004. [79] R. Lupas and S. Verd´ u. Linear multiuser detectors for synchronous code–division multiple–access channels. IEEE Trans. Inform. Theory, 35(1):123–136, 1989. [80] R. Lupas and S. Verd´ u. Near-far resistance of multiuser detectors in asynchronous channels. IEEE Trans. Commun., 38(4):496–508, 1990.

254

References

[81] D. J. C. MacKay. Good error-correcting codes based on very sparse matrices. IEEE Trans. Inform. Theory, IT-45(2):399–431, 1999. [82] U. Madhow and M. L. Honig. MMSE interference suppression for direct– sequence spread–spectrum CDMA. IEEE Trans. Commun., 42(12): 3178–3188, 1994. [83] J. L. Massey. Information theory aspects of spread-spectrum communications. In The Fifth International Symposium on Personal, Indoor and Mobile Radio Communications, pages 16–20, The Hague, The Netherlands, 1994. [84] J. L. Massey and P. Mathys. The collision channel without feedback. IEEE Trans. Inform. Theory, IT-31(2):192–204, 1985. [85] J. L. Massey and T. Mittelholzer. Welch’s bound and sequence sets for code-division multiple-access systems. In Sequences II, Methods in Communication, Security and Computer Science. Springer-Verlag, New York, 1993. [86] P. Mathys. A class of codes for a T-active users out of N multiple access communication system. IEEE Trans. Inform. Theory, 36(6):1206–1219, 1990. [87] E. C. v. d. Meulen. A survey of multi-way channels in information theory. IEEE Trans. Inform. Theory, IT-23(1):1–37, 1977. [88] M. Moher. An iterative multiuser decoder for near-capacity communications. IEEE Trans. Commun., 46(7):870–880, 1998. [89] M. Moher. Multiuser decoding for multibeam systems. IEEE Trans. Veh. Technol., 49(4):1226–1234, July 2000. [90] M. Moher, L. Erup, and R. G. Lyons. Interference statistics for multibeam satellites. In Second European Workshop on Mobile and Personal Satellite Comunications, pages 366–384, Oct. 1996. [91] H. Nyquist. Certain topics in telegraph transmission theory. AIEE Trans., 47:617–644, 1928. [92] E. Ordentlich. On the factor-of-two bound for Gaussian multiple-access channels with feedback. IEEE Trans. Inform. Theory, 42(6):2231–2235, 1996. [93] L. H. Ozarow. The capacity of the white Gaussian multiple access channel with feedback. IEEE Trans. Inform. Theory, IT-30:623–629, 1984. [94] P. Patel and J. Holtzman. Analysis of a simple successive interference cancellation scheme in a DS/CDMA system. IEEE J. Select. Areas Commun., 12(5):796–807, June 1994. [95] R. Peterson and D. Costello. Error probability and free distance bounds for two-user tree codes on multiple access channels. IEEE Trans. Inform. Theory, IT-26(6):658–670, 1980. [96] R. L. Peterson and D. J. Costello. Binary convolutional codes for a multiple-access system. IEEE Trans. Inform. Theory, IT-25(1):101–105, 1979.

References

255

[97] R. L. Peterson, R. E. Ziemer, and D. E. Borth. Introduction to Spread Spectrum Communications. Prentice-Hall, 1995. [98] E. Plotnik. Code construction for asynchronous random multiple-access to the adder channel. IEEE Trans. Inform. Theory, 39(1):195–197, 1993. [99] M. Pohst. On the computation of lattice vectors of minimal length, succesive minima and reduced basis with applications. ACM SIGSAM, 15:37–44, 1981. [100] G. S. Poltyrev. Coding for channel with asynchronous multiple access. Probl. Peredachi Informatsii, 19(3):12–21, 1983. [101] H. Poor and S. Verdu. Probability of error in MMSE multiuser detection. IEEE Trans. Inform. Theory, pages 858–871, May 1997. [102] H. V. Poor. An introduction to signal detection and estimation. Springer-Verlag, New York, 2 edition, 1994. [103] K. A. Post. Convexity of the nonachievable rate region for the collision channel without feedback. IEEE Trans. Inform. Theory, IT-31(2):205– 206, 1985. [104] J. G. Proakis. Digital Communications. McGraw-Hill, 4 edition, 2001. [105] C. R. Rao. Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc., 37:81–91, 1945. [106] P. B. Rapajic and D. Popescu. Information capacity of a random signature multiple-input multiple-output channel. IEEE Trans. Commun., 48(8):1245–1248, Aug. 2000. [107] P. B. Rapajic and B. S. Vucetic. Adaptive receiver structures for asynchronous CDMA systems. IEEE J. Select. Areas Commun., 12(4):685– 697, 1994. [108] M. Reed, C. Schlegel, P. Alexander, and J. Asenstorfer. Iterative multiuser detection for CDMA with FEC: near-single-user performance. IEEE Trans. Commun., 46(12):1693–9, Dec. 1998. [109] A. B. Reid, A. J. Grant, and P. D. Alexander. List detection for multiaccess channels. In IEEE Global Commun. Conf., pages 1083–1087, 2002. [110] A. B. Reid, A. J. Grant, and A. P. Kind. Low-complexity list-detection for high-rate multiple-antenna channels. In IEEE Int. Symp. Inform. Theory, page 273, Yokohama, Japan, 2003. [111] M. Rupf. Coding for CDMA Channels and Capacity. PhD thesis, Eidgen¨ ossiche Technische Hochschule, 1994. [112] M. Rupf and J. L. Massey. Optimum sequence multisets for synchronous code-division multiple-access channels. IEEE Trans. Inform. Theory, 40 (4):1261–1266, 1994. [113] C. Sankaran and A. Ephremides. Solving a class of optimum multiuser detection problems with polynomial complexity. IEEE Trans. Inform. Theory, 44:1958–1961, 1998. [114] L. L. Scharf. Statistical Signal Processing. Addison-Wesley, Reading, Massachusetts, 1991.

256

References

[115] C. Schlegel, P. Alexander, and S. Roy. Coded asynchronous CDMA and its eﬃcient detection. IEEE Trans. Inform. Theory, 44(7):2837–2847, 1998. [116] C. Schlegel, P. D. Alexander, S. Roy, and Z. Xiang. Multi-user projection receivers. IEEE J. Select. Areas Commun., 14(8):1610–1618, 1996. [117] C. Schlegel and A. Grant. Polynomial complexity optimal detection of certain multiple access systems. IEEE Trans. Inform. Theory, 46(6): 2246–8, Sept. 2000. [118] C. Schlegel and L. Wei. A simple way to compute the minimum distance in multiuser CDMA. IEEE Trans. Commun., 45(5):532–535, May 1997. [119] C. Schlegel and Z. Xiang. Multiuser projection receivers. In IEEE Int. Symp. Inform. Theory, page 318, Whistler, Canada, 1995. [120] C. B. Schlegel and L. Perez. Trellis and Turbo Coding. IEEE Press, Piscataway, NJ, 2004. [121] K. S. Schneider. Optimum detection of code division multiplexed signals. IEEE Transactions on Aerospace and Electronic Systems, AES-15(1): 181–185, Jan. 1979. [122] C. P. Schnorr and M. Euchner. Lattice basis reduction: improved practical algorithms and solving subset sum probelms. Mathematical Programming, 66:181–191, 1994. [123] C. E. Shannon. A mathematical theory of communication. Bell Syst. Tech. J., 27:379–423, 623–656, 1948. [124] C. E. Shannon. The zero error capacity of a noisy channel. IRE Trans. Inform. Theory, IT-2:8–19, 1956. [125] Z. Shi and C. Schlegel. Asymptotic iterative multi-user detection of coded random CDMA. Submitted to IEEE Trans. Signal Processing., 005. [126] Z. Shi and C. Schlegel. Joint iterative decoding of serially concatenated error control coded CDMA. IEEE J. Select. Areas Commun., pages 1646–1653, 2001. [127] J. W. Silverstein. Eigenvalues and eigenvectors of large dimensional sample covariance matrices. Contemporary Mathematics, 50:153–159, 1986. [128] J. W. Silverstein and Z. D. Bai. On the emperical distribution of eigenvalues of a class of large dimensional random matrices. J. Multivariate Anal., 54(2):175–192, Aug. 1995. [129] D. Slepian and J. K. Wolf. A coding theorem for multiple access channels with correlated sources. Bell Syst. Tech. J., 52(7), 1973. [130] R. Sorace. Trellis coding for the multiple access channel. IEEE Trans. Inform. Theory, IT-29(4):606–611, 1983. [131] G. Strang. Linear Algebra and its Applications. Harcourt Brace Jovanovich, 1988. [132] I. E. Telatar. Capacity of multi-antenna Gaussian channels. Europ. Trans. Telecommun., 10(6):585–595, Nov.-Dec. 1999. Origninally pub-

References

[133] [134] [135] [136] [137]

[138]

[139]

[140]

[141]

[142] [143] [144]

[145]

[146]

[147]

[148]

257

lished as Tech. Memo., Bell Laboratories, Lucent Technologies, October 1995. S. ten Brink. Convergence behavior of iteratively decoded parallel concatenated codes. IEEE Trans. Commun., 49(10):1727–1737, Oct. 2001. TIA. Tia/is-2000 interim standard, Introduction to cdma2000 spread spectrum systems, July 2004. TIA/EIA. IS-95: Mobile station-base station compatibility standard for dual-mode wideband spread spectrum cellular system, July 1993. H. v. Tilborg. An upper bound for codes in a two-access binary erasure channel. IEEE Trans. Inform. Theory, IT-24(1):112–117, 1978. L. G. F. Trichard, J. S. Evans, and I. B. Collings. Large system analysis of linear parallel interference cancellation. IEEE Trans. Commun., 50 (11):1778–1786, Nov. 2002. D. Tse and S. Hanly. Linear multiuser receivers: Eﬀective interference, eﬀective bandwidth and user capacity. IEEE Trans. Inform. Theory, 45 (2):641–657, 1999. S. Ulukus and R. Yates. Optimum mutliuser detection is tractable for synchronous CDMA using m-sequences. IEEE Commun. Lett., 2:89–91, 1998. G. Ungerboeck. Adaptive maximum–likelihood receiver for carrier– modulated data–transmission systems. IEEE Trans. Commun., 22(5): 624–636, 1974. R. Urbanke and Q. Li. The zero-error capacity region of the 2-user synchronous BAC is strictly smaller than its Shannon capacity region. In IEEE Information Theory Workshop, page 61, Killarney, Ireland, 1998. W. van Etten. Maximum likelihood receiver for multiple channel transmission system. IEEE Trans. Commun., 24(2):276–283, 1976. H. L. van Trees. Detection, Estimation, and Modulation Theory, Part I. John Wiley & Sons, 1968. P. Vanroose. Code construction for the noiseless binary switching multiple-access channel. IEEE Trans. Inform. Theory, 34(5):1100–1106, 1988. M. Varanasi and B. Aazhang. Multistage detection in asynchronous code-division multiple-access communication. IEEE Trans. Commun., 38(4):509–519, April. 1990. M. Varanasi and B. Aazhang. Near optimum detection in synchronous code-division multiple-access systems. IEEE Trans. Commun., 39(5): 725–736, May. 1991. S. Verd´ u. Minimum probability of error for asynchronous Gaussian multiple–access channels. IEEE Trans. Inform. Theory, 32(1):85–96, 1986. S. Verd´ u. The capacity region of the symbol-asynchronous Gaussian multiple-access channel. IEEE Trans. Inform. Theory, 35(4):733–751, 1989.

258

References

[149] S. Verd´ u. Computational complexity of optimum multiuser detection. Algorithmica, pages 303–312, 1989. [150] S. Verd´ u and S. Shamai. Spectral eﬃciency of CDMA with random spreading. IEEE Trans. Inform. Theory, 45(2):622–640, 1999. [151] H. Vikalo, B. Hassibi, and T. Kailath. Iterative decoding for mimo channels via modiﬁed sphere decoder. IEEE Trans. Wireless Commun., 3(6):2299–2311, Nov. 2004. [152] A. J. Viterbi. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Inform. Theory, IT-13: 260–269, 1967. [153] A. J. Viterbi. Very low rate convolutional codes for maximum theoretical performance of spread-spectrum multiple-access channels. IEEE J. Select. Areas Commun., 8:641–649, 1990. [154] A. J. Viterbi. CDMA: Principles of Spread Spectrum Communication. Addison-Wesley, 1995. [155] A. J. Viterbi and J. K. Omura. Principles of Digital Communication and Coding. McGraw–Hill, 1979. [156] K. W. Wachter. The strong limits of random matrix spectra for sample matrices of independent elements. Ann. Probab., 6:1–18, 1978. [157] X. Wang and V. Poor. Iterative (turbo) soft interference cancellation and decoding for coded CDMA. IEEE Trans. Commun., 47(7):1046–61, July 1999. [158] L. Wei and C. Schlegel. Synchronous DS-SSMA with improved decorrelating decision–feedback multiuser detection. IEEE Trans. Veh. Technol., 43(3):767–772, 1994. Special Issue on Future PCS Technologies. [159] L. R. Welch. Lower bounds on the maximum cross correlation of signals. IEEE Trans. Inform. Theory, IT-20(3):397–399, 1974. [160] E. J. Weldon. Coding for a multiple access channel. Information and Control, 36:256–274, 1978. [161] K. Wesolowski. Mobile Communication Systems. John Wiley & Sons, Chichester, England, 2002. [162] L. Wilhelmsson and K. S. Zigangirov. On the transmission over the T-user q-ary multiple access permutation channel. In IEEE Int. Symp. Inform. Theory, page 56, Trondheim, Norway, 1994. [163] F. M. J. Willems. The feedback capacity of a class of discrete memoryless multiple access channels. IEEE Trans. Inform. Theory, IT-28(1):93–95, 1982. [164] F. M. J. Willems and E. C. v. d. Meulen. Partial feedback for the discrete memoryless multiple access channel. IEEE Trans. Inform. Theory, 29 (2):287–290, 1983. [165] J. K. Wolf. Multi-user communications networks. In J. Skwirynski, editor, Communication Systems and Random Process Theory, NATO Advanced Study Institute Series, pages 37–53. Sijthoﬀ and Noordhoﬀ, Alphen aan den Rijn, The Netherlands, 1978.

References

259

[166] J. M. Wozencraft and I. M. Jacobs. Principles of Communications Engineering. J. Wiley and Sons, New York, 1965. [167] J. M. Wozencraft and I. M. Jacobs. Principles of Communications Engineering. Waveland Press, 1993. [168] D. Wubben, B. Bohnke, J. Rinas, V. Kuhn, and K. D. Kammeyer. Efﬁcient algorithm for decoding layered space-time codes. IEE Electron. Lett., 37:1384–1350, Oct. 2001. [169] A. D. Wyner. Recent results in Shannon theory. IEEE Trans. Inform. Theory, IT-20(1):2–10, 1974. [170] A. D. Wyner. Shannon-theoretic approach to Gaussian cellular multipleaccess channel. IEEE Trans. Inform. Theory, 40(6):1713–27, 1994. [171] Z. Xie, R. T. Short, and C. K. Rushforth. A family of suboptimum detectors for coherent multiuser communications. IEEE J. Select. Areas Commun., 8(4):683–690, 1990. [172] Y. Q. Yin and Z. D. Bai. Spectra for large-dimensional random matrices. Contemporary Mathematics, 50:161–167, 1986. [173] Y. Yoon, R. Kohno, and H. Imai. A spread-spectrum multiaccess system with cochannel interference cancellation for multipath fading channels. IEEE J. Select. Areas Commun., 11:1067–1075, Sept. 1993.

Author Index

Agrell et al. [2002] 168, 249 Ahlswede [1971] 6, 53, 60, 249 Alexander and Rasmussen [1994] 133, 249 Alexander et al. [1997] 178, 184, 186, 249 Alexander et al. [1998] 179, 249 Alexander et al. [1999] 179, 249 Alexander [1996] 133, 249 Axelsson [1994] 139, 142, 146, 147, 249 Bahl et al. [1974] 107, 109, 249 Bai and Yin [1993] 127, 249 Baro et al. [2003] 173, 249 Berrou and Glavieux [1996] 99, 176, 249 Bertsekas and Gallager [1992] 94, 250 Braak and Tilborg [1985] 76, 77, 250 Burnashev et al. [2004] 207, 250 Caire et al. [2004] 233, 234, 250 Chang and Weldon [1979] 58, 59, 78–80, 250 Chang and Wolf [1981] 58, 59, 80, 250 Chang [1984] 80, 250 Chevillat [1981] 76, 81, 250 Corazza and Vanelli-Coralli [2003] 219, 250 Cover and Leung [1981] 86, 250 Cover and Thomas [1991] 4, 53, 58, 227, 240, 243, 244, 250 Cover [1975] 60, 250 Cram´er [1946] 243, 247, 250 Damen et al. [2003] 168, 250 Deaett and Wolf [1978] 76, 79, 250

Divsalar and Simon [1995] 141, 250 Divsalar et al. [2001] 196, 250 Duel-Hallen [1993] 130, 251 Duel-Hallen [1995] 130, 251 ETSI/SMG/SMG2 [1998] 139, 251 ETSI [1988] 9, 251 Eggleston [1958] 53, 84, 251 El-Gamal and Geraniotis [2000] 179, 251 Elders-Boll et al. [1997] 139, 143, 251 Elders-Boll et al. [1998] 139, 143, 251 Eppstein [1994] 172, 251 Farhang-Boroujeny [2005] 149, 251 Farrell [1981] 74, 251 Ferguson [1982] 79, 251 Fincke and Pohst [1985] 165, 168, 251 Fisher [1921] 244, 251 Fisher [1925] 244, 251 Forney [1973] 106, 251 Foschini [1996] 175, 251 Foschnini et al. [1999] 168, 252 Gaarder and Wolf [1975] 85, 252 Gallager [1968] 16, 17, 101, 252 Gallager [1985] 53, 252 Gamal and Cover [1980] 53, 252 Giallorenzi and Wilson [1996] 176, 178, 252 Gold [1967] 163, 186, 252 Golub and Loan [1989] 110, 119, 128, 132, 142, 252 Grant and Alexander [1998] 67, 252 Grant and Schlegel [2001] 143, 145–148, 150, 252

262

Author Index

Grant et al. [2001] 81, 91, 252 Hagenauer [2003] 173, 252 Horn and Johnson [1990] 125, 142, 144, 146, 148, 252 Hughes and Cooper [1996] 79, 252 Hui and Humblet [1985] 91, 252 Hui [1984] 67, 252 Jonsson [1982] 65, 253 Jung and Blanz [1995] 120, 253 Juntti et al. [1998] 143, 253 Kasami and Lin [1976] 76, 77, 253 Kasami and Lin [1978] 77, 253 Kasami et al. [1983] 78, 253 Kay [1993] 237, 253 Kay [1998] 237, 253 Khachatrian [1987] 76, 253 Kramer [1999] 87, 253 Kuhn and Hagenauer [2004] 173, 253 Liao [1972] 53, 60, 253 Lin and Costello [1983] 77, 106, 253 Lin and Costello [2004] 175, 253 Lupas and Verd´ u [1989] 120, 253 Lupas and Verd´ u [1990] 120, 253 MacKay [1999] 99, 253 Madhow and Honig [1994] 126, 254 Massey and Mathys [1985] 74, 91, 93, 254 Massey and Mittelholzer [1993] 44, 254 Massey [1994] 33, 254 Mathys [1990] 80, 254 Meulen [1977] 4, 254 Moher et al. [1996] 41, 254 Moher [1998] 113, 179, 194, 254 Moher [2000] 41, 254 Nyquist [1928] 1, 254 Ordentlich [1996] 90, 254 Ozarow [1984] 88, 254 Patel and Holtzman [1994] 141, 254 Peterson and Costello [1979] 76, 81, 254 Peterson and Costello [1980] 81, 254 Peterson et al. [1995] 98, 254 Plotnik [1993] 80, 255 Pohst [1981] 165, 168, 255 Poltyrev [1983] 91, 255 Poor and Verdu [1997] 125, 179, 255 Poor [1994] 237, 255 Post [1985] 94, 255

Proakis [2001] 15, 101, 110, 184, 186, 255 Rao [1945] 243, 255 Rapajic and Popescu [2000] 67, 255 Rapajic and Vucetic [1994] 126, 255 Reed et al. [1998] 179, 255 Reid et al. [2002] 172, 255 Reid et al. [2003] 172, 255 Rupf and Massey [1994] 64, 255 Rupf [1994] 64, 255 Sankaran and Ephremides [1998] 112, 255 Scharf [1991] 237, 248, 255 Schlegel and Grant [2000] 115, 256 Schlegel and Perez [2004] XVII, 99, 102, 106, 107, 109, 111, 116, 162, 175, 176, 185–187, 192, 193, 200, 225, 227, 256 Schlegel and Wei [1997] 110, 112, 256 Schlegel and Xiang [1995] 184, 256 Schlegel et al. [1996] 178, 256 Schlegel et al. [1998] 178, 255 Schneider [1979] 28, 29, 97, 100, 256 Schnorr and Euchner [1994] 165, 168, 256 Shannon [1948] 1, 175, 256 Shannon [1956] 85, 93, 256 Shi and Schlegel [005] 179, 256 Shi and Schlegel [2001] 179, 256 Silverstein and Bai [1995] 69, 215, 256 Silverstein [1986] 65, 256 Slepian and Wolf [1973] 53, 60, 256 Sorace [1983] 81, 256 Strang [1988] 119, 142, 146, 256 TIA/EIA [1993] 10, 97, 101, 102, 139, 219, 257 TIA [2004] 3, 10, 97, 102, 139, 257 Telatar [1999] 175, 256 Tilborg [1978] 78, 257 Trichard et al. [2002] 153, 154, 257 Tse and Hanly [1999] 51, 135, 211, 212, 257 Ulukus and Yates [1998] 113, 257 Ungerboeck [1974] 104, 257 Urbanke and Li [1998] 75, 76, 257 Vanroose [1988] 80, 257 Varanasi and Aazhang [1990] 141, 145, 257 Varanasi and Aazhang [1991] 145, 257

Author Index Verd´ u and Shamai [1999] 66, 67, 72, 127, 258 Verd´ u [1986] 28, 97, 100, 102, 257 Verd´ u [1989] 29, 64, 91, 92, 110, 158, 257 Vikalo et al. [2004] 172, 258 Viterbi and Omura [1979] 185, 258 Viterbi [1967] 81, 106, 258 Viterbi [1990] 228, 258 Viterbi [1995] 10, 98, 118, 134, 258 Wachter [1978] 65, 258 Wang and Poor [1999] 179, 209, 258 Wei and Schlegel [1994] 162, 258 Welch [1974] 43, 65, 258 Weldon [1978] 78, 258 Wesolowski [2002] 219, 258 Wilhelmsson and Zigangirov [1994] 80, 258

263

Willems and Meulen [1983] 86, 258 Willems [1982] 87, 258 Wolf [1978] 59, 79, 258 Wozencraft and Jacobs [1965] 101, 119, 258 Wozencraft and Jacobs [1993] 15, 203, 259 Wubben et al. [2001] 168, 259 Wyner [1974] 60, 259 Wyner [1994] 39, 259 Xie et al. [1990] 126, 259 Yin and Bai [1986] 65, 259 Yoon et al. [1993] 141, 259 ten Brink [2001] 196, 257 van Etten [1976] 28, 29, 97, 100, 257 van Trees [1968] 237, 257

Subject Index

a posteriori probability 29, 30, 193 achievable rate 50 region 51, 53, 86 algorithm M - 162 T 111, 169 BCJR 107 bi-directional 107 shortest path 172 sphere detector 168 Viterbi 106, 185 amplitude 16 matrix 19, 24, 35 approximate APP 170 asymmetric operation 219 asymptotic negligibility 134, 222 asynchronous 91, 132 frame 79, 91 users 103 band-diagonal 102, 139 bandwidth 17, 31, 33, 62 basis lattice 164 orthonormal 17, 189 Bayes criterion 238 estimation 237 estimator 239, 240, 242, 244, 246 risk 238 rule 29, 246 bias 241 block code 75

bound Shannon 116, 123, 136, 226 Welch 43, 65, 67, 118 branch metric 104, 159, 161, 172 branch-and-bound 99 cancellation 58 parallel interference 143 perfect 195 serial 148 signal 194 successive 227 capacity information theoretic 116 per chip 208 per dimension 177 polytope 228 upper bound 156 capacity region 48, 53 asynchronous 91–93 binary adder channel 55, 87 binary multiplier channel 59 code-division multiple-access 64, 66 Gaussian multiple-access channel 60, 88 vertex 58, 81 Cayley Hamilton theorem 146 CDMA 175 asynchronous 186 joint decoding 99 synchronous 186 time-varying 101 cellular network 39 central limit theorem 179, 195

266

Subject Index

channel binary input 204 CDMA 178 MIMO 175 multiple-access 116, 193 single-user 116, 175 channel layering 116 chip 32, 33 matched ﬁlter 34 Cholesky factorization 110 code block 75 convolutional 81 Gold 186 LDPC 175 multiple-access 49, 73 outer 192 parity check 197 powerful 222 repetition 203, 206, 217 serial concatenated 200 Shannon-type 224 simple 202 strong 199, 231 trellis 175 turbo 175 weak 200 code-division multiple-access 32, 63 capacity region 64, 66 channel model 22, 35 direct sequence 33 codebook 187 codeword estimator 202 coding error control 192 low-density parity-check 99 turbo 99 collision channel 93 complexity state space 107 sub-exponential 112 concatenation 178 serial 192 convergence asymptotic factor 143, 147 bottleneck 200 optimal 153 speed 147 tunnel 234

convex hull 53, 55, 58, 84, 91 convexity 191 convolutional code 81 cooperation receiver 50 transmitter 47, 55, 57, 59 correlation detector 98 equal 113 reception 97 correlation matrix 24, 25, 27 asynchronous channel 26 narrow-band channel 37 orthogonal 31 synchronous channel 27 correlator 23, 27, 42, 67 decision-feedback 130 decoder APP 193 iterative multiuser 193 state 104 decoding iterative 192, 196 optimal 178 single-user 178, 179 trellis 102 turbo 107 decomposition Cholesky 128, 160 QR 159 spectral 152 decorrelator 99, 119 improved decision feedback 162 information theoretic capacity 123 partial 131 random spreading 122 Shannon bound 123 delay 16–18, 21, 33, 35, 92 detector 238 a posteriori probability 107 correlation 118 individually optimal 30 jointly optimal 29, 158 minimum mean-square error 124, 125, 135 optimal 29, 67, 100, 194, 246, 247 sub-exponential 112 single-user 23, 27, 42, 56, 61, 67, 81

Subject Index sphere 165 initial radius 168 list 171 Schnorr-Euchner 169 distance Euclidean 109, 158, 202 Hamming 188 minimum 109 dynamical system 222 eﬀective interference 212 eigenvalue 64, 65, 142, 215 distribution 65, 66, 70, 153 minimum 144 energy degradation 185 energy loss 183 equation characteristic 146 error cliﬀ 198, 201 error event pair-wise 188 error ﬂoor 201 error probability 27–30, 42, 50, 246 projection receiver 189 two-codeword 192 estimator 238 consistent 242, 247 eﬃcient 244, 247 Euclidean distance 109, 202 exponential distribution 224 factorization Cholesky 110 asynchronous 133 feedback 84–88 ﬁlter loop low-complexity 214 two-stage 216 loop matched 205 MMSE per-user 212, 217 non-linear 204 whitening 128, 160 asynchronous 132 Fincke-Pohst enumeration 168 ﬁnite-state machine 102 ﬁrst-order stationary method 147 ﬁx point 198

forward error control 175 frequency-division multiple-access 42, 62

267

31,

Gauss-Seidel 142 Gaussian assumption 27 distribution 24, 36, 42, 64 linear model 241, 244, 245 multiple-access channel 59, 64 noise 16, 18, 24, 29, 35, 60, 101, 196 generator polynomial 186 geometry 120 projection receiver 182 sphere detector 172 gradient 149 Hamming distance 245 weight 188, 191 hard decision 145, 179 independence assumption 193 inequality Jensen 135, 154, 191 inter-symbol interference 26, 48, 185 interference cancellation statistical 99 successive 99 interleaver 176 random 177 isotropic 42 iterated map 198 iterative inversion 139 minimization 150 update equation 141 iterative cancellation 208 iterative decoding 192 iterative layering capacity 208 Jacobi decorrelator 145 method 143 performance 143 receiver 145 Jensen’s inequality 134

268

Subject Index

large systems analysis 65, 66, 70, 143, 145, 212 Lindeberg condition 134, 135, 222 linear preprocessor 116 linear optimization 234 load average system 232 partial maximum 231 supportable 233 uncoded 208 log-likelihood ratio 193, 196 extrinsic 206 loop ﬁlter 194, 195, 209 low-complexity 194 lower diagonal 149 lower triangular 110, 128, 148 marginalization 107 matched ﬁlter 24, 27–29, 31, 34, 67, 92, 196, 209, 241 whitened 160 matched ﬁltering 118 matrix block 192 cross-correlation 103 inversion lemma 125 iterative inversion 139 lower triangular 160 partitioned inversion lemma 192 permutation 110 projection 189 splitting 142 maximum a posteriori detector 30, 246 maximum likelihood detector 29, 247 maximum-likelihood approximate 158 estimate 98, 106 method Chebyshev 146 Gauss-Seidel 148 lattice 164 tree search 158, 161 metric branch 159, 166 Euclidean 160 monotonic 159

monotonic property 166 recursive 158 Ungerb¨ ock 161 minimum distance 109 minimum mean-square error detector 99, 124, 125 system capacity 135 performance 126 random spreading 127 minimum mean-squared error 242–244 layering 208 linear 245 minimum variance ﬁlter 125 MMSE solution 149 modulation baseband 15 BPSK 15, 204 matrix 18, 22, 35, 42, 100 asynchronous 23 waveform 16, 18, 32, 36, 92 modulation matrix 31 modulation waveform 16 multi-stage approximation 214 multiple antenna channel 37 multiple-access channel 46 asynchronous 90 code 49, 73 continuous time model 14 discrete memoryless 47 discrete time model 17 feedback 84 Gaussian 59 asynchronous 92 feedback 88 matrix model 18, 35 narrow band 36 orthogonal 61 synchronous 21 multiplicity 110 mutual information 51, 53 near-far problem 97 resistance 163 near-optimal 112 noise limitation 199 NP-hard 29, 158

Subject Index optimal detector 67 individually optimal 30 jointly optimal 29 orthogonality principle 209 parent node 161 partial load 222 partitioning 180 permutation function 167 phase-shift keying 15 power average 223 control 219 equal 225 group 222 optimal distribution 222, 226 received 119 received levels 134 transmitted 16 unequal 219 power group ﬁnite 232 pre-processing 178 projection ﬁltering 184 matrix 189 projection receiver performance 185, 191 pseudo-inverse 121 quadratic form 102, 104 function 149 rate unequal 228 Rayleigh fading 72 receiver iterative multistage 151 multistage 139, 141 projection 179 relaxation parameter 148 root node 162 sampling 17, 18, 22, 34, 35 satellite 41 scaling factor 210 scenario worst-case 136

269

Schnorr-Euchner enumeration 168 sequence m 113 detector extended 184 maximum-likelihood 178 metrics 103 signature 112 Shannon bound 116, 123, 155 unequal power 137 signal layering 115 signal-to-interference asymptotic 211 signal-to-noise ratio threshold 200 single-user detector 23, 27, 42, 56, 61, 67, 81 sliding window 133 soft bits 195 soft-output 193 spectral eﬃciency 206 spectral radius 147 sphere decoder 99 spreading 16, 18, 22, 32, 100 periodic 36, 64 random 36, 65, 69, 70, 101, 112, 122, 127, 139, 143, 185, 212 sequence 33 sequences 33 stationarity parameter 216 stationary method 147 statistic 238 Stieltjes transform 69, 70 strong user 136 successive relaxation 148 suﬃcient statistic 17, 24, 28, 29, 34, 102, 240, 243 minimal 241 superposition 58, 81 supportable load 203, 233 survivor 106 symbol period 15 synchronous chip 32, 34, 35 phase 182 symbol 21, 22, 27, 31, 49, 91, 101 system capacity correlation receiver 134 decorrelator 135

270

Subject Index

system load

uniquely decodeable 81 unitary transformation 160 upper bound 191 upper diagonal 149 upper triangular 110

216

theorem of irrelevance 106 time sharing 53, 84, 91 region 56 time-division multiple-access 61, 62 trajectory 198 tree search 99 breadth-ﬁrst 162 depth-ﬁrst 167, 173 trellis code nonlinear 81 turbo cliﬀ 201, 216

31, 42,

variance transfer 232 variance transfer analysis 195 variance transfer function 213 variational problem 234 Viterbi algorithm 106, 185 weak user 136 whitening ﬁlter local optimality

130