Texts in Computer Science Editors David Gries Fred. B. Schneider
For other titles published in this series, go to www.springer.com/series/3191
David Salomon
The Computer Graphics Manual
Prof. David Salomon (emeritus) Department of Computer Science California State University Northridge, CA 91330-8281, USA
[email protected] Series Editors David Gries Department of Computer Science Upson Hall Cornell University Ithaca, NY 14853-7501, USA
Fred. B. Schneider Department of Computer Science Upson Hall Cornell University Ithaca, NY 14853-7501, USA
ISSN 1868-0941 e-ISSN 1868-095X ISBN 978-0-85729-885-0 e-ISBN 978-0-85729-886-7 DOI 10.1007/978-0-85729-886-7 Springer London Dordrecht Heidelberg New York British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress Control Number: 2011937970 © Springer-Verlag London Limited 2011 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licenses issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc., in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
To users of computer graphics everywhere
There’s no better sensation than image. It’s so in-your-face!
—Jimmy Lai
Preface The field of quantum mechanics is the cornerstone of modern physics. This field was rapidly developed in the 1920s and 1930s by a small group of (mostly young) researchers. They generally agreed that we cannot (and indeed, should not even try to) visualize atoms, photons, and elementary particles. These objects, their attributes and their behavior are best described in terms of mathematical abstractions, not pictures. Indeed, one of the first textbooks in this area The Principles of Quantum Mechanics by P. A. M. Dirac, does not include a single diagram in its 314 pages. This style of writing reflects one extreme approach to graphics, namely considering it irrelevant or even detracting as a teaching tool and ignoring it. Today, of course, this approach is unthinkable. Graphics, especially computer graphics, is commonly used in texts, advertisements, and videos to illustrate concepts, to emphasize points being discussed, and to entertain. Our approach to graphics has been completely reversed since the 1930s, and it seems that much of this change is due to the wide use of computers. Computer graphics today is a mature, successful, and growing field. It is employed by many people for many purposes and it is enjoyed by even more people. One criterion for the maturity of a field of study is its size. When a certain discipline becomes so big that no one person can keep all of it in their head, we say that that discipline has matured (or has come of age). This is what happened to computer graphics in the last decade or so. It is now a large field consisting of many subfields such as curve and surface design, rendering methods, and computer animation. Even a person who has written a book covering the entire field cannot claim that they keep all that material in their head all the time, which is precisely the reason why textbooks are being written.
In its 357 pages, The Principles of Quantum Mechanics featured neither a single diagram, nor an index, nor a list of references, nor suggestions for further reading. —Graham Farmelo, The Strangest Man, 2009.
vii
viii
Preface
Overview and Goals Today (in 2011), the power of computer-generated images is everywhere. Computer graphics has pervaded our lives to such an extent that sometimes we don’t even realize that an image we are watching is artificial. The average person comes into contact with computer graphics mostly in three areas, computers, television, and electronic devices. Current computers and operating systems are based on GUI (graphical user interface). Computer programs often display results graphically. Television programs and commercials employ sophisticated, computer-generated graphics that are often hard to distinguish from the real thing. Many television programs (mostly documentaries) and recent movies mix real actors and artificial imagery to such an extent that the viewer may find it difficult to distinguish a real object or scene from a computer-generated image. (A real actor trying to outrun a computer-generated dinosaur is a common example.) More and more digital cameras, electronic devices, and instruments have small screens that display messages, options, controls, and results in color and are often touch sensitive, enabling the user to enter commands by finger gesturing instead of from the traditional keyboard. Many cell telephones even have two screens, and some new digital cameras also feature two LCD displays. With this in mind, the goal of this manual is to present the reader with a wide picture of computer graphics, its history and its pioneers, the hardware tools it employs, and most important, the techniques, approaches, and algorithms that are at the core of this field. Thus, this textbook/reference tries to describe as many concepts and algorithms as possible, paying special attention to the important ones. It would have been nice to include everything in this book and title it, like other texts by the same author, Computer Graphics: The Complete Reference, but computer graphics has grown to a point where I cannot hope to be an authority on the entire field, which is why some readers may not find every topic, term, concept, and algorithm they may be looking for. New material for Volume 4 will first appear in beta-test form as fascicles of approximately 128 pages each, issued approximately twice per year. These fascicles will represent my best attempt to write a comprehensive account; but computer science has grown to the point where I cannot hope to be an authority on all the material covered in these books. Therefore I’ll need feedback from readers in order to prepare the official volumes later. —Donald E. Knuth. On the other hand, those same readers may find in this manual/textbook topics they did not know existed, which might serve as compensation. The many examples and exercises sprinkled throughout the book enhance its usefulness. By paying attention to the examples and working out the exercises, readers will gain deeper understanding of the material.
Organization and Features This manual is large and is organized in seven parts as follows: Part I covers the history, basic concepts, and techniques used in computer graphics. The concepts of pixel, vector scan, and raster scan are discussed. It is shown how an
Preface
ix
image given as a bitmap of pixels can be scaled (zoomed) and rotated. Many important scan-conversion methods are explained and illustrated. Part II is devoted to transformations and projections. It starts with the important two- and three-dimensional transformations, including translation, rotation, reflection, and shearing. This is followed by the main types of projections, namely parallel, perspective, and nonlinear. Part III is by far the largest. It includes many methods, algorithms, and techniques employed to construct curves and surfaces, which are the building blocks of solid objects. Six important interpolation and approximation methods for curves and surfaces are included, as well as sweep surfaces and subdivision methods for surface design. Part IV goes into advanced techniques such as rendering an object, eliminating those parts that are hidden from view, and bringing objects to life (animating them) by interpolation. Several chapters included in this part discuss the nature and properties of light and color, graphics standards and graphics file formats, and fractals. Part V describes the principles of image compression. It concentrates on two important approaches to this topic, namely orthogonal and subband (wavelet) transforms. The important JPEG image compression standard is described in detail. Part VI is devoted to many of the important input/output graphics devices. Chapter 26 describes them and explains their operations. Part VII consists of appendixes, most of which discuss certain mathematical concepts. The following features enhance the usefulness and appearance of this textbook: The powerful MathematicaTM and Matlab software systems are used throughout the book to implement the various concepts discussed. When a figure is computed in one of these programs, the code is listed with the figure. These codes, which are available in the book’s website, are meant to be readable rather than efficient and fast, and are therefore easy to read and to modify even by inexperienced Mathematica and Matlab users. The book has many examples. Experience shows that examples are important for a thorough understanding of the kind of material discussed in this manual. The conscientious reader should follow each example carefully and try to work out variations of the examples. Many examples also include Mathematica code. The many exercises sprinkled throughout the text are not a cosmetic feature. They deal with important topics, and should be worked out. Answers are provided but they should be consulted only as a last resort. A quotation is a phrase that reflects its author’s profound thoughts. Quotations and epigrams enliven a book, which is why they have been generously used throughout this manual. I hope that they add to the book and make it more interesting. The ability to quote is a serviceable substitute for wit. —W. Somerset Maugham. This manual/textbook aims to be practical, not theoretical. After reading and understanding a topic, the reader should be able to design and implement the concepts
x
Preface
discussed there. The few mathematical arguments found in the book are simple, and there is no attempt to present an overall theory encompassing the entire field of computer graphics. An important feature of this text is the attention paid to orphans. Those are topics that most texts on computer graphics either mention briefly or disregard completely. Examples are perspective projections, nonlinear projections, nonlinear bitmap transformations, curves, surfaces, I/O devices, and image compression. The reader will find that this manual discusses orphans in great detail, including numerous examples and exercises. Most of the necessary mathematical background (such as vectors and matrices) is covered in the Appendixes. However, some math concepts that are used only once (such as the mediation operator and points vs. vectors) are discussed right where they are introduced.
The Two Volumes This textbook/reference is big because the discipline of computer graphics is big. There are simply many topics, techniques, and algorithms to discuss, explain, and illustrate by examples. Because of the large number of pages, the book has been produced in two volumes. However, this division of the book into volumes is only technical and the book should be considered a single unit. It is wrong to consider volume I as introductory and volume II as advanced, or to assume that volume I is important while volume II is not. The volumes are simply two halves of a single, large entity.
The Color Plates This extensive manual features more than 100 color plates, placed at the very beginning, at the end, and between individual parts. They serve to liven up the book and to illustrate many of the topics discussed. It is planned to place information about these plates in the book’s website, for the benefit of readers who want to recreate or extend them. The plates were prepared over several months, using a variety of graphics software. Appendix F has more information about the plates, their content, and the graphics applications used to generate them.
A Word on Notation It is common to represent nonscalar quantities such as points, vectors, and matrices with boldface. Here are examples of the notation used in this manual: x, y, z, t, u, v
Italics are used for scalar quantities such as coordinates and parameters.
P, Qi , v, M
Boldface is used for points, vectors, and matrices.
CP
An alternative notation for vectors, used when the two endpoints of the vector are known.
Preface P(t), P(u, v)
a12 a22
a11 a21
a11 a21
Boldface with arguments is used for nonscalar functions such as curves and surfaces.
a12 a22
Parentheses (and sometimes square brackets) are used for matrices. Vertical bars are used for determinants.
|v|
The absolute value (length) of vector v.
AT
The transpose of matrix A.
x∗ , P∗
The transformed values of scalars and points.
f u (u), Pt (t), Ptt (t)
The derivatives (first, second,. . . ) of scalar and vector functions.
df (u) dP(t) , du dt
Alternative notation for derivatives.
df 2 (u) dP2 (t) , du2 dt2
Alternative notation for higher-order derivatives.
∂f (u, v) ∂P(u, v) , ∂u ∂v
Partial derivatives.
f (x)|x0 or f (x0 )
Value of function f (x) at point x0 .
n
xi
xi
The sum x1 + x2 + · · · + xn .
xi
The product x1 x2 . . . xn .
i=1 n i=1
Exercise 1: What is the meaning of (P1 , P2 , P3 , P4 )?
Target Audiences The material presented here has been developed over many years of teaching computer graphics. It has been revised, updated, and distilled many times, with many figures, examples, and exercises added. The text emphasizes the simplicity of the mathematics behind computer graphics and tries to show how graphics software works and how current computer graphics can generate and display realistic-looking curves, surfaces, and solid objects. The key ideas are introduced slowly, are examined, when possible, from several points of view, and are illustrated and illuminated by figures, examples, and (solved) exercises. The discussion must employ mathematics, but it is
Preface
xii
mostly non-rigorous (no theorems or proofs) and therefore easy to grasp. The mathematical background required includes polynomials, matrices, vector operations, and elementary calculus. This manual/textbook was written with three groups of readers in mind. Textbook/student. The book can serve as the primary text for a two-semester class on computer graphics for graduate and advanced undergraduate students. The many fragments of Mathematica code found here may serve as a core around which students can build larger programs. The exercises are especially valuable for students, and the lack of rigorous theorems and proofs should encourage those who consider computer science distinct from mathematics. Reference/professional use. Professionals in engineering, computers, and other scientific disciplines generate, watch, and process digital images all the time. They often would like to know more about how such images are generated. Artists, photographers, and publishing professionals use computer graphics routinely and may be interested in a solid background in this field. Those readers can benefit from two features of this book, namely the detailed index and the thorough and precise exposition of the principles, methods, and techniques used in computer graphics. This textbook/reference tries to cover all the important topics of the graphics field, some in more detail than others. The index is exceptionally detailed and constitutes 2.5% of the book. Individuals/handy resource. All of us, not just certain professionals, are constantly exposed to digital images, digital effects, and computer animation. Intelligent persons who try to widen their horizons may wonder how digital images are created, edited, and distributed. It is my belief that this manual can serve the needs of this group of individuals as well. The book may prove useful to them because of the simple, straightforward descriptions of graphics devices and processes. The book may also be a handy resource because of the detailed index, which makes it easy to locate any topic.
Supplementary Resources Because of the importance of computer graphics, a vast number of supplementary resources is available. Section 1.5 has a long list of seven classes of resources and this only scratches the surface of what is available. The Internet has many thousands of resources in the form of websites dedicated to graphics, class notes, scientific publications, and software. Please bear in mind that graphics, like any discipline associated with computers and computations, develops continually, so readers should search the Internet for new, useful, and exciting sources of information. Search items may be as general as “computer graphics,” “computer animation,” “image processing,” “computer vision,” and “computer-aided design (CAD),” or as specific as “how to twirl in Photoshop,” “what is quaternion rotation,” and “Java code for dithering.” Shaw’s plays are the price we pay for Shaw’s prefaces. —James Agate.
Preface
xiii
Acknowledgements A book of this magnitude is generally written with the help, dedicated work, and encouragement of many people, and this large textbook/reference is no exception. First and foremost should be mentioned my editor, Wayne Wheeler and the copyeditor, Francesca White. They made many useful comments and suggestions, and pointed out many mistypes, errors, and stylistic blemishes. In addition, I would like to thank H. L. Scott for permission to use Figure 2.82, CH Products for permission to use Figure 26.24b, Andreas Petersik for Figure 6.61, Shinji Araya for Figure 7.27, Dick Termes for many figures and paintings, the authors of Hardy Calculus for the limerick at the end of Chapter 13, Bill Wilburn for many Mathematica notebooks, and Ari Salomon for photos and panoramas in several plates. I welcome any comments, suggestions and corrections. They should be emailed to
[email protected]. An errata list and other information will be maintained in the book’s website http://www.davidsalomon.name/CGadvertis. And now, to the matter at hand. This preface made me so impatient, being conscious of my own merits and innocence, that I was going to interrupt; when he entreated me to be silent, and thus proceeded.
— Jonathan Swift, Gulliver’s Travels
Contents Preface
vii
Introduction
1
Part I Basic Techniques 1
2
Historical Notes 1.1 Historical Survey 1.2 History of Curves and Surfaces 1.3 History of Video Games 1.4 Pioneers of Computer Graphics 1.5 Resources For Computer Graphics Raster Graphics 2.1 Pixels 2.2 Graphics Output 2.3 The Source-Target Paradigm 2.4 Interpolation 2.5 Bitmap Scaling 2.6 Bitmap Stretching 2.7 Replicating Pixels 2.8 Scaling Bitmaps with Bresenham 2.9 Pixel Art Scaling 2.10 Pixel Interpolation 2.11 Bilinear Interpolation 2.12 Interpolating Polynomials 2.13 Adaptive Scaling by 2 2.14 The Keystone Problem 2.15 Bitmap Rotation 2.16 Nonlinear Bitmap Transformations 2.17 Circle Inversion 2.18 Polygons (2D) 2.19 Clipping
7 9 9 13 14 17 19 29 30 32 40 41 45 46 46 48 54 56 57 58 65 73 76 79 85 88 91 xv
Contents
xvi 2.20 2.21 2.22 2.23 2.24 2.25 2.26 2.27 2.28 2.29 2.30 2.31 2.32 3
Cohen–Sutherland Line Clipping Nicholl–Lee–Nicholl Line Clipping Cyrus–Beck Line Clipping Sutherland–Hodgman Polygon Clipping Weiler–Atherton Polygon Clipping A Practical Drawing Program GUI by Inversion Points Halftoning Dithering Stippling Random Numbers Image Processing The Hough Transform
92 93 95 97 99 100 104 107 109 118 122 126 131 135
Scan Conversion 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14
Scan-Converting Lines Midpoint Subdivision DDA Methods Bresenham’s Line Method Double-Step DDA Best-Fit DDA Scan-Converting in Parallel Scan-Converting Circles Filling Polygons Pattern Filling Thick Curves General Considerations Antialiasing Convolution
135 136 137 142 147 151 153 156 167 175 177 179 180 192
Part II Transformations and Projections
193
4
199
Transformations 4.1 4.2 4.3 4.4 4.5
5
Introduction Two-Dimensional Transformations Three-Dimensional Coordinate Systems Three-Dimensional Transformations Transforming the Coordinate System
201 204 232 233 250 251
Parallel Projections 5.1 5.2 5.3
Orthographic Projections Axonometric Projections Oblique Projections
252 255 262
Contents 6
7
Perspective Projection 6.1 One Two Three . . . Infinity 6.2 History of Perspective 6.3 Perspective in Curved Objects, I 6.4 Perspective in Curved Objects, II 6.5 The Mathematics of Perspective 6.6 General Perspective 6.7 Transforming the Object 6.8 Viewer at an Arbitrary Location 6.9 A Coordinate-Free Approach: I 6.10 A Coordinate-Free Approach: II 6.11 The Viewing Volume 6.12 Stereoscopic Images 6.13 Creating a Stereoscopic Image 6.14 Viewing a Stereoscopic Image 6.15 Autostereoscopic Displays Nonlinear Projections 7.1 False Perspective 7.2 Fisheye Projection 7.3 Poor Man’s Fisheye 7.4 Fisheye Menus 7.5 Panoramic Projections 7.6 Cylindrical Panoramic Projection 7.7 Spherical Panoramic Projection 7.8 Cubic Panoramic Projection 7.9 Six-Point Perspective 7.10 Other Panoramic Projections 7.11 Panoramic Cameras 7.12 Telescopic Projection 7.13 Microscopic Projection 7.14 Anamorphosis 7.15 Map Projections
xvii 267 269 275 282 283 294 305 310 314 322 326 329 332 336 340 351 355 355 357 369 369 372 373 380 386 389 391 396 402 404 405 408
Part III Curves and Surfaces 8
Basic 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 8.10 8.11
Theory Points and Vectors Length of Curves Example: Area of Planar Polygons Example: Volume of Polyhedra Parametric Blending Parametric Curves Properties of Parametric Curves PC Curves Curvature and Torsion Special and Degenerate Curves Basic Concepts of Surfaces
429 433 433 441 442 443 444 445 447 453 461 469 470
Contents
xviii 8.12 8.13 8.14 8.15 8.16 9
Straight Segments Polygonal Surfaces Bilinear Surfaces Lofted Surfaces
483 487 493 499
Four Points The Lagrange Polynomial The Newton Polynomial Polynomial Surfaces The Biquadratic Surface Patch The Bicubic Surface Patch Coons Surfaces Gordon Surfaces
505 506 510 519 521 521 522 527 542
Hermite Interpolation 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 11.10
12
483
Polynomial Interpolation 10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8
11
473 475 476 478 481
Linear Interpolation 9.1 9.2 9.3 9.4
10
The Cartesian Product Connecting Surface Patches Fast Computation of a Bicubic Patch Subdividing a Surface Patch Surface Normals
Interactive Control The Hermite Curve Segment Degree-5 Hermite Interpolation Controlling the Hermite Segment Truncating and Segmenting Hermite Straight Segments A Variant Hermite Segment Ferguson Surfaces Bicubic Hermite Patch Biquadratic Hermite Patch
545 546 547 557 558 562 564 566 568 571 573
Spline Interpolation 12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8 12.9
The Cubic Spline Curve The Akima Spline The Quadratic Spline The Quintic Spline Cardinal Splines Parabolic Blending: Catmull–Rom Curves Catmull–Rom Surfaces Kochanek–Bartels Splines Fitting a PC to Experimental Points
577 578 599 602 604 606 610 615 617 624
Contents 13
14
B´ ezier Approximation 13.1 The B´ezier Curve 13.2 The Bernstein Form of the B´ezier Curve 13.3 Fast Calculation of the Curve 13.4 Properties of the Curve 13.5 Connecting B´ezier Curves 13.6 The B´ezier Curve as a Linear Interpolation 13.7 Blossoming 13.8 Subdividing the B´ezier Curve 13.9 Degree Elevation 13.10 Reparametrizing the Curve 13.11 Cubic B´ezier Segments with Tension 13.12 An Interpolating B´ezier Curve: I 13.13 An Interpolating B´ezier Curve: II 13.14 Nonparametric B´ezier Curves 13.15 Rational B´ezier Curves 13.16 Circles and B´ezier Curves 13.17 Rectangular B´ezier Surfaces 13.18 Subdividing Rectangular Patches 13.19 Degree Elevation 13.20 Nonparametric Rectangular Patches 13.21 Joining Rectangular B´ezier Patches 13.22 An Interpolating B´ezier Surface Patch 13.23 A B´ezier Sphere 13.24 Rational B´ezier Surfaces 13.25 Triangular B´ezier Surfaces 13.26 Joining Triangular B´ezier Patches 13.27 Reparametrizing the B´ezier Surface 13.28 The Gregory Patch Volume II
xix 629 630 632 639 644 647 648 653 658 660 663 672 673 676 684 684 690 693 698 699 701 702 704 707 707 709 719 723 725 729
B-Spline Approximation 14.1 14.2 14.3 14.4 14.5 14.6 14.7 14.8 14.9 14.10 14.11 14.12 14.13 14.14
The Quadratic Uniform B-Spline The Cubic Uniform B-Spline Multiple Control Points Cubic B-Splines with Tension Cubic B-Spline and B´ezier Curves Higher-Degree Uniform B-Splines Interpolating B-Splines A Knot Vector-Based Approach Recursive Definitions of the B-Spline Open Uniform B-Splines Nonuniform B-Splines Matrix Form of the Nonuniform B-Spline Subdividing the B-spline Curve Nonuniform Rational B-Splines (NURBS)
731 732 736 743 745 748 748 750 751 760 761 766 775 779 782
Contents
xx 14.15 14.16 14.17 14.18 14.19 15
788 792 796 798 801 803
Subdivision Methods 15.1 15.2 15.3 15.4 15.5 15.6 15.7 15.8 15.9 15.10
16
The Cubic B-Spline as a Circle Uniform B-Spline Surfaces Relation to Other Surfaces An Interpolating Bicubic Patch The Quadratic-Cubic B-Spline Surface Introduction Chaikin’s Refinement Method Quadratic Uniform B-Spline by Subdivision Cubic Uniform B-Spline by Subdivision Biquadratic B-Spline Surface by Subdivision Bicubic B-Spline Surface by Subdivision Polygonal Surfaces by Subdivision Doo Sabin Surfaces Catmull–Clark Surfaces Loop Surfaces
803 804 811 812 816 821 826 826 828 829
Sweep Surfaces 16.1 16.2 16.3 16.4
Sweep Surfaces Surfaces of Revolution An Alternative Approach Skinned Surfaces
833 834 839 842 846
Part IV Advanced Techniques 17
851
Rendering 17.1 17.2 17.3 17.4 17.5 17.6 17.7 17.8 17.9 17.10
18
849
Introduction A Simple Shading Model Gouraud and Phong Shading Palette Optimization Ray Tracing Photon Mapping Texturing Bump Mapping Particle Systems Mosaics
852 853 864 866 867 876 877 879 881 883
Visible Surface Determination 18.1 18.2 18.3 18.4 18.5 18.6 18.7 18.8
Ray Casting Z-Buffer Method Explicit Surfaces Depth-Sort Method Scan-Line Approach Warnock’s Algorithm Octree Methods Approaches to Curved Surfaces
891 893 893 895 898 901 905 907 910
Contents 19
911 914 915 916 919 923 924 930 937 943 944
GKS IGES PHIGS OpenGL PostScript Graphics File Formats GIF TIFF PNG CRC
947 947 949 951 954 956 960 961 963 967 972
Color 21.1 21.2 21.3 21.4 21.5 21.6 21.7 21.8 21.9 21.10 21.11 21.12 21.13
22
Background Interpolating Positions Constant Speed: I Constant Speed: II Interpolating Orientations: I SLERP Summary Interpolating Orientations: II Nonuniform Interpolation Morphing Free-Form Deformations
Graphics Standards 20.1 20.2 20.3 20.4 20.5 20.6 20.7 20.8 20.9 20.10
21
911
Computer Animation 19.1 19.2 19.3 19.4 19.5 19.6 19.7 19.8 19.9 19.10 19.11
20
xxi
975 Light Color and the Eye Color and Human Vision The HLS Color Model The HSV Color Model The RGB Color Space Additive and Subtractive Colors Complementary Colors The Color Wheel Spectral Density The CIE Standard Luminance Converting Color to Grayscale
975 976 978 982 984 984 986 991 992 994 997 1000 1001
Fractals 22.1 22.2 22.3 22.4 22.5 22.6
Introduction Fractal Landscapes Branching Rules Iterated Function Systems (IFS) Attractors Gaussian Distribution
1005 1006 1009 1013 1013 1017 1021
Contents
xxii
Part V Image Compression 23
Compression Techniques 23.1 23.2 23.3 23.4 23.5 23.6 23.7 23.8 23.9 23.10 23.11 23.12 23.13
24
25
1025
Redundancy in Data Image Types Redundancy in Images Approaches to Image Compression Intuitive Methods Variable-Length Codes Codes, Fixed- and Variable-Length Prefix Codes VLCs for Integers Start-Step-Stop Codes Start/Stop Codes Elias Codes Huffman Coding
Transforms and JPEG 24.1 Image Transforms 24.2 Orthogonal Transforms 24.3 The Discrete Cosine Transform 24.4 Test Images 24.5 JPEG
1027 1027 1031 1032 1038 1051 1052 1052 1055 1056 1058 1060 1062 1069 1079 1079 1084 1092 1128 1132
The Wavelet Transform 25.1 25.2 25.3 25.4 25.5
The Haar Transform Filter Banks The DWT SPIHT QTCQ
1147 1148 1166 1176 1188 1199
Part VI Graphics Devices 26
1201 1203
Graphics Devices 26.1 26.2 26.3 26.4 26.5 26.6 26.7 26.8 26.9 26.10 26.11 26.12 26.13 26.14
Displays The CRT LCDs The Digital Camera The Computer Mouse The Trackball The Joystick The Graphics Tablet Scanners Inkjet Printers Solid-Ink Printers Laser Printers Plotters Interactive Devices
1203 1208 1213 1217 1236 1240 1242 1244 1252 1262 1271 1274 1279 1282
Contents
xxiii
27 Appendixes
1287
A
Vector Products
1289
B
Quaternions
1295
C
Conic Sections
1299
D
Mathematica Notes
1305
E
The Resolution of Film
1311
F
The Color Plates
1315
References
1321
Answers to Exercises
1339
Index
1469
To me style is just the outside of content, and content the inside of style, like the outside and the inside of the human body both go together, they can’t be separated.
— Jean-Luc Godard
PlateA.1.WaterSplashinaGreenGarden(Modo).
Plate A.2. A Cluttered Room, Day and Night (Live Interior).
PlateA.3.AJigsawPuzzleofChateauChambord(AVbrothers).
PlateA.4.TheMandelbrotSetandTwoDetails(FractalDomains).
PlateA.5.ThreeShinyDice(Modo).
PlateA.6.POVRAYTextures(MegaPOV).
PlateA.7.RationalFractals(FractalDomains).
Varythelighting
Makethefloorshiny
Addtexture
Addalightsource
PlateA.8.StepstoVisuallyImproveaSimpleScene(MegaPOV).
Addawoodenfloor
Startwithafewrods
Introduction Computer graphics is a vast field with applications to presentations (slides and video), computer art, cartography, medicine, entertainment, training, visualization (of large quantities of data), image processing, design (computer aided or CAD), and many other areas. The adage “a picture is worth a thousand words” explains why this field is so important, and it also explains why this book is so big. A video (or even a single image) tells us so much more than text, but it also requires more resources and longer preparation. There simply is much to learn about computer graphics, the special input/output devices it requires, the specific approaches, techniques, and algorithms it employs, and the specialized language, concepts, and terms that are commonly used in this field. This introduction covers the chief terms used in computer graphics, the concept of graphics processing unit (GPU), and the fundamental equation of computer graphics.
Terms and Concepts Here are a few informal definitions of the graphics field and its “relatives” (as is common with any informal definitions, certain experts and users may disagree). Image processing (more precisely, digital image processing) is the field that deals with methods, techniques, and algorithms for image manipulation, enhancement, and interpolation. Researchers in this field test, publish, and implement their algorithms to make them available to users who often may not be technically savvy. (A technically-savvy person is one who is proficient enough to read a user manual, run software and equipment, and notice when things go wrong, but who is not an expert, does not develop algorithms, and does not write code.) Image editing (or simply imaging) is concerned with using software (implemented by image processing professionals) to manipulate images. Computer graphics is the discipline concerned with generating images. An image is generated (or synthesized) from geometrical descriptions. In addition to just generating images, this field is also concerned with rendering them accurately (so they look real) and fast (so that generating a long video of computer animation does not take forever). 1
2
Introduction
Computer vision is that branch of engineering concerned with constructing devices that comprehend and correctly interpret real objects the way our brains do. Pattern recognition is the mathematical field that deals with finding patterns in data and signals. The data can be text, audio, still images, or video. A special case of pattern recognition is data mining, a branch of computer science concerned with finding trends in large data bases. Image analysis includes anything that has to do with the extraction of meaningful information from images. Image analysis is a wide field that often overlaps other imagerelated areas. Reading a bar code, for example, can be considered image analysis, but is also pattern recognition. Identifying a face in a digital image is image analysis, but may use techniques from artificial intelligence. Detecting an edge in a digital image is image analysis, but it uses algorithms from image processing. Here is a short list of the most important concepts and terms used in computer graphics: Graphics (from the Greek γραφικ´ oς graphia or graphikos) are visual presentations on some surface, such as a canvas, screen, or paper. Graphics are used to inform, illustrate, or entertain. Examples of graphics presentations are paintings, photographs, drawings, digital images, graphs, diagrams, geometric designs, and maps. Graphics may be in black-and-white, grayscale, or color and may contain text. Computer graphics is the field concerned with the generation, manipulation, and storage of digital graphical data. This includes still images (two- and three-dimensional), animated graphics, and interactive images (virtual reality). In fact, most digital data that is not text, software, or audio, is graphics data. A pixel (from picture element) is the smallest unit of a digital image. In the computer, a pixel is represented by its color code, which is either a grayscale value or the three components of a color. We tend to think of a pixel as a small dot, circular or square, but Section 2.1 shows that in principle, a pixel is a mathematical, dimensionless point. Figure Intro.1 shows a small, 128 × 128-pixel image and some of its constituent pixels.
Figure Intro.1: An Image and Pixels.
Introduction
3
Digital image. Such an image is a rectangular array of pixels. Early images had only black-and-white or grayscale pixels, and featured low resolutions (only a few dozen pixels per centimeter or per inch). Current images are mostly color, where the colors are selected from huge palettes of millions of colors. Today’s images can have resolutions of hundreds of pixels per centimeter, more often measured as dpi (dots per inch). 3D (three-dimensional) images. Such an image contains much more information than a two-dimensional image. In order to fully view a three-dimensional image, it has to be rotated or transformed in some way. True viewing devices for such images are rare and expensive (holograms come to mind), so most three-dimensional images are projected on a two-dimensional display and have to be viewed from different directions. Computer animation. Much as a film is a sequence of still images, a digital animation is a sequence of digital images. An animation is organized in scenes, where consecutive images in a scene differ slightly in order to achieve the illusion of smooth movement. Vector graphics. Older graphics displays were of this type. In a vector graphics computer system, the program running in the computer creates a picture out of graphics components such as points, lines, and circles. A set of such components is stored in memory and becomes the description of an image. Special hardware translates each graphics component in memory into a visible part on the output (normally a display). The total of all the parts on the display constitutes the image. Raster graphics. Most current graphics displays are of this type. The screen displays an image that consists of small dots (pixels). The program running in the computer creates an image by assigning color codes to the pixels in memory. The graphics hardware scans the color codes in memory and actually paints the pixels on the display. Scan conversion. This is the process of converting a smooth geometric figure into pixels, such that the result looks as smooth as possible. Scan conversion is a very common operation, which is why scan conversion algorithms should be fast, using only logical operations, shifts, and simple arithmetic operations. Transformations. Once an image has been generated, the user often transforms it in order to view different parts of the image or to modify its shape in regular ways. Transformations are a must when a three-dimensional image is viewed on a two-dimensional display. Projections. A three-dimensional image has to be projected in order to print it or view it on a two-dimensional display. The most common projection is perspective, but parallel projections and nonlinear projections are suitable for special applications. Rendering is the process of generating a digital image from a mathematical model by means of algorithms implemented in computer programs. The mathematical model describes the objects that constitute the image in terms of points and curves. Inputs to the rendering algorithm include the orientation of the objects in space, the surface textures of the objects (including shading information), and the lighting configuration (the positions, intensities, and colors of the light sources). The term rendering has its origins in a painter’s rendering of a scene.
4
Introduction
Modeling. A mathematical model of an object is a collection of points and curves. If the object is three-dimensional, the model also contains surface information. The first step in rendering an object is to display its mathematical model as a wireframe (Section 8.11.2). Color. An understanding of color is essential for dealing with color images and animation. Chapter 21 discusses the physical meaning and psychological implications of color, as well as the various color spaces and human vision. Image compression. Images tend to be large, compared to texts, which is why image compression is such an important field of research. Images can be compressed because their original (raw) representations are inefficient and feature much redundancy. The various image compression methods remove this redundancy in different ways, and Chapters 23 through 25 describe several approaches to image compression, especially orthogonal and subband (i.e., wavelet) transforms.
The Graphics Processing Unit (GPU) Graphics is a computationally intensive application. The task of computing and displaying a color, high-resolution image on a large screen involves the following steps: (1) The surfaces that constitute the image have to be computed (as discussed in Part III of the book), either as polygonal surfaces or as smooth, curved surfaces that are then converted to triangles. (2) Each small triangle has to be rendered by simulating the light reflected from it (Chapter 17). The software has to know the positions, orientations, and colors of the light sources, and has to perform intensive computations for each triangle. (3) A texture is sometimes embedded in a surface. The texture is a small bitmap that is wrapped around the surface to enhance its look. (4) Once rendered, a triangle has to be projected to two dimensions (see Part II of the book). The projection can be parallel or perspective. (5) When the projection of a triangle has been determined (i.e., the two-dimensional coordinates of its three vertices are known), its visibility must be determined (Chapter 18). A triangle (a small part of a larger surface patch) may be completely or partially obscured by other surface parts that are closer to the camera (or the observer). Only those parts of the triangle that are visible to the camera should participate in the next step. (6) Those parts need to be scan converted (Chapter 3). A special algorithm is needed to determine the best pixels for each visible part of the triangle. It has therefore been recognized in the early days of computer graphics that there is a need for special, dedicated hardware to perform most of these operations, which is why several companies started developing such hardware as early as the 1970s. The first units were special-purpose processors for creating bitmaps of text and simple graphics shapes and sending them to the video output. The 1980s saw graphics controllers for the IBM PC and the Commodore Amiga. Those devices tried to accelerate graphics
Introduction
5
operations and implement various two-dimensional graphics primitives in hardware. In the case of the Amiga computer, the graphics unit also handled scan conversion of lines, filling of regions, and performed block image transfers. As hardware manufacturing capabilities improved during the 1990s, more graphics cards, graphics accelerators, and support devices for three-dimensional graphics operations were introduced for both personal computers and console games. In some cases, three-dimensional graphics operations became possible only if an accelerator was installed in the computer. The graphics programming language OpenGL appeared in the early 1990s and became so popular that users started demanding graphics processors that can be programmed in OpenGL. It was only in the late 1990s that the first GPUs, made by Nvidia (which also coined the term GPU) were introduced. These devices (in combination with high-resolution display monitors and drawing and illustration software) have turned our personal computers into fast, high-resolution graphics generators, and have made it possible for anyone to create high-quality original graphics such as maps, plans, drawings, animations, games, and illustrations on a personal computer. A GPU is a processor mounted on a special graphics card which also includes support devices, memory, and a bus. The graphics card can be plugged into the computer and later replaced with a more powerful version, but in most current personal computers the card is simply part of the main circuit board. The main functions of the GPU and its support devices are fast floating-point operations and vector operations. These operations are used all the time in surface computations, rendering, and visible surface determination. Today (2011), GPUs feature parallel computational units that can perform their operations simultaneously. A parallel processor is notoriously difficult to program, which is why software writing for parallel GPUs is now considered an important field which is dubbed “general-purpose computing on GPU” or GPGPU. All problems in computer graphics can be solved with a matrix inversion. —James F. Blinn, What’s the Deal with the DCT (1993). The Fundamental Equation of Computer Graphics Mathematicians like fundamental theorems. The fundamental theorem of a field of mathematics is the theorem considered central to that field. There are fundamental theorems of calculus, algebra, arithmetic, vector analysis, Galois theory, Riemannian geometry, and more. Theorems and proofs are rare in computer graphics, which is not as rigorous as mathematics. However, there is one equation that appears many times in this book and can be considered fundamental in computer graphics. This fundamental equation is the expression for the basic linear interpolation, Equation (9.1) P(t) = (1 − t)P0 + tP1 , where P0 and P1 can be numbers, points, curves, and even images (as, for example, in Sections 2.31.1 and 8.5). It is easy to see that P(0) = P0 and P(1) = P1 . In general,
6
Introduction
P(t) is a mixture (or a combination) of (1 − t)% of P0 and t% of P1 . Notice that these percentages add up to 1. When the parameter t is varied from 0 to 1, P(t) varies linearly from P0 to P1 . (See the epigram at the end of Chapter 13.) You know what? I can’t stand introductions. Weird coming from an author, right? It’s like some committee got together and said that you’ve got to have an introduction in your book. Oh, and please make it long. Really long! In fact, make it so long that it will ensure no one reads introductions.
—Matt Kloskowski, The Complete Guide to Photoshop Layers, 2011.
Part I Basic Techniques The first part of this book deals with the basic concepts and techniques employed in computer graphics. Chapter 1 is a short survey of the history of this field. It also lists many pioneers and their achievements, and follows with a detailed list of graphics resources. Chapter 2 introduces the basic concepts of vector scan and raster scan. It also discusses many topics and operations that relate to bitmaps. Included are windows, the important BitBlt operation, bitmap scaling, bitmap rotation, inversion points, and many others. The important operation of scan conversion is the topic of Chapter 3. Geometrical figures such as lines, circles, and ellipses are ideally straight or curved and are smooth, but digital images consist of pixels and are therefore rough, blocky, and pixelated. Scan conversion is the task of determining the best pixels for a given geometric figure. Several algorithms for scan converting lines and circles are described in this chapter, and it is shown that scan converting these figures is a simple problem in the sense that it can be solved by efficient DDA methods using just integers and simple arithmetic operations (additions, subtractions, and shifts). Many other scan conversion algorithms have been developed for generating polygons, parabolas, hyperbolas, and other common graphics figures. The chapter also discusses antialiasing, an operation that is related to scan conversion in that it employs grayscales to reduce the annoying effect of jagged edges.
1 Historical Notes We start our long voyage in computer graphics with a short survey of the history of this field and the names of some of its pioneers. This is followed by a detailed list of several types of available resources.
1.1 Historical Survey The term “Computer Graphics” was coined in 1960 by William Fetter, to describe what he was doing at Boeing at the time, but the history of computer graphics started in the early 1950s. (In 1950, Ben Laposky, a mathematician and artist from Iowa, created the first graphic images generated by an electronic machine. These were Lissajous figures that he dubbed oscillons and displayed on an oscilloscope, which is an analog device.) This is very early, considering that the history of the modern digital electronic computer itself began in the late 1940s. However, because of high hardware prices, the field was originally the domain of a few lucky individuals, and it was only in the 1970s that it started growing fast and eventually became the vast discipline that we know today. Here is a short chronology. A curious note. For many years, the CRT (Cathode Ray Tube) was the main graphics output device. History tells us that this device was invented in 1885 and became practical in 1897, when Ferdinand Braun in Germany developed a CRT with a fluorescent screen. The screen would emit a visible light when struck by a beam of electrons and the device became known as the cathode ray oscilloscope. By 1951 the Whirlwind computer installed at MIT had two 16-inch graphics displays (actually, modified oscilloscopes). Surprisingly, there were no immediate users. (This computer was part of the Whirlwind project, an electronic controller for a United States Navy flight simulator to train bomber crews. The project started in 1945.) Plotters (Section 26.13) came into use as graphics output devices in 1953. D. Salomon, The Computer Graphics Manual, Texts in Computer Science, DOI 10.1007/978-0-85729-886-7_1, © Springer-Verlag London Limited 2011
9
10
1.1 Historical Survey
In 1955, the SAGE (Semi-Automatic Ground Environment) air defense system started its operations. It used vector-scanned monitors as its main output and light pens as its input devices. Digital Equipment Corporation (DEC) was founded in 1957. It started making minicomputers that were later used in the early development of computer graphics. Light pens (Section 26.2.3) came into wide use in 1958, the same year as the first microfilm recorder. In 1959, a partnership of General Motors and IBM produced the first piece of drawing software, the DAC-1 (Design Augmented by Computers). Users could input the three-dimensional description of a car, view the car in perspective, and rotate it. It was in the 1960s that the field got its first big push. In 1961, Ivan Sutherland developed Sketchpad, a drawing program, as his Ph.D. thesis at MIT. Sketchpad used a light pen as its main input device and an oscilloscope (modified to do vector scan) as its output device. The first version handled two-dimensional figures only, and was later extended to draw, transform, and project three-dimensional objects, and also to perform engineering calculations such as stress analysis. One important feature of Sketchpad was its ability to recognize constraints. The user could draw, for example, a rough square, then instruct the software to convert it to an exact square. Another feature was the ability to deal with objects, not just individual curve segments. The user could build an object out of segments, then ask the software to scale it. Because of his pioneering work, Sutherland is often acknowledged as the grandfather of interactive computer graphics and graphical user interfaces. There are many Internet resources with information about and images of Sketchpad, see, for example, [sketchpad.wiki 10]. I just need to figure out how things work. —Ivan Sutherland. At about the same time, Steven Russell, another MIT student, developed the first video game, Spacewar! This program was written for the PDP-1 and was later used by DEC salespersons to demonstrate that minicomputer. In 1963, the first computer-generated film, titled Simulation of a two-giro gravity attitude control system, was created by Edward E. Zajac at Bell laboratories. Others at Bell, Boeing, and Lawrence Radiation Laboratory followed soon with more films. The first digitizer (Section 26.8), the RAND tablet, appeared in 1964. Also in 1964, the first commercially available graphics computer, the IBM 2250 Graphics Display Unit, was announced as part of the historically-important System/360. Like many old displays, the 2250 employed vector graphics on a 1024×1024 CRT that was refreshed up to 40 times per second. Characters were constructed of short line segments and any characters and symbols could be displayed. Like any vector-scan graphics device, the refresh time became longer as more and more symbols were displayed, and the display eventually started to flicker. The 2250 used a light pen as an interactive input device. In the mid-1960s, interest in computer graphics was picking up. More and more companies—such as TRW, Lockheed-Georgia, General Electric, and Sperry Rand— became active in the graphics field. At about the same time, David Evans and Ivan Sutherland founded their company which made, among other things, vector-scan dis-
1 Historical Notes
11
plays. Those displays are historically important since they gave a tremendous boost to computer graphics throughout the 1960s. In 1966, Sutherland developed the first three-dimensional head-mounted display (HMD, section 26.14.2). It displayed a stereoscopic pair of wire-frame images. This device was rediscovered in the 1980s and is commonly used today in virtual-reality applications. In the late 1960s, both Sutherland and Evans were invited to develop a program in computer science at the University of Utah in Salt Lake City. Computer graphics quickly became the specialty of their department, and for years maintained its position as the primary world center for this field. Many important methods and techniques were developed at the UU computer graphics lab, among them illumination models, hiddensurface algorithms, and basic rendering techniques for polygonal surfaces. Names of UU students such as Phong Bui-Tuong, Henri Gouraud, James Blinn, and Ed Catmull are associated with many basic algorithms still in use today. Several accounts of computer graphics persons and projects at UU can be found on the Internet at URL http://www.cs.utah.edu/school/history/. Computer graphics in the 1960s was out of the reach of most computer users because the special graphics hardware was expensive. There were no personal computers or workstations. Users had to pay for mainframe time by the second or buy expensive minicomputers. Display monitors used vector scan and were black and white. The result was that few computer professionals could develop computer graphics techniques and algorithms and the software was noninteractive and non-portable. The advent of the microprocessor, in the mid-1970s, was another factor in the rapid progress of computer graphics. Personal computers appeared on the market and suddenly anyone could afford to own a computer. This encouraged the formation of small companies that developed computer animation, mostly to be used in television commercials. Names such as Abel and Associates, Information International Inc., Digital Effects, and Systems Simulation Ltd. became well known and produced short pieces that demonstrated dazzling effects. SIGGRAPH, the Special Interest Group on Computer Graphics (part of the ACM), was formed in 1969 and has grown in size and importance as the field of computer graphics expanded. The first of the many famous SIGGRAPH conferences was held in 1973. It attracted 1200 attendees and later conferences boasted as many as 30,000 participants and hundreds of exhibitors. The famous Utah teapot (see Page 704 and Plate Z.6) was constructed in 1975. This is perhaps the best known three-dimensional model in computer graphics. The original teapot this model is based on is displayed at the Computer Museum in Boston. During the 1970s, activity in basic computer graphics research started moving from UU first to NYIT, the New York Institute of Technology, then to Lucasfilm. Computer animation and computer painting were two topics seriously developed at those places. The technique of (and hardware for) raster scan was developed in the 1970s by Richard Shoupe at Xerox Palo Alto Research Center (PARC). Workers in the field soon realized the advantages of raster scan and the word “pixel” entered the field of computer graphics. Like any other mature discipline, computer graphics eventually got its first periodic publication. Computer Graphics World started carrying news and reviews in late 1977.
12
1.1 Historical Survey
Fractals, developed by Benoˆıt Mandelbrot in the 1960s and 1970s, were applied to computer graphics in the late 1970s by Loren Carpenter and others. It was in the 1980s that personal computers, most notably, the Macintosh and Amiga, employed graphical user interfaces (GUI) to interact with the user and to graphically display results with symbols, icons, and pictures, rather than text. The term “multimedia,” which originated around 1985, refers to the use of text, images, animation, and audio in computers in an integrated way, which is why computer graphics is one of the main components of multimedia. Ray tracing, a sophisticated rendering method (Section 17.5), was developed by Turner Whitted of Bell labs and published in 1980. Silicon Graphics Inc. (SGI) was founded in 1982 and has been building highperformance graphics computers since. The technique of particle systems (Section 17.9) was developed in the early eighties at Lucasfilm. Morphing (Section 19.10) was developed at the same time at NYIT. The data glove (Section 26.14.1), currently very popular for virtual-reality applications, was developed at Atari in 1983. Radiosity came out of Cornell University in 1984. This is a sophisticated rendering method that simulates light reflection between surfaces by determining the exchange of energy between them. GUI, graphical user interfaces, appeared in 1984 with the release of the first Macintosh computer. This personal computer immediately became, and still remains, a highly popular tool for computer graphics and is currently used by graphics amateurs and professionals, as well as by graphic-oriented businesses. The Amiga computer, made by Commodore, also had much success in the graphics field. In 1985 came the first ISO standard, the High Sierra, for CD-ROMs. The Commodore Amiga personal computer was also introduced in the same year. It immediately became popular for what today is called multimedia applications. PostScript, the all-important page-description language (Section 20.5), came out of Adobe Inc. at about the same time. The 1980s saw the emergence of raster-scan display monitors as the main graphics output device. This technology has benefited from experience gained with television and has resulted in the affordable, reliable color monitors of today. The late 1980s and early 1990s also saw the developments of graphics standards such as GKS and PHIGS. In the late 1980s, graphics computers made by SGI (Silicon Graphics Inc.) were used to create some of the first fully computer-generated short films at Pixar. The Microsoft Windows 3.0 operating system was first shipped in 1990 and, of course, gave a tremendous boost to the concept of GUI. More and more applications were developed to run under MS Windows. OpenGL (Open Graphics Library) (Section 20.4), was originated by SGI in 1992. The 1990s also saw rapid developments in three-dimensional graphics, especially in gaming, multimedia, and animation. Quake, one of the first fully 3D games, was released in 1996. Released in 1997, Toy Story is the first full-length (79 minutes, which translates to more than 114,000 animation frames at 24 frames per second) feature film that’s completely computer-animated. It represents a milestone in computer graphics and
1 Historical Notes
13
it marks the beginning of an era when computer graphics rendering techniques have become so sophisticated that viewers may find it impossible to tell if an image is real or if it is a clever rendering of a mathematical model. The explosion of CPU speeds and memory capacities since the late 1990s have resulted in more detailed and realistic digital images and animation, partly also due to powerful 3D-modeling software. Reference [graphics.timeline 09] is a detailed timeline of important developments in computer graphics. Check also [hocg 06] and [masson 11]. Reference [morrison 10] is Michael Morrison’s history of computer animation. A short visual history of this area is [animation-tube 10]. Search YouTube.com for others.
1.2 History of Curves and Surfaces Section 9.4 discusses lofted surfaces but does not explain the reason for this unusual name. Historically, shipbuilders were among the first to mechanize their operation by developing mathematical models of surfaces. Ships tend to be big and the only dry place in a shipyard large enough to store full-size drawings of ship parts was the sail lofts above the shipyard’s dry dock. Certain parts of a ship’s surface are flat in one direction and curved in the other direction, so such surfaces became known as lofted. In the 1960s, both car and aircraft manufacturers became interested in applying computers to automate the design of their vehicles. Traditionally, artists and designers had to make clay models of each part of the surface of a car or airplane and these models were later used by the production people to produce stamp molds. With the help of the computer it became possible to design a section of surface on the computer in an interactive process, and then have the computer drive a milling machine to actually make the part. The box on Page 629 mentions the work of Pierre B´ezier at Renault and Paul de Casteljau at Citro¨en, the contributions of Steven Coons to Ford Motors and William Gordon and Richard F. Riesenfeld to General Motors, and the efforts of James Ferguson in constructing airplane surfaces. As a result of these developments in the 1960s and 1970s, the area of computer graphics that deals with curves and surfaces has become known, in 1974, as computeraided geometric design (CAGD). Several sophisticated CAGD software systems have been developed in the 1980s for general use in manufacturing and in other fields such as chemistry (to model molecules), geoscience (for specialized maps), and architecture (for three-dimensional models of buildings). Hardware developments in the 1980s made it possible to use CAGD techniques in the 1990s to produce computer-generated special effects for movies (an example is Jurassic Park), followed by full-length movies, such as Toy story, Finding Nemo, and Shrek, that were entirely generated by computer. A detailed survey of the history of this field can be found in [Farin 04]. Several first-person historical accounts by pioneers in this field are collected in [Rogers 01].
14
1.3 History of Video Games
1.3 History of Video Games When personal computers started appearing, in the mid 1970s, many, among them computer professionals, questioned their usefulness. A common question was: Why would anyone want to have a computer at home? Typical answers were: To balance your checking account and To store your recipe collection. Few realized the correct answer which is: To entertain and communicate. Today, those who have computers (and who hasn’t) use them to communicate (by email, Internet telephone, and video chat), to be entertained (by watching movies, listening to music, and playing games), or to do both. This is why video games are so important. Because games are based on graphics, the following short history is included here. 1958. The experimental game Tennis for Two is implemented by William Higinbotham at Brookhaven Laboratory. Even though it was interactive, today this is considered more an experiment than a game. 1966. A short paper by Ralph Baer describes ways to use a television receiver as an output device for interactive games. 1968. Spacewar! is finally finished and is demonstrated at MIT. This early game has inspired the 1971 Computer Space by Nolan Bushnell. 1971. Computer Space, by Nolan Bushnell, is introduced and becomes the first coin-operated video game. 1972. Ralph Baer releases Magnavox Odyssey, the first home video game. The very successful Pong game, also by Nolan Bushnell, is introduced. 1973. Several companies, among them Chicago Coin, Midway, Ramtek, Taito, Allied Leisure, and Kee Games, enter the video game market and give this young field a big boost. 1974. Tank, by Kee Games, becomes the first game to employ ROM (Read-Only Memory) for storing game data. TV Basketball, by Midway, features human figures as players. 1975. Gun Fight, by Midway, is introduced and becomes the first game to be based on a microprocessor. Atari releases Steeplechase, the first six-player arcade video game. Kee Games announces Indy 800, the first eight-player game. Lots of innovations in one year. 1976. The first video game chip, the AY-3-8500, is built by General Instruments. The first cartridge-based home game system, the Fairchild/Zircon Channel F is introduced. Night Driver, by Atari, becomes the first game to simulate a first-person perspective. Atari also releases Breakout. 1977. After two active, successful years, the game market becomes saturated and very competitive. Several companies give up and quit the video game field as a result. The winners continue and Atari introduces its VCS home console system (later renamed the 2600). Super Bug, by Kee Games, becomes the first game to offer four-directional scrolling, and Nintendo, a Japanese newcomer, releases its first home video game, Color TV Game 6. 1978. The very successful and much familiar Space Invaders game, by Taito, appears this year. Many are familiar with how this game became the inspiration for
1 Historical Notes
15
the many vertical shooting games that were introduced later. The technique of twodirectional scrolling is introduced by Football, from Atari. 1979. Warrior, the first one-on-one fighting game, is made public by Vectorbeam. Two vector graphics games, Asteroids and Lunar Lander are released by Atari. Namco introduces Galaxian, the first game with 100% RGB colors, and Puck-Man (later renamed Pac-Man), another success story and an inspiration to many. 1980. Pac-Man, Battlezone (the first arcade game to feature a true 3D environment), and Defender are released. All are influential and a source of ideas to competitors. Ultima becomes the first home computer game with four-directional scrolling. Star Fire becomes both the first cockpit game and the first video game to feature a high-score table using players’ initials. 1981. Donkey Kong, from Nintendo, and Tempest, by Atari, are released. The video game industry is growing fast and experts are predicting a backlash. 1982. Q*bert, by Gottlieb, appears. Zaxxon, by Sega, is introduced and is advertised on television (probably the first game with this distinction). The predicted economic crash starts this year (could it be a self-fulfilling prophecy?). 1983. After several years of continued growth, the video game industry suffers from another recession. Nevertheless, new games appear. I, Robot, by Atari, is the first raster video game with filled-polygon three-dimensional graphics. Atari also comes up with Star Wars, and the Famicon system, from Nintendo, is released in Japan. 1984. Even though the general economic recession that started in 1981 is now over, the video game industry is still suffering economically. Halcyon, by RDI, becomes the first laserdisc-based home video game system. 1985. A new, improved version of Famicon, renamed the Nintendo Entertainment System (NES), is released in America. It becomes so popular that it single-handedly reverses the crash and revives the video game industry. Super Mario Bros. is introduced by Nintendo and immediately becomes a best seller (it seems to be the best selling game ever). In the Soviet Union, Alex Pajitnov designs Tetris, another influential game. 1986. Nintendo’s Famicon game system benefits from the first of the many Zelda games. Taito’s Arkanoid and Bubble Bobble games appear in video game arcades. Sega releases its successful Sega Master System (SMS). 1987. The Manhole, by Cyan, becomes the first computer game released on CDROM. Maniac Mansion, by LucasArt, is the first adventure game with a point-and-click interface. Driller, from Incentive Software, is a personal computer game with stunning 3D graphics. Double Dragon is released by Taito. 1988. The notable game production for this year includes Assault by Namco, NARC, by Williams (the first game to run on a 32-bit processor), and Super Mario Bros. 2, by Nintendo. 1989. The list for this year includes Atari’s Hard Drivin’ and S.T.U.N. Runner, Gottlieb’s Exterminator (the first game to have all digitized imagery for its backgrounds), Nintendo’s Game Boy and Atari’s Lynx (two handheld video game consoles), and Sega Genesis (a home console, not a game). 1990. This is the year SimCity is released, by Maxis. Designed by Will Wright, SimCity is the first in a long line of Sim games. Among other new games, Nintendo releases Super Mario Bros. 3, Sega’s Game Gear is released in Japan, and Squaresoft’s Final Fantasy series is sold in North America.
16
1.3 History of Video Games
1991. New releases continue with the Super Nintendo Entertainment System (SNES), Street Fighter, II by Capcom, and Sonic the Hedgehog, by Sega. 1992. Mortal Kombat is developed by Midway (psychologists recommend replacing “C” by “K” as a means of attracting customers’ attention). The best seller The 7th Guest is released by Virgin Games. Virtua Racing, a 3D racing game, is a new game by Sega. Another familiar title is Wolfenstein 3D, by id Software. Dactyl Nightmare, by Virtuality, employs a virtual reality headset and gun interface. 1993. Another best seller, Myst, from Cyan, appears this year, together with Doom, by id Software, and Virtua Fighter, by Sega. This is also the year when the world-wide-web expands rapidly. 1994. New, important games include Donkey Kong Country (Nintendo), Saturn (Sega), Warcraft (Blizzard), Daytona USA, (Sega), and Warcraft II (Blizzard). Two new game systems, the Sega Saturn and the Sony PlayStation are released in Japan and SNK’s NeoGeo home console system is introduced. 1995. The Sega Saturn and the Sony PlayStation are now available in America. Donkey Kong Country 2 by Nintendo and Kong Quest, by Diddy, are among new game releases this year. 1996. Virtual Boy, a portable, stereo game system with a separate screen for each eye, is released by Nintendo. At last, it is possible to obtain a degree in video game development (from Digipen Institute of Technology). 1997. The first GameWorks arcade opens in Seattle by DreamWorks, Sega, and Universal. The first of the MMORPG Ultima genre games appears. Designed by Richard Garriott these are multiplayer games where the action occurs in an RPG (Role-Playing Game) world. This genre has influenced many of the most successful and popular titles that followed. Other notables are Riven, the sequel to Myst, by Sega, Top Skater, with a skateboard interface, also by Sega, and Mario Kart 64, by Nintendo. 1998. Dance Dance Revolution and the first games in the Beatmania series and GuitarFreaks series, all by Konmai. Boy Color (Nintendo), Half-Life (Sierra Studios), and Grand Theft Auto (Rockstar Games) are among the notables this year. SNK releases the NeoGeo Pocket handheld video game system. 1999. The list includes Dreamcast (Sega), Donkey Kong 64 (Nintendo), and Pro Skater (Tony Hawk). The first Independent Games Festival is held at the Game Developers Conference. 2000. An important announcement: Nintendo has just sold its 100 millionth Game Boy console. Sony introduces its PlayStation 2. Another milestone: The United States Post Office issues a stamp depicting video games. Have video games finally arrived? 2001. This is the year of the Xbox, by Microsoft, but the Nintendo GameCube is also introduced in 2001. Two surprises: Sega is leaving the home video game consoles market and Midway Games quits the arcade video game industry. 2002. The Sim series of games becomes a best seller and the MMORPG Sims Online game starts. Microsoft announces its Xbox Live online gaming service. Sega releases Rez for the PlayStation 2. 2003. This year marks the start of the MMORPG Star Wars Galaxies. Enter the Matrix, by atari, is released. Nokia releases the N-Gage handheld video game system. 2004. PlayStation Portable and PlayStation 2 (Sony) are released in Asia. The Nintendo DS (dual screen) handheld video game system is also introduced.
1 Historical Notes
17
2005. The Xbox 360 (Microsoft) and the Gizmondo (Tiger Telematics) are released. The Sims series appears on postage stamps in France. We thus conclude that: Yes, video games have arrived. 2006. Wii (Nintendo), PlayStation 3 (Sony), and Xbox 360 (Microsoft) are released. 2007. World of Warcraft, an MMORPG game, is estimated to have more than nine million players worldwide. 2008. Apple entered the field of mobile gaming hardware with the release of the iPhone and iPod Touch in the summer. Software for these platforms is sold only online. Nintendo announces its Wii MotionPlus module. 2009. Sony releases its PSP Go. This device is a newer, slimmer version of the PSP. Microsoft and Sony present their new motion controllers: Project Natal (later renamed Kinect) and PlayStation Move, respectively. A few cloud computing services are announced, targeted at video games. 2010. The Nintendo 3DS, the successor to the Nintendo DS, has been released in early 2011. The new Xbox 360 console (referred to as the Xbox 360 S or Slim) is revealed by Microsoft. See also [Wolf 08]. Life is a video game. No matter how good you get, you are always zapped in the end. —Anonymous.
1.4 Pioneers of Computer Graphics The following is an alphabetical list of many pioneers, researchers, and notable figures of this important field. (A writer’s apology. Any such list is necessarily incomplete. I apologize in advance for any omissions. They are unintentional, and when brought to my attention would be included in the errata list of the book.) Bill Atkinson, developed graphics algorithms, implemented the revolutionary MacPaint software for the Macintosh, as well as Hypercard, QuickDraw, and the Macintosh menu bar. Michael Barnsley, worked on fractals. Brian A. Barsky, developed beta splines (with tension). Richard H. Bartels, codeveloper of the Kochanek-Bartels splines. ´ Pierre Etienne B´ezier, created efficient algorithms for curves and surfaces. Jim F. Blinn, artist-mathematician-programmer, the originator of many graphics algorithms, implementations, and video productions. Jack Elton Bresenham, came up with extremely efficient methods for scan converting straight lines and circles. Tom Brigham worked on morphing in 1982.
18
1.4 Pioneers of Computer Graphics Nolan Kay Bushnell, created early video games.
Loren Carpenter, programmer and researcher, implemented fractal methods to draw realistic terrain and mountains. Paul de Faget de Casteljau, developed B´ezier curves and surfaces using an approach radically different from that of B´ezier. Ed Catmull, founded Pixar, a maker of digital films and videos. George Merrill Chaikin, developed subdivision methods for surfaces. Jim Clark, entrepreneur and scientist. Founder, in 1982, of Silicon Graphics. Steven Anson Coons, an early researcher in the field of surface design and implementation. Charles Csuri, an early pioneer in computer animation and digital fine art. Carl Wilhelm Reinhold de Boor, a pioneer in the application of splines to curves. Tom A. DeFanti is a distinguished computer graphics researcher and pioneer. Tony DeRose is a distinguished computer graphics researcher. Donald Doo, developed Doo–Sabin subdivision surfaces. Gerald Farin is a distinguished computer graphics researcher. Charles Geschke developed PostScript and cofounded Adobe. Henri Gouraud, originated the shading algorithm named after him. Donald Peter Greenberg, a leading educator and innovator in computer graphics and the chief developer of radiosity. Charles Hermite, a 19th century mathematician who originated the interpolation method named after him. Alan Kay Originated the notion of a graphical user interface (GUI). He said “the best way to predict the future is to invent it.” Doris H. U. Kochanek, codeveloper of the Kochanek-Bartels splines. Kenneth C. Knowlton, is a computer graphics pioneer, artist, mosaicist, and portraitist. Joseph-Louis Lagrange is the father of the Lagrange polynomial. Charles T. Loop developed Loop subdivision surfaces. Benoˆıt B. Mandelbrot, a major figure in the field of fractals. Martin Newell, developed the Newell algorithm for hidden surface removal and created the Utah teapot which was made famous by Frank Crow. A. Michael Noll is a pioneer of computer art. Phong Bui Tuong, originated the shading algorithm named after him.
1 Historical Notes
19
Richard F. Riesenfeld, developer of B-splines. Ton Roosendaal is the original creator of Blender. Steve Russell is a programmer and scientist known for creating Spacewar!, one of the earliest videogames, in 1961. Malcolm A. Sabin, codeveloper of the Doo–Sabin subdivision surfaces. Daniel J. Sandin is a video and computer graphics artist and researcher. He was part of the team that developed the first data glove. Alvy Ray Smith, entrepreneur and researcher. Founder, or active in, several computer graphics enterprises. Ivan Edward Sutherland, designed and implemented Sketchpad, an early 2D and 3D graphics system. Cofounded Evans and Sutherland. Other achievements too numerous to list here. Dick A. Termes, an internationally acclaimed artist painting in n-point perspective on spheres. John Warnock developed PostScript and cofounded Adobe. Turner Whitted, a pioneer of ray tracing. Edward E. Zajac created the first computer-generated film.
1.5 Resources For Computer Graphics The following types of resources are listed here: (1) organizations and societies, (2) research institutions, (3) universities, (4) journals, (5) books, (6) other graphics-related websites, and (7) software. There are hundreds of academic websites and the ones listed here were selected (somewhat arbitrarily) from hundreds of easily-located similar URLs. Similarly, there are hundreds of textbooks on computer graphics and the few listed here have been selected because they offer complete coverage and are also familiar to me. The reader should bear in mind that the field and its resources develop and change constantly, so readers should search the Internet for new, useful, and exciting sources of information. Search items may be selected from “history of computer graphics,” “computer graphics,” “computer animation,” “image processing,” “computer vision,” “computer-aided design (CAD),” and many other phrases. A few resources are also listed on Pages 196 and 431. 1. Societies. SIGGRAPH, the ACM special interest group on graphics, is perhaps the single most important source of information about computer graphics. It is located at http://www.siggraph.org/ and offers a wealth on information on art and design, computer graphics events, a computer graphics bibliography database, jobs and careers in the field, industry directory, and many related links worldwide. SIGGRAPH is also
20
1.5 Resources For Computer Graphics
known for its publications (Computer Graphics Quarterly, SIGGRAPH Video Review, and ACM Transactions on Graphics) and annual conferences. Eurographics is the European Association for Computer Graphics. Find it at http://www.eg.org/. CGsociety, at http://forums.cgsociety.org/, is the society of digital artists. Its mission is to cater for 3D Animation and Visual Effects developers. http://www.vrs.org.uk/ is The Virtual Reality Society (VRS). This is an international society dedicated to the discussion and advancement of virtual reality and synthetic environments. 2. Research Organizations. http://www.wavelet.org/. This site offers several services intended to foster the exchange of knowledge and viewpoints related to theory and applications of wavelets. It is relevant to computer graphics because of the applications of wavelets to image compression. http://www.igd.fhg.de/ is the address of the Fraunhofer Institute for Computer Graphics Research (IGD) which concentrates on the development of product prototypes (hardware and software) and the realization of concepts, models, and solutions for computer graphics. Look at http://www.ccg.pt/ for information on the Computer Graphics Center, devoted to research and development in virtual reality, multimedia systems, electronic commerce, and other, graphics-related, topics. 3. Academic Websites. Free lecture notes, image gallery, and students’ assignments, projects, and examinations are available at MIT http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/. http://www.cs.brown.edu/exploratories/freeSoftware/home.html is located at Brown University. It offers a collection of Java applets for learning about computer graphics, including color theory, imaging (convolution and filters), viewing techniques, coordinate systems, splines), and texture mapping. Site http://www.eecs.berkeley.edu/Research/Projects/Areas/GR.html lists graphics projects at the University of California, Berkeley. Graphics-related research at the California Institute of Technology (Caltech) can be found at http://www.gg.caltech.edu/. It focuses primarily on the mathematical foundations of computer graphics. Cornell university, at http://www.graphics.cornell.edu/, has an advanced program in computer graphics. http://www.cg.inf.ethz.ch/ will tell you all about the Computer Graphics Laboratory at the Eidgen¨ ossische Technishce Hochschule (ETH), in Z¨ urich.
1 Historical Notes
21
The Georgia Tech (GaTech), at http://gvu.cc.gatech.edu/, has a strong graphics program. Check out http://groups.csail.mit.edu/graphics/ for the graphics program at MIT. Computer Graphics, Visualization and Animation at the University of Utah is located at http://www.cs.utah.edu/research/areas/graphics/. http://mambo.ucsc.edu/psl/cg.html is a jumping point to many sites that deal with computer graphics. A similar site is http://www.cs.rit.edu/~ncs/graphics.html that also has many links to computer graphics sites. A very extensive site of computer-graphics-related pointers is http://ls7-www.informatik.uni-dortmund.de/html/englisch/servers.html. See http://graphics.idav.ucdavis.edu/education/GraphicsNotes/ for computer graphics course notes at the Computer Science Department, University of California, Davis. Site http://ls7-www.cs.uni-dortmund.de/cgotn/ lists many links to computer graphics resources. 4. Journals, Magazines, and Conference Proceedings. See http://www.siggraph.org/publications/newsletter for The ACM SIGGRAPH Computer Graphics Quarterly. This is the official publication of the ACM SIGGRAPH organization. It is published in February, May, August, and November of each year. See http://www.siggraph.org/publications/video-review for the SIGGRAPH Video Review, an important video-based publication. It illustrates the latest concepts in computer graphics and interactive techniques. ACM Transactions on Graphics is the premiere peer-reviewed journal of graphics research. See more information at http://www.siggraph.org/publications/tog. The proceedings (including visual material, such as the Computer Animation Festival) of the annual ACM SIGGRAPH Conferences are published extensively. See http://www.siggraph.org/publications/acm-siggraph-conference-documentation. A satire on the 1992 conference, by Steve Connelly and Tim Hall, can be found at http://steve.hollasch.net/cgindex/misc/sgsatire.html. Computer Graphics World is the premiere authority on innovative graphics, technology, and applications. It is located at http://www.cgw.com/. http://crossings.tcd.ie/ is a multidisciplinary online journal that explores the areas where technology and art intersect. http://computer.org/cga is the home of the IEEE Computer Graphics and Applications, a bimonthly magazine that covers a variety of topics catering to both computer
1.5 Resources For Computer Graphics
22
graphics practitioners and researchers. This popular publication bridges the theory and practice of computer graphics, from specific algorithms to full system implementations. See http://jgt.akpeters.com for the journal of graphics, gpu, and game tools, a quarterly journal whose primary mission is to provide the computer-graphics research, development, and production communities with practical ideas and techniques that solve real problems. Ray Tracing News Guide is a website maintained by Eric Haines who also compiles its content. It is located at http://tog.acm.org/resources/RTNews/html/. This is not a formal journal. It periodically features news of ray tracing. Animation Magazine is a monthly publication covering the entire animation field, computer and otherwise. Check it at http://www.bcdonline.com/animag/. Digital Imaging is a bimonthly reporting on the digital imaging industry. 5. Books. James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes, Computer Graphics: Principles and Practice in C, Addison-Wesley Professional, 2nd ed., 1995. This textbook is a classic. It covers all the important topics and areas of the field. It is not easy reading, though, and some may claim that it is showing its age. Peter Shirley and Steve Marschner, Fundamentals of Computer Graphics, A K Peters; 3rd revised ed., 2009). Covers the basic topics (except computer animation). Francis S. Hill Jr. and Stephen M. Kelley, Computer Graphics Using OpenGL, Prentice Hall, 3rd. ed., 2006. Very clear and easy to read. The treatment of ray tracing is especially comprehensive. There are many practice exercises. Donald D. Hearn and M. Pauline Baker, Computer Graphics with OpenGL, Prentice Hall, 3rd ed., 2003. In addition to its full coverage of the CG field, this book offers a nice OpenGL user’s manual. Graphics Gems is a series of books, started in 1990 by Andrew S. Glassner and written by him and others. See http://www.graphicsgems.org/. There are hundreds of other books on this subject, mostly concentrating on specific topics, such as ray tracing, curves and surfaces, programming in OpenGL, and computer animation. The colonists did not have a library at their disposal, but the engineer was a book that was always ready, always open to the page needed, a book which answered all their questions and which they often leafed through. —Jules Verne, The Mysterious Island, 1874. 6. Other Websites. The following UseNet groups are dedicated to various aspects of computer graphics: comp.graphics.animation, comp.graphics.digest, comp.graphics.opengl, comp.graphics.raytracing, and comp.graphics.visualization.
1 Historical Notes
23
3D ARK (http://www.3dark.com/) is a free online archive of 3D related content and resources for 3D enthusiasts. http://www.colormatters.com/ is a resource for all things color. http://www.computergraphica.com/ is all about human-pixel interaction. The Graphics File Formats page is at http://www.martinreddy.net/gfx/. http://www.opengl.org/ is the chief site for OpenGL, the open-ended graphics language. Site http://steve.hollasch.net/cgindex/index.html is Steve’s Computer Graphics Index. This is a collection of topics related to computer graphics. It is maintained by Steve Hollasch. Nan’s Computer Graphics Page (http://www.cs.rit.edu/~ncs/graphics.html) is a list of links to places that offer help and information related to computer graphics. The site is owned by Nan Schaller. http://www.refdesk.com/compgrah.html is an encyclopedia of facts and topics of computer graphics. The Persistence of Vision Raytracer is a high-quality, totally free tool for creating stunning three-dimensional graphics. It is available in official versions for several popular platforms. See http://www.povray.org/. GRAFICA Obscura, at http://www.graficaobscura.com/?/, is an evolving computer graphics notebook. This is a compilation of technical notes, pictures, and essays that Paul Haeberli has accumulated over the years. It seems that this site is no longer being maintained. Website http://i33www.ira.uka.de/applets/mocca/html/noplugin/ has applets for computer-aided geometric design (CAGD). http://www.faqs.org/faqs/graphics/faq by John Grieggs contains answers to frequently asked questions on graphics. 7. Graphics Software. Most graphics applications can be classified as two-dimensional or three-dimensional (abbreviated here as 2D and 3D, respectively). In the former class we find painting, drawing, illustration, drafting, and CAD programs. The latter class consists of modelers and renderers. A painting program includes tools such as brush, spray paint, pencil, and eraser. A picture can be painted by moving these tools with the mouse, pad, or other pointing device. Editing a painting is very difficult, because the individual graphics elements are not saved by the program. Once a stroke has been painted, it may be impossible to erase. The (now obsolete) MacPaint program of 1984–1988 was perhaps the first successful painting program. A drawing program offers tools such as a rectangle, ellipse, line, arc, and curve. Each element drawn is saved by the program as a set of data (or control) points. Thus, it is possible to select any element, edit it, move it, or delete it. An illustration program
24
1.5 Resources For Computer Graphics
includes colors, patterns, and textures. A drafting program allows the accurate drawing of graphics elements with precise dimensions. A CAD program may output the drawing in a special format to another program that drives a cutting tool to actually manufacture an item. A 3D modeler allows the user to construct any three-dimensional object in the computer and save it in a special 3D format, whereas a renderer may render such an object accurately, often by tracing every ray of light and computing its reflections from several surfaces. For many years, graphics software remained primitive because computers that were fast enough to process images and compute accurate renderings tended to be expensive. It was only in the late 1980s that several 2D programs were introduced and were slowly developed over the years. The rapid development of fast personal computers and large color display monitors in the 1990s paved the way for sophisticated 2D and 3D graphics software, and today, in 2011, we witness an explosion of such programs. The following survey is necessarily incomplete, but it lists the most important graphics applications available today. I apologize in advance for any omissions; they are unintentional. Mathematica is general-purpose mathematical software that can perform numeric calculations and symbolic manipulations and display its results graphically. Both 2D and 3D graphics are supported, with many options allowing the user to precisely specify what will be displayed and how. This software was conceived by Stephen Wolfram and was first introduced in 1988. It is currently in version 8. Even though Mathematica is not a graphics application, it is listed here because of its powerful graphics capabilities. See http://en.wikipedia.org/wiki/Mathematica. Matlab, by The Mathworks (http://www.mathworks.com/) is a software tool designed mostly for numerical computations. Its capabilities can be extended by individual packages that allow detailed computations and study of topics such as wavelets, symbolic manipulations, and neural networks. Matlab can plot the curves and surfaces it computes, which makes it an important graphics program. The name Matlab is short for Matrix Lab, because all variables in Matlab, even scalars, are matrices (there is also an upazila, a subdistrict, called Matlab in Bangladesh). 2D Applications Adobe Photoshop is a graphics editing program developed by Adobe Systems. The software can edit and process bitmap images (i.e., images where only the pixels are given, without any geometric information). The main features of the program include layers with masks, color spaces, ICC profiles, transparency, text, alpha channels and spot colors, clipping paths, and duotone settings. See http://www.adobe.com. Adobe Illustrator is a vector-based image editor, now at version 15, designed for illustrations. First released in 1988, the software immediately became popular due to its chief innovation, a simple, natural way to draw curves, based on cubic B´ezier curves (Section 13.2). Over the years, many features have been added, including some 3D capabilities to extrude and revolve shapes. Among the most recent sophisticated features and tools of Illustrator are Live Trace, Live Paint, a control palette and custom workspaces, aligning individual points, multiple
1 Historical Notes
25
crop areas, the Color Guide panel, the Live Color feature, the ability to create multiple artboards, a blob brush, a gradient tool, a perspective grid tool, and a bristle brush. In spite of the vast literature (books, videos, and other training materials) that exists for this program, it takes years to master all its capabilities and power. For more information see http://en.wikipedia.org/wiki/Adobe_Illustrator. GIMP is a free image manipulation program for tasks such as photo retouching, image composition, and image authoring. It is available from [GIMP 05] for many operating systems, in many languages. Inkscape is an editor for images in vector format. It runs on several computer platforms, it is free and is distributed under the GNU license. Its developer intends Inkscape to become a powerful graphics tool while being fully compliant with the XML, SVG, and CSS standards. At the time of writing, Inkscape is under active development, with new features being added regularly. See http://www.inkscape.org/. CorelDRAW is a vector-based graphics suite, offering more than just a vector graphics editor. It has been developed by http://www.corel.com/. It supports many powerful, useful features such as layout, tracing, photo editing, Web graphics, and animation. Having powerful competitors, CorelDRAW tries to improve on them in several ways, the most important of which is its being a suite of programs. It consists of the following applications: * * * * *
CorelDRAW: Vector graphics editing software Corel PHOTO-PAINT: Raster image creation and editing software Corel CONNECT: Content organizer Corel CAPTURE: Enables several methods of image-capture Corel PowerTRACE: Converts raster images to vector graphics (available inside the CorelDRAW suite)
Graphing Calculator is a tool for quickly visualizing mathematical objects and results. The user types an equation and the software computes and displays it without complicated dialogs or commands. See http://www.nucalc.com/. K3DSurf is software for plotting surfaces or higher-dimensional manifolds. See http://k3dsurf.sourceforge.net/. It can plot equations of three variables, parametric expressions for higher-dimension surfaces, and can also morph two images based on a variable. 3D-XplorMath, at http://3d-xplormath.org/, is software for visualization of geometric objects and processes. Surfaces, both 2D and 3D curves, complex-valued functions, and differential equations can all be plotted and animated. This software is particularly useful for those working in differential geometry, differential equations, or minimal surfaces. Virtual Math Labs. A group at the technical university of Berlin has developed a number of programs for exploring curves and surfaces. See http://www.math.tuberlin.de/geometrie/lab/curvesnsurfaces.shtml.
26
1.5 Resources For Computer Graphics
Gnuplot, at http://www.gnuplot.info, is a portable command-line driven graphing utility for linux, OS/2, MS Windows, OSX, VMS, and many other platforms. It was originally created to allow scientists and students to visualize mathematical functions and data interactively, but has grown to support many non-interactive uses such as web scripting. Windows Programs for Plotting Curves and Surfaces MathGV: http://www.mathgv.com/. 3D Grapher: http://www.romanlab.com/. Graphmatica: http://www8.pair.com/ksoft/. Graphis: http://www.kylebank.com/. DPGraph: http://www.dpgraph.com/. Advanced Grapher: http://www.serpik.com/. MathGrapher: http://www.mathgrapher.com/. 3D Graphics Applications POV-Ray, at http://www.povray.org/ is mostly a 3D ray-tracing renderer, but it has many commands and options that make it possible to build quite complex models. The acronym POV stands for persistence of vision. Several modelers, such as Bishop3D and LionSnake, output POV-Ray code that can be rendered directly. The program SU2POV converts the output of SketchUp to POV-Ray code. SketchUp, at http://sketchup.google.com/ is a 3D modeler designed for architects, civil engineers, filmmakers, game developers, and related professions. It has been acquired in 2006 by Google, so it also includes features to facilitate the placement of models in Google Earth. It was originally developed in 1999–2000 and it immediately became clear that its developers had hit on the right way to manipulate 3D objects on the 2D monitor screen. The basic SketchUp is free, while the pro version is commercial. The chief innovation of SketchUp is its Push/Pull technology, described in its patent application as follows: “System and method for three-dimensional modeling: A threedimensional design and modeling environment allows users to draw the outlines, or perimeters, of objects in a two-dimensional manner, similar to pencil and paper, already familiar to them. The two-dimensional, planar faces created by a user can then be pushed and pulled by editing tools within the environment to easily and intuitively model three-dimensional volumes and geometries.” There are quite a few books on this useful software. Many training videos are available at http://sketchup.google.com/training/videos.html. A feature of SketchUp is the 3D Warehouse that lets SketchUp users search for models made by others and contribute models. Maya, by Autodesk, is a high-level application for 3D animation, 3D modeling, simulation, visual effects, rendering, matchmoving, and compositing. Maya is used for animation in film and television, in commercials, video games, and architectural visualization and design. See http://en.wikipedia.org/wiki/Maya_(software). The name Maya has nothing to do with the Maya civilization or with Maya Angelou. It is simply the Sanskrit term for “illusion.”
1 Historical Notes
27
Modo is a powerful polygon and subdivision surface modeling and rendering tool. It also supports morphing, sculpting, 3D painting, and animation. Modo is developed by Luxology (http://www.luxology.com/modo/) and is currently at version 501. Modo is heavily used both by commercial entities (television, film, and video games) and by private individuals. The program incorporates features such as n-gons, 3D painting, and edge weighting, and runs on Mac OS X and Microsoft Windows platforms. Because of its large user base, huge libraries of textures, scenes, studio lighting, and special effects (such as hair, splashes, and water/fog) are available for Modo. There is also much training material. See http://en.wikipedia.org/wiki/Modo_(software). Carrara, from http://www.daz3d.com, is a 3D modeler and renderer. Judging from the examples displayed in its gallery, its users tend to develop models of humans, animals, and aliens. Bryce, also from http://www.daz3d.com, is a 3D modeling and animation package that purports to be the first name in 3D landscapes. It combines powerful features with a smart and simple user interface to create realistic digital landscapes. Blender, from http://www.blender.org, is free software with all the features of commercial programs. It has many active users and its price makes it the default choice for many people. However, its user interface is non-standard so it creates a feeling of user-unfriendliness. Some users complain about incomplete documentation. Zbrush, from http://www.pixologic.com, is a 3D painting and sculpting tool. An object is created by selecting a brush and using it to paint, chisel, and mold. 3d Studio Max, from http://www.autodesk.com/3dsmax, is considered by many the equivalent of Maya because of its similar set of features. Softimage, from http://www.softimage.com, is one of the most advanced 3D animation and character creation software for video games and movies. It uses several non-proprietary languages for its scripting, but its current user base seems small, perhaps because of its price. Many other modelers and renderers are currently available. A detailed list can be found at http://en.wikipedia.org/wiki/3D_computer_graphics_software. The following is an alphabetical list with just the names of the most important ones: 3ds Max (Autodesk), AC3D (Inivis), Aladdin4D (DiscreetFX), Cinema 4D (MAXON), CityEngine (Procedural Inc.), Cobalt, Electric Image Animation System (EI Technology Group), form’Z (AutoDesSys, Inc.), Houdini (Side Effects Software), Inventor (Autodesk), LightWave 3D (NewTek), MASSIVE, NX (Siemens PLM Software), Silo (Nevercenter), solidThinking (solidThinking), Solid Edge (Siemens PLM Software), SolidWorks (SolidWorks Corporation), Swift 3D (Electric Rain), trueSpace (Caligari Corporation), and Vue (Eon Software). Yesterday is history. Tomorrow is a mystery. And today? Today is a gift. That’s why we call it the present.
—Babatunde Olatunji
2 Raster Graphics “An image is worth a thousand words” is a well-known saying. It reflects the truth because the human eye–brain system is a high-capacity, sophisticated data processor. We can “digest” a huge amount of information if we receive it visually, as an image, rather than as a list of numbers. This is the reason for the success of computer graphics. However, if one image conveys a lot of information, many images are even more informative. This is why computer animation is so popular. It can bring to life impossible or non-existent objects and can teach and entertain. So it is not just science—reasoning about the physical world—that involves virtual reality. All reasoning, all thinking, and all external experience are forms of virtual reality. —David Deutsch, The Fabric Of Reality. Computer graphics has progressed over the years in three stages. The first stage was to develop hardware and algorithms and software to compute, construct, and display a single image consisting of smooth, curved, realistic-looking surfaces. The second stage was to extend the basic algorithms in order to create and display an entire animation made of many frames, where each frame is an image. The third stage is virtual reality, where computer graphics goes one step beyond passively watching animation. The main features of virtual reality are the following: 1. Interaction. Once a virtual three-dimensional world is built, a user can walk through it at will, often also “grabbing” objects and manipulating them. 2. Realistic views. With the use of a special helmet (or a head-mounted display, Section 26.14.2) where each eye watches its own display, the user can look in one direction while moving in another. We say that this adds degrees of freedom to the display. The helmet also allows a stereo pair of images to be displayed, adding to the visual realism. Exercise 2.1: Use your crystal ball (or, in its absence, the answer provided here) to predict the next stage beyond virtual reality. D. Salomon, The Computer Graphics Manual, Texts in Computer Science, DOI 10.1007/978-0-85729-886-7_2, © Springer-Verlag London Limited 2011
29
30
2.1 Pixels
2.1 Pixels A digital image is a rectangular array of dots, or picture elements, arranged in m rows and n columns. The expression m × n is called the resolution of the image, and the dots are called pixels (except in the cases of fax images and video compression, where they are referred to as pels). The term resolution is sometimes also used to indicate the number of pixels per unit length of the image. Thus, dpi stands for dots per inch. Images are all around us. We see them in color and in high resolution. Many objects (especially artificial objects) seem perfectly smooth and continuous, with no jagged edges and no graininess. Computer graphics, on the other hand, deals with images that consist of small dots, pixels. The term pixel stands for “picture element” (see [Lyon 09] for a lively survey of the history of this important term). When we first hear of this feature of computer graphics, we tend to dismiss the entire field as trivial. It seems intuitively obvious that an image that consists of dots would always look discrete, grainy, rough, and inferior to what we see with our eyes. Yet state-of-the-art computer-generated images are often difficult or impossible to distinguish from their real counterparts, even though they are discrete, made of pixels, and not continuous. (See also Section 21.3 for a discussion of human vision and the resolution of the eye.) A similar dichotomy between discrete and continuous exists in art. Many painters try to mimic nature and employ wide, continuous brush strokes to paint smooth and continuous pictures, while others choose to be pointillists. They create a painting by placing many small dots on their canvas (see Section 2.29 for a similar approach). The most important pointillist was the 19th century French impressionist Georges Seurat. Seurat was a leader in the late 19th century neo-impressionism movement, a school of painting that uses tiny brushstrokes of contrasting colors to achieve a delicate play of light and create subtle changes in form. Seurat used this technique, which became known as pointillism or divisionism, to create large paintings made entirely of small dots of pure color. The dots are too small to be distinguished when looking at the work in its entirety, but they make his paintings shimmer with brilliance. His most well-known works are Une Baignade (1883–1884) and Un Dimanche Apr`esmidi a ` l’Ile de la Grande Jatte (1884–1886). The art critic Ars`ene Alexandre had this to say about the latter painting: “Everything was so new in this immense painting—the conception was bold and the technique one that nobody had ever seen or heard before. This was the famous pointillism.” Most engineers, programmers, and users think of pixels as small squares, and this is generally true for pixels on computer monitors. Pixels in other digital output devices (displays or printers) may be rectangular or circular. However, in principle, a pixel should be considered a mathematical, dimensionless, point (see [Smith 09]). It seems impossible to reconstruct a continuous image from an array of discrete pixels, but this is precisely what the surprising Nyquist-Shannon sampling theorem [Nyquist 03] tells
2 Raster Graphics
31
Figure 2.1: Pointillism.
us (in fact, what it guarantees). This theorem is often applied to digitized audio (a one-dimensional signal), but here we apply it to two-dimensional images. Audio is a good starting point to understand the sampling theorem. Sound fed into a microphone is converted to an electrical voltage that varies with time; it becomes a wave. A wave has a frequency, and a wave that varies all the time consists of many frequencies. We denote the maximum frequency contained in a wave by B (cycles per second, or Hertz). The samping theorem says that it is possible to reconstruct the original wave if it is sampled at a rate greater than 2B samples per second. An image is a rectangular array of point samples (pixels). The sampling theorem guarantees that we’ll be able to reconstruct the image (i.e., to compute the color of every mathematical point in the image) if we sample the image at a rate greater than 2B pixels per unit length, where B is the maximum pixel frequency in the image. Section 24.2 explains the meaning of the term “pixel frequencies in an image,” but in practice, pixels, their values, and their frequencies depend on the accuracy of the capturing device. An ideal device should measure the color of an image at certain points, but image sensors (CCDs and CMOS) used in real devices (cameras and scanners) are often far from ideal. Because of physical limitations, manufacturing defects, and the need to capture enough light, an image sensor often measures the average color (or intensity) of a small area of the image, instead of the color at a point. Assuming that we have enough pixels for a given digital image, we compute the color of a given point in the image by interpolation. Section 2.11 discusses bilinear interpolation and Part III of this book discusses this and other interpolation methods and illustrates them with examples. In this chapter, we are interested in interpolation of pixels (Sections 2.4, 2.5, and 2.10). The discussion assumes a grayscale image, where each pixel is a number indicating a shade of gray (an intensity), but interpolation can easily be extended to color images, where a pixel is a triplet of primary colors.
2.2 Graphics Output
32
2.2 Graphics Output The two traditional graphics output devices are the display monitor (CRT or LCD, Chapter 26) and the printer (mostly inkjet and laser, because hard copy is often needed). These output devices are two-dimensional, which is why a three-dimensional image has to be projected before it can be output (see Part II for projections). Many graphics output devices are currently available, but the discussion in this section employs the CRT to illustrate the important concepts of vector and raster scans. For such an advanced civilization as ours to be without images that are adequate to it is as serious a defect as being without memory. —Werner Herzog.
2.2.1 Vector Scan Current computer graphics hardware is virtually always implemented as raster scan. Very few display monitors are of the vector scan type (also known as random scan). In vector scan, where the display unit itself is normally a CRT, the program prepares graphics commands in a buffer in memory. Each command starts with a code (for example, 0 for point, 1 for line, 2 for circle, and so on), followed by command-specific data. A few examples are listed below. The program then starts the graphics controller, a hardware unit that controls all the graphics operations in the computer, and gives it the start address of the buffer. The controller fetches the commands from the buffer in memory and executes each to display a graphics object on the screen. Each command is executed by moving on the screen to where the object should start, turning the display on, and moving a beam to draw the object. At the end of the buffer, the program should place a branch command that sends the graphics controller to the start of the buffer, to refresh the display. Examples of common commands are the following: Point:
0
x1
y1
Line:
1
x1
y1
x2
y2
2 x1 y1 r 3 address Circle: Goto: The main advantage of vector scan is a smooth display. Slanted lines, arcs, circles, and other objects come out smooth and perfect. However, the hardware is complex (it has to be able, for example, to sweep the beam in a perfect circle around the screen); also, a complex image implies many commands. A good-quality, static display requires at least 20 refreshes per second. If the graphics controller takes more than about one 20th of a second to execute all the commands in the buffer, the refresh rate becomes too low, which causes the display to flicker. Vector scan is suitable for applications such as drafting. A technical drawing consists of a few types of graphics objects such as lines, arcs, rectangles, text, and arrowheads. Each becomes a command in the buffer. General graphics applications, however, often require the display of a smooth, curved surface whose color may vary continuously from point to point. In a vector scan graphics system, such a surface can only be represented by drawing a large number of points in different colors, which causes slow refresh and results in flickering.
2 Raster Graphics
33
2.2.2 Raster Scan Virtually all current graphics displays are raster scan devices. The principle of raster scan is to prepare the entire image in a buffer (called the bitmap) in memory in terms of dots. The final image consists of dots, and a number is stored in the bitmap for each dot, specifying the color of the dot. The graphics controller scans the bitmap number by number and draws a dot, called a pixel, on the screen for each number. If the display is bilevel (or monochromatic, displaying just two colors, black and white or foreground and background), then the color of a pixel is specified by one bit in the bitmap, normally 0 for background and 1 for foreground. If the display supports 2n shades of gray, then the bitmap should contain an n-bit number for each pixel. This is sometimes referred to as a bitmap with n bitplanes. For color displays, each pixel must be represented by three numbers in the bitmap. Figure 2.2 illustrates how a large, complex color image consists of a large number of pixels.
Figure 2.2: Pixels in a Raster Scan Display.
Mouse droppings: [MS-DOS] n. Pixels (usually single) that are not properly restored when the mouse pointer moves away from a particular location on the screen, producing the appearance that the mouse pointer has left droppings behind. —Eric Raymond, The Hacker’s Dictionary. It is useful to imagine the screen as a rectangle with r rows and c columns. A pixel has screen coordinates (x, y), where x is in the range [0, c − 1] and y is in the range [0, r − 1]. However, the bitmap is stored in memory as a one-dimensional array with index values from 0 to r × c − 1. If the bitmap is stored row by row in array bitmap, then the index of pixel (x, y) is yc+x, and the pixel is accessed in the array by bitmap[yc+x]:=.... Unfortunately, the common operation of accessing a pixel requires
2.2 Graphics Output
34
a multiplication. To simplify it, the computer designer should choose a value for c that’s a power of 2. The multiplication can then be replaced by a shift. Exercise 2.2: On some screens, the rows are numbered from the bottom up (the top screen line is row r − 1 instead of row 0). What would be the pixel’s index in array bitmap in such a case? A typical bitmap in current computers represents a pixel by three bytes, each corresponding to one of the three primary colors. Each primary color can therefore have 256 intensities and a pixel can have one of 224 ≈ 16.8 million colors. For a 1K ×1K screen resolution, this requires a bitmap of size 3×210 ×210 = 3×220 = 3 Mbytes. Doubling the resolution to 2K ×2K requires a bitmap four times as big. Graphics applications are one reason why today’s computers have such large memories. R G B 0 1 2 3 110 0 255
3 Bitmap
256 entries
255 Color Lookup Table
Figure 2.3: A Bitmap and a Color Lookup Table.
To save memory it is possible to use a color lookup table (Figure 2.3). The bitmap may consist of just one byte per pixel, containing a number between 0 and 255. That number is used by the graphics controller as a pointer to the color lookup table, which has 256 entries, each three bytes wide, specifying the intensities of red, green, and blue. The program first decides what colors are going to be used, and stores their values in the lookup table. Each color can be selected from a palette of 224 ≈ 16.8 million colors, but because of the size of the lookup table, a maximum of 256 colors can be displayed simultaneously. The program then scan-converts the objects to be displayed and stores pointers in the bitmap. Palette—board on which paints are laid and mixed. The graphics controller has a simple task. It scans the screen line by line and, at the same time, reads numbers from the bitmap and uses them to turn pixels on and off during each scan. The entire image is drawn as dots on scan lines (with nothing between the scan lines). In the case of a color lookup table, the controller reads a byte from the bitmap, uses it as a pointer, goes to the table, and uses the values in the three bytes to adjust the intensities of the three electron beams (in a CRT) or three pixels (in an LCD). This process is repeated for every pixel.
2 Raster Graphics
35
Some graphics controllers perform an interlaced scan. Such a controller first scans all the odd-numbered lines, then all the even-numbered ones, and then starts the refresh. This is supposed to create a smooth display even for low refresh rates, because of the way our eyes work. The advantages of raster scan are simple hardware, fast scan, and no flickering (the time it takes for a complete scan does not depend on the complexity of the image). The main downside is that an image appears on the screen as a large collection of pixels. Increasing the resolution of the screen allows for more complex images, but slanted lines and curved objects always have jagged edges. Vector scan, in contrast, does not involve any pixels and generates smooth lines and curves on the screen. Another disadvantage is that, in addition to the bitmap, another data structure (a geometric one) is needed, to keep track of the individual image elements. Imagine a line whose pixels are turned on in the bitmap. If the line has to be erased, there is no way to tell what bits in the bitmap belong to the line. The geometric data (code and coordinates of endpoints) of the line should be stored in the geometric data structure, then scan-converted from there. To erase the line, its geometric data should first be located in the geometric data structure. It should be scan converted and each pixel, in turn, erased. Historically, vector scan was the first method used, in the 1960s and 1970s, in computer graphics. In the early 1980s, however—with the advent of high-resolution, low-cost color monitors (and especially LCD monitors in the 1990s)—raster scan became the dominant display method. The reason for the popularity of raster scan is that in a complex image with many colors, a vector display requires many short vectors, thus increasing the scan time close to flickering. A raster scan takes the same time to complete a scan, regardless of the number of different pixels displayed, so the complexity of the image and the number of colors do not degrade the performance.
2.2.3 The Refresh Operation The graphics controller saves the coordinates of the currently refreshed pixel in two internal registers, x and y. The registers are used to calculate the bitmap address, yc+x, of the pixel (Figure 2.4), allowing the graphics controller to read the pixel’s value from the bitmap. Notice how the multiplication yc is achieved by simply placing y to the left of x and concatenating them. The value read is then converted to a voltage that’s used to turn the electron beam (in a CRT) or the current pixel (in an LCD) on or off. In a color display, three numbers are read from the bitmap for each pixel, and each is converted to a voltage that’s used to control the intensity of three beams. While the bitmap is read, the graphics controller increments register x, to point to the next pixel. When register x overflows, it automatically returns to zero (think of an odometer going from 9 . . . 99 to 0 . . . 00) and the overflow signal is used to clock (i.e., to increment) register y and also to move the beam to the start of the next scan line. When register y overflows, it also returns to zero and its overflow signal is used to return the beam to the start of the first scan line.
2.2.4 Windows and BitBlt In current computers, the operating system maintains the screen as a bitmap. Everything displayed on the screen—text, cursor, and graphics—is saved in the bitmap as
2.2 Graphics Output
36
Memory
CRT controller
CRT
Data bus Converter
Beam on/off Start new scan
Overflow Clock Overflow Clock Bitmap
y register
Start new scan line
x register
Bitmap address
Internal computer clock
Address bus Figure 2.4: CRT Refresh.
pixels and is constantly refreshed on the screen by the graphics controller. A useful feature supported by current operating systems is multiple windows. At any time, the screen may display several windows (or parts thereof). They can be opened, moved, resized, and closed. The window manager, part of the operating system, is responsible for these operations. The main task of the window manager is to display the windows on the screen in a way that makes sense to the user. Imagine a window A fully displayed on the screen. If another window B is moved by the user over A, completely or partially obscuring it, the window manager has to erase all or part of A and replace it with B. When B is moved again, window A (or part of it) should be restored. If A is static, its content can be saved in memory and later used to restore A. However, if window A is displayed by a program, its content may change all the time. The program must therefore keep the pixels of A in a buffer in memory and update the buffer all the time. The window manager uses this buffer to restore A when necessary. Many graphics packages offer a set of procedures collectively referred to as BitBlt— an acronym that stands for “BIT Boundary bLock Transfer” and is pronounced “bitblit.” These procedures can save parts of the bitmap in, and also restore parts from, memory. The operating system must have routines to provide BitBlt with fresh memory areas and to reclaim unneeded areas. This is done by a dynamic memory allocation method such as first fit, best fit, or the buddy system. Because BitBlt operations are common, they should be fast; today there are VLSI circuits that do BitBlt in hardware at speeds of about 100 million pixels per second. Such a circuit can do more than just save and restore. It can also perform a logical operation while restoring the image, a feature that can be applied to create useful effects. We call the new image “source” and the current bitmap (in which the new image is going to be written) the “destination.” Instead of simply writing each source bit into
2 Raster Graphics
37
the bitmap, thereby erasing a destination bit, the BitBlt circuit writes a bit that’s a function of the source and destination bits. The rule is dst:=dst op src, where op is any logical operation on bits. Table 2.5 shows examples of logical operations (the rows are the source and the columns are the destinations). 01 0 00 1 01
01 0 01 1 10
01 0 01 1 11
01 0 01 1 00
Table 2.5: Some Logical Truth Tables.
Exercise 2.3: How many logical operations are possible in principle? Here are a few examples of logical operations performed by BitBlt (zero represents a white pixel and one represents a black pixel). Op Source
= Destination
replace
or
xor
and
dst:=src. This is the replace operation. It draws the new source on the screen, regardless of what’s on the screen now (i.e., regardless of the destination). It amounts to overpainting the new source on the screen (a destructive write). This is common when windows are created and moved. It can also be used to erase areas by drawing in white (the background color). dst:=0. This clears the screen. dst:=not dst. This inverts the screen. A process of inverting bits, changing ones to zeros and vice versa. For example, in a graphics program, to invert a black-and-white bit-mapped image (to change black to white and vice versa), the program could simply flip the bits that compose the bitmap.
dst:=dst or src. This adds the source to the destination (a nondestructive write). Oring a gray pattern on a white screen results in the pattern. Oring the same pattern on a black background has no effect. The paint brush in many paint programs uses this mode. The and operation dst:=dst and src can be used to erase destination pixels selectively. dst:=dst xor src. A pixel would be black whenever the source and destination pixels differ. This turns out to be an important mode. A common example is erasing an
2.2 Graphics Output
38
object from the bitmap. Imagine an image consisting of simple geometric objects. To erase an object, the coordinates of all its pixels have to be determined, so that they can be erased. The problem is that objects may intersect; erasing all the pixels of an object will therefore leave holes in all the objects that happen to intersect it. The solution is to draw in xor mode when the object is originally drawn and also when it is erased. The term xor stands for exclusive-or. This is a simple logical operation that is defined as follows. The xor of two bits a and b is 1 when a is 1 or b is 1 or both are 1’s. This simple operation has many surprising and powerful applications in computer programming. When objects are drawn and erased in xor, the intersection of several objects (which is normally just one pixel or a few pixels) flips between black and white each time an object is drawn or is erased. Figures 2.6 and 2.7 illustrate an example. In Figure 2.6a, a horizontal line is drawn in xor mode by flipping pixels from white to black. In Figure 2.6b, a vertical line is drawn, also in xor mode, so the pixel at the intersection point becomes white. In Figure 2.6c, a slanted line is drawn, so the same pixel becomes black again. In Figure 2.6d, the vertical line is erased, and in Figure 2.6e, the horizontal line is erased as well. The pixel at the intersection keeps changing its color, but the picture as a whole does not look bad. Note that both the drawing and erasing are done with a source of 1. The BitBlt operation is dst:=dst xor 1.
• • • • • • •
• • • • • • • • • • • •
• • • • • • • • • • • • • • • • • • •
• • • • • • • • • • • •
• • • • • • •
(a)
(b)
(c)
(d)
(e)
Figure 2.6: Xor at Line Intersections.
001=
100
010=
100 xor 001=101
011=
100=
101 xor 010=111
101=
110=
111 xor 001=110
111=
110 xor 100=010
Figure 2.7: Drawing and Erasing Color Lines in Xor Mode.
Exercise 2.4: Is the erase operation common enough to justify xor drawing? Exercise 2.5: Can xor drawing be used when drawing in color (i.e., when a pixel consists of more than 1 bit)? Xor drawing can also be used with a color lookup table. If an original entry in the bitmap is 3 (=0011) and we draw a new object with a source of 5 (=0101), then
2 Raster Graphics
39
the bitmap entry is going to change to (0011 xor 0101)=0110=6. The pixel at the intersection point now points to entry 6 in the color lookup table. If we now erase the original pixels of 3, then (3 xor 6) will result in 5. Example: The concept of transparency is important when adding images to a bitmap. A common example is a web page. Such a page normally has a certain background and new images added to the page obscure that background. When adding a new image to our page, we may want certain parts of the image to be transparent. Such parts should show the background instead of any image pixels. Transparency is achieved by declaring one of the colors used in the image a transparent color (a notable example is the GIF89 graphics file format). The browser displaying the page finds this declaration in the image file. This example shows how several BitBlt operations can be used to implement transparency. We assume that a bitmap and an image are given (parts (a) and (b), respectively, of Figure 2.8) and that the image is to be added to a certain region of the bitmap (the bottom-left 4 × 4 part). We also assume that the image file contains information about the transparent parts. The operation proceeds in the following four steps: 1. Create a bitmap the size of the image, set all the transparent pixels in it to 1 and all the opaque pixels to 0. This bitmap is called a mask (Figure 2.8c). 2. Perform an exclusive-or (xor) of the image with the proper bitmap region (in our example, the bottom-left 4 × 4 part). Store the result in the bitmap (Figure 2.8d). 3. Perform a logical AND of the bitmap and the mask and store it in the bitmap (Figure 2.8e). This operation leaves all the transparent areas unchanged and sets the opaque areas to zeros. 4. Xor the proper bitmap region and the image. This operation sets all the transparent areas back to their original values (because two xor operations cancel each other out). The opaque areas now contain the desired image bits (Figure 2.8f) because an xor with a zero leaves a bit unchanged. 0 0 1 1 0
0 0 1 0 1
0 1 0 1 0
1 1 0 0 1
1 1 1 0 1
(a) Bitmap 0 0 0 1 1
0 1 1 0 0
0 1 1 0 0
1 0 0 1 1
(d)
1 1 1 0 1
0 1 0 1
1 0 0 1
0 1 1 0
1 0 1 0
0 0 1 1
(b) Image 0 0 0 1 1
0 0 0 0 0
0 0 0 0 0 (e)
1 0 0 1 1
0 0 1 1
0 0 1 1
0 0 1 1
(c) Mask 1 1 1 0 1
0 0 1 1 0
0 1 0 0 1
0 0 1 1 0 (f)
Figure 2.8: Transparency in a Bitmap.
1 1 0 0 1
1 1 1 0 1
2.3 The Source-Target Paradigm
40
2.3 The Source-Target Paradigm The source-target paradigm is a simple, useful concept that has to do with bitmap operations. An image taken with a digital camera or produced by a scanner exists only as a bitmap, but users may want to perform many operations on such an image. A bitmap operation starts with a source bitmap S and uses its pixels (x, y) to produce a target bitmap G with pixels (x∗ , y ∗ ). We can denote such an operation by G = T ·S or (x∗ , y ∗ ) = T (x, y), where T is the operation or transformation from S to G. The most common examples of bitmap operations are bitmap scaling, stretching, and rotation, image sharpening, blurring, and pixelating, color enhancement, and converting to grayscale. The source-target paradigm has to do with bitmap operations that change the size of the bitmap (the target bitmap may be larger or smaller than the source bitmap). When a bitmap is enlarged, the target bitmap has more pixels than the source bitmap. When we write a program to perform such scaling, we tend to implement it as a loop that goes over the pixels of the source bitmap. For each pixel (x, y), the program computes new coordinates x∗ and y ∗ and a new value. It then stores the value in location (x∗ , y ∗ ) of the target bitmap G. We can refer to such a program as a source-to-target algorithm and it is clear that this type of algorithm involves a problem. If the target is greater than the source, some target locations will be left empty. If the target is smaller than the source (bitmap shrinking), some target locations may be “covered” with multiple new values from several source pixels. Thus, implementing a bitmap operation that changes the size of the bitmap should be done in a target-to-source algorithm. The program loops over the pixels of the target bitmap G and computes coordinates (x, y) for every target pixel (x∗ , y ∗ ). This can be denoted by (x, y) = T −1 (x∗ , y ∗ ), where T −1 is the inverse of the original transformation T . If the inverse transformation T −1 cannot be figured out, the program has to use a source-to-target algorithm, but must include an extra step that loops over the target pixels, identify each empty pixel and compute a value for it. Figure 2.9 illustrates these types of algorithms.
S
y
T
G
S y*
x
x*
T -1
G y*
y x
x*
Figure 2.9: The Source-Target Paradigm.
Figure 2.9 illustrates another important problem in bitmap operations. The coordinates of pixels are integers, but the coordinates of transformed pixels may be nonintegers, because T and its inverse may transform many pairs of integers to pairs of
2 Raster Graphics
41
real numbers. If an inverse transformation T −1 (x∗ , y ∗ ) results in a pair (x, y) of real numbers, the bitmap-operation algorithm must decide how to convert them to integers. This is an interpolation problem (more specifically, two-dimensional interpolation) and it can be solved in various ways as illustrated in the next section.
2.4 Interpolation Interpolation (in the context used here) is the process of determining intermediate values from a set of discrete values (but see Section 2.10 for a different definition). The word comes from the Latin inter (between) and polare (to polish) and it means to compute new values that lie between (or that are an average of) certain given values. This book is about synthesizing images, which involves pixels, points, and vectors. Therefore, most of the interpolations considered in the book are two dimensional. Here are some examples of two-dimensional interpolation. Rounding. Certainly the simplest solution is to round any real coordinates to the nearest integer. This is denoted by round(x) and is done by x + 0.5. Bilinear. The next obvious interpolation technique assigns a value to pixel (x∗ , y ∗ ) that is a bilinear interpolation of the four near neighbors of the pair (x, y) of real numbers. This type of interpolation is the topic of Section 9.3 (but see also Section 2.11) and is summarized in Equation (9.6), duplicated here P(u, w) = P00 (1 − u)(1 − w) + P01 (1 − u)w + P10 u(1 − w) + P11 uw =
1 1
B1i (u)Pij B1j (w),
i=0 j=0
= [B10 (u), B11 (u)]
P00 P10
P01 P11
(9.6)
B10 (w) , B11 (w)
where Figure 2.10a illustrates the meanings of u and w in this case (notice that these parameters are in the interval [0, 1]).
P00
P10 w
y
2w P 11
u P01
x (a)
P00
P11
2u P22 (b)
(c)
Figure 2.10: Bilinear, Biquadratic, and Bicubic Interpolations.
2.4 Interpolation
42
Biquadratic. A better approach for certain applications, such as free-form deformations (Section 19.11), may be biquadratic interpolation. The new value stored in target pixel (x∗ , y ∗ ) is a weighted sum of a group of 3 × 3 source pixels centered on the pixel nearest (x, y). This is illustrated in Figure 2.10b and the weighted sum is given by Equation (10.21), duplicated here ⎛
⎞⎛ 2 −4 2 P22 4 −1 ⎠ ⎝ P12 P(u, w) = (u , u, 1) ⎝ −3 1 0 0 P02 ⎞T ⎛ 2 ⎞ ⎛ 2 −4 2 w 4 −1 ⎠ ⎝ w ⎠ . × ⎝ −3 1 0 0 1 2
P21 P11 P01
⎞ P20 P10 ⎠ P00
(10.21)
Notice that the distances from the top-left corner of the group to the (red) pixel at (x, y) are denoted by 2u and 2w. These quantities vary from zero (on the left or the top) to 2, so the parameters u and w of Equation (10.21) vary from 0 to 1. If the red pixel happens to be at the center of the group, both u and w will equal 0.5. Since this book is about computer graphics, it is best to explain biquadratic interpolation in graphical terms. Imagine that the 3×3 pixels of the group are three-dimensional points with x and y coordinates that vary from 0 to 2 and z coordinates that are the colors of the pixels. The biquadratic interpolation computes a smooth surface that spans the nine points. Once the location of the red pixel within the surface is determined, the color of the target pixel (x∗ , y ∗ ) is set to the height of the surface at that location. Bicubic. Extending the principles of bilinear and biquadratic interpolations results in the popular bicubic interpolation (Sections 2.12.3 and 10.6). The new value stored in target pixel (x∗ , y ∗ ) in this type of interpolation is a weighted sum of a group of 4 × 4 source pixels selected such that the red pixel at (x, y) is located somewhere in the center square of the group. Figure 2.10c illustrates this type of interpolation, while Equation (10.25) lists the 16 weights of the sum. Catmull-Rom. The Catmull–Rom surface patch of Section 12.7 can be used to interpolate a pixel as a weighted sum of a group of 4 × 4 pixels, similar to bicubic interpolation. The details of this weighted sum are listed in Equation (12.55). The interpolating B´ezier surface patch of Section 13.22 and the interpolating bicubic B-spline patch of Section 14.18 may also be used for two-dimensional interpolation.
2.4.1 The Sinc Function Section 2.1 discusses pixels and shows that a pixel should be considered a mathematical, dimensionless point. It seems impossible to reconstruct a continuous image from an array of discrete pixels, but it has long been known (from the so-called Nyquist-Shannon sampling theorem [Nyquist 03]) that such a feat is possible, provided that the pixels are dense enough. The sampling theorem guarantees that a set of discrete points can perfectly reconstruct a continuous function if the points are dense enough (if there are enough points per unit length). To get a better understanding of this theorem and its consequences, we examine the one-dimensional case. We know that a function g(t) can be plotted as a curve. Such a curve may go up and down and behave like a wave. In any given interval, this wave has
2 Raster Graphics
43
a frequency. We select the highest frequency of the function, and denote it by f . The sampling theorem guarantees (subject to certain conditions) that g(t) can be perfectly reconstructed if it is sampled at a rate greater than 2f . To sample a function means to evaluate it at many points. Thus, if f = 1200, we should sample the function at, say 2500 points for each unit length. For example, from g(3) to g(4) we need to evaluate the function 2500 times and create 2500 equally-spaced samples. Such a set of samples can reconstruct the original, continuous function perfectly. Given the samples, we can compute g(t) at any point t to full precision. Thus, the sampling theorem provides an ideal interpolation method, but like any other ideal concept, solution, or construction, it cannot be fully applied in practice, because (1) the set of samples is often infinite and (2) even if it is finite, too many samples are required for ideal interpolation. In order to be practical, interpolation must include only a relatively small number of samples. Perfect reconstruction is done with the well-known sinc function, which is defined as 1, for x = 0, sinc(x) = sin(πx) , for |x| > 0. πx The name sinc is a contraction of sinus cardinalis (Latin for cardinal sine). Figure 2.11 shows a finite portion of this function, which is a sine wave, scaled vertically to become higher around zero. 1.0 0.8 0.6 0.4 0.2
15
10
5
5
10
15
0.2
Figure 2.11: The Sinc Function.
Reconstruction is done by the infinite sum g(t) =
∞
sinc(t − u)s(u) = sinc ∗ s,
u=−∞
where the “*” symbol stands for convolution. Figure 2.12 illustrates this process graphically (it is obvious that there are not enough samples in the figure). The most striking feature of this figure is that the sinc function is often negative, and many samples are
2.4 Interpolation
44
multiplied by negative weights as a result. It makes sense to multiply nearby samples by large weights and decrease the weights for samples that are farther away, but what is the meaning of negative weights? The explanation is found in the answers to Exercises 10.13 and 2.12.
Discretesamples
Convolution
sinc
Continuousfunction Figure 2.12: Ideal Interpolation with Sinc.
For the two-dimensional case we start with a set s(u, w) of samples taken from a function G(x, y) and we use this set to compute any value of G with the double sum G(x, y) =
∞
∞
sinc(x − u)sinc(y − w)s(u, w) = SINC ∗ s,
u=−∞ w=−∞
where SINC(x, y) = sinc(x)sinc(y).
2 Raster Graphics
45
2.5 Bitmap Scaling An image is made of pixels, so its size is measured in pixels, not in inches or centimeters. When an image has to be displayed or printed, it often has to be scaled to fit the available space. Scaling can be up or down. Scaling up (or zooming in) enlarges the bitmap, while scaling down (or zooming out) reduces it. Both types of scaling are important, they represent different problems, and are often implemented differently. The amount of scaling is the first feature to be discussed and we start with onedimensional scaling. When a straight segment of length L is left as is, we say that its length has increased by 0%. When it is stretched to 2L, we say that its size has increased by 100%. Because the length has doubled (and because the word “double” is associated with the integer 2) we also say that the segment has been stretched or scaled by a factor of 2. Similarly, when the segment is stretched to 3L, it has been scaled by a factor of 3 (because 3L/L = 3) or by 200%. This is a little confusing, and the confusion increases when we consider noninteger scaling, when we take into account shrinking, and when we discuss images, which are two dimensional. When a segment is stretched to three times its length, it has been scaled by 200%, because the difference in lengths 3L − L equals 2L. This answers the question what if a segment has been stretched from L to 2.35L? We simply subtract 2.35L − L = 1.35L and say that the length has increased by 135% or by a factor of 2.35. When a segment is shrunk to 0.4L, it makes sense to claim that its length has increased by 0.4L − L = −0.6L or −60% (or by a factor of 0.4L/L = 0.4). We find it more convenient, however, to talk about 60% shrinking or scaling down by 0.4. When an image of size m × n is left alone, it makes sense to say that it has been scaled by 0%. When both its dimensions have doubled, we say that it has been scaled by a factor 2 or by 100%. The point is that the area of the image is now four times what it used to be, but the terms “factor” and “percent” refer to the increase in the linear dimensions, not the area. In principle, the height and width of an image may be scaled by different amounts, and in such a case we have to use two numbers (factors or percentages) to express the amount of scaling. Thus, increasing the height of an image from H to 2.35H and decreasing its width from W to 0.4W is considered scaling by the pair (135%, −60%) or by (2.35, 0.4). This long section discusses several approaches to bitmap scaling, as follows: Replicate rows and columns as needed (or remove some of them for shrinking). Generate new pixels by assigning them the values of original pixels. This is better than simply replicating entire rows or columns and is also fast, because pixels are simply copied and no computations are needed. Use interpolation to compute values for new pixels. Each new pixel becomes a weighted average of several of its near neighbors, original pixels. This approach involves many calculations and is slow, but normally produces better results than simply replicating rows or individual pixels. Use one of the approaches above, but also examine the original pixels for any edges in the image and try to preserve them. Pixels on or close to an edge should be treated
46
2.6 Bitmap Stretching
differently from pixels located away from a sharp edge. Such techniques are more time consuming, but may produce better results, especially for large scale factors.
2.6 Bitmap Stretching One way to scale a bitmap is to stretch it. This is done by replicating rows and columns several times. To stretch a bitmap to three times its width and twice its height, replicate each column three times and replicate each row twice, creating six identical target pixels for each source pixel. This is simple, but the result is a coarse image, visibly made of large rectangular blocks. To shrink a bitmap (by negative stretching) to a third of its width and half its height, delete one-third of its columns and every other row. Leave the rest of the original bitmap untouched. This is also fast, but the resulting image may be missing important details and may be noticeably different from the original image.
2.7 Replicating Pixels Instead of stretching a bitmap by replicating entire columns and rows, the approach described here generally achieves better results by replicating individual pixels. When a small bitmap U is scaled up to become a larger bitmap V , each of its pixels is replicated several times to become a small, uniform, rectangular region in V . Notice that these regions cannot all be the same size. If we start from a bitmap with P pixels and replicate each pixel into a small region of size Q, then the resulting scaled bitmap will consist of P · Q pixels, but the user may want a different target size. Any algorithm that scales by replicating pixels is given the dimensions of the original and final bitmaps, and it has to generate regions of different dimensions (but not very different) that together will have the required dimensions of the final bitmap. This approach is illustrated here by an algorithm due to [Andres et al. 96], which has two important properties (1) it can scale a bitmap up by any rational factors (and they can be different for the rows and columns) and (2) it is reversible. The main advantage of such an algorithm is its simplicity, which translates to speed. Replicating a pixel several times is much faster than interpolating values. When a bitmap is scaled by clicking and dragging, the program normally displays an empty outline that follows the mouse. The scaled bitmap is computed and displayed only when the dragging is complete. When this method is used, the software may be able to compute and display the scaled bitmap several times per second, while also following the dragged mouse. We assume that a small bitmap UK,L is to be scaled to a bigger bitmap VM,N . The pixels of U are denoted by (x, y) and the pixels of V are denoted by (x , y ). Figure 2.13 illustrates the basic parameters involved in this process and it shows that a pixel in U with row and column coordinates (x, y) should be moved to location (x , y ) = (M x/K, N y/L) in V . However, coordinates of pixels must be integers, so we modify the above to (x , y ) = (M x/K, N y/L). Because of the use of the floor
2 Raster Graphics L
N
x K
47
x’
(x,y) M
(x’,y’)
Figure 2.13: Relation of x to x.
(integer truncation) operation it is obvious that several values of x may yield the same x , and this fact is the basis for the reverse transformation. The reverse transformation, from V to W , is extremely simple. Given a pixel (x , y ) in V , it is moved to become pixel (x, y) = (K x /M , L y /N ) ,
(2.1)
in U . This moves all the (identical) pixels in a rectangular region of V to the same location in U . Mathematical experience, combined with a little thinking, produces the forward transformation. Given a pixel (x, y) in U , we want to determine the region in V where it should be replicated, i.e., the region of pixels (x , y ) that will be transformed to this particular (x, y) by equation (2.1). A little tinkering yields the inequalities
Mx + K − 1 Mx + M − 1 ≤ x ≤ or C0 (x) ≤ x ≤ C1 (x) K K and
Ny + N − 1 Ny + L − 1 ≤ y ≤ or R0 (y) ≤ y ≤ R1 (y). L L The scaling algorithm is now obvious. It loops over the pixels of U and for each pixel (x, y) it computes C0 (x), C1 (x), R0 (y), and R1 (y) from the equations above. It then fills that region of V bounded by rows R0 (y) and R1 (y) and by columns C0 (x) and C1 (x) with copies of pixel (x, y). The rectangular regions in V can have the four sizes [M/K × N/L], [M/K × (N/L + 1)],
[(M/K + 1) × N/L], [(M/K + 1) × (N/L + 1)].
When the reverse transformation is applied to V , all the pixels of a region are moved to the same location in U . This scaling algorithm is perfectly reversible. Example: Given the small bitmap U4,3 , we scale it to become the large V15,13 . The scale factors are 15/4 (vertical) and 13/3 (horizontal). The bounds of the regions in V are
15x + 3 15x + 14 13y + 2 13y + 12 , C1 (x) = , R0 (y) = , R1 (y) = . C0 (x) = 4 4 3 3
2.8 Scaling Bitmaps with Bresenham
48
Thus, for example, pixel (3, 2) of U is transformed as follows:
(3, 2) →
59 28 38 48 , × , = [12, 14] × [9, 12]. 4 4 3 3
This pixel is therefore replicated 12 times into the 3 × 4 region of V that is bounded by rows 12 through 14 and by columns 9 through 12. Figure 2.14 shows the 12 regions of V and the pixel of U that is replicated in each region. This figure also illustrates the main downside of this algorithm. Scaling factors of more than 2–3 result in a pixelated bitmap.
U V
Figure 2.14: Bitmap Scaling by Pixel Replication.
2.8 Scaling Bitmaps with Bresenham In this approach, originally proposed by [Kientzle 95] and later improved and extended by [Riemersma 02], we scale a source bitmap S to a destination bitmap D by replicating pixels. No interpolation or other computations are used. Pixels are selected in S and are moved to D (however, the smooth algorithm that is based on this approach computes weighted averages). We first discuss our method for a single row of pixels. Recall that the goal of bitmap scaling is to determine the destination bitmap D, which means that whatever algorithm we develop, it will have to loop over the pixels of D one by one, and for each pixel Di decide which source pixel Sj to move to Di . When scaling a bitmap up (to zoom in or enlarge it), each iteration advances to the next pixel Di+1 in the destination and a source pixel S is moved there, but this source pixel can only be the one used for Di (let’s denote it by Sj ) or its immediate successor Sj+1 . This brings to mind Bresenham’s algorithm for scan converting straight lines (Section 3.4), where x is incremented by 1 in each iteration, and the only decision is whether or not to increment y. This decision depends on the slope of the line, but the slope is a a real number, which is why Bresenham’s algorithm and other DDA methods for lines use instead the two integers Δx and Δy. The simple bitmap scaling algorithm of Figure 2.15 is based on the same principle. In each iteration, the index destPos of the destination bitmap is incremented by 1, but the index srcPos of the source bitmap is
2 Raster Graphics
49
incremented only from time to time, when needed, based on the relative sizes srcWidth and destWidth of the source and destination bitmaps, respectively. When scaling a bitmap down (shrinking), each iteration still advances to the next destination pixel Di+1 , but selecting the source pixel is more complex, because that pixel may be a distant successor Sj+k of Sj . This is why the algorithm of Figure 2.15 has an inner while loop where index srcPos may be incremented several times as needed depending on the relative sizes of the source and destination bitmaps. var srcWidth, destWidth: integer; var srcPos=0, destPos=0, numerator=0: integer; while(destPos < destWidth) dest[destPos]:=src[srcPos]; destPos:=destPos+1; numerator:=numerator+srcWidth; while(numerator > destWidth) numerator:=numerator-destWidth; srcPos=:srcPos+1 endwhile; endwhile; Figure 2.15: Bitmap Scaling Following Bresenham.
Example: Given a source bitmap of five pixels and a destination bitmap of 13 pixels, Table 2.16 lists the 13 iterations needed to scale the source to the destination. Before the main loop starts, five variables are initialized as follows: srcWidth=5, destWidth=13, srcPos=destPos=0, and Num=0. Figure 2.18a illustrates this process graphically.
destPos Num srcPos Pixel moved destPos Num srcPos Pixel moved 1 5 dest[0]←src[0] 8 14,1 3 dest[7]←src[2] 2 10 dest[1]←src[0] 9 6 dest[8]←src[3] 3 15,2 1 dest[2]←src[0] 10 11 dest[9]←src[3] 4 7 dest[3]←src[1] 11 16,3 4 dest[10]←src[3] 5 12 dest[4]←src[1] 12 8 dest[11]←src[4] 6 17,4 2 dest[5]←src[1] 13 13 dest[12]←src[4] 7 9 dest[6]←src[2] Table 2.16: Bitmap Scaling, 5 to 13 Example.
For the opposite scaling, from a 13-pixel row to a 5-pixel row, the initialization is similar and Table 2.17 lists the individual steps (see also Figure 2.18b). This scaling algorithm is simple and fast, but has serious drawbacks. In particular, enlarging a bitmap may result in an irregular, blocky image, while shrinking skips source pixels and thus generates artifacts that are often very noticeable. It is possible to extend the basic algorithm of Figure 2.15 so that instead of skipping pixels it computes their
50
2.8 Scaling Bitmaps with Bresenham destPos 1 2 3 4 5 6
Num 13,8,3 16,11,6,1 14,9,4 17,12,7,2 15,10,5
srcPos 1,2 3,4,5 6,7 8,9,10 11,12
Pixel moved dest[0]←src[0] dest[1]←src[2] dest[2]←src[5] dest[3]←src[7] dest[4]←src[10] dest[5]←src[12]
Table 2.17: Bitmap Scaling, 13 to 5 Example.
(a)
(b)
Figure 2.18: Scaling Example.
(weighted) average and stores it in the current destination pixel. Such an algorithm is listed in Figure 2.19 and it results in smooth shrinking of the source bitmap (albeit at the price of more operations and slower execution). var srcWidth, destWidth, pixelFrac, num: integer; var srcPos=0, destPos=0: integer; pixelFrac:=destWidth; while(destPos < destWidth) p:=0; num:=0; /* Handle whole pixels first */ while(num+pixelFrac ≤ srcWidth) num:=num+pixelFrac; p:=p+pixelFrac × src[srcPos]; srcPos:=srcPos+1 pixelFrac:=destWidth; endwhile if(num<srcWidth) /* Partial pixel? */ p:=p+(srcWidth-num) × src[srcPos]; pixelFrac:=pixelFrac-(srcWidth-num); endif dest[destPos]:=p/srcWidth; destPos:=destPos+1; endwhile; Figure 2.19: Smooth Bitmap Scaling.
2 Raster Graphics
51
The algorithm of Figure 2.19 consists of a main while loop with three parts (1) an inner while for collecting a number of whole source pixels, (2) an if statement to include a partial source pixel, if needed, and (3) an assignment statement to store a weighted average of the collected pixels in a destination pixel. This is best illustrated by an example. Example. We manually execute the algorithm of Figure 2.19 to shrink a row of 13 pixels to a row of five pixels. Table 2.22 lists the five iterations and Figure 2.20 illustrates how each of the five destination pixels is assigned a weighted average of approximately 13/5 = 2.6 source pixels.
Figure 2.20: Scaling Example.
The methods of this section operate on a single row of pixels. To apply this approach to a complete, rectangular bitmap A, we can first scale every row of A to create B, then scale every column of B to obtain C. Figure 2.21 illustrates how this works and also the downside of this simple technique. Scaling the rows of the original bitmap A creates the wider bitmap B with lower visual quality. Scaling B vertically creates the final bitmap C but degrades the image quality even more in the process. Thus, this algorithm performs best if only the rows or the columns, but not both, need be scaled.
A
B
C Figure 2.21: Scaling by Rows and by Columns.
2.8.1 A Slight Variation The variation on Bresenham described in this section is due to Tomas M¨ oller (page 4 of [Kirk 92]) and it is fast because it employs just integers. In fact, it is similar to the DDA methods of Section 3.3. Any DDA method for lines can be used, and the developer selected Bresenham’s algorithm of Section 3.4. The method is illustrated in Figure 2.23, which is a modified version of the original Bresenham algorithm, Figure 3.8. The original algorithm loops from x1 to x2 and draws several pixels at each y value from y1 to y2 . The modified algorithm assumes a source array where each array location from y1 to y2 is copied into several locations, from x1 to x2, of the destination array.
52
2.8 Scaling Bitmaps with Bresenham
Initialization srcWidth=13; destWidth=5; pixelFrac=5; srcPos=destPos=0; p:=0; num:=0; \* Iteration 1 *\ while(num+pixelFrac≤13) num:=0+5=5; p:=p+5×src[0]; srcPos:=1; pixelFrac:=5; num:=5+5=10; p:=p+5×src[1]; srcPos:=2 pixelFrac:=5; endwhile if(10<13) p:=p+[13-10]×src[2]; pixelFrac:=5-(13-10)=2; endif dest[0]:=(5src[0]+5src[1]+3src[2])/13; destPos:=1;
p:=0; num:=0; \* Iteration 4 *\ while(num+pixelFrac≤13) num:=0+1=1; p:=p+1×src[7]; srcPos:=8; pixelFrac:=5; num:=1+5=6; p:=p+5×src[8]; srcPos:=9; pixelFrac:=5; num:=6+5=11; p:=p+5×src[9]; srcPos:=10; pixelFrac:=5; endwhile if(11<13) p:=p+[13-11]×src[10]; pixelFrac:=5-(13-11)=3; endif dest[3]:=(src[7]+5src[8]+5src[9]+2src[10])/13; destPos:=4;
p:=0; num:=0; \* Iteration 2 *\ p:=0; num:=0; \* Iteration 5 *\ while(num+pixelFrac≤13) while(num+pixelFrac≤13) num:=0+2=2; num:=0+3=3; p:=p+2×src[2]; p:=p+3×src[10]; srcPos:=3; srcPos:=11; pixelFrac:=5; pixelFrac:=5; num:=2+5=7; num:=3+5=8; p:=p+5×src[3]; p:=p+5×src[11]; srcPos:=4; srcPos:=12; pixelFrac:=5; pixelFrac:=5; num:=7+5=12; endwhile p:=p+5×src[4]; if(8<13) srcPos:=5; p:=p+[13-8]×src[12]; pixelFrac:=5; pixelFrac:=5-(13-8)=0; endwhile endif if(12<13) dest[3]:=(3src[10]+5src[11]+5src[12])/13; p:=p+[13-12]×src[5]; destPos:=5; pixelFrac:=5-(13-12)=4; endif dest[1]:=(2src[2]+5src[3]+5src[4]+src[5])/13; destPos:=2; p:=0; num:=0; \* Iteration 3 *\ while(num+pixelFrac≤13) num:=0+4=4; p:=p+4×src[5]; srcPos:=6; pixelFrac:=5; num:=4+5=9; p:=p+5×src[6]; srcPos:=7; pixelFrac:=5; endwhile if(9<13) p:=p+[13-9]×src[7]; pixelFrac:=5-(13-9)=1; endif dest[2]:=(4src[5]+5src[6]+4src[7])/13; destPos:=3;
Table 2.22: Smooth Bitmap Shrinking, 13 to 5 Example.
2 Raster Graphics
53
Proc line(source, y1, y2, destin, x1, x2); var y, dx, dy, d: integer; y:=y1; dx:=x2-x1; dy:=y2-y1; d:=2dy-dx; for i:=x1 to x2 do dest(i):= source(y); while d ≥ 0 do y:=y+1; d:=d+dx; endwhile d:=d+2dy endfor Figure 2.23: Stretching an Array.
The reader should compare the pseudo-code above to that of Figure 3.8, but perhaps an example serves as the best illustration. Assume that locations 1 through 3 of the source array S are to be stretched into locations 1 through 10 of the destination array D. Table 2.24 lists the 10 steps of this stretch and shows how the decision variable d swings between positve and negative values to guide the algorithm in its decisions. D(1)← S(1), D(2)← S(1), D(3)← S(1), y ← 2, D(4)← S(2), D(5)← S(2), D(6)← S(2), D(7)← S(2), y ← 3, D(8)← S(3), D(9)← S(3), D(10)← S(3),
d ← −5 + 4 = −1 d ← −1 + 4 = 3 d ← 3 − 18 = −15 d ← −15 + 4 = −11 d ← −11 + 4 = 7 d ← −7 + 4 = −3 d ← −3 + 4 = 1 d ← 1 − 18 = −17 d ← −17 + 4 = −13 d ← −13 + 4 = −11 d ← −11 + 4 = −7 d ← −7 + 4 = −3
Table 2.24: Ten Steps in Stretching an Array.
The developer of this method suggests several applications for it, such as converting a rectangular image to a circular one by stretching the center rows, or converting it to a trapezoid by stretching the bottom rows and shrinking the top rows.
2.9 Pixel Art Scaling
54
2.9 Pixel Art Scaling Digital images are generally classified (Section 23.2) as continuous-tone (or natural) and discrete-tone (or artificial), but this classification is very general. The methods described in this section are intended for line-art images (especially images with thin lines) and for low-resolution images. The main requirements of such a scaling algorithm are (1) to preserve the sharpness of lines and (2) to be simple and fast (which makes it a candidate for use in video games). The Scale2 algorithm [Scale2 10] was developed by Andrea Mazzoleni and has evolved from the 1992 EPX (Eric’s pixel expansion) factor 2 scaling method. The source E0 E1 bitmap is scanned pixel by pixel. A pixel E is expanded into four new pixels E2 E3 by examining its eight nearest neighbors and constructing a 3 × 3 region of the source bitmap A B C D E F G H I.
Each of the four new Ei pixels is assigned the value of one of the nine source pixels as shown in the pseudocode of Figure 2.25. if ((B = H) and (D = F)) then E0:=if (D = B) then D else E1:=if (B = F) then F else E2:=if (D = H) then D else E3:=if (H = F) then F else E0:=E1:=E2:=E3:=E endif
E E E E
endif; endif; endif; endif; else
Figure 2.25: The Scale2 Algorithm.
When E is on the border of the source bitmap, any missing pixels in the 3 × 3 region are replaced by their nearest neighbors. Thus, if E is on the left border, the region becomes B B C E E F H H I.
The Scale3 algorithm [Scale2 10] is similar. It expands a source pixel E to nine pixels Ei as listed in Figure 2.26 by examining neighbor pixels in the same 3 × 3 region and assigning each new pixel the value of one of the source pixels. No computations are required. A B C D E F G H I.
E0 E1 E2 E3 E4 E5 E6 E7 E8.
The Scale4 algorithm is performed by executing Scale2 twice. These are simple, fast algorithms, but when applied to the right type of image, they produce excellent results. Figure 2.27 illustrates a few examples of how Scale2 performs on a few simple, monochromatic items.
2 Raster Graphics if ((B = H) and (D = F)) then E0:=if (D = B) then D else E endif; E1:=if ((D = B) and (E = C)) or ((B = F) and E2:=if (B = F) then F else E endif; E3:=if ((D = B) and (E = G)) or ((D = H) and E4:=E; E5:=if ((B = F) and (E = I)) or ((H = F) and E6:=if (D = H) then D else E endif; E7:=if ((D = H) and (E = I)) or ((H = F) and E8:=if (H = F) then F else E endif; else E0:=E1:=E2:=E3:=E4:=E5:=E6:=E7:=E8:=E endif
55
(E = A)) then B else E endif; (E = A)) then D else E endif; (E = C)) then F else E endif; (E = G)) then H else E endif;
Figure 2.26: The Scale3 Algorithm.
Figure 2.27: Examples of the Scale2 Algorithm.
The Eagle algorithm [eagle 10] adopts the same principle as Scale2. Given a 3 × 3 region of pixels centered on the current pixel E, the algorithm scales the source bitmap by generating four new pixels Ei, each a copy of one of the nine source pixels in the region. In step 1, each Ei is set to E. In step 2, the algorithm performs comparisons and assignments as shown in Figure 2.28. E0:=E1:=E2:=E3:=E; if (A = B = D) then E0:=A if (B = C = F) then E1:=C if (D = G = H) then E2:=G if (F = I = H) then E3:=I
endif; endif; endif; endif;
Figure 2.28: The Eagle Scaling Algorithm.
The hqnx method is a family of algorithms hq2x, hq3x, and hq4x by Maxim Stepin. They scale a bitmap by factors of 2, 3, and 4, respectively. A reference is [hq3x 10], which includes a source code, but is skimpy on details.
56
2.10 Pixel Interpolation
A 3 × 3 region around the current pixel is examined. For each of the eight near neighbors, the color difference between it and the current pixel is computed and is compared to a predetermined threshold. If the difference is less than the threshold, the difference is considered “close” and a zero bit is generated; otherwise, the difference is “distant” and a 1-bit is generated. The eight bits thus generated become an 8-bit integer that is used as an index to a table of 256 entries. Each entry in this table describes how to combine the colors of the nine source pixels to obtain four, nine, or 16 target pixels. This table is the main part of the algorithm. For each index (each combination of eight close and distant color differences) the corresponding table entry has to specify the most probable target pixels (keeping in mind edges in the source bitmap). Tests show that the hqnx methods preform very well for images with clear sharp edges, such as line graphics or cartoons, but produce poor results for natural images. Better scaling techniques are based on interpolation, so we next discuss this important concept in some details.
2.10 Pixel Interpolation Interpolation is the process of estimating the intermediate values of a continuous event from discrete samples (see also Section 2.4). This section starts with an intuitive explanation of interpolation (see also the beginning of Chapter 10), and then shows how to apply it to the bilinear and bicubic cases. We start with an intuitive discussion of the term interpolation. Given two numbers a and b, their average (a+b)/2 is always located midway between them, so we can use the average to interpolate them. However, given four numbers a, b, c, and d, their average (a + b + c + d)/4 is not a good interpolation, because it is not located “midway” between the four. A simple example is the four numbers 1, 1, 1, and 100. Their average is close to 25 and so does not seem to be “in the middle” of the four numbers. Interpolating four numbers is therefore done by (1) converting the numbers to two-dimensional points, (2) calculating a smooth, polynomial curve that passes through the points, and (3) finding the midpoint of the curve. Any numbers a, b, c, and d can be converted to the points (1, a), (2, b), (3, c), and (4, d). It is intuitively clear that the midpoint (x, y) of a smooth curve that passes through those points is a good candidate for the title “the interpolation of the four points.” The y coordinate of the midpoint becomes the interpolation of the four numbers, and the x coordinate is disregarded. This method is called one-dimensional interpolation. It can be extended to more than four numbers, and also to pixels, where it becomes two-dimensional interpolation. As mentioned in Section 2.12.3, the principle of bicubic interpolation is to use a group of 16 neighboring pixels to predict the value of a pixel at the center of the group. The main idea is to consider the 16 neighbor pixels a set of 4 × 4 equally-spaced points on a surface (where the value of a pixel is interpreted as the height of the surface) and to derive a polynomial function P(u, w) that passes through all 16 points. Graphically, P(u, w) can be thought of as a surface. The value of the pixel at the center of the 4×4 group can then be predicted by computing the height of the center point P(0.5, 0.5) of the surface.
2 Raster Graphics
57
Mathematically, this surface is the two-dimensional polynomial interpolation of the 16 points.
2.11 Bilinear Interpolation This type of interpolation is similar to the construction of bilinear surfaces. Thus, the reader is advised to review Section 9.3 before continuing. Given enough pixels of an image and given a point Q anywhere in the image, we use bilinear interpolation to compute the intensity (grayscale) of the image at Q in the following steps: 1. We select the four pixels surrounding Q, denote their values by a, b, c, and d (Figure 2.29), and convert them to three-dimensional points by considering the value of a pixel the z coordinate of the point and adding appropriate x and y coordinates. We end up with the points P00 = (0, 0, a), P01 = (0, 1, b), P11 = (1, 1, c), and P10 = (1, 0, d). 2. We compute the parametric equations of the straight segment L1 (u) running from P00 to P10 and the segment L2 (u) running from P01 to P11 . These equations are the simple linear interpolations L1 (u) = P00 (1−u)+P10 u and L2 (u) = P01 (1−u)+P11 u. 3. We compute the parametric equation Pu,w of the bilinear surface generated by interpolating segments L1 (u) and L2 (u). The equation is P (u, w) = L1 (u)(1−w)+L2 (u)w = P00 (1−u)(1−w)+P10 u(1−w)+P01 (1−u)w+P11 uw. Figure 2.29 is an example of such a surface. The entire surface is spanned when parameters u and w are varied independently in the interval [0, 1]. 4. We determine the values of u and w for the given point Q and compute its intensity as the z coordinate of surface P (u, w) at these values. b
L
2
(u
) d
b•
a•
Q•
z
•c (u L1
•d
a
Figure 2.29: A Bilinear Surface.
c
)
y x
2.12 Interpolating Polynomials
58
2.12 Interpolating Polynomials This section shows how to predict the value of a pixel from those of 16 of its near neighbors by means of a two-dimensional interpolating polynomial.
2.12.1 One-Dimensional Interpolation A surface can be viewed as an extension of a curve, so we start by deriving a onedimensional polynomial (a curve) that interpolates four points, then extend it to a two-dimensional polynomial (a surface) that interpolates a grid of 4 × 4 points. Given four points P1 , P2 , P3 , and P4 we look for a polynomial that will pass through them. In general, a polynomial of degree n in x is defined as the function Pn (x) =
n
ai xi = a0 + a1 x + a2 x2 + · · · + an xn ,
(2.2)
i=0
where ai are the n+1 coefficients of the polynomial and the parameter x is a real number. The one-dimensional interpolating polynomial that is of interest to us is special, and differs from the definition above in two respects as follows: 1. This polynomial goes from point P1 to point P4 . Its length is finite, and it is therefore better to describe it as the function Pn (t) =
n
ai ti = a0 + a1 t + a2 t2 + · · · + an tn ; where 0 ≤ t ≤ 1.
i=0
This is the parametric representation of a polynomial. We want this polynomial to go from P1 to P4 when the parameter t is varied from 0 to 1. 2. The only given data are the four points and we have to use them to calculate all n + 1 coefficients of the polynomial. This suggests the value n = 3 (a polynomial of degree 3, a cubic polynomial; one that has four coefficients). The idea is to set up and solve four equations, with the four coefficients as the unknowns, and with the four points as known quantities. Thus, we use the notation (T indicates transpose) P(t) = at3 + bt2 + ct + d = (t3 , t2 , t, 1)(a, b, c, d)T = T(t) · A.
(2.3)
The four coefficients a, b, c, d are shown in boldface because they are not numbers. Keep in mind that the polynomial has to pass through the given points, so the value of P(t) for any t must be the three coordinates of a point. Each coefficient should therefore be a triplet. T(t) is the row vector (t3 , t2 , t, 1), and A is the column vector (a, b, c, d)T . Computing the curve therefore involves finding the values of the four unknowns a, b, c, and d. P(t) is called a parametric cubic (or PC) polynomial (Section 8.8). It turns out that degree 3 is the smallest one that is still useful for an interpolating polynomial. A polynomial of degree 1 has the form P1 (t) = ct + d and is therefore a straight line, so it can be used only in special cases. A polynomial of degree 2 (quadratic) has the form P2 (t) = bt2 +ct+d and is a conic section, so it can take only a few different shapes. A polynomial of degree 3 (cubic) is therefore the simplest one that can take on complex shapes, and can also be a space curve.
2 Raster Graphics
59
Exercise 2.6: Show that a quadratic polynomial must be a plane curve. Our ultimate problem is to interpolate pixels. Pixels are always spaced uniformly, so we assume that the two interior points P2 and P3 are equally spaced between P1 and P4 . The first point P1 is the start point P(0) of the polynomial, the last point P4 is the endpoint P(1), and the two interior points P2 and P3 are the two equally-spaced interior points P(1/3) and P(2/3) of the polynomial. We therefore write P(0) = P1 , P(1/3) = P2 , P(2/3) = P3 , P(1) = P4 , or a(0)3 + b(0)2 + c(0) + d = P1 , 3
a(1/3) + b(1/3)2 + c(1/3) + d = P2 , a(2/3)3 + b(2/3)2 + c(2/3) + d = P3 , a(1)3 + b(1)2 + c(1) + d = P4 . These equations are easy to solve, and the solutions are: a = −9/2P1 + 27/2P2 − 27/2P3 + 9/2P4 , b = 9P1 − 45/2P2 + 18P3 − 9/2P4 , c = −11/2P1 + 9P2 − 9/2P3 + P4 , d = P1 . Substituting into Equation (2.3) gives P(t) =(−9/2P1 + 27/2P2 − 27/2P3 + 9/2P4 )t3 + (9P1 − 45/2P2 + 18P3 − 9/2P4 )t2 + (−11/2P1 + 9P2 − 9/2P3 + P4 )t + P1 , which, after rearranging, becomes P(t) =(−4.5t3 + 9t2 − 5.5t + 1)P1 + (13.5t3 − 22.5t2 + 9t)P2 + (−13.5t3 + 18t2 − 4.5t)P3 + (4.5t3 − 4.5t2 + t)P4 =G1 (t)P1 + G2 (t)P2 + G3 (t)P3 + G4 (t)P4 =G(t) · P,
(2.4)
where G1 (t) = (−4.5t3 + 9t2 − 5.5t + 1),
G2 (t) = (13.5t3 − 22.5t2 + 9t),
G3 (t) = (−13.5t3 + 18t2 − 4.5t),
G4 (t) = (4.5t3 − 4.5t2 + t);
(2.5)
P is the column (P1 , P2 , P3 , P4 )T , and G(t) is the row vector
G1 (t), G2 (t), G3 (t), G4 (t) . The functions Gi (t) are called blending functions, because they create any point on the curve as a blend of the four given points. Note that they add up to 1 for any value of
2.12 Interpolating Polynomials
60
t. This property must be satisfied by any set of blending functions, and such functions are called barycentric. We can also write G1 (t) = (t3 , t2 , t, 1)(−4.5, 9, −5.5, 1)T and, similarly, for G2 (t), G3 (t), and G4 (t). In matrix notation this becomes ⎞ −4.5 13.5 −13.5 4.5 18 −4.5 ⎟ ⎜ 9.0 −22.5 G(t) = (t3 , t2 , t, 1) ⎝ ⎠ = T(t) · N. −5.5 9.0 −4.5 1.0 1.0 0 0 0 ⎛
(2.6)
The curve can now be written P(t) = G(t) · P = T(t)·N·P. N is called the basis matrix, and P is the geometry vector. From Equation (2.3) we know that P(t) = T(t) · A, so we can write A = N · P. The word barycentric is derived from barycenter, meaning “center of gravity,” because such weights are used to calculate the center of gravity of an object. Barycentric weights have many uses in geometry in general and in curve and surface design in particular. —From Dr. Dobbs Journal, March 2000. Given the four points, the interpolating polynomial can be computed in two steps: 1. Set-up the equation A = N · P and solve it for A = (a, b, c, d)T . 2. The polynomial is P(t) = T(t) · A.
2.12.2 Example (This example is in two dimensions, each of the four points Pi and each of the four coefficients a, b, c, and d is a pair. For three-dimensional curves, the method is the same, except that triplets should be used instead of pairs.) Given the four two-dimensional points P1 = (0, 0), P2 = (1, 0), P3 = (1, 1), and P4 = (0, 1), we set up the equation ⎛ ⎞ ⎛ ⎞⎛ ⎞ a −4.5 13.5 −13.5 4.5 (0, 0) 18 −4.5 ⎟ ⎜ (1, 0) ⎟ ⎜b⎟ ⎜ 9.0 −22.5 ⎝ ⎠=A=N·P=⎝ ⎠⎝ ⎠, c −5.5 9.0 −4.5 1.0 (1, 1) d 1.0 0 0 0 (0, 1) which is easy to solve a = −4.5(0, 0) + 13.5(1, 0) − 13.5(1, 1) + 4.5(0, 1) = (0, −9), b = 19(0, 0) − 22.5(1, 0) + 18(1, 1) − 4.5(0, 1) = (−4.5, 13.5), c = −5.5(0, 0) + 9(1, 0) − 4.5(1, 1) + 1(0, 1) = (4.5, −3.5), d = 1(0, 0) − 0(1, 0) + 0(1, 1) − 0(0, 1) = (0, 0). Thus
P(t) = T · A = (0, −9)t3 + (−4.5, 13.5)t2 + (4.5, −3.5)t.
2 Raster Graphics
61
It is now easy to calculate and verify that P(0) = (0, 0) = P1 , and P(1/3) = (0, −9)1/27 + (−4.5, 13.5)1/9 + (4.5, −3.5)1/3 = (1, 0) = P2 , P(1) = (0, −9)13 + (−4.5, 13.5)12 + (4.5, −3.5)1 = (0, 1) = P4 . Exercise 2.7: Calculate P(2/3) and verify that it is equal to P3 . Exercise 2.8: Imagine the circular arc of radius 1 in the first quadrant (a quarter circle). Write the coordinates of the four points that are equally spaced on this arc. Use the coordinates to calculate a PC interpolating polynomial approximating this arc. Calculate point P(1/2). How far does it deviate from the midpoint of the true quarter circle? The main advantage of this method is its simplicity. Given the four points, it is easy to determine the PC polynomial that passes through them. Exercise 2.9: This method makes sense if the four points are (at least approximately) equally spaced along the curve. If they are not equally spaced, the following may be done: Instead of using 1/3 and 2/3 as the intermediate values, the user may specify values α, β such that P2 = P(α) and P3 = P(β). Generalize Equation (2.6) such that it depends on α and β.
2.12.3 Two-Dimensional Interpolation The PC polynomial, Equation (2.3), can easily be extended to two dimensions by means of a technique called Cartesian product. The polynomial is generalized from a cubic curve to a bicubic surface (Section 10.6). 3 A one-dimensional PC polynomial has the form P(t) = i=0 ai ti . Two such curves, P(u) and P(w), can be combined by means of this technique to form the surface: P(u, w) =
3 3
aij ui w j
i=0 j=0
= a33 u3 w3 + a32 u3 w2 + a31 u3 w + a30 u3 + a23 u2 w3 + a22 u2 w 2 + a21 u2 w + a20 u2 + a13 uw3 + a12 uw2 + a11 uw + a10 u + a03 w3 + a02 w 2 + a01 w + a00 ⎛ ⎞⎛ 3 ⎞ a33 a32 a31 a30 w ⎜ a23 a22 a21 a20 ⎟ ⎜ w 2 ⎟ 3 2 (2.7) = (u , u , u, 1) ⎝ ⎠ , where 0 ≤ u, w ≤ 1. ⎠⎝ a13 a12 a11 a10 w 1 a03 a02 a01 a00 This is a double cubic polynomial (hence the name bicubic) with 16 terms, where each of the 16 coefficients aij is a triplet. Note that the surface depends on all 16 coefficients. Any change in any of them produces a different surface. Equation (2.7) is the algebraic representation of a bicubic surface. In order to use it in practice, the
2.12 Interpolating Polynomials
62
16 unknown coefficients have to be expressed in terms of the 16 known, equally-spaced points. We denote these points by P03 P02 P01 P00
P13 P12 P11 P10
P23 P22 P21 P20
P33 P32 P31 P30 .
To determine the 16 unknown coefficients, we write 16 equations, each based on one of the given points: P(0, 0) = P00 P(1/3, 0) = P10 P(2/3, 0) = P20 P(1, 0) = P30
P(0, 1/3) = P01 P(1/3, 1/3) = P11 P(2/3, 1/3) = P21 P(1, 1/3) = P31
P(0, 2/3) = P02 P(1/3, 2/3) = P12 P(2/3, 2/3) = P22 P(1, 2/3) = P32
P(0, 1) = P03 P(1/3, 1) = P13 P(2/3, 1) = P23 P(1, 1) = P33 .
Solving, substituting the solutions in Equation (2.7), and simplifying produces the geometric representation of the bicubic surface ⎛
P33 ⎜ P23 3 2 P(u, w) = (u , u , u, 1)N ⎝ P13 P03
P32 P22 P12 P02
P31 P21 P11 P01
⎞ ⎛ 3⎞ P30 w P20 ⎟ T ⎜ w 2 ⎟ ⎠, ⎠N ⎝ P10 w 1 P00
(2.8)
where N is the Hermite matrix of Equation (2.6). The surface of Equation (2.8) can now be used to predict the value of a pixel as a polynomial interpolation of 16 of its near neighbors. All that is necessary is to substitute u = 0.5 and w = 0.5. The following Mathematica code Clear[Nh,P,U,W]; Nh={{-4.5,13.5,-13.5,4.5},{9,-22.5,18,-4.5}, {-5.5,9,-4.5,1},{1,0,0,0}}; P={{p33,p32,p31,p30},{p23,p22,p21,p20}, {p13,p12,p11,p10},{p03,p02,p01,p00}}; U={u^3,u^2,u,1}; W={w^3,w^2,w,1}; u:=0.5; w:=0.5; Expand[U.Nh.P.Transpose[Nh].W] does that and produces P(0.5, 0.5) = 0.00390625P00 − 0.0351563P01 − 0.0351563P02 + 0.00390625P03 − 0.0351563P10 + 0.316406P11 + 0.316406P12 − 0.0351563P13 − 0.0351563P20 + 0.316406P21 + 0.316406P22 − 0.0351563P23 + 0.00390625P30 − 0.0351563P31 − 0.0351563P32 + 0.00390625P33 ,
2 Raster Graphics
63
where the 16 coefficients are the ones used in Table 2.30. Table 2.30 shows the 16 weights that should be used when we want to predict a pixel by a weighted average of 16 of its nearest neighbors. These weights are calculated by bicubic polynomial interpolation and are normalized such that they add up to 1. (Notice that in Table 2.30a the weights are not normalized; they add up to 256. When these integer weights are used, the weighted sum should be divided by 256.) To predict a pixel near an edge of the image, where some of the 16 neighbors do not exist, only those neighbors that exist are used, and their weights are renormalized, to bring their sum to 1. 1 −9 −9 −9 81 −9 1 81 81 1 −9 81 −9 −9 −9 1
0.0039 −0.0351 −0.0351 −0.0351 0.3164 −0.0351 0.0039 0.3164 0.3164 0.0039 −0.0351 0.3164 −0.0351 −0.0351 −0.0351 0.0039
Table 2.30: 16 Weights. (a) Integers. (b) Normalized.
Exercise 2.10: Why do the weights have to add up to 1? Exercise 2.11: How can this method be used in cases where not all 16 points are known? Exercise 2.12: The center point of the surface is computed as a weighted sum of the 16 equally-spaced data points. It makes sense to assign small weights to points located away from the center, but our result assigns negative weights to eight of the 16 points. Explain the meaning of negative weights and show what role they play in interpolating the center of the surface. Readers who find it difficult to follow the details above should compare the way two-dimensional polynomial interpolation is presented here to the way it is discussed by [Press et al. 88]. The following quotation is from page 125: “. . . The formulas that obtain the c’s from the function and derivative values are just a complicated linear transformation, with coefficients which, having been determined once, in the mists of numerical history, can be tabulated and forgotten.” If high-quality results are important, the new pixels should be calculated from the original ones, not simply copied. To expand an image with scale factors of, say, 3×4, we need to create 3 × 4 = 12 new pixels for every original pixel. Let P [i, j] represent the value of the pixel at row i and column j of the original image. The 12 new pixels generated from P [i, j] should be computed by interpolating the values of P [i, j] and its three near neighbors. The neighbor on the right of P [i, j] is P [i, j + 1] and the two neighbors below it are P [i+1, j] and P [i+1, j +1]. Each of the 12 new pixels is computed as a weighted average of these four pixels, where the weights (which add up to 1) are given by Table 2.31. The table consists of 12 subtables, each of size 2 × 2. They are
2.12 Interpolating Polynomials
64
arranged in three columns and four rows. Each entry in the table equals the product of the weights of its row and column. The four entries of each subtable add up to one. Each of the 12 new pixels is calculated as a weighted sum of the 4 pixels P [i, j], P [i + 1, j],
P [i, j + 1], P [i + 1, j + 1],
where the weights are taken from the corresponding subtable.
1: 0: 3/4: 1/4: 1/2: 1/2: 1/4: 3/4:
1
0
2/3
1/3
1/3
2/3
1 0 3/4 · 1 1/4 · 1 1/2 · 1 1/2 · 1 1/4 · 1 3/4 · 1
0 0 3/4 · 0 1/4 · 0 1/2 · 0 1/2 · 0 1/4 · 0 3/4 · 0
2/3 0 3/4 · 2/3 1/4 · 2/3 1/2 · 2/3 1/2 · 2/3 1/4 · 2/3 3/4 · 2/3
1/3 0 3/4 · 1/3 1/4 · 1/3 1/2 · 1/3 1/2 · 1/3 1/4 · 1/3 3/4 · 1/3
1/3 0 3/4 · 1/3 1/4 · 1/3 1/2 · 1/3 1/2 · 1/3 1/4 · 1/3 3/4 · 1/3
2/3 0 3/4 · 2/3 1/4 · 2/3 1/2 · 2/3 1/2 · 2/3 1/4 · 2/3 3/4 · 2/3
Table 2.31: Interpolation Weights for 3 × 4 Scaling.
The first of the 12 new pixels should thus be 1 × P [i, j] + 0 × P [i, j + 1] + 0 × P [i + 1, j] + 0 × P [i + 1, j + 1]. The last one is 1 2 3 6 P [i, j] + P [i, j + 1] + P [i + 1, j] + P [i + 1, j + 1]. 12 12 12 12 To shrink an image by factors of n×m, we have to replace a group of n × m original pixels by a single pixel. This pixel is simply computed as an average of the original n × m pixels. An important point that should be mentioned is that the values stored in the bitmap are not always the intensities or colors of the pixels. Often the bitmap contains pointers to a color lookup table. In such a case, the interpolation shown in Table 2.31 should be performed on the pixel values in the lookup table, not on the pointers in the bitmap. If the scale factors are not integers, then a simple, fast algorithm can be developed to determine how the new pixels depend on the original ones. We present the basic ideas assuming that an image is to be expanded by copying each original pixel several times. When we think of expanding an image, we normally imagine it to be stretched from all sides until it fits the new, larger space. To understand the algorithm, we should think of the expansion process as starting with a grid consisting of a few, large pixels and
2 Raster Graphics
65
creating another grid with the same area as the original, but with more, smaller pixels. Figure 2.32a shows an example of an original 4 × 4 image (whose pixels are numbered 1 through 16) and an expanded image of 10 × 10 pixels superimposed on it. Both are the same size, which suggests that each original pixel has to be copied 10/4 = 2.5 times. In practice, a pixel will be copied either two or three times to obtain the required scaling. Our algorithm uses the quantity 4/10 = 0.4 to determine how many copies of a pixel to generate. A variable diff (for differential) is set to zero and is incremented by 0.4 each time a pixel is copied. Each time diff crosses an integer boundary, the algorithm moves to the next original pixel and starts copying it. The process is summarized in Table 2.33. It shows how pixels 1 and 3 are copied three times each, and pixels 2 and 4, twice each. Figure 2.32b shows the final 100 pixels. The discussion above assumes an expansion, but this method also works for shrinking, where the scale factor is less than 1. Shrinking a 10 × 10 image to, for example, 4× 4 is done by incrementing diff by 10/4 = 2.5. The results are summarized in Table 2.34. Exercise 2.13: Show the original 10 × 10 image (with pixels numbered 1 to 100) and the final 4 × 4 image using a diagram similar to Figure 2.32b. In general, diff is incremented by the inverse of the scale factor, but the main point is that both diff and the amount it is incremented by should be integers, since floatingpoint operations are considerably slower than the same operations on integers. We therefore declare diff an integer and increment it by the rounded value of “1000/scale factor.” Each time diff crosses another multiple of 1000, we move to the next pixel in the original image and start replicating it. Figure 2.35 is a pseudo-code algorithm for scaling one scan line. It copies pixels from the original bitmap P[i,j] to a scaled bitmap Q[x,y] where each scan line is N pixels wide. The values of i and x don’t change in this example. The code of Figure 2.35 can be extended to loop over scan lines. Another pair of diff and accum_diff variables is needed, since the scaling factors in the horizontal and vertical directions may be different. The loop shown in Figure 2.35 now becomes the inner loop of a double-loop algorithm whose outer loop goes over the scan lines. Once the inner loop creates the first scan line Q[1,y], the outer loop copies it as many times as necessary, using the vertical differential. It should be mentioned that this algorithm, which uses integers and a differential, is called a DDA algorithm (for Digital Differential Analyzer, or Discrete Differential Analyzer). Section 3.3 discusses DDA algorithms for scan-converting lines.
2.13 Adaptive Scaling by 2 Section 2.8 shows how to apply the principles of Bresenham’s line algorithm to bitmap scaling. The method of that section is simple and fast. It also allows for any rational scale factors, not just integers. Unfortunately, for large scale factors (for both enlarging and reducing), the method produces poor results. The methods described in this section are limited to scaling by 2, but produce results comparable with bilinear and cubic interpolations. Thus, if a bitmap has to be scaled by a large rational factor, an ideal solution may be to first scale it by 2, and then by a small non-integer factor.
2.13 Adaptive Scaling by 2
66
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
1 1 1 2 2 3 3 3 4 4 1 1 1 2 2 3 3 3 4 4 1 1 1 2 2 3 3 3 4 4 5 5 5 6 6 7 7 7 8 8 5 5 5 6 6 7 7 7 8 8 9 9 9 10 10 11 11 11 12 12 9 9 9 10 10 11 11 11 12 12 9 9 9 10 10 11 11 11 12 12 13 13 13 14 14 15 15 15 16 16 13 13 13 14 14 15 15 15 16 16
(a)
(b) Figure 2.32: Bitmap Scaling.
diff: 0 0.4 0.8 1.2 1.6 2.0 2.4 2.8 3.2 3.6 Pixel: 1 1 1 2 2 3 3 3 4 4 Table 2.33: A 10/4 Expansion by Copying.
diff: 0 2.5 5 7.5 Pixel: 1 3 6 8 Table 2.34: A 4/10 Shrinking by Copying.
Isn’t life a series of images that change as they repeat themselves? —Andy Warhol.
diff:=round(1000/scale); accum_diff:=0; j:=1; for y:=1 to N do Q[x,y]:=P[i,j]; a:=accum_diff/1000; accum_diff:=accum_diff+diff; b:=accum_diff/1000; j:=j+(b-a); endfor; Figure 2.35: Scaling One Scan Line.
2 Raster Graphics
67
Gray level
Shrinking a bitmap by a factor of 2 is simple. Just replace a group of 2 × 2 pixels with its average (this is often referred to as a box filter). Section 2.13.2 discusses color averaging and presents an operator Av(A,B) that computes the average of two colors using only logical operations and a shift, thereby avoiding any overflows. Given four colors A, B, C, and D, this operator can be employed twice Av(Av(A,B),Av(C,D)) to compute their average. Intuitively, enlarging a bitmap seems more complex than reducing it, and we present two adaptive algorithms for this task. The term adaptive signifies that these algorithms perform different operations for different pixels, depending on the content of the bitmap in the vicinity of the pixel. The main idea is to detect edges in the image stored in the bitmap, and to split a source pixel into four target pixels such that sharp edges remain sharp. An image is two dimensional and it is quite difficult to identify an image edge in the bitmap. It is also not immediately clear how to keep such an edge sharp and well defined after the image is enlarged. Therefore, we start with a one-dimensional example. Figure 2.36a shows two source pixels x1 and x2 . Several target pixels y have to be placed between these two and have values assigned. It is clear from the figure that even though pixel y is closer to x2 , the grayscale value assigned to y should be closer to that of x1 , because the slope of the grayscale curve around x2 is higher. In two dimensions, the term gradient is used instead of slope, so any adaptive scaling method should check several pairs of neighbor pixels to determine the gradients around the current pixel (in two dimensions, there may be several gradients), and assign colors to the new target pixels according to the smallest gradient.
x1 y (a)
x1
x4
x7
x2
y1 y3 x5 y2 y4
x8
x3
x6
x9
x2 (b) Figure 2.36: A 3 × 3 Scaling Filter.
The first algorithm described here is due to [Carrato and Tenze 00] and the second algorithm is the work of [Riemersma 06]. The Carrato and Tenze algorithm scans the source bitmap pixel by pixel and splits each pixel into four target pixels. The algorithm examines the eight nearest neighbors of the current pixel (it looks at a 3 × 3 grid of nine pixels centered on the current pixel) and uses their colors to compute the four new target pixels. Each of the four new pixels is first set to the current pixel, and then a weighted sum is added to that, a sum that depends on five of the eight near neighbors. The sum includes differences and averages of the five neighbors and it depends on four weights qj . The details of the algorithm
2.13 Adaptive Scaling by 2
68
are in how to determine the weights, and this is done by a complex process where each weight is (again) expressed as a weighted sum of nine differences of neighbor pixels, each weighted by a parameter pij . Thus, the total number of parameters is 4 × 9 = 36 and these 36 parameters have been computed once by an optimization process where 25 different patterns of six pixels each were used as training patterns. Figure 2.36b shows a typical 3 × 3 grid of eight neighbor pixels centered on the current source pixel x5 . This pixel becomes the source of four target pixels yi , also shown in the figure. Each of the target pixels is determined by x5 and by five more pixels. The five pixels for y1 are shown surrounded by a red frame. The main step is to compute the three differences d1 = x1 − x5 ,
d2 = (x2 + x4 )/2 − x5 ,
d3 = (x3 + x7 )/2 − x5 ,
and compute y1 as a weighted sum added to x5 y1 = x5 +
3
wk dk .
k=1
It is obvious that images can be very complex and it is impossible to select a set of only three weights wi that will produce satisfactory results in all cases. Thus, the developers went through two more steps. They replaced the three wi s with a set of four weights qj and then expressed each weight in terms of nine parameters pij , as follows: 3
y1 = x5 +
1 qk dk , q4 + k=1
where the four qk are weights to be determined and is a small number, used as safety in case q4 turns out to be zero ( was set to 1 in the tests performed by the algorithm developers). In order to assign fixed values to the four weights, the developers of this algorithm expressed each weight as another weighted sum of differences of pixels (Figure 2.37), with nine weights pij as follows: qj = p1j (x1 − x5 )2 + p2j (x2 − x4 )2 + p3j (x3 − x7 )2 + p4j (x1 − x2 )2 + (x1 − x4 )2 + p5j (x1 − x3 )2 + (x1 − x7 )2 + p6j (x2 − x5 )2 + (x4 − x5 )2 + p7j (x3 − x5 )2 + (x7 − x5 )2 + p8j (x3 − x2 )2 + (x7 − x4 )2 + p9j (x3 − x4 )2 + (x7 − x2 )2 , for j = 1 . . . 4. This has increased the size of the problem, because now we have to determine the values of 4 × 9 = 36 parameters pij , but the point is that it is possible to assign these values in a precise and well-understood mathematical process that takes into account the ideal values that should be assigned to y1 in 25 different situations. The developers have selected 25 6-pixel patterns (Figure 2.38), of which 20 patterns contain pixels of only three different colors (they represent sharp edges, common in
2 Raster Graphics
69
Figure 2.37: Pixel Differences for Nine Parameters.
Figure 2.38: Patterns of Six Pixels.
artificial images) and the remaining five contain more colors (they represent smooth edges, common in natural images). These are used as training patterns to compute the best values of the 36 parameters. The “training” is done as follows. For each pattern, it is easy to determine the ideal target pixel y manually, just by looking at the six source pixels x. Figure 2.39 illustrates two simple examples of this process. In part (a) of the figure, three of the six source pixels are dark (we denote their grayscale by a) and three are bright (grayscale b). This pattern corresponds to an ideal 45◦ edge in the image (shown in red), and it is easy to figure out that the best grayscale values for the four target pixels are y1 = (a + b)/2 and y2 = y3 = y4 = b. Exercise 2.14: Explain why the ideal value for y1 is the average (a + b)/2.
Exercise 2.15: What are the ideal values for the four target pixels in the case of Figure 2.39b? Once the ideal target pixel values have been determined for all 25 patterns, training continues with an optimization process that determines the set of 36 parameters that produce the best values for the target pixels for all 25 patterns. Those interested in the details of such optimization methods should consult references [Quasi-Newton 10] and
2.13 Adaptive Scaling by 2
70
y1 y3
y1 y3
y2 y4
y2 y4
(a)
(b) Figure 2.39: Two Edges.
[BFGS 10]. The result obtained by the developers was the set p1j p2j p3j p4j
= 14.1, 112.4, 0.2, −19.5, 5.0, 19.2, −2.9, −29.5, −7.7, = 64.4, −81.0, 158.2, 24.0, 38.3, 70.5, 59.6, 65.6, 22.5, = 61.5, 94.9, −6.5, 45.6, 31.2, 35.8, 55.9, −52.3, −52.5, = 97.5, 96.3, −3.9, 159.5, 115.2, 59.6, 117.9, 132.4, 96.4.
With this set and with = 1, the algorithm computes the four target pixels for each of the 25 pattern with a mean square error (MSE) of only 0.0053. Tests of this method show that it produces results at least as good as bilinear and bicubic interpolations. When the source image contains long, well-defined edges (such as the brim of the hat and the eyes in the well-known Lena test image, Figure 24.40), this algorithm produces smooth, enlarged edges, while interpolation methods that are not edge sensitive tend to produce stair-step edges. In addition, the method is simple and fast, but it is limited to a scale factor of 2. A faster version of the algorithm is obtained if the parameters are replaced with the powers of 2 that are closest to the original values (such as 64 instead of 59.6) and by replacing small parameter values with zeros. The second algorithm of this section is due to [Riemersma 06]. It is similar but simpler. The principle is to consider only the three neighbors x1 , x2 , and x4 of each source pixel x5 (Figure 2.36b), to compute four gradients (each as the color difference between x5 and one of its three neighbors), select the smallest gradient, and use it to compute y1 as the color average of x5 and one or two of its neighbors. The algorithm is listed in Figure 2.40. There is a point in the algorithm that deserves attention, namely how to determine the color difference between pixels and how to compute the average of two or three given colors. For grayscale images, each pixel is a number, which makes it easy to compute differences and averages. For color images, where a pixel consists of three color components, each normally a byte, there are several ways to compute the distance between colors as follows: 2
Compute the Euclidean distance R1 − R22 , G21 − G22 , B12 − B22 . This is too slow for any fast algorithms.
2 Raster Graphics
71
d1:=|x_5-x_2|; d2:=|x_5-x_4|; d3:=|x_5-x_1|; d4:=|x_5-(x_2+x_4)/2|; min:=minimum(d1,d2,d3,d4); if(min=d1) then y_1:=(x_5+x_2)/2 else if(min=d2) then y_1:=(x_5+x_4)/2 else if(min=d3) then y_1:=(x_5+x_1)/2 else if(min=d4) then y_1:=x_5/2+x_2/4+x_4/4; Figure 2.40: A Simpler Scaling Algorithm.
Convert each pixel to grayscale and subtract the grayscale values. This may also be too slow. Compute the three differences |R1 − R2 |, |G1 − G2 |, and |B1 − B2 | and select the largest. Interleave the bits of the color components as shown in Section 2.13.1 and subtract the resulting 24-bit integers. This idea can also be applied to averaging colors, as described in Section 2.13.2. The developer of this algorithm has selected the last alternative above as the one that worked best in tests of the algorithm.
2.13.1 Matching Colors Current computers and graphics cards support many colors. A pixel in a color image is often stored in three bytes, containing the relative intensities of the red, green, and blue components of the pixel’s color. With 24 bits per pixel, the size of the color palette (the number of possible colors) is 224 ≈ 16.8 million. Any given image may feature only a few hundred or a few thousand colors, but they are selected from this huge palette. Current monitors can also display this palette, although a typical 1920 × 1080 monitor consists of a little more than two million pixels. Nevertheless, there may be situations (especially on small monitors of the type used in mobile devices) where the output device is limited to a small palette of colors. Section 2.28 shows how to generate grayscale images on a monochromatic output device by dithering. Similarly, it is possible to employ color dithering to approximate images with many colors on an output device with a limited color palette. The method described here is due to [Kientzle 96] (with an error correction). Given a small set S of 24-bit colors and a color C, this method shows how to locate the best match of C in S. Assuming that C is given as the 24 bits R7R6R5R4R3R2R1R0, G7G6G5G4G3G2G1G0, and B7B6B5B4B3B2B1B0, we interleave these bits to create the 24-bit integer G7R7B7G6R6B6G5R5B5G4R4B4G3R3B3G2R2B2G1R1B1G0R0B0. This integer is a good candidate for the title “average of the three color components of C” because (1) its three most-significant bits are the most-significant bits of the RGB components (and the same is true for all the other triplets of bits) and (2) it takes
2.13 Adaptive Scaling by 2
72
into account the fact that the eye is most sensitive to green and least sensitive to blue (Figure 21.4). We denote This average by AC. Two colors can be considered close if they match in several of their most-significant bits. Thus, all we have to do is interleave each of the colors stored in S in the same way to become a 24-bit integer, and then sort these integers. Given a color C, it is now easy to find its best match in S by computing AC and performing a binary search (Figure 2.41, where each table entry has a 24-bit interleaved color on the left and the original three color components on the right).
R7R6 ... R0
G7G6 ... G0
B7B6 ... B0 Available colors (in 24-bit interleaved) Binary search for closest match
A 24-bit interleaved integer
0 107 51 0 163 61 0 255 0 141 129 179 119 88 179 106 66 129 76 30 71 191 0 0 222 0 0 222 91 58 222 136 91 222 169 151 255 255 146 255 255 98 255 255 45 255 255 0
Figure 2.41: Matching Interleaved Colors.
This method is not very accurate, but it is fast, simple to implement, and does not require large tables. Given that such color matching is needed in computing environments that are already limited, this method offers an excellent compromise between accuracy and simplicity. It is easy to compute the interleaved encoding of a given color with only logical operations and shifts. We construct a 256× 3-byte lookup table that maps an input byte of the form A7A6A5A4A3A2A1A0 to the three bytes 00(A7)00(A6)00, (A5)00(A4)00(A3)0, and 0(A2)00(A1)00(A0). Three table lookups, for the three color components of C, produce nine bytes. The three bytes of the red component are shifted one byte to the left and are ORed with those of the green component to produce the three bytes T. Then the three bytes of the blue component are shifted two positions to the left and are ORed with T.
2.13.2 Averaging Colors Given two numbers a and b, their average is (a + b)/2. This is easy, but how do we average two colors? Even more, how do we average three colors? Given the two colors A and B, we can average them by averaging each of their color components separately. Thus, [(AR + BR)/2, (AG + BG)/2, (AB + BB)/2], but this involves a subtle problem; arithmetic addition may cause overflow. If both colors are close to red, their red components will be large, so adding these components may result in overflow. The following identity, known to many programmers, can be employed to compute the average of two numbers with only logical operations and a shift. Avoiding arithmetic operations eliminates the chance of overflow:
2 Raster Graphics
73
Av(A, B) = (A + B)/2 = (A and B) + (A xor B)/2 where xor stands for exclusive-or and the division by 2 is done with a right shift. A least-significant bit may disappear after the shift, but such bits are the least important ones. This identity can be applied to each of the three color components of two colors to obtain their average color, but what about three colors? Here, we use the simple identity (a + b)/2 + c (a + b)/2 + , (a + b + c)/2 = 2 2 which implies that the average of three colors equals Av(Av(a,b),c)+Av(a,b)/2.
2.14 The Keystone Problem A projector is a device that projects images on a screen. Projectors are used in movie theaters, large public gatherings, and at home. Old projectors would project the image in a rectangle, symmetric about the lens. If the screen was not centered with the projector’s lens but was located higher, the projector had to be tilted upward, which projected the image on the screen as a trapezoid, with a wide top and narrow bottom, as illustrated in Figure 2.42. This effect became known as the keystone problem, because the keystone placed by builders at the top of an arch has a similar shape. In addition to distorting the image, this effect may cause some areas of the screen to appear blurred, because the projector lens is focused at the average distance only.
Scr een
Figure 2.42: Projecting at an Angle.
74
2.14 The Keystone Problem
Modern projectors can correct this problem either manually or digitally. Manual correction is done by adjusting the lens manually to project the image above or below the projector (Figure 2.43), thereby eliminating the need to tilt the projector. Adjusting a lens is normally limited to about 15◦ , so the projector often has to be moved away from the screen. Digital keystone correction is done by software that distorts the image inside the projector before it is sent to the lens. If we denote the keystone distortion by D(α), where α is the tilt angle, then the software applies the inverse transform D −1 (α) to the image and sends it to the lens. Once projected by the lens on its way to the screen, the image is distorted by D(α), so it ends up on the screen in its original form, free of any distortion.
Figure 2.43: Projecting Up or Down.
Digital keystone correction is done when the projector is installed in the projection room. A test image is projected, and the user turns a knob (whose position is continually input by the software) until the projected image becomes rectangular. This type of correction tolerates large tilt angles but may degrade image quality because applying a distortion by software (i.e., transforming a rectangular image to an upside-down trapezoid) is done by manipulating pixels. New pixels may be added to the bottom part of the image, while some pixels may be deleted from the top part, and any such manipulation reduces the visual quality of the original. Computing the widths wt and wb of the top and bottom of the trapezoid is straightforward. Figure 2.44a shows a side view of a tilted projector (with projection angle α and tilt angle β) at a distance d from the screen. The figure implies that cos(β − α/2) = d/b and cos(β + α/2) = d/t. Thus, the quantity t cos(β − α/2) = b cos(β + α/2) is a reasonable measure of the distortion. Notice that β = 0 (i.e., no tilt), results in t = b (no distortion). Figure 2.44b shows the top view of the same scene, and it is clear that the width wt of the top of the trapezoid satisfies (wt /2)/t = tan(α/2) and (wb /2)/b = tan(α/2). Thus, knowing d, β, and α and computing t and b yields the values of wt and wb . We denote the width and height of the original image by w and h, respectively. The top row of the projected image will be wt pixels wide (i.e., the keystone distortion will stretch it from w to wt pixels) and the bottom row will be wb pixels wide (shrunk
2 Raster Graphics
t
Projector
75
t b d
w t wb
b
d
(b)
(a) Figure 2.44: Side and Top Views.
from w pixels). As we go down the rows from top (row 0) to bottom (row h − 1), each projected row r will be (wt − wb )/(h − 1) pixels shorter than its predecessor, so its length L(r) will be L(r) = wt −
r(wt − wb ) , h−1
for r = 0, 1, . . . , h − 1. It is now clear that in order for the projector to compensate for the distortion and project a rectangle of width w instead of a trapezoid, each row will have to be stretched or shrunk by a factor of F (r) = w/L(r), where F (r) > 1 implies stretching and F (r) < 1 implies shrinking. Exercise 2.16: It may make more sense to project the image as a rectangle of width wt . How can this be done? Reference [keystoning 10] shows how Adobe Photoshop can be used to correct the keystone problem. Reference [Sukthankar et al. 00] describes a sophisticated approach to solving the keystone problem. A low-cost camera is positioned such that it sees both the projector and the screen. The image from this camera is sent to a computer that automatically generates the correct parameters for the inverse distortion and sends them to the projector. The user can interact with the computer in order to improve the final projected image, if needed. This way, the projector can be moved out of the way and does not have to be directly in front of, or above the screen.
2.15 Bitmap Rotation
76
2.15 Bitmap Rotation Rotating a bitmap is a common operation. Imagine that an image has been scanconverted and is already stored in a small bitmap. It is easy to transfer this image to become part of a large bitmap, but what if we want to rotate it before transferring? We show three approaches to this problem. All three assume that an n × n bitmap A is given, with its center at the origin, and has to be rotated clockwise θ degrees about its center into another bitmap B. The first approach is based on the two-dimensional rotation matrix R, Equation (4.4). Each bitmap pixel (x, y) is multiplied by this matrix to produce the coordinates (x∗ , y ∗ ) of the rotated pixel. The transformation is (x∗ , y ∗ ) = (x, y)R = (x, y)
cos θ sin θ
− sin θ cos θ
= (x cos θ + y sin θ, −x sin θ + y cos θ).
This is simple but produces unacceptable results because it is easy to show that some pixels of B may remain empty (i.e., not get any of the pixels of A mapped to them), while two pixels of A may get mapped to the same pixel of B. Figure 2.45 illustrates this problem, while Figure 2.46 is a more specific example. A 9 × 9 bitmap is rotated in this figure 45◦ and the mapping is computed by the Mathematica code t=Pi/4; Brot[x_,y_]:={x Cos[t]+y Sin[t],-x Sin[t]+y Cos[t]}; Do[Print[x,",",y,N[Brot[x,y],1]], {x,-4,4},{y,-4,4}] The resulting list of rotated points shows that, for example, pixel (0, 4) is mapped to (3, 3) (the pixels numbered “1” circled), pixel (1, 3) is mapped to (3, 1) (“2” circled), pixel (0, 3) goes to (2, 2) (“3” circled), pixel (1, 4) is mapped to (4, 2) (“4” circled), but no pixel is mapped to (3, 2) (the one with “?”). Also, the two pixels (2, 3) and (2, 4) are mapped to (4, 1) (the gray pixels).
Figure 2.45: Rotating a Bitmap.
2 Raster Graphics
-4,4
1
4
3
2
77
4,4 1 3
?
4
2 0,0
-4,-4
4,4
4,-4
Figure 2.46: A 45◦ Bitmap Rotation.
Exercise 2.17: Perform the same calculation and analysis for a 3 × 3 bitmap. The second approach is based on the source-target paradigm (Section 2.3). It tries to overcome the earlier problems by looping over the pixels B[i, j] of the final bitmap B and determining the pixel of A that should map to B[i, j]. Since the rotation is represented by (x∗ , y ∗ ) = (x, y)R, we use the relation (x, y) = (x∗ , y ∗ )R−1 to find a pixel of A for every pixel of B. The matrix inverse R−1 can be computed once, before we start the loop. This approach produces better results but is computationally expensive because the relation above requires four multiplications and two additions for each pixel that is being mapped. The third approach is due to [Paeth 86]. It is based on Exercise 4.24 which shows that a two-dimensional rotation about the origin can be expressed as the sequence of three shears: 1 0 1 − tan θ2 cos θ − sin θ 1 − tan θ2 = A · B · A. = 0 1 0 1 sin θ 1 sin θ cos θ It is easy to see that the first shear transforms a pixel (x, y) to the pixel (x, y−x tan(θ/2)).
2.15 Bitmap Rotation
78
This transformation preserves the column x of pixel (x, y) and changes only its row. Figure 2.47a shows how each column is moved by this shearing vertically, up or down, and how the extreme columns, where the column number x is large, are translated more than the inner columns. The center column, where x = 0, is not moved at all. The same is true for the third shear, while the second shear, which transforms (x, y) to (x + y sin θ, y), preserves rows and changes only columns.
(x,y) x=4 f
x=-4
(x,y*)
1-f f
(x,y*+1)
1-f
x=0 (a)
(b) Figure 2.47: Rotation by Shearing.
The following points summarize the principles of this approach: Let’s assume that the image is in grayscale and let P (x, y) denote the intensity of pixel (x, y). Since the quantity tan(θ/2) is in general noninteger, the source pixel (x, y) is normally moved such that it covers f percent of one of the destination pixels (x, y ∗ ) and (1 − f ) percent of the adjacent pixel (x, y∗ + 1) (Figure 2.47b, where the source pixels are dashed and the destination pixels are shown in light blue). The intensity of destination pixel (x, y ∗ ) should therefore be set to the sum f P (x, y) + (1 − f )P (x, y − 1). In this way, the sum of the intensities of all the destination pixels in bitmap column x will equal the sum of the intensities of all the source pixels in the same bitmap column. Figure 2.46 shows that the source and destination bitmaps are incompatible in the sense that certain destination pixels have no corresponding source pixels and certain source pixels are moved outside the bitmap by the rotation. This incompatibility becomes worse the larger the rotation angle, up to an angle of 45◦ . Because of the symmetry of a bitmap, our method assumes that the rotation does not exceed 45◦ . A rotation θ of more than 45◦ can be achieved by rotating in the opposite direction by 90◦ − θ. A rotation of more than 90◦ can be performed by first swapping pixels (x, y) and (y, n − x), which amounts to rotating the bitmap clockwise 90◦ , and then using the present approach to rotate by less than 45◦ . We also assume that the rotation is done about the center of the bitmap and that the coordinates of the center pixel are (0, 0). Figure 2.48 is a pseudo-code description of the routine for shearing in the x direction. The yshear routine for shearing in the y direction is similar. The main program inputs
2 Raster Graphics
79
procedure xshear(α,r,c); for i:=-r to r do skew:=i*α; int:=floor(shear); f:=frac(shear); PrevLeft:=0; for j:=-c to c do p:=Sbitmap[i,j]; LeftPart:=p*f; Dbitmap[i,j+int]:=p-LeftPart+PrevLeft; PrevLeft:=LeftPart; endfor; Dbitmap[i,int]:=PrevLeft; endfor; end; Figure 2.48: Shearing in the x Direction.
(or computes) the rotation angle θ, and then makes the three procedure calls xshear(− tan(θ/2),r,c); yshear(sin θ,r,c);
xshear(− tan(θ/2),r,c);
Notice that p-LeftPart equals p*(1-f) which is the right part of the contribution to the intensity of the new pixel. This part is added to PrevLeft, which is the left part of the contribution.
2.16 Nonlinear Bitmap Transformations The bitmap transformations described in this section are not as basic as bitmap scaling and rotation and are used mostly for artistic and ornamental purposes. Many more beautiful nonlinear transformations are possible, and are an integral part of bitmap manipulation software such as Adobe Photoshop. These transformations are nonlinear in the sense that the transformation applied to a given pixel depends on its location in the bitmap. Chapter 7 discusses nonlinear projections, which are transformations from three dimensions to two dimensions. Figure 2.49 shows three popular nonlinear transformations, twirl, spheroid, and wave. Twirl. The term twirl means to rotate or revolve briskly, to swing in a circle, or to spin. This transformation (Figure 2.49a) rotates each pixel of the bitmap around a center, but the angle of rotation diminishes with the distance of the pixel from the center. In a general twirl, the user has to specify the coordinates (x0 , y0 ) of the twirl center, the rotation angle α, and the radius r of the twirl. To derive the transformation, we first assume that α is constant. In this case, a pixel P has to be rotated about pixel (x0 , y0 ), and this is done in three steps, as described in Section 4.2.2. Pixel P is first translated such that the twirl center is brought to the origin, a rotation rotates P about
2.16 Nonlinear Bitmap Transformations
80
(b)
(a)
(c)
Figure 2.49: Nonlinear Bitmap Transformations.
the origin (i.e., about the twirl center), and a reverse translation brings P to its new location. Combining these transformations produces the final transformation matrix T ⎡
⎤⎡ ⎤⎡ 0 1 cos α sin α 0 0 ⎦ ⎣ − sin α cos α, 0 ⎦ ⎣ 0 x0 0 0 1 1
⎤ 0 0⎦ 1 ⎤ cos α sin α 0 − sin α cos α 0⎦. =⎣ x0 (1 − cos α) + y0 sin α y0 (1 − cos α) − x0 sin α 1
1 T=⎣ 0 −x0 ⎡
0 1 −y0
0 1 y0
(See also the answer to Exercise 4.15.) As mentioned earlier, the rotation angle α is not constant but drops linearly with the distance d of the current pixel from the center, until it reaches zero when d = r. Pixels at distances greater than r from the center are not transformed. The distance of pixel (x, y) from the twirl center (x0 , y0 ) is d = (x − x0 )2 + (y − y0 )2 , so the rotation angle as a function of d and r is α r−d r . Section 2.3 explains why, in the case of twirl and other nonlinear transformations, it is better to use a target-to-source algorithm. Such an algorithm loops over the pixels of the target bitmap G and computes coordinates (x, y) for every target pixel (x∗ , y ∗ ). This can be denoted by (x, y) = T−1 (x∗ , y ∗ ), where T−1 is the inverse of the original transformation T. Exercise 2.18: Compute T−1 . Spherize. The term spherize means to cause to appear as if on the surface of a sphere. This transformation simulates a thick lens, shaped as a half-sphere, that is placed on top of the image. Image areas outside the lens are not affected by the transformation. Figure 2.49b illustrates the effect of spherical transformation. The user has to input the center (x0 , y0 ) of this transformation, the radius r beyond which no pixels are modified, and the strength of the transformation, expressed in terms of the refraction index ρ of the (imaginary) lens. A pixel P = (x, y) is transformed by shifting (or translating) it along the line from the center (x0 , y0 ) to P. The amount of the shift is ρ-reduced linearly for pixels located away from the center. For pixels at a
2 Raster Graphics
81
Figure 2.50: Twirl and Spherize.
distance r from the center, the shift amount should be zero. The mathematics is easy to follow, but readers unfamiliar with the properties of points and vectors should first consult Section 8.1. The vector from (x0 , y0 ) to (x, y) is (x − x0 , y − y0 ). We normalize it by dividing it by its length to obtain (x − x0 , y − y0 ) V= . (x − x0 )2 + (y − y0 )2 (x − x0 )2 + (y − y0 )2 is the distance The amount of the shift is A = ρ r−d r , where d = of P from the transformation center. The shift itself is done by adding vector AV to P. Thus, (x − x0 , y − y0 ) r−d (x∗ , y ∗ ) = (x, y) + AV = (x, y) + ρ . r (x − x0 )2 + (y − y0 )2 Figure 2.50 shows a simple example of spherical transformation. Wave. The wave transformation transforms each row of pixels from a straight line to a sine wave, and similarly for the columns (Figure 2.49c). The user has to input the periods Tx and Ty (in pixels) and the amplitudes ax and ay of the waves in the x and y directions. The wave transformation can easily result in an unfamiliar, even meaningless image, so the user should start with large periods and small amplitudes and experiment. In many cases, it may make sense to set one of the amplitudes to zero, so as to have waves in one direction only. The mathematics of the wave transformation is simple. To convert a row of pixels to a sine wave, pixels have to be shifted up or down by an amount that depends on the sine of their x coordinate. Thus, y∗ = y + ay sin
2π x , Tx
and a similar expression for x∗ . When x = 0 and x = Tx , the sine function is zero, and when x = Tx /2 and x = 3Tx /2, the sine function is at its maximum or minimum.
82
2.16 Nonlinear Bitmap Transformations
Figure 2.51: Fish, Original and Wave.
In addition to the three nonlinear transformations above, current image manipulation software offers many more. The following is only a short list of what such software can often do: Glass (looking at the image through a thick, rough glass plate, see color Plate F.2), ripple (like a wave, but with random amplitudes), zigzag (like a wave, with a zigzag pattern instead of a smooth sine wave), pixelate (convert a square group of pixels to a uniform, average color, see Plates H.1 and N.3), add various types of noise, sharpen, blur, emboss (Section 2.31 and Plate H.4), and add textures.
2.16.1 Arbitrary Bitmap Deformations The nonlinear bitmap transformations described earlier are artistic or ornamental and are also global; the same operations (with perhaps different parameters) are applied to each pixel in the bitmap. Sometimes, a more general transformation is needed, where the bitmap is partitioned into regions and each region is transformed separately, with perhaps different transformations. Such an operation can be referred to as a deformation or as a local nonlinear transformation. Figure 2.52 illustrates an example. Two bitmaps are shown, one divided into triangles and the other partitioned into squares. Each square or triangular region is then mapped by perturbing its three or four corner points (in the figure, the perturbations are random, but in practice they may be regular and determined computationally). Once partitioning is complete, each region of the source bitmap is transformed by mapping each of its pixels to the corresponding region in the target bitmap. Transforming the pixels of a square region is a simple bilinear interpolation. Given a square source region with corners at a = (ax , ay ), b, c, and d, Figure 2.53 shows that the local coordinates (u, w) of a general point (x, y) are u = (x−ax )/(dx −cx ) and w = (y−ay )/(dy −by ). These coordinates are in the interval [0, 1] and they serve to compute the location of the transformed point (x∗ , y ∗ ) in the quadrilateral target region. Assuming that the target is bounded by points A, B, C, and D, we use the expression of bilinear interpolation, Equation (9.6), duplicated here (x∗ , y ∗ ) = A(1 − u)(1 − w) + B(1 − u)w + Cu(1 − w) + Duw.
(9.6)
2 Raster Graphics
83
Remove["Global‘*"]; L = Table[{i+RandomReal[{-0.15, 0.15}], j+RandomReal[{-0.15,0.15}]},{i,0,9},{j,0,9}]; P1=ListLinePlot[L, Axes->False]; P2=ListLinePlot[Transpose[L], Axes->False]; Show[{P1, P2}, AspectRatio->Automatic] Figure 2.52: Square and Triangular Grids for Arbitrary Image Deformation.
Once the values of u and w are substituted, this equation produces the transformed point (x∗ , y ∗ ).
a
A
b
B
w
(x*,y*)
u
c
d
C
D
Figure 2.53: Mapping Square and Triangular Regions.
Mapping the pixels of one triangle to another triangle is done by Equation (9.5) (Section 9.2.7), duplicated here with a small, important modification u(P2 − P1 ) + w(P3 − P1 ).
84
2.16 Nonlinear Bitmap Transformations
Thus, given a pixel (x, y) in a triangular region in the source bitmap, we first determine the local coordinates u and w of the pixel, then substitute the corner points of the target triangle in the same equation to obtain the local coordinates of the mapped pixel. Arbitrary bitmap deformations can be useful in advanced image processing techniques. Here are just three examples of where such image operations are invaluable: Morphing (Section 19.10) is the process of varying one image into another. Figure 2.54 illustrates a common morphing problem. The eyes of the girl and cat are positioned at different relative locations in the two images and are also slanted in opposite directions. Once the user indicates these regions to the morphing software, the software “matches” them by partitioning the images and matching the blue rectangle on the left to the red quadrilateral on the right (see also Plates K.1, K.3, and T.2).
Figure 2.54: Morphing Images.
Panoramas are commonly created by “stitching” together several images (see also Section 7.5 and Plates Q.2, R.1, and S.1). The individual images to be stitched must have significant overlap, for the software to identify their relative locations in the final panorama. In the simplest case, an image A is photographed, the camera is rotated to the right, and an overlapping image B is taken. The right-hand part of A and the lefthand part of B show the same scene, but they may not be identical because the rotation of the camera has introduced perspective effects. To compensate for such effects, the panorama software may have to undistort parts of several images, and this can be done by partitioning them and matching triangles or squares. Wrapping one image as a texture around another is a common operation in computer graphics and may also be implemented by arbitrary bitmap deformations. (See Plate J.3 and cup in Plate S.2.)
2 Raster Graphics
85
2.17 Circle Inversion This projection is different from the ones described earlier because it is neither artistic nor ornamental. Nevertheless, it is included here because of its simplicity and mathematical elegance. Circle inversion was the brainchild, around 1830, of Jakob Steiner. It has been researched and studied extensively since its first publication, and much is known about it (as is shown by a simple Internet search). He [Steiner] is a middle-aged man, of pretty stout proportions, has a long intellectual face, with beard and moustache and a fine prominent forehead, hair dark rather inclining to turn grey. The first thing that strikes you on his face is a dash of care and anxiety, almost pain, as if arising from physical suffering—he has rheumatism. He never prepares his lectures beforehand. He thus often stumbles or fails to prove what he wishes at the moment, and at every such failure he is sure to make some characteristic remark. —Thomas Hirst, Diary (1852).
y
P*
r
P
1 Q*
x Q
Figure 2.55: Circle Inversion.
Figure 2.55 illustrates the principle. The figure shows the unit circle centered on the origin and an arbitrary point P with polar coordinates (r, θ). Circle inversion projects P to P∗ = (1/r, θ). Both P and P∗ have the same angle θ, which places them on the same straight line that passes through the origin. If r > 1, then P is outside the unit circle and P∗ is inside it (because 1/r < 1). Thus, this projection inverts points with respect to the unit circle centered on the origin. It is easy to see that points on the circumference of the circle are projected to themselves and that circle inversion is undefined for the origin, where r = 0. (Although we can say that the origin is projected to the point at infinity, but this claim is not very useful and may cause confusion with parallel lines, which are also sometimes said to meet at infinity.) Since P is moved to P∗ along the line that connects P to the origin, we can think of this projection as scaling.
2.17 Circle Inversion
86
From P∗ = (1/r, θ), we obtain x∗ 2 + y ∗ 2 = 1/r2 and this implies P∗ = (x∗ , y ∗ ) =
(x, y) P = 2 = sP x2 + y 2 x + y2
because this relation means that x∗ 2 + y ∗ 2 =
(x2
x2 y2 1 + 2 = 2 = 1/r2 . 2 2 +y ) (x + y 2 )2 (x + y 2 )
Notice that the scale factor s depends on P, indicating that this type of projection is nonlinear. There are several applets on the Internet that make it easy to explore the properties of circle inversion. This projection has a number of interesting features, the most important of which are the following: 1. Any circle that intersects the unit circle at right angles is projected to itself. 2. The angle between two projected lines is preserved. Thus, circle inversion is a conformal projection. 3. Circles that do not pass through the origin are projected into circles (that do not pass through the origin and generally have a different radius). 4. Similarly, lines that do not pass through the origin are projected into circles that do pass through the origin (Figure 2.56). 5. A circle centered on the origin is projected to another circle similarly centered. 6. Lines through the origin are projected to themselves (except that the projection of the origin is undefined). 7. The inverse of an inverse is the original point. Thus, (P∗ )∗ = P. (This is trivial.)
4 1
3 2
2 3
Figure 2.56: Four Circles and Lines.
1 4
2 Raster Graphics
87
Curves that are their own inverse are called anallagmatic. Exercise 2.19: Search the mathematical literature or the Internet (or just think) to find another anallagmatic curve.
L
ci Unit rcle
Proje cti on
L of
Q*
Q O
P
P*
Figure 2.57: Circular Inversion of a Line.
Here is an explanation of feature 4. Figure 2.57 shows a line L that does not pass through the origin. Consequently, there must be a perpendicular to L from the origin. The point where this perpendicular meets L is denoted P and its projection is denoted P∗ . We now select another arbitrary point Q on L and denote its projection Q∗ . It is obvious that OP ·OP ∗ = 1 and OQ·OQ∗ = 1, so we conclude that OP/OQ∗ = OQ/OP ∗ . This shows that triangles OP Q and OP ∗ Q∗ are similar (notice that they have a common angle), which, in turn, implies that angles OP Q and OQ∗ P ∗ are equal. Since the former is a right angle, the latter must be also. However, point Q is an arbitrary point on L, so angle OQ∗ P ∗ equals 90◦ for any point Q on L, showing that the projection Q∗ lies on a circle that passes through the origin O and has a diameter OP ∗ . The projection of P is P∗ , and the projection of the origin is the point (or points) at infinity. Line L of Figure 2.57 passes inside the unit circle. For lines outside this circle, the diagram looks different but the proof is identical. Exercise 2.20: Use similar arguments to explain feature 3. Exercise 2.21: The discussion so far has assumed inversion with respect to the unit circle. Given a circle C of radius R about the origin, show how to project a point P with respect to it. Figure 2.58 shows a simple geometric construction of the inverse of a point P. In part (a) of the figure, P is inside the circle. Line L1 is constructed from the center through P and continues outside the circle. Line L2 is then constructed perpendicular to L1 . Point A is the intersection of L2 with the circle. A tangent L3 to the circle is
2.18 Polygons (2D)
88
A
A
L3
L2 P
R
P*
L1
L2
P0
(a)
L3
P*
L1
P
(b) Figure 2.58: Construction of Circle Inversion.
constructed at A, and P∗ is placed at the intersection of the tangent and L1 . Part (b) shows the similar construction when P is outside the circle. Figure 2.58b illustrates another feature of circle inversion. Up to now, we assumed that the inversion is about a unit circle centered on the origin. Given a circle of radius R, the two triangles PP0 A and P∗ P0 A are similar, implying that P0 P/R = R/P0 P∗ or R2 = P0 P×P0 P∗ . The quantity R2 is termed the circle power. The inverse P∗ of a point P with respect to an inversion circle of radius R centered at P0 is given by P∗ = P0 + R2
P − P0 . |P − P0 |2
As is common with nonlinear projections, it is possible to come up with many variants of circle inversions. For example, project point (r, θ) to (1/r, 180◦ + θ). An obvious (but perhaps not very useful) extension of circle inversion is sphere inversion, where the spaces inside and outside a sphere are swapped. Reference [Coxeter 69] presents the complete theory of circle inversions. A more general treatment of inversive geometry can be found in [Stothers 05]. Figure 2.59 (after [Gardner 84]) shows the circle inversion of a chessboard.
2.18 Polygons (2D) This section discusses two-dimensional polygons, their definition, parts, and basic properties. Section 3.9 discusses methods for filling a polygon with a given color or texture. Section 9.2 discusses three-dimensional polygons and their applications to polygonal surfaces. A polygon is a closed, two-dimensional figure that consists of three or more straight line segments. Each segment (edge) is joined to two other segments at a point (vertex). A polygon is a closed plane figure where line segments are joined together. The word is derived from the Greek πoλυσ (many) and γων´ıα (knee or angle). A convex polygon satisfies the following: when a straight line is drawn through it, the line crosses at most two sides. The interior angles of a convex polygon add to 360◦ , but each is less than 180◦ .
2 Raster Graphics
89
Figure 2.59: The Circle Inversion of a Chessboard.
If it is possible to draw a line that crosses more than two edges of a polygon, then the polygon is concave and at least one of its interior angles is greater than 180◦ . A polygon can be regular (all the edges are the same length), irregular, simple (its boundary does not cross itself), or self-intersecting (also known as complex). All convex polygons are simple. Figure 2.60 shows elementary examples of polygons and lines crossing them.
Figure 2.60: Polygons and Intersecting Lines.
Exercise 2.22: What if the angle between two edges is exactly 180◦ ? The area of a polygon with vertices (xi , yi ) is tion 8.3).
1 2
i (xi yi+1
− xi+1 yi ) (see also Sec-
Polygons naming convention. We are familiar with the terms triangle, pentagon, octagon, and a few others, but what about the 60-sided polygon? The standard polygon naming convention is based on the number of sides and it combines a numerical prefix (derived from Greek) with the suffix -gon (there are some exceptions and also disagreements with this standard). The table below lists the names of many polygons.
2.18 Polygons (2D)
90
Name Triangle Quadrilateral Pentagon Hexagon Heptagon Octagon Nonagon, Enneagon Decagon Undecagon, Hendecagon Dodecagon Tridecagon, Triskaidecagon Tetradecagon, Tetrakaidecagon Pentadecagon, Pentakaidecagon Hexadecagon, Hexakaidecagon
n 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Name n Heptadecagon, Heptakaidecagon 17 Octadecagon, Octakaidecagon 18 Enneadecagon, Enneakaidecagon 19 Icosagon 20 Triacontagon 30 Tetracontagon 40 Pentacontagon 50 Hexacontagon 60 Heptacontagon 70 Octacontagon 80 Enneacontagon 90 Hectogon, Hecatontagon 100 Chiliagon 1,000 Myriagon 10,000
Even more, given a polygon with a “weird” number of sides such as 63 or 99, the following table proposes eight prefixes and nine suffixes that should satisfy most cases. Thus, a 63-sided polygon is named Hexacontakaitrigon, and a 99-sided polygon has the right to be addressed as Enneacontakaienneagon. Sides
Prefix
20 30 40 50 60 70 80 90
Icosikai Triacontakai Tetracontakai Pentacontakai Hexacontakai heptacontakai Octacontakai Enneacontakai
Sides +1 +2 +3 +4 +5 +6 +7 +8 +9
Suffix henagon digon trigon tetragon pentagon hexagon heptagon octagon enneagon
In computer graphics, we often need the vector N that is normal to a given edge. Given an edge between vertices P1 = (x1 , y1 ) and P2 = (x2 , y2 ), its slope is m = (y2 − y1 )/(x2 − x1 ). The slope of a vector perpendicular to the edge is therefore −1/m, so such a vector is described by the pair (1, −1/m). For each polygon edge, there are two such vectors, N and −N, pointing in opposite directions. In a convex polygon, it is meaningful to talk about the inside and outside of the polygon, which is why in certain applications it may be important to determine the outer or the inner normal for a given edge. To identify the inner and outer normals, we first construct the vector M from vertex P1 to a vertex Pk , where k is neither 1 nor 2. If the polygon is convex, vector M will point inside the polygon, so the sign of the dot product N • M tells whether N is inner (positive sign, indicating that the angle between
2 Raster Graphics Pk N P1
91
(2.5,3)
(−1,4)
P2
(5,2) (1,1)
Figure 2.61: Inner and Outer Normals.
these vectors is less than 90◦ , as in Figure 2.61) or outer (negative sign). To reverse the direction of N, just reverse its sign. Example: Given three vertices P1 = (1, 1), P2 = (5, 2), and Pk = (2.5, 3), the slope of edge P1 P2 is m = 2−1 5−1 = 1/4, so N = (1, −4) and M • N = (1.5, 2) • (1, −4) = −6.5. The dot product is negative and Figure 2.61 shows that vector (−1, 4) points inside the polygon, which is why vector (1, −4) is an outer normal. Data Structure. It is easy to store a polygon in memory. Simply construct a list of consecutive vertices. To actually display the polygon, a program needs only a pointer to this list. Scanning the list, the software creates an edge from each vertex to its successor in the list. The last edge is from the last vertex to the first one.
2.19 Clipping Often, a large scene is Constructed in memory, but only part of it should be displayed; the rest must be identified and clipped. A typical example is zoom. When zooming on a scene, only part of it can be displayed, so it is unnecessary and time consuming to compute the other parts of the scene; they should be identified and clipped. Clipping is especially important with three-dimensional scenes, where the software has to transform, project, and render (i.e., shade and texture) the individual objects. In such cases, clipping can considerably speed up the program. If the image exists only as a bitmap, clipping is trivial, but if the image exists in vector form (points, lines, curves, polygons, and filled areas), then special algorithms are needed to identify those parts of the image that should be kept and remove or disregard the rest. Any method that identifies those parts of an image that are either inside or outside a given region is referred to as a clipping algorithm. The region against which a graphics object is to be clipped is called clipping window. The discussion here includes two types of clipping algorithms, those that clip lines against rectangles or convex polygons and those that clip polygons against other polygons. To clip a line segment against a given polygon means to find their intersection points and retain only that part of the line located inside the polygon. This is relatively simple with convex polygons but complex (and perhaps rarely needed) with other types of polygons. Similarly, to clip a polygon A (the subject) against another polygon B (the
2.20 Cohen–Sutherland Line Clipping
92
clip) means to locate their intersection points and retain only those parts of A that are inside B. The most important line clipping algorithms are Cohen–Sutherland, Cyrus–Beck, Liang–Barsky, and Nicholl–Lee–Nicholl. Among the common polygon clipping algorithms we find Sutherland–Hodgman, Weiler, Liang–Barsky, Maillot, Vatti, and Greiner– Hormann.
2.20 Cohen–Sutherland Line Clipping The Cohen–Sutherland method is one of the oldest clipping algorithms. The main idea is to classify lines according to the relation between their endpoints and the boundaries of the clip rectangle. It is easy to see that if both endpoints of a segment are above the rectangle, the entire segment is outside the rectangle and no clipping is needed; the segment should be disregarded. The same is true if the two endpoints are below, to the left of, or to the right of the rectangle. If both endpoints are inside the rectangle, the entire segment should be displayed (no clipping). The algorithm proceeds as follows: 1. Each of the two endpoints of the segment is compared to the boundaries of the rectangle and is assigned a 4-bit number according to the result (Figure 2.62). Thus, if an endpoint is above and to the left of the rectangle, it is assigned the number 1001. 2. The two numbers assigned to the endpoints are logically “anded.” If the result is nonzero, the two endpoints are on the same side of the rectangle, meaning the entire segment is outside the rectangle and should be ignored (segment 1 in Figure 2.62). If the result is zero, there are three subcases as follows: 2.1. Both numbers are “0000,” meaning that both endpoints are inside the clip rectangle. There is no need to clip and the entire segment should be displayed (segment 2 in Figure 2.62). 2.2. Only one number is “0000.” One endpoint is inside the rectangle and the other outside it. One clipping point should be calculated (point a in Figure 2.62). 2.3. Neither of the two numbers is “0000.” Both endpoints are, in this case, outside the rectangle and two clipping points (b and c in Figure 2.62) should be computed. Bit # 3 2 1 0
If set then point is
d e
above window below window right of window left of window
3210
3210
3210
1001
1000
1010
0001
0000
0010
0101
0100
0110
f
2
5
g a
b
h 3
c
1 Clip Rectangle
Figure 2.62: The Cohen–Sutherland Clipping Algorithm.
4
2 Raster Graphics
93
It is obvious that cases 2.2 and 2.3 need further treatment. In these cases the line segment has to be clipped and cannot be trivially accepted or discarded. Thus, the algorithm proceeds by splitting the segment at a point where it crosses one of the rectangle’s (infinite) edges. The two resulting sections are examined and the one located on the outside of the edge (it always exists) is discarded. Section 2.18 explains how to distinguish the inside and outside directions of an edge of a convex polygon. A case in point is segment 4 of Figure 2.62. The algorithm determines its intersection with the left edge of the clip rectangle (point b). The section on the left of b lies outside the rectangle and is discarded, but the remaining section still cannot be accepted or discarded, so the algorithm must split it at the point where it crosses the bottom edge of the rectangle (point c). Of the two new sections, the one on the right is outside the rectangle and is discarded, while the one on the left (section bc) is immediately accepted. How does the algorithm know to test segment 4 first against the left edge and then against the bottom edge? It does not. It simply tests such a segment methodically against all four edges to locate an intersection point. The order of the tests is irrelevant, but should always be the same. Exercise 2.23: List the steps taken by the algorithm for segment 5 (dh) of Figure 2.62.
2.21 Nicholl–Lee–Nicholl Line Clipping The Cohen–Sutherland clipping method is simple but may require several iterations for clipping a segment, where the intersection of two lines is computed in each iteration. The Cyrus–Beck line clipping algorithm of Section 2.22 often has to test certain segments methodically against all four edges of the clip rectangle to locate an intersection point. Computing an intersection point is slow, which is why the Nicholl–Lee–Nicholl (NLN) algorithm described here, due to [Nicholl et al. 87], is interesting. It reduces the chances of multiple clipping of a single line segment and therefore requires fewer computations but more tests than other line clipping methods. Its developers have included a detailed analysis that shows that this algorithm requires fewer operations than its predecessors and can also save the results of certain computations for later use. Given a rectangular clip window and a line segment PQ, the NLN method divides the entire xy plane into nine regions, as illustrated in green in Figure 2.63. The algorithm distinguishes the following three cases, point P is inside the rectangle, it is in a corner region, and it is in an edge region (parts (a), (b), and (c) of the figure, respectively). If P is located in other regions, not the ones shown in the figure, the treatment is symmetric. For each of the three cases, the algorithm draws the four rays from P through the corners of the clip rectangle (the central region). These are shown in the figure in black. The algorithm then performs tests to identify those regions that satisfy the following property. For example, if P is in an edge region and Q is in a corner region (as in part (c) of the figure), the algorithm identifies the regions labeled LB in the figure. Regardless of where Q is located in these regions, the algorithm will have to determine the intersections of segment PQ with the left and bottom edges of the clip rectangle. There is no need to try any other intersections.
2.21 Nicholl–Lee–Nicholl Line Clipping
94
T
L
L
P
R
R B B B (a)
P L
T T
TR
P
TR TB
L
T TR
LB
(b)
L
LT L
LT
LR
LR LR
LB LB Q (c)
Figure 2.63: Regions in NLN Clipping.
Abbreviations T, L, B, and R in the figure indicate intersections with the top, left, bottom, and right edges of the rectangle, respectively. Abbreviations LT, LR, LB, TR, and TB indicate intersections of segment PQ with the top and left, left and right, left and bottom, top and right, and top and bottom boundaries of the rectangle, respectively. Thus, the essence of the NLN algorithm is to perform simple tests that indicate whether segment PQ lies to the left or right of each of the four rays, in order to identify the regions that satisfy the property above. Once these regions have been identified, the algorithm knows which edge or edges are intersected by the segment, and it computes the intersection points. It is clear that this algorithm has to perform many tests, but has to compute only one or two intersection points. Overall, the analysis done by the developers claims that the NLN algorithm performs the fewest number of divisions (one for each intersection point that is computed) and the fewest number of comparisons among the three line clipping algorithms described here. Another interesting property of this method is that it uses certain results several times. The tests performed by the algorithm require the perpendiculars to the four rays, and these can be computed once and stored for future use. The following paragraphs explain how to determine whether a given point lies to the left or to the right of a given ray. The negate and exchange rule (Page 207) states that rotating a point (x, y) 90◦ counterclockwise transforms it to point (−y, x). Thus, given the two points P = (Px , Py ) and Q = (Qx , Qy ), their difference is the vector (Qx − Px , Qy − Py ) which is transformed by a 90◦ rotation to its perpendicular (Py − Qy , Qx − Px ). Given a ray from P to Q, it divides the xy plane into two halfplanes. Given another point R, it is now easy to determine whether it lies to the left or to the right of the ray. We construct the vector from P to R and compute its dot product with the perpendicular (Py − Qy , Qx − Px ). The result is (Rx − Px )(Py − Qy ) + (Ry − Py )(Qx − Px ) and it is if this dot product is positive. easy to see that R lies to the left of PQ
2 Raster Graphics
95
2.22 Cyrus–Beck Line Clipping The Cyrus–Beck algorithm [Cyrus and Beck 78] clips a line against a convex polygon. Figure 2.64a shows that a straight segment (shown in red) from P0 to P1 may intersect the (infinite) edges of a polygon (in green) at up to four points, only two of which may be relevant for clipping purposes. The algorithm computes the intersection points and then performs a series of comparisons to decide which points are relevant for the clipping.
L P0
1
2
E
P0
P1
E
L
L
L
P1 Pk
3
P0
E P0
P1
E
P1 >0
(a)
=0
<0
(b)
Figure 2.64: Lines Intersecting a Convex Polygon.
Figure 2.64b shows a straight segment from P0 to P1 crossing an edge of a convex polygon. The segment’s parametric equation can be written P(t) = P0 + (P1 − P0 )t (see also Equation (9.1)). We pick a point Pk on the edge (it may be any point, but an endpoint is convenient) and examine the vector P(t) − Pk . This vector points from Pk to some point on the segment. We select three such points on the segment, located outside, on, and inside the edge, as indicated in the figure. These points are denoted by P(ti ) for three different values ti . We compute the outer normal N of the edge and perform the three dot products N•(P(ti )−Pk ). Section 2.18 shows why the dot product of an outer normal with a vector pointing inside the polygon is positive. Similarly, the dot product of an outer normal with a vector pointing outside the polygon is negative. The dot product of an outer normal with a vector pointing along the edge is zero, which can be shown in two ways as follows: The normal is perpendicular to the edge, so these two vectors are perpendicular. As vector P(t) − Pk swings from inside the polygon outside, the value of the dot product varies from positive to negative. Since the dot product is a smooth, continuous function, it must be zero when the vector points in the direction of the edge. From N • (P(t) − Pk ) = 0, we can solve for the value of t at the intersection point. We substitute N • [P0 + (P1 − P0 )t − Pk ] = 0, then perform the dot product N(P0 − Pk ) + N(P1 − P0 )t = 0,
96
2.22 Cyrus–Beck Line Clipping
and finally isolate t
N • (P0 − Pk ) . −N • (P1 − P0 ) In order for this expression to be valid, its denominator must be nonzero. Clearly, the normal is nonzero, and the difference (P1 − P0 ) is nonzero (because the two points must be different). The dot product N • (P1 − P0 ), however, can be zero, but only when the segment is parallel to the edge (i.e., no intersection points). Thus, the algorithm has to check whether the dot product in the denominator is zero. If yes, there are no intersection points and no need to clip this segment against the edge (but it may have to be clipped against another edge). If no (the dot product is nonzero), the algorithm computes t and then selects another edge and computes its intersection point (if any) with the same line segment (and the corresponding value of t). As Figure 2.64a shows, there can be up to four intersection points of the segment with various (infinite) edges, but at most two of them are relevant. Line 1 in the figure has two intersection points, both irrelevant. Line 2 also has two intersection points, both relevant. Line 3 has four intersection points, two of which are relevant. To determine the relevant values of t, the algorithm can immediately discard any t values outside the interval [0, 1] (they correspond to points outside the finite segment). The remaining values of t are now sorted. In the case of line 3, there are four values of t and it seems that we can simply select the two interior values, but would this always work? Also, each of lines 1 and 2 have two values, but these cases are different. Clearly, the algorithm needs a test to correctly handle all possible cases. This test is based on the following idea. We distinguish between points that enter the polygon and points that leave it (denoted by E and L, respectively, in Figure 2.64a). If the segment from P0 to P1 intersects an edge and then continues outside the polygon, the intersection point is designated L; otherwise it becomes an E point. The direction of an intersection, inside or outside, is determined by the direction of the outer normal (some outer normals are shown in blue in the figure). If the angle between the segment and the normal is less than 90◦ , the segment continues outside and the intersection point is L. Thus, the first intersection point of line 1 (as we move on the segment from P0 ) is L, but the first intersection point of line 2 is E. The test our algorithm needs is now obvious. Out of all the t values, the algorithm should select the largest value t0 that corresponds to an E intersection point and the smallest value t1 that corresponds to an L intersection point. These two t values are used to compute the endpoints P(t0 ) and P( t1 ) of the clipped segment. This test also identifies cases such as line 1. This line should not be clipped at all against our polygon, and the algorithm knows that because it ends with t0 > t1 for this line. The Cyrus–Beck algorithm becomes much simplified when the clip polygon is a rectangle with edges that are parallel to the coordinate axes. This version of the algorithm is known as the Liang–Barsky line clipping algorithm. Given a rectangle with its bottom-left corner at point (xmin , ymin ) and its top-right corner at (xmax , ymax ), Table 2.65 summarizes the quantities and operations required to compute up to four t values for the intersection points of a segment from P0 = (x0 , y0 ) to P1 = (x1 , y1 ). t=
2 Raster Graphics Clip edge
N
left (x = xmin ) right (x = xmax ) bottom (y = ymin ) top (y = ymax )
Pk
97
P0 − Pk
t −(x0 −xmin ) (x1 −x0 ) (x0 −xmax ) −(x1 −x0 ) −(y0 −ymin ) (y1 −y0 ) (y0 −ymax ) −(y1 −y0 )
(−1, 0)
(xmin , y)
(x0 − xmin , y0 − y)
(1, 0)
(xmax , y)
(x0 − xmax , y0 − y)
(0, −1)
(x, ymin )
(x0 − x, y0 − ymin )
(0, 1)
(x, ymax )
(x0 − x, y0 − ymax )
Table 2.65: Intersections of a Segment and a Rectangle.
2.23 Sutherland–Hodgman Polygon Clipping The Sutherland–Hodgman algorithm clips a subject polygon S (that can be convex or concave) against a clip convex polygon C. The method is iterative, clipping S against an edge of C in each iteration. An edge E of C is selected and is extended to become a halfplane. The inside and outside normals of E are also computed (these normals are well defined because polygon C is convex). Polygon S is traversed, edge by edge, and each edge D is compared to E and is clipped. Clipping D against E may result in zero, one, or two output vertices. These vertices are appended to an output list of vertices, and the algorithm passes to the next edge of S. Once the algorithm has processed every edge of S, it starts the next iteration, where it selects another edge of C and it uses the output list from the previous iteration as its input list of vertices. When all the edges of C have been selected in this way, the final output list constitutes the vertices of the clipped version of S. Figure 2.66 shows an example of a concave polygon (blue) clipped against four edges (red) of a square.
Figure 2.66: Clipping Against a Convex Polygon.
In a typical step, the algorithm examines an edge D of polygon C. Assume that the edge goes from vertex A to vertex B, the software has to determine the locations of these vertices with respect to edge E. Specifically, whether they lie inside or outside polygon C. Figure 2.67 is an example where three vertices of S are inside polygon C (in blue) and two vertices (plus more that are not shown) are outside. In part (a) of the figure, vertices A and B are both inside C and the algorithm outputs B (in red). In part (b), point A is inside and point B is outside. The algorithm computes the intersection point P and outputs it. In part (c), both points are outside C, and the algorithm outputs nothing. In part (d), point A is outside, while point B is inside, so the algorithm outputs the intersection point P , followed by B.
2.23 Sutherland–Hodgman Polygon Clipping
98
×
D B
E B
A
P
B
B
A
A (a)
P A
×
(b)
(c)
(d)
(e)
Figure 2.67: Sutherland–Hodgman Clipping Rules.
Thus, clipping these four edges against edge D appends points B(a), P (b), P (d), and B(d) to the output list. Point A(a) was added to this list earlier, when the algorithm examined the short edge that leads to A(a). The result of clipping S against edge D is shown in part (e) of the figure. Next, the algorithm examines the remaining edges of polygon S, but all their vertices are outside D, so nothing is added to the output list. Once all the edges of S have been clipped against edge D, the algorithm replaces the original list of vertices with the newly-generated output list and clips this set against another edge of C. Exercise 2.24: Figure 2.67d shows two vertices marked ×. How were they added to the output list? To summarize, when the algorithm examines edge AB against edge D, it follows these simple rules: If both vertices A and B are inside polygon C, output B. If A is inside and B is outside, output the intersection point of AB and D. If A is outside and B is inside, output the intersection point, followed by B. If both vertices are outside, output nothing. The algorithm involves many tests of points for being inside or outside a polygon, but determining intersection points, a more complex computation, is done infrequently.
2 Raster Graphics
c1 s4
s1
p6
p5 p4
p3
s3 p2 c4
c2
99
c3
p1
s2
Figure 2.68: Weiler–Atherton Clipping Example.
2.24 Weiler–Atherton Polygon Clipping In 1977, Kevin Weiler and Peter Atherton were working on an algorithm for visiblesurface determination and serendipitously came up with the clipping algorithm that is briefly described here. In 1980, Weiler improved this method with concepts from graph theory. The first step is to number the vertices of the subject (S) and clip (C) polygons such that the inside of each polygon is always on the right as we traverse it in order of the vertices. In Figure 2.68, polygon S is shown in green, with vertices si , and polygon C is in blue, with vertices ci . Next, all the intersection points of S and C are determined (they are red in the figure and are denoted by pi ). Note that these intersections come in pairs. If S enters C at point pi , it leaves C at point pi+1 . The algorithm starts at a vertex of S, say s1 , and traverses S until it reaches an entering intersection point (p1 in the figure). It then continues traversing S until it reaches a leaving intersection (p2 in the figure). At that point the algorithm “turns to the right” and starts traversing C until an intersection is found, at which point it again turns to the right and starts traversing S. This process is repeated until the algorithm reaches a vertex it has already visited (p1 in our example). This loop results in the polygon p1 s2 p2 c2 . Next, the algorithm repeats this process starting at another vertex of S, and iterates in this way until all the intersection points have been visited. Exercise 2.25: Assume that the algorithm starts at vertex s2 , what is the clipped polygon resulting from that iteration? Note. Here is how to compute the intersection of two edges. Given the edge from P0 to P1 , its parametric equation is P0 (t) = P0 + (P1 − P0 )t (see also Equation (9.1)). Similarly, given the edge P2 to P3 , its parametric equation is P2 (t) = P2 + (P3 − P2 )t. If an intersection point exists, it satisfies P0 + (P1 − P0 )t = P2 + (P3 − P2 )t, which is easily solved for t.
2.25 A Practical Drawing Program
100
If the equations are contradictory, then the edges do not intersect. If the equations are dependent, then the edges are parallel. (End of note.) Figure 2.69 is another example of this algorithm. The first iteration starts at s1 and produces the clipped polygon p1 p2 p3 p4 and the second iteration start at s3 and results in p5 s4 s5 p6 . At this point, all the intersection points have been visited, so the algorithm terminates. c1
s1
p1
p2
p4 s3 s6
c2 s2
p3
p5 p6
s4
s5
c3
c4
S: s1
p1
p2
s2
p3
p4
s3
p5
s4
s5
p6
C: c1
c2
p2
p3
c4
c3
p6
p5
p4
p1
S: s1
p1
p2
s2
p3
p4
s3
p5
s4
s5
C: c1
c2
p2
p3
c4
c3
p6
p5
p4
p1
s6
p6
s6
Figure 2.69: Weiler–Atherton Clipping Example.
2.25 A Practical Drawing Program Programs for artistic and technical drawing and illustration have been popular since the mid-1980s, and this section describes their main data structures and operations. There are also painting programs, but they are not discussed here. The only background necessary for this section is an understanding of the xor operation (Section 2.2.4) and of the bitmap in general. A typical drawing/illustration program starts with a blank screen and a menu of graphics objects that it can draw and manipulate. These normally include line, circle/oval, box (square/rectangle, optionally with rounded corners), polygon, cubic B´ezier curves (Chapter 13), and text. It may also be possible to paste in a small bitmap prepared outside the program (perhaps a scanned painting or a photograph). The user selects a menu option, supplies the necessary data, and the program displays the graphics object on the screen. In this way, the screen fills up with different items, each a simple graphics object, that may intersect, obscure each other, and have different colors.
2 Raster Graphics
101
The user then starts editing the image by selecting various objects and deleting, moving, or modifying them. The discussion here concentrates on selecting one of possibly many graphics objects on the screen. Figure 2.70a shows a simple drawing consisting of a line, a circle, and a rectangle. We initially assume that those objects do not intersect. The user points the cursor (the arrow in the figure) anywhere on the line (or within a few pixels of the line) and clicks to select the line. The program has to identify the object selected and highlight it, thereby confirming the selection to the user. The problem is that the program knows only the position of the cursor on the screen (its screen coordinates). These are the coordinates of a pixel on the selected object, and the immediate task of the program is to quickly find (1) all the other pixels of this object, (2) its type (line, circle, etc.), and (3) its other data (in the case of a line, the coordinates of the two endpoints; in the case of a circle, its center and radius).
Bitmap
Codemap 1
1
1
333333333 3 3 3 3 333333333 1 1 1 22 2 2 2 1 2 2 1 1 2 2 2 2 2 2 22 2 2 2
(a)
avail: no no no yes code:
1
1
1
1
L C R
color: R B G pointer:
(b) 1
(d)
Geometric structure 1 2 3 4
1 333333333 1 22 32 2 3 2 3 231 2 333333 33 3 2 2 2 2 22 2 2 2
(e)
(c)
(f)
String (g)
Figure 2.70: Data Structures for a Drawing Program.
Two data structures are used by the program to perform this task. The first is the codemap, an array the size of the bitmap where each location contains the serial number of a graphics object on the screen. The second is a geometric data structure containing the type of each graphics object drawn so far and its color and specific data. Figure 2.70b shows how the serial numbers of the three objects drawn so far are stored in the codemap (the rest of the codemap is assumed to have been initialized to zeros).
102
2.25 A Practical Drawing Program
Figure 2.70c shows the geometric data structure. Notice that the serial numbers are implied in that structure and don’t actually have to be stored there. Once the program detects a click, it inputs the cursor coordinates and checks the corresponding location of the codemap. In our example, it finds serial number 1. (Both the bitmap and the codemap are shown as two-dimensional arrays, but, in practice, they are normally one-dimensional.) The program examines location 1 of the geometric data structure, finds that the graphics object with serial number 1 is a line, and finds the pointer to its specific data. That data consists of the coordinates of the two endpoints. The program then calculates all the pixels of the line (using the same scan-converting method that was originally used to draw the line) and highlights them. The two endpoints may be highlighted with a different color, making them more visible to the user. This takes about the same amount of time as drawing the line in the first place. The program starts waiting for the next user response/command. Exercise 2.26: Rewrite the preceding paragraph for the case in which the user selects the circle. Text presents a special problem. The drawing program gets the characters of text from the different fonts and knows nothing about their shapes. In order to select a string of text, the user normally has to click on its reference point (which is either the bottom-left corner of the text or the left point of the baseline, Figure 2.70g). Any clicks within the text string are ignored by the program (although a sophisticated program may obtain the width of the character from the font file, and this enables it to identify clicks anywhere on the baseline). After selecting a graphics object, typical user responses may be to delete the object, move it, or reshape it. The user may also want to copy the object, group it with some other objects, or perform other operations, but we discuss only the first three operations. Deleting an object is done by using the same scan-converting method. Every pixel calculated by the method is erased (or is xored with the color of the object being erased) and the program also clears the serial number of the pixel in the codemap. When done, the program uses the serial number to mark the corresponding entry of the geometric data structure “available” and deletes the storage used for the specific data. To move a graphics object, the user grabs it at a certain pixel or anchor point and drags it to its new position. The program creates an outline of the object and moves it with the cursor. When the dragging stops, the program calculates the x and y displacements (the final cursor position minus its initial position) and moves the object to its new location pixel by pixel. Each pixel at position (x, y) is erased and drawn at position (x + dx , y + dy ). An xor may be used instead of straight erase and draw. After moving a pixel, the program moves its serial number from position (x, y) to position (x+dx , y +dy ) in the codemap. There is no need to change the geometric data structure, but the specific object information may have to be updated. (If the object is simple, such as a line or a circle, the outline of the object is the object itself. If the object is more complex, such as text or a bitmap, the outline of the object may be its bounding rectangle.) Reshaping an object can be done by rotating, scaling, or shearing it. These transformations are discussed in Chapter 4, where it is shown that they can all be expressed by a 3 × 3 transformation matrix where only six of the nine elements vary. Thus, re-
2 Raster Graphics
103
shaping is similar to moving. The user may enter a command or make their wish known by dragging. Once the program knows what transformation is required, it prepares a specific transformation matrix T. Each pixel P is then moved from its original position to position P·T, a step that’s followed by moving its serial number in the codemap in the same way. The geometric data structure may have to be updated, since reshaping may change the type of an object. A circle may become an oval; a square may turn into a rectangle or a parallelogram. Notice that scaling or rotating a bitmap cause special problems. These operations are discussed in Sections 2.5 and 2.15. We now turn to the case where graphics objects may intersect. Figure 2.70d shows three intersecting objects. We assume that the line was drawn first, followed by the circle, and then by the rectangle. This behavior is best visualized by thinking of each object as a layer that is drawn on top of all the previous objects. A glance at Figure 2.70e shows that the codes at the intersection points are those of the most-recently drawn object that passes through the point. If the user clicks on an intersection point, the latest object through it will be selected. This does not seem bad, until we consider the case where that object is moved. If the rectangle of Figure 2.70d is moved, its two intersection points would be set to zero. After many such moves, the codemap may become meaningless. We outline two solutions to this problem: 1. When an object with serial number n is drawn, the program checks each codemap location before it sets it to n. If a location is nonzero, its value is saved in a linked list before the location is set to n. We now have to have a linked list associated with each codemap location. We visualize the codemap as an array of structures, where each structure consists of two fields—a code and a pointer to a list. Initially, all the codes are set to zero and all the pointers are set to null. When the first code is stored in a location, the pointer remains null. When another code is stored in the location, the original code is saved in a list node and the pointer is set to point to that node. When a code is removed from a location, the pointer is checked. If it is not null, it is followed and the most recent code is restored. Modern computers can perform these operations fast enough to make them appear instantaneous. When this method is used, clicking on an intersection point will select the most recent object that passes through it. 2. When an object with serial number n is drawn, the program xor’s n with the codemap locations. The value stored in a codemap location is therefore the xor of the serial numbers of all the objects that pass through it. When an object with serial number m is deleted or moved, the program performs an xor of a codemap location with m. No lists are necessary, so this method is simple and fast. The following examples show the values stored in a hypothetical codemap location when objects 1, 5, 7, 1, and 5 are added then deleted (we assume 4-bit serial numbers) 0000 xor 0001 = 0001, 0001 xor 0101 = 0100, 0100 xor 0111 = 0011, 0011 xor 0001 = 0010, 0010 xor 0101 = 0111. The downside is that a codemap location corresponding to an intersection point does not contain any of the serial numbers of the intersecting objects. In our example, the contents of the hypothetical location varies from 4 to 3 to 2, ending with 7. Thus, the program should flag all codemap locations where objects intersect, and should disregard any user clicks at those points.
104
2.26 GUI by Inversion Points
Exercise 2.27: What is a good way to implement such flags?
2.26 GUI by Inversion Points Computer users are familiar with the various flavors of the Microsoft Windows operating system and the many versions of the Macintosh operating system. These operating systems are based on a graphical user interface (GUI), which makes it natural for the user to view the status of the computer at a glance and to perform many operations on files. In a typical application, a GUI-based operating system displays several disks, files, and folders (or subdirectories) on the screen, and the user can select any of them, open it, move it, change its name and content, or delete it. New folders may also be added, named, and dragged to new locations. Any folder may partly obscure others, thereby creating the illusion of a three dimensional display. At any time, only one file or folder (in this section we’ll refer to both files and folders as folders) can be active and its icon is graphically enhanced to distinguish it from the inactive folders. The user can easily select another folder by pointing at any location inside it and clicking. The selected folder, which may have been partly covered by others, is immediately brought to the front and may now obscure other folders. This section describes an approach to implementing such a display. The method described here was developed by Bill Atkinson of Apple Computer [Atkinson 86] who used it in his revolutionary MacPaint software. The first method shown here is an extension of BitBlt and is based on the concept of inversion points. It is described here for a monochromatic display, but can also be adapted to color displays. Any shape (such as folders, icons, and characters of text) that the operating system has to display is prestored in a special area in memory and is referred to as a source image. When the operating system is instructed by the user to display a certain source image in a certain region of the bitmap it “holds” the source image over the bitmap region and performs an xor of each bitmap bit with the source image bit that “covers” it. This embeds the source image in the region in such a way that (1) any other images in the region can still be recognized and (2) those images are automatically restored when the source image is later deleted. Nothing outside the region is affected. As described so far, this operation is a direct application of BitBlt. There are, however, two extensions that make this method more general and useful than BitBlt. (1) The source images don’t have to be rectangular. They can have any shape and can even consist of disjoint parts. (2) A source image is stored in memory in terms of inversion points, which effectively creates a compressed representation of the source image. A large, complex source image that consists of thousands of pixels may be represented by a short, sorted list of tens or perhaps hundreds of inversion points. An inversion point in the source image is defined as a point such that all the pixels to its right and below it should be inverted (flipped). Figure 2.71a (1) illustrates the effect of a single inversion point and (2) shows how two inversion points on a horizontal line create a vertical strip in the bitmap. Such a strip is infinite but can be turned into a rectangle by adding two more points as shown in part 3 of the figure. An L-shaped source image can be represented by six inversion points as depicted in the same figure.
2 Raster Graphics
105
A source image with a hole and another image with two disjoint parts are shown in parts 3 and 4 of Figure 2.71b, respectively, while part 5 of the same figure illustrates a source image with a slanted line. Such an image requires many more inversion points, but the use of inversion points makes it possible to represent any shape, with straight or curved boundaries, with holes or with separate parts.
4
2 3 3 1
5
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Figure 2.71: Inversion Points.
It is also clear, from Figure 2.71, that the inversion points are simply the endpoints of the horizontal boundary segments of the source image. This fact makes it easy for both the user and the software to determine the inversion points of a given image. The user does it by looking at the image, while the software has to scan the image row by row. Figure 2.71c shows how a source image in the shape of an L is embedded in a bitmap that contains a circle and a triangle. The six steps of this process are listed in parts (d) through (i) of the figure and they illustrate the order in which the inversion points are utilized. They are taken from top to bottom and points with identical y
2.26 GUI by Inversion Points
106
coordinates are employed from left to right. This is why the inversion points of a source image should be stored in memory already sorted in this order. The result of this process is that the new source image is xored with the existing bitmap. The circle of Figure 2.71 is unaffected because it is located outside the region and the triangle is still recognizable. The advantage of the xor is that deleting an object restores any other objects intersecting it. Thus, deleting the L-shaped source image restores the triangle and deleting the triangle restores the new source image. In principle, this method is slow. Each new inversion point may require the inversion of those bits located to its right and below it in the bitmap, and there may be many thousands of such bits. Many may have to be inverted several times, as illustrated in Figure 2.71. It is easy to come up with an efficient version once we realize that embedding a source image in a bitmap region affects only the bits within the region. A faster version of this method starts by computing a bounding box around the source image and identifying the matching box in the bitmap. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000001 0000000011 0000000111 0000001111 0000011111 0000001111 0000000111 0000000011 0000000001 0000000000 0000000000
1111110000 1111110000 1111110000 1111110000 1111110000 1111110000 1111110001 1111110011 1111111000 1111110000 1111100000 1111110000 1111111000 1111111100 1111111110 1111111111 1111111111
(a)
(b)
FOLDER A
FOLDER A FOLDER B
FOLDER A FOLDER B
(c)
Figure 2.72: A Bitmap Region Before and After Bit Inversions.
Figure 2.72a,b illustrates this version. The bounding box is 10 × 17 bits (the six inversion points are shown in red). In the top eight rows, the 6 bits between the two consecutive inversion points are inverted. Starting with row 9, the range of inversion is widened to include all 10 bits of a row. The two inversion points on row 17 stop the inversion. Figure 2.72c shows how this method can be combined with clipping. When a new folder B is added to the bitmap, folder A is clipped to the boundaries of B, with the result that B seems to float above A. A GUI-based operating system allows the user to select folders on the screen by moving a cursor and clicking. When the user moves the cursor to an arbitrary point P inside an image and clicks, the operating system receives only the screen coordinates of P . It turns out that the use of inversion points makes it easy to decide whether P is located inside a given folder. The operating systems starts by checking all the folders, finding those that contain P . It then performs an operation that depends on P . If P is not inside any folder, nothing should be done. If P is inside the top (active) folder, that folder is selected and highlighted. If P is outside the top folder and inside several inactive folders, the topmost of those folders is selected and becomes the active folder.
2 Raster Graphics
107
In order to determine whether point P is located inside a given folder, P is compared with all the inversion points of the folder as follows: A Boolean variable v is set to false. For each inversion point located above and to the left of P , v is toggled. If v is true at the end (i.e., v has been toggled an odd number of times), then P lies within the folder.
2.27 Halftoning For many decades, newspapers were printed in black-and-white. This is fine for text, but images require at least grayscale and preferably color. For years, CRT monitors used with computers were also monochromatic, which was fine for text, but too restrictive for images. Even when color CRT displays became popular, the really high-resolution CRTs were still monochromatic. Color printers became low cost and popular only in the early 1990s, with the development of reliable, high-resolution inkjet printers. Thus, for many years, both text and images had to be printed on black-and-white printers. The method used to generate grayscales on a bilevel device is known as halftoning. Halftoning is a method that makes it possible to display images with shades of gray on a black and white (i.e., bilevel) output device. The trade-off is loss of resolution. Instead of small, individual pixels, halftoning uses groups of pixels where only some of the pixels in a group are black. Halftoning is important because it makes it possible to print images in grascale on a black and white printer. It is commonly used in newspapers and books. A classic reference is [Ulichney 87]. The human eye can resolve details as small as 1 minute of arc under normal light conditions. This is called the visual acuity. If we view a very small area from a normal viewing distance, our eyes cannot see the details in the area and end up integrating them, such that we only see an average intensity coming from the area. This property is called spatial integration and is very nicely demonstrated by Figure 2.73. The figure consists of black circles and dots on a white background, but spatial integration creates the effect of a gray background. ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦ ◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦◦
.................... .................... .................... .................... .................... ....................
Figure 2.73: Gray Backgrounds Obtained by Spatial Integration.
The principle of halftoning is to use groups of n×n pixels (with n usually in the range 2–4) and to set some of the pixels in a group to black. Depending on the blackto-white percentage of pixels in a group, the group appears to have a certain shade of gray. An n×n group of pixels contains n2 pixels and can therefore provide n2 + 1 levels of gray. The only practical problem is to find the best pattern for each of those levels. The n2 + 1 pixel patterns selected must satisfy the following conditions: 1. Areas covered by copies of the same pattern should not show any textures.
2.27 Halftoning
108
2. Any pixel set to black for pattern k must also be black in all patterns of intensity levels greater than k. This is considered a good growth sequence and it minimizes the differences between patterns of consecutive intensities. 3. The patterns should grow from the center of the n×n area, to create the effect of a growing dot. 4. All the black pixels of a pattern must be adjacent to each other. This property is called clustered dot halftoning and is important if the output is intended for a printer (laser printers cannot always fully reproduce small isolated dots). If the output is intended for a monochromatic display only, then dispersed dot halftoning can be used, where the black pixels of a pattern are not adjacent. As a simple example of condition 1, a pattern such as • • • should be avoided, since large areas with level-3 groups would produce long horizontal lines. Other patterns may result in similar, annoying textures. With a 2×2 group, such effects may be impossible to avoid. The best that can be done is to use the patterns • • • • • • • •. • • A 3 × 3 group provides for more possibilities. The 10 patterns below (= 32 + 1) are the best ones possible (reflections and rotations of these patterns are considered identical) and usually avoid the problem of annoying textures. They were produced by the matrix ⎤ ⎡ 7 9 5 ⎣2 1 4⎦ 6 3 8 using the following rule: To create a group with in the above matrix should be black. • • • • • • • • • • • • • • •
intensity n, only cells with values ≤ n • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •
The halftone method is not limited to a monochromatic display. Imagine a display with four levels of gray per pixel (2-bit pixels). Each pixel is either black or can be in one of three other levels. A 2×2 group consists of four pixels, each of which can be in one of three levels of gray or in black. The total number of levels is therefore 4×3 + 1 = 13. One possible set of the 13 patterns is shown here. 00 00
10 00
10 01
11 01
11 11
21 11
21 12
22 12
22 22
32 22
32 23
33 23
33 33
2 Raster Graphics
109
2.28 Dithering The downside of halftoning is loss of resolution. It is also possible to display continuoustone images (i.e., images with different shades of gray) on a bilevel device without losing resolution. Such methods are sometimes called dithering and their trade-off is loss of image detail. If the device resolution is high enough and if the image is watched from a suitable distance, then our eyes perceive an image in grayscale, but with fewer details than in the original. It should be noted that dithering is the opposite of antialiasing. Antialiasing (Section 3.13) adds gray pixels to a black and white image in order to improve its appearance on a grayscale output device, while dithering converts a grayscale image to a black-andwhite image, in order to get better output on a monochromatic output device. The dithering problem can be phrased as follows: given an m×n array A of pixels in grayscale, compute an array B of the same size with zeros and ones (corresponding to white and black pixels, respectively) such that for every pixel B[i, j] the average value of the pixel and a group of its near neighbors will approximately equal the normalized value of A[i, j]. (Assume that pixel A[i, j] has an integer value I in the interval [0, a]; then its normalized value is the fraction I/a. It is in the interval [0, 1].) (Dithering can also be exploited to create new colors by mixing several existing colors, as illustrated by Figure 26.30 which shows how purple is obtained by dithering red and blue.) The simplest dithering method uses a threshold and the single test: Set B[i, j] to white (0) if A[i, j] is bright enough (i.e., less than the value of the threshold); otherwise, set B[i, j] to black (1). This method is fast and simple but generates very poor results, as the next example shows, so it is never used in practice. As an example, imagine a human head. The hair is generally darker than the face below it, so the simple threshold method may quantize the entire hair area to black and the entire face area to white, a very poor, unrecognizable, and unacceptable result. (It should be noted, however, that some images are instantly recognizable even in just black and white, as Figure 2.74 aptly demonstrates.) This method can be improved a little by using a different, random threshold for each pixel, but even this produces low-quality results. Four approaches to dithering—ordered dither, constrained average dithering, diffusion dither, and dot diffusion—are presented in this section. Another approach, called ARIES, is discussed in [Roetling 76] and [Roetling 77].
2.28.1 Ordered Dither The principle of this method is to paint a pixel B[i, j] black or leave it white, depending on the intensity of pixel A[i, j] and on its position in the picture [i.e., on its coordinates (i, j)]. If A[i, j] is a dark shade of gray, then B[i, j] should ideally be dark, so it is painted black most of the time, but sometimes it is left white. The decision whether to paint it black or white depends on its coordinates i and j. The opposite is true for a bright pixel. This method is described in [Jarvis et al. 76]. The method starts with an m×n dithering matrix Dmn which is used to determine the color (black = 1 or white = 0) of all the B pixels. In the example below we assume that the A pixels have 16 gray levels, with 0 as white and 15 as black. The dithering matrices for m = n = 2 and m = n = 4 are shown below. The idea in these matrices is
2.28 Dithering
110
Figure 2.74: A Familiar Black and White Image.
The giant panda resembles a bear, although anatomically it is more like a raccoon. It lives in the high bamboo forests of central China. Its body is mostly white, with black limbs, ears, and eye patches. Adults weigh 200 to 300 lb (90 to 140 kg). Low birth rates and human encroachment on its habitat have seriously endangered this species.
to minimize the amount of texture in areas with a uniform gray level.
D22 =
0 2 , 3 1
⎤ 0 8 2 10 ⎢ 12 4 14 6 ⎥ =⎣ ⎦. 3 11 1 9 15 7 13 5 ⎡
D44
The rule is: Given a pixel A[x, y] calculate i = x mod m, j = y mod n, then select black (i.e., set B[x, y] to 1) if A[x, y] ≥ Dmn [i, j] and select white otherwise. To see how the dithering matrix is used, imagine a large, uniform area in the image A where all the pixels have a gray level of 4. Since 4 is the fifth of 16 levels, we would like to end up with 5/16 of the pixels in the area set to black (ideally they should be randomly distributed in this area). When a row of pixels is scanned in this area, y is incremented, but x does not change. Since i depends on x, and j depends on y, the pixels scanned are compared to one of the rows of matrix D44 . If this happens to be the first row, then we end up with the sequence 10101010 . . . in bitmap B. When the next line of pixels is scanned, x and, as a result, i have been incremented, so we look at the next row of D44 , that produces the pattern 01000100 . . . in B. The final result is an area in B that looks like
2 Raster Graphics
111
10101010. . . 01000100. . . 10101010. . . 00000000. . . Ten out of the 32 pixels are black, but 10/32 = 5/16. The black pixels are not randomly distributed in the area, but their distribution does not create annoying patterns either. Exercise 2.28: Assume that image A has three large uniform areas with gray levels 0, 1, and 15 and calculate the pixels that go into bitmap B for these areas. Ordered dither is easy to understand if we visualize copies of the dither matrix laid next to each other on top of the bitmap. Figure 2.75 shows a 6×12 bitmap with six copies of a 4×4 dither matrix laid on top of it. The threshold for dithering a pixel A[i, j] is that element of the dither matrix that happens to lie on top of A[i, j].
Figure 2.75: Ordered Dither.
Matrix D44 above was created from D22 by the recursive rule Dnn =
4Dn/2,n/2 4Dn/2,n/2 + 3Un/2,n/2
4Dn/2,n/2 + 2Un/2,n/2 4Dn/2,n/2 + Un/2,n/2 ,
,
(2.9)
where Unn is an n×n matrix with all ones. Other matrices are easy to generate with this rule. Exercise 2.29: Use the rule of Equation (2.9) to construct D88 . The basic rule of ordered dither can be generalized as follows: Given a pixel A[x, y], calculate i = x mod m and j = y mod n, then select black (i.e., assign B[x, y] ← 1) if Ave[x, y] ≥ Dmn [i, j], where Ave[x, y] is the average of the 3×3 group of pixels centered on A[x, y]. This is computationally more intensive but tends to produce better results in most cases since it considers the average brightness of a group of pixels. Reference [Wolfram-dither 10a] illustrates ordered dither.
2.28 Dithering
112
Ordered dither is a simple, fast method, but it tends to create images that have been described by various people as “computerized,” “cold,” or “artificial.” The reason for that is probably the recursive nature of the dithering matrix.
2.28.2 Constrained Average Dithering In cases where high speed is not critical, constrained average dithering [Jarvis and Roberts 76] produces good results, although it requires more computations than or¯ j] of the dered dither. The idea is to compute, for each pixel A[i, j], the average A[i, pixel and its eight near neighbors. The pixel is then compared to a threshold of the form 2γ ¯ γ+ 1− A[i, j], M where γ is a user-selected parameter and M is the maximum value of A[i, j]. Notice that the threshold can have values in the range [γ, M − γ]. The final step is to compare A[i, j] to the threshold and set B[i, j] to 1 if A[i, j] ≥ threshold and to 0 otherwise. The main advantage of this method is edge enhancement. An example is Figure 2.76 which shows a 6×8 bitmap A, where pixels have 4-bit values. Most pixels have a value of 0, but the bitmap also contains a thick slanted line indicated in the figure. It separates the 0 (white) pixels in the upper-left and bottom-right corners from the darker pixels in the middle.
0 0 0 0 0 0
0 0 0 0 8 15 15 0 0 0 7 14 15 15 0 0 5 13 15 15 10 0 3 11 15 15 8 0 1 10 15 15 8 0 0 9 15 15 6 13 0 0
0 3 19 54 95 117 107 63 1 14 45 87 112 104 71 33 10 38 79 105 113 80 44 8
(a)
0 0 0
0 0 0
(b)
0 0 1
0 1 1
1 1 1
1 1 1 0 1 0
0 0 0
(c)
Figure 2.76: Constrained Average Dithering.
Figure 2.76a shows how the pixels around the line have values approaching the maximum (which is 15). Figure 2.76b shows (for some pixels) the average of the pixel and its eight near neighbors (the averages are shown as integers, so an average of 54 really indicates 54/9). The result of comparing these averages to the threshold (which in this example is 75) is shown in Figure 2.76c. It is easy to see how the thick line is sharply defined in the bilevel image. So I’m a ditherer? Well, I’m jolly well going to dither, then! —Roland Young (as Cosmo Topper) in Topper (1937).
2.28.3 Diffusion Dither Imagine a photograph, rich in color, being digitized by a scanner that can distinguish millions of colors. The result may be an image file where each pixel A[i, j] is represented
2 Raster Graphics
113
by, say, 24 bits. The pixel may have one of 224 colors. Now, imagine that we want to display this file on a computer that can only display 256 colors simultaneously on the screen. A good example is a computer using a color lookup table whose size is 3×256 bytes (Page 34 and Section 20.7). We begin by loading a palette of 256 colors into the lookup table (see Section 17.4 for a good method to select such a palette). Each pixel A[i, j] of the original image will now have to be displayed on the screen as a pixel B[i, j] in one of the 256 palette colors. The diagram below shows a pixel A[i, j] with original color (255, 52, 80). If we decide to assign pixel B[i, j] the palette color (207, 62, 86), then we are left with a difference of A[i, j] − B[i, j] = (48, −10, −6). This difference is called the color error of the pixel. R=48 R=255 R=207 − = G=−10 G=52 G=62 B=−6 B=80 B=86 Large color errors degrade the quality of the displayed image, so an algorithm is needed to minimize the total color error of the image. Diffusion dither does this by distributing the color errors among all the pixels such that the total color error for the entire image is zero (or very close to zero). The algorithm is very simple. Pixels A[i, j] are scanned line by line from top to bottom. In each line, they are scanned from left to right. For each pixel, the algorithm performs the following: 1. Pick up the palette color that’s nearest the original pixel’s color. This palette color is stored in the destination bitmap B[i, j]. 2. Calculate the color error A[i, j] − B[i, j] for the pixel. 3. Distribute this error to four of A[i, j]’s nearest neighbors that haven’t been scanned yet (the one on the right and the three centered below) according to the Floyd– Steinberg filter [Floyd and Steinberg 75] where the X represents the current pixel: X 7/16 3/16 5/16 1/16 Consider the example of Figure 2.77a. The current pixel is (255, 52, 80) and we (arbitrarily) assume that the nearest palette color is (207, 62, 86). The color error is (48, −10, −6) and is distributed as shown in Figure 2.77c. The five nearest neighbors are assigned new colors as shown in Figure 2.77b. The algorithm is shown in Figure 2.78a where the weights p1, p2, p3, and p4 can be assigned either the Floyd–Steinberg values 7/16, 3/16, 5/16, and 1/16 or any other values. The total color error may not be exactly zero because the method does not work well for the leftmost column and for the bottom row of pixels. However, the results can be quite good if the palette colors are carefully selected. This method can easily be applied to the case of a monochromatic display (or any bilevel (monochromatic) output device), as shown by the pseudo-code of Figure 2.78b, where p1, p2, p3, and p4 are the four error diffusion parameters. They can be the ones already given (i.e., 7/16, 3/16, 5/16, and 1/16) or different ones, but their sum should be 1.
2.28 Dithering
114 R=255 Before G=52 B=80 R=192 R=250 G=45 G=49 B=75 B=83
R=178 G=20 B=60 R=191 G=31 B=72
R=207 G=62 B=86 R=201 R=265 G=43 G=46 B=74 B=81 After
(a) 7 16 × 48 = 7 16 ×(−10) = 7 16 × (−6) =
21, −4, −3,
R=199 G=16 B=57 R=194 G=30 B=72
(b)
1 16 × 48 1 16 ×(−10) 1 16 × (−6)
=
3,
= −1, = 0,
5 16 × 48 = 5 16 ×(−10) = 5 16 × (−6) =
15, −3, −2,
3 = 16 × 48 3 ×(−10) = 16 3 16 × (−6) =
(c) Figure 2.77: Diffusion Dither.
for i := 1 to m do for j := 1 to n do begin B[i, j]:=SearchPalette(A[i, j]); err := A[i, j] − B[i, j]; A[i, j + 1] := A[i, j + 1] + err ∗ p1; A[i + 1, j − 1] := A[i + 1, j − 1] + err ∗ p2; A[i + 1, j] := A[i + 1, j] + err ∗ p3; A[i + 1, j + 1] := A[i + 1, j + 1] + err ∗ p4; end. (a) for i := 1 to m do for j := 1 to n do begin if A[i, j] < 0.5 then B[i, j] := 0 else B[i, j] := 1; err := A[i, j] − B[i, j]; A[i, j + 1] := A[i, j + 1] + err ∗ p1; A[i + 1, j − 1] := A[i + 1, j − 1] + err ∗ p2; A[i + 1, j] := A[i + 1, j] + err ∗ p3; A[i + 1, j + 1] := A[i + 1, j + 1] + err ∗ p4; end. (b) Figure 2.78: Diffusion Dither Algorithm. (a) For Color. (b) For Bilevel.
9, −2, −1.
2 Raster Graphics
115
Exercise 2.30: Consider an all-gray image, where A[i, j] is a real number in the range [0, 1] and it equals 0.5 for all pixels. What image B would be generated by diffusion dither in this case? Exercise 2.31: Imagine a grayscale image consisting of a single row of pixels where pixels have real values in the range [0, 1]. The value of each pixel p is compared to the threshold value of 0.5 and the error is propagated to the neighbor on the right. Show the result of dithering a row of pixels all with values 0.5. Error diffusion can also be used for color printing. A typical low-end inkjet color printer has four ink cartridges for cyan, magenta, yellow, and black ink. The printer places dots of ink on the page such that each dot has one of the four colors. If a certain area on the page should have color L, where L isn’t any of CMYK, then L can be simulated by dithering (see also Page 1256 and color Plate U.2). This is done by printing adjacent dots in the area with CMYK colors such that the eye (which integrates colors in a small area) will perceive color L. This can be done with error diffusion where the palette consists of the four colors cyan (255, 0, 0), magenta (0, 255, 0), yellow (0, 0, 255), and black (255, 255, 255) and the error for a pixel is the difference between the pixel color and the nearest palette color. A slightly simpler version of error diffusion is the minimized average error method. The errors are not propagated but rather calculated and saved in a separate table. When a pixel A[x, y] is examined, the error table is used to look up the errors E[x + i, y + j] already computed for some previously seen neighbors A[i, j] of the pixel. Pixel B[x, y] is assigned a value of 0 or 1 depending on the corrected intensity: A[x, y] +
1 ij
αij
αij E[x + i, y + j].
ij
The new error, E[x, y] = A[x, y] − B[x, y], is then added to the error table to be used for future pixels. The quantities αij are weights assigned to the near neighbors of A[x, y]. They can be assigned in many different ways, but they should assign more weight to nearby neighbors, so the following is a typical example: ⎛
⎞ 1 3 5 3 1 ⎝ α = 3 5 7 5 3 ⎠, 5 7 x − − where x is the current pixel A[x, y] and the weights are defined for some previously seen neighbors above and to the left of A[x, y]. If the weights add up to 1, then the corrected intensity above is simplified and becomes A[x, y] +
αij E[x + i, y + j].
ij
Floyd–Steinberg error diffusion generally produces better results than ordered dither but has two drawbacks, namely it is serial in nature and it sometimes produces annoying
116
2.28 Dithering
“ghosts.” Diffusion dither is serial since the near neighbors of B[i, j] cannot be calculated until the calculation of B[i, j] is complete and the error A[i, j] − B[i, j] has been distributed to the four near neighbors of A[i, j]. To understand why ghosts are created, imagine a dark area positioned above a bright area (for example, a dark sky above a bright sea). When the algorithm works on the last dark A pixels, a lot of error is distributed below, to the first bright A pixels (and also to the right). When the algorithm gets to the first bright pixels, they have collected so much error from above that they may no longer be bright, creating perhaps several rows of dark B pixels. It has been found experimentally that ghosts can be “exorcised” by scaling the A pixels before the algorithm starts. For example, each A[i, j] pixel can be replaced by 0.1+0.8A[i, j], which “softens” the differences in brightness (the contrast) between the dark and bright pixels, thereby reducing the ghosts. This solution also changes all the pixel intensities, but the eye is less sensitive to absolute intensities than to changes in contrast, so changing intensities may be acceptable in many practical situations. Reference [Wolfram-dither 10b] illustrates this type of dither.
2.28.4 Dot Diffusion This section is based on [Knuth 87], a very detailed article that includes thorough analysis and actual images dithered using several different methods. The dot diffusion algorithm is somewhat similar to diffusion dither, it also produces good quality, sharp bilevel images, but it is not serial in nature and may be easier to implement on a parallel computer. We start with the 8 × 8 class matrix of Figure 2.79a. The way this matrix was constructed will be discussed later. For now, we simply consider it a permutation of the integers (0, 1, . . . , 63), which we call classes. The class number k of a pixel A[i, j] is found at position (i, j) of the class matrix. The main algorithm is shown in Figure 2.80. The algorithm computes all the pixels of class 0 first, then those of class 1, and so on. Procedure Distribute is called for every class k and diffuses the error err to those near neighbors of A[i, j] whose class numbers exceed k. The algorithm distinguishes between the four orthogonal neighbors and the four diagonal neighbors of A[i, j]. If a neighbor is A[u, v], then the former type satisfies (u − i)2 + (v − j)2 = 1, while the latter type is identified by (u−i)2 +(v −j)2 = 2. It is reasonable to distribute more of the error to the orthogonal neighbors than to the diagonal ones, so a possible weight function is weight(x, y) = 3 − x2 − y2 . For an orthogonal neighbor, either (u − i) or (v − j) equals 1, so weight(u − i, v − j) = 2, while for a diagonal neighbor, both (u − i) and (v − j) equal 1, so weight(u − i, v − j) = 1. Procedure Distribute is listed in pseudo-code in Figure 2.80. Once the coordinates (i, j) of a pixel A[i, j] are known, the class matrix gives the pixel’s class number k that is independent of the color (or brightness) of the pixel. The class matrix also gives the classes of the eight near neighbors of A[i, j], so those neighbors whose classes exceed k can be selected and linked in a list. It is a good idea to construct those lists once and for all, since this speeds up the algorithm considerably. It remains to show how the class matrix, Figure 2.79a, was constructed. The main consideration is the relative positioning of small and large classes. Imagine a large class surrounded, in the class matrix, by smaller classes. An example is class 63, which is surrounded by the “lower classes” 43, 59, 57, 51, 60, 39, 47, and 55. A little thinking
2 Raster Graphics 34 48 40 32 29 15 23 31 42 58 56 53 21 5 7 10 50 62 61 45 13 1 2 18 38 46 54 37 25 17 9 26 28 14 22 30 35 49 41 33 20 4 6 11 43 59 57 52 12 0 3 19 51 63 60 44 24 16 8 27 39 47 55 36
(a)
117
34 48 40 32 29 15 23 31 42 58 56 53 21 50 62 61 45 13 18 38 46 54 37 25 17 26 28 14 22 30 35 49 41 33 20 11 43 59 57 52 12 19 51 63 60 44 24 16 27 39 47 55 36
34 48 40 32 29 23 31 42 58 56 53 21 50 62 61 45 38 46 54 37 25 26 28 22 30 35 49 41 33 43 59 57 52 51 63 60 44 24 27 39 47 55 36
(b)
(c)
34 48 40 32 42 58 56 53 50 62 61 45 38 46 54 37 35 49 41 33 43 59 57 52 51 63 60 44 39 47 55 36
(d)
Figure 2.79: 8×8 Matrices for Dot Diffusion.
for k := 0 to 63 do for all (i, j) of class k do begin if A[i, j] < .5 then B[i, j] := 0 else B[i, j] := 1; err := A[i, j] − B[i, j]; Distribute(err, i, j, k); end. procedure Distribute(err, i, j, k); w := 0; for all neighbors A[u,v] of A[i,j] do if class(u, v) > k then w := w+weight(u − i, v − j); if w > 0 then for all neighbors A[u, v] of A[i, j] do if class(u, v) > k then A[u, v] := A[u, v] + err×weight(u − i, v − j)/w; end; Figure 2.80: The Dot Diffusion Algorithm.
shows that as the algorithm iterates toward 63, more and more error is absorbed into pixels that belong to this class, regardless of their brightness. A large class surrounded by lower classes is therefore undesirable and may be called a “baron.” The class matrix of Figure 2.79a has just two barons. Similarly, “near-baron” positions, which have only one higher-class neighbor, are undesirable and should be avoided. Our class matrix has just two of them. Exercise 2.32: What are the barons and near-barons of our class matrix? Exercise 2.33: Consider an all-gray image where A[i, j] = 0.5 for all pixels. What image B would be generated by dot diffusion in this case? Another important consideration is the positions of consecutive classes in the class matrix. Figure 2.79b,c,d shows the class matrix after 10, 21, and 32 of its lowest classes have been blackened. It is easy to see how the black areas form 45◦ grids that grow and eventually form a 2×2 checkerboard. This helps create diagonal, rather than rectilinear dot patterns in the bilevel array B, and we know from experience that such patterns are less noticeable to the eye. Figure 2.81a shows a class matrix with just one baron and
2.29 Stippling
118
25 21 13 39 47 57 53 45 48 32 29 43 55 63 61 56 40 30 35 51 59 62 60 52 36 14 22 26 46 54 58 44 16 6 10 18 38 42 50 24 8 0 2 7 15 31 34 20 4 1 3 11 23 33 28 12 17 9 5 19 27 49 41 37
(a)
14 13 1 2 4 6 11 9 0 3 15 12 10 8 5 7
(b)
Figure 2.81: Two Class Matrices for Dot Diffusion.
one near-baron, but it is easy to see how the lower classes are mostly concentrated at the bottom-left corner of the matrix. A close examination of the class matrix shows that the class numbers in positions (i, j) and (i, j + 4) always add up to 63. This means that the grid pattern of 63 − k white pixels after k steps is identical to the grid pattern of 63 − k black pixels after 63 − k steps, shifted right four positions. This relation between the dot pattern and the diffusion pattern is the reason for the name dot diffusion. Exercise 2.34: Figure 2.81b shows a 4×4 class matrix. Identify the barons, near-barons, and grid patterns. Experiments with the four methods described in this section seem to indicate that the dot diffusion method produces best results for printing because it tends to generate contiguous areas of black pixels, rather than “checkerboard” areas of alternating black and white. Modern laser and ink-jet printers have resolutions of 600 dpi or more, but they generally cannot produce a high-quality checkerboard of 300 black and 300 white alternating pixels per inch.
2.29 Stippling Stippling is the process of generating a random pattern of black dots that we perceive as a grayscale image. Stippled images are found in nature and are sometimes painted by artists manually with a pen or brush. The pen and ink drawing of locomotive 1356 (Figure 2.82, courtesy of H. L. Scott) is a beautiful example. Stippling as an art form may include strokes in addition to dots. Manual stippling is an attempt to direct the viewer’s attention to important features and details, to simplify features, and to expose features that are otherwise hidden. When done properly, stippling can selectively include and enhance certain details that make the stippled image more expressive than a photograph. In computer graphics and image processing, stippling may be used instead of halftoning and dithering, because it is fast and because it avoids the annoying artifacts that often result from the regular halftone patterns. Pointillism (Section 2.1) also works by spreading small dots, and some experts claim that pointillism and stippling are the same. Others maintain that stippling employs black
2 Raster Graphics
119
Figure 2.82: A Hand-Drawn Stippled Image (Courtesy H. L. Scott).
dots on a white background to create a grayscale image, while pointillism is based on colored dots placed close together, so that their colors are combined and averaged in the eye. Other meanings of stippling. A random pattern of small depressions made in a surface to increase friction and provide better grip. This is also referred to as knurling or checkering. The damage caused by spider mites. They leave tiny but growing white spots on the leaves such that eventually the entire leaf appears silvery. In medicine, the term stippling refers to the circular pattern of dots that appear around a gunshot wound when the bullet was fired from close range. The stippling method described here is simple and intuitive. Given a grayscale image, where each pixel is a number between 0 and 255, the idea is to select certain pixels in the original, grayscale image at random and change them to black, while changing all the other pixels to white. More pixels should be selected in dark regions of the image, which suggests the following approach. Scan the image pixel by pixel. For each pixel P(i, j), draw a random number R in the interval [0, T ] (where T is a user-controlled threshold parameter) and compare it to the gray intensity of the pixel. If P(i, j) > R, paint the pixel black, otherwise change it to white. (Notice that in the Mathematica code of Figure 2.83, the condition is “<” instead of “>” because in the test files, 255 was the value of white and 0 was the value of black. Also, the rows of the bitmap were in reverse order, which is why the code has stp[[row+1-i,j]]=1 instead of stp[[i,j]]=1.) This method makes sense because (1) the black pixels are randomly distributed and (2) in dark regions of the image, where many gray pixels have large values, more pixels are selected. Figure 2.83 shows an original grayscale image and two stippled versions, a dark version where 70% of the pixels were chosen, and a brighter version, with a threshold of 150, where only 57% of the pixels were chosen. Notice that reversing the relational btmp[[i,j]]
RandomInteger[150] results in a stippled negative of the original image. This simple stippling method gives good results, but can be improved, at least for
120
2.29 Stippling
(* Stippling an image *) ar=Import["A1965.jpg"]; (*Input grayscale image*) d=ImageData[ar]; btmp=255 d[[All,All,1]]; (* Grayscale values in [0,255] *) {row,col}=Dimensions[btmp] stp=Table[0,{i,1,row},{j,1,col}];(*Init stp to zeros*) Do[If[btmp[[i,j]]
Glory be to God for dappled things, For skies of couple-colour as a brinded cow, For rose-moles all in stipple upon trout that swim. —Gerard Manley Hopkins.
Figure 2.84: Uniform, Sobol, and Niederreiter-Distributed Points.
2 Raster Graphics
121
some grayscale images, by using quasi-random numbers instead of uniformly-distributed random numbers. Quasi-random numbers and random numbers in general are described in Section 2.30. Specifically, the Sobol sequence of quasi-random numbers (Figures 2.84 and 2.86) seems promising because of the following property it possesses. Points with coordinates drawn from this sequence fill up the plane in order and at increasingly finer resolutions. Thus, the first 10% of the points are uniformly distributed in the plane, the next 10% locate themselves evenly in between the spaces left by the first 10%, and so on. When such points are plotted, the graph looks both random and structured. Next, a completely different, physics-based approach to stippling is described. The algorithm starts by injecting imaginary tiny electrically-charged particles into the center of the original, grayscale image. The electrical charges are identical, so the particles start repelling each other. At any given time, each particle is located over one of the original grayscale pixels. The particle checks the shade of gray of its pixel. If the shade is dark, the particle reduces its electrical charge, which allows other particles to get near it and thereby increase the particle density in its neighborhood. If the shade is bright, the particle increases its charge, which repels its near neighbors and results in a bright region around the particle. The number of particles injected by the algorithm is a percentage of the size of the image. In order to end up with a bright stippled image, the number of particles should be about 50–60% of the number of pixels, while for a dark stippled image, 70–80% seems the right amount. The main part of the algorithm is a loop where a new position for each particle is computed in each iteration. When an iteration is reached where no particle moves more than one step (i.e., from a pixel to its next neighbor), the stippling pattern is considered stable and the loop terminates. The electrical force between two charged particles is inversely proportional to the square of their distance, but in order to speed up this algorithm, the following, simplified expression is used to compute the force acting on particle i F=
vij j
dij
,
where vij is the unit vector from particle i to particle j and dij is the distance between these particles. The sum is taken over all particles j where the distance dij is less than a user-controlled threshold parameter. The result of this sum is a force, a vector quantity which has direction and magnitude. Once F has been computed, its magnitude is rounded to the nearest integer G and particle i is moved G pixels in the direction of F. Reference [roberthodgin 10] employs such an algorithm and obtains results that look pleasing to the eye and where no annoying patterns or artifacts can be discerned. It is possible to create a random-stippled image in Adobe Photoshop. Experiment with the following steps: 1. Convert the color image to grayscale (Image/Mode/Grayscale). 2. Set to high contrast (Image/Adjustments/Brightness/Contrast). 3. Blur it somewhat (Filters/Blur/Surface blur).
122
2.30 Random Numbers
4. Create the random dots (Image/Mode/Bitmap). Select diffusion dither and try resolutions of 100, 200, and 300. Figure 2.85 shows an image in color and grayscale, followed by three typical stippled versions.
Figure 2.85: A Stippled Image in Photoshop.
2.30 Random Numbers The algorithm of Section 2.29 employs random numbers, and these numbers are used in many areas of computer graphics and image processing. This section is a short survey of this interesting topic. An individual number is neither random nor non-random. Only a sequence of numbers can be random, and only if it passes certain statistical tests. These tests are necessary but unfortunately not sufficient. A sequence that does not pass these tests is not random, but a sequence that passes them is not necessarily random. Random numbers pose an interesting dilemma, they are needed in many areas of computing, but computers and computations are deterministic. A computer works by executing a program instruction by instruction, without leaving any room for chance. Each time a program is executed with the same input data, the results are the same. Random numbers, however, are, well, random. How do we include an element of randomness in a computer program? Exercise 2.35: The statement above is not completely accurate. There are cases where a program that’s repeatedly executed with the same input data produces different results. What are those cases? The trick is to develop an algorithm that will produce a sequence of numbers that passes the statistical tests. Such a sequence is not random, because each time the program is executed, it produces the same sequence, but the resulting sequence can be used as random. Such a sequence is called pseudo-random.
2 Raster Graphics
123
2.30.1 Statistical Tests for Randomness Here is a list of some important statistical tests for randomness, mostly without explanations. 1. The entropy of a random sequence should be at its maximum. 2. A random sequence must be incompressible. 3. A Chi-Square Test. 4. Arithmetic Mean. In a random sequence of numbers in the interval [a, b], the average should be (a + b)/2. Any other values indicate a trend and therefore a lack of randomness. 5. Monte Carlo value for π. Imagine a square, 2 units on a side, with a unit circle inscribed in it. The area of the square is 4 and the area of the circle is π r 2 = π. The ratio of the areas is π/4 ≈ 0.7854. We draw two random numbers, each in the interval [−1, +1] and interpret them as the (x, y) coordinates of a point in the square. The point will be in or on the circle if x2 + y2 ≤ 1. The chance that the point will be inside the circle is the ratio of the areas. If we draw many pairs of random numbers, we will find that approximately 78.5% of them are in the circle. This can serve as a test of randomness. 6. Serial Correlation Coefficient. This statistical quantity measures the extent to which each item in the sequence depends on its predecessor. If the sequence is random, this measure yields zero. 7. Number of runs of identical digits. Statistics shows that in a sequence of 1000 decimal digits, we can expect about 90 runs of identical digits (pairs, triplets, and so on). A good algorithm for pseudo-random numbers depends on the value of a seed. Before running this algorithm, the user should assign a value to the seed, and this value determines the numbers generated by the algorithm. True random numbers can be generated outside the computer from natural processes that are believed to be random. Examples of such processes are radioactive decay, static noise heard in a radio receiver tuned between stations, and atmospheric noise (the latter is discussed in http://www.random.org/). It is also possible to employ mathematics to generate true random numbers. The decimal expansion of any irrational number is infinite and nonrepeating. Such an expansion may be used as a source of random numbers √ (digits, pairs of digits, or even longer numbers). A single irrational number, such as π, 2, or e, may provide all the random numbers a user may ever need. In addition to pseudo- and true random numbers, there are also quasi-random numbers (sometimes referred to as low-discrepancy number sequences). When a sequence of such numbers is plotted, the points seem to be distributed regularly, but the sequence is not random because its members are not statistically independent. Quasi-random numbers can be considered a compromise between pseudo-random numbers (which are random) and a grid of equally-spaced points (which are easy to generate, but are useless as random numbers). The following sheds some light on the nature of quasi-random numbers. When uniform random numbers are drawn in the interval [0, 1), each new number generated has the same probability of falling in any subinterval. The next random number has
2.30 Random Numbers
124
a chance of 1/3 of being in any of the subintervals [0, 1/3), [1/3, 2/3), and [2/3, 1) regardless of any past numbers in the sequence. Thus, for example, in the rare case where n consecutive numbers happen to fall in subinterval [1/3, 2/3), the next number drawn still has probability 1/3 of falling in any of the three subintervals. In contrast, the next number drawn in a quasi-random sequence depends on its predecessors, so it has different probabilities of falling in various subintervals. The most well-known quasi-random sequences are termed Niederreiter, Sobol, and Faure, after their developers. The mathematics of quasi-random numbers is outside the scope of this book, but is easy to find, for example in [Sobol-wiki 10]. Figure 2.86 illustrates the Sobol sequence in two-dimensions. Each element in this sequence is a pair of numbers and is plotted as a point. It is obvious that the point distribution is regular and thus non-uniform, but it is also easy to see that the first 100 points fill up the entire space in a regular way, the next 900 points fit regularly in-between the first 100, and so on.
100
1000
10,000
Figure 2.86: Three Densities of Quasi-Random Sobol Sequences.
2.30.2 Random Points in a Circle As an added example, we discuss how to convert a uniform distribution of random numbers to a ramp distribution. This problem arises when we consider how to distribute random points uniformly in a circle. Given a circle of radius R centered on the origin, we can generate random points by drawing pairs of random numbers, each in the interval [−R, R]. The Mathematica statement P=Table[{Random[Real,{-R,R}], Random[Real,{-R,R}]},{n}]; generates n such points. However, about 32% of the points will fall outside the circle, because the circle is the set of points (x, y) that satisfy x2 +y 2 = R2 (see also Figure 7.3). Following are two ways to generate random points inside the circle: Proceed as above, but disregard any point that does not satisfy x2 + y 2 = R2 . Generate pairs of random numbers (θ, r) where θ is in the interval [0, 2π] and r ∈ [0, R]. For each pair, plot the point (r cos θ, r sin θ) which will lie in the circle.
2 Raster Graphics
125
(* Place random points in a circle *) n=300; R=10; theta=Table[Random[Real, {0,2 Pi}], {n}]; r=Table[Random[Real, {0,R}], {n}]; (* uniform distribution *) P=Point[ Table[{r[[i]] Cos[theta[[i]]], r[[i]] Sin[theta[[i]]]}, {i,1,n}]]; (* points are denser toward the center *) Show[ Graphics[{Green, P}], Graphics[{Blue, Circle[{0, 0}, 10]}], Graphics[{Red, Point[{0, 0}]}], AspectRatio->1] (* Now give r a ramp distribution *) r=Table[Max[Random[Real, {0,R}], Random[Real, {0,R}]], {n}]; P = Point[ Table[{r[[i]] Cos[theta[[i]]], r[[i]] Sin[theta[[i]]]}, {i,1,n}]]; (* points are uniformaly distributed *) Show[Graphics[{Green, P}], Graphics[{Blue, Circle[{0,0},10]}], Graphics[{Red, Point[{0,0}]}], AspectRatio->1] Figure 2.87: Two Distributions of Random Points in a Circle.
The left part of Figure 2.87 shows the results of the latter method. Surprisingly, the 300 points are not distributed uniformly but are denser toward the center. The righthand part of the figure shows points that are uniformly distributed. An examination of the code verifies that the only difference between the two parts is the way the random r numbers are generated. The statement Max[Random[Real,0,R], Random[Real,0,R]] results in a random sequence of numbers with a ramp distribution, which causes the points to be distributed uniformly. Here is why. Assume that both θ and r are uniformly distributed. Assume further that the 300 random numbers that we draw for r can take only 10 values, r1 through r10 , between 0 and R and similarly the 300 numbers for θ are limited to 10 values θj between 0 and 2π. For each ri there are now about 30 points with various θj , and they are arranged in a circle of radius ri . This explains why inner circles (closer to the center)
126
2.31 Image Processing
are denser than outer circles. It is obvious that the 300 random numbers for r should have more large values than small values, i.e., r should be generated with a ramp distribution. If among the 300 numbers there are m random numbers with r = 0.15, then there should be 2m numbers with r = 0.30. It remains to understand how the Max function converts the distribution of r from uniform to ramp. The code above shows that we draw a pair of random numbers a and b distributed uniformly in the interval [0, R], and select the larger. Once the first number, a, is drawn, it partitions the interval [0, R] into two subintervals [0, a] and [a, R]. In order for a to be larger than the next number b, the latter has to be located in [0, a] and the probability of that is the ratio a/R of intervals. Thus, the probability of selecting a is a/R, proportional to a. With such probability, larger values are selected more often than smaller values. See also Section 22.6 for the Gaussian distribution of random numbers. One might have thought that from traditional mathematics and statistics there would long ago have emerged some standard definition of randomness. But despite occasional claims for particular definitions, the concept of randomness has in fact remained quite obscure. And indeed I believe that it is only with the discoveries in this book that one is finally now in a position to develop a real understanding of what randomness is. —Stephen Wolfram, A New Kind of Science (2002).
2.31 Image Processing The rapid development of computer graphics (and related hardware such as scanners and digital cameras) in the last four decades has created a flood of images. As a result, the field of image processing has grown side by side with computer graphics. Image processing is a vast set of techniques that input an image and output either another image or a set of characteristics or parameters related to the input image. Many of those techniques modify a given image in ways that make it more useful or more interesting. Many times, an image taken by a satellite needs to be sharpened or painted with false colors. On the other hand, an artist may want to take a sharply focused photograph and intentionally blur it, or make it look as if it was originally painted by watercolors (Plates E.3, E.4, H.2, and H.4), or make it resemble an image embossed on paper—in order to achieve interesting, beautiful, or useful effects. The input image to be processed is a bitmap, with 1 or more bits per pixel. The processing software must be given the three dimensions of the bitmap (number of rows, number of columns, and number of bits per pixel) and it normally creates the new image in another bitmap, pixel by pixel. A typical image processing algorithm consists of a loop that iterates over all the pixels of the image, processing each in the same way. The original pixel stays in the original bitmap (because its value may be needed to process neighboring pixels) and the newly computed pixel is stored in the new bitmap.
2 Raster Graphics
127
The techniques described here produce very different results, but most are based on the same principle. The principle is to define a small matrix of values, called a convolution kernel, to place it centered on the current pixel, to multiply the value of the pixel and the values of its neighbors by the values in the kernel, to sum the results, and to store the sum in the new bitmap, as the value of the newly computed pixel. If the color of a pixel is specified by means of three numbers (normally the red, green, and blue components), then this process is applied separately to each of the three components of the current pixel. Blurring: This is achieved by the convolution kernel of Figure 2.88b. The values of the current pixel and eight of its nearest neighbors are multiplied by the weights shown and the products are added. The result is a color value that still has 20% of the old pixel value but has contributions of 8% and 12% from neighboring pixels. Note that the weights add up to 1. As an example, suppose that the current pixel is the center pixel of Figure 2.88a. The calculation is 5×0.8 + 5×0.12 + 8×0.8 + 19×0.12 + 5×0.20 + 8×0.12 + 11×0.8 + 1×0.12 + 8×0.8 = 30.56, yielding a value of 31 for the new pixel. Note that more blurring can be achieved by having a 5×5 convolution kernel (the weights should add up to 1) which will spread the color of a pixel to 24 of its neighbors. (See also Exercise 25.9.)
5 5 8 19 5 8 11 1 8 (a)
.8 .12 .8 .12 .20 .12 .8 .12 .8 (b)
0 −1 0 −1 5 −1 0 −1 0 (c)
−1 0 0 0 0 0 0 0 1 (d)
0 0 0 10 10
0 4 1 3 1
4 4 8 7 7
4 4 8 8 9
4 6 6 6 5
(e)
Figure 2.88: Image Processing Techniques.
Sharpening: Sharpening, which often seems a miracle, is obtained by the convolution kernel of Figure 2.88c. The weights again add up to 1, but the negative weights magnify any contrasts between the original pixels. More sharpening may be achieved by repeating the process on the new bitmap. Embossing: This is achieved by the convolution kernel of Figure 2.88d. Note that the weights here add up to 0. To understand how this works, we should think of pixels along an edge, as opposed to pixels away from an edge. Pixels located away from an edge tend to be similar and we can call them “background” pixels. The convolution kernel of Figure 2.88d sets such pixels to 0 or close to 0. In contrast, if the current pixel is part of an edge that goes from bottom-left to top-right (if it is a nonbackground pixel), then its two diagonal neighbors should have different colors and our kernel will create a new nonzero pixel.
128
2.31 Image Processing
Our kernel creates a white background since it sets all background pixels to 0. Visually, it is better to have a medium gray background, and this is easily achieved by adding 128 to each pixel generated. Background pixels will now have a value of 128 (or close to 128) and nonbackground ones will have any values. If they have a value greater than 255, it can be truncated to 8 bits (calculated modulo 256). Note that the embossing kernel can be written in a number of ways. All that is needed are the numbers 1 and −1 in opposite corners. Different kernels create the effect of the light hitting the embossed picture from different directions. Watercoloring: An image can be modified to look as if it has been painted in watercolor by examining a group of neighbors centered on the current pixel and replacing the original value of the pixel by the median of the group. Assuming that the current pixel is the center one in Figure 2.88e, we sort the values in the group of 5×5 neighbor pixels to obtain 0, 0, 0, 0, 1, 1, 3, 4, 4, 4, 4, 4, 4, 5, 6, 6, 6, 7, 7, 8, 8, 8, 9, 10, 10. The median value is 4 (since there are 12 smaller values and 12 greater ones). Our center pixel of 8 is therefore replaced, in the new bitmap, by 4. If the result is too soft, it can later be sharpened. Gaussian Blur: A linear blur assigns lighter and lighter weights to pixels that are farther away from the center. A blurred pixel may be assigned a% of its original value and contributions from its neighbors that go down linearly to (a − b)%, (a − 2b)%, (a − 3b)%, and so on, for farther away neighbors. Thus, if a = 24% and b = 2%, then a blurred pixel will retain 24% of its value and will be assigned 22%, 20%, 18%, and 16% of its neighbors at distances of 1, 2, 3, and 4 units. Notice that the percentages add up to 100. Gaussian blur (also known as Gaussian smoothing) is nonlinear. It may blur a pixel by assigning it 22% of its original value, but only 11% of each of its nearest neighbors, 1.3% of each neighbor at distance 2, 0.03% of each neighbor at distance 3, and so on, down to 0.000067% for each neighbor at distance 6. The percentages have to add up to 100, so the only user-controlled parameter is the maximum distance. Figure 2.89 shows four examples of Gaussian blur (with maximum distances of 10, 20, 30, and 40 pixels) and two linear blurs, one of them radial. We know from experience that linear blur results in an image that seems unfocused (this is also called the bokeh effect). Gaussian blur, on the other hand, reduces the higher frequencies of the image. It has the effect of a low-pass filter and it results in an image that seems to be viewed through translucent glass. Bokeh (or boke) is a term referring to the quality of the out-of-focus (blurred) parts of an image. The amount of bokeh in an image is not well-defined. The bokeh of a photo is determined by certain characteristic of the lens, such as its aperture, the circles of confusion (Section 26.4.7), and how far out-of-focus the lens is. The mathematics of Gaussian blur is based on the well-known Gaussian distribution (or bell curve, Section 22.6), but in two dimensions. We start with the two-dimensional Gaussian distribution, select a value for the standard deviation σ, and compute the
2 Raster Ra aster Graphics Grap phics
10
20
129
30
Linear 40
Figure 2.89: Gaussian and Linear Blurrings.
expression G(x, y) =
2 x + y2 1 exp − , 2πσ2 2σ 2
for several neighbor pixels distributed symmetrically around the current pixel. For example, x and y may each vary seven steps from −3 to 3, producing a 7 × 7 array of Gaussian coefficients. This array is normalized such that the sum of all 49 coefficients is 1, as shown in Figure 2.90, and the current pixel is replaced by the weighted sum of itself and its 48 near neighbors, each multiplied by the corresponding Gaussian coefficient (i.e., it is convolved with a Gaussian distribution). This is done for every pixel in the image (or in the region to be blurred) and the new pixels then replace the original ones. Notice how the elements of the kernel are normalized by dividing them by their sum, so that their new sum is 1. Exercise 2.36: Why is the normalization necessary?
2.31.1 An Alternative Approach It is possible to process images by means of the fundamental relation (1 − t)P0 + tP1 (this is the all-important Equation (9.1)). The idea is to blend two images (Section 8.5) by interpolating (i.e., using 0 ≤ t ≤ 1) or by extrapolating (using t < 0 or t > 1) them. Values t > 1 subtract part of P0 while scaling P1 . Negative values of t do the reverse. The examples below show how a general image P0 can be blended with a special image P1 (a mask) to obtain the following useful results: Brightness: We select a bitmap of all black as the mask P1 . Interpolation darkens the image while extrapolation brightens it. The original image is obtained for t = 0. Contrast: We compute the average intensity I of all the pixels in the original image. We build the mask as a gray bitmap where every pixel has value I. Interpolation (0 ≤ t ≤ 1) reduces contrast, while extrapolation increases it. Negative values of t generate inverted images. The average intensity of the final image is always I.
2.31 Image Processing
130 0.00000067 0.00002292 0.00019117 0.00038771 0.00019117 0.00002292 0.00000067
0.00002292 0.00078633 0.00655965 0.01330373 0.00655965 0.00078633 0.00002292
0.00019117 0.00655965 0.05472157 0.11098164 0.05472157 0.00655965 0.00019117
0.00038771 0.01330373 0.11098164 0.22508352 0.11098164 0.01330373 0.00038771
0.00019117 0.00655965 0.05472157 0.11098164 0.05472157 0.00655965 0.00019117
0.00002292 0.00078633 0.00655965 0.01330373 0.00655965 0.00078633 0.00002292
0.00000067 0.00002292 0.00019117 0.00038771 0.00019117 0.00002292 0.00000067
(* Gaussian Kernel (normalized) *) sigma=0.84089642; sigmat=2.sigma^2; cc=1/sigmat Pi; gausskernel[x_, y_]:=cc E^(-(x^2+y^2)/sigmat); GC=Table[gausskernel[x,y], {x,-3,3}, {y,-3,3}]; GC=GC/Total[Flatten[GC]] (* Normalize *) Plot3D[gausskernel[x,y], {x,-3,3}, {y,-3,3}, PlotRange->All] Figure 2.90: Gaussian Kernel.
Saturation: We first compute the luminance of every pixel in P0 and set the corresponding pixel in the mask P1 to a shade of gray with that luminance. The mask is then used to change the luminance of every pixel in the original image P0 . Interpolation decreases saturation, while extrapolation increases it. Negative t also inverts the hue of image P0 . Sharpening: Section 3.14 discusses convolution. Sharpening and blurring are examples of convolutions. If the mask is a blurred version of the original, then interpolation blurs the original and extrapolation sharpens it. For more information, see [Haeberli and Voorhies 94].
2 Raster Graphics
131
2.32 The Hough Transform A bubble chamber is a container filled with liquid hydrogen (or any superheated transparent liquid). For many years, bubble chambers were used as the main instrument for detecting electrically-charged elementary particles moving through it. It was invented in 1952 by Donald A. Glaser. A nuclear reaction would be initiated by accelerating particles and letting them collide, and the results were photographed through a glass window. A bubble chamber photograph may be very complex (Figure 2.91) and may include hundereds of curves and spirals. Certain elementary particles are elusive and are created only rarely. Thus, very often thousands of photographs had to be taken and analyzed in order to discover such a particle, which is why the analysis had to be automated.
Figure 2.91: A Bubble Chamber Photograph.
In 1962, Paul Hough introduced a method, based on a simple transform, that became the basis for analyzing bubble chamber photographs and can be used to detect lines and curves in digital images. Reference [Hough 62] is the original patent application, reference [Hough 10] is a general introduction to this transform, and [Hart 09] is the history of the Hough transform. In modern computer graphics, the Hough transform is used to detect patterns in images. We first describe how this method is employed to detect straight lines, and then show how to extend it to detect arbitrary parametric curves. To understand the problem, let’s consider the two straight segments y = x and y = x/3 + 4 displayed on an 11 × 11 grid, where 0 ≤ x, y ≤ 10. Looking at the grid, it is easy for a person to realize that it features two lines. Identifying the two lines by software, however, is much more difficult, because all that the software can “see” is the 121-bit bitmap 00000000001 00000000010 00000000100 00000001111 00000111000 00111100000 11001000000 00010000000 00100000000 01000000000 10000000000 (without the spaces), which corresponds to the pixels shown in Figure 2.92. How can a program find the equations of the lines, or even discover the fact that there are two lines, from the pixels in the bitmap? In practical cases, there may be many lines in the bitmap,
132
2.32 The Hough Transform
with missing pixels and with some pixels at slightly wrong positions, complicating the problem even further. O 10 9 O 8 O 7 x 6 O 5 O 4 O 3 O 2 O 1 O 0 0 1 2 3 4 5 6 7 8 9 10 O
Figure 2.92: An 11×11 Bitmap.
The explicit equation of a straight line is y = ax+b, where a is the slope and b is the y intercept. For a given line, the parameters a and b are fixed and the coordinates x and y vary. The principle of the Hough transform is to reverse the roles of the parameters and the coordinates. Given a pair of coordinates (x0 , y0 ), the transform calculates all the possible values of (a, b) that satisfy y0 = a x0 + b. In other words, given a point (x0 , y0 ), the transform computes all the pairs (a, b) of straight lines that pass through the point. When these pairs are plotted in the ab plane, they define a line, because the values of each pair are linearly related. The ab plane is called the parameter space, to distinguish it from the image space (the xy plane) where the pixels actually exist. Example: Given the point (x0 , y0 ) = (10, 10), we calculate the 11 pairs (a, b), where b = 0, 1, . . . , 9, 10. From 10 = 10a + b we obtain a = (10 − b)/10. The 11 pairs are (1, 0), (0.9, 1), (0.8, 2), (0.7, 3),. . . ,(0, 10). Since there can be infinitely many straight lines passing through any given point, we have to limit the calculation to a subset of these lines (i.e., the values of a and b have to be “quantized”). We may decide to calculate only (a, b) values that are integers in a certain range or that have just one digit to the right of the decimal point. For example, we may limit the calculation to the set of quantized values a = 0, 0.1, . . . , 0.9, 1 and b = 0, 1, . . . , 9, 10 (lines with 11 slopes between 0◦ and 45◦ and 11 intercept values between 0 and 10). Each parameter can take on 11 values. The pairs (a, b) being calculated are accumulated in a two-dimensional array ab of integers in memory, whose rows and columns correspond to values of a and b. The array should be large enough for all the possible values of pairs (a, b). In the above example, we need an array of 11×11 integers. All array elements are initially cleared. Each time a pair (a, b) is calculated, the corresponding array element is incremented by 1. The algorithm works by scanning the bitmap, looking for bits of 1. For each 1-bit found, the following three steps are performed: 1. Determine the coordinates (x, y) of the pixel. 2. Compute all the quantized pairs (a, b) for point (x, y). 3. For each pair (a, b), increment array location ab[a,b] by 1.
2 Raster Graphics
133
Exercise 2.37: Assuming that the 121 bits of the bitmap are indexed 0 through 120 and given the row and column numbering of the pixels in Figure 2.92, figure out the row and column numbers of the pixel at bitmap index i. The Hough algorithm is a transform because it transforms points (x, y) in the image space (i.e., pixels) to points (a, b) in the parameter space (i.e., lines). It has the following properties: 1. A single point in image space is transformed to many points in the parameter space. Those points are on a line, so each image point is transformed to a line in the parameter space. 2. Any pair (a, b) defines the unique line y = ax + b, so any point in parameter space corresponds to a line in image space. 3. Imagine two points (x1 , y1 ) and (x2 , y2 ) on the line y = mx + n in image space. Many lines go through each point, so each will cause many elements of array ab to be incremented. Specifically, line y = mx + n goes through both points, so each point will cause array element ab[m,n] to be incremented. This element will therefore be incremented twice. In our example, there are 11 pixels on the line y = x (for which a = 1, b = 0), so array element ab[1,0] will be incremented 11 times. Similarly, line y = x/3 + 4 will cause element ab[1/3,4] to be incremented 11 times. Any other elements of ab will be incremented fewer than 11 times. 4. Imagine two points (a1 , b1 ) and (a2 , b2 ) in parameter space. Each of them defines a line in image space. Imagine that these two lines intersect at point (x0 , y0 ). All points (a, b) between (a1 , b1 ) and (a2 , b2 ) also define lines that intersect at (x0 , y0 ). Property 3 above is the important one. It means that the two array elements ab[1,0] and ab[1/3,4] will have the largest counts. Identifying the two lines is therefore reduced to scanning array ab and finding the elements with the largest counts. Each such element ab[m,n] tells us that the line y = mx + n exists in the image. The first 1-bit is found in the bitmap at index 10. It therefore corresponds to the pixel at location (10, 10). The parameter pairs for this point satisfy 10 = 10a + b or a = (10 − b)/10. The 11 pairs are therefore (1, 0), (0.9, 1), (0.8, 2), (0.7, 3),. . . , (0, 10). Exercise 2.38: The next 1-bit is found in the bitmap at index 20, so it corresponds to the pixel at location (9, 9). Calculate the parameter pairs for this point. Using the slope and y-intercept as parameters has the disadvantage that both can grow without a limit. A better set of parameters, the normal parameters, is shown in Figure 2.93. Parameter α is the angle between the x axis and the normal to the line (0◦ ≤ α < 180◦ ) and parameter β is the distance of the line from the origin (0 ≤ β ≤ ∞). The straight line itself has the equation x cos α + y sin α = β, but if we consider α and β to be the variables, instead of x and y, we end up with the function F (α, β) = x cos α + y sin α − β, a sinusoidal. Each point (x, y) in the image space is therefore transformed into a sinusoidal in the parameter space. However, when several points on a straight line are transformed into sinusoidals, all the sinusoidals intersect at one point. The four properties above also hold, but have to be rephrased as follows: 1. A point (x, y) in the image space is transformed to a sinusoidal curve in the parameter space.
134
2.32 The Hough Transform y
Li β
ne
x
Figure 2.93: The Normal Parameters for a Line.
2. A point (α, β) in the parameter space corresponds to a straight line in the image space. 3. Points lying on a straight line in the image space correspond to sinusoidals that intersect at one point in the parameter space. 4. Points lying on a sinusoidal curve in the parameter space correspond to lines intersecting at one point in the image space. We next discuss the extension of the Hough transform to arbitrary parametric curves. Suppose that P(t1 , t2 , . . . , tn ) is a parametric curve defined by n parameters. For each n-tuple (t1 , t2 , . . . , tn ), the curve passes through a point (x, y). The Hough transform maps each point (x, y) in the image space to many points (t1 , t2 , . . . , tn ) in the n-dimensional parameter space. If n = 2, each point is transformed into a twodimensional curve. For n = 3, each point is transformed into a three-dimensional surface. For larger values of n, each point is transformed into an n-dimensional hypersurface. The advent of the computer, not as a computer but as a drawing machine, was for me a major event in my life.
—Benoˆıt Mandelbrot.
3 Scan Conversion In a raster-scan graphics system, two steps are necessary in order to display a geometric figure: (1) a scan-converting algorithm should be executed to select the best pixels (the ones closest to the ideal figure) and (2) the selected pixels should be turned on. Step 2 is simple. It only requires setting bits in the bitmap (perhaps with xor). Most current compilers have a built-in function to do this. All that the program has to say is putpixel(row,col); or plot(r,c,color); or something similar. Step 1, however, is more complex. The scan-converting algorithm has to be fast and it must depend on the shape of the figure. This chapter discusses scan-converting algorithms for straight lines and for circles.
3.1 Scan-Converting Lines After the point, the straight line is the simplest geometric figure. Its explicit equation is y = a x + b, where a is the slope and b is the y-intercept (see Section 9.1 for other ways to represent lines). In practice, the coordinates of the two endpoints (x1 , y1 ) and (x2 , y2 ) are given, instead of a and b, but it is easy to express a and b in terms of the endpoints. The slope a is simply (y2 − y1 )/(x2 − x1 ) or Δy/Δx. The value of b is obtained from y1 = ax1 + b, which implies b = y1 − ax1 = y1 −
y2 − y1 y1 (x2 − x1 ) − x1 (y2 − y1 ) y1 x2 − x1 y2 x1 = = . x2 − x1 x2 − x1 x2 − x1
Our first algorithm uses (x1 , y1 ) and (x2 , y2 ) to compute a and b, and then executes the loop of Figure 3.1. However, this loop is very slow because it uses multiplications and also because it works with real quantities that eventually have to be rounded to integers. A better algorithm should use just additions/subtractions and just integers.
D. Salomon, The Computer Graphics Manual, Texts in Computer Science, DOI 10.1007/978-0-85729-886-7_3, © Springer-Verlag London Limited 2011
135
136
3.2 Midpoint Subdivision var a, b, x, y, x1, x2, y1, y2: real; a:=(y2-y1)/(x2-x1); b:=y1-a*x1; x:=x1; repeat y:=a*x+b; point(round(x),round(y)); x:=x+1; until x>x2; Figure 3.1: Scan Convert y = ax + b.
Exercise 3.1: The loop of Figure 3.1 has another drawback; what is it?
3.2 Midpoint Subdivision Midpoint subdivision is a completely different approach to the problem of scan-converting lines. It employs a procedure Midpoint that divides the line segment in two, calculates the midpoint of the segment, plots this pixel, and calls itself recursively twice, to do the same for the two halves of the segment. The following is a listing in the C language: void Midpoint(int a1,int b1, int a2, int b2) { int midx,midy; midx=(a1+a2)/2; midy=(b1+b2)/2; putpixel(midx,midy,Color); /* Turbo C */ if(abs(a1-midx)>1 || abs(b1-midy)>1) Midpoint(a1,b1,midx,midy); if(abs(midx-a2)>1 || abs(midy-b2)>1) Midpoint(midx,midy,a2,b2); } The main program needs only input the two endpoints and then invoke Midpoint(x1,y1,x2,y2);. In practice, the procedure may be written as a nonrecursive one, manipulating the recursion stack explicitly. An attractive feature of this method is the simple arithmetic. Only two divisions are required and even they can be replaced by a shift. The proverbial German phenomenon of the verb-at-the-end about which droll tales of absentminded professors who would begin a sentence, ramble on for an entire lecture, and then finish up by rattling off a string of verbs by which their audience, for whom the stack had long since lost its coherence, would be totally nonplussed, are told, is an excellent example of linguistic recursion. —Douglas Hofstadter.
3 Scan Conversion
137
3.3 DDA Methods Better scan-conversion algorithms should be arithmetically simple and should use just integers. There is a large class of such methods, called DDA (for Digital Differential Analyzer) methods. Section 2.5 discusses one such method, for bitmap scaling. Several DDA methods for scan-converting lines are described here. The original Differential Analyzer was an analog computer built, in 1927, by the American scientist and engineer Vannevar Bush (1890–1974). It was based on the use of mechanical integrators that could be interconnected in any desired manner.
3.3.1 Simple DDA This algorithm uses the relationship a(x + 1) + b = (ax + b) + a = y + a, which implies that if we increment x by 1 and would like to stay on the line, we should increment y by a. The following code assumes that the two endpoints (x1, y1) and (x2, y2) of the line segment are given. This algorithm does not work for vertical lines, where a = ∞. var x, x1, y1, x2, y2: integer; a, y: real; a:=(y2-y1)/(x2-x1); x:=x1; y:=y1; repeat point(x,round(y)); x:=x+1; y:=y+a; until x>x2; This is still inefficient because y is still a real quantity, but it has another, more important drawback. Imagine a very steep line, where a 1. Since our loop increments x by 1, successive pixels will have y coordinates that differ greatly and the result may be a fragmented line, made of a few disconnected pixels. To correct this problem, we have to check the slope, and if it is greater than 45◦ , increment y, not x, in steps of 1. From y = ax + b, we get y + 1 = ax + b + 1 = ax + 1 + b = a(x + 1/a) + b. Therefore, when y is incremented by 1, x should be incremented by 1/a. The algorithm becomes var Δx,Δy,L:integer; x,y,a,G,H:real; Δx:=x2-x1; Δy:=y2-y1; x:=x1; y:=y1; if Δx>Δy then G:=1; H:=a; else G:=1/a; H:=1 endif; for L:=1 to max(Δx,Δy)+1 do point(round(x),round(y)); x:=x+G; y:=y+H; endfor; Ideally, the number of pixels generated when a line is scan-converted should equal the length of the line. This is because the length is measured in screen units, which are pixels. A line of length L displayed by fewer than L pixels would be fragmented. If the same line is generated by more than L pixels, it would look brighter than other lines. The total number of pixels drawn by simple DDA is max(Δx, Δy). For lines that are close to horizontal or close to vertical, max(Δx, Δy) ≈ length, which is the ideal
3.3 DDA Methods
138
◦ case. For a 45 line, √ Δx = Δy, √ so Δx pixels are drawn. The length of such a line equals 2 2 Δx + Δy = 2Δx2 = 2Δx ≈ 1.41Δx which implies that Δx ≈ 0.71 length. For such a line, this algorithm generates only about 71% of the ideal number of pixels. Consequently, such lines look dim.
Exercise 3.2: Given the two endpoints (x1, y1) = (1, 2) and (x2, y2) = (4, 6), execute the simple DDA algorithm manually and show the pixels generated. Also, calculate the length of the line.
3.3.2 A Variation The following pseudo-code is a variation of the simple DDA. The quantity length is not the length of the line but is related to it. Because of the way it is defined, one of the quantities x_incr, y_incr equals ±1, which simplifies the algorithm. procedure SimpleDDA(x1,y1,x2,y2: integer); begin Δx,Δy,length,i:integer; x,y,x_incr,y_incr:real; Δx:=x2-x1; Δy:=y2-y1; length:=max(abs(Δx), abs(Δy)); x_incr:=Δx/length; y_incr:=Δy/length; x:=x1; y:=y1; for i:=1 to length+1 do point(round(x),round(y)); x:=x+x_incr; y:=y+y_incr; endfor; end; Notice that the other quantity (x_incr or y_incr) is still real.
3.3.3 Symmetrical DDA The simpler DDA methods are based on a loop, where during each iteration, the (x, y) coordinates of the previous pixel are incremented by Δx and Δy, respectively, for some quantity . A typical procedure is procedure SymmDDA(x1,y1,x2,y2: integer); calculate eps; xIncr:=eps*Δx; yIncr:=eps*Δy; x:=x1+.5; y:=y1+.5; repeat Plot(trunc(x),trunc(y)); x:=x+xIncr; y:=y+yIncr; until x=x2 or y=y2; end; (See the end of this section for an explanation of x:=x1+.5; y:=y1+.5; and for the use of trunc instead of round.) In the symmetrical DDA method, we set = 2−n , where n is defined by 2n−1 ≤ max(|Δx|, |Δy|) < 2n .
3 Scan Conversion
139
This sets to 1 over the (approximate) length of the line. It also sets the x and y increments to values less than 1. As an example, consider the line from (0, 0) to (7, 5). Its equation is y = (5/7)x. For this line, we have Δx = 7 and Δy = 5, leading to 22 = 4 ≤ max(7, 5) < 8 = 23 . Therefore, n is set to 3, implying = 2−3 = 1/8. The (x, y) coordinates of the previous pixel are incremented by Δx = 7/8 and Δy = 5/8, respectively. The nine steps of the loop are summarized in Table 3.2. The last column of the table (+.−) compares the ideal y coordinate of the point (which equals 5/7 times the x coordinate) to the y coordinate actually displayed. A “+” is shown if the point displayed is above the ideal point, a “−” is shown in the opposite case, and a period is shown in the ideal case. Point No. 1 2 3 4 5 6 7 8 9
Start (x, y) (.5, .5) (.5 + 78 , .5 + 7 9 ( 11 8 + 8, 8 + 18 7 14 ( 8 + 8, 8 + 7 19 ( 25 8 + 8, 8 + 32 ( 8 + 78 , 24 8 + 7 29 ( 39 + , 8 8 8 + 7 34 + ( 46 8 8, 8 + 54 ( 8 + 78 , 39 8 +
5 8) 5 8) 5 8) 5 8) 5 8) 5 8) 5 8) 5 8)
= (.5, .5) = (11/8, 9/8) = (18/8, 14/8) = (25/8, 19/8) = (32/8, 25/8) = (39/8, 29/8) = (46/8, 34/8) = (54/8, 39/8) = (61/8, 44/8)
Truncated to
Ideal y
+.−
(0, 0) (1, 1) (2, 1) (3, 2) (4, 3) (4, 3) (5, 4) (6, 4) (7, 5)
0 5/7 10/7 15/7 20/7 20/7 25/7 30/7 35/7
. + − − + + + − .
Table 3.2: Symmetrical DDA Example.
Note that point 6 is identical to point 5 and should not be displayed. If we ignore point 6, we end up with a line where the first and last points are smack on the ideal line, points 3, 4, and 8 are below the line, and points 2, 5, and 7 are above it. This is the reason for the name Symmetrical DDA. How can we avoid plotting point 6? The ideal solution is to use special hardware where both x and y are stored in registers that have two parts. The left part of each register holds the integer value of the variable, and the right part, the fractional value. The x and y increments, which are less than 1, are added to the fractional parts. Whenever a fractional part overflows, the overflow signal increments the corresponding integer part. If neither fractional register overflows, the point is not plotted. This method creates the truncated value in the integer parts, rather than the rounded values, since this is faster. Since we still need the rounded, not truncated, values, we initialize both fractional parts to 0.5 rather than to zero. Notice that trunc(x+0.5) equals round(x).
3.3.4 Quadrantal DDA Quadrantal DDA is a more sophisticated DDA method. Its principle is to increment, in each step, either x or y but not both. From step to step, we move along the line either horizontally or vertically, but not diagonally (Figure 3.3).
140
3.3 DDA Methods
Figure 3.3: Quadrantal DDA.
The key to the algorithm is to realize that the ratio number of y increments number of x increments should equal the slope Δy/Δx. The implementation uses an auxiliary variable, Err, that is initially set to zero. In each step, if Err>0, we increment x by 1 and decrement Err by Δy. If Err<0, we increment y by 1 and increment Err by Δx. Thus, if Err is positive, we move along the line horizontally and decrement Err. After a number of steps, Err becomes negative and we start moving vertically, incrementing Err. If Δx > Δy, we end up with more x than y increments, which is appropriate since Δx > Δy implies a slope that’s less than 45◦ . If Δx < Δy, there will be more steps in the y direction. The quantity Err, thus, oscillates around zero all the time. The method is called quadrantal because the details of the steps depend on the direction of the line. If the direction is in the range 0◦ –90◦ , we increment both x and y by 1. If the direction is in the range 90◦ –180◦ , we increment x by −1 and y by 1; similarly, for the other two ranges. Hence, the algorithm must start by determining the range of the direction, and this is done by comparing Δx and Δy and looking at their signs. The pseudo-code below lists the loop for the first quadrant. It assumes that the two endpoints (x1, y1) and (x2, y2) of the line segment are given. var x1,y1,x2,y2,Δx,Δy: integer; Δx:=x2-x1; Δy:=y2-y1; Err:=0; repeat plot(x1,y1); if Err>0 then x1:=x1+1; Err:=Err-Δy else y1:=y1+1; Err:=Err+Δx endif; until x1>=x2 and y1>=y2; The quadrantal DDA method features the following: No multiplications, divisions, or real numbers are used. There are no diagonal moves, just horizontal and vertical ones. The line is therefore made of segments with a one-pixel overlap.
3 Scan Conversion
141
If Err happens to be zero in some loop iteration, it means that the current pixel is located right on the ideal line. Our program is written such that for Err=0, it executes the else part and therefore increments y1. When the current pixel is positioned on the ideal line, we normally don’t care whether the next pixel is drawn in the x or in the y direction. There is one exception, however—the second pixel! The first pixel is drawn at point (x1, y1), and since initially Err is set to zero, the second pixel will always be drawn at (x1, y1 + 1). This may be annoying if the line is close to horizontal, because it produces the pattern instead of . The solution is to set Err initially to +Δx/2, if Δx > Δy, Err = −Δy/2, otherwise, instead of to zero. This produces better looking lines, but notice that Err is no longer an integer. The number of pixels in a quadrantal line segment is easily computed. For a line close to horizontal or close to vertical, the number equals the length of the line (one pixel per x or per y value, as in the simple DDA method). For a line slanted at 45◦ , quadrantal DDA produces two pixels per x value (except √ the two extreme values), so the line looks as in Figure 3.3. The length of the line is 2Δx, so the ratio (number of √ √ pixels/line length) equals 2Δx/ 2Δx = 2 ≈ 1.41. Thus, there is an excess of 41%. Such lines are still too bright and also slow to compute. Exercise 3.3: Given the two endpoints (1, 1) and (5, 5), calculate the pixels for the straight line between them obtained by simple DDA and by quadrantal DDA.
3.3.5 Octantal DDA The main idea behind octantal DDA is to move along the line either horizontally or diagonally, but not vertically (the precise rule depends on the slope). The main feature of such lines is that they are made of segments that do not overlap (Figure 3.4), so they look smoother than similar quadrantal lines.
Figure 3.4: Octantal DDA.
Eight sets of rules are necessary. If the direction of the line is in the range 0◦ –45◦ (the first octant, where 0 ≤ slope ≤ 1), only horizontal and diagonal moves are allowed. If the direction is in the second octant (45◦ –90◦ ), only vertical and diagonal moves are allowed, not horizontal. In the third octant, the moves are the same as in the second one, but with negative x increments. The precise rules for four of the eight octants are summarized as follows: In octant 1: 1. If Err<=0, move horizontally (+x) and update Err:=Err+Δy. 2. If Err>0, move diagonally (+x, +y) and update Err:=Err+Δy − Δx.
142
3.4 Bresenham’s Line Method
In octant 2: 1. If Err<0, move diagonally (+x, +y) and update Err:=Err+Δy − Δx. 2. If Err>=0, move vertically (+y) and update Err:=Err−Δx. In octant 7: 1. If Err<0, move diagonally (+x, −y) and update Err:=Err−Δy − Δx. 2. If Err>=0, move vertically (−y) and update Err:=Err−Δx. In octant 8: 1. If Err<=0, move horizontally (+x) and update Err:=Err−Δy. 2. If Err>0, move diagonally (+x, −y) and update Err:=Err−Δy − Δx. The rules for octants 3–6 are similar. Note that the program can be made more compact (although a bit slower) by using the symmetry between certain octants. The only difference between octants 2 and 7, for example, is in the y coordinate. If the endpoints are (1, 1) and (2, −7) (which corresponds to octant 7), then the program can calculate the line for points (1, −1) and (2, 7) (octant 2), and transform the line to octant 7 by reversing the y coordinates of all the pixels. Exercise 3.4: Do it. The main advantage of the octantal over the quadrantal DDA is the number of pixels per line, which is closer to the length of the line. Lines drawn using this method look finer and more precise. For lines close to horizontal or vertical, the number of pixels is the same as in the quadrantal DDA method. For a 45◦ line, however, there is an improvement. The line looks as in Figure 3.4, one pixel per x value.√ Hence, the √ number of pixels is Δx and the ratio (# of pixels/line length) equals Δx/ 2Δx = 1/ 2 ≈ 0.71 = 1 − 0.29. There is therefore a pixel shortage of 29% (compared to an excess of 41% in quadrantal DDA). Exercise 3.5: Intuitively it seems that the 45◦ line shown in Figure 3.4 is the best representation of such a line. It looks straight and precise. Any additional pixels would make it too thick. It seems that the octantal DDA method produces ideal 45◦ lines, so how can we say that they have a shortage of pixels?
3.4 Bresenham’s Line Method Currently, high-resolution, color displays are common, but they were rare and very expensive in the 1960s, the period that saw the emergence of computer graphics. This is why the main graphics output device in the early days of this discipline was the pen plotter (Section 26.13), and this is also why the first DDA method was originally developed, by Jack Bresenham [Bresenham 65], for use on a pen plotter. Two versions of this method are described here, together with ideas for improving it and speeding it up. The first version of this algorithm is derived here for the first octant (i.e., for line segments with slopes between 0 and 45◦ ), but its extension to other octants is not difficult. Since the line is close to horizontal, the algorithm should increment x in each iteration and the only decision that needs to be made is whether to also increment y. If the current pixel was drawn at (x, y), then the next pixel should be plotted either at
3 Scan Conversion
143
(x + 1, y) or at (x + 1, y + 1). In principle, this decision can be made by computing the y value of the actual line at x + 1 (i.e., h = a(x + 1) + b) and determining the smaller of the two distances (y + 1) − h and h − y. These computations, however, involve the two real quantities a and b. Instead, the Bresenham algorithm starts by computing the difference d between the height of a general pixel (x, y) and the height of the true line at x. That difference is d = (ax + b) − y = (Δy/Δx)x + b − y. In the first octant, Δx is nonzero, so d is well defined. We now multiply both sides by Δx to obtain the decision variable D = dΔx = xΔy − yΔx + bΔx (note that D is an integer because bΔx is an integer). Each time x is incremented inside the loop, if the algorithm decides to increment y, then D is modified to Dnew ← (x + 1)Δy − (y + 1)Δx + bΔx = Dold + (Δy − Δx), so D should be incremented by (Δy − Δx), a negative quantity. If, on the other hand, the algorithm decides to keep the same y, D is modified to Dnew ← (x + 1)Δy − yΔx + bΔx = Dold + Δy, so D should be incremented by Δy, which is positive. Based on this, the decision whether to increment y is made by trying to keep D close to zero. The principle of Bresenham’s algorithm for the first octant is as follows: Set D to zero and start a loop. In each iteration, if D is zero or negative, keep y the same and increase D by Δy. If D is positive, increment y by 1 and decrease D by (Δy − Δx). The resulting code, listed in Figure 3.5, is both compact and fast. var dx, dy, dxdy, D: integer; x dxdy:=dy-dx, D:=0; 10 x:=x1; y:=y1; 11 repeat 12 pixel(x,y); if d>0 then y:=y+1; D:=D+dxdy 13 14 else D:=D+dy 15 endif; 16 x:=x+1; until x=x2; 17 Figure 3.5: Bresenham’s Method.
y 10 10 11 11 12 12 13 13
D 0 6 −2 4 −4 2 −6 0
Inc y? n y n y n y n n
x 18 19 20 21 22 23 24
y 13 14 14 15 15 16 16
D 6 −2 4 −4 2 −6 0
Inc y? y y y n y n
Table 3.6: An Example.
As an example, Table 3.6 lists the pixels selected by this algorithm for the line from (10, 10) to (24, 16). For this line, Δy = 6, Δx = 14, and Δy − Δx = −8. Table 3.7 can be used to determine the octant of the slope. Given a line segment from (x1 , y1 ) to (x2 , y2 ), first reorder the points, if necessary, such that x1 ≤ x2 , then use the table. The top row of the table reads: If Δy ≥ 0 and Δx ≥ Δy, then the slope is positive and is less than or equal 1. The octant is either 1 or, if the points had to be swapped, it is 5.
144
3.4 Bresenham’s Line Method Δy
Δx?Δy
slope
≥0 ≥0 <0 <0
≥ < ≤ >
pos ≤ 1 pos > 1 neg ≥ −1 neg < −1
octant 1 2 7 8
(5) (6) (3) (4)
Table 3.7: Determining the Octant.
For the four octants where the slope is in the range [−1, +1], the algorithm loops over x as shown in Figure 3.5. For the other four octants, the roles of x and y should be swapped. The above derivation of Bresenham’s method is simple, but most references in the literature derive it from a different approach, and end up with a slightly different algorithm. This alternative approach is described here. We again assume a line segment from (x1 , y1 ) to (x2 , y2 ) with a slope in the first octant. The loop iterates on x values from x1 to x2 and the only decision to be made in iteration k is whether to increment the y coordinate or not. Assume that iteration k has plotted a pixel at position (xk , yk ). The next iteration should plot a pixel at (xk+1 , yk+1 ) where xk+1 = xk + 1 and yk+1 is either yk or yk + 1. The decision is based on the distances of the true line from the two candidate pixels P0 = (xk + 1, yk ) and P1 = (xk + 1, yk + 1). We use the notation y = a xk+1 + b. This is the true height of the line between points P0 and P1 . The equation of a straight line is y = a x+b. The slope a equals (y2 −y1 )/(x2 −x1 ) = Δy/Δx and the y-intercept b is y1 − a x1 . The distance of the line from P1 is m = (yk +1)−y = yk +1−a(xk +1)−b and its distance from P0 is n = y−yk = a(xk +1)+b−yk . We subtract these distances to obtain the difference n − m = a(xk + 1) + b − yk − yk − 1 + a(xk + 1) + b = 2a(xk + 1) − 2yk + 2b − 1. (3.1) The sign of this difference determines the point to be selected in iteration k + 1. If the difference is positive (m is smaller), then P1 should be selected (i.e., y should be incremented); otherwise, P0 is the point closer to the line. True to the DDA spirit, however, we don’t want to perform so many computations in each iteration, so the algorithm has to be improved. We multiply both sides of Equation (3.1) by Δx to obtain dk = Δx(n − m) = 2xk Δy + 2Δy − 2yk Δx + Δx(2b − 1) = 2xk Δy − 2yk Δx + C, (3.2) where C = 2Δy + Δx(2b − 1) is a constant that does not vary from iteration to iteration. Equation (3.2) can be used to compute the initial value d1 d1 = 2x1 Δy − 2y1 Δx + 2Δy + Δx(2b − 1) = 2Δy − Δx,
(3.3)
but it is clear that computing dk is as bad as computing the difference n − m. The main idea of Bresenham’s algorithm is to compute dk in each iteration indirectly, from
3 Scan Conversion
145
its predecessor. This turns out to be simple, since the difference dk+1 − dk is the simple expression dk+1 − dk = 2Δy(xk+1 − xk ) − 2Δx(yk+1 − yk ) = 2Δy − 2Δx(yk+1 − yk ). If iteration k + 1 decides (based on the sign of dk ) to increment y, then yk+1 = yk + 1, so the updated value dk+1 becomes dk + 2(Δy − Δx). If the decision is not to increment y, then the new value becomes dk + 2Δy. Thus, the algorithm starts by computing the initial value of d from Equation (3.3), then iterates from x1 to x2 . In each iteration it selects either P0 or P1 depending on the sign of d, then updates d. Figure 3.8a shows a pseudo-code version of this algorithm, with a C version shown in Figure 3.8b. var x,y,dxy,dy,d: integer; y:=y1; Δ x=x2-x1; Δ y=y2-y1; dy:=2Δ y; dxy:=2(Δ y −Δ x); d:=2Δ y − Δ x; for x:=x1 to x2 do pixel(x,y); if d< 0 then d:=d+dy; else d:=d+dxy; y:=y+1 endif endfor (a)
bresenham(int x1,y1,x2,y2) {int y=y1, dx=x2-x1, dy=y2-y1; int d=2*dy-dx; for (int x=x1; x<=x2; x++) { pixel(x,y); if(d<0) d+=2*dy; else{y++; d+=2*(dy-dx); } } } (a)
Figure 3.8: Bresenham Line Method (a) Pseudo-Code, (b) C Code.
Exercise 3.6: Use the algorithm of Figure 3.8 to calculate the line segment from (1, 1) to (10, 3). The following observations apply to lines generated by DDA methods in general, not just Bresenham’s method. 1. The best pixels of a line segment should be symmetric about the center of the segment, as in Figure 3.9d and not as in 3.9e. The line can therefore be drawn from both ends toward its middle, plotting two pixels in each iteration. This is easy for line segments that have an even number of pixels. If the segment consists of an odd number of pixels, the last step draws two pixels next to each other, causing the line to “bulge” at the center. Figure 3.9 illustrates this problem. In part (a), a 5-pixel segment is drawn from A to B. In part (b), the same segment is drawn from B to A. In part (c), it is drawn from both side, so the last step plots two pixels on top of each other. The algorithm has to be improved such that it draws just one pixel in the last iteration. 2. A glance at Table 3.6 shows that the values of D go through cycles. When D reaches its initial value of 0 (in step 8), it cycles through the same values again, causing the algorithm to make the same yes/no decisions. It is possible to store the sequence of decisions for one cycle of D in an array, and use it as many times as needed to produce
3.4 Bresenham’s Line Method
146
the rest of the line segment. This is particularly useful for long segments (hundreds of pixels). 3. DDA methods (especially line methods) are fast. There are just a few simple operations to perform in each iteration, and those are operations on integers. As a result, such a program spends most of its time in the system routine that plots a pixel (routine pixel in the code listings shown here). This routine receives the (x, y) screen coordinates of a pixel and its color. It has to calculate the location of the pixel in the bitmap (a single bit, a byte, or some other chunk of memory), then set that location to the right color or perform a logical operation between the new color and the existing color. Any DDA method can therefore be speeded up considerably if the program can have direct access to the bitmap instead of calling a procedure. This is true for operating system programs, and sometimes for programs written in assembler. 4. Certain DDA methods can be speeded up by calculating and plotting two pixels in each iteration. One example is an extension of the Bresenham method, due to [Rokne et al. 90] that employs symmetry and double-step to speed up the original algorithm by a factor of 3. Another example is the doube-step method of Section 3.5.
....B ...X. .XX.. A....
....B ..XX. .X... A....
....B ..XX. .XX.. A....
(d)
(a)
(b)
(c)
(e)
Figure 3.9: Segments with Odd and Even Numbers of Pixels.
Historical Note In November 2001, Jack E. Bresenham wrote, “I was working in the computation lab at IBM’s San Jose development lab. A Calcomp plotter had been attached to an IBM 1401 via the 1407 typewriter console. [The algorithm] was in production use by summer 1962, possibly a month or so earlier. Programs in those days were freely exchanged among corporations so Calcomp (Jim Newland and Calvin Hefte) had copies. When I returned to Stanford in Fall 1962, I put a copy in the Stanford comp center library.”
3 Scan Conversion
147
3.5 Double-Step DDA The double-Step DDA algorithm is an extension of the quadrantal and octantal methods, where in each iteration, two pixels, instead of just one, are computed and displayed. The method is due to [Wu and Rokne 87]. It is based on a loop, where in each iteration, a simple decision is made based on the sign of a discriminator D. The method is fast, because only half the number of iterations is required. We discuss the details of the method for lines in the first quadrant (i.e., with slopes between 0◦ and 45◦ ), but it can easily be extended to the other three quadrants. Figure 3.10 shows that if pixel P was selected in some iteration, then the following iteration must select “its” two pixels according to patterns 1–4. No other patterns are possible for a line in the first quadrant. In order to select one of four patterns, we need to ask three questions, and this translates to three if statements in each iteration. Since other DDA methods execute only one if per iteration, our method must be improved, otherwise it will turn out to be slow, even though it does two pixels per iteration. The key to improving the method is to realize that lines with higher slopes (close to 45◦ ) do not use pattern 1 and lines close to horizontal do not use pattern 4. Hence, our method should start by checking the slope.
P
P 1
P 2
P 3
P 4
5
Figure 3.10: The Five Patterns for Quadrant-1 Lines.
There are two approaches to improving the double-step method: 1. We reduce the number of patterns to three by selecting pattern 5 (with the two gray pixels) whenever we need patterns 2 or 3. The algorithm checks the slope, as shown below. If the slope is small enough, only patterns 1 and 5 will be used. The algorithm goes into a loop, where, in each iteration, it uses a simple test to select either pattern 1 or 5. If the slope is high, the algorithm goes into a similar loop, where it selects either pattern 4 or 5 in each iteration. This results in a fast algorithm that has another advantage. Since pattern 5 has two gray pixels, the resulting line is somewhat antialiased. (The important topic of antialiasing is discussed in Section 3.13.) The following pseudo-code procedure summarizes this approach (but does not show how D is computed):
148
3.5 Double-Step DDA procedure dbstep(x1,y1, x2,y2); dx=x2-x1; dy=y2-y1; x:=x1; y:=y1; if dy/dx>.5 (high slope) then while x≤x2 do if D<0 then pattern(5) else pattern(4); endwhile; else (low slope) while x≤x2 do if D<0 then pattern(1) else pattern(5); endwhile; endif; procedure pattern(patt: integer); case patt of 1: pixel(x+1,y); pixel(x+2,y); 2: pixel(x+1,y); pixel(x+2,y+1); y:=y+1; 3: pixel(x+1,y+1); pixel(x+2,y+1); y:=y+1; 4: pixel(x+1,y+1); pixel(x+2,y+2); y:=y+2; end (case) x:=x+2; end;
2. We stay with the four patterns and divide the algorithm into two loops, as in method 1. This time, each loop has to have two if statements to select one of three patterns. This is somewhat slow but can be used in a monochromatic display, which can display only black and white (or black and green) and no gray. Both methods are discussed here, but note again that they are restricted to lines in the first quadrant. To restrict our choices, let’s first distinguish higher slopes (where we select patterns 2, 3, and 4, but never 1) from lower slopes (where pattern 1, 2, or 3 should be used, but never pattern 4). Figure 3.11 shows the pixel that was selected at iteration i − 1. We name this pixel P and denote its coordinates (xi−1 , yi−1 ). The next iteration (iteration i) should select one of the three pixels L (pattern 1), K (patterns 2 or 3), or J (pattern 4). Three important dimensions in the figure are a, the distance of the line from pixel P, b, its distance from L, and c, its distance from J. A look at the diagram should convince the reader that a should satisfy a ≤ 0.5, since otherwise, iteration i − 1 would have selected the pixel above P. Note that the line may go below pixel P, but even in this case, it should satisfy |a| ≤ 0.5. Pattern 4 is only selected for lines with higher slopes. The smallest slope for which pattern 4 (pixel J) will be selected satisfies a = 0.5 and b = 1.5 (the line is farthest from P and is midway between K and J, see Figure 3.11a). This corresponds to a slope of 0.5. Pattern 1 is only selected for lines with lower slopes. The highest slope for which this pattern could possibly be selected corresponds to a = −0.5 and b = 0.5 (the line is below pixel P and midway between L and K). This, again, corresponds to a slope of 0.5. We therefore say that slope 0.5, which corresponds to a slope angle of 22.5◦ , sepa-
3 Scan Conversion i−1
M
149
J c’
c
K a
b’
b L
i−1 P (a)
(b)
(c)
Figure 3.11: Details of Calculations.
rates patterns 1 and 4, so they cannot occur in the same line. Figure 3.11a shows that the condition for the line to be closer to pixel K (patterns 2 or 3) than to J (pattern 4) is (b − c)/2 < 0.5, which corresponds to b − c < 1 or b − c − 1 < 0. Similarly, Figure 3.11c shows that the line will be closer to pixel L (pattern 1) than to K (patterns 2 or 3) if it satisfies (c − b)/2 > 0.5, which corresponds to c − b > 1 or b − c + 1 < 0. The algorithm should therefore first look at the slope and do the following: 1. If the slope is < 0.5, there should be a loop that checks the sign of b − c + 1. If it is negative, pattern 1 should be selected; otherwise, patterns 2 or 3. 2. Similarly, if the slope is > 0.5, there should be loop where the sign of b − c − 1 should be checked in each iteration. If it is negative, patterns 2 or 3 should be selected; otherwise, pattern 4. What is needed now is a way to compute the sign of both b − c + 1 and b − c − 1 by using just integer quantities and simple operations. We first develop a simple test for the sign of b − c + 1. We first observe that our line, which goes from point (x1 , y1 ) to (x2 , y2 ), can be translated such that it starts at the origin and this will not change the pattern of pixels. We therefore assume that our line has the equation y = (Δy/Δx)x (the y-intercept is zero), without loss of generality. We next observe that the height of the line at x = xi−1 + 2 is (Δy/Δx)(xi−1 + 2). The same height can also be written as the sum yi−1 + b, so we get b = (Δy/Δx)(xi−1 + 2) − yi−1 . To find a similar expression for c, we subtract the height of the line at x = xi−1 + 2 from the height of pixel J (which is yi−1 + 2). The result is c = yi−1 + 2 − (Δy/Δx)(xi−1 + 2). The quantity b − c + 1 can now be written Δy Δy (xi−1 + 2) − yi−1 − yi−1 + 2 − (xi−1 + 2) + 1 Δx Δx Δy (xi−1 + 2) − 2yi−1 − 1. =2 Δx
b−c+1=
We now define a new quantity Di (the discriminator for the sign of b − c + 1) by means
3.5 Double-Step DDA
150 of Δx and Δy as
Di = Δx(b − c + 1) = 2Δy(xi−1 + 2) − 2Δx yi−1 − Δx. Note that since the line is in the first quadrant, Δx is never negative (this assumption should be changed when extending the algorithm to other quadrants). This means that the sign of Di is the same as that of b − c + 1 and we can use Di in our loop to determine which pattern to use. Before starting the loop, we should set D1 = 4Δy − Δx. In each iteration, a new value Di+1 should be calculated from the previous value Di . From Di+1 = 2Δy(xi +2)− 2Δx yi − Δx, we get Di+1 − Di = 2Δy(xi − xi−1 ) − 2Δx(yi − yi−1 ). Since xi − xi−1 = 2 and since yi − yi−1 equals either 1 (for patterns 2 and 3) or 0 (for pattern 1), we can write if Di < 0 (pattern 1), Di + 4Δy, Di+1 = Di + 4Δy − 2Δx, otherwise (patterns 2 and 3). The discriminator for the sign of b − c − 1 is derived in a similar way, and the result is D1 = 4Δy − 3Δx and Di + 4Δy − 2Δx, if Di < 0 (patterns 2 and 3), Di+1 = Di + 4(Δy − Δx), otherwise (pattern 4). This completes the algorithms for approach 1, where patterns 1, 4, and 5 are used. In order to implement approach 2, where patterns 1–4 are used, we need another discriminator Dt . This is calculated by a process similar to the one used to obtain Di , except that point M is used in Figure 3.11 instead of pixel P as the base for the calculations. Its distance c from the line is shown in Figure 3.11b. By the definition of a discriminator, we get Δx(b − c + 1), if slope< 0.5, Dt = (3.4) Δx(b − c − 1), if slope≥ 0.5, where Figure 3.11 provides the relations b = b − Δy/Δx and c = c + Δy/Δx, which imply Dt = Di − 2Δy. Also, from Figure 3.11 we see that when b − c + 1 < 0, pattern 2 should be used, otherwise pattern 3. Combining this knowledge with Dt = Di − 2Δy and with Equation (3.4), we find that the relation b − c + 1 < 0 is equivalent to 2Δy, if slope< 0.5, Di < 2(Δy − Δx), if slope≥ 0.5. The last point to discuss is the termination of the loop. Since each iteration increments x by 2, we cannot simply check to see whether x is greater than x2 , as this may result in one extra pixel. The solution is to realize that the extra pixel will result only if the line is made of an odd number of pixels. Since our lines are in the first quadrant, the number of pixels equals Δx. We therefore change the loop if Δx is odd, and terminate it when x = x2 − 1. The last pixel is then drawn separately, outside the loop, at (x2 , y2 ).
3 Scan Conversion
151
3.6 Best-Fit DDA Best-fit is an interesting, fast method based on Euclid’s algorithm. It is discussed here for the first octant only (not including slopes of 0◦ and 45◦ , but drawing lines for these slopes is trivial), but it is easy to extend to other octants since it produces a string of bits indicating the moves from pixel to pixel. For the first octant, a 0 bit means a horizontal move and a bit of 1 means a diagonal move. For the second octant, a 0 bit means a vertical move and a 1 bit means a diagonal move. Section 3.3.5 discusses the symmetry between certain octants. The only difference between octants 2 and 7, for example, is in the y coordinate. Since the algorithm is based on strings, we use the string notation rev(str) for the reverse of the string in variable str, we use str1+str2 to denote the concatenation of the two strings, we use substring(str,a,b) for the substring from position a to position b of str, and we use len(str) for the length of string str. (String operations are discussed in any introductory text on computer programming.) Figure 3.12 is a pseudo-code listing of the algorithm The algorithm calls for several string reversals. This operation is slow when done by software. On a graphics computer, however, a special machine instruction may be added that reverses a register, making it possible to reverse a string in just one clock cycle. Example: A line from (1, 2) to (18, 12). The slope is (12 − 2)/(18 − 1) = 10/17 (i.e., in the first octant). Table 3.13 summarizes the steps. The final result is the string 10101011010110101. Note that it consists of 10 ones and seven zeros. It indicates 17 steps—10 of them horizontal and seven diagonal—following the first pixel at (1, 2). Exercise 3.7: Apply this algorithm manually to the lines from (4, 4) to (12, 6), and from (4, 4) to (14, 7). DDA methods generate different types of lines, but they all obey the following rule: In a line that is close to horizontal, steps in the y direction (or diagonal steps) are isolated (never two consecutive steps) and steps in the x direction occur in runs of the same size (except, perhaps, the first and last runs). In a line that’s close to vertical, the situation is the reverse.
3.6.1 Scanning a Bitmap in an Arbitrary Direction It is easy to scan a bitmap by rows or by columns, and this short section shows how a scan-converting algorithm for lines can be employed to scan a bitmap in an arbitrary direction. Figure 3.14a shows a bitmap of R rows and C columns, and a line drawn at an angle α, specifying the direction of the scan. Notice that α = 0 is in the direction of the columns while α = 90◦ implies scanning by rows. Let’s call this line the main diagonal. It goes from (0, 0) to a point (x, y) and our first task is to determine the coordinates of this point.
3.6 Best-Fit DDA
152
procedure bestfit(x1,y1,x2,y2: integer); var str1,str2: 0..1; x,y,i: integer; done: boolean; begin done:=false; str1:=‘0’; str2:=‘1’; y:=y2-y1; x:=(x2-x1)-y; repeat case x>y: str2:=rev(str2)+str1; x:=x-y; x=y: done:=true; x
x
y
str1
str2
17 7 7 4 1 1 1
10 10 3 3 3 2 1
0 0 01 01 01 1010101 101010110101
1 1 1 101 10101 10101 10101
Table 3.13: Best-Fit DDA Example.
3 Scan Conversion
153
(0,0) R
(a)
(x,y)
(b)
C
(c)
(x,y)
Figure 3.14: Diagonals in a Bitmap.
The figure produces the equations y = R and x/y = tan α, and these equations can be solved to yield the values of x and y. Notice that for large angles α (Figure 3.14c, where tan α > C/R) we have to substitute x = C for y = R. Once the coordinates (x, y) are known, the next task is to determine the coordinates of the pixels closest to the ideal, mathematical main diagonal from (0, 0) to (x, y). This can be done with an algorithm such as quadrantal DDA (Section 3.3.4) or octantal DDA (Section 3.3.5). The algorithm determines the individual steps of moving from a pixel to its successor and saves these steps. For example, move one pixel to the right and draw three pixels vertically down. The last task is to repeat this process for all the diagonals (some are shown as green lines in Figure 3.14b). This is easy, because we already have the steps for the main diagonal. We simply move one pixel to the right and then proceed from pixel to pixel in the steps saved earlier by the DDA algorithm.
3.7 Scan-Converting in Parallel Parallel computers are becoming more and more common, and researchers are constantly looking for parallel versions of important algorithms. It turns out that scan-conversion algorithms can be modified to run on parallel computers. This section shows how to generalize scan-conversion methods for lines so that they can run on an MIMD (Multi Instruction, Multi Data) computer. Such a computer consists of several processors, all running simultaneously, communicating with each other either through shared memory or by means of message passing.
3.7 Scan-Converting in Parallel
154
The main principle in developing a parallel algorithm is to divide (or partition) the problem into subproblems that are identical. In this way, only one program has to be written, and the individual processors execute identical copies. The problem of scan-converting lines can be divided into subproblems in several ways as follows: 1. Assuming that the MIMD computer has p processors, divide the greater of the intervals Δx and Δy into p equal subintervals, and assign each subinterval to a processor. Each processor calculates the best pixels in its subinterval, and the total time is reduced in this way by a factor of p. The main problem is the error variable. Its start value in each subinterval should equal its last value in the preceding subinterval, which is initially unknown. Assigning Err:=0 in every subinterval leads to less than ideal results. 2. Consider the bounding rectangle of the line segment (Figure 3.15a). It contains Δx × Δy pixels. We assume that the MIMD computer has at least Δx × Δy processors arranged in a two-dimensional grid where the processor in position (i, j) becomes “responsible” for the pixel at relative position (i, j) in the rectangle. The processor calculates the distance d of its pixel from the line. If d is less than a preset value, the processor turns the pixel on (or, if XOR is used, flips it). This method is fast, but requires many processors.
(x2,y2) (xi,yj) yj-y0
(4,2)
(4,3)
(4,4)
(3,1)
(3,2)
(3,3)
(3,4)
(2,1)
(2,2)
(2,3)
(2,4)
(1,1)
(1,2)
(1,3)
(1,4)
d (x0,y0) Δy
x0-xi L
(x1,y1)
(4,1)
Δx (a)
(b)
Figure 3.15: (a) A Bounding Rectangle. (b) Divided Among 16 Processors.
Figure 3.15 shows the details of the calculation. The line segment L goes from (x1 , y1 ) to (x2 , y2 ). We draw a perpendicular d from point (xi , yj ) to the line. The point where it intersects the line is denoted (x0 , y0 ). The two triangles (L, Δy, Δx) and (d, yj − y0 , x0 − xi ) are similar. We can, thus, multiply corresponding sides to get d × L = Δy(x0 − xi ) + Δx(yj − y0 ),
3 Scan Conversion
155
yielding d=−
Δx x0 Δy − y0 Δx Δy xi + yj + = Axi + Byj + C. L L L
The three constants A, B, and C are the same for all pixels (see Exercise 3.8). They are calculated—normally by processor 0—and sent to all the other processors. When a processor receives these values, it calculates the distance d of “its” pixel from the line and decides whether or not to turn it on. Exercise 3.8: Show why C = (x0 Δy − y0 Δx)/L is a constant that does not depend on x0 or y0 . 3. If the MIMD computer does not have enough processors, each processor can be assigned a row or column in the bounding rectangle. The processor then calculates the intersection of the line with “its” row or column. If the line is close to horizontal, there should be one pixel per column, so each processor should be assigned a column xi . The processor then solves the equation
y = axi + b =
y2 − y1 y1 x2 − x1 y2 xi + x2 − x1 x2 − x1
for y and sets pixel (xi , y). If the line is close to vertical, there should be one pixel per row and each processor should be assigned a row yj . The processor then solves the equation yj = ax + b =
y2 − y1 y1 x2 − x1 y2 x+ x2 − x1 x2 − x1
for x and sets pixel (x, yj ). 4. If the number of processors is very limited, each may be assigned more than just a row or a column of pixels. As an example, let’s assume that we have an MIMD computer with 16 processors, organized in a two-dimensional grid and numbered from processor (1, 1) to processor (4, 4). If the bounding box contains n×n pixels, we can divide them evenly among the 16 processors and assign each processor a square area of m × m pixels, where m = n/4. Processor number (i, j) would, in this case, be “responsible” for the m×m square of pixels whose bottom-left corner is at position (i × Δx/m, j × Δy/m) (Figure 3.15b).
156
3.8 Scan-Converting Circles
3.8 Scan-Converting Circles Because of the high symmetry of the circle, it is possible to scan-convert it in many ways. Only a few such methods are discussed here, but many more can be found in the charming article [Blinn 87].
3.8.1 Obvious Methods Obvious methods for scan-converting a circle are easy to derive and to implement but are slow and inefficient. 2 2 2 The first obvious √ method is based on the Cartesian equation of a circle x +y = R , which yields y = R2 − x2 . This expression is used in the loop below to determine onequarter of the circle, which is then duplicated to complete the circle. for x:=0 to R step eps do y:=sqrt(R*R-x*x); plot(x,y); plot(-x,y); plot(x,-y); plot(-x,-y); end; The method is slow, but a more important drawback is that the pixels are not uniformly distributed over the quarter circle. This is a result of the equal x increments of the loop (Figure 3.16). y
x Figure 3.16: Equal Increments of x.
The next obvious method solves this problem by employing the parametric equation x = R cos θ, y = R sin θ, which expresses the circle in terms of polar coordinates. for theta:=0 to pi/2 step eps do x:=R*cos(theta); y:=R*sin(theta); plot(x,y); plot(-x,y); plot(x,-y); plot(-x,-y); end;
3 Scan Conversion
157
This method is still very inefficient because of the use of trigonometric functions and also because some pixels may be set multiple times.
3.8.2 A Circle in Polar Coordinates Expressing the circle in polar coordinates allows for a fast algorithm, even though it uses real numbers. This algorithm is presented here for a complete circle (the loop goes from 0◦ to 360◦ ), but it can easily be modified to compute just one-quarter or one octant and use symmetry to complete the circle. The circle equation in polar coordinates is x = R cos θ, y = R sin θ. A computer program computes the circle pixel by pixel, by iterating and varying a variable k from 0 to n − 1 (where n, the number of steps, is specified by the user). As a result, it is convenient to write the above equations as xk = R cos(θk ), yk = R sin(θk ), where θk = 2πk/n. The main point of this method is to compute certain trigonometric functions just once. The user has to input a small value Δθ, which the program uses as an angle increment. In each step, it increments θk+1 = θk + Δθ. We use the trigonometric identities for the sum of angles: cos(α + β) = cos α cos β − sin α sin β,
sin(α + β) = sin α cos β + cos α sin β;
from this we get xk+1 = R cos(θk+1 ) = R cos(θk + Δθ) = R [cos(θk ) cos(Δθ) − sin(θk ) sin(Δθ)] = xk cos(Δθ) − yk sin(Δθ). Similarly, yk+1 = xk sin(Δθ) + yk cos(Δθ). The program needs to compute sin(Δθ) and cos(Δθ) only once, then loop n times. A pseudo-code algorithm is listed here. (Note the quantities a and b. They are added to every pixel, which creates a circle centered at point (a, b) rather than at the origin.) input(n,delta,a,b,R); xk:=R; yk:=0; dcos:=Cos(delta); dsin:=Sin(delta); for k:=0 to n-1 do xn:=xk*dcos-yk*dsin; yn:=xk*dsin+yk*dcos; xk:=xn; yk:=yn; pixel(round(xn)+a,round(yn)+b); end; Exercise 3.9: Select Δθ = 5◦ , a = b = 0 and use this method to calculate the 18 equally-spaced points of the first quadrant of a circle of radius 1.
3.8 Scan-Converting Circles
158
3.8.3 Bresenham–Michener Circle Method The Bresenham–Michener circle method is a DDA algorithm [Bresenham √ √ 77] that is based on a loop that starts at point (0, R) and ends at point (R/ 2, R/ 2) to create one octant of the circle. Each pixel calculated is used to determine seven more pixels, in the remaining seven octants, to√create the complete circle (see also Exercise 4.18). To √ move from (0, R) to (R/ 2, R/ 2), we need to increment x and decrement y. The loop of Figure 3.17 is set such that the x coordinate is incremented in every step, but the y coordinate is only decremented in certain steps (i.e., conditionally). The results are good (i.e., the pixels are fairly uniformly distributed) because in this octant the circle is close to horizontal. In each step (except the first), the algorithm examines two points, S and T, that differ only in their y coordinates, and it selects the one that’s closer to the true circle. Exercise 3.10: Which point should be selected in case of a tie (i.e., when the circle passes exactly between Si and Pi )? The algorithm maintains a variable di (calculated using just additions, subtractions, and shifts) that is updated every step. The sign of di is used as an indicator, telling the program whether to decrement y at the step or not. The general form of the loop is as in Figure 3.17.
x:=0; y:=R; while x0 then . . y:=y-1; else ... endif; x:=x+1; endwhile;
yi−1
Pi
xi−1
Si
Ti
Figure 3.17: Main Loop of Bresenham’s Circle Algorithm.
If a point Pi−1 = (xi−1 , yi−1 ) has been selected in a certain step, then the next step should increment x from xi−1 to xi−1 + 1 and either set yi = yi−1 or decrement yi = yi−1 − 1. The next step should therefore select either point Si = (xi−1 + 1, yi−1 ) or Ti = (xi−1 + 1, yi−1 − 1). The two quantities DS and DT are defined based on the
3 Scan Conversion
159
distances from the points to the circle (Figure 3.17): 2 DS = (xi−1 + 1)2 + yi−1 − R2 ,
DT = R2 − [(xi−1 + 1)2 + (yi−1 − 1)2 ], 2 + (yi−1 − 1)2 − 2R2 . di = DS − DT = 2(xi−1 + 1)2 + yi−1
Note that these quantities are based on the distances of points S and T from the true circle. They are not the distances themselves, since this would require a square root calculation. If the circle passes closer to point S, then DS < DT and di is negative. If the circle passes closer to T, then di is positive. Hence, the sign of di indicates which point to select in iteration i. The only remaining detail is the recalculation of di in each iteration. It turns out that calculating di+1 is very simple if it is done in terms of di . This, in fact, is the main advantage of the method. We start with di+1 = 2(xi + 1)2 + yi2 + (yi − 1)2 − 2R2 . We already know that xi always equals xi−1 + 1, but the value of yi depends on the choice of point. If di > 0 (point T selected), then yi = yi−1 − 1 and di+1 = 2(xi−1 + 2)2 + (yi−1 − 1)2 + (yi−1 − 2)2 − 2R2 2 = 2[(xi−1 + 1)2 + 2xi−1 + 3] + [yi−1 − 2yi−1 + 1]
+ (yi−1 − 1)2 − 2yi−1 + 3 − 2R2 = di + 4xi−1 + 2 · 3 − 2yi−1 + 1 − 2yi−1 + 3 = di + 4(xi−1 − yi−1 ) + 10. If, however, di < 0 (point S selected), then yi = yi−1 and 2 + (yi−1 − 1)2 − 2R2 di+1 = [2(xi−1 + 2)2 ] + yi−1 2 = [2(xi−1 + 1)2 + 4xi−1 + 6] + yi−1 + (yi−1 − 1)2 − 2R2 = di + 4xi−1 + 6.
Hence, updating the value of di in either case is simple and does not require any arithmetic operations beyond addition, subtraction, and shifting. Notice also that di+1 depends on (xi−1 , yi−1 ) and not on (xi , yi ). The program should therefore update the value of d before updating the values of x and y. The initial value of di , namely d1 , is easily found by substituting (xi−1 , yi−1 ) = (0, R). This gives d1 = 2(0 + 1)2 + R2 + (R − 1)2 − 2R2 = 2 + R2 + R2 − 2R + 1 − 2R2 = 3 − 2R. The final program is shown in Figure 3.18.
160
3.8 Scan-Converting Circles procedure Bresenham(R); x:=0; y:=R; d:=3-2*R; while x0 then d:=d+4*(x-y)+10; y:=y-1; else d:=d+4*x+6; endif; x:=x+1; endwhile; if x=y then Plot8(x,y) end; {Bresenham}
procedure Plot8(x,y); Plot(x,y); Plot(-x,-y); Plot(-x,y); Plot(x,-y); Plot(y,x); Plot(-y,-x); Plot(-y,x); Plot(y,-x); end; {Plot8}
Figure 3.18: Bresenham’s Circle Algorithm.
3.8.4 The DCS Circle Method The circle and the square are different mathematical entities. Squaring the circle (constructing a square with the same area as a given circle by using only a finite number of steps with compass and straightedge) was one of the three great problems of classical geometry (the other two were the trisection of the angle and the duplication of the cube). In 1882 it was finally shown (as a direct consequence of the proof that π is transcendental) that squaring the circle is impossible, which is why today circle squarers are considered crackpots. Nevertheless, there have been connections and associations between squares and circles (or between quantities related to those geometric figures). One such relation is the sum of the infinite series of inverse squares ∞ 1 , k2 k=1
which in 1735 was shown by Leonhard Euler to equal π2 /6, a quantity related to circles because of the definition of π. The DCS circle algorithm of this section is another unexpected connection between circles and perfect squares. This simple and elegant algorithm (DCS stands for digital circle squares) determines the best pixels for a circle with radius r in terms of the distribution of square numbers in discrete intervals. Square numbers (or perfect squares) are denoted by Si and are integers of the form i2 where i is a nonnegative integer. They have a property that speeds up this algorithm. Given the square Si , its successor is easily computed by Si+1 = (i + 1)2 = Si + 2i + 1. (Notice that no multiplication is needed because 2i can be computed with a left shift of i.) This property is illustrated by the gnomon of Figure 3.19. (The term gnomon refers to the part that should be added to a figure to produce a larger figure of the same shape.) The main reference is [Bhowmick and Bhattacharya 08], while [Jha 10] is a free Mathematica package that employs animation to demonstrate the algorithm.
3 Scan Conversion
161
Figure 3.19: Square Numbers Computed by a Gnomon.
As in the Bresenham–Michener circle method (Section 3.8.3), the idea is to scan the first octant of the circle from x = 0 to x = y, start with y = r, plot a horizontal run of pixels, and decrement y by 1. While other scan conversion methods compute a y coordinate for each x coordinate, the DCS method determines the length of each horizontal run of pixels with the same y. Thus, DCS can be referred to as an interval searching algorithm. Figure 3.20 illustrates the meaning of the term “run” in DCS. The figure shows the first octant of a circle with r = 41. It is easy to see that the pixels selected by DCS for this circle form horizontal runs of lengths 7, 4, 4, 2, 2, 2, 2, 1, 1, 2, 1, 1, and 1. The run lengths generally decrease, but they may sometimes slightly increase.
Figure 3.20: First Octant Runs of Pixels in DCS.
The run lengths in the figure were determined as follows. The first run-length (7) is the number of perfect squares in the interval I0 = [0, r − 1] = [0, 40] (these are the squares 0, 1, 4, up to 36). The second run-length (4) is the number of perfect squares in the interval I1 = [r, 3r − 3] = [41, 120] (these are the squares 49, 64, 81, and 100). The third length (also 4) is the number of perfect squares in the interval I2 = [3r − 2, 5r − 7] = [121, 198] (these are the squares 121, 144, 169, and 196). In general, run length k (where k starts at 0) is the number of perfect squares in the interval Ik = [(2k − 1)r − k(k − 1), (2k + 1)r − k(k + 1) − 1]. The principle of DCS is surprisingly simple and is illustrated in Figure 3.21. At an arbitrary step in the algorithm, we are at point P = (i, j). The true circle lies on point Q = (i, j − δ), located between P and M = (i, j − 1/2) (but Q could also be between P
162
3.8 Scan-Converting Circles
and N = (i, j + 1/2) and have coordinates (i, j + δ)). We can therefore write r2 = i2 + (j − δ)2 , 0 = δ 2 − 2jδ + (i2 + j 2 − r2 ), 1 δ= 2j ± 4j 2 − 4(i2 + j 2 − r2 ) , 2 δ = j ± r2 − i2 .
y=j+1
N P
y=j
Q
M
y=j−1 x=i−1
x=i
x=i+1
Figure 3.21: Principle of DCS.
The answer to Exercise 3.10 tells us that Q cannot be M or N (i.e., a tie is impossible), which is why |δ| < 1/2 or − 12 < δ < 12 . This implies that we have to select the minus sign in the last equation and write it as δ=j−
r 2 − i2 .
(3.5)
Extending Equation (3.5) while keeping in mind that − 12 < δ < 12 , we obtain 1 1 < j − r 2 − i2 < , 2 2 1 1 2 2 − − j < − r − i < − j, 2 2 1 1 2 2 +j > r −i >j− , 2 2 2 2 1 1 < r 2 − i2 < j + , j− 2 2 1 1 j 2 − j + < r2 − i2 < j 2 + j + . 4 4 −
The terms j 2 − j, r2 − r, and j 2 + j on the last line are integers, which is why the inequality j 2 − j + 14 < r2 − i2 implies (j 2 − j) − (r 2 − i2 ) < − 14 < 0 or j 2 − j < r2 − i2 .
3 Scan Conversion
163
Similarly, the quantity (r2 − i2 ) − (j 2 + j) is an integer and is less than 1/4, so it must be nonnegative. We can therefore write j 2 − j < r2 − i2 ≤ j 2 + j, −j 2 − j ≤ i2 − r 2 < −j 2 + j, r 2 − j 2 − j ≤ i2 < r2 − j 2 + j.
(3.6)
At first, Equation (3.6) does not seem special, but it is in fact the heart of the DCS algorithm. The DCS loop starts with j = r and the first step has to determine the run length of pixels for this j. Substituting j = r in Equation (3.6) yields 0 ≤ i2 < r 2 − r2 + r = r or 0 ≤ i2 ≤ r − 1. This means that the x coordinates i of the pixels in the topmost run are the perfect squares in the interval I0 = [0, r − 1]. This result is an unexpected connection between number theory and digital circles. The DCS algorithm computes the number l0 of these squares and draws the topmost horizontal run of pixels. Variable j is now decremented by 1, and we substitute r − 1 for j in Equation (3.6). The result is the next interval I1 = [r2 − (r − 1)2 − (r − 1), r2 − (r − 1)2 + (r − 1) − 1] = [r, 3r − 3]. It indicates that the horizontal run-length of pixels for j = r − 1 equals the number of perfect squares in the interval [r, 3r − 3]. In the kth step of DCS, we substitute j = k in Equation (3.6) to obtain the kth interval where we’ll have to look for perfect squares
Ik = r2 − (r − k)2 − (r − k), r 2 − (r − k)2 + (r − k) − 1 = [(2k − 1)r − k(k − 1), (2k + 1)r − k(k + 1) − 1]. The lengths of these intervals are easily computed. The length of a general closed interval [a, b] is b − a + 1, so Equation (3.6) implies that l1 = (3r − 3) − r + 1 = l0 + (r − 2) and the length of Ik for k ≥ 1 is lk = [(2k + 1)r − k(k + 1) − 1] − [(2k − 1)r − k(k − 1)] + 1 = 2r − 2k, which suggests that the length of the next interval is lk+1 = 2r −2(k +1) = lk −2 (again, this is for k ≥ 1). Notice that the lengths get shorter by 2, but these are the lengths of intervals containing perfect squares. These are not the lengths of horizontal runs of pixels. The length of each horizontal run of pixels is computed by counting the number of perfect squares included in the corresponding interval. The authors of this algorithm show how to speed up the determination of the intervals Ik . If we denote Ik = [uk , vk := uk + lk − 1], then uk and lk can be computed from their predecessors as follows uk−1 + lk−1 k ≥ 1, uk = 0 k = 0. lk−1 − 2 k ≥ 2, lk = 2r − 2 k = 1, r k = 0. The algorithm is listed here in the C programming language (where Plot8 is a procedure that draws the pixel at location (i, j) and the seven pixels that correspond to it in the other seven octants).
164
3.8 Scan-Converting Circles
DCS (int r){ int i = 0, j = r, s = 0, w = r − 1; int l = w << 1; while (j ≥ i){ do{Plot8(i, j); s = s + i; i + +; s = s + i;} while (s ≤ w); w = w + l; l = l − 2; j − −; } } The originators of this method provide an analysis of its performance, compare it to Bresenham’s method, and even propose two variations, dubbed DCR and DCH, of the basic DCS.
3.8.5 Circle and Ellipse The parametric representation of a circle of radius R centered on the origin is C(t) = (R cos(2πt), R sin(2πt)),
where 0 ≤ t ≤ 1.
Note that this expression can also be used to determine the vertices of an n-sided regular polygon. A circle with a center at point (m, n) is given by the expression (m + R cos(2πt), n + R sin(2πt)). It is possible to represent the circle by C(t) = (x(t), y(t)) =
√
2 t 1−t ,R , R 1+t 1+t
0 ≤ t ≤ ∞.
(3.7)
This is a circle of radius R because it satisfies x2 (t) + y 2 (t) = R2 . When t varies in the range [0, 1], Equation (3.7) creates the first quadrant of the circle, since C(0) = (R, 0) and C(1) = (0, R). When t varies in the range [1, ∞], it describes the second quadrant since C(1) = (0, R) and C(∞) = (−R, 0). This is easy to see when one realizes that x(1/t) = −x(t) and y(1/t) = y(t). Hence, if a value t0 in the range [0, 1] creates a point (x, y), then the value 1/t0 (which is in the range [1, ∞]) will create the point (−x, y). An ellipse is the locus of all the points for which the sum of the distances to two fixed points, called the foci, is constant. An ellipse centered on the origin with foci at points (−c, 0) and (c, 0) is called canonical. Its implicit representation is (x/a)2 + (y/b)2 = 1, where 2a and 2b are the major and minor axes, respectively.
3 Scan Conversion T
(x,y) d
L (−c,0)
165
(c,0)
2b
d´
R
L
2a (a)
d
d´
(b)
(c)
Figure 3.22: Ellipse and Oval.
Exercise 3.11: Show that a2 = b2 + c2 . Ovals: In much the same way that an ellipse can be considered an extension of the circle, the shape of an egg can be considered a generalization of the ellipse. The shape of an egg isn’t well defined, but the great 19th-century physicist James Maxwell discovered, at the tender age of 15, how to create an entire family of oval shapes that resemble an egg [May 62]. Figure 3.22b shows how to attach a string to the left focus and to the pen (the hollow circle) such that the distance d of the pen from the left focus equals twice its distance d from the right focus. Moving the pen traces an oval shape reminiscent of an egg. Figure 3.22c shows a similar oval created by attaching a string to the right focus and to the pen in a way that keeps the sum 2d + 3d constant. The entire family of oval shapes can be created by keeping the sum ad + bd constant for various integer values of a and b. Exercise 3.12: Express the maximum and minimum values of d in terms of the length S of the string and the distance F between the foci. The ellipse can be represented parametrically similar to a circle: E(t) = (a cos(2πt), b sin(2πt)), or
E(t) =
√
2 t 1−t ,b , a 1+t 1+t
0 ≤ t ≤ 1, 0 ≤ t ≤ ∞.
The eccentricity of the ellipse measures how much it deviates from a circle. It is defined as e = c/a. For a circle, e = 0. When e = 1, the ellipse reduces to a line from (−c, 0) to (c, 0). The eccentricity of the Earth’s orbit around the Sun is ≈ 1/60. When an area is scaled (expanded or shrunk), the determinant of the scaling matrix (Page 205) equals the scaling factor. This can be used to determine the area πab of the ellipse. The equations of the circle and the ellipse are x2 +y 2 = R2 and (x/a)2 +(y/b)2 = 1. Therefore, if point (x, y) is on the circle, it can be transformed to the ellipse by the scaling transformation
a/R 0 . 0 b/R The determinant of this matrix equals ab/R2 , so the area of the ellipse equals the circle area times ab/R2 or πR2 × ab/R2 = πab.
3.8 Scan-Converting Circles
166
Superellipse This interesting and little known curve was developed by the French mathematician Gabriel Lam´e in 1818. The parametric representation of the ellipse is (a cos θ, b sin θ) where 0 ≤ θ ≤ 2π. The superellipse has the generalized representation (a cosn θ, b sinn θ), where θ varies in the same range and n is any non-negative real number. The implicit representation is |x/a|n + |y/b|n − 1 = 0. It is easy to see that the extreme values of x are −a and a, and the extreme values of y are −b and b. Thus, the curve lies inside the rectangle of width 2a and height 2b, centered on the origin. The “corner” points of the superellipse are (±σa, ±σb) where σ = 2−1/n is called the superness of the curve. The following values of n are especially interesting (Figure 3.23): 1. n = 0 (superness = 0). The superellipse becomes a rectangle. If n = 0 and a = b, it becomes a square. 2. 0 < n < 1. The superellipse becomes a rounded-corner rectangle. 3. n = 1 (superness = 0.5). The superellipse becomes an ellipse. If n = 1 and a = b, it becomes a circle. 4. n = 2 (superness ≈ 0.7071). The superellipse becomes diamond shaped. 5. n > 2 (superness > 0.7071). The superellipse becomes a pinched diamond. When n → ∞ (superness → 1), it approaches the shape of a plus sign.
(a)
(e)
(c) (d) (b)
Figure 3.23: Five Superellipses.
Check also [Lam´e 98] for an interactive JAVA applet. The superellipse was popularized by the multi-artist Piet Hein, who designed one of the Stockholm city squares as a superellipse with n = 2.5 (or superness of 2−0.4 ≈ 0.7578). Regardless of its shape, the superellipse passes through the same points when θ = 0, π/2, π, 3π/2. When a = b, the superellipse is reduced to the supercircle (a cosn θ, a sinn θ). It can be generalized to the three-dimensional superellipsoid (a cosn θ cosm φ, b cosn θ sinm φ, c sinn θ),
where − π/2 ≤ θ ≤ π/2 and − π ≤ φ ≤ π.
This solid can take many shapes, but all are bounded in the box whose dimensions are 2a × 2b × 2c.
3 Scan Conversion
167
A circle no doubt has a certain appealing simplicity at first glance, but one look at an ellipse should have convinced even the most mystical of astronomers that the perfect simplicity of the circle is akin to the vacant smile of complete idiocy. Compared to what an ellipse can tell us, a circle has little to say. Possibly our own search for cosmic simplicities in the physical universe is of this circular kind—a projection of our uncomplicated mentality on an infinitely intricate external world. —Eric Temple Bell, Mathematics: Queen and Servant of Science.
3.9 Filling Polygons Polygons and their applications to polygonal surfaces are discussed in Sections 2.18 and 9.2. Here, we discuss two types of algorithms, seed fill and scan-line fill, for filling any polygon with a given color. The problem of polygon fill is to fill the interior of a given polygon with pixels of a certain color. A related problem is boundary fill, where we want to fill a given area, bounded by pixels of a certain color, with pixels of another color.
3.9.1 Seed Fill The first approach to the polygon fill problem is a recursive seed fill algorithm. The user has to specify a start point by clicking inside the polygon, and the software starts at that point, paints it and its neighbors the required color, examines the neighbors of the neighbors, and so on. The basic seed fill algorithm works as follows: Push the seed pixel onto the stack While the stack is not empty Pop a pixel P from the stack Set P to color f For each of the four (or eight) nearest neighbors of P , If any is a boundary pixel or is already painted f , then disregard it else push it onto the stack.
This algorithm works even for polygons with holes and is simple to implement in a programming language that supports recursion. However, the recursion can get very deep (which results in slow execution) and the recursion stack may have to be very large. We therefore need a sophisticated, nonrecursive boundary seed fill algorithm and such a method is discussed here in detail. We denote the boundary pixels by • and the fill pixels by ×. The area can be of any shape, but it must be completely bounded. If the boundary is not complete, the fill algorithm may not know where to stop and it may spill out of the area. To specify the area to the fill program, the user has to point to and select one of the interior pixels to become the seed. A straightforward algorithm is the following: 1. Set the seed pixel to the fill color ×. 2. Push the four nearest neighbors of the seed into a stack, unless any of them is a boundary pixel • , or has already been colored. 3. Pop the stack. Set that pixel to the fill color, and push its four near neighbors, as indicated in step 2, into the stack.
3.9 Filling Polygons
168
4. Repeat step 3 until the stack is empty. This method is slow because of the excessive use of the stack. Also, it may easily push thousands of pixels into the stack and overflow it. A better algorithm is outlined below. It still uses a stack, but it works with rows of pixels rather than individual ones. The stack is used to indicate future rows to be filled, instead of future pixels, so its use is not excessive. Consider the area in Figure 3.24a. The seed pixel is shown as . The algorithm is the following: 1. Fill the line where the seed pixel is located with the fill color ×. Scanning left and right of the seed as far as necessary until a boundary pixel • is reached (Figure 3.24b). 2. Examine the two rows immediately above and below the current row (if there are none, go to step 3). Scan each of the two rows from right to left looking for pixels that lie immediately to the left of a boundary pixel and that haven’t been colored already. Those pixels are pushed into the stack. Figure 3.24c shows three such pixels, labeled in the order in which they were found. ◦ ◦ • • • • • • • • • •
◦ • ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ •
• ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ •
••• ◦◦◦ ◦◦◦ ••• •◦◦ ••• ◦◦ ◦◦◦ ••• •◦◦ •◦◦ •◦◦
• ◦ ◦ • ◦ • ◦ ◦ • ◦ ◦ ◦
• ◦ ◦ • • • ◦ ◦ • • • •
• ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ •
◦ • ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ •
◦ ◦ • • • • • • • • • •
(a) ◦◦•••••••◦◦ ◦•◦◦◦◦◦◦◦•◦ •◦◦◦◦◦◦◦◦◦• •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• 2• ◦• • • • • ◦•1 • ××××××××× • • ××××××××× • 4• ◦• • • • • ◦•3 •◦◦•◦◦◦•◦◦• •◦◦•◦◦◦•◦◦• ••••◦◦◦••••
◦◦•••••••◦◦ ◦•◦◦◦◦◦◦◦•◦ •◦◦◦◦◦◦◦◦◦• •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• •◦◦•••••◦◦• • ××××××××× • •◦◦◦◦◦◦◦◦◦• •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• •◦◦•◦◦◦•◦◦• ••••◦◦◦••••
◦◦•••••••◦◦ ◦•◦◦◦◦◦◦◦•◦ •◦◦◦◦◦◦◦◦◦• •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• 3• ◦• • • • • ◦•1 • ××××××××× • • ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦•2 •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• •◦◦•◦◦◦•◦◦• ••••◦◦◦••••
(b) ◦◦•••••••◦◦ ◦•◦◦◦◦◦◦◦•◦ •◦◦◦◦◦◦◦◦◦• •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• 2• ◦• • • • • ◦•1 • ××××××××× • • ××××××××× • • ×× • • • • • ◦ • 3 4• ◦• ◦ ◦ ◦ • ◦ ◦ • •◦◦•◦◦◦•◦◦• ••••◦◦◦••••
(c) ◦◦•••••••◦◦ ◦•◦◦◦◦◦◦◦•◦ •◦◦◦◦◦◦◦◦◦• •◦◦•••••◦◦• •◦◦•◦◦◦•◦◦• 2• ◦• • • • • ◦•1 • ××××××××× • • ××××××××× • • ×× • • • • • ◦ • 3 • ×× • ◦ ◦ ◦ • ◦ ◦ • • ×× • ◦ ◦ ◦ • ◦ ◦ • ••••◦◦◦••••
(d) ◦ ◦ ◦ ◦ ◦
◦ • • • ◦
• • ◦ • •
• ◦ ◦ ◦ •
• • ◦ • •
(e) ◦• •• •◦ •• ◦•
(f) • ◦ ◦ ◦ •
• • ◦ • •
◦ • • • ◦
◦ ◦ ◦ ◦ ◦
(g) Figure 3.24: Boundary Fill.
3 Scan Conversion
169
3. The pixel at the top of the stack is popped out (if the stack is empty, the algorithm terminates) and steps 1 and 2 are repeated on that pixel. The result is shown in Figure 3.24d, where two new pixels, numbered 3 and 4, have been pushed into the stack. Repeating steps 2 and 3 results in Figure 3.24e. After a few more repetitions, the situation (which is now much advanced) is as shown in Figure 3.24f. It should be an easy exercise to apply this algorithm to the rest of the example and fill up the entire figure. Note that the region to be filled may include holes and may also be concave, but it should be connected. For the purposes of boundary fill, a region such as in Figure 3.24g is considered two separate, disconnected areas, consisting of five pixels each.
3.9.2 Scan-Line Fill A better approach is to develop a scan line-fill algorithm. This type of algorithm scans the polygon in rows of pixels (which tells us that this type of algorithm is imageprecision), determining the locations of interior pixels in each row, and painting them with the fill color. The determination is done by observing the points where a scan line intersects the edges of the polygon, as illustrated in Figure 18.11. The scan line at y = 5 places three pixels in one span and four pixels in another span of the polygon. The principle is to scan from left to right, to start painting pixels the fill color as soon as the first edge is located, to stop filling when the next edge is found, and to alternate in this way until the last intersection point is found. 12 10
a
b
8 6 4 2 2
4
6
8
10 12
14
Figure 3.25: Pixels on a Scanline.
This simple operation has to solve the following problems: How to compute the intersection of each scan line with the polygon’s edges. Scan line 5 in Figure 18.11 intersects the polygon at four points, two of which (indicated by small triangles) have noninteger x coordinates. We certainly do not want complex computations, involving floating-point numbers, just to determine the intersection points. How to identify the last intersection point between the scan line and an edge. If the algorithm cannot do this, it may have to continue scanning until the right edge of the entire display monitor is reached.
3.9 Filling Polygons
170
The scan line at y = 7 intersects the polygon at three points, one of which corresponds to a vertex of the polygon. An odd number of intersections confuses the basic algorithm. Thus, an intersection of a scan line with a vertex should count as either zero or two intersections. Edge ab of the polygon is horizontal and may result in many intersection points. A correct algorithm has to pay special attention to this case. Perhaps the simplest solution to these problems is to triangulate the given polygon. Any polygon can be decomposed into a mesh of triangles, and triangles are simple geometrical figures that don’t suffer from most of the problems listed above. As we scan a triangle row by row, each row intersects the triangle at either one point (a vertex), two points (edges), or a horizontal edge. Thus, each triangle is easy to fill, which seems to solve the problem of polygon fill. Unfortunately, concave polygons (and also polygons that intersect themselves) are not easy to triangulate, so this approach can be used to fill only convex polygons. Exercise 3.13: What other geometric figure is easy to scan and fill? The following idea solves most of the problems above. Start with a list of the edges of the polygon. Each node in this list is a pair of vertices (actually, it is a pointer to a pair of vertices). Scan-convert each edge with any scan conversion method for straight lines (Section 3.1). The result of scan converting all the edges of a polygon are shown in Figure 18.12a. The points determined by the scan conversion process are placed in a list T where they are sorted by their y coordinates, and within each y, by their x coordinates. Unfortunately, list T is not what we expected. We were hoping for a list where there is an even number of points for each scan line (each y coordinate), but the list resulting from the pixels of Figure 18.12a is different. 4
2
3
1 5
(a)
(b)
Figure 3.26: Boundary Pixels of a Polygon.
A closer look at the figure shows the source of the problem. For edges whose slope is greater than 1, the scan conversion produces one pixel per y value, but scanning a shallow edge produces one pixel for each x value (the gray pixels in Figure 18.12a). Thus, the scan conversion algorithm has to be modified to produce one pixel per y value for any slope, as illustrated in Figure 18.12b. Notice that this modification results in
3 Scan Conversion
171
holes and gaps in the boundary pixels, but each scan line now has an even number of pixels (the black/white pixels are counted twice, because each is created twice, from two edges). There is still one problem. The white pixel (labeled 1) appears in list T twice, because it is a vertex (the common point of two edges), but it is obvious that we want it to appear in T only once, otherwise the third scan line from the bottom would have three points. We therefore examine pixel 1 carefully. It is a vertex of the polygon, but there are four other vertices, labeled 2 through 5, and each of them should appear in T twice. What distinguishes pixel 1 from the other four vertices? The answer, when it finally becomes clear, is obvious. The other four vertices are extreme in some sense but vertex 1 isn’t (we will call such a vertex moderate). Vertex 2, for example, is located at the highest point (maximal y coordinate) of its two member edges (edge 1–2 and edge 2–3). Vertex 5 is located at the lowest point of its two edges. These vertices are extreme, but vertex 1 is located at the ymin of one edge and the ymax of the other edge, which makes it moderate. Thus, we have to add another rule or test to our fill algorithm. Scan list T after it is sorted, looking for pairs of adjacent identical points. The two points of such a pair are endpoints of edges that meet at a vertex. If the vertex is moderate (located at the top of one edge and the bottom of the other edge), then one of the two identical points should be deleted from T . In principle, it does not matter which of the two is deleted, but further experience with this algorithm (with polygons that have horizontal edges) suggests that we should be consistent. We therefore arbitrarily decide to delete the point that is located at the bottom of its edge. In the polygon of Figure 18.12b, we delete the point at the bottom of edge 1–2 and retain the point at the top of edge 5–1. Both points correspond to pixel 1, but now this pixel appears in list T only once. Once list T is complete, the algorithm scans it. The list is sorted by scan lines (y values) and has an even number of points for each scan line. Each of these points causes the algorithm to switch its behavior. The first point starts a fill, the second point terminates the fill, the third point starts another fill, and so on. The third scan line from the top serves as an example. The first boundary pixel starts a string of four fill pixels (shown as triangles). Pixel 3 stops the fill, but the second occurrence of the same pixel starts another fill that includes this pixel and six more. Finally, the rightmost boundary pixel on this scan line stops the fill, and the next point encountered by the algorithm in T corresponds to the next scan line. Our algorithm has to be checked and modified, if needed, so that it deals properly with horizontal edges. It turns out that only the following, simple rule is needed: Delete the two endpoints of every horizontal edge from list T (rather, if the two endpoints of an edge have the same y coordinate, don’t even place them in T ). Figure 3.27 illustrates how this rule works. The figure shows a simple polygon with horizontal and vertical edges. At vertex 1 (moderate), the endpoint of edge 1–2 has been deleted, but the other endpoint (at vertex 2) remains in T . Edge 2–3 is horizontal, so it does not contribute any points to T . Thus, there is only one point at vertex 2 and it starts a fill that is terminated at vertex 3, because this vertex is also moderate (it includes only the endpoint of edge 3–4). The situation at vertex 4 is more complex. Edge 4–5 is horizontal and therefore does not contribute endpoints at vertices 4 and 5. Edge 3–4 has its bottom endpoint at
172
3.9 Filling Polygons
a 1
2
3 4
10
5 6
9 8
7
Figure 3.27: Horizontal edges in a Polygon.
vertex 4, so this endpoint is deleted from T . Thus, there are no points in T for vertex 4, and the scan line at a starts a fill that continues uninterrupted to vertex 5. The horizontal edge 9–10 does not contribute points to T , and the bottom endpoint of edge 1–10 is deleted. Thus, no points in T correspond to vertex 10 and edge 9–10 is not drawn, but the top endpoint of edge 8–9 starts a fill at vertex 9 (which terminates at edge 5–6). Exercise 3.14: What about edge 7–8? Our algorithm works (it works also for polygons with holes), but our arbitrary decision to delete the points located at the bottom of their edges has resulted in horizontal edges missing (i.e., not filled) at the bottom of the polygon. As with many other algorithms, there is more than one way to do things. Earlier we said “scan convert each edge with any scan conversion method for straight lines. . . .” We now describe an efficient method for determining the intersections of edges with scan lines. This method is based on the following observations: (1) If certain edges intersect a scan line, chances are that the same edges will intersect the next scan line (this is referred to as edge coherence and is similar to the concept of pixel correlation, discussed on Page 1035). (2) Once we have determined the intersection (the x value) of an edge with a scan line y, it is easy to compute the intersection (the new x value) of the same edge with the next scan line y + 1. Figure 3.28 illustrates the method. It shows a polygon with six vertices A = (0, 2), B = (10, 6), C = (13, 2), D = (14, 11), E = (6, 9), and F = (2, 10). We assume that the polygon is stored in memory as a list of edges, each of its nodes contains two pointers to the end vertices of the edge (see Section 9.2 for other ways to represent polygons). Thus, the polygon of Figure 3.28 is represented as the list [A, B] → [B, C] → [C, D] → [D, E] → [E, F ] → [F, A] where B is a pointer to the node for vertex B, containing (10, 6), and similarly for the other vertices. (This is not the best polygon representation, but it works well to explain our method). Two simple data structures are employed by the method, an edge table (ET) and an active-edge list (AEL). The ET is a list of nodes (often called buckets) for all the scan lines, i.e., all the y values spanned by the polygon (in our example, the polygon spans y values from 2 to 11). The bucket corresponding to k is the start of a list of edges whose ymin = k.
3 Scan Conversion 11
173 D
F
9
E
7
B
5 3 1
A
1
3
5
7
9
11
C
13
Figure 3.28: Illustrating the Coherence-Based Method.
Thus, many buckets may be empty, as illustrated by the figure. Each bucket of the ET list is either empty or points to a list of edges where each node contains the ymax of the edge, the x value of the bottom endpoint of the edge (the list is kept sorted by these values), and a term of the form 1/a, where a is the slope of the edge (i.e., (ymax − ymin )/(xmax − xmin )). The example that follows explains how the value 1/a is used to determine the intersections of the edge with consecutive scan lines. Notice that 1/a is 0 for vertical edges and is undefined for horizontal edges, but such edges are ignored by our algorithm anyway. 11 10
EF DE 10, 6, −4 11, 6, 4 → → 9 8
.. . 3
AF AB BC CD 10, 0, 1/4 6, 0, 5/2 6, 13, −1/4 11, 13, 1/9 → → → → 2 The AEL is a list of active edges (edges that intersects the current scan line). The algorithm loops over the scan lines in order of increasing y and the AEL is updated in each iteration. Updating the AEL involves the following steps: Edges in the AEL for which ymax = y are deleted. These edges will not be needed in future iterations. Intersections (i.e., x values) are computed for all the edges in the AEL, based on the intersections computed for the previous iteration. This step uses the 1/a term and is illustrated in the example below. Edges for which ymin = y + 1 are added to the AEL. They will be needed in future iterations. Once the functions of the ET and AEL are clear, the method is easy to understand. The main steps are as follows: 1. Set up the ET. Initialize the AEL to an empty list.
174
3.9 Filling Polygons
2. Set y to the smallest nonempty entry of the ET (in our case y ← 2). 3. Repeat the following steps until both the ET and AEL are empty. 4. Bucket y in the ET consists of edge nodes for edges whose ymin = y. Those are the new edges to be added to the AEL. Move them to the AEL and sort the AEL on the x field of the nodes. (Sorting may not be needed because the ET is already sorted.) 5. For each pair (xi , xi+1 ), (xi+2 , xi+3 ),. . . of successive x values in the AEL, compute the endpoints of a span as shown below, convert them to integers by rounding up (in a left edge) or down (in a right edge), and fill the pixels of the span. 6. Scan the AEL for nodes (edges) where ymax = y and remove those nodes. Those edges will not intersect future scan lines. 7. Increment y by 1. 8. Scan the AEL for non-vertical edges and update their x fields for the new y (this is explained below). In our example, we start at y = 2. We move the four nodes of ET bucket 2 to the AEL, and compute two spans, for edges AF and AB (at x = 0) and for edges BC and CD (at x = 13). Each span is a single pixel. No edge should be deleted. We increment y to 3. The ET bucket for y = 3 is empty, so nothing new is moved to the AEL in step 4. Step 5 computes the new endpoints for the span AF–AB as x = 0 + 1/4 and x = 0 + 5/2. The first value is closer to 0, but since AF is a right edge, the algorithm may round 1/4 up to 1. The second value is rounded down to 2. The two endpoints are therefore (1, 3) and (2, 3). Two more endpoints are computed for the span BC–CD. Their x values are 13 − 1/4 → 12 or 13 and 13 + 1/9 → 13. No edges are deleted from the AEL in step 8. Next, y is incremented to 4. No new edges are moved to the AEL (in fact, nothing new will be moved until y reaches 9). The new x values for the AF–AB span are 0 + 2/4 → 1 and 0 + 10/2 → 5. The span from (1, 4) through (5, 4) is filled. The new x values for the span BC–CD are 13 − 2/4 → 12 and 13 + 2/9 → 13. The span from (12, 4) to (13, 4) is also filled. No edges are deleted. Exercise 3.15: When will the first edges be deleted? At y = 5, the x values computed for the two spans are 0 + 3/4 → 1 through 0 + 15/2 → 7 and 13 − 3/4 → 12 through 13 + 3/9 → 13. The rest of the example is easy to complete. The last point that needs be discussed is computing the extreme x values for the spans (step 5 above). The explicit equation of a straight line (Section 9.1) is y = a x + b where a is the slope (yi+1 − yi )/(xi+1 − xi ) of the line. In the special case where yi+1 = yi + 1, the slope becomes a = 1/(xi+1 − xi ) or xi+1 = xi + 1/a. Thus, given an x value of an endpoint for scan line y, the x value for the next scan line y + 1 is x + 1/a rounded up or down to the nearest integer depending on the location (left or right) of the edge in the polygon. As y is incremented, successive values for x have the form x + k/a, which makes them easy to compute. The software has to save the integer part of x and the values of k and m. When k becomes equal to m, the integer part is incremented by 1 and k is reset to 0. Stroke and fill The concept of a stroke was introduced by PostScript (Section 20.5). PostScript is based on entities called paths, where a path can be a polyline, a polygon, or a curved region (open or closed). A path is a two-dimensional object with
3 Scan Conversion
175
two attributes, stroke and fill. The term stroke refers to the edge of the path. It can be thick or thin, it can be in any shade of gray or in any color, and it can be a pattern. Figure 20.3 shows examples of strokes and fills. The point is that the pixels shown in Figure 18.12a constitute the stroke of the polygon. They are not sufficient to fill the polygon, but they are nevertheless useful. Note. Because of the discrete nature of pixels, certain polygons may look bad when filled, regardless of the filling algorithm used. This is especially true for very narrow polygons (and for narrow regions in general), such as the pathological one depicted in Figure 3.29.
Figure 3.29: A Very Narrow (Pathological) Polygon.
3.10 Pattern Filling Once we know how to fill a polygon with a given color, the next step is to find out how a polygon can be filled with copies of a pattern. The pattern is a small M × N rectangle of pixels, and the problem is to fill the interior pixels of the polygon with bits from the pattern such that the pattern will repeat. The main concern here is the placement of the pattern within the polygon. Figure 3.30 shows three variations of pattern fill that illustrate this issue. In part (a) of the figure, The first pixel selected by the fill algorithm (normally the bottom endpoint of the lowest-left edge) is considered the anchor and is filled with the bottom-left pixel of the pattern. If the polygon is moved, the pattern inside the polygon does not change. In part (b) of the figure, the user selects an anchor in the polygon (the thick, short arrow) and the fill algorithm sets this to the bottom-left pixel of the pattern. This is handy in cases where the user wants to try various placements of the pattern in the polygon. In part (c) we assume that the pattern covers the entire screen permanently, but is invisible. When a polygon is placed on the screen and the pattern-fill algorithm is executed, that part of the pattern behind the polygon becomes visible. When the polygon is moved, the pattern in it seems to move in the opposite direction. The technique of part (c) is referred to as absolute anchoring and it has the advantage that polygons may overlap and butt each other seamlessly. A programmer implementing a pattern-fill algorithm may include various options, such as selecting the anchor at random, permitting the user to specify gaps between copies of the pattern, or allowing for triangular patterns that are placed like this . . . .
3.10 Pattern Filling
176
(b) (a) (c)
Figure 3.30: Polygons with Patterns.
Exercise 3.16: Suggest another option that makes sense for a pattern-fill algorithm. The basic mathematics of pattern fill is easy to derive. It employs the modulo operation p mod q, which computes the remainder r of the division p/q. This operation can also be written in terms of the floor operator r =p−q
p , q
where the remainder has the sign of the divisor q. Figure 3.31 illustrates the application of the modulo to our problem. Once the anchor point (a, b) has been determined, by any method, it is painted the color of the bottom-left pixel (pixel (0, 0)) of the pattern. Thus, (a, b) ↔ (0, 0). Obviously, point (a + 1, b) in the polygon corresponds to point (1, 0) in the pattern, and so on for M points. Beyond that, the pattern repeats, so (a + M, b) ↔ (0, 0). x-a g=M M (x,y) r M-r
N
(x,y)
M (a,b)
r = x-a-g
Figure 3.31: The Modulo Operation.
In general, to determine the pattern point that corresponds to polygon point (a + k, 0), we take the distance x − a between the points, and subtract the largest multiple of M that fits into the distance. If the remainder is r, then (a + k, 0) ↔ (r, 0). This computation is easily applied to the y coordinate. The figure shows that polygon point (x, y) should be painted the color of pattern point x−a y−b x−a−M ,y − b − N . M N What about points (x, y) to the left of or below the anchor (a, b)? Figure 3.31 suggests the following. First, count the largest integer multiple of M between x and a,
3 Scan Conversion
177
then compute the remaining distance r, and finally subtract M − r to obtain the correct pattern point. Thus, given polygon point (x, y) to the left of (a, b), the x coordinate of the pattern point that corresponds to it is a−x , M −r =M − a−x−M M and similarly for the y coordinate. In practice, the modulo computations shown here are needed only once in a while. Recall that polygon filling is often done in scan lines, filling one or several spans in each line. If the leftmost point of a span is (x, y), then the modulo computations need be done once, to determine the pattern point (p, q) corresponding to (x, y). Filling the rest of the span is done by selecting pattern point (p + k, q) and incrementing k from 0 to M − 1 (i.e., modulo M ). Skipping to the start of the next span (a distance d) is done by incrementing k by d modulo M .
3.11 Thick Curves So far, we have implicitly assumed that our curves are one-pixel wide. How can we scanconvert thick lines and other geometric primitives? Two simple methods are described here: 1. Replicating pixels. The user specifies the width w of the line. Any desired scan-converting method is used to determine the best pixels. If the line is close to horizontal, each pixel selected is replicated by turning on several pixels above it and below it (Figure 3.32a), to obtain a column that is w pixels high. If the line is close to vertical, several pixels to its left and right are turned on instead. This works better when w is odd; even values of w result in a pixels replicated on one side of the line and a + 1 pixels replicated on the other side. This is a simple, fast method, but it produces lines shaped like parallelograms instead of rectangles (Figure 3.32b). Such lines do not connect√very well and do not always have the right width (if the slope is 45◦ , the width is w/ 2; see Figure 3.32b). When drawing a curve with this method, the program should constantly check the slope to determine whether horizontal or vertical replication is needed. For lines that are not very thick, however, this method may be satisfactory. 2. Using a drawing pen. The user specifies the shape of a pen, which can be a square, a rectangle, a circle, or anything else. The program employs any scan-converting method to determine the best pixels, and draws the shape of the pen (its footprint) centered on every pixel selected (Figure 3.32c). This is slow, since successive copies of the pen may highly overlap, but the results are often visually acceptable. Neither method is ideal. Working with thick lines and curves raises new questions. If the pen is not circular, does it turn while drawing a thick curve? What shape should the endpoints be? How should thick straight segments connect? How to “fatten” a thin line if its thickness should be an even number of pixels? Figure 3.33 illustrates several variations that may be ideal in some applications (but may look bad in others). On the left, we see three types of caps (or line ends), butt,
3.11 Thick Curves
178
w
... ... 0.7071w ...
... ... ... ... ...
w (a)
(b)
(c)
Figure 3.32: Thick Lines.
round, and projecting square. These are followed by four types of joins. Line segments with butt ends do not join well, so drawing and illustration software often offer joining options such as round caps, mitered join, and trimmed (or beveled) miter.
Figure 3.33: Thick Lines Caps and Joins.
How can we draw thick straight segments with uniform thickness regardless of their slopes? Given a segment between (x1 , y1 ) and (x2 , y2 ), its slope a is (y2 − y1 )/(x2 − x1 ). The negate and exchange rule (Page 207) tells us that rotating the segment by 90◦ changes its slope to b = −1/a (Figure 3.34a). Thus, when the original segment is scan converted, we need to replicate each pixel in the direction given by b. Lets concentrate on one pixel (p, q) selected by whatever scan conversion algorithm that we use. If the line segment is close to horizontal (i.e., if the slope satisfies −1 < a ≤ 1), then it will consist of short horizontal strings of pixels (the black pixels in Figure 3.34b), so thickening the line by extending pixels should be done in the general vertical direction (the white pixels in the figure).
/a
1 e=− Slop
nt Segme
e=a Slop (a)
(b)
Figure 3.34: Fattening a Thin Line Segment.
3 Scan Conversion
179
The pixels added around (p, q) should have y coordinates of q + 1, q − 1, q + 2, q − 2, and so on. The first of those will therefore have coordinates (x, q + 1), where x has to be determined. The slope from (p, q) to (x, q + 1) is [(q + 1) − q]/(x − p), which implies x = p − a. The difference p − a is rounded off and the pixel is drawn. Proceeding in the same way, it is easy to verify that the x coordinate of the next pixel (with y = q + 2) is p − 2a. Similarly, the x coordinates of the pixels with y coordinates q − 1, q − 2,. . . are the rounded values of p + a, p + 2a, and so on. The point is that once these coordinates have been determined for one pixel (p, q), there is no need to repeat the computations for other pixels of the segment. Once the software has determined the positions of the white pixels around (p, q), it can paint new pixels (the white empty pixels in the figure) in the same relative positions around any of the black pixels of Figure 3.34b.
3.12 General Considerations When a new scan-converting algorithm is developed or when an existing algorithm is selected for practical use, the following three points should be considered: 1. Is the algorithm symmetric? A symmetric scan-conversion algorithm results in the same set of pixels when run from P1 to P2 as when run from P2 to P1 . Most scan-conversion algorithms are not symmetric, which may result in issues when paths are being drawn and erased. In such a case, it is important not only to use the same algorithm to draw and to erase a path but also to remember which endpoint is P1 and which is P2 . Exercise 3.17: Is the quadrantal DDA method symmetric? 2. Does the algorithm compute any pixels more than once? This may not be important in current computers, where the bitmap is located in memory and the output device is a display monitor. It was, however, important in some old types of displays, such as the (long obsolete) Tektronix storage tubes, where setting a pixel multiple times made it brighter. Exercise 3.18: Does the symmetrical DDA method satisfy point 2? 3. Are points generated in order of minimal adjacent distance? This is important when the output device is a pen plotter. In such a device, it is essential to minimize pen travel in order to save time and to end up with high-quality output. An algorithm that sends the pen all over the place would not only be very slow on a plotter, but may result in poor plotting quality. This feature of the algorithm may even be significant when the bitmap is stored in RAM, since parts of the RAM may be loaded into a cache memory, and setting pixels all over the bitmap may cause unnecessary loading of memory pages into cache. Exercise 3.19: The Bresenham–Michener circle method (Section 3.8.3) calculates pixels in the second octant and uses each pixel to calculate and plot seven more pixels. Suggest a way to modify it to satisfy point 3.
180
3.13 Antialiasing
3.13 Antialiasing All scan-converting methods result in jagged lines. It turns out that the human eye is very sensitive to jagged edges. Reducing the pixel size improves the appearance of jagged edges, but the problem still persists. Better-looking lines and curves can be generated with an antialiasing method. All the antialiasing methods are based on the observation that a pixel, rather than being a mathematical, dimensionless dot, occupies a small area on the screen. Similarly, a line displayed on the screen or printed on paper has some finite width and is not infinitely thin. For a given straight line, even when the best pixels are selected, only some will be centered on the ideal line, and the rest will be off to some extent. The amount by which a pixel is off the line is the distance between its center and the ideal line. This suggests an intuitive method for antialiasing; paint each pixel a shade of gray (or some other color) inversely proportional to its distance from the line. Several examples are shown in Figure 3.35. Especially interesting is example (a) at the bottom-left corner of the figure, where the shades of gray are listed inside each pixel. It is immediately clear that the result (shown in real size in the inset) is very unsatisfactory. It consists of dark and bright areas and looks worse than an aliased line. It is obvious that a good antialiasing method should generate lines where each region has the same average brightness. Figure 3.35b (the three lines at the bottom-right corner) shows how this can be achieved. The top line consists of several nonoverlapping horizontal segments, similar to those generated by octantal DDA (Section 3.3.5) and is aliased; its smaller version is jagged. The middle line illustrates an attempt to blend the individual segments. Each segment receives additional pixels preceding and following it. The pixels preceding the segment become darker, while those following the segment become progressively brighter. The resulting line has two pixels for each x position. The small version looks good, since the eye is directed gradually from one segment to the next. The bottom line is similar but looks even better because the original black segments have been changed to gray. The principle behind such a line is that the two pixels at each x position should have intensities that add up to 100% and are also inversely proportional to their distances from the ideal line. Jaggies: /jag’eez/ n. The “stairstep” effect observable when an edge (especially a linear edge of very shallow or steep slope) is rendered on a pixel device (as opposed to a vector display). —Eric Raymond, The Hacker’s Dictionary.
It should be noted that antialiasing is the opposite of dithering (Section 2.28). Antialiasing adds gray pixels to a black and white image in order to improve its appearance on a grayscale output device, while dithering converts a grayscale image to a black and white image, in order to obtain better output on a monochromatic output device. This section uses the term “grayscale” but the methods described here apply to shades of any color, not just gray. It is possible to antialias a red and white image by using shades of red instead of just maximum red. Reference [Chang 10] employs animation and shades of green to illustrate antialiasing in color.
3 Scan Conversion
181
15 15 35 90
85
25
25
10
15
35
(a)
(b) Figure 3.35: Antialiasing and Gray Pixels.
Another way of looking at antialiasing is as a way to increase the effective resolution of the output device. There are two main approaches to antialiasing. One approach is to extend an existing scan-conversion algorithm so it can compute the correct grayscales for the pixels while the image is being generated. This approach is illustrated in Sections 3.13.1 through 3.13.3. The other approach uses unmodified algorithms to create an entire
3.13 Antialiasing
182
aliased image, then antialiases it by scanning the bitmap and performing certain operations on the values stored there. This approach is illustrated in Sections 3.13.4 and 3.13.5.
3.13.1 The Wu Line Antialiasing Method We start with simple DDA, a method described in Section 3.3.1, and extend it to generate an antialiased line. The discussion here is limited to lines that are close to horizontal (with slopes between zero and 45◦ ), but the generalization to any slopes is straightforward. Given the two points (x1 , y1 ) and (x2 , y2 ), simple DDA starts by calculating the slope a of the line segment between them. If a is in the interval [0, 1], the method sets x to x1 and y to y1 , then goes into a loop where in each iteration it plots a pixel at (x, y), increments x by 1 and increments y by a to stay on the line. Since y is a noninteger, its rounded value should be used when plotting a pixel. This process is summarized by the psuedo-code of Figure 3.36a and it results in an aliased line. (Notice that the code in the figure uses truncation instead of rounding. This is done to accommodate the simple DDA to Wu’s method.) var x,x1,x2: integer; a,y: real; a:=(y2-y1)/(x2-x1); x:=x1; y:=y1; repeat
var x,x1,x2,numgray: integer; a,b,y:real; calculate a and b; x:=x1; y:=y1; repeat d=y-trunc(y); gray1=(1-d)*numgray; gray2=d*numgray; pixel(x,trunc(y),gray1); pixel(x,trunc(y)+1,gray2); x:=x+1; y:=y+a; until x>x2;
pixel(x,trunc(y)); x:=x+1; y:=y+a; until x>x2;
(a)
(b)
Figure 3.36: Simple DDA (a) Aliased, (b) Antialiased.
increment y 0
13
26
39
52
100
87
74
61
48
(a)
0 65
77
90 100
35
23
c d
10
(b)
Figure 3.37: Distances Between Line and Pixels.
Figure 3.37 shows how antialiasing is added to this simple method (the numbers inside the pixels are grayscales in percentages). In each iteration, the distance d between the center of the selected pixel P = (x, trunc(y)) and the line y = ax+b is measured and a grayscale inversely proportional to d is selected for the pixel. (Since d is in the range [0, 1], that grayscale should be proportional to 1 − d.) In addition, the pixel immediately
3 Scan Conversion
183
above P (or, if the slope is negative, the pixel below P ) is also plotted, with a grayscale inversely proportional to its distance c from the line (or, similarly, proportional to 1 − c, but 1 − c = d, since the distance between the centers of the two pixels is one unit). This pixel is the companion of P . Because of the nature of human vision, the effect of plotting two pixels with complementary intensities (i.e., intensities that add up to 1) at each x position creates the impression of a single, full-intensity pixel, positioned precisely on the line. A simple analogy is two masses positioned at different points. Their combined effect is identical to that of a single mass, positioned at the center of gravity of the two original masses. We assume 2n grayscales, ranging from white (0) to black (2n − 1, numgray in Figure 3.36b.) Under this assumption, the grayscale for the main pixel should be (1 − d)(2n − 1) and the intensity of its companion pixel should be d · (2n − 1). Figure 3.36b is pseudo-code for this extended method. Even though this method is easy to understand and produces lines that are pleasing to the eye, it is inefficient. The value of y is real and has to be rounded off before a pixel can be plotted. Also, the calculation of the grayscale value for each of the two pixels involves a multiplication followed by rounding the product to the nearest integer (because d is real). The rest of this section shows how this simple algorithm can be redesigned and made efficient. The ideas presented here are due to Xiaolin Wu [Wu 91] and are also discussed (with C code included) in [Abrash 92]. An alternative C code is available at [Elias 01]. The slope of a line is a real number and real numbers are normally represented in the computer in floating-point. Floating-point operations are slow and should be avoided, but the slope is definitely not an integer. It turns out that in addition to integer and floating-point there is another way to represent numbers in the computer, namely fixedpoint. When an integer is stored in a register or in memory, we implicitly assume that there is a binary point to the right of the number. In a fixed-point representation, the binary point can be located at any position. The binary point itself is not stored in the computer. A fixed-point number consists of the magnitude bits (and perhaps a sign bit) and the routines that operate on these numbers are told where the point is assumed to be. In our application, the slope is a fraction in the range (0, 1), so it can be stored as a fixed-point number where the binary point is assumed to be to the left of the number. We can call such a representation a 0.16 fixed-point scheme (a 16-bit number where the point is at position 0). Before discussing any details, two observations that justify the use of this scheme should be mentioned. (1) Horizontal, vertical, and diagonal (45◦ ) lines do not require antialiasing because they pass through the center of all their pixels, so they are better dealt with separately. This is why we can assume that the slope is in the open interval (0, 1). (2) For lines close to vertical, the slope a is greater than 1, but Section 3.3.1 shows that for these lines we need the quantity 1/a, and this number is again in the interval (0, 1). When the 16-bit unsigned number 110. . . 0 is interpreted as an integer, it is a large number. It equals 214 +215 = 49,152. When the same bit pattern is interpreted as a 0.16 fixed-point number, it equals 0.112 = 2−1 + 2−2 = 0.75. Similarly, the 0.16 fixed-point number 1010. . . 0 equals 0.1012 = 2−1 + 2−3 = 0.625. To avoid the use of floating-point numbers, the slope is computed as a 0.16 fixed-point number in a variable v, and another
3.13 Antialiasing
184
variable V (which starts from zero) is incremented by v in each iteration of the loop. We use the line segment of Figure 3.37a as an example. This segment goes up one unit in the y direction for every eight units in the x direction, so its slope is 1/8 = 0.0012 . . . 0 . When v When stored as a 0.16 fixed-point number in v, the slope becomes 0010 . is repeatedly added to V , the latter variable goes through the values 00000 . . . 0, 00100 . . . 0, 01000 . . . 0, 01100 . . . 0, 10000 . . . 0, 10100 . . . 0, 11000 . . . 0, 11100 . . . 0,
13
(3.8)
and then wraps back to 00000 . . . 0. This is an indication to the algorithm to increment y. In pseudo-code, this part is VT:=V; V:= V + v; if V < VT then y:=y+1; The main point is that incrementing V by v is done by integer addition, as if the two variables were integers. This is how floating-point numbers and operations are avoided. The next point is the computation of the grayscale values for the two pixels in each iteration. This process is also greatly simplified because of the way variable V is maintained. This variable indicates how far the current pixel is from the ideal line. Since V is represented in fixed-point, its significant bits are on the left, so the leftmost n bits of V can serve as an indication of the grayscale needed for the current pixel. A glance at Equation (3.8) shows a simple relation between the successive values of V and the grayscales of Figure 3.37a. It turns out that the n most significant bits of V are the correct grayscale for the companion pixel. Since the grayscales for the two pixels are complementary (add up to 1), the grayscale for the main pixel is given by the complement of those n bits (one’s complement, since these bits represent an unsigned number). As an example, consider the case of 32 = 25 grayscales. In this case, n = 5 and the five leftmost bits of V get the values 00000, 00100 = 4, 01000 = 8, 01100 = 12, 10000 = 16, 10100 = 20, 11000 = 24, and 11100 = 28,
(3.9)
before they reset to 00000. When computed as percentages of 31 (the largest 5-bit integer), these numbers become 0, 13, 26, 39, 52, 65, 77, and 90 %, respectively. These are the values shown in the figure for the grayscales of the companion pixels. The complements of the values in Equation (3.9) are 11111 = 31, 11011 = 27, 10111 = 23, 10011 = 19, 01111 = 15, 01011 = 11, 00111 = 7, and 00011 = 3. When computed as percentages of 31, these numbers become 100, 87, 74, 61, 48, 35, 23, and 10 %. These are the grayscales shown in the figure for the main pixels.
3 Scan Conversion
185
The pseudo-code below summarizes the main steps in computing the two grayscales. The quantity shft is set to 16 (the size of V ) minus 5 (the value of n). Thus, V is shifted to the right 11 positions, which brings its five most significant bits to the right end. The constant 31 is simply five bits of 1. When used as the second operand of an xor, this constant complements the bits of the first operand. Thus, calculating the two grayscales involves a shift followed by an xor, two simple (i.e., fast) operations. shft:=16-5; companion:=V>>shft; main:=companion xor 31; pixel(x,y,main); pixel(x,y+1,companion); Because of the two features above (the use of fixed-point arithmetic and the way pixel intensities are computed) this algorithm is extremely fast, in addition to producing excellent results. Another way to speed it up is to use the natural symmetry of a line segment (point 1 on Page 145). The line can be drawn from both ends toward its middle, plotting four pixels, two main and two companions, in each iteration. The main steps are listed in Figure 3.38, where it is assumed that x1 < x2 and y1 < y2 . After plotting four pixels, two at each end of the line, the algorithm brings x1 and x2 closer. If a decision is made to increment the y coordinate, then y1 is incremented while y2 is decremented, thereby creating a line segment symmetric about its center. var x1,y1,x2,y2,maxgray,shft,V,v: integer; shft:=16-5; V:=0; v:=(y2-y1)<<16 /(x2-x1); repeat companion:=V>>shft; main:=companion xor 31; pixel(x1,y1,main); pixel(x1,y1+1,companion); pixel(x2,y2,main); pixel(x2,y2+1,companion); VT:=V; V:= V + v; if V < VT then y1:=y1+1; y2:=y2-1; x1:=x1+1; x2:=x2-1; until x2>x1; Figure 3.38: Symmetric Wu Algorithm.
3.13.2 The Wu Circle Antialiasing Method The Wu line antialiasing method of Section 3.13.1 can be modified to generate antialiased circles. The resulting algorithm, described in this section, is simple to implement, is fast, and produces circles that look pleasing to the eye. However, since a circle is geometrically
3.13 Antialiasing
186
more complex than a straight line segment, the Wu circle method [Wu 91] is not as elegant as his line method and uses precomputed data from a built-in table. The algorithm determines the best pixels for the first octant of the circle (Figure 3.39d) by looping from point (r, 0) to the end of the octant, where x = y. The other seven octants are done by symmetry. Since the first octant is