Official New Features Guide to SYBASE ASE 15

“Sybase has been dead a very long time. May they rest in peace.” Larry Ellison CEO, Oracle Inc.

This page intentionally left blank.

Several years later…

The Official New Features Guide to Sybase ASE 15 ®

Brian Taylor, Naresh Adurty, Steve Bradley, and Carrie King Taylor with Mark A. Shelton and Jagan Reddy

Wordware Publishing, Inc.

Library of Congress Cataloging-in-Publication Data Taylor, Brian. The official new features guide to Sybase ASE 15 / by Brian Taylor ... [et al.]. p. cm. Includes index. ISBN-13: 978-1-59822-004-9 ISBN-10: 1-59822-004-7 (pbk.) 1. Client/server computing. 2. Sybase. I. Taylor, Brian (Brian Robert), 1972QA76.9.C55O4 2006 005.2'768--dc22 2005036445

.

© 2006, Wordware Publishing, Inc. All Rights Reserved 2320 Los Rios Boulevard Plano, Texas 75074 No part of this book may be reproduced in any form or by any means without permission in writing from Wordware Publishing, Inc. Printed in the United States of America

ISBN-13: 978-1-59822-004-9 ISBN-10: 1-59822-004-7 10 9 8 7 6 5 4 3 2 1 0602

Sybase, Adaptive Server, and the Sybase Fibonacci symbol are registered trademarks of Sybase, Inc. in the United States of America and/or other countries. Other brand names and product names mentioned in this book are trademarks or service marks of their respective companies. Any omission or misuse (of any kind) of service marks or trademarks should not be regarded as intent to infringe on the property of others. The publisher recognizes and respects all marks used by companies, manufacturers, and developers as a means to distinguish their products. This book is sold as is, without warranty of any kind, either express or implied, respecting the contents of this book and any disks or programs that may accompany it, including but not limited to implied warranties for the book’s quality, performance, merchantability, or fitness for any particular purpose. Neither Wordware Publishing, Inc. nor its dealers or distributors shall be liable to the purchaser or any other person or entity with respect to any liability, loss, or damage caused or alleged to have been caused directly or indirectly by this book. Portions of this book contain charts, graphs, tables, and other materials that are copyrighted by Sybase, Inc., and are used with permission.

All inquiries for volume purchases of this book should be addressed to Wordware Publishing, Inc., at the above address. Telephone inquiries may be made by calling: (972) 423-0090

SYBASE, INC., AND ITS SUBSIDIARIES DO NOT TAKE ANY RESPONSIBILITY FOR THE CONTENT OF THE BOOK. SYBASE DISCLAIMS ANY LIABILITY FOR INACCURACIES, MISINFORMATION, OR ANY CONTENT CONTAINED IN, OR LEFT OUT OF THIS BOOK.

This page intentionally left blank.

Dedications

To my family, tons! — BRT To Karen, Mom, and Dad — NA To my best friend and wife, Carol — SWB To my boys — CKT

vii

This page intentionally left blank.

Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . xxi About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . xxv Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Part I: New Features Overview Chapter 1

Exploring the Sybase Galaxy . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Part I — New Features Overview . . . . . . . . . . . . . . . . . . . 4 System Maintenance Improvements. . . . . . . . . . . . . . . . 4 Partition Management — Semantic Partitions . . . . . . . . . . 5 Scrollable Cursors . . . . . . . . . . . . . . . . . . . . . . . . . 5 Overview of Changes to Query Processing . . . . . . . . . . . . 6 Detection and Resolution of Performance Issues in Queries . . . 6 Computed Columns . . . . . . . . . . . . . . . . . . . . . . . . 7 Functional Indexes. . . . . . . . . . . . . . . . . . . . . . . . . 7 Capturing Query Processing Metrics . . . . . . . . . . . . . . . 8 Plan Viewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Sybase Software Asset Management (SySAM) 2.0 . . . . . . . . 9 Installation of ASE 15 . . . . . . . . . . . . . . . . . . . . . . . 9 Part II — Pre-15 Improvements . . . . . . . . . . . . . . . . . . . 10 Multiple tempdb Databases . . . . . . . . . . . . . . . . . . . 10 MDA Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Java and XML . . . . . . . . . . . . . . . . . . . . . . . . . . 11 The Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Sample Certification Exam. . . . . . . . . . . . . . . . . . . . 11 Use Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3, 2, 1, Contact! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Chapter 2

System Maintenance Improvements . . . . . . . . . . . . . Recent Pre-ASE 15 Improvements . . . . . . . . . . Multiple tempdb . . . . . . . . . . . . . . . . . Native Data Encryption/Security Enhancements Automatic Database Expansion . . . . . . . . . The Basics . . . . . . . . . . . . . . . . . . Job Scheduler. . . . . . . . . . . . . . . . . . . Basic Components. . . . . . . . . . . . . . Installation of Job Scheduler . . . . . . . . ASE 15 Improvements . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13 13 14 14 15 16 18 18 18 19 ix

Contents

Row Locked System Catalogs . . . . . . . . . . . . . . . . . Update Statistics . . . . . . . . . . . . . . . . . . . . . . . . Updates to Partition Statistics . . . . . . . . . . . . . . . Automatic Update Statistics . . . . . . . . . . . . . . . . . . Datachange . . . . . . . . . . . . . . . . . . . . . . . . . . . Why Use datachange? . . . . . . . . . . . . . . . . . . . Datachange, Semantic Partitions, and Maintenance Schedules Local Indexes. . . . . . . . . . . . . . . . . . . . . . . . . . Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . sp_helpindex . . . . . . . . . . . . . . . . . . . . . . . . . . Partition-level Utilities . . . . . . . . . . . . . . . . . . . . . Partition Configuration Parameters . . . . . . . . . . . . Utility Benefits from Semantic Partitions . . . . . . . . . Partition-specific Database Consistency Checks (dbccs) . Reorg Partitions . . . . . . . . . . . . . . . . . . . . . . Changes to the bcp Utility. . . . . . . . . . . . . . . . . Truncate Partitions . . . . . . . . . . . . . . . . . . . . Very Large Storage System . . . . . . . . . . . . . . . . . . . . . Disk Init . . . . . . . . . . . . . . . . . . . . . . . . . . . . Large Identifiers . . . . . . . . . . . . . . . . . . . . . . . . Long Identifiers . . . . . . . . . . . . . . . . . . . . . . Short Identifiers . . . . . . . . . . . . . . . . . . . . . . Unicode Text Support. . . . . . . . . . . . . . . . . . . . . . . . New Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . New Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . Deprecated Functions . . . . . . . . . . . . . . . . . . . . . . . . New Configuration Parameters . . . . . . . . . . . . . . . . . . . Eliminated Configuration Parameters. . . . . . . . . . . . . . . . New Global Variables. . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 3

x

Semantic Partitions and Very Large Database (VLDB) Support Introduction . . . . . . . . . . . . . . . . . . . . . . . Why Partition Data? . . . . . . . . . . . . . . . . Benefits of Partitioning. . . . . . . . . . . . . . . Partition Terminology . . . . . . . . . . . . . . . Semantic Partitions . . . . . . . . . . . . . . . . . . . Configuring ASE for Semantic Partitioning . . . . Partition Support in ASE 15 . . . . . . . . . . . . Partition Types . . . . . . . . . . . . . . . . . . . Range Partitioning. . . . . . . . . . . . . . . Hash Partitioning . . . . . . . . . . . . . . . List Partitioning . . . . . . . . . . . . . . . . Round-robin Partitioning . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19 20 20 22 22 27 28 31 31 32 33 34 35 35 38 39 43 44 45 45 46 46 47 47 48 50 51 52 52 52

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

53 53 55 56 57 61 62 63 64 65 70 75 78

Contents

Partitioning Strategies. . . . . . . . . . . . . . . . . . . . . . . . . 83 Inserting, Updating, and Deleting Data in Partitions . . . . . . . . . 84 Inserting Data into Semantic Partitions . . . . . . . . . . . . . 84 Inserting Data into Range Partitions . . . . . . . . . . . . . . . 84 Inserting Data into Hash Partitions. . . . . . . . . . . . . . . . 86 Inserting Data into List Partitions . . . . . . . . . . . . . . . . 86 Deleting Data from All Semantic Partitions . . . . . . . . . . . 86 Updating Data in All Semantic Partitions . . . . . . . . . . . . 86 Built-in Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Data Partition Implementation and Upgrade Strategies . . . . . . . 89 Index Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Local Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Clustered Prefixed Index on Range Partitioned Table . . . 95 Clustered Non-Prefixed Index on Range Partitioned Table . . . . . . . . . . . . . . . . . . . . . 97 Clustered Prefixed Index on List Partitioned Table . . . . . 99 Clustered Non-Prefixed Index on List Partitioned Table . . . . . . . . . . . . . . . . . . . . 101 Clustered Prefixed Index on Round-robin Partitioned Table . . . . . . . . . . . . . . . . . . . . 104 Clustered Non-Prefixed Index on Round-robin Partitioned Table . . . . . . . . . . . . . . . . . . . . 106 Clustered Non-Prefixed Index on Hash Partitioned Table . . . . . . . . . . . . . . . . . . . . 108 Clustered Prefixed Index on Hash Partitioned Table . . . 110 Global Index . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Global Nonclustered Prefixed Index on Range Partitioned Table . . . . . . . . . . . . . . . . . . . . 114 Global Nonclustered Prefixed Index on List Partitioned Table . . . . . . . . . . . . . . . . . . . . 116 Global Nonclustered Prefixed Index on Round-robin Partitioned Table . . . . . . . . . . . . . 118 Global Nonclustered Prefixed Index on Hash Partitioned Table . . . . . . . . . . . . . . . . . . . . 120 Query Processor and Partition Support . . . . . . . . . . . . . . . 122 ASE 15 Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Partition Maintenance . . . . . . . . . . . . . . . . . . . . . . . . 124 Altering Data Partitions. . . . . . . . . . . . . . . . . . . . . 124 Unpartition a Table. . . . . . . . . . . . . . . . . . . . . 125 Change the Number of Partitions . . . . . . . . . . . . . 126 Add a Partition to a Table . . . . . . . . . . . . . . . . . 126 Drop Partitions . . . . . . . . . . . . . . . . . . . . . . . 130 Modifications to the Partition Key . . . . . . . . . . . . . 131 Partition Information . . . . . . . . . . . . . . . . . . . . . . . . 134 xi

Contents

Influence of Partitioning on DBA Activities . . . . . . . . . . . . 143 Influence of Partitioning on Long-time Archival . . . . . . . . . . 143 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Chapter 4

Chapter 5

xii

Scrollable Cursors . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . Scrollable Cursor Background. . . . . . . . . . . . . . . Cursor Scrollability . . . . . . . . . . . . . . . . . . . . Cursor-related Global Variables . . . . . . . . . . . . . . Changes to the sp_cursorinfo System Procedure . . . . . Be Aware of Scrollable Cursor Rules! . . . . . . . . . . Cursor Sensitivity . . . . . . . . . . . . . . . . . . . . . Demonstration 1: Update to a Row Already Fetched. Demonstration 2: Update to a Row Not Yet Fetched. Cursor Sensitivity — An Exception . . . . . . . . . Locking Considerations with Cursors . . . . . . . . Impact on tempdb Usage . . . . . . . . . . . . . . . Worktable Materialization with Scrollable Sensitive Cursors. . . . . . . . . . . . . . . . . . . . . . . Conclusion of Sensitive vs. Insensitive Cursors . . . Sybase Engineer’s Insight . . . . . . . . . . . . . . . . . Summary. . . . . . . . . . . . . . . . . . . . . . . . . . Future Direction. . . . . . . . . . . . . . . . . . . .

. . . . . . 145 . . . . . 145 . . . . . 146 . . . . . 146 . . . . . 148 . . . . . 150 . . . . . 151 . . . . . 152 . . . . . 154 . . . . . 156 . . . . . 157 . . . . . 158 . . . . . 159

Overview of Changes to the Query Processing Engine. . . Introduction . . . . . . . . . . . . . . . . . . . . . Optimization Goals . . . . . . . . . . . . . . . . . allrows_oltp . . . . . . . . . . . . . . . . . . . allrows_mix . . . . . . . . . . . . . . . . . . . allrows_dss . . . . . . . . . . . . . . . . . . . Determining the Current Optimization Goal . . Optimization Criteria . . . . . . . . . . . . . . . . merge_join . . . . . . . . . . . . . . . . . . . merge_union_all . . . . . . . . . . . . . . . . merge_union_distinct . . . . . . . . . . . . . . multi_table_store_ind . . . . . . . . . . . . . . opportunistic_distinct_view . . . . . . . . . . parallel_query . . . . . . . . . . . . . . . . . . hash_join . . . . . . . . . . . . . . . . . . . . Optimization Timeout Limit. . . . . . . . . . . . . Query Processor Improvements . . . . . . . . . . . Datatype Mismatch . . . . . . . . . . . . . . . Partition Elimination and Directed Joins . . . . Tables with Highly Skewed Histogram Values. Group By and Order By . . . . . . . . . . . .

. . . . . . 167 . . . . . 167 . . . . . 168 . . . . . 168 . . . . . 169 . . . . . 169 . . . . . 170 . . . . . 170 . . . . . 171 . . . . . 171 . . . . . 171 . . . . . 171 . . . . . 171 . . . . . 171 . . . . . 172 . . . . . 172 . . . . . 174 . . . . . 175 . . . . . 177 . . . . . 179 . . . . . 181

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

160 163 164 164 165

Contents

or Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Star Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Chapter 6

Detection and Resolution of Query Performance Issues . . . . . . . . . . . 187 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 An Approach to Poor Query Performance Diagnosis . . . . . . . . 188 Common Query Performance Factors . . . . . . . . . . . . . . . . 190 Eliminating Causes for Sub-Optimal Plan Selection . . . . . . 191 Find Missing or Invalid Statistics . . . . . . . . . . . . . 191 Consider Range Cell Density on Non-Unique Indexes . . 191 Identify Index Needs . . . . . . . . . . . . . . . . . . . . 192 Identify Poor Index Strategy . . . . . . . . . . . . . . . . 192 Fragmentation of Data . . . . . . . . . . . . . . . . . . . 192 Resolve Partition Imbalance . . . . . . . . . . . . . . . . 193 Reset Server- or Session-level Options . . . . . . . . . . 193 Overengineered Forceplan . . . . . . . . . . . . . . . . . 194 Invalid Use of Index Force. . . . . . . . . . . . . . . . . 194 Inefficient Query Plan Forced by Abstract Plan . . . . . . 195 Query Processor “set options” — The Basics . . . . . . . . . . . . 195 Query Optimizer Cost Algorithm. . . . . . . . . . . . . . . . 198 ASE 15 vs. 12.5.x Cost Algorithm . . . . . . . . . . . . . . . 199 Query Processor “set options” — Explored . . . . . . . . . . . . . 200 show_missing_stats . . . . . . . . . . . . . . . . . . . . . . . 200 show_elimination . . . . . . . . . . . . . . . . . . . . . . . . 203 show_abstract_plan . . . . . . . . . . . . . . . . . . . . . . . 204 Why Use Abstract Plans for ASE 15? . . . . . . . . . . . . . 207 Application of Optimization Tools . . . . . . . . . . . . . . . . . 208 Optimization Goal Performance Analysis . . . . . . . . . . . 208 Optimization Criteria Performance Analysis . . . . . . . . . . 210 Optimization Timeout Analysis . . . . . . . . . . . . . . . . 212 Suggested Approach to Fix Optimization Timeout Problems . . . . . . . . . . . . . . . . . . . 216 Detection, Resolution, and Prevention of Partition-related Performance Issues . . . . . . . . . . . . . . . . . . . . . . . . 217 Data Skew Due to Incorrect Partition Type or Poor Partition Key Selection . . . . . . . . . . . . . . . . 218 Effect of Invalid Statistics on Table Semantically Partitioned . . . . . . . . . . . . . . . . . . . . . . . . . . 220 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

Chapter 7

Computed Columns . . . . . . . . Introduction . . . . . . . . . Key Concepts . . . . . . . . Materialization . . . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . 225 . . . . . 225 . . . . . 226 . . . . . 226 xiii

Contents

Deterministic Property . . . . . . . . . . . . . . . . . . Relationship between Deterministic Property and Materialization. . . . . . . . . . . . . . . . . . . . . Deterministic and Materialized Computed Columns Deterministic and Nonmaterialized Computed Columns. . . . . . . . . . . . . . . . . . . . . . Nondeterministic and Materialized Computed Columns. . . . . . . . . . . . . . . . . . . . . . Nondeterministic and Nonmaterialized Computed Columns. . . . . . . . . . . . . . . . . . . . . . Benefits of Using Computed Columns . . . . . . . . . . . . Provide Shorthand and Indexing for an Expression . . . Composing and Decomposing Datatypes. . . . . . . . . User-defined Sort Order . . . . . . . . . . . . . . . . . Rules and Properties of Computed Columns . . . . . . . . . Sybase Enhancements to Support Computed Columns . . . . Create Table Syntax Change . . . . . . . . . . . . . . . Alter Table Syntax Change . . . . . . . . . . . . . . . . System Table Changes . . . . . . . . . . . . . . . . . . Stored Procedure Changes . . . . . . . . . . . . . . . . Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 8

xiv

. . . 228 . . . 229 . . . 229 . . . 229 . . . 230 . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

230 231 231 231 232 233 235 235 235 236 237 239

Functional Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Computed Column Index . . . . . . . . . . . . . . . . . . . . . . 242 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Rules and Properties of a Computed Column Index . . . . . . 246 Feature Benefits. . . . . . . . . . . . . . . . . . . . . . . . . 246 Feature Limitations . . . . . . . . . . . . . . . . . . . . . . . 248 Impacts to tempdb . . . . . . . . . . . . . . . . . . . . . . . 248 Impact to Existing Application Code . . . . . . . . . . . . . . 249 Determining When to Use a Computed Column Index. . . . . 250 Optimizer Statistics . . . . . . . . . . . . . . . . . . . . . . . 251 Function-based Index . . . . . . . . . . . . . . . . . . . . . . . . 251 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Rules and Properties of a Function-based Index . . . . . . . . 252 Feature Benefits. . . . . . . . . . . . . . . . . . . . . . . . . 253 Feature Limitations . . . . . . . . . . . . . . . . . . . . . . . 256 Impacts to tempdb . . . . . . . . . . . . . . . . . . . . . . . 256 Impact to Existing Application Code . . . . . . . . . . . . . . 257 Determining the Use of a Function-based Index . . . . . . . . 257 Optimizer Statistics . . . . . . . . . . . . . . . . . . . . . . . 258 Behind the Scenes . . . . . . . . . . . . . . . . . . . . . . . . . . 258 Getting Index Information . . . . . . . . . . . . . . . . . . . . . . 258 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

Contents

Chapter 9

Capturing Query Processing Metrics . . . . . . . . . . . . . . . . . . . . . 261 Alternatives to Query Processing Metrics . . . . . . . . . . . . . . 261 Introduction to Query Processing Metrics. . . . . . . . . . . . . . 262 Contents of sysquerymetrics. . . . . . . . . . . . . . . . . . . . . 263 Contents of the sysquerymetrics View . . . . . . . . . . . . . 264 How to Enable QP Metrics Capture . . . . . . . . . . . . . . . . . 265 Captured Information Explored . . . . . . . . . . . . . . . . . . . 266 Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . 266 Triggers and Views . . . . . . . . . . . . . . . . . . . . . . . 270 Execute Immediate . . . . . . . . . . . . . . . . . . . . . . . 270 Accessing Captured Plans . . . . . . . . . . . . . . . . . . . . . . 271 How Is the QP Metrics Information Useful? . . . . . . . . . . . . 273 Identification of Performance Regression . . . . . . . . . . . . . . 276 Comparing Metrics for a Specific Query between Running Groups . . . . . . . . . . . . . . . . . . . . . . . . . 277 Comparing Metrics for All Queries between Running Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Why Separate the QP Metrics Data by gid? . . . . . . . . . . 280 Syntax Style Matters; Spacing Does Not . . . . . . . . . . . . . . 281 Clearing and Saving the Metrics. . . . . . . . . . . . . . . . . . . 283 Relationship between Stats I/O and QP Metrics I/O Counts . . . . 284 Information for Resource Governor . . . . . . . . . . . . . . . . . 285 Space Utilization Considerations . . . . . . . . . . . . . . . . . . 285 Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

Chapter 10

Graphical Plan Viewer . . . . . . . . . . . . . . . . . . Graphical Plan Viewer from Interactive SQL . . Graphical Query Tree Using Set Options . . . . Summary. . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . 287 . . . . . 287 . . . . . 294 . . . . . 296

Chapter 11

Sybase Software Asset Management (SySAM) 2.0. . . Introduction . . . . . . . . . . . . . . . . . . . Prior to ASE 15 . . . . . . . . . . . . . . . . . With ASE 15. . . . . . . . . . . . . . . . . . . Components of Asset Management . . . . . . . SySAM Server . . . . . . . . . . . . . . . SySAM Utility Program — lmutil . . . . . SySAM Reporting Tool. . . . . . . . . . . System Environment Variables . . . . . . . License File . . . . . . . . . . . . . . . . . Options File . . . . . . . . . . . . . . . . . Properties File. . . . . . . . . . . . . . . . The SySAM Environment . . . . . . . . . Standalone License Server . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . 297 . . . . . 297 . . . . . 298 . . . . . 299 . . . . . 299 . . . . . 299 . . . . . 300 . . . . . 300 . . . . . 301 . . . . . 302 . . . . . 303 . . . . . 303 . . . . . 304 . . . . . 304 xv

Contents

Networked License Server . Redundant License Server . Acquiring Product Licenses . . . . . Product Licenses. . . . . . . . . . . Try and Buy. . . . . . . . . . . License Activation . . . . . . . SySAM Administration . . . . . . . sp_lmconfig . . . . . . . . . . . ASE 15 SySAM Upgrade Process. . SySAM Reporting . . . . . . . . . . Summary Reports . . . . . . . . Server Usage Reports . . . . . . Raw Data Reports. . . . . . . . Summary. . . . . . . . . . . . . . . Chapter 12

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

305 305 306 309 309 309 310 310 312 313 313 319 320 321

Installation of ASE Servers . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Prior to Installation for All Methods . . . . . . . . . . . . . . . . 324 Installation with Resource Files . . . . . . . . . . . . . . . . . . . 325 Notes for Resource File Installation of ASE . . . . . . . . . . 325 Installation of ASE Components with a Resource File . . . . . 330 GUI Installation Method with srvbuild Executable . . . . . . . . . 332 Installation with the Dataserver Executable . . . . . . . . . . . . . 352 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

Part II: Pre-15 Improvements Chapter 13

xvi

Multiple Temporary Databases . . . . . . . . . . . . . . . . . . . . . . . . 359 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 Purposes for Multiple Temporary Databases . . . . . . . . . . . . 360 Prior to ASE 15 . . . . . . . . . . . . . . . . . . . . . . . . . 360 With ASE 15 . . . . . . . . . . . . . . . . . . . . . . . . . . 361 System Catalog Changes. . . . . . . . . . . . . . . . . . 361 directio Support . . . . . . . . . . . . . . . . . . . . . . 361 update statistics . . . . . . . . . . . . . . . . . . . . . . 363 Insensitive Scrollable Cursors . . . . . . . . . . . . . . . 364 Semi-sensitive Scrollable Cursors . . . . . . . . . . . . . 364 Sensitive Scrollable Cursors . . . . . . . . . . . . . . . . 364 How to Decide When to Add a Temporary Database . . . . . . . . 365 Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 What Are Your Needs? . . . . . . . . . . . . . . . . . . . . . 366 Implementation Steps . . . . . . . . . . . . . . . . . . . . . . . . 367 Determining Available Temporary Databases. . . . . . . . . . . . 368 Sample Setup for Temporary Database for “sa” Use Only . . . . . 369 Other Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

Contents

Dropping Temporary Databases Altering a Temporary Database. @@tempdb . . . . . . . . . . . Summary. . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

371 372 372 372

Chapter 14

The MDA Tables . . . . . . . . . . . . . . . . . . . . . What Are the MDA Tables? . . . . . . . . . . . Past Solutions . . . . . . . . . . . . . . . . . . MDA Table Installation . . . . . . . . . . . . . MDA Table Server Configuration Options . . . The Parent Switch. . . . . . . . . . . . . . The MDA Tables . . . . . . . . . . . . . . . . Changes from ASE 12.5.3 . . . . . . . . . . . . What Is Meant by “stateful” Tables? . . . . . . Stateful MDA Table Data Management . . . . . SQL Use . . . . . . . . . . . . . . . . . . . . . Useful MDA Table Queries . . . . . . . . . . . MDA Alternatives . . . . . . . . . . . . . . . . Summary. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . 373 . . . . . 373 . . . . . 374 . . . . . 376 . . . . . 377 . . . . . 379 . . . . . 380 . . . . . 382 . . . . . 383 . . . . . 385 . . . . . 391 . . . . . 391 . . . . . 393 . . . . . 393

Chapter 15

Java, XML, and Web Services in ASE. . . . . . . . . . . . . . . . . . . . . 395 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 Java in the Database . . . . . . . . . . . . . . . . . . . . . . . . . 396 Installing Java Classes . . . . . . . . . . . . . . . . . . . . . 397 Creating Java Classes and JARs . . . . . . . . . . . . . . 397 Using the installjava Utility . . . . . . . . . . . . . . . . 398 Configuring Memory for Java in the Database . . . . . . 398 Java Classes as Datatypes. . . . . . . . . . . . . . . . . . . . 399 An Example of Table Definition Using a Java Class . . . 400 Performance Considerations . . . . . . . . . . . . . . . . 400 An Example of Using a Java Class within a Select . . . . 400 Executing Java Methods . . . . . . . . . . . . . . . . . . . . 400 Class Static Variables . . . . . . . . . . . . . . . . . . . 401 Recommendations and Considerations . . . . . . . . . . . . . 401 XML in the Database . . . . . . . . . . . . . . . . . . . . . . . . 402 XML Stored in the Database . . . . . . . . . . . . . . . . . . 402 Option 1: Store the XML Document into a Text Datatype . . . . . . . . . . . . . . . . . . . . . . 403 Option 2: Store the XML Document into an Image Datatype Using xmlparse . . . . . . . . . . . . 404 Option 3: Store the XML Document into an Image Datatype Using Compression . . . . . . . . . . 404 Option 4: Store the XML Document Outside the Database. . . . . . . . . . . . . . . . . . . . . . . 405 HTML Stored in the Database . . . . . . . . . . . . . . . 406 xvii

Contents

Recommendations and Considerations . . . . . . . . Performance and Sizing . . . . . . . . . . . . . . . . SQL Result Sets Converted to Return an XML Document. Web Services . . . . . . . . . . . . . . . . . . . . . . . . . . Web Services Producer . . . . . . . . . . . . . . . . . . . Web Services Consumer . . . . . . . . . . . . . . . . . . Recommendations and Considerations . . . . . . . .

. . . . . . .

. . . . . . .

406 407 410 411 411 413 415

Appendix A

Sybase ASE 15 Certification Sample Questions and Answers . . . . . . . 417

Appendix B

Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

xviii

Foreword Sybase Adaptive Server Enterprise (ASE) has been a leading RDBMS for mission-critical applications for almost two decades. With ASE 15, Sybase has continued the tradition of introducing leading edge database technologies to address our customers’ needs. These technologies cover security, unstructured data management, and operational intelligence. ASE 15 provides significant improvements for DSS and OLTP applications thanks to innovative query processing technology. The result is a clear understanding of the optimal usage of the features and functionality. The Official New Features Guide to Sybase ASE 15 is the first book describing ASE 15. Of particular interest are details about new features such as semantic partitions, computed columns, function indexes, and scrollable cursors. The book is valuable also for its guidance for diagnosing, resolving, and optimizing the overall system performance. The authors collectively have more than 40 years of experience with Sybase ASE as DBAs at some of Sybase’s largest and most demanding customers. Their background has enabled them to create a book with great practical value. The authors’ material is presented in a way that is useful to readers independent of their experience level with ASE. The Official New Features Guide to Sybase ASE 15 will be an invaluable asset to DBAs, developers, and consultants working with Sybase ASE.

Dr. Raj Nathan Senior Vice President Information Technology Solutions Group Sybase, Inc.

xix

This page intentionally left blank.

Acknowledgments The authors would like to thank Celeste Noren for starting this whole process. Your continual support and upbeat attitude have allowed us to stay focused on the job at hand. We thank Joe Pearl for his tireless and thorough editing and brilliant input. Special thanks to Irfan Khan at Sybase Engineering for providing us with early access to the new features and documentation for ASE 15. We need to thank Rob Verschoor, Mark Kusma, and Jeff Tallman of Sybase for their thorough technical reviews and timely feedback. We thank Joan Fronske and Tom Traubitz from Sybase Marketing for making all the arrangements with Sybase. We would like to thank Tim McEvoy at Wordware Publishing for getting this book put together in such a timely manner. A special thanks goes to Dean Kent for his legal counsel during the process and for all the good lunches. Mostly, we would like to thank our families for being understanding and gracious.

xxi

Acknowledgments

I would like to start by recognizing the talented and dedicated group of authors — Steve Bradley, Naresh Adurty, and Carrie King Taylor — who accepted my challenge to create this book about the new features for Sybase ASE 15. With all of the personal challenges and commitments we each faced during the development of this book, I’m even more pleased with the final product. As our careers diverge in the future, I’m sure each of us will fondly recall the many challenges, good times, and late nights spent on the development of this book. Next, I want to personally thank the many Sybase employees who helped with this book in many capacities: Celeste Noren, Joe Pearl, Rob Verschoor, Irfan Khan, Jeff Tallman, Mark Kusma, Joan Fronske, and Tom Traubitz. I also need to thank the many friends and family members in my life: my parents, Art and Nancy, my brother, Grant, Aunt Soozie, Frank, Carol, and my grandfather Robert Taylor, who left this world during the writing of this book. I also want to thank some of my friends who have made life very interesting and with whom I have shared many of life’s milestones: Steve, Daryl, Dave, Chris, and Mike. I owe the biggest thanks to my family, especially my wife and co-author, Carrie. I appreciate how you are able to keep me motivated, and still find time for our boys and your contributions to this book. You are the reason and motivation for my success. I sure love you! I also owe many great big hugs and kisses to my two big boys, Elliot and Jonah, for giving up their daddy for many weekends and nights. Each of you did your best to remind me of how important it is to spend quality time with family, despite the many demands of authorship. — BRT First, I would like to thank Brian for considering me talented enough to be a contributing author and my poor judgment that made me agree to his request. When Brian approached me with this idea, I had just been married for a month to my wonderful and naïve wife, Karen, and was about to start my Executive MBA from Duke, for which I had to fly from Florida to Raleigh every other weekend. My mind envisioned mansions and big boats following the success of this xxii

Acknowledgments

book. After talking to the publisher, I realized that we will not be anywhere close to buying a boat. We will be lucky to be able to buy a life jacket with the new riches. By that time, it was too late to say no. I would like to thank Karen for supporting me, at least outwardly, throughout this madness; my parents, for giving me the freedom and courage to make insane decisions and for constantly reminding me that I am the cutest kid in the world and thus stroking my fragile ego; and my failed stand-up comedy career, which forced me to continue my career as a Sybase DBA for the past decade. — NA First, I would like to thank Brian for taking on this initiative and considering me to be an author in developing this work. Second, I would like to thank the other authors for the time and patience that each of you have shown during this process. As difficult as it is for one person to produce such a work, four people can make it even more strenuous. Thank you, guys, for keeping the process lighthearted and fun. It was a pleasure working with you. I would like to thank my parents, Charlie and Betty, for instilling in me the desire and pride to do the best I can in whatever I attempt. And most importantly, I would like to thank my best friend for all of her support during this time — my wife and my love, Carol. — SWB I would like to recognize my co-authors Brian, Naresh, and Steve. You guys rock. Special thanks go to Carol and Karen for sacrificing your family time so we could complete this book. I’d like to acknowledge my dad (posthumously), my mom, and my siblings (“The Chills”…eyes loaf) for making me the person I am today. Without them, I might have turned out ordinary. A special thanks to my friends Linda, Katrina, Colleen, Gina, Heather, Chris, Daryl, Terri, Allie, and September for providing so much fun and inspiration during this process. Most importantly, I’d like to thank my husband, Brian, and wonderful sons, Elliot and Jonah. You are the lights of my life. I love you more, more, more, more, more! — CKT

xxiii

This page intentionally left blank.

About the Authors Brian Taylor is a Sybase Certified Professional DBA with over 11 years of experience in the Information Technology (IT) industry. Brian has been a presenter at the Sybase TechWave conference. He has also presented an ASE 15 case study through a live web seminar, as well as delivered web seminars based on his beta testing efforts with ASE 15. He is a contributing author to Wordware’s Administrator’s Guide to Sybase ASE 12.5. Brian has a BS in management information systems and finance from Florida State University and is currently pursuing his MBA from Florida State University. Naresh Adurty has been working with Sybase for the past 12 years. He has an MS in mechanical engineering from Oklahoma State University and is currently pursuing his MBA from Duke’s Fuqua School of Business. He is also a professional stand-up comedian with radio and TV credits including HBO.

Steve Bradley is a Brainbench Certified Sybase DBA with over 25 years of experience in the IT field in the areas of data center operations, application development, MVS administration, database administration, and vendor technical sales support. Steve has worked with Sybase products for over 12 years and has been a presenter at the 2004 and 2005 Sybase TechWave conferences. Steve has a BS in computer science from the University of South Carolina and an MBA from University of Phoenix, Tampa.

xxv

About the Authors

Carrie King Taylor has been working with Sybase as a DBA for 8 years with over 15 years of experience in the IT industry. Carrie has a diverse IT background as she has served as a Unix systems administrator, software quality analyst, and software developer, and has worked in software configuration and implementation. She has a BS in business information systems from Indiana Wesleyan University.

About the Contributing Authors Mark A. Shelton is an enterprise architect with Nielsen Media Research. He has more than 25 years of experience in the IT industry and has been working with Sybase technologies since 1993. His work positions have included a DBA, developer, business analyst, project manager, and architect. Mark holds a BS in computer science and a BS in electrical engineering from the University of Dayton, and an MBA from the University of South Florida. Jagan Reddy is a certified professional senior DBA at America Online, Inc. (AOL). He has more than 15 years of database administration experience, and has been a presenter at the Sybase TechWave conference. He started using Sybase technologies in 1989 as a Sybase DBA. He has spent most of his career as a DBA, project lead, and system administrator. He holds a master’s degree in plant genetics and plant breeding from Andhra Pradesh Agricultural University in India, and a master’s degree in computer science from Bowie State University in Maryland.

xxvi

Introduction We’ve read the newsgroups. We’ve seen the blogs. We know you’re out there. The talented Sybase database administrators, developers, managers, and users are out there. This book is for you. We will attempt to guide you, the reader, through the enhancements of ASE 15. In addition to the enhancements, we will also discuss how these enhancements can be applied to a “real-life” situation. We have polled many fellow database administrators about their thoughts on Sybase ASE regarding the new features. We have tried to incorporate as many of these ideas as possible.

Audience The audience for this book is wider than just database administrators. This book can be used by managers to foresee possible project direction for more than just the next release of a product or application. It can assist the system architect in determining the best solution for enterprise applications. It can be used to educate the developer about features that are now available that will make the development process easier and more manageable. But most importantly, this book is for the database administrator. This book will guide you through the practical use of the new features in Sybase ASE 15.

Scope This book is an overview of the new features available in ASE 15. We also touch on some existing features that were implemented since the last series of administration guides was released. These preexisting features were deemed worthy of reference based on the overall ASE road map for future releases and the direction of Sybase toward a flexible product geared toward service-oriented architecture (SOA). We will demonstrate and clarify the new features and how they integrate into existing systems.

xxvii

Introduction

Conventions This book follows certain typographic conventions, as outlined below: Arial font is used for commands, options, utility names, and

other keywords within the text Italics

is used to show generic variables and options; these should be replaced when actually writing code is used to show syntax, the contents of files, and the output from commands

Constant width

%, $

are used in some examples as the operating system prompt

[]

surround optional elements in a description of syntax (the brackets themselves should never be typed)

{}

indicates you must choose at least one of the enclosed options

()

are part of the command

|

is used in syntax descriptions to separate items for which only one alternative may be chosen at a time

,

indicates as many of the options shown can be selected but must be separated by commas.

…

indicates the previous option is repeatable

An example of these conventions is: update statistics table_name [[partition data_partition_name] [(column_list)] | index_name [partition index_partition_name]] [using step values] [with consumers = consumers] [,sampling=N percent]

xxviii

Introduction

A final word about syntax: No capitalization of objects, databases, or logins/user were used where syntax would be incorrect and would cause errors. Sentences starting with keywords may start with lowercase letters. Code and output are presented separately from the regular paragraph.

Disclaimer Some of this material is based upon a prereleased version of ASE 15. Some of the details, syntax, and output of some of the commands and procedures may differ slightly from the General Availability (GA) release of ASE 15. While the authors have attempted to ensure the screen shots, syntax, and output reflect the GA release of ASE 15, it is possible the book may contain some information from the prereleased version of ASE 15, which may or may not be materially different from the GA release of ASE 15.

xxix

This page intentionally left blank.

Part I

New Features Overview

1

This page intentionally left blank.

Chapter 1: Exploring the Sybase Galaxy

|

3

Chapter 1

Exploring the Sybase Galaxy The times, they are a-changin’. This observation applies to relational database management systems as well. Historically, databases have revolved around structured data and how it is managed. The database administrator must now look beyond managing structured data and must now consider managing information. Sybase rightly code-named the latest release of Sybase Adaptive Server Enterprise (ASE) “Galaxy.” We are moving beyond the world of data management as we know it and rocketing into the new galaxy of information management. What is the difference between data and information? Data is simply the atomic component of information, whereas information is taking the data and making it mean something. As we begin to open our silos and release the data within, that data can then become useful information to everyone in our individual galaxies. What does this mean to the database administrator? It means supporting more complex queries for greater flexibility and operational scalability. It also means looking beyond our previous ideals of data storage and broadening our concepts to the possibilities of larger amounts of data and faster access. Let’s take a look at the new features available in ASE 15 as we begin to explore the galaxy of information management. Additionally, our journey will cover a few of the recent pre-ASE 15 enhancements and explore how they enhance and expand the ASE product.

4

|

Chapter 1: Exploring the Sybase Galaxy

Part I — New Features Overview Unlike previous releases, the release of ASE 15 incorporates sweeping changes to the ASE product. Many of these changes are designed to increase the database server’s ability to handle a high volume and wide variety of data while responding with greater velocity. Today’s database servers must have the ability to handle an increasing load of information. Database administrators are given the responsibility of balancing the responsiveness of systems and the availability of data. ASE 15 concurrently assists the database administrator with both of these often conflicting objectives. ASE 15 arrives with internal enhancements to keep data moving at the highest possible velocity while minimizing the server’s need for performance tweaks, thus lowering the total cost of ownership (TCO). Features such as automatically logged execution metrics assist the database administrator in lowering the cost of development while increasing the ability to address more complex data. This chapter gives a brief overview of all of the new features of ASE 15.

System Maintenance Improvements To achieve the goals of simplifying the maintenance of ASE as well as enhancing the performance of query processing, several system maintenance enhancements are introduced in the ASE 15 release. Included in these changes is the ability to automatically update statistics using the datachange function in conjunction with Job Scheduler, the expansion of row-level locking to system tables, and the introduction of partition-based utilities and maintenance tools. The system administration enhancements in ASE 15 are designed to lower the maintenance requirements of ASE, enabling the database administrator to target maintenance efforts on areas of ASE that truly need attention. Several of the changes allow the database administrator to reduce or altogether close the regularly scheduled maintenance windows where full system maintenance is performed. Leading up to the ASE 15 release, many new features were added to the ASE 12.5.x versions of ASE that work hand in hand with the enhancements included in ASE 15. In Chapter 2, we discuss the following pre-ASE 15 topics: multiple tempdb, automatic database expansion, and the Job Scheduler, and how these features work in conjunction with enhancements in ASE 15 to lower TCO.

Chapter 1: Exploring the Sybase Galaxy

|

5

Partition Management — Semantic Partitions Sybase ASE semantic partitions address two main issues with today’s storage systems: data velocity and data explosion. Today, systems demand access to more data, and with increasing velocity. These goals are often contradictory. The introduction of Sybase ASE 15 semantic partitions simultaneously assists with each of these goals through the physical separation of data into partitions. The storage of current and historic data in the same database adds to the data management complexities already inherent in most environments. Semantic partitions can decrease the difficulty of managing these complexities. Semantic partitions provide a mechanism for allowing access to this data through the same interface, with no extraneous processing modifications to retrieve archived or historical data. Semantic partitions also introduce partition elimination. With semantic partitions, the optimizer is able to choose which partitions are necessary to resolve the query and therefore eliminate unnecessary data access. The concept of data partitions is not new to Sybase ASE; however, the concept of semantic partitions is new. Semantic partitions allow the database administrator to maintain control over the placement of partitioned data and indexes within the database. This ability was not present in prior releases of Sybase ASE. Chapter 3 elaborates on the advantages provided by semantic partitions. Additionally, the chapter presents an approach to best match the appropriate semantic partition type with a system’s data and usage requirements. The chapter specifies the differences between partition types as well as the new terminology introduced in support of this new functionality.

Scrollable Cursors Scrollable cursors are in line with Sybase’s goal to increasingly lower the TCO of Sybase ASE. Scrollable cursors can also ease or speed the development process by replacing supplemental middleware or client tools used to manipulate a cached result set. Chapter 4 describes how to create a scrollable cursor, explains how to manipulate a scrollable cursor, and discusses the global variables associated with a scrollable cursor.

6

|

Chapter 1: Exploring the Sybase Galaxy

Additionally, Sybase ASE 15 introduces cursor sensitivity. Cursor sensitivity relates to the ability of ASE to allow changes to data underlying a cursor to be represented in the cursor’s result set. Cursor sensitivity also allows for a cursor performance benefit, which is explained within this chapter.

Overview of Changes to Query Processing In ASE 15, the Query Optimizer has undergone numerous internal changes. These changes are intended to accomplish the following objectives: t Improve query performance t Limit the need for manual intervention to achieve optimal query plans t Ease the monitoring and diagnostic efforts needed to maintain query performance Chapter 5 discusses the “out-of-the-box” improvements to the ASE 15 Query Optimizer. These Query Optimizer changes will lower TCO by requiring less tuning. By simply running existing queries through the new Query Optimizer, the performance can be improved. Additionally, new optimization goals are explored in conjunction with the optimization time limitations. These time limitations will provide the database administrator the ability to specify the percentage of time the Query Optimizer is allowed to spend on query optimization.

Detection and Resolution of Performance Issues in Queries An ASE 15 goal is to ease the monitoring and diagnostic efforts required by the database administrator to scrutinize query performance. As discussed in Chapter 5, ASE 15’s Query Optimizer, or optimizer, contains “out-of-the-box” improvements designed to result in little or no manual intervention for the query processor to arrive at the most efficient query plan. To accomplish these goals in ASE 15, the Query Optimizer and optimizer toolset have undergone numerous changes to accommodate the demand for simplified detection and resolution of problems. A

Chapter 1: Exploring the Sybase Galaxy

|

7

new set of optimizer diagnostic tools for the database administrator is included in this release of ASE. Chapter 6 discusses how to enable these new diagnostics. This chapter also defines the diagnostics and offers insight into how the database administrator can use the new diagnostics to detect query-level problems. A specific problem resolution slant involving diagnostic examples with use of the new ASE 15 features, most specifically semantic partitions, is represented through example. It is also important to note that the name of the optimizer is now often referred to as the “query processor” and equally as often as the “Query Optimizer.” This is a departure from the normal reference to the “Optimizer” as represented in prior versions of ASE.

Computed Columns In accordance with the Sybase road map to provide operational excellence within the database, ASE 15 introduces computed columns. The implementation of computed columns affords the database administrator the opportunity to ease development costs and provide a foundation to offer increased data velocity. The utilization of computed columns can reduce TCO by simplifying the application development effort. The simplification is accomplished by the placement of the computational burden on the server as opposed to within the client layer. This enhancement is especially applicable to the handling of unstructured data, such as XML documents. Chapter 7 provides the technical details on the implementation of computed columns. The chapter outlines the syntax to implement computed columns and includes several examples. Materialization and the deterministic properties of computed columns are also explained.

Functional Indexes Sybase offers computed column indexes and function-based indexes to maximize the scalability and performance capabilities of Sybase ASE. These new index types allow Sybase ASE to maintain indexes based on derived data. This enhancement further emphasizes the data velocity initiatives incorporated into the release of ASE 15.

8

|

Chapter 1: Exploring the Sybase Galaxy

Function-based indexes provide the database administrator an additional tool to improve the performance of application logic while reducing TCO. As a DBA, when faced with optimization scenarios for queries issued from the application, previous solutions were to modify the application code or upgrade the underlying hardware. ASE 15 provides the ability to optimize application code with the addition of function-based indexes to the database. These performance-enhancing indexes can be applied without the impact of application maintenance or hardware upgrades. The underlying benefit of function-based indexes and computed column indexes is the reduction of physical and logical I/O necessary to satisfy queries. A reduction in physical and logical I/O allows ASE to maintain performance as the storage requirements of databases increase. This provides greater flexibility to the database administrator for managing data explosion. Chapter 8 provides details on how to implement functional indexes. The chapter provides guidelines on when to implement these new features as well as provides an analysis of the impact and limitations.

Capturing Query Processing Metrics As database administrators, we are often asked to scrutinize the performance of queries. As an initial diagnostic step, many database administrators examine the statistics I/O or statistics time output associated with a query. While the statistics I/O and time output provide initial information on a track to take to diagnose query performance, basic information is often lacking (for example, baseline performance for the specific query). The Query Processing Metrics capture process provides a mechanism to maintain a query baseline with minimal setup or maintenance effort. In Chapter 9 we demonstrate the setup and value of the Query Processing Metrics capture process as well as specifically identify what and where information is maintained by the metrics capture process. This feature is in line with the Sybase goal to increasingly lower the TCO of Sybase ASE. The Query Processing Metrics can supplement or replace third-party tools designed to capture the SQL executed against ASE installations.

Chapter 1: Exploring the Sybase Galaxy

|

9

Plan Viewer The introduction of the Plan Viewer in ASE 15 is aligned with Sybase’s goal to ease the overall management needs of ASE. The Plan Viewer is a graphical user interface (GUI) utility wrapped in the interactive ISQL tool included with the ASE 15 open client product. This new feature provides database administrators and developers a tool to help visualize query plans for the purpose of quick and easy identification of performance issues. Chapter 10 provides instructions on how to set up and launch the Plan Viewer. The chapter continues with a demonstration on how to extract useful information from the Plan Viewer, and discusses the information and concepts contained in the Plan Viewer’s output.

Sybase Software Asset Management (SySAM) 2.0 In conjunction with the release of ASE 15, Sybase provides a new version of the Sybase Software Asset Management (SySAM). SySAM 2.0 provides a mechanism for Sybase to ensure customer compliance with license agreements. For most customers, the question is not “Should we monitor for compliance?” but “How do we monitor for compliance?” SySAM 2.0 provides the capability of monitoring and reporting license compliance, which is required for ASE 15. In Chapter 11, SySAM 2.0 is explored in detail. The components of the Sybase Software Asset Management are explained, as well as the various setup options, evaluation licenses, and grace periods.

Installation of ASE 15 The focus in Chapter 12 is the installation of ASE 15. This chapter provides an overview of the server installation process, using three distinctly different installation methods. With each method, server installation is covered in an easy-to-follow step-by-step manner.

10

|

Chapter 1: Exploring the Sybase Galaxy

Part II — Pre-15 Improvements Why include pre-15 features in a book about ASE 15 new features? As database administrators, the authors found very little information about these features outside of the Sybase-provided manuals and web-based seminars. The pre-15 features that have been included in this book are flexible, fascinating, and handy tools. Not only are these features useful, but they set the groundwork for future expansion toward self-management and scalability in managing data explosion and very large database (VLDB) growth.

Multiple tempdb Databases The use of multiple tempdb databases was first made available in the 12.5.0.3 release of Sybase ASE. Until now, little has been published on the use of multiple temporary databases. Chapter 13 offers insight to organizations debating the use of multiple tempdb databases and explains the steps that are needed for implementation. Once the decision to implement multiple tempdb databases is made, follow this chapter’s step-by-step example on how to implement the feature. Additionally, the chapter discusses the impact of multiple tempdb databases on several of the ASE 15 new features.

MDA Tables MDA tables are another ASE feature first available in release 12.5.0.3. While there are a few known publications on this feature, we decided to incorporate the MDA tables into this 15 book to provide another resource on MDA table application. In Chapter 14, the new MDA table and column additions are identified. The chapter also walks through the basics of MDA table setup, identifies the information collected by the MDA tables, suggests how to manage the “statefulness” of MDA tables, and discusses the impact of changing some of the MDA table’s configuration parameters. The MDA tables are another ASE feature in line with the Sybase goal to increasingly lower the TCO of Sybase ASE. The MDA tables

Chapter 1: Exploring the Sybase Galaxy

|

11

can supplement or replace tools designed to monitor ASE at the server, database, and session levels.

Java and XML Java and XML capabilities within the database are not new to ASE. For several releases of ASE, Java and XML utilities have been present. Chapter 15 provides an overview of the implementation of Java and XML in the database server as well as the impact of XML and Java on the database. Additionally, an explanation of how to install and enable Java in the database is presented. The chapter concludes with the authors’ recommendations for appropriate use and implementation of these features.

The Appendices Sample Certification Exam Appendix A contains an 80-question sample Sybase ASE 15 certification practice exam with answers. The intent of this appendix is to enhance the learning experience of the Sybase ASE 15 material presented in this book. Additionally, the sample exam provides supplemental preparation for DBA certification for the Sybase ASE 15 product. Approximately 60% of the questions are based purely on ASE 15 material, while 40% of the questions are based on common Sybase ASE knowledge that would apply to ASE 15 and earlier versions. The general Sybase ASE material is based on actual test questions from previous tests.

Use Cases All ASE 15 material covered in this book is put to practical use through the business cases presented in Appendix B. Through practical examples, the use cases demonstrate how Sybase ASE 15 can assist businesses with lowering TCO. Additionally, the examples highlight the many advantages provided by the ASE 15 “Galaxy” features.

12

|

Chapter 1: Exploring the Sybase Galaxy

3, 2, 1, Contact! It is a new and exciting time in the universe of database technology. The use of the database will broaden to handle the new challenges of not only data management, but information management. The future of information management is here today. So sit back, buckle up, and enjoy the ride as we explore the outer reaches of the “Galaxy” release known as Sybase ASE 15.

Chapter 2: System Maintenance Improvements

|

13

Chapter 2

System Maintenance Improvements For a database administrator, system maintenance can be one way to separate the wheat from the chaff. As a rule of thumb, a good database administrator should spend less time fixing server issues and more time preventing them. With the release of ASE 15, Sybase has added new features to give the database administrator a helping hand. The new features can be implemented to not only help the database administrator, but to perform certain tasks for the database administrator. The new features will assist database administrators in preparing their systems for next-generation platforms. These features will also enable ASE to handle very large, data-intensive environments. This chapter focuses on the system maintenance improvements designed to help all database administrators, especially those pertaining to VLDB installations. The chapter also reviews the evolution of ASE 15’s new features as they progressed from ASE 12.0 through 12.5.x.

Recent Pre-ASE 15 Improvements As mentioned in Chapter 1, Sybase has taken steps toward automating and streamlining system administration and server maintenance tasks. Many of the new features in ASE 15 are available with help from the building blocks Sybase placed within the ASE 12.5.x point releases. Listing these pre-ASE 15 enhancements, we have: multiple

14

|

Chapter 2: System Maintenance Improvements

tempdb, native data encryption, automatic database expansion, and the Job Scheduler feature.

Multiple tempdb The multiple tempdb capability was introduced in ASE 12.5.0.3. This feature allows the database administrator to create special “user-defined” temporary databases and to distribute tempdb activity across multiple temporary databases. The enhancement helps to eliminate a long-standing point of contention within the ASE architecture. Constructed properly, the multiple tempdb feature can allow for a “back door” into ASE in the event of a full system tempdb. A prudent database administrator can create a privileged login bound to a user-defined tempdb. From this user-defined tempdb, the privileged user can perform maintenance on the system tempdb in the event the system tempdb experiences a log full problem. The setup of this “back door” is relatively simple, and can save the database administrator from an unplanned recycle of ASE. For further information on the implementation of multiple tempdb in ASE, please see Chapter 13, as well as the Sybase Adaptive Server System Administration Guide.

Native Data Encryption/Security Enhancements Sybase plans to integrate the native data encryption feature into the ASE 15 product line in the 15.0.1 interim release. For now, native data encryption exists for ASE 12.5.3a. For detailed information on this subject, please see Sybase’s “New Features Adaptive Server Enterprise 12.5.3a” document. The following items are a few of the highlights of the native data encryption feature: t Database administrators or security officers (sso_role) have the ability to create encryption keys at the database level using the Advanced Encryption Standard (AES) algorithm. t Multiple encryption keys are permitted within a single ASE database. t Encrypted data is stored on disk or in memory as encrypted data and not plain text. The encryption can be verified with the dbcc page command:

Chapter 2: System Maintenance Improvements

|

15

dbcc page (dbid, pageno,[printopt [,cache [,logical [,cachename]]]]) dbcc traceon(3604) go dbcc page (5,1282,1) -- specify 1 as the third option to dump the contents of the page. go t

Encrypted data, as opposed to plain text, is stored in the transaction log. This can be verified with the dbcc log command: dbcc log (dbid, objid, pageno, rowno, nrecs, type, printopt) dbcc traceon(3604) go dbcc log (6, 553769030) go

t

No external programming logic is necessary to decrypt encrypted data.

Recommendation: To enhance the level of security with ASE 15, separate the sso_role from the sa_role in ASE. Grant these roles to two or more different privileged users. Additionally, it is recommended to lock the sa login for ASE as this login has both the sa_role and sso_role upon installation of ASE.

Automatic Database Expansion Automatic database expansion was first introduced in ASE 12.5.1. This feature is used to automatically increase database and device sizes. In order to accomplish the resize, the auto-expand feature uses administrator-defined thresholds to determine when the expansion will occur. The management of automatic database expansion can be accomplished with the sp_dbextend system procedure. Prior to implementing automatic database expansion, the sp_dbextend system procedure will need to be installed by the installdbextend script. According to Sybase’s “What’s New in ASE” guide for ASE 12.5.1, new rows will be added to the sysattributes table in the master database in order to manage and document the expansion thresholds set up by the database administrator. Additionally, 93 system procedures are added to the sybsystemprocs database by the installdbextend script.

16

|

Chapter 2: System Maintenance Improvements

The Basics The sp_dbextend system procedure is the tool used to set up and maintain automatic database expansion. The procedure can be used to provide a listing of the syntax options, which are included here for reference: sp_dbextend "help" go Usage: sp_dbextend [arguments ...] Usage: sp_dbextend help [,] Usage: sp_dbextend 'set', 'threshold', @dbname, @segmentname, @freespace Usage: sp_dbextend 'set', 'database', @dbname, @segmentname {[,@growby] [,@maxsize]} Usage: sp_dbextend 'set', 'device', @devicename {[,@growby] [,@maxsize]} Usage: sp_dbextend 'clear', 'threshold', @dbname, @segmentname [,@freespace] Usage: sp_dbextend 'clear', 'database' [,@dbname [,@segmentname]] Usage: sp_dbextend 'clear', 'device' [,@devicename] Usage: sp_dbextend 'modify', 'database', @dbname, @segmentname, {'growby' | 'maxsize'}, @newvalue Usage: sp_dbextend 'modify', 'device', @devicename, {'growby' | 'maxsize'}, @newvalue Usage: sp_dbextend {'list' | 'listfull'} [,'database' [,@dbname [,@segmentname [,@ORDER_BY_clause]]]] Usage: sp_dbextend {'list' | 'listfull'} [,'device' [,@devicename [,@ORDER_BY_clause]]] Usage: sp_dbextend 'check', 'database' [,@dbname [,@segmentname]] Usage: sp_dbextend {'simulate' | 'execute', @dbname, @segmentname [,@iterations] Usage: sp_dbextend 'trace', {'on' | 'off'} Usage: sp_dbextend 'reload [defaults]' Usage: sp_dbextend {'enable' | 'disable'}, 'database' [,@dbname [,@segmentname]] Usage: sp_dbextend 'who' [,'' | 'block' | 'all']

The Sybase reference manual provides a detailed explanation of the commands and parameters for the sp_dbextend procedure shown in the above example. To allow automatic database expansion, the database administrator must accomplish two main tasks with the sp_dbextend procedure: Set up the sp_dbextend action, and define the sp_dbextend threshold. The setup of the action and definition of the threshold is demonstrated in the following example, where the logsegment of database Properties is set up to extend by 50 MB

Chapter 2: System Maintenance Improvements

|

17

(the action) when the logsegment reaches 5 MB (the threshold) of remaining freespace. Example: sp_dbextend 'set', 'database', properties, "logsegment", '50m' sp_dbextend 'set', 'thresh', properties, "logsegment", '5m'

When the threshold of 5 MB is reached, the database’s logsegment is extended by 50 MB. Messages similar to the following will be sent to ASE’s errorlog: 00:00000:00026:2005/09/12 21:59:35.09 server background task message: Threshold action procedure 'sp_dbxt_extend_db' fired in db 'properties' on segment 'logsegment'. Space left: 2560 logical pages ('5M'). 00:00000:00026:2005/09/12 21:59:35.11 server background task message: ALTER DATABASE properties log on logspace1 = '50.0M' -- Segment: logsegment 00:00000:00026:2005/09/12 21:59:35.13 server Extending database by 25600 pages (50.0 megabytes) on disk logspace1.

The above example demonstrates where the database is mapped to a device on which the logsegment has space available to expand. The output from the errorlog indicates the device logspace1 had free space remaining on which to expand. Recommendation: Use the automatic database expansion feature of ASE with caution. Automatically expanding a database is not always the best solution for dealing with space utilization.

As opposed to automatic expansion of a database, often data may need to be archived or deleted from a system, or perhaps a long-running transaction needs to be flushed from ASE. The enablement of automatic expansion can allow for databases to become inconsistent in size between production, development, and test environments. Many database administrators support environments where production data is cascaded to development or test servers. Automatic expansion of the production database, for example, would create a waterfall effect of database resizing for these servers where data is duplicated from production to development or test regions via a dump and load process. Additionally, with a larger database, more time is needed to perform restore operations since all pages allocated to a database are recovered, regardless of the presence of data within

18

|

Chapter 2: System Maintenance Improvements

the disk allocations. Finally, for users of the dbcc checkstorage consistency checker, alteration of the dbccdb configuration may be necessary for databases where automatic expansion has occurred.

Job Scheduler As a precursor to some of the automatic administration tasks, Job Scheduler was introduced in ASE 12.5.1. In introducing this feature, Sybase took the first steps toward offering automatic administration from within a Sybase product. Job Scheduler allows an administrator to create and schedule jobs, as well as share jobs and schedules. This means one database administrator can create a job and other database administrators can then schedule and run that job on another server. Job Scheduler jobs can be created from scratch using the command line or GUI, or from a SQL batch file. Jobs can also be created from a predefined template. Job Scheduler can also be used as a tool for messaging. It captures the results and output of jobs and records that information in log tables. This data can then be queried and used for creating meaningful messages. In addition, Job Scheduler keeps a history of scheduled jobs. Job Scheduler is self-monitoring and removes outdated, unnecessary history records. By doing this, a limit can be kept on the size of the history table. In addition, many of the new features in ASE 15 have been integrated with Job Scheduler.

Basic Components Job Scheduler has two main functional components: the Job Scheduler Task (JS Task) and the Job Scheduler Agent (JS Agent). The JS Task manages the schedules and notifies the JS Agent to execute a job. The JS Agent then carries out job execution.

Installation of Job Scheduler The Job Scheduler can be installed in conjunction with the server executed through the installation GUI. Additionally, Job Scheduler can be installed manually after the ASE server is built. Please see Chapter 12 for installation of the Job Scheduler through the Sybase installation GUI. Additionally, the installation methods and creation of jobs are well outlined in Sybase’s Job Scheduler User’s Guide.

Chapter 2: System Maintenance Improvements

|

19

With the Job Scheduler, it is important to note the scheduler is backwardly compatible between versions of ASE, to a degree. For example, basic maintenance jobs such as dbcc checkstorage launched against a target database by the Job Scheduler can be expected to operate against a 12.5.x ASE server when launched from a JS within ASE 15. Some of the newer job templates will not work against a 12.5.x server, such as the job template to automatically update statistics.

ASE 15 Improvements Row Locked System Catalogs Yes, that’s right. You read correctly. It does say “row locked system catalogs.” At last, the feature that many Sybase database administrators have been waiting for. Key tables in all databases will be row locked. However, it is important to note this feature is only partially implemented in the ASE 15 GA release. For the ASE 15 GA release, modifications to any system catalog will continue to use exclusive table locks, similar to the behavior exhibited by pre-ASE 15 systems. This was a deliberate multi-phased release. Since the catalog and catalog index changes were required to support DOL locking, the DOL locking schema changes are implemented as part of the 15 upgrade process. This is the “preparatory” phase for the full implementation of row locked catalogs in which the DOL locking would be enabled in the code. Two phases were decided upon as catalog changes were already needed to support partitioning (avoids having a catalog change/mini-upgrade when RLC is finally implemented), allowing it to be implemented in a simple EBF vs. IR or upgrade. Once this handy new feature is fully implemented, it will add grace and ease to server maintenance and administration tasks by allowing similar operations, such as DDL execution, to occur simultaneously. By allowing increased concurrency to system catalogs, maintenance operations will no longer have contention on the system catalogs. Another advantage of row locked system catalogs is the reduction of lock contention for DDL operations, allowing higher throughput for applications, especially those applications where

20

|

Chapter 2: System Maintenance Improvements

tempdb usage is high. This advantage is accomplished by easing and even eliminating lock contention in the temporary database(s). Blocking and deadlocks due to stored procedure renormalization are largely eliminated. Less contention means less waiting, which in turn means improved application performance. Tip: Multiple tempdbs are still a good idea to relieve log and cache contention.

Update Statistics The update statistics command for ASE 15 is enhanced twofold. First, the statistics can be updated at the partition level in ASE 15. Additionally, Sybase introduces the datachange function, which allows the database administrator to measure the amount of change to the data underlying the collected statistics at the table, index, column, and partition level. Based on the values returned by the datachange function, database administrators can make update statistics operations optional, as the datachange function provides a basis to measure the amount of changed data at a specified level. Let’s begin by exploring the enhanced syntax for update statistics in ASE 15.

Updates to Partition Statistics Prior to ASE 15, update statistics operations were only possible on the columns or indexes of a table in their entirety. Often, the maintenance windows are too minimal to perform update statistics on tables. Some database administrators were forced to reduce the frequency of update statistics operations, or possibly compromise the accuracy of update statistics operations with statistics sampling. With the introduction of semantic partitions in ASE 15, database administrators can now update statistics at the partition level, offering an opportunity to split the update statistics operations for large objects between multiple maintenance windows.

Chapter 2: System Maintenance Improvements

|

21

Syntax: update statistics table_name [[partition data_partition_name] [(column_list)] | index_name [partition index_partition_name]] [using step values] [with consumers = consumers] update index statistics table_name [[partition data_partition_name] |[index_name [partition index_partition_name]]] [using step values] [with consumers = consumers] update all statistics table_name [partition data_partition_name] update table statistics table_name [partition data_partition_name] delete [shared] statistics table_name [partition data_partition_name] [(column_name [,column_name] ...)]

It is important to note the old syntax to perform statistics updates for partitions of update partition statistics is a deprecated command in ASE 15. The above syntax indicates updates to partition-level statistics is accomplished with the update statistics table command. Prior to ASE 15, the rationale behind update partition statistics was to get a better picture of partition skew for parallel query optimization. In other words, the more pronounced the skew, the less likely the optimizer would choose a parallel query plan. With semantic partitioning, data skew is virtually guaranteed as the data is partitioned by value and the values are not likely to be evenly distributed. However, this effectively deprecates “partition skew” as an issue as partition elimination, directed joins, and other parallel query optimization techniques now rely on the data semantics vs. the skew for optimization decisions. Round-robin partitions, however, likely are still treated the same. Examples: This example updates statistics on the data partition part1. For partition-level update statistics, Adaptive Server creates histograms for each major attribute of the local indexes for the specified partition and creates densities for the composite attributes. Adaptive Server also creates histograms for all non-indexed columns. update statistics ckt_table partition part1

This example updates all statistics on the data partition part1: update all statistics ckt_table partition part1

22

|

Chapter 2: System Maintenance Improvements

This example regenerates systabstats data for data partition part2 of table user_table: update table statistics user_table partition part2

Automatic Update Statistics Automated statistics updates can be accomplished with the combination of different scheduling techniques, and even made optional with the utilization of the datachange feature. As discussed in the earlier section covering the Job Scheduler, the control of update statistics jobs can be managed from within ASE. The goal of Automatic Update Statistics is to automatically determine when to run update statistics and minimize the impact on ASE performance. Automatic Update Statistics allows users to determine the objects, schedules, and datachange thresholds to automate the process. In other words, it allows the database administrator to choose the optimum time to run update statistics, and only run update statistics as required. Implemented properly, the addition of the datachange function can minimize or eliminate one of the database administrator’s largest maintenance windows. With ASE 15, it is possible for the database administrator to run update stats on systems where this maintenance effort was otherwise infeasible due to time constraints, especially when combining the datachange feature with the ability to update stats at the partition level, and the statistics sampling concept introduced in ASE 12.5.

Datachange The datachange function is the key to identifying whether update statistics operations on a table, index, partition, or column is necessary. The datachange function returns a percentage value to indicate how much the data has changed within a table, partition, index, or column. A value of 0% indicates no changes to the object are measured by the datachange function. As the data changes, the value returned will increase. Syntax: select datachange(object_name, partition_name, column_name)

Chapter 2: System Maintenance Improvements

|

23

Examples: Measure the data changed at the table level: select datachange("authors", null, null)

Measure the data change to the identification column of the authors table: select datachange("authors", null, "identification")

Measure the data change to the authors_part2 partition of the authors table: select datachange("authors", "authors_part2", null)

Measure the data change to the identification column contained in the authors_part4 partition of the authors table: select datachange("authors", "authors_part4", "identification")

It is possible for the datachange function to return values greater than 100%. These values are possible due to the way the datachange function measures updates. Updates are measured as a delete and an insert against the measured object. Thus, a count of 2 is contributed to the datachange counter for updates. For inserts and deletes, the count is incremented by 1 for each single row affected. Take a look at the following example, which demonstrates how the datachange function can report values of over 100%. First, consider how the datachange value is calculated for updates using the table data described below. Information about the identification table: select identification, "rowcount" = count(*) from authors group by identification identification Carrie Taylor Steve Bradley Naresh Adurty Jagan Reddy Brian Taylor

rowcount 40000 30000 20000 10000 10000

24

|

Chapter 2: System Maintenance Improvements

Step 1: Update statistics on the identification column of the authors table: update statistics authors(identification) go

Step 2: Verify the datachange function reports no data changed to the identification column of the authors table. The datachange value should report 0.0 since no data has changed in the identification column since update statistics on this column last took place: select datachange("authors", null, "identification")

Output: 0.0

Step 3: Update 63% of the data in the identification column of the authors table: update authors set identification = "DataChange Demo" where identification = "Carrie Taylor" update authors set identification = "Testing" where identification = "Steve Bradley"

Step 4: Run the datachange function against the identification column of the authors table: select datachange("authors", null, "identification")

Output: 127.272727

Note that 63% of the data changed based on the contents of the table; however, the datachange function is reporting that 127% of the data in the identification column has changed. Since this operation was an update, the data changes are counted twice, therefore the datachange value is computed by ASE as follows: 70,000 updates = 70,000 inserts + 70,000 deletes = 140,000 datachange counts = 140,000 datachange counts/110,000 rows in table = 1.272727, or 127.27%

Chapter 2: System Maintenance Improvements

|

25

Note: For this example with datachange, a bulk update is employed, which will effectively be processed as a deferred update by ASE. Had the example employed a row by row update, perhaps through a cursor, the doubling of the counted changes shown in this example may not have occurred.

Now let’s consider how the datachange value is calculated for deletes using the same table data described above. Note: Data reset to the original data, as displayed on page 23.

Step 1: Update the statistics on the identification column of the authors table: update statistics authors(identification) go

Step 2: Verify the datachange function reports no data changed to the identification column of the authors table. The datachange value should report 0.0 since no data has changed in the identification column since update statistics on this column last took place: select datachange("authors", null, "identification")

Output: 0.0

Step 3: Delete 30,000 rows from the authors table, reducing the rowcount from 110,000 to 80,000: delete authors where identification = "Steve Bradley" go

Step 4: Run the datachange function against the identification column of the authors table: select datachange("authors", null, "identification") go

Output: 37.5

Note that according to the datachange function, 37.5% of the identification column has changed since the statistics for this column were last updated. So why is datachange reporting 37.5% of the column’s data changed when only 27% of the table’s data was deleted?

26

|

Chapter 2: System Maintenance Improvements

Because ASE measures the change counts against the current rowcount in the table, and not the rowcount prior to the delete operation. The same measurement principle applies for insert operations as demonstrated in the next example. Evaluate the datachange calculation: 110,000 30,000 80,000 30,000/80,000

= rowcount prior to delete = number of rows deleted = number of rows in the table after the delete = 37.5% of the data is changed

Finally, let’s consider how the datachange value is calculated for inserts using the same table data described above. Note: Data reset to the original data, as displayed on page 23.

Step 1: Update the statistics on the identification column of the authors table: update statistics authors(identification) go

Step 2: Verify the datachange function reports no data changed to the identification column of the authors table. The datachange value should report 0.0 since no data has changed in the identification column since update statistics on this column last took place: select datachange("authors", null, "identification")

Output: 0.0

Step 3: Insert 30,000 rows into the authors table, increasing the rowcount from 110,000 to 140,000. Step 4: Run the datachange function against the identification column of the authors table: select datachange("authors", null, "identification") go

Output: 21.428571

Chapter 2: System Maintenance Improvements

|

27

Evaluate the datachange calculation: 110,000 30,000 140,000 30,000/140,000

= rowcount prior to insert = rows inserted = number of rows in the table after the delete = 21.42% of the data is changed

Tip: Two conclusions can be drawn from this example. First, the datachange function evaluates the percentage of data changed based upon the current rowcount of the table. Second, deletes have a greater impact than inserts on the datachange calculation as demonstrated in this example.

Why Use datachange? The datachange function measures the amount of data that has changed at the column, table, or partition level where statistics are maintained. The value reported by datachange is useful for determining if additional runs of the update statistics command are necessary to synchronize data statistics with the actual data. In other words, scrutiny of the datachange function can determine the need to run the update statistics process at the table, column, index, or partition level. Tip: Use the datachange function prior to the launch of any update statistics command to determine whether the update statistics command is necessary. Use of the datachange function provides an opportunity for the database administrator to minimize or eliminate one of the traditionally largest maintenance windows for Sybase ASE databases.

Evaluation of the datachange column prior to the issuance of any update statistics command is presented with the following: declare @datachange numeric(5,1) select @datachange = datachange("authors", null, "identification") if (@datachange > 20.0) begin select "updating statistics of identification column for table authors" update statistics authors(identification) end else if (@datachange sp_countmetadata "open partitions" 2> go There are 239 user partitions in all database(s), requiring 505 Kbytes of memory. The 'open partitions' configuration parameter is currently set to 500. (return status = 0)

Open partitions within specific database: 1> sp_countmetadata "open partitions", "events" 2> go There are 9 user partitions in events database(s), requiring 505 Kbytes of memory. The 'open partitions' configuration parameter is currently set to 500. (return status = 0)

Note: The section of Chapter 3 called “Semantic Partitions” discusses the fact that all tables in ASE are now considered to be partitioned. By default, all tables are partition type round-robin. The sp_countmetadata procedure will return a count on user-created partitions of all types; however, round-robin partitions will only be counted by sp_countmetadata where number of partitions is greater than one. t

partition spinlock ratio — Used to set the ratio of spinlocks to par-

tition caches. This configuration option is only relevant to ASE when the number of online engines is greater than one.

Chapter 2: System Maintenance Improvements

|

35

Utility Benefits from Semantic Partitions Adaptive Server 15 supports horizontal partitioning, wherein a collection of table rows are distributed among multiple disk devices. In addition, ASE 15 supports semantic partitioning, wherein valuebased partitioning schemes, such as range and hash, partition data according to value. Significant benefits of semantic partitions include: t Improved scalability t Improved performance — Concurrent multiple I/O on different partitions, and multiple threads on multiple CPUs working concurrently on multiple partitions t Faster response time t Partition transparency t VLDB support — Concurrent scanning of multiple partitions of very large tables t Range partitioning to manage historical data, hash partitioning, and round-robin partitioning In addition to the aforementioned benefits of semantic partitions, system maintenance on ASE 15 is improved. The improvements are a result of partition-specific maintenance capabilities with the enhancements of several utilities to perform system administration and maintenance tasks at the partition level.

Partition-specific Database Consistency Checks (dbccs) For ASE 15, a portion of the database consistency checks are enhanced with the ability to check consistency at the partition level for semantically partitioned tables. The partition-enabled dbcc commands are dbcc checktable, dbcc checkindex, dbcc indexalloc, and dbcc tablealloc. dbcc checktable Syntax dbcc checktable({table_name | table_id} [,skip_ncindex | "fix_spacebits" [,"partition_name" | partition_id]])

For dbcc checktable, a specific partition name or partition ID can be supplied. The partition name or ID limits checktable’s analysis to the data within a specific partition, and the local indexes for the specified partition.

36

|

Chapter 2: System Maintenance Improvements

Example: dbcc checktable(authors, null, batch1)

Output: Checking partition 'batch1' (partition ID 569769087) of table 'authors'. The logical page size of this table is 2048 bytes. The total number of data pages in partition 'batch1' (partition ID 569769087) is 981. Partition 'batch1' (partition ID 569769087) has 55000 data rows. DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.

dbcc checkindex Syntax dbcc checkindex({table_name | table_id}, index_id, bottom_up [,partition_name | partition_id]])

The dbcc checkindex command is similar to dbcc checktable, except that checks are limited to the named index of the table. Additionally, with dbcc checkindex it is possible to perform the consistency checks for DOL tables from the leaf level upward toward the root level with the bottom_up parameter. Example: -- checkindex executed on user-named index partition: dbcc checkindex(authors, 5, bottom_up, batch1)

Output: Checking partition 'batch1' (partition ID 841770056) of table 'authors'. The logical page size of this table is 2048 bytes. Table has 55000 data rows. Index has 55000 leaf rids. The total number of data pages in this table is 1002. DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.

Example: -- checkindex executed on system named index partition: dbcc checkindex(authors, 3, null, authors_idx2_729769657)

Output: Checking partition 'authors_idx2_729769657' (partition ID 729769657) of table 'authors'. The logical page size of this table is 2048 bytes. DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.

Chapter 2: System Maintenance Improvements

|

37

dbcc indexalloc Syntax The dbcc indexalloc command can perform analysis on a specific index partition. The first argument of the dbcc indexalloc command accepts partition_id as an input parameter. An example of dbcc indexalloc is executed against a single partition index where index_id = 3. dbcc indexalloc(object_name | object_id | partition_id, index_id [,{full | optimized | fast | null} [,fix | nofix]])

Example: dbcc indexalloc(713769600, 3, full, fix)

Output: *************************************************************** TABLE: authors OBJID = 553769030 PARTITION ID=713769600 FIRST=610281 ROOT=610280 SORT=0 Indid : 3, partition : 713769600. 47 Index pages allocated and 7 Extents allocated. TOTAL # of extents = 7 Alloc page 610048 (# of extent=2 used pages=9 ref pages=9) Alloc page 610304 (# of extent=5 used pages=39 ref pages=39) Total (# of extent=7 used pages=48 ref pages=48) in this database DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.

dbcc tablealloc Syntax The tablealloc check can be performed at the partition level when passed partition_id. The tablealloc command at the partition level will also perform checks on all of the local indexes for the stated partition_id. dbcc tablealloc(object_name | object_id | partition_id, [,{full | optimized | fast | null} [,fix | nofix]])

Example: dbcc tablealloc(569769087, full, fix)

Output: *************************************************************** TABLE: authors OBJID = 553769030 PARTITION ID=569769087 FIRST=16642 ROOT=75755 SORT=0 Data level: indid 0, partition 569769087. 1002 Data pages allocated and 126 Extents allocated.

38

|

Chapter 2: System Maintenance Improvements

TOTAL # of extents = 126 Alloc page 16640 (# of extent=2 used pages=16 ref pages=16) Alloc page 30464 (# of extent=1 used pages=8 ref pages=8) Alloc page 34816 (# of extent=1 used pages=8 ref pages=8) . . Alloc page 75008 (# of extent=9 used pages=72 ref pages=72) Alloc page 75264 (# of extent=10 used pages=80 ref pages=80) Alloc page 75520 (# of extent=8 used pages=60 ref pages=60) Total (# of extent=126 used pages=1004 ref pages=1004) in this database DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.

Reorg Partitions ASE 15 enables the reclaim_space, forwarded_rows, and compact parameters of the reorg command to run on a single partition. The reorg rebuild command can also be run against single index partitions. Benefits of Reorg Partitions Traditionally, reorg operations on the largest of tables is a very timeconsuming operation for ASE servers. Additionally, reorg operations have only been permitted to operate at the table or index level. Reorg at the partition level is a new feature for ASE 15. Using reorg at the partition level provides the database administrator a mechanism to divide the reorg tasks for very large tables to multiple maintenance windows. Additionally, if the database administrator knows only a subset of the table or index’s partitions are fragmented, reorg operations can be directed to only those partitions where reorgs are necessary. Syntax: reorg forwarded_rows table_name partition partition_name [with {resume, time = no_of_minutes}] reorg reclaim_space table_name [index_name] partition partition_name with {resume, time = no_of_minutes}] reorg compact table_name partition partition_name with {resume, time = no_of_minutes}] reorg rebuild table_name [index_name [partition index_partition_name]

Chapter 2: System Maintenance Improvements

|

39

where partition_name is the name of the partition that contains the table on which you are running reorg, and index_partition_name is the name of the partition index. Note: For ASE 15 GA, reorg rebuild works at the table level, even when passed a partition name as an input parameter. The ability to perform reorg rebuild at the partition level may be available in a future IR release of ASE 15.

Examples: Run reorg rebuild on the rental_idx local index of the Rental table. The rental_idx2 local index resides on the rental_idx2_681769486 system-named index partition: reorg rebuild Rental rental_idx2 partition rental_idx2_681769486

Run reorg reclaim_space on the Rental table’s clusered index. This is a local index, so perform the reorg on only the portion of the local index that is specific to the partition named range_part1: reorg reclaim_space Rental rental_clustered_idx partition range_part1

Run reorg compact on the portion of the Rental table specific to the range partition range_part2: reorg compact Rental partition range_part2

Run reorg forwarded_rows on the portion of the Rental table specific to the range partition range_part4: reorg forwarded_rows Rental partition range_part4

Run reorg rebuild on the Rental table: reorg rebuild Rental partition batch2

Note: Remember for ASE 15 GA, reorg rebuild on a specific partition performs the reorg operation on the table as a whole.

Changes to the bcp Utility The bcp utility is enhanced for ASE 15 to support bcp operations at the semantic partition level. This is not the only improvement that has occurred for the bcp utility. The capability of using the bcp utility on computed columns has also been included. These enhancements address pre ASE-15 limitations of the bcp utility.

40

|

Chapter 2: System Maintenance Improvements

The limitations of the pre-ASE 15 version of bcp are: t Can only specify one partition and one file in a bcp command t The --maxconn parameter is not available bcp for ASE 15 allows the database administrator to: t

t t t t

bcp data in or out from any type of partitioned table: range, hash, list, or round-robin. Parallel bcp supports partitions in both slow

and fast modes. Bulk copy data out from all partitions or a subset of partitions Bulk copy data out to a single file or partition-specific files Bulk copy data in from a single file, multiple files, or partition-specific files Bulk copy data in parallel to specific partitions

The new syntax for bcp is in bold: bcp [[db_name.]owner.]table_name [:slice_num] [partition pname] {in | out} [filename] [-m maxerrors] [-f formatfile] [-e errfile] [-F firstrow] [-L lastrow] [-b batchsize] [-n] [-c] [-t field_terminator] [-r row_terminator] [-U username] [-P password] [-I interfaces_file] [-S server] [-a display_charset] [-z language] [-v] [-A packet size] [-J client character set] [-T text_or_image size] [-E] [-g id_start_value] [-N] [-X] [-M LabelName LabelValue] [-labeled] [-K keytab_file] [-R remote_server_principal] [-V [security_options]] [-Z security_mechanism] [-Q] [-Y] [--maxconn maximum_connections] [--show-fi] [--hide-vcc]

where: t

t

slice_num — A number designating the partition from which you are bulk copying data. This number ranges from 1 to the total number of partitions in the table. For example, if the table has 15 partitions, the range for slice_num would be from 1 to 15. slice_num is only valid for bcp in round-robin partitioned tables. partition pname — Specifies a comma-delimited set of one or more unique partitions. This parameter can be used for both bcp out and bcp in. This parameter cannot be used in conjunction with the slice_num option.

Chapter 2: System Maintenance Improvements

t t

t t

|

41

filename — Specifies a comma-delimited set of one or more datafiles. Can be specified for both bcp in and bcp out. maximum_connections — The maximum number of parallel connections bcp can open to the server. If this is not specified, bcp automatically determines the number of connections. show-fi — Tells bcp to copy functional index data. This parameter can be used for both bcp out and bcp in. hide-vcc — Tells bcp not to copy virtual computed columns. This parameter can be used for both bcp out and bcp in.

Examples: Copies the Rental table to the rentals.dat file: bcp Properties..Rental out rentals.dat

Copies the contents of the datafile rentals.dat file to Rental: bcp Properties..Rental in rentals.dat

Copies the contents of partitions p2, p3, and p4 to the rentals.dat file: bcp Properties..Rental partition p2, p3, p4 out rentals.dat

Copies rentals.dat into the first partition of the round-robin partitioned table titles: bcp Properties..Rental:1 in rentals.dat

Copies multiple datafiles into the Rental table: bcp Properties..Rental in p2.dat, p3.dat, p4.dat

Copies Rental to a file: bcp Properties..Rental out

Because a name is not specified in the command, the file is named the same as the partition with a .dat extension added. For example, if the table consists of the partitions p1, p2, p3, and p4, bcp creates four output datafiles: p1.dat, p2.dat, p3.dat, and p4.dat. If the table is not partitioned, the datafile is named after the single default partition that comprises the table. The Tenants table below is non-partitioned, and a bcp out with the following syntax will result in one bcp file, named in conjunction with the single partition name for this table: bcp Properties..Tenants out

42

|

Chapter 2: System Maintenance Improvements

The following command specifies a subset of the partitions for the Rental table. Since the command does not specify an output file but does specify a subset of the partitions for the Rental table, bcp creates two datafiles: p3.dat and p4.dat. bcp Properties..Rental partition p3, p4 out

Usage t For bcp out, if you specify both the partition pname clause and filename clause, either: • The filename parameter contains a single datafile and all data will be copied to that file, or There is a one-to-one mapping between the partition names • and the datafiles, and the data from a partition will be copied to its specific datafile. t If a specified partition does not exist, the bcp out command fails and does not copy out any of the partitions. t You cannot use partition_id with bcp out. t For bcp out, if you do not specify a file name, each partition’s data is copied to a file that is named after the partition and appended with a .dat extension. t For bcp in, if you do not specify any partitions, you can list multiple data files. If you specify partitions, you must include a one-to-one mapping between the partition names and the datafiles. When you bcp data into a table, the input file can contain data for one or more partitions. Clients can send data across parallel connections, and each connection is a dedicated channel for a set of one or more partitions. If the bcp client has partition-level information to map files to partitions, bcp can initiate dedicated connections for each partition into which data is being copied. bcp: Client-side Parallelism The bcp client determines the number of connections to the server. For maximum parallelism, this includes a separate connection for each partition to the file mapping provided as parameters to the bcp command. The maxconn value indicates the maximum number of connections the bcp client is allowed to set up. Without this parameter, the

Chapter 2: System Maintenance Improvements

|

43

bcp client automatically chooses the number of connections. One connection per partition is ideal, but if that is not possible, bcp will multiplex batches for more than one partition across a connection. The batch_size option for bcp in applies to each file that is copied into a table, regardless of whether the data goes into a single partition or is distributed across multiple partitions.

Truncate Partitions With ASE 15, truncate table has been enhanced to deallocate the data from ASE 15 partitions. Truncate partitions can be performed on the default round-robin style partition in addition to tables semantically partitioned. As with truncations at the table level, pages are deallocated with truncate, but only at the partition level. Similar to truncate table, using the partition argument does not log the deletes one row at a time, and is much faster than a delete with a where clause to delete data from a single partition. Note: Truncate table at the partition level can only truncate one partition at a time.

The syntax is: truncate table [[database.] owner.] table_name [partition partition_name [,partition_name] ...]

Truncate the data from a partition that is partitioned as a four-way hash partitioned table. Prior to truncate: partition_name p1 p2 p3 p4

partition_id 1433772165 1449772222 1465772279 1481772336

pages 131 129 132 130

Truncate syntax: truncate table authors partition authors_1449772222

After truncate: partition_name p1

partition_id 1433772165

pages 131

44

|

Chapter 2: System Maintenance Improvements

p2 p3 p4

1449772222 1465772279 1481772336

1 132 130

Note that after a truncate partition of the hash partitioned table, the database administrator will need to consider partition rebalancing. Looking Forward — Drop Partition Note: Since drop partition is not available in the GA release of ASE 15, use truncate partition where drop partition would otherwise be used.

In the future, the drop partition command may be included in ASE. If so, some potentially interesting scenarios could occur. For example, if a database administrator drops a partition and then later needs to restore the data, re-adding the partition may get interesting, particularly for range partitions. Think of what happens if you have a semantic partitioned table by range