www.dbebooks.com - Free Books & magazines
EXPERT SYSTEMS IN CHEMISTRY RESEARCH
5323X.indb 1
11/13/07 2:08:37 PM
53...
96 downloads
2066 Views
25MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
www.dbebooks.com - Free Books & magazines
EXPERT SYSTEMS IN CHEMISTRY RESEARCH
5323X.indb 1
11/13/07 2:08:37 PM
5323X.indb 2
11/13/07 2:08:37 PM
EXPERT SYSTEMS IN CHEMISTRY RESEARCH Markus C. Hemmer
Boca Raton London New York
CRC Press is an imprint of the Taylor & Francis Group, an informa business
5323X.indb 3
11/13/07 2:08:37 PM
CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487‑2742 © 2008 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed in the United States of America on acid‑free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number‑13: 978‑1‑4200‑5323‑4 (Hardcover) This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the conse‑ quences of their use. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978‑750‑8400. CCC is a not‑for‑profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Hemmer, Markus C. Expert systems in chemistry research / Markus C. Hemmer. p. cm. Includes bibliographical references and index. ISBN 978‑1‑4200‑5323‑4 (hardback : alk. paper) 1. Chemistry‑‑Data processing. 2. Chemistry‑‑Research. I. Title. QD39.3.E46.H46 2007 542’.85633‑‑dc22
2007031226
Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com
5323X.indb 4
11/13/07 2:08:38 PM
Dedicated to my father, who has aided and accompanied my scientific career all these years.
5323X.indb 5
11/13/07 2:08:38 PM
5323X.indb 6
11/13/07 2:08:38 PM
Contents Preface....................................................................................................................xvii Acknowledgments....................................................................................................xix Trademark Information............................................................................................xxi Chapter 1 Introduction........................................................................................... 1 1.1 Introduction...................................................................................................... 1 1.2 What We Are Talking About...........................................................................1 1.3 The Concise Summary.....................................................................................3 1.4 Some Initial Thoughts......................................................................................3 References................................................................................................................... 8 Chapter 2 Basic Concepts of Expert Systems........................................................ 9 2.1 2.2 2.3
What Are Expert Systems?.............................................................................. 9 The Conceptual Design of an Expert System................................................ 10 Knowledge and Knowledge Representation.................................................. 12 2.3.1 Rules................................................................................................... 12 2.3.2 Semantic Networks............................................................................. 14 2.3.3 Frames................................................................................................. 16 2.3.4 Advantages of Rules........................................................................... 18 2.3.4.1 Declarative Language............................................................ 18 2.3.4.2 Separation of Business Logic and Data................................ 18 2.3.4.3 Centralized Knowledge Base................................................ 18 2.3.4.4 Performance and Scalability................................................. 19 2.3.5 When to Use Rules.............................................................................. 19 2.4 Reasoning.......................................................................................................20 2.4.1 The Inference Engine..........................................................................20 2.4.2 Forward and Backward Chaining....................................................... 22 2.4.3 Case-Based Reasoning........................................................................ 22 2.5 The Fuzzy World............................................................................................24 2.5.1 Certainty Factors................................................................................24 2.5.2 Fuzzy Logic........................................................................................25 2.5.3 Hidden Markov Models......................................................................26 2.5.4 Working with Probabilities — Bayesian Networks............................ 27 2.5.5 Dempster-Shafer Theory of Evidence.................................................28 2.6 Gathering Knowledge — Knowledge Engineering....................................... 29 2.7 Concise Summary.......................................................................................... 31 References................................................................................................................. 32 vii
5323X.indb 7
11/13/07 2:08:38 PM
viii
Expert Systems in Chemistry Research
Chapter 3 Development Tools for Expert Systems.............................................. 35 3.1 Introduction.................................................................................................... 35 3.2 The Technical Design of Expert Systems...................................................... 35 3.2.1 Knowledge Base................................................................................. 35 3.2.2 Working Memory................................................................................ 35 3.2.3 Inference Engine................................................................................. 36 3.2.4 User Interface...................................................................................... 36 3.3 Imperative versus Declarative Programming................................................ 37 3.4 List Processing (LISP)...................................................................................40 3.5 Programming Logic (PROLOG)................................................................... 41 3.5.1 PROLOG Facts................................................................................... 41 3.5.2 PROLOG Rules................................................................................... 42 3.6 National Aeronautics and Space Administration’s (NASA’s) Alternative — C Language Integrated Production System (CLIPS)............. 43 3.6.1 CLIPS Facts........................................................................................44 3.6.2 CLIPS Rules....................................................................................... 45 3.7 Java-Based Expert Systems — JESS............................................................. 47 3.8 Rule Engines — JBoss Rules......................................................................... 48 3.9 Languages for Knowledge Representation.................................................... 49 3.9.1 Classification of Individuals and Concepts (CLASSIC)..................... 50 3.9.2 Knowledge Machine........................................................................... 51 3.10 Advanced Development Tools........................................................................ 53 3.10.1 XpertRule............................................................................................ 55 3.10.2 Rule Interpreter (RI)........................................................................... 56 3.11 Concise Summary.......................................................................................... 57 References................................................................................................................. 58 Chapter 4 Dealing with Chemical Information................................................... 61 4.1 Introduction.................................................................................................... 61 4.2 Structure Representation................................................................................ 61 4.2.1 Connection Tables (CTs)..................................................................... 61 4.2.2 Connectivity Matrices......................................................................... 62 4.2.3 Linear Notations................................................................................. 63 4.2.4 Simplified Molecular Input Line Entry Specification (SMILES)....... 63 4.2.5 SMILES Arbitrary Target Specification (SMARTS).........................64 4.3 Searching for Chemical Structures................................................................64 4.3.1 Identity Search versus Substructure Search.......................................64 4.3.2 Isomorphism Algorithms.................................................................... 65 4.3.3 Prescreening........................................................................................66 4.3.4 Hash Coding.......................................................................................66 4.3.5 Stereospecific Search.......................................................................... 67 4.3.6 Tautomer Search................................................................................. 67 4.3.7 Specifying a Query Structure............................................................. 68 4.4 Describing Molecules.................................................................................... 69 4.4.1 Basic Requirements for Molecular Descriptors.................................. 70 4.4.1.1 Independency of Atom Labeling........................................... 71
5323X.indb 8
11/13/07 2:08:39 PM
Contents
4.5
4.6
4.7
4.8
4.9
5323X.indb 9
ix
4.4.1.2 Rotational/Translational Invariance...................................... 71 4.4.1.3 Unambiguous Algorithmically Computable Definition........ 71 4.4.1.4 Range of Values..................................................................... 71 4.4.2 Desired Properties of Molecular Descriptors..................................... 72 4.4.2.1 Reversible Encoding.............................................................. 73 4.4.3 Approaches for Molecular Descriptors............................................... 73 4.4.4 Constitutional Descriptors.................................................................. 73 4.4.5 Topological Descriptors...................................................................... 74 4.4.6 Topological Autocorrelation Vectors.................................................. 74 4.4.7 Fragment-Based Coding..................................................................... 75 4.4.8 3D Molecular Descriptors................................................................... 76 4.4.9 3D Molecular Representation Based on Electron Diffraction............ 77 4.4.10 Radial Distribution Functions............................................................. 77 4.4.11 Finding the Appropriate Descriptor................................................... 78 Descriptive Statistics...................................................................................... 79 4.5.1 Basic Terms......................................................................................... 79 4.5.1.1 Standard Deviation (SD)....................................................... 79 4.5.1.2 Variance................................................................................ 79 4.5.1.3 Covariance.............................................................................80 4.5.1.4 Covariance Matrix................................................................80 4.5.1.5 Eigenvalues and Eigenvectors...............................................80 4.5.2 Measures of Similarity....................................................................... 81 4.5.3 Skewness and Kurtosis....................................................................... 83 4.5.4 Limitations of Regression................................................................... 85 4.5.5 Conclusions for Investigations of Descriptors.................................... 86 Capturing Relationships — Principal Components....................................... 87 4.6.1 Principal Component Analysis (PCA)................................................ 87 4.6.1.1 Centering the Data................................................................ 89 4.6.1.2 Calculating the Covariance Matrix....................................... 89 4.6.2 Singular Value Decomposition (SVD)................................................ 91 4.6.3 Factor Analysis...................................................................................94 Transforming Descriptors.............................................................................. 95 4.7.1 Fourier Transform............................................................................... 95 4.7.2 Hadamard Transform..........................................................................96 4.7.3 Wavelet Transform..............................................................................96 4.7.4 Discrete Wavelet Transform...............................................................97 4.7.5 Daubechies Wavelets.......................................................................... 98 4.7.6 The Fast Wavelet Transform...............................................................99 Learning from Nature — Artificial Neural Networks................................. 102 4.8.1 Artificial Neural Networks in a Nutshell.......................................... 103 4.8.2 Kohonen Neural Networks — The Classifiers................................. 105 4.8.3 Counterpropagation (CPG) Neural Networks — . The Predictors................................................................................... 107 4.8.4 The Tasks: Classification and Modeling........................................... 109 Genetic Algorithms (GAs)........................................................................... 110
11/13/07 2:08:39 PM
Expert Systems in Chemistry Research
4.10 Concise Summary........................................................................................ 112 References............................................................................................................... 115 Chapter 5 Applying Molecular Descriptors....................................................... 119 5.1 5.2
5.3
5.4
5.5
5.6 5.7
5.8 5.9 5.10 5.11 5.12 5.13 5.14
5323X.indb 10
Introduction.................................................................................................. 119 Radial Distribution Functions (RDFs)......................................................... 119 5.2.1 Radial Distribution Function............................................................ 119 5.2.2 Smoothing and Resolution................................................................ 120 5.2.3 Resolution and Probability................................................................ 122 Making Things Comparable — Postprocessing of RDF Descriptors......... 123 5.3.1 Weighting.......................................................................................... 123 5.3.2 Normalization................................................................................... 124 5.3.3 Remark on Linear Scaling................................................................ 124 Adding Properties — Property-Weighted Functions................................... 125 5.4.1 Static Atomic Properties................................................................... 125 5.4.2 Dynamic Atomic Properties............................................................. 126 5.4.3 Property Products versus Averaged Properties................................ 126 Describing Patterns...................................................................................... 128 5.5.1 Distance Patterns.............................................................................. 129 5.5.2 Frequency Patterns............................................................................ 129 5.5.3 Binary Patterns................................................................................. 130 5.5.4 Aromatic Patterns............................................................................. 130 5.5.5 Pattern Repetition............................................................................. 130 5.5.6 Symmetry Effects............................................................................. 130 5.5.7 Pattern Matching with Binary Patterns............................................ 131 From the View of an Atom — Local and Restricted RDF Descriptors...... 131 5.6.1 Local RDF Descriptors..................................................................... 132 5.6.2 Atom-Specific RDF Descriptors....................................................... 132 Straight or Detour — Distance Function Types.......................................... 133 5.7.1 Cartesian RDF.................................................................................. 133 5.7.2 Bond-Path RDF................................................................................. 133 5.7.3 Topological Path RDF....................................................................... 134 Constitution and Conformation.................................................................... 135 Constitution and Molecular Descriptors...................................................... 136 Constitution and Local Descriptors............................................................. 139 Constitution and Conformation in Statistical Evaluations........................... 140 Extending the Dimension — Multidimensional Function Types................ 145 Emphasizing the Essential — Wavelet Transforms..................................... 147 5.13.1 Single-Level Transforms................................................................... 150 5.13.2 Wavelet-Compressed Descriptors..................................................... 151 A Tool for Generation and Evaluation of RDF Descriptors — ARC.......... 151 5.14.1 Loading Structure Information......................................................... 153 5.14.2 The Default Code Settings................................................................ 153 5.14.3 Calculation and Investigation of a Single Descriptor....................... 154 5.14.4 Calculation and Investigation of Multiple Descriptor Sets............... 155 5.14.5 Binary Comparison........................................................................... 155
11/13/07 2:08:39 PM
Contents
xi
5.14.6 Correlation Matrices......................................................................... 155 5.14.7 Training a Neural Network............................................................... 155 5.14.8 Investigation of Trained Network..................................................... 157 5.14.9 Prediction and Classification for a Test Set...................................... 157 5.15 Synopsis....................................................................................................... 157 5.15.1 Similarity and Diversity of Molecules.............................................. 162 5.15.2 Structure and Substructure Search................................................... 162 5.15.3 Structure–Property Relationships..................................................... 162 5.15.4 Structure–Activity Relationships...................................................... 162 5.15.5 Structure–Spectrum Relationships................................................... 162 5.16 Concise Summary........................................................................................ 163 References............................................................................................................... 165 Chapter 6 Expert Systems in Fundamental Chemistry...................................... 167 6.1 6.2
6.3 6.4 6.5
6.6
6.7
5323X.indb 11
Introduction.................................................................................................. 167 How It Began — The DENDRAL Project.................................................. 167 6.2.1 The Generator — CONGEN............................................................. 168 6.2.2 The Constructor — PLANNER....................................................... 168 6.2.3 The Testing — PREDICTOR........................................................... 169 6.2.4 Other DENDRAL Programs............................................................ 171 A Forerunner in Medical Diagnostics......................................................... 171 Early Approaches in Spectroscopy.............................................................. 175 6.4.1 Early Approaches in Vibrational Spectroscopy................................ 176 6.4.2 Artificial Neural Networks for Spectrum Interpretation.................. 177 Creating Missing Information — Infrared Spectrum Simulation............... 178 6.5.1 Spectrum Representation.................................................................. 178 6.5.2 Compression with Fast Fourier Transform....................................... 179 6.5.3 Compression with Fast Hadamard Transform.................................. 179 From the Spectrum to the Structure — Structure Prediction...................... 179 6.6.1 The Database Approach.................................................................... 181 6.6.2 Selection of Training Data................................................................ 181 6.6.3 Outline of the Method....................................................................... 182 6.6.3.1 Preprocessing of Spectrum Information............................. 182 6.6.3.2 Preprocessing of Structure Information.............................. 182 6.6.3.3 Generation of a Descriptor Database.................................. 182 6.6.3.4 Training............................................................................... 182 6.6.3.5 Prediction of the Radial Distribution Function (RDF) Descriptor............................................................................ 183 6.6.3.6 Conversion of the RDF Descriptor...................................... 184 6.6.4 Examples for Structure Derivation................................................... 184 6.6.5 The Modeling Approach................................................................... 187 6.6.6 Improvement of the Descriptor......................................................... 188 6.6.7 Database Approach versus Modeling Approach.............................. 189 From Structures to Properties...................................................................... 190 6.7.1 Searching for Similar Molecules in a Data Set................................ 191
11/13/07 2:08:40 PM
xii
Expert Systems in Chemistry Research
6.7.2 Molecular Diversity of Data Sets...................................................... 193 6.7.2.1 Average Descriptor Approach............................................. 194 6.7.2.2 Correlation Approach.......................................................... 194 6.7.3 Prediction of Molecular Polarizability............................................. 199 6.8 Dealing with Localized Information — Nuclear Magnetic Resonance (NMR) Spectroscopy................................................................................... 201 6.8.1 Commercially Available Products.................................................... 201 6.8.2 Local Descriptors for Nuclear Magnetic Resonance Spectroscopy.....................................................................................202 6.8.3 Selecting Descriptors by Evolution...................................................205 6.8.4 Learning Chemical Shifts.................................................................206 6.8.5 Predicting Chemical Shifts...............................................................207 6.9 Applications in Analytical Chemistry.........................................................208 6.9.1 Gamma Spectrum Analysis..............................................................208 6.9.2 Developing Analytical Methods — Thermal Dissociation . of Compounds...................................................................................209 6.9.3 Eliminating the Unnecessary — Supporting Calibration................ 215 6.10 Simulating Biology...................................................................................... 217 6.10.1 Estimation of Biological Activity..................................................... 217 6.10.2 Radioligand Binding Experiments.................................................... 218 6.10.3 Effective and Inhibitory Concentrations........................................... 219 6.10.4 Prediction of Effective Concentrations............................................. 221 6.10.5 Progestagen Derivatives.................................................................... 221 6.10.6 Calcium Agonists.............................................................................. 223 6.10.7 Corticosteroid-Binding Globulin (CBG) Steroids............................224 6.10.8 Mapping a Molecular Surface.......................................................... 226 6.11 Supporting Organic Synthesis..................................................................... 229 6.11.1 Overview of Existing Systems.......................................................... 230 6.11.2 Elaboration of Reactions for Organic Synthesis............................... 232 6.11.3 Kinetic Modeling in EROS............................................................... 233 6.11.4 Rules in EROS.................................................................................. 233 6.11.5 Synthesis Planning — Workbench for the Organization of Data for Chemical Applications (WODCA).............................................. 234 6.12 Concise Summary........................................................................................ 236 References............................................................................................................... 239 Chapter 7 Expert Systems in Other Areas of Chemistry................................... 247 7.1 7.2
5323X.indb 12
Introduction.................................................................................................. 247 Bioinformatics.............................................................................................. 247 7.2.1 Molecular Genetics (MOLGEN)......................................................248 7.2.2 Predicting Toxicology — Deductive Estimation of Risk . from Existing Knowledge (DEREK) for Windows.......................... 249 7.2.3 Predicting Metabolism — Meteor.................................................... 251 7.2.4 Estimating Biological Activity — APEX-3D................................... 251 7.2.5 Identifying Protein Structures..........................................................254
11/13/07 2:08:40 PM
Contents
xiii
7.3
Environmental Chemistry............................................................................ 257 7.3.1 Environmental Assessment — Green Chemistry Expert . System (GCES)................................................................................. 257 7.3.2 Synthetic Methodology Assessment for Reduction Techniques....... 258 7.3.3 Green Synthetic Reactions................................................................ 259 7.3.4 Designing Safer Chemicals...............................................................260 7.3.5 Green Solvents/Reaction Conditions................................................ 261 7.3.6 Green Chemistry References............................................................ 261 7.3.7 Dynamic Emergency Management — Real-Time Expert System (RTXPS)............................................................................... 262 7.3.8 Representing Facts — Descriptors................................................... 262 7.3.9 Changing Facts — Backward-Chaining Rules................................. 263 7.3.10 Triggering Actions — Forward-Chaining Rules.............................. 263 7.3.11 Reasoning — The Inference Engine.................................................264 7.3.12 A Combined Approach for Environmental Management................. 265 7.3.13 Assessing Environmental Impact — EIAxpert................................266 7.4 Geochemistry and Exploration.................................................................... 267 7.4.1 Exploration........................................................................................ 267 7.4.2 Geochemistry.................................................................................... 268 7.4.3 X-Ray Phase Analysis....................................................................... 268 7.5 Engineering.................................................................................................. 269 7.5.1 Monitoring of Space-Based Systems — Thermal Expert System (TEXSYS)......................................................................................... 269 7.5.2 Chemical Equilibrium of Complex Mixtures — CEA..................... 270 7.6 Concise Summary........................................................................................ 271 References............................................................................................................... 274 Chapter 8 Expert Systems in the Laboratory Environment............................... 277 8.1 8.2
8.3
5323X.indb 13
Introduction.................................................................................................. 277 Regulations................................................................................................... 277 8.2.1 Good Laboratory Practices............................................................... 278 8.2.1.1 Resources, Organization, and Personnel............................. 278 8.2.1.2 Rules, Protocols, and Written Procedures.......................... 278 8.2.1.3 Characterization.................................................................. 278 8.2.1.4 Documentation.................................................................... 278 8.2.1.5 Quality Assurance............................................................... 279 8.2.2 Good Automated Laboratory Practice (GALP)................................ 279 8.2.3 Electronic Records and Electronic Signatures (21 CFR Part 11).....280 The Software Development Process............................................................ 281 8.3.1 From the Requirements to the Implementation................................ 282 8.3.1.1 Analyzing the Requirements............................................... 282 8.3.1.2 Specifying What Has to Be Done....................................... 282 8.3.1.3 Defining the Software Architecture.................................... 282 8.3.1.4 Programming...................................................................... 282 8.3.1.5 Testing the Outcome........................................................... 283
11/13/07 2:08:40 PM
xiv
Expert Systems in Chemistry Research
8.3.1.6 Documenting the Software.................................................. 283 8.3.1.7 Supporting the User............................................................. 283 8.3.1.8 Maintaining the Software................................................... 283 8.3.2 The Life Cycle of Software............................................................... 283 8.4 Knowledge Management.............................................................................. 287 8.4.1 General Considerations..................................................................... 287 8.4.2 The Role of a Knowledge Management System (KMS)................... 288 8.4.3 Architecture...................................................................................... 289 8.4.4 The Knowledge Quality Management Team....................................290 8.5 Data Warehousing........................................................................................290 8.6 The Basis — Scientific Data Management Systems.................................... 293 8.7 Managing Samples — Laboratory Information Management Systems (LIMS)......................................................................................................... 295 8.7.1 LIMS Characteristics........................................................................ 296 8.7.2 Why Use a LIMS?............................................................................ 297 8.7.3 Compliance and Quality Assurance (QA)........................................ 297 8.7.4 The Basic LIMS................................................................................ 298 8.7.5 A Functional Model.......................................................................... 298 8.7.5.1 Sample Tracking.................................................................. 298 8.7.5.2 Sample Analysis.................................................................. 299 8.7.5.3 Sample Organization........................................................... 299 8.7.6 Planning System............................................................................... 299 8.7.7 The Controlling System....................................................................300 8.7.8 The Assurance System......................................................................300 8.7.9 What Else Can We Find in a LIMS?................................................ 301 8.7.9.1 Automatic Test Programs.................................................... 301 8.7.9.2 Off-Line Client.................................................................... 301 8.7.9.3 Stability Management......................................................... 301 8.7.9.4 Reference Substance Module..............................................302 8.7.9.5 Recipe Administration........................................................302 8.8 Tracking Workflows — Workflow Management Systems...........................302 8.8.1 Requirements.................................................................................... 303 8.8.2 The Lord of the Runs........................................................................ 303 8.8.3 Links and Logistics...........................................................................304 8.8.4 Supervisor and Auditor.....................................................................304 8.8.5 Interfacing......................................................................................... 305 8.9 Scientific Documentation — Electronic Laboratory Notebooks (ELNs).... 305 8.9.1 The Electronic Scientific Document.................................................307 8.9.2 Scientific Document Templates........................................................309 8.9.3 Reporting with ELNs........................................................................ 310 8.9.4 Optional Tools in ELNs.................................................................... 310 8.10 Scientific Workspaces.................................................................................. 312 8.10.1 Scientific Workspace Managers........................................................ 313 8.10.2 Navigation and Organization in a Scientific Workspace.................. 315 8.10.3 Using Metadata Effectively.............................................................. 315
5323X.indb 14
11/13/07 2:08:41 PM
Contents
8.11
8.12 8.13
8.14
8.15
8.16
5323X.indb 15
xv
8.10.4 Working in Personal Mode............................................................... 319 8.10.5 Differences of Electronic Scientific Documents.............................. 319 Interoperability and Interfacing................................................................... 320 8.11.1 eXtensible Markup Language (XML)-Based Technologies............. 320 8.11.1.1 Simple Object Access Protocol (SOAP).............................. 321 8.11.1.2 Universal Description, Discovery, and Integration (UDDI)................................................................................ 321 8.11.1.3 Web Services Description Language (WSDL).................... 321 8.11.2 Component Object Model (COM) Technologies.............................. 321 8.11.3 Connecting Instruments — Interface Port Solutions........................ 322 8.11.4 Connecting Serial Devices................................................................ 322 8.11.5 Developing Your Own Connectivity — Software . Development Kits (SDKs)................................................................. 324 8.11.6 Capturing Data — Intelligent Agents............................................... 325 8.11.7 The Inbox Concept............................................................................ 327 Access Rights and Administration.............................................................. 328 Electronic Signatures, Audit Trails, and IP Protection................................ 329 8.13.1 Signature Workflow.......................................................................... 329 8.13.2 Event Messaging............................................................................... 331 8.13.3 Audit Trails and IP Protection.......................................................... 331 8.13.4 Hashing Data.................................................................................... 331 8.13.5 Public Key Cryptography................................................................. 332 8.13.5.1 Secret Key Cryptography.................................................... 333 8.13.5.2 Public Key Cryptography.................................................... 333 Approaches for Search and Reuse of Data and Information....................... 333 8.14.1 Searching for Standard Data............................................................. 334 8.14.2 Searching with Data Cartridges........................................................ 334 8.14.3 Mining for Data................................................................................ 335 8.14.4 The Outline of a Data Mining Service for Chemistry...................... 336 8.14.4.1 Search and Processing of Raw Data................................... 336 8.14.4.2 Calculation of Descriptors.................................................. 337 8.14.4.3 Analysis by Statistical Methods.......................................... 337 8.14.4.4 Analysis by Artificial Neural Networks.............................. 337 8.14.4.5 Optimization by Genetic Algorithms.................................. 338 8.14.4.6 Data Storage........................................................................ 338 8.14.4.7 Expert Systems.................................................................... 338 A Bioinformatics LIMS Approach.............................................................. 338 8.15.1 Managing Biotransformation Data................................................... 339 8.15.2 Describing Pathways.........................................................................340 8.15.3 Comparing Pathways........................................................................ 342 8.15.4 Visualizing Biotransformation Studies............................................. 343 8.15.5 Storage of Biotransformation Data...................................................344 Handling Process Deviations.......................................................................344 8.16.1 Covered Business Processes............................................................. 345 8.16.2 Exception Recording.........................................................................346 8.16.2.1 Basic Information Entry......................................................346 8.16.2.2 Risk Assessment..................................................................346
11/13/07 2:08:41 PM
xvi
Expert Systems in Chemistry Research
8.16.2.3 Cause Analysis.................................................................... 347 8.16.2.4 Corrective Actions............................................................... 347 8.16.2.5 Efficiency Checks................................................................348 8.16.3 Complaints Management..................................................................348 8.16.4 Approaches for Expert Systems........................................................ 349 8.17 Rule-Based Verification of User Input......................................................... 350 8.17.1 Creating User Dialogues................................................................... 350 8.17.2 User Interface Designer (UID)......................................................... 351 8.17.3 The Final Step — Rule Generation.................................................. 354 8.18 Concise Summary........................................................................................ 354 References............................................................................................................... 358 Chapter 9 Outlook.............................................................................................. 361 9.1 9.2 9.3
Introduction.................................................................................................. 361 Attempting a Definition............................................................................... 361 Some Critical Considerations....................................................................... 362 9.3.1 The Comprehension Factor............................................................... 363 9.3.2 The Resistance Factor....................................................................... 363 9.3.3 The Educational Factor..................................................................... 363 9.3.4 The Usability Factor.........................................................................364 9.3.5 The Commercial Factor.................................................................... 365 9.4 Looking Forward......................................................................................... 365 Reference................................................................................................................ 366 Index....................................................................................................................... 367
5323X.indb 16
11/13/07 2:08:41 PM
Preface Sitting at the breakfast room of my hotel I thought — and not for the first time — about this term expert. Would I consider myself a specialist for jam, just because I eat it every morning at breakfast? And then — why not? Isn’t a consumer a specialist because of his experience with consuming products? Certainly, he would not be considered an expert for the raw materials, the production process, or the quality control in jam production. Still, he is the consumer, so he knows the most about consuming jam. Whatever constitutes an expert, one of the fascinating topics of computer science arises from the question of how to take advantage of the expert’s knowledge in a computer program. Wouldn’t it be great if we could transfer some of the expert’s knowledge and reasoning to a computer program and to use this program for education and problem solving? If we continue thinking in this direction, we encounter a serious problem: We all know that human reasoning and decision making are a result of knowledge, experience, and intuition. How can such a thing like intuition be expressed in logical terms? The answer is that it cannot. In fact, knowledge and reasoning cannot really be expressed in static terms, since it is a result of a complex combination of all three properties. However, computer software is able to store facts. If we were able to describe the relationships between facts and the more complex topics of knowledge, experience, and intuition, we could imagine software that does reasoning and makes decisions. At the beginning of a lecture about artificial intelligence at the university, I explained to my students that this topic is basically simple, since it has to do with our perception of the world rather than the sometimes complex computational point of view. One of them looked at me and asked, “Why then do I have to learn it here?” The answer is, “Because simpler things may be much more difficult to understand due to their general nature.” This is the very crux of expert systems: On one the hand, we have a somewhat complex logic that we have to develop and encode in a computer program, to make things easier for the expert sitting in front of the screen; on the other hand, the more generalized context of the expert might be even harder to understand than the program running in the background. And, there is still a big gap between the expert and the expert system. During my Internet research for this book I stumbled across a nice phrase in a presentation from Joy Scaria from the Biological Sciences Group of the Birla Institute of Technology and Science in Pilani, India. In an introductionary slide, she stated, “Bioinformatics is … complicating biology with introducing algorithms, scripts, statistics, and confusing softwares so that no one understands it anymore.…” Sometimes, I think, this applies to expert systems and the underlying field of artificial intelligence, as well. We tend too often to complicate things rather than to simplify them. Consequently, one of my goals in writing this book was to simplify things as well as possible; unfortunately, I did not succeed entirely. xvii
5323X.indb 17
11/13/07 2:08:41 PM
xviii
Expert Systems in Chemistry Research
It lies in the nature of the topic that the mathematical, information technology, and regulative aspects have to be formulated in a more complex manner; whereas conceptual aspects could be described in a more entertaining fashion. The book shall finally be a good mixture of scientific literature and a captivating novel. There is nothing else than to wish an exciting journey into chemistry’s future. Markus C. Hemmer Bonn, Germany
5323X.indb 18
11/13/07 2:08:42 PM
Acknowledgments This book is finally a result of a series of discussions and contributions around the scientific idea of expert systems, and particularly around simplifying the matters. A series of people contributed to this goal in one way or the other, and I would like to express my gratitude to them: Dr. Joao Aires de Sousa (Department of Chemistry, New University of Lisbon, Caparica, Portugal), Dr. Jürgen Angerer (Institute and Outpatient Clinic of Occupational, Social and Environmental Medicine, University of Erlangen, Germany), Ulrike Burkard (Institute for Didactics in Physics, University of Bremen, Germany), Dr. Roberta Bursi (N.V. Organon, The Netherlands), Dr. Antony N. Davies (Division of Chemistry and Forensic Science, University of Glamorgan, United Kingdom), Dr. Thomas Engel (Chemical Computing Group AG, Köln, Germany), Dr. Thorsten Fröhlich (Waters GmbH, Frechen), Dr. Johann Gasteiger (Computer Chemistry Center, University of Erlangen, Germany), Dr. Wolfgang Graf zu Castell (Institute of Biomathematics and Biometrics, GSF, Neuherberg, Germany), Alexander von Homeyer (Computer Chemistry Center, University of Erlangen, Germany), Dr. Ulrich Jordis (Institute for Applied Synthesis Chemistry, University of Vienna, Austria), Michael McBrian (Advanced Chemistry Development, Inc., Toronto, Canada), Dr. Reinhard Neudert (Wiley-VCH, Weinheim, Germany), Dr. Livia Sangeorzan (Department of Computer Science, University Transilvania of Brasov, Romania), Dr. Thomas Sauer (Institute of Mathematics, University of Erlangen, Germany), Dr. Axel Schunk (Institute for Didactics of Chemistry, University of Frankfurt Main, Germany), Dr. Christoph Schwab (Molecular Networks GmbH, Erlangen, Germany), Dr. Valentin Steinhauer (Danet Group, Darmstadt, Germany), Dr. Jürgen Sühnel (Bioinformatics Group, Institute of Molecular Biology, Jena, Germany), Dr. Lothar Terfloth (Computer Chemistry Center, University of Erlangen, Germany), and Dr. Heiner Tobschall (Department of Applied Geology, University of Erlangen, Germany). Parts of the research in this work were supported by the German Federal Ministry of Education and Research (BMFT), the German National Research and Education Network (DFN), the German Academic Exchange Service (DAAD), the National Cancer Institute (U.S. National Institutes of Health), and Waters Corporation, Milford, Massachusetts.
xix
5323X.indb 19
11/13/07 2:08:42 PM
5323X.indb 20
11/13/07 2:08:42 PM
Trademark Information Apache® is a registered trademark of Apache Software Foundation. CA® and CAS® are registered trademarks of the American Chemical Society. Citrix® is a registered trademark of Citrix Systems, Inc. Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin®, and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc. CleverPath® and Aion® are registered trademarks of Computer Associates International, Inc., New York. Contergan® is registered trademark of Grünenthal GmbH, Aachen, Germany. CORINA® and WODCA® are registered trademarks of Molecular Networks GmbH, Erlangen, Germany. EXSYS® and EXSYS CORVID® are registered trademarks of EXSYS Inc., Albuquerque, New Mexico. HTML®, XML®, XHTML®, and W3C® are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology. HyperChem® is a registered trademark of Hypercube, Inc. IBM®, DB2®, OS/2®, Parallel Sysplex®, and WebSphere® are trademarks of IBM Corporation in the United States and/or other countries. Java® is a registered trademark of Sun Microsystems, Inc. JavaScript® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape®. JBoss® is a registered trademark of Red Hat, Inc., New Orleans, Louisiana. Jess® is a registered trademark of Sandia National Laboratories. Loom® is a registered trademark of the University of Southern California. MDL® and ISIS® are trademarks of Elsevier MDL. Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint®, ODBC®, OLE®, .NET®, ActiveX®, and SQL Server® are registered trademarks of Microsoft Corporation. MOLGEN® is a registered trademark of Stanford University. MySQL®is a trademark of MySQL AB, Sweden. NUTS® is a registered trademark of Acorn NMR, Inc. Oracle® is a registered trademark of Oracle Corporation. SAP® is a registered trademark of SAP AG, Germany. SMILESTM is a trademark and SMARTS® is a registered trademark of Daylight Chemical Information Systems Inc., Aliso Viejo, California. Sun®, Sun Microsystems®, Solaris®, Java®, JavaServer Web Development Kit®, and JavaServer Pages® are trademarks or registered trademarks of Sun Microsystems, Inc. UNIX®, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. UNIX®is a registered trademark of the Open Group. xxi
5323X.indb 21
11/13/07 2:08:42 PM
xxii
Expert Systems in Chemistry Research
VAX® and VMS® are registered trademarks of Digital Equipment Corporation, Maynard, Massachusetts. V-Modell® is a registered trademark of Bundesministerium des Innern (BMI), Germany. All other product names mentioned in this book are trademarks of their respective owners.
5323X.indb 22
11/13/07 2:08:42 PM
1
Introduction
1.1 Introduction Allen Newell — a researcher in computer science and cognitive psychology in the School of Computer Science at Carnegie Mellon University — was involved in the design of some of the earliest developments in artificial intelligence. In 1976 he presented a speech at Carnegie Mellon University that was later published in his essay “Fairy Tales,” from which the following statement was taken [1]: Exactly what the computer provides is the ability not to be rigid and unthinking but, rather, to behave conditionally. That is what it means to apply knowledge to action: It means to let the action taken reflect knowledge of the situation, to be sometimes this way, sometimes that, as appropriate.
What is interesting about this statement is that it apparently contradicts our perception of computers: Computers usually have the stigma of being not as reasonable and adaptive as the human brain. However, if we think about what a human decision characterizes, we find that exactly what Newell described as behaving conditionally is an ultimate basis for reasoning, evaluation, and assessment. A second fundamental idea behind this phrase is the concept of applying knowledge to action. Although theoretical research is important for modeling a scientific basis, the application of scientific ideas in the best sense is what finally leads to advancement, a fundamental concept of evolution on Earth. Expert systems particularly use conditional methodologies and are designed to apply knowledge to practical problems. Knowledge and experience play a major role in the handling of scientific information. Computers are indispensable tools for processing and retrieval of the huge amounts of laboratory data, and expert systems can aid an expert in making decisions about a certain problem. Human experts rely on experience as well as on knowledge. Experience can be regarded as a specialized kind of knowledge created by a complex interaction of rules and decisions. Instead of representing knowledge in a static way, rule-based systems represent knowledge in terms of rules that lead to conclusions. Computer software will never be able to replace the human expert in interpreting information. However, expert systems can assist the human expert by organizing information and by making estimations and predictions.
1.2 What We Are Talking About The patient reader will find out that this book is unconventional in several aspects. This is partially due to the fact that expert systems are an unconventional topic and require having from time to time a different point of view than one would expect at the beginning.
5323X.indb 1
11/13/07 2:08:42 PM
Expert Systems in Chemistry Research
This book is primarily written for scientists — particularly chemists and biochemists — and the thread leading through this book is the application of expert systems and similar software in chemistry, biochemistry, and related research areas. I know that scientists expect an overview on existing research before the book or article starts with the news. We will follow this approach, however, only if necessary for the context of the new content that is presented. The information technology aspect is described to an extent that allows a scientist to understand the principles rather than instructions given on how to develop expert systems. Nevertheless, we will deal with a series of mathematical aspects from cheminformatics and chemometrics that are required to represent chemical information in a computer. The mathematical techniques are introduced at a reasonable level and provide the background to understand the examples of expert systems and their application that are introduced afterward. The expert systems introduced are selected to cover different application areas. Even though the list is far from being complete, we will find some historical systems and more recent noncommercial and commercial software. Since applying expert systems requires integrating them with other software, we will finally have a look at the laboratory software environment. This final topic will give the reader an idea about the challenges occuring with the practical implementation of such systems in the research laboratories and, hopefully, will support his own ideas for developing or applying expert systems. The following gives a summary on what to find in this book. Later in this chapter, we will start thinking about some initial aspects of intelligence to get a framework for what we want to address. Chapter 2 gives an overview of the ideas and concepts underlying the term expert systems. This is a necessary basis for understanding the approaches that are subsequently described. At this point, we will focus on two domains: the conceptional background and the scientific methodologies that support the concepts. Chapter 3 provides a concise summary on technical design, programming paradigms, and development tools for expert systems. Chapter 4 covers technologies for representing and processing chemical information in a computer as well as a summary of supporting technologies that typically appear in expert systems. Chapter 5 deals with a particular method for describing molecules in a computer, which acts as representative for molecular descriptors, a topic of particular importance when dealing with chemistry in software. It introduces applications for the different molecular descriptors depicted in the previous chapter and closes with the introduction of a software package developed for the investigations. Chapter 6 covers experts systems and their application in chemistry research, starting with the historical forerunners of these systems and then focusing on different application areas. Chapter 7 comprises expert system applications in areas related to chemistry research, such as bioinformatics and industrial areas. Chapter 8 deals with the software environment in the laboratory, gives an overview on typical software packages, and describes requirements that have to be taken into account when using expert systems in conjunction with the existing laboratory software. Chapter 9 closes with a generic definition and a critical assessment of expert systems and an outlook.
5323X.indb 2
11/13/07 2:08:43 PM
Introduction
1.3 The Concise Summary A book about expert systems in chemistry has to bring two disciplines together: chemistry and information technology (IT). Since the chemists might be unfamiliar with the considerable amount of IT aspects for this topic (and vice versa), I added an additional section to each chapter: the “Concise Summary.” The concise summary gives the reader a chance to revisit the important terms and concepts described in the context of the respective chapter. It might also be helpful to reexamine the new concepts and to make sure that the new information has been appropriately digested before starting with the next chapter.
1.4 Some Initial Thoughts Before we go into some detail for the theoretical background for expert systems, we should take some time to get familiar with some of the concepts of dealing with information in a computer. I decided to use one of the newer developments in chemistry research to focus on a problem that is essential for many aspects in computational chemistry: the topic of molecular descriptors. Molecular descriptors describe a molecule in a way that is appropriate for processing in a computer program. Like the term suggests, a molecular descriptor describes certain features of a molecule. We could now start focusing on the details and algorithms of descriptors. We will do this later, but let us try another approach here. Let us take a step back and have a more general view of the meaning of the term descriptor. A typical example is a spectrum; spectra describe specific features or the behavior of a molecule under defined circumstances. An infrared spectrum describes the movement of a molecule under infrared radiation, whereas a nuclear magnetic resonance spectrum describes how a molecule’s nuclei behave when the surrounding magnetic field changes. All of these descriptors have something in common: They cover just a specific portion of characteristics of a molecule, and that is the reason why so many different descriptors exist. One of the approaches for using molecular descriptors is to compare an artificial descriptor with one evolving from a measurable physical or chemical interaction. As an example, we are able to record an infrared spectrum for a compound. If we use an artificial descriptor, which is calculated from properties that are essential for the thermal movement of a molecule, and we compare this artificial descriptor with a measured infrared spectrum, we are able to find a correlation between these descriptors. Finding a correlation allows us — in the best of all worlds — to convert one descriptor into the other. In fact, it would be possible to derive an infrared spectrum from an artificial descriptor. What good is it to relate an artificial and measured descriptor? At this point, we have to think about the application of measured descriptors. Infrared spectra are usually used to confirm the structure of a compound: If an infrared spectrum for an unknown compound is measured and we find a similar spectrum in a database, the probability is high that the measured unknown compound is identical to the database compound. Here is an important fact for this approach: We do not need to know the details about infrared spectroscopy; the simple fact that it is similar assumes the similarity of the unknown compound. This approach is typical for artificial
5323X.indb 3
11/13/07 2:08:43 PM
Expert Systems in Chemistry Research
intelligence systems: We are not looking at the details, but we try to make use of the bigger picture to solve our problem. Expert systems use this approach for problem solving by separating technical aspects clearly from the domain aspects. Consequently, the domain experts are seldom development experts who create an expert system. However, developers of expert systems are able to create a reusable shell that allows domain experts to enter knowledge in an understandable fashion. Facts, like descriptors, are seldom directly interpretable. Looking again at an infrared spectrum, it might be possible to assign each peak to a certain movement in a molecule; however, these facts do not really help to solve the problem of structure elucidation. Chemists know that a complete structure cannot be derived from an infrared spectrum. This is simply because thermal movement and constitution — even they are related to some extent — are two different things. But there is an interesting detail: Nearly all structures produce their unique spectrum. Should an infrared spectrum be specific enough to provide all information to derive a structure? Structure elucidation with an infrared spectrum cannot be performed simply by interpreting the individual peaks one after another. As a result, we are in need of some mechanism that is able to find patterns in infrared spectra or to use the entire spectrum as pattern, like it is done in the conventional infrared software to identify a compound. If we are not able to obtain a structure from the details, we can either try to use patterns or even the entire data set for comparison. This is how infrared spectra are typically used: as pattern for structure elucidation. The best pattern-matching machine currently available is the human brain. How else can we explain that we recognize a face by looking at someone from behind? Imagine a typical situation: You walk in town along a street heading for your favorite restaurant to have your lunch break. The street is full of people, and for a short moment you see a face of someone smiling at you. The person passes by; you ponder for a moment and then turn your head to look at the crowd of people from behind. In almost any case it would be easy for you to recognize the person from the back — quite normal, isn’t it? Now let us think about software that should be able to do exactly the same: recognize a person from the back while having a picture from the face of the person. Very few techniques would be able to handle such a task in a short time frame. No conventional software would succeed. Additionally supported by parallel processing, it is not much of a problem for a brain to predict the right person within a few seconds. The brain works more effectively because it is not working with all details. It does not memorize the detail structure of the iris, nor does it count the number of hairs or look at each hair individually. The brain is working with more or less complex patterns instead; the medium color of all hairs is categorized in very simple classes, like blond, brown, black, or red; the shape of the clothes is stored as, for example, a shirt, jacket, or coat. Patterns seem to be very important for analyzing complex problems, and expert systems particularly take advantage of those and other concepts to deal with experimental data. A typical example is artificial neural networks (ANN), which are complex statistical models that leverage certain mechanisms known from the functioning of the
5323X.indb 4
11/13/07 2:08:43 PM
Introduction
mammal brain for analyzing patterns. One advantage of pattern matching is that we can try to mimic human behavior with the simplest algorithm if we are able to define the pattern appropriately. However, as we all know, our way of describing the world is not based on clear facts. Almost any result we get from our perception underlies certain fuzziness. For instance, gray hair may have different shadings or color tones that can be further classified. The same applies to experimental data: Almost every result from an experiment is unsharp, as chemists know well when they try to classify a peak in a spectrum. Since computer programs are strictly logic systems, which (at first sight) are unable to handle unsharp data, we need to introduce a technology that is capable of generalizing a fact to assign it to a specific value. These so-called fuzzy techniques, which are able to handle not just black and white problems but also the natural gray problems, allow creating systems that support the concept of decision making with fuzzy data. Another aspect of the situation just described is this: In many cases we are not even able to specify the exact criteria for how our brain classifies things. The term elegant, for instance, would be hard to describe with few facts. Still, if we are talking about elegance, we all seem to know what is meant. And here is another interesting point: Although it is easy to define blond hair in a series of parameters (e.g., brightness, saturation, luminance) for an image processing system, we are with the term elegance already in the next level of interpretation. We definitely know what elegance means to us, but it is obvious that there exist different opinions about being elegant. This knowledge has three aspects: • We have knowledge about what elegance means to us. • Each individual has different perceptions and understandings of this term. • There is a common thing about elegance, or else we would not be able to use this term in our conversations. Let us stay a moment with the first aspect: How do we know what elegance is? Obviously this has something to do with the experience that we obtain when we learn to use such term. We apply rules to our decision as to whether something is elegant or not. An interesting question would now be, are we able to express this knowledge in a computer program? Since we used rules to memorize the meaning, it is a straightforward method to define rules in the program. We have seen already that behaving conditionally is one of the strengths of a computer program; in fact, if–then constructions can be found in any programming language and most probably in every software. Simple if–then rules are helpful in describing hard facts. But there are situations where we need to describe facts in the more general context, like in the aforementioned example. A concept of elegance, for instance, might be better described in a system of concepts and relationships. This is one of the available methods we will deal with later. Another important aspect of the situation just described is the action that followed the perception: You saw the person smiling at you and decided to turn around. This decision was derived from a series of perceptions and memorized patterns. The first aspect is the smiling face; your experience tells you that this is a sign of sympathy, which causes you to be more open and also to perceive sympathy for the person.
5323X.indb 5
11/13/07 2:08:44 PM
Expert Systems in Chemistry Research
The second aspect is the appearance, derived by the face, the clothes, and the behavior of the person. Everything that makes a person friendly to you is included in the decision process. You will automatically rank the facts; for instance, the eyes might be more important to you than the hair. If all patterns are evaluated and ranked, you will summarize the outcomes. The next step is to include a certain probability (e.g., maybe the person did not smile at you but at another person) and to account for certain fuzziness (e.g., you still cannot be sure whether there was an intent behind the smile). Finally, you will conclude with a decision as to whether to turn around or not. The inference mechanism is again something that has been trained and is affected by experience: Maybe you will not turn around because you had a bad experience with a similar situation before. External facts are also accounted for (e.g., you do not have the time to talk to the person). All of these influences have to be covered if we want to create a computer system that mimics human reasoning and decision making. An important aspect of what we have done so far is the generalization: We tried to create generic concepts, rules, and relationships to describe the situation and the course of activity of the beholder. Things in chemistry and related sciences are usually a bit simpler than the example previously given. This is due to their systematic nature. Sciences are nothing else than the attempt to create a system around observations. This systematic approach makes things a lot easier. We already have a series of rules in sciences that can be more or less directly transformed into a rule that a computer program can interpret. A Diels-Alder reaction, for instance, is clearly defined by their inventors as a reaction between a diene and a dienophil. We can now create a hierarchy — starting from the generalized view to the detail — to describe a Diels-Alder reaction: Transformation — Reaction — Dienophilic — Diels-Alder The more general the point of view is (the higher the level), the less applicable becomes a rule to describe a detail. On the other hand, a more specific description lets us derive the underlying general aspects. An example is the definition of life. In a specific rule we can define life from our experience with life forms on our planet: consumption, metabolization, and growth are some commonly used criteria for life from a discrete point of view. If we define a rule on the basis of these criteria in a computer program, it would look like this: IF a system consumes AND transforms AND grows THEN it is a life form. Let us query now our software with the concept of fire. Fire also consumes raw material, transforms it into degradation products, and is able to grow. Our software would evaluate that fire is a life form; still it is not understood as life. Obviously, our definition was not correct, and we have to be a bit more specific. Let us include the criterion of reproduction. Still, the software would evaluate fire as life form,
5323X.indb 6
11/13/07 2:08:44 PM
Introduction
since it is able to create offspring. However, now we are running into another issue: Members of some species, such as ant workers, do not reproduce since they are a sterile variety of the species; still, they are considered forms of life. There are now two ways to approach this problem. We can try to include all necessary constraints and exceptions that lead exactly to our expected result: Ant workers are life forms; fire is not. However, it can be foreseen that we will run into other similar situations where we have to add more and more constraints and exceptions, and at some point, we would find that we have an unreasonable amount of criteria. Another approach is to generalize the definition. Probably one of the most general scientific definitions of life can be derived from the essay Chance and Necessity, written by French biochemist Jaques Monod [2]. Monod derives two basic concepts from a biochemical point of view: • Proteins cover controlling, organizational, and energetic aspects of biochemical transformations. • Nucleic acids store the information required by proteins for the transformation. If we focus on the two aspects of control and information, we are able to derive all of the specific requirements mentioned before and are able to distinguish clearly between living and nonliving systems, as in the case of fire. Interestingly, the separation of control and information leads exactly to one of the important differentiators between conventional computer programs and expert systems: the separation between the information pool (i.e., the knowledge base) and the controlling (i.e., executing) environment. This might be one of the underlying reasons why expert systems are able to behave somewhat intelligently. However, other authors are more competent to go deeper into this kind of philosophical discussion; let us return to the more practical aspects. With these initial thoughts, we are prepared to go now into some details of the theory of expert systems. For an expert systems to handle real-world problems, we need to incorporate a series of supporting technologies, in particular to deal with the following: • Describing a fact, property, or a situation; this leads us to the topic of Mathematical Descriptors that are able to represent such information in a computer program. • Finding, defining, and searching for patterns; Pattern Recognition and Pattern Matching are essential technologies supporting searches in large data sets. • Handling natural fuzziness; experimental data underlie certain fuzziness that is addressed by Fuzzy Logic and similar techniques. • Estimating the appropriateness of properties or events; Probability Theory and related approaches let us handle the likelihood of proposed results. • Describing knowledge in a computer system; the topic of Knowledge Representation deals with specifying knowledge rather than static facts. • Deducing from facts and knowledge; Inference Engines allow us to apply existing knowledge to new situations. The following chapter shows how we can approach these requirements.
5323X.indb 7
11/13/07 2:08:44 PM
Expert Systems in Chemistry Research
References
5323X.indb 8
1. Newell, A., Fairy Tales, AI Magazine, 13, 46, 1992. 2. Monod, J., Chance and Necessity: An Essay on the Natural Philosophy of Modern Biology, Alfred A. Knopf, New York, 1971.
11/13/07 2:08:44 PM
2
Basic Concepts of Expert Systems
2.1 What Are Expert Systems? Expert systems are computer programs that are derived from artificial intelligence (AI) research. AI’s scientific goal is to understand intelligence by building computer programs that exhibit intelligent behavior. It is concerned with the concepts and methods of symbolic inference, or reasoning, by a computer and how the knowledge used to make those inferences will be represented inside the machine. However, there is a difference between conventional software algorithms and expert systems. Whereas conventional algorithms have a clearly defined result, an expert system may provide no answer or just one with a certain probability. The methodology used here is heuristic programming, and, depending on the point of view, the terms expert system and knowledge-based or rule-based system are often used synonymously. Expert systems are computer programs that aid an expert in making decisions about a certain problem. Knowledge-based systems can generally be defined as computer systems that store knowledge in the domain of problem solution. Human experts rely on experience as well as on knowledge. This is the reason why problem-solving behavior cannot be performed using simple algorithms. Experience can be regarded as a specialized kind of knowledge created by a complex interaction of rules and decisions. The term expert system is typically used for programs in which the knowledge base contains the knowledge used by human experts, in contrast to knowledge-based systems, which use additional information sources to solve a problem. An expert system is software that typically operates with rules that are evaluated to predict a result for a certain input. For the generation of rules, prior knowledge about the relationship between query and output data is necessary. Inductive learning processes that are typically performed by feeding the system with experimental data of high reliability can establish such a relationship. As a consequence, expert systems pertain to at least a knowledge base that contains predefined knowledge in the domain of problem solution (e.g., assignments of bands in spectra). Many expert systems contain a knowledge base in the form of a decision tree that is constructed from a series of decision nodes connected by branches. For instance, in expert systems developed for the interpretation of vibrational spectra, decision trees are typically used in a sequential manner. Similar to the interpretation of a spectrum Although there is still discussion about what distinguishes expert systems from knowledge-based systems or which term is the correct one, we will drop out of this formality as it is not useful in the context of understanding the principles and will use both terms more or less synonymously.
5323X.indb 9
11/13/07 2:08:44 PM
10
Expert Systems in Chemistry Research
by an expert, decisions can be extended to global or restricted to special problems; a larger tree can include more characteristics of a spectrum and can be considerably more accurate in decision making. By combining other AI techniques, like pattern recognition techniques, fuzzy logic approaches, or artificial neural networks, many problems in chemistry can be solved by expert systems automatically.
2.2 The Conceptual Design of an Expert System Let us think about how we can make a first hypothetical approach to what an expert system would constitute. A standard model in the science of knowledge management is the knowledge pyramid, which basically describes the quantitative and logical relationship among data, information, knowledge, and action (please also refer to Figure 2.1). • Data: These represent the basis of the knowledge pyramid and are a result of measurements, observations, and calculations. They are usually available in a large amount (as the broad basis indicates) and are usually meaningless if they are disconnected from their context. The raw data points of a spectrum are a typical example: These data points are useless without additional data describing the measurement technique, parameters, conditions, and attributes, such as sample identifier. The additional data that describe the context are usually referred to as metadata (i.e., data about data). • Information: If we start to work on data by converting them into a different representation, analyzing or interpreting them, we create information. With our example of a spectrum, we could already see the step of creating a graph from the measured data as a first approach to create information from data. A graph can be interpreted much more efficiently than the raw data points. • Knowledge: The concept of knowledge is a bit different from the previous ones. Knowledge is something that can result from information. The continued interpretation of a spectrum graph (i.e., information) leads to a certain probability that a peak in the graph is related to a particular — usually structural — feature. This observation leads to the experience that the peak relates to a property. By validating this result with experimental methods, we gain the knowledge about this relationship. • Action: The final step in the concept of a knowledge pyramid is applying the knowledge to make decisions and to finally take certain actions. Without applying the knowledge, the entire approach is purely theoretical. Now that we have defined the basic terms, the next logical step would be to think about the transition from data to information, knowledge, and action. Let us select three methods for turning data into information: calculation, analysis, and interpretation. Calculation and analysis can be appropriately performed using algorithms based on linear — or imperative — programming. There is also a series of software algorithms available for automatic interpretation of data; in many cases, a human expert is even required to validate the results. In contrast to that, the transition of
5323X.indb 10
11/13/07 2:08:45 PM
11
Basic Concepts of Expert Systems Transitions
Expert
Intuition
Action Knowledge
Patterns Context
Data
Theory
Software
Information
Analysis Calculation
Probability Fuzziness
Experience Interpretation
Certainty
Reality
Reasoning
Factors
Amount
Figure 2.1 The relationship among data, information, knowledge, and action is described by the knowledge pyramid. The transition from data to information is performed by calculation, analysis, and interpretation performed by computer software, whereas in the transition from information to knowledge, human interpretation, experience, and intuition play a major role. The final step of reasoning leads to actions to be taken. Amount, context, and patterns can be considered as theoretical factors; in contrast to that, real data have to be described by concepts of certainty, probability, and fuzziness.
information to knowledge and action can barely be solved by conventional software products. Here the subject-matter expert (SME, or simply expert) comes into play. For instance, the concept of experience plays a significant role for the interpretation of data and for gaining knowledge from information. After a certain amount of training improves the experience and knowledge, another term evolves: intuition. A spectroscopist might be able to intuitively derive from a look at a spectrum whether a compound type is probable or not. He does this by recognizing a pattern within part of the spectrum or even from the entire spectrum. Finally, a human expert performs reasoning by combining the available information with his experience, his intuition, and his knowledge to make conclusions and to finally take the appropriate actions. How can we approach these issues in a computer program? The first step is to have a look at the factors and constraints that play a role for the transitions. Starting again with data, the first factor we have to take into account is the context. Data outside of their context and without metadata describing their meaning are basically useless. A second factor is the amount. In particular with computers data are produced in such masses that only computer software can handle them appropriately. What is needed here are algorithms that allow us to reduce the amount of data to the important facts that can be further analyzed. One approach for analyzing data is to find patterns, similarities, and correlations. Statistical analysis and certain methods developed in the field of AI, or soft computing, can be easily implemented in computer software to perfom these tasks more or less automatically. However, we have to face the fact that most data come from experimental observations. The results from these observations rarely underlie a linear and straightforward model; experimental data exhibit certain fuzziness, usually as a result of the
5323X.indb 11
11/13/07 2:08:46 PM
12
Expert Systems in Chemistry Research
complexity of the observed system. It is particularly interesting that the concept of probability has gained so much use in a scientific area like quantum physics, which seems at first sight to be an exact science. A quantum physicist has to deal with systems of such complexity that well-defined states are rarely found in observations and probabilities have to be used instead. Probabilities are used to describe the likelihood of events, usually based on the frequency of their appearance. A similar — but nevertheless different — factor is the concept of certainty. Only data, information, and knowledge of a particular certainty are adequate for deriving a decision that finally triggers an action. Taking these factors into account, we can now try to investigate the elementary concepts of expert systems.
2.3 Knowledge and Knowledge Representation The term expert system is often used synonymously in the literature with the term knowledge-based system to point out a more general approach. Knowledge-based systems use not only human expertise but also other available information sources. However, expert or knowledge-based systems are different from other computer information systems. Knowledge engineering is one aspect of information system engineering, but it focuses on the epistemological status of information that is defined as knowledge. Instead of representing knowledge in a static way, rule-based systems represent knowledge in terms of rules that lead to conclusions. A simple rule-based system consists of a set of if–then rules, a collection of facts, and an interpreter controlling the application of the rules by the given facts. Other important knowledge representation techniques are frames and semantic networks [1].
2.3.1 Rules Rule-based programming is one of the most commonly used techniques for developing expert systems. In this programming paradigm, rules are used to represent heuristics, which specify a set of actions to be performed for a given situation. A rule is composed of an if-portion and a then-portion. The if-portion of a rule is a series of patterns that specify the facts, or data, that cause the rule to be applicable. The process of matching facts to patterns is called pattern matching. A simple if–then rule in a login procedure can look like the following: if (userName = “Administrator”) then grantAccess In this case, access is granted only if the user logs in as administrator; the decision is based on a single condition. A combination of two conditions leading to a decision can look like the following: if (username = “Administrator” AND password = “admin”) then grantAccess
5323X.indb 12
11/13/07 2:08:46 PM
Basic Concepts of Expert Systems
13
Here, the AND combines both facts and leads to a positive result only if the first and the second fact is true. By combining more facts, complex rule sets can be set up (the brackets indicate groups of commands): if (username = “Administrator”) then { if (password = “admin”) then { if (systemStatus = “online”) then grantAccess else showMessage (“System is offline”) } else showMessage (“Password is wrong”) } else denyAccess These program statements are hard coded; that is, each rule is placed directly in the program code. This is how reasoning would look like in a conventional program. As mentioned before, an important differentiator between conventional programs and expert systems is the separation of rules from the program logic. A bit more complex example — still developed in a conventional programming language — is shown in the following. This is a C++ code of a function that evaluates whether a compound is aromatic or not, based on the general definition that an aromatic system is characterized by coplanarity, cyclic conjugation, and the Hückel rule (the number of π-electrons is 4n + 2, where n is any positive integer beginning with zero): bool Molecule::AromaticRing (ring) { if (mod((ring.piElectrons – 2) % 4) != 0) return (false); For (int i = 0; i < ring.nAtoms - 2; i++) { if ((ring.coordX[i] != ring.coordX[i+1]) return(false); if ((ring.bondOrder[i,i+1]] == comp.bondOrder[i+1,i+2]) ) return(false); } return(true); } We will not go into all details of this C++ code but will focus on the evaluation part. The function Molecule::AromaticRing requires an input of the ring atoms (ring) and returns a Boolean value (bool) of true or false. The first statement checks whether the Hückel rule applies or not; the system returns false (and leaves the function) if the modulus of π-electrons – 2 divided by 4 is not zero. The second and third
5323X.indb 13
11/13/07 2:08:46 PM
14
Expert Systems in Chemistry Research
evaluation takes place in the within the loop (For statement) that goes over all atoms in the molecule. The second if statement returns false if the x-coordinates of any atom pair are unequal; in this case the system would not be planar. The third if statement evaluates to false if the bond order of neighboring bonds is equal — that is, no cyclic conjugation appears. If the function is never left by any of the return (false) statements, it finally returns true in the last statement. This example shows how a rule could be implemented in a conventional program; however, a rule-based system would typically just include the statement for calling the rule rather than the inherent logic. For instance, consider the following: (rule AromaticRing if (ring aromatic) then (stability = high) ) This rule evaluates whether a ring system is aromatic and, if true, asserts a high stability for the compound without showing how the evaluation is actually achieved. This is typical for so-called declarative programming languages, whereas the first example is written in an imperative language that includes the evaluation logic. Declarative languages are also easier to understand since they are more descriptive. We will have a closer look at these languages later in this book. What we need to know right now is that in an expert system the evaluation of if (ring aromatic) may call the C++ function shown before; however, both codes will reside in separated modules — that is, rules and evaluation logic are separated. This allows the rule base to be changed and extended — using a more descriptive language — without the need for changing the underlying imperative logic, written in a language that is harder to understand. These types of if–then rules are most often used for rule bases; however, there are other forms of knowledge representation in a computer.
2.3.2 Semantic Networks Semantic networks are a specialized form of knowledge representation in a computer. The basis is a formal model describing concepts and their relations in a directed graph; concepts are represented by nodes that are connected by their relations. The term semantic network was proposed by Ross Quillian in his Ph.D. thesis in 1966 and was published a year later [2]. As a linguist, Quillian was looking for a way to represent the meaning of words in an objective manner as psychological model for the architecture of human semantic knowledge. The basic assumption is that the meaning of a word can be represented by a series of verbal associations. A semantic network can simply be constructed by connecting concepts with one or more relational terms. As an example, let us look at the concepts and relations in Figure 2.2. Starting with the concept molecule we can define a relationship to structure with the relational term consists of. Following the route we can relate structure to substructure, and to functional group. Here we introduce the new relation term defines to describe the relationship between functional group and reactivity. As we follow through all cross-references, we build up a complex picture of the concept of molecule and its relations to other concepts, like biological activity.
5323X.indb 14
11/13/07 2:08:46 PM
15
Basic Concepts of Expert Systems Consists of
Molecule
has
Defines
Structure
Molecular Surface
Consists of Defines
Substructure Consists of Functional Group
Defines
Defines
Spectrum
Reactivity
Defines
Biological Activity
Consists of Atom
has
Defines Partial Charge
has Polarizability
Defines
Figure 2.2 A semantic network of a molecule describing the relationship between entities with the relational terms consists of or defines and their attributes with the term has. Following the route from the molecule we can relate structure to substructure, functional group, and atom. The atom entity provides attributes, such as partial charge or polarizability, which finally are part of the definition of biological activity.
The semantic network reflects the way of how human knowledge is structured, and the knowledge of a chemist can be represented by creating such graph of concepts and relationships; getting from a molecule to its biological activity is defined in such a graph. This generic approach can be used for any specific instance of a molecule and allows not only storing concepts and realtionships in an effective manner but also opens the way for deductive reasoning by systematically traversing through the nodes of the graph. Semantic networks have several features that make them particulary useful [3,4]: • They define hierarchy and inheritance between concepts easily in a network format. • Their network structure can be dynamically adapted to new information. • Their abstract relationships represent conclusions and cause–effects and allow for deductive reasoning. • They are an effective and economic approach for storing and retrieving information. Semantic networks represent ontologies, and the inherent knowledge can be retrieved by using automatic reasoning methods [5]. An ontology in computer science is a data model that represents a set of concepts within a domain and the relationships between those concepts. Semantic networks have been successfully applied in various applications for bioinformatics.
5323X.indb 15
11/13/07 2:08:47 PM
16
Expert Systems in Chemistry Research
One of the most well-known ontologies is hosted by the Open Biomedical Ontologies (OBO) consortium and is called the Gene Ontology project. This project provides a vocabulary to describe gene functions and gene product attributes for organisms, as well as mammalian phenotypes and cell types [6–9]. Gene Ontology organizes biological terms into three directed and acyclic graphs that cover cellular component, molecular function, and biological process, each of which consists of concepts connected by is a or part of relationships. Since 2007 the biological process ontology incorporates a complete tree where each concept in the hierarchy has at least one is a relationship to the top node. The following example shows the definition of a concept (called term in the Gene Ontology project): [Term] id: GO:0000079 name: regulation of cyclin-dependent protein kinase activity namespace: biological_process def: “Any process that modulates the frequency, rate or extent of CDK activity.” [GOC:go_curators] synonym: “regulation of CDK activity” EXACT [] is_a: GO:0000074 ! regulation of progression through cell cycle is_a: GO:0045859 ! regulation of protein kinase activity Each term requires a unique identifier (id), a name, as well as at least one is_a relationship to another term. Additional data are optional: in the example, a namespace, its definition, and a synonym. The namespace modifier, for instance, allows the definition of a group in which the term is valid and can also be defined for other modifiers, like the is a relationship. The current project contains more than 20,000 terms for biological processes, cellular components, and molecular functions. The ontologies can be downloaded in different formats including eXtensible Markup Language (XML) and the opensource relational database scheme MySQL. A series of other projects is available from Open Biomedical Ontologies. Among these, the Generic Model Organism Project (GMOD) comprises several organism databases and is intended to develop reusable components suitable for creating new databases for the biology community. The Microarray Gene Expression Data (MGED) project covers microarray data generated by functional genomics and proteomics experiments. The Sequence Ontology (SO) is a part of the Gene Ontology project for developing ontologies for describing biological sequences. An overview of recent applications of semantic networks in bioinformatics is given by Hsing and Cherkasov [10].
2.3.3 Frames Frames are a technique for representing knowledge in a modular way that is ideally suited for object-oriented programming. The term frames was first proposed by Marvin Minsky in the early 1970s [11].
5323X.indb 16
11/13/07 2:08:47 PM
17
Basic Concepts of Expert Systems Frame
Slots
Compound
Applications
Application
Solvent
Risk
Reactant Catalyst
Appearance
Dessicator
Availability Risks
Flammable Toxic Catalyst Dessicator
Figure 2.3 A frame-based representation of a compound. The frame compound includes so-called slots that define attributes and relationships between multiple frames. The example shows how the compound’s slots application and risk are connected to specific slots in the generic frames applications and risks.
Frames represent an object as a group of attributes and relationships (Figure 2.3). Each relationship is stored in a separate slot. The contents of a slot may be data types, functions, or procedures. Frames may be linked to other frames providing inheritance. Frames have the following general form: (frame framename (type frametype) (attributes (slots (slotname slotvalue) (slotname slotvalue) ... ) ) A specific instance of a frame describing a solvent might look like the following: Frame Dimethylformamide Application Solvent Risk Flammable Appearance Clear liquid Availability Catalog Fields named (Application), (Risk), (Appearance), and (Availability) are called slots; they represent the relation between frames. Frames are similar to object-oriented programming; however, there is an important difference between a frame and an
5323X.indb 17
11/13/07 2:08:48 PM
18
Expert Systems in Chemistry Research
object‑oriented approach. In the object-oriented paradigm the objects are constructed in a hierarchical manner that focuses on the data exchange between specific instances of classes. They are constructed dynamically but underlie a procedural approach focusing on messaging between instances. Frames represent a declarative point of view that is more or less static and focuses on the relationships between frames. However, both concepts rely on inheritance — that is, the capability for deriving a frame, or class, from another. Frames are created taking expectations into account that are typical for the human notion of information. For instance, with a chemist the term compound would raise the question for its application rather than for the systematic name. Slots can represent variables of different types (e.g., integer, real, symbol, interval) or functions (e.g., procedure, link) used in basic algorithms. The majority of systems using case-based reasoning, which is described later in this chapter, work with frames.
2.3.4 Advantages of Rules There are several advantages to using rule engines in expert systems. 2.3.4.1 Declarative Language They key advantage of rule engines is the declarative language. Declarative programming allows expressing problems in an easy and understandable way. Rule verification is much easier to read than usual code in a programming language. Since rule systems are capable of solving very complex problems, they provide solutions in a simple manner, thus reducing the complexity of a given problem to a simple set of answers or even just a single one. This simplified representation allows often explaining why a decision was made in a much easier manner than with results from other AI systems like neural networks. By creating domain specific languages that model the problem domain, rules can look very close to natural language. They lend themselves to logic that is understandable to domain experts who may be nontechnical, like auditors. 2.3.4.2 Separation of Business Logic and Data Conventional object-oriented programming approaches tend to couple data with business logic. These entities are clearly separated in rule systems; business logic is much more easily maintained and managed in a separate entity. This is particularly important for environments, where business rules change on a regular basis, which is in fact typical for business areas in the scientific field. Typical examples are the regulations that authorities like the U.S. Food and Drug Administration (FDA) impose on the pharmaceutical industry in drug development. 2.3.4.3 Centralized Knowledge Base Storing business rules separated from data supports the creation of a centralized knowledge base: a single point of authentic knowledge, which can be used at multiple sites and on different occasions, reducing maintenance and validation effort and costs.
5323X.indb 18
11/13/07 2:08:48 PM
Basic Concepts of Expert Systems
19
Auditing can be done more effectively at a single site, and decisions and decision paths are easy to explain since conclusions of the inference engine can be logged. 2.3.4.4 Performance and Scalability Pattern-matching algorithms are usually fast compared with complex statistical models or AI approaches. In particular, Rete and performance-improved variation Leaps recall previous matches and are quite efficient as long as no major changes occur in the input data sets.
2.3.5 When to Use Rules In any situation, where no conventional programming approaches exist to solve the problem, rule engines might be an alternative. Here are some typical reasons to decide for a rule-based approach. • Conventional algorithms, commercial software, or solution approaches are not available. Scientific problems often arise from new analytical results and insights. Many of the problems to be solved resulting from new findings are rarely covered by literature, not to mention software algorithms. Rule-based systems are one of the alternatives for an easier development of expert systems. • A conventional approach would lead to instability and undetermined results due to the complexity of the resulting program code. The more complex a problem is, the more relations have to be considered. Translating these relationships to a static program code becomes tedious due to required nested code statements. The larger the code, the higher is the chance of unexpected results, which are hard to reproduce and to explain. Extracting rules from the code base allows storing them in a maintainable fashion and reduces the risk of unwanted results. • The complexity of the problem would lead to code that is not maintainable; in particular, if rules are built in using if–then approaches, the complexity of code may increase dramatically. If code contains more than 20 lines of if–then statements, it is usually a good idea to think about a rule-based approach. • The business rules change regularly; updating rules in a conventional program code is tedious and time consuming. Rule-based systems allow rule changes and updates independent from the back-end program. • The release cycle of a commercial software product is too long; if a commercial solution is used, the release cycles are usually too long to make fast adaptation to a change in the problem-solving process. An individual rule base can be updated much easier, either using in-house personnel or the software provider’s professional software services. • Domain or subject matter experts are nontechnical; most of the domain experts are not programmers. The problem is well understood by nontechnical domain experts but is not fully understood by the programmers. Consequently, the domain expert would have to train the developer until he becomes the domain expert — or the other way around — which would
5323X.indb 19
11/13/07 2:08:48 PM
20
Expert Systems in Chemistry Research
contradict any effective business model. The declarative programming of a rule base allows nontechnical staff to understand rules and to create new rules without requiring a complete software development background. Finally, the centralized knowledge base is often a good reason for a decision toward rule-based systems. Scientific processes in an enterprise are often the same or at least very similar. Regulations concerning the use of scientific results for commercial or academic purpose are standardized all over the world. Centralizing knowledge and rules does not just make maintenance easier; it is also the basis for effective auditing, regulatory, and legal compliance. Rule-based systems are quite effective for problem solving; however, they work dynamically in the sense of rule interpretation and updating. If a static problem shall be solved, a simple look-up table might provide a better performance while retaining the ease of maintenance and updating. The following drawbacks have to be taken into account. • Whereas individual rules are relatively simple, their logical relations and interactions within a large set of rules are usually not straightforward. Complex rule-based systems make it difficult to understand how individual rules contribute to the overall problem solution strategy. • Another aspect of reality that we often have to deal is with incomplete and uncertain knowledge. Most rule-based expert systems are capable of representing and reasoning with incomplete and uncertain knowledge. • The generic rule-based expert system does not provide the ability to learn from experience. Unlike human experts, an expert system does not automatically modify its knowledge base, adapt it to a problem, or add new ones. The amount of data is increasing exponentially. On March 11, 2002, the Chemical Abstracts Service (CAS) and REGISTRY databases contained 19,446,779 organic and inorganic substances as well as 17,787,700 sequences — more than 37 million total registrations. On March 13, 2007, five years later, the count was 31,010,591 organic and inorganic substances and 58,693,490 sequences — nearly 90 million total registrations. This is equivalent to a 60% increase in reactions and a 330% increase in sequences in five years. The current number can be retrieved from the World Wide Web [12]. Rule-based systems are a part of a solution but do not constitute the entire solution. Problem solving in scientific areas is quite often part of a large workflow that incorporates dozens or hundreds of individual processes. A rule base is a part of an expert system, which itself is a part of an extensive process. Looking, for instance, at a drug discovery process, rule bases might be used to predict the docking behavior of a new molecule; however, they would not be able to predict adverse reactions.
2.4 Reasoning 2.4.1 The Inference Engine In rule-based systems, an interpreter module controls the application of the rules and, thus, the systems activity. In a basic cycle of activity (i.e., recognize–act cycle)
5323X.indb 20
11/13/07 2:08:49 PM
Basic Concepts of Expert Systems
21
the system first checks to find all the rules in which the conditions hold. In a second step one rule is selected, and the actions are performed that apply to the rule. The selection of a rule is based on fixed, or conflict resolution strategies. The actions that are chosen lead either to a final decision or to an adaptation of the existing rule environment. By continuous repetitions a final decision is made or final state of the rule system is established. An expert system tool provides a mechanism, the inference engine, which automatically matches facts against patterns and determines which rules are applicable. Essentially, the inference engine performs three steps at each iteration: (1) comparing the current rule to stored or given patterns; (2) selecting the most appropriate rule; and (3) executing the corresponding action. This process continues until no applicable rules remain. The inference engine is the brain of a rule-based system; it manages a large number of facts and rules, matches rules against facts (data), and performs or delegates actions based on the outcome. Whereas simple if–then conditions suffice for small or unique data, complex data require specialized pattern-matching algorithms. Pattern-matching algorithms consider a group of data (a pattern) at the same time rather than single values. For instance, the recognition of patterns in a spectrum may be performed by comparing groups of peaks that represent characteristic structures or substructures. Patterns found in a database spectrum are mapped to peaks occurring in a query spectrum, and their similarity is assessed by various statistical and related methods [13,14]. There are two prerequisites for the application of pattern methods. First, the patterns are retrieved from calculated data, and, thus, the accuracy and reliability of a pattern in a given context must be validated. In terms of descriptors, the accuracy relies mainly on the experimental conditions or the raw data used for calculation. The second requirement is a suitable similarity measure for the comparison of patterns. Once the patterns are defined and the quality of the experimental base data is good, pattern-recognition methods are valuable. Nevertheless, if patterns change irregularly and cannot be explicitly defined, the similarity measure no longer describes the difference between query and experimental pattern even if a fuzzy logic approach is implemented. In general, systems based on the comparison of patterns can provide a series of candidates, and, finally, the expert has to decide the target compound by experience or by using additional information. One approach that is implemented in most expert system shells is the Rete algorithm. Rete was developed by Charles Forgy of Carnegie Mellon University [15]. It consists of a a generalized logic for matching facts against rules. The rules consist of one or more conditions and a set of actions that are executed if a set of facts matches the conditions. Rete works with directed acyclic graphs (Retes) that represent the rule sets as a network of nodes, each of which represents a single pattern. New facts entering the system propagate along the network, and each node matches its pattern to the facts. When successfully matched, the node is annotated. If all facts match all of the patterns for a given rule, the corresponding action is triggered. Rules are defined by subject-matter experts using a high-level rules language. They are collected into rule sets that are then translated at run time into an executable Rete.
5323X.indb 21
11/13/07 2:08:49 PM
22
Expert Systems in Chemistry Research
The Rete algorithm stores not just exact matches but also partial matches; this avoids reevaluating the complete set of facts when the rule base is changed. Additionally, it uses a decision-node-sharing technique, which eliminates certain redundancies.
2.4.2 Forward and Backward Chaining From a theoretical point of view two kinds of rule-based systems can be distinguished: forward-chaining systems and backward-chaining systems. A forwardchaining system starts with initial facts and uses the rules to draw new conclusions or take certain actions. In a backward-chaining system the process starts with a hypothesis, or target, that is to be proved and tries to find rules that would allow concluding that hypothesis. Forward-chaining systems are primarily data driven, whereas backward-chaining systems are target driven. Let us consider an example consisting of four rules made up of several conditions (i.e., server status, user type, user name) and actions (i.e., deny access, grant access, connect to server): if (server = “online” AND usertype = “User”) then denyAccess if (server = “online” AND usertype = “Administrator”) then grantAccess if (user = “Jill”) then usertype = “Administrator” if (grantAccess) then connectToServer Suppose we need to check if the server is connected, given the server is online and Jill tries to log on. In forward-chaining technique, a rule that applies is searched for in the first iteration. Rule 3 applies, and the user type is defined as Administrator. In the second iteration rule 2 applies and access is granted, which finally leads to application of rule 4 in the third iteration. In backward chaining the rules are evaluated starting from the conclusion (i.e., server is connected) and finding out the necessary condition. In first iteration the access must be granted (rule 4) to connect to the server. To grant access, the server must be online — one of the given assertions — and the user must be an administrator (rule 2). Since the online criterion is one of the given assertions, rule 3 holds the condition to be fulfilled.
2.4.3 Case-Based Reasoning Case-based reasoning (CBR) is a concept for problem solving based on solutions for similar problems. The central module of a CBR system is the case memory that stored the previously solved problems in form of a problem description and a problem solution. One of the first comprehensive approaches for this method was described by Aamodt and Plaza in 1994 [16]. The process of case-based reasoning consists of four steps:
5323X.indb 22
11/13/07 2:08:49 PM
Basic Concepts of Expert Systems
23
• Retrieval: Problems stored in the case memory are compared with a new target problem. Each case consisting of a problem description is compared to the new problem and is assessed for its similarity. • Reuse: The solution from the case memory is mapped to the target problem and is adapted to fit best to the new target problem. • Revision: After mapping the solution to the target problem, the new solution is validated by testing or simulation. Eventual corrections are made and are included in the new solution. • Retaining: After successful adaptation, the new solution is stored as as a new case in the case memory. Cases can be represented in a variety of formalisms like frames, objects, predicates, semantic nets, and rules. Cases are usually indexed to allow fast and efficient retrieval. Several guidelines on indexing have been proposed by CBR researchers [17,18]. Both manual and automated methods have been used to select indices. Choosing indices manually involves deciding a case’s purpose with respect to the aims of the reasoner and deciding under what circumstances the case will be useful. Indices can be automatically constructed on several paradigms: • Properties: Cases are indexed by properties — typically one- or multidimensional experimental or mathematical descriptors. Descriptors allow a specific definition of the case according to the relevant features of a case. • Differences: Indexing cases based on the difference between cases requires the software to analyze similar cases and evaluates which features of a case differentiate it from other similar cases. Those features that differentiate cases best are chosen as indices. • Similarities: Cases are investigated for common set of features. The ones that are not common are used as indices to the cases. Indices may also be created by using inductive learning methods — for instance, artificial neural networks — for identifying predictive features that are then used as indices. However, despite the success of many automated methods, Janet Kolodner believes that people tend to do better at choosing indices than algorithms, and therefore for practical applications indices should be chosen manually [19]. The case base is organized in a structure that allows efficient search and retrieval. Several case-memory models have been proposed. The two most widely used methods are the dynamic memory model of Schank and Kolodner [20,21] and the category-exemplar model of Porter and Bareiss [22]. The dynamic memory model uses memory organization packets (MOPs), either based on instances that represent specific cases or on abstractions representing generalized forms of instances. The category-exemplar model takes advantage of categories and semantic relations organized in a network. This network represents a background of general domain knowledge that enables explanatory support to some CBR tasks. The retrieval algorithm relies on the indices and the organization of the memory to direct the search to potentially useful cases. Several algorithms have
5323X.indb 23
11/13/07 2:08:49 PM
24
Expert Systems in Chemistry Research
been implemented to retrieve appropriate cases: for example, serial search [23], hierarchical search [24], and simulated parallel search [25]. Case-based reasoning has been applied successfully to a variety of applications. Leng et al. developed a two-level case-based reasoning architecture for predicting protein secondary structure [26]. They divided the problem into two levels: (1) reasoning at the protein level and using the information from this level to focus on a more restricted problem space; and (2) reasoning at the level of internal structures decomposed as segments from proteins. Conclusions from second level were then combined to make predictions on the protein level. An article on practical reasoning in chemistry was published by Kovac [27].
2.5 The Fuzzy World Though an expert system consists primarily of a knowledge base and an inference engine, a couple of other technologies have to be used to deal with several issues that are inherent to the processing of natural. Problem solving in expert systems is often based on experimental data. The accuracy of the data that are used for the knowledge base in an expert system is normally controlled by experienced personnel; inaccurate, diffuse, and unspecific data are eliminated manually. To process larger sets of potential source data for knowledge bases, a method must be used that takes inaccuracies as well as natural fuzziness of experimental data into account —ideally automatically and without the help of an expert. Let us consider an example to illustrate this: John has a meeting at 8:30 a.m. at his company. He leaves his house at 8 a.m. heading toward the bus station. It takes him 5 minutes to reach the station, where the bus departs at 8:10; this will alow him to be right on time to attend to the meeting. What if John leaves his house at 8:05 a.m. instead? According to logical rules, he should reach the bus in time. However, experience says that this is not sure, and he probably misses the meeting. Obviously, we think about these exact times with certain fuzziness. Nobody would be really surprised if a guest for dinner arrives at 8:01 p.m. when he has been invited to arrive at 8:00. That is why systems that can handle this kind of uncertainty are mandatory elements of an expert system. Uncertainty in expert systems can be handled in a variety of approaches: certainty factors, fuzzy logic, and Bayesian theroy.
2.5.1 Certainty Factors Certainty theory is an approach to inexact reasoning that describes uncertain information in a certainty factor [28]. Certainty factors are used as a degree of confirmation of a piece of evidence. Mathematically, a certainty factor is the measure of belief minus the measure of disbelief. Again, using an example with John, an uncertainty could be the following: if (JohnsDeparture = 8.00) then (JohnsAtMeeting equals 0.8)
5323X.indb 24
11/13/07 2:08:50 PM
25
Basic Concepts of Expert Systems
1
On Time
Early
Late
0.5
8.20
8.25
8.30
8.40
Time
Figure 2.4 Simple mathematical functions can be used to relate certainty factors with a corresponding outcome. Each outcome — early, on time, late — is associated with a function. Early arrival is defined for a time frame between 8:20 and 8:30, with certainty factors between 0 and 1. An input time of 8:25 with a certainty factor of 1 would result in an early arrival. The smaller the certainty factor, the wider the time frame for arrival. The same applies to the other functions.
The rule says that if John leaves the house at 8:00, the certainty that he will reach the bus in time and attend to the meeting is 80%. Depending on the time frame, we can create functions that relate certainty factors with the time of arrival. Figure 2.4 show how the aforementioned example of John’s arrival can be described by means of a transfer function. Even the concept of certainty factors is not derived from a formal mathematical basis; it is the most commonly used method to describe uncertainty in expert systems. This is mainly due to the fact that certainty factors are easy to compute and can be used to effectively reduce search by eliminating branches with low certainty. However, it is difficult to produce a consistent and accurate set of certainty factors. In addition, they are not consistently reliable; a certainty factor may produce results opposite to probability theory.
2.5.2 Fuzzy Logic Problem solving in computational chemistry is often based on experimental data that may be more or less accurate. Experienced scientist can control the accuracy of the data that are used for the knowledge base in an expert system; inaccurate diffuse and unspecific data are manually eliminated. For example, the main reasons for inaccuracies in vibrational spectra are instrumental errors, measuring conditions and poor purity of the sample. Although most of these errors can be kept small by adequate handling of samples and instruments, random errors that lead to a loss in precision cannot be completely avoided. Even errors may not necessarily concern the entire spectrum; instead, a certain important spectral range may be affected. Complex dependencies of spectral signals from the chemical environment of the corresponding substructure lead to a certain fuzziness of the spectral signal. This is the reason why strict assignment rules (e.g., a strong peak between 3000 cm–1
5323X.indb 25
11/13/07 2:08:50 PM
26
Expert Systems in Chemistry Research
and 3100 cm–1 in the infrared spectrum proves at least one benzene-ring) cannot be applied in most cases. Accuracy checks are generally performed by comparing measured data with data from certified reference materials. When measured data are not accurate because of relative or systematic errors, or a lack of precision (noise), the comparison between measured data and reference values cannot lead to any useful conclusion in an expert system. To process larger sets of potential source data for knowledge bases, a method must be used that takes inaccuracies as well as natural fuzziness of experimental data into account — ideally automatically and without the help of an expert. Problems of uncertainty and inaccuracy can be addressed by using statistical and stochastic methods that have been described before [29,30]. The fuzzy logic approach provides a mathematical framework for representation and calculation of inaccurate data in AI methods [31,32]. Fuzzy logic is a superset of conventional (Boolean) logic that has been extended to handle the values between exactly true and exactly false. The general principle in fuzzy logic is that a reference value x0 is associated with a fuzzy interval dx, and experimental data within an interval of x0 ± dx are identified as reference data. Since natural, or experimental, data are always inaccurate, and the representation of knowledge is quite like that in fuzzy logic, expert systems have to use fuzzy logic or some techniques similar to fuzzy logic [33]. In a computer system based on the fuzzy logic approach, fuzzy intervals for reference values are defined a priori. The term fuzzy logic leads to the association of a logic that is fuzzy itself. In fact, the term concerns the logic of fuzziness — not a logic that itself is fuzzy. Fuzzy sets and logic are used to represent uncertainty, which is crucial for handling natural systems. A fuzzy expert system is an expert system that uses a collection of fuzzy functions and rules to reason about data. In other words, a fuzzy expert system is a collection of membership functions and rules that are used to reason about data. Unlike conventional expert systems, which are mainly symbolic reasoning engines, fuzzy expert systems are oriented toward numerical processing. However, there is another aspect for using fuzzy logic in the context of expert systems. The conclusion of a fuzzy rule requires converting the result back into objective terms before they can be used in a nonfuzzy method. If we describe John’s probability to arrive in time by using fuzzy logic, the result from the inference engine might be high, medium, or low. If we want to use this result in an algorithm that calculates the most probable arrival time, the conclusions from the inference engine have to be translated to a real number. This process is called defuzzification. Defuzzification is done by mapping the confidence factors that result from a fuzzy outcome onto the membership function. The defuzzified value can then be calculated by different methods, like the center of gravity, the smallest or largest maximum or mean of maximum of the function.
2.5.3 Hidden Markov Models Hidden Markov models (HMMs) are statistical models based on the Markov property [34]. A stochastic process has the Markov property if the conditional probability
5323X.indb 26
11/13/07 2:08:51 PM
27
Basic Concepts of Expert Systems
distribution of future states is independent of the path of the process. A typical example is a dice game: Each step in the process of a dice game is independent of former states of the game. In contrast to that, each turn in a card game depends on previously played cards. HMMs are statistical models for systems that behave like a Markov chain, which a discrete-time stochastic process formed on the basis of the Markov property. A Markov chain is a sequence of events, the probability of each of which is dependent only on the event immediately preceding it. An HMM represents stochastic sequences as Markov chains where the states are not directly observed but are associated with a probability density function. The generation of a random sequence is then the result of a random walk in the chain. HMMs are dynamic models, in the sense that they are specially designed to account for some macroscopic structure of the random sequences. In statistical pattern recognition, random sequences of observations were considered as the result of a series of independent draws in one or several Gaussian densities. To this simple statistical modeling scheme, an HMM adds the speciation of some statistical dependence between the Gaussian densities from which the observations are drawn. In a hidden Markov model the states are less simple, and instead of having a single known outcome they can have several possible outcomes. The model is called hidden because the outcome of any given state is uncertain. HMMs have been successfully used in a number of Bioinformatics applications, like the modeling of proteins and nucleic acids or for quantitative analysis of biological sequence data using statistical approaches [35,36].
2.5.4 Working with Probabilities — Bayesian Networks Bayesian networks are based on Bayes’ Theorem, which gives a mathematical framework for describing the probability of an event that may have been the result of any of two or more causes [37]. The questions is this: What is the probability that the event was the result of a particular cause, and how does it change if the cause is changing? Bayesian networks are statistic models for describing probabilistic dependencies for a set of variables. They trace back to a theorem in the eighteenth century found by Thomas Bayes, who first established a mathematical base for probability inference [38]. Bayes’ theorem is based on two different states:
P ( A| B ) =
P ( A| B ) P ( A ) P ( B)
(2.1)
This equation describes the probability P for state A existing for a given state B. To calculate the probability, Bayes used the probability of B existing given that A exists, multiplied by the probability that A exists, and normalized by the probability that B exists. This admittedly complicated explanation can be interpreted as follows: For an existing state B, what is the probability that state B is caused by state A? The importance of this theorem is that probabilities can be derived without specified knowledge about P(A|B), if information about P(B|A), P(A), and P(B) is available.
5323X.indb 27
11/13/07 2:08:52 PM
28
Expert Systems in Chemistry Research
The essence of the Bayesian approach is to provide a mathematical rule explaining how a hypothesis changes in light of new evidence [39]. Back to John for an example: John has to reach his bus not only once, but every morning. The experience he makes if he leaves his house at different times each day affects his feeling of what the probability is for reaching his bus. In a Bayesian analysis, a set of observations should be seen as something that changes opinion. In other words, Bayesian theory allows scientists to combine new data with their existing knowledge or expertise. A Bayesian network can be used to model the dependencies between variables that directly influence each other, which are usually few. The rest of the variables are assumed conditionally independent. A Bayesian network is a directed graph in which each node is annotated with quantitative probability information. It is constructed by selecting a set of variables that define the nodes of the network. The nodes are connected via directed links that indicate their inheritance, and each node has a conditional probability distribution that quantifies the effect of the parents on the node. Dynamic Bayesian networks (DBNs) are directed graph models of stochastic processes. They generalize HMMs by representing the hidden and observed state in terms of state variables, which can have complex interdependencies. The graphical structure provides an easy way to specify these conditional independencies, and, hence, to provide a compact parameterization of the model. Although this approach has a clear mathematical basis for inexact reasoning, this approach requires the gathering of data about all possible states of a problem, which is rarely applicable in practice. One of the most noted expert systems developed using a Bayesian approach for inexact reasoning is PROSPECTOR, a system designed for mineral exploration [40]. F. V. Jensen provides a complete overview on Bayesian networks and decision graphs [41].
2.5.5 Dempster-Shafer Theory of Evidence The Dempster-Shafer theory, also known as the theory of belief functions, is a generalization of the Bayesian theory of subjective probability [30,42]. Whereas the Bayesian theory requires probabilities for each question of interest, belief functions allow us to base degrees of belief for one question on probabilities for a related question. These degrees of belief may or may not have the mathematical properties of probabilities; how much they differ from probabilities will depend on how closely the two questions are related. The Dempster-Shafer theory is based on two ideas: (1) the idea of obtaining degrees of belief for one question from subjective probabilities for a related question; and (2) Dempster’s rule for combining such degrees of belief when they are based on independent items of evidence. To illustrate the idea of obtaining degrees of belief for one question from subjective probabilities for another, suppose John has subjective probabilities for the reliability of his alarm clock. Let us say the reliability that his alarm clock shows the right time is 0.9 and the probability that it is unreliable is 0.1. Suppose he will get to his bus if he leaves his house at 8:00 in the morning. This statement, which is true if his alarm clock is reliable, is not necessarily false if the
5323X.indb 28
11/13/07 2:08:52 PM
Basic Concepts of Expert Systems
29
clock is unreliable. So the reliability of the clock alone justifies a 0.9 degree that he will reach the bus but only a zero degree of belief (not a 0.1 degree of belief) that he will miss the bus. This zero does not mean that he will reach the bus always in time, as a zero probability would; it merely means that there is no reason for John to believe that he will miss the bus. The 0.9 and the zero together constitute a belief function. In summary, we obtain degrees of belief for one question (i.e., Will John reach the bus in time?) from probabilities for another question (i.e., Is the alarm clock reliable?). Dempster’s rule begins with the assumption that the questions for which we have probabilities are independent with respect to our subjective probability judgments, but this independence is only a priori; it disappears when conflict is discerned between the different items of evidence.
2.6 Gathering Knowledge — Knowledge Engineering After all, one of the most important problems in dealing with expert systems is the stage of getting the knowledge of a human expert into the expert system. The subject of this task is knowledge engineering [43]. Knowledge engineering is still the bottleneck in the development of expert systems. It is difficult for a chemist to explicitly state the knowledge he is using in a form that is suitable for computational processing. For a spectroscopist it would be hard to write down his experience in interpreting nuclear magenetic resonance spectra on a sheet of paper; a synthesis chemist would have the problem of defining the basic reasoning for his retrosynthesis approach in if–then rules, and a metabonomics expert would not be able to define the rules for his experience when he is proposing a metabolic pathway for a drug based on hundreds of individual results. One of the reasons that a scientist, currently working on a problem, would have problems explicitly stating his knowledge is that information for rules has to be abstracted, and this is a challenging task when a specific problem is in mind. Three main questions arise from this problem: (1) How to derive a rule from the intuitive decision-making skills of an expert: This problem can be addressed by allowing the expert himself to use the software without requiring special technical or development skills. Declarative languages are the first step in this direction. Intuitive and easy-to-use graphical user interfaces can support the expert considerably with this task. Although a few methods for guiding rule entry have been proposed, the problem of intuitive knowledge remains. (2) How to assess the logic of the acquired rules for conflicts, overlaps, and gaps: The logic for conflict handling is closely related to the subject of interest. In many cases, only the expert is able to find the reason for a conflict that itself emerges usually during validation of the system. However, software can again assist the expert by presenting the conflict in an understandable way that is easy for the expert to evaluate. (3) How to generalize the explicit knowledge for reusing it in similar cases: The only solution to this issue is to separate the development of the rule base from a specific problem.
5323X.indb 29
11/13/07 2:08:53 PM
30
Expert Systems in Chemistry Research
In a knowledge engineering process, the scientist would be interviewed and posed representative problems. Based on his responses, the knowledge he applies needs to be understood and encoded in the form of the knowledge representation used: the rules. In a second stage, the scientist would then need to validate the programmed rules to make sure that the outcome from expert systems is valid in the problems domain. This cycle needs continuous repetition until the system performs in the desired manner. Depending on the complexity of the domain, knowledge engineering could take anywhere from a few days to a few years. Expert system tools have been created that provide support in the creation of this knowledge and carry out checks on the completeness and correctness of the knowledge represented in the system. The process of knowledge engineering can be considerably improved through graphical knowledge representation tools. A number of methods for structuring rule sets have been traditionally used, such as trees, spider diagrams, and concept maps. These methodologies aim to model the application by establishing a hierarchy of rule sets. The developer can add control rules to force the flow of the inference engine through a tree view or a diagram. Modern applications allow the knowledge to be structured into intuitive units of knowledge, each of which is represented as decision tree or a cases table and an overall summary map representing the structure of the entire knowledge base for a particular task. Based on these examples, the requirements for knowledge engineering from a software point of view can be summarized as follows: • Declarative languages and intuitive and easy-to-use graphical user interfaces are required for problem assessment. • Multiple knowledge representations shall be available to express knowledge in the most appropriate way. • Structuring elements shall be available for easy construction, analysis, and adaptation of hierarchies and relationships. • Method-like rule induction shall be available to convert tables or lists of examples into decision trees. • Tools are required that support the expert in finding the reason for a conflict, a gap, or an overlap by presenting the conflict in an understandable way that is easy for the expert to evaluate. • Experts shall be absolved from their day-to-day work and shall have the chance to concentrate on the knowledge engineering process. • Tools shall be available to test and validate the inserted knowledge, preferably on the basis of constructed test cases. • Revision and approval of the system shall be supported by an appropriate user management system. Knowledge engineering is related to many computer science domains such as artificial intelligence, databases, data mining, and decision support. The quality of the primary data for the knowledge base determines the validity and reliability of results. However, knowledge engineering is just a part of the task for an expert system. Expert systems rely on a series of supporting technologies that ensure that
5323X.indb 30
11/13/07 2:08:53 PM
Basic Concepts of Expert Systems
31
complex data can be managed and evaluated in an efficient manner. The following chapters will cover some of these techniques.
2.7 Concise Summary Backward Chaining is a problem-solving procedure that starts with a statement and a set of rules leading to the statement and then works backward, matching the rules with information from a database of facts until the statement can be either verified or proven wrong. Bayes’ Theorem is a mathematical framework describing the probability that an event was the result of a particular cause and how it changes if the cause changes. Bayesian Networks are statistic models for describing probabilistic dependencies for a set of variables based on Bayes’ theorem. Case-Based Reasoning (CBR) is a problem-solving system that relies on stored representations of previously solved problems and their solutions. Certainty Factors are used as a degree of confirmation of a piece of evidence. Mathematically, a certainty factor is the measure of belief minus the measure of disbelief. Data are printed, stored, visualized, or in any other state delocalized and temporally decoupled available facts that can be preserved in a documentation process. Dempster-Shafer Theory of Evidence is a generalization of the Bayesian theory based on degrees of belief rather than probabilities. Expert System Shells are software packages that facilitate the building of knowledgebased systems by providing a knowledge representation scheme and an inference engine. The developer adds domain knowledge. Expert Systems are application programs making decisions or solving problems in a particular field by using knowledge and analytical rules defined by experts in the field. They aid an expert in making decisions for a certain problem. An expert system typically operates with rules that are evaluated to predict a result for a certain input. Forward Chaining is a problem-solving procedure that starts with a set of rules and a database of facts and works to a conclusion based on facts that match all the premises set forth in the rules. Frames are a technique for representing knowledge in a group of attributes and relationships, which are stored in slots. Hidden Markov Models (HMMs) are statistical models based on the assumption that the probability of future states of a process is independent of its path (Markov property). If–Then Rules are collected in a rule base and describe problem situations and the actions an expert would perform in those situations. Inference Engine is the processing portion of an expert system. With information from the knowledge base, the inference engine provides the reasoning ability that derives inferences, or conclusions, on which the expert system acts. Information is the process of perception and cognition and is typically followed by interpretation.
5323X.indb 31
11/13/07 2:08:53 PM
32
Expert Systems in Chemistry Research
Knowledge is related to individuals and is stored in the cognitive system, the brain. It is subject as well as purpose oriented and is related to a certain context. Consequently, knowledge carriers are all individuals belonging to an enterprise. Knowledge Acquisition defines the gathering of expertise from a human expert for entry into an expert system. Knowledge Engineering is the process of gathering knowledge from subject-matter experts and transforming it into a computable format. Knowledge Induction describes the generation of knowledge either directly (communicaton), indirectly (via information), or through self-inducting (by training). Knowledge Logistics ensures that knowledge is available in adequate form at any time and place. Knowledge Representation describes the notation or formalism used for coding the knowledge to be stored in a knowledge-based system. Knowledge-Based Systems use stored knowledge to solve problems in a specific domain. Pattern Matching is the process of matching facts against patterns to determine rules that are applicable. Rule-Based Systems, instead of representing knowledge in a static way, represent knowledge in terms of rules that lead to conclusions. A simple rule-based system consists of a set of if–then rules, a collection of facts, and an interpreter (inference engine) controlling the application of the rules by the given facts. Semantic Networks are a form of knowledge representation in a directed graph, where the nodes represent concepts and the edges, or connectors, describe their relations. Subject-Matter Experts (SMEs) are persons that have knowledge of a specific subject or domain. Task refers to some goal-oriented, problem-solving activity, whereas domain refers to the area within which the task is being performed. Task Domain is the area of human intellectual endeavor to be captured in an expert system.
References
5323X.indb 32
1. Barr, A. and Feigenbaum, E.A., Eds., The Handbook of Artificial Intelligence, Vol. 1. William Kaufman Inc., Los Altos, CA, 1981. 2. Quillian, M.R., Word Concepts: A Theory and Simulation of Some Basic Semantic Capabilities, Behavioral Sci., 12, 410, 1967. 3. Shastri, L., Why Semantic Networks? in Principles of Semantic Networks: Explorations in the Representation of Knowledge, Sowa, J.F., Ed., Morgan Kaufmann Publishers, San Mateo, CA, 1991, 109. 4. Schubert, L.K., Semantic Nets Are in the Eye of the Beholder, in Principles of Semantic Networks: Explorations in the Representation of Knowledge, Sowa, J.F., Ed., Morgan Kaufmann Publishers, San Mateo, CA, 1991, 95. 5. Lehmann, F., Semantic Networks, Comp. Mathemat. Applicat., 23, 1, 1992. 6. Open Biomedical Ontologies, The OBO Foundry, http://obo.sourceforge.net/. 7. Harris, M.A., et al., The Gene Ontology (GO) database and informatics resource, Nucleic Acids Res., 32, D258, 2004.
11/13/07 2:08:54 PM
Basic Concepts of Expert Systems
5323X.indb 33
33
8. Smith, C.L., Goldsmith, C.A., and Eppig, J.T., The Mammalian Phenotype Ontology as a Tool for Annotating, Analyzing and Comparing Phenotypic Information, Genome. Biol., 6, R7, 2005. 9. Bard, J., Rhee, S.Y., and Ashburner, M., An Ontology for Cell Types, Genome. Biol., 6, R21, 2005. 10. Hsing, M. and Cherkasov, A., Integration of Biological Data with Semantic Networks, Current Bioinformatics, 1(3), 1, 2006. 11. Minsky, M.L., Form and Content in Computer Science, ACM Turing Lecture, JACM, 17, 197, 1970. 12. American Chemical Society, Chemical Abstracts Service, Registry Number and Substance Counts, http://www.cas.org/cgi-bin/cas/regreport.pl. 13. Jalsovszky, G. and Holly, G., Pattern Recognition Applied to Vapour-Phase Infrared Spectra: Characteristics of nuOH Bands, J. Mol. Struct., 175, 263, 1988. 14. Sadtler Research Laboratories, Spectroscopy Group, Cambridge, MA, http://www. sadtler.com. 15. Forgy, C., Rete: A Fast Algorithm for the Many Pattern/Many Object Pattern Match Problem, Artificial Intelligence, 19, 17, 1982. 16. Aamodt, A. and Plaza, E., Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches, AI Communications, 7(1), 39, 1994. 17. Birnbaum, L. and Collings, G., Remindings and Engineering Design Themes: A Case Study in Indexing Vocabulary, in Proceedings of the Second Workshop on Case-Based Reasoning, Pensacola, FL, 1989, 47. 18. Hammond, K.J., On Functionally Motivated Vocabularies: An Apologia, in Proceedings of the Second Workshop on Case-Based Reasoning, Pensacola, FL, 1989, 52. 19. Kolodner, J.L., Case-Based Reasoning, Morgan Kaufmann, San Mateo, CA, 1993. 20. Kolodner, J.L., Maintaining Organization in a Dynamic Long-Term Memory, Cognitive Science, 7, 243, 1983. 21. Schank, R., Dynamic Memory: A Theory of Reminding and Learning in Computers and People, Cambridge University Press, Cambridge, UK, 1982. 22. Porter, B.W. and Bareiss, E.R., PROTOS: An Experiment in Knowledge Acquisition for Heuristic Classification Tasks, in Proceedings of the International Meeting on Advances in Learning, Les Arcs, France, 1986, 159. 23. Navichandra, D., Exploration and Innovation in Design: Towards a Computational Model, Springer Verlag, New York, 1991. 24. Maher, M.L. and Zhang, D.M., CADSYN: Using Case and Decomposition Knowledge for Design Synthesis, in Artificial Intelligence in Design, Gero, J.S., Ed., ButterworthHeinmann, Oxford, 1991. 25. Domeshek, E., A Case Study of Case Indexing: Designing Index Feature Sets to Suit Task Demands and Support Parallelism, in Advances in Connectionnist and Neural Computation Theory, Vol.2: Analogical Connections, Barenden, J. and Holyoak, K., Eds., Norwood Publishing, NJ, 1993. 26. Leng, B., Buchanan, B.G., and Nicholas, H.B., Protein Secondary Structure Prediction Using Two-Level Case-Based Reasoning, J. Comput. Biol., 1(1), 25, 1994. 27. Kovac, J., Theoretical and Practical Reasoning in Chemistry, Found. Chem., 4, 163, 2002. 28. Shortliffe, E.H., Computer Based Medical Consultations: MYCIN, Elsevier, New York, 1976. 29. Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann Publishers, New York, 1988. 30. Shafer, G.A., A Mathematical Theory of Evidence, Princeton University Press, Princeton, NJ, 1976. 31. Negoita, C.V. and Ralescu, D., Simulation, Knowledge-Based Computing, and Fuzzy Statistics, Van Nostrand Reinhold, New York, 1987.
11/13/07 2:08:54 PM
34
Expert Systems in Chemistry Research
32. Zadeh, L.A., Fuzzy Sets, Information and Control, 8, 338, 1965. 33. Zadeh, L.A., Fuzzy Sets as the Basis for a Theory of Possibility, Fuzzy Sets Syst., 1, 3, 1978. 34. Baum, L.E., et al., A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains, Ann. Math. Statist., 41, 164, 1970. 35. Durbin, R., et al., Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids, 1st ed., Cambridge University Press, Cambridge, UK, 1999. 36. Pachter, L. and Sturmfels, B., Eds., Algebraic Statistics for Computational Biology, 1st ed., Cambridge University Press, Cambridge, UK, 2005. 37. Jensen, F.V., Bayesian Networks and Decision Graphs, 1st ed., Springer, New York, 2001. 38. Bayes, T., An Essay towards Solving a Problem in the Doctrine of Chances, by the late Rev. Mr. Bayes, communicated by Mr. Price, in a letter to John Canton, M. A. and F. R. S., Philosophical Transactions of the Royal Society of London, 53, 370, 1763. 39. Dempster, A.P., A Generalization of Bayesian Inference, Journal of the Royal Statistical Society, Series B, 30, 205, 1968. 40. Duda, R., et al., Development of a Computer-Based Consultant for Mineral Exploration, SRI Report, Stanford Research Institute, Stanford, CA, 1977. 41. Jensen, F.V., Bayesian Networks and Decision Graphs, 1st ed., Springer, New York, 2001. 42. Shafer, G., Perspectives on the Theory and Practice of Belief Functions, International Journal of Approximate Reasoning, 3, 1, 1990. 43. Negnevitsky, M., Artificial Intelligence: A Guide to Intelligent Systems, Addison Wesley, 2004.
5323X.indb 34
11/13/07 2:08:54 PM
3
Development Tools for Expert Systems
3.1 Introduction Conventional programming languages are designed for the procedural manipulation of data. Humans, however, often solve complex problems using very abstract, symbolic approaches, which cannot be directly implemented in conventional languages. Although abstract information can be modeled in these languages, considerable programming effort is required to transform the information to a format usable with procedural programming paradigms. One of the results of research in the area of artificial intelligence has been the development of techniques, which allow the modeling of information at higher levels of abstraction. These techniques are embodied in languages or tools, which allow programs to be built that closely resemble human logic in their implementation and are therefore easier to develop and maintain. Most expert systems are developed via specialized software tools called shells. These shells come equipped with an inference mechanism — backward chaining, forward chaining, or both — and require knowledge to be entered according to a specified format. We will have a closer look at several tools for developing expert system in this chapter.
3.2 The Technical Design of Expert Systems Expert systems consist of several modules (Figure 3.1).
3.2.1 Knowledge Base The knowledge base contains knowledge on a particular domain in the form of logic rules or similar representation technologies. Rules are stored as declarative statements and can be added, changed, and edited using a knowledge engineering module. Rules are formulated in a cause-and-effect manner that is similar to how an expert would formulate his knowledge. The rules are either obtained from a human expert in the knowledge engineering process or entered by the expert himself. The definition of rules is the most common technique used today for representing knowledge in an expert system.
3.2.2 Working Memory Specific information on a current problem is represented as case facts and is entered in the expert system’s working memory. The working memory contains both the facts 35
5323X.indb 35
11/13/07 2:08:54 PM
36
Expert Systems in Chemistry Research
Descriptive Language Interaction
Knowledge Engineer/Expert
User
Inference Control
Engineering Module
User Interface
Working Memory
Inference Engine
Task-Specific Data
Search
Rule Definition
Pattern Matching
Knowledge Base
Evaluation
Prediction
Rules
Simulation
Fuzzy Logic
Certainty Factors
Figure 3.1 The general design of an expert system comprises the user-oriented parts of user interface and the working memory that stores specific information for a current problem as facts. The knowledge base contains knowledge on a particular domain in the form of logic rules or other representation technologies. The inference engine performs the reasoning by using the knowledge to perform an evaluation similar to the way a human reasons with available information. Inference engine and knowledge base are handled by a knowledge engineering module for defining rules and controlling the inference mechanism. A series of underlying technologies, such as searches or pattern matching, support the process of inference.
entered by the user from questions asked by the expert system and facts inferred by the system. The working memory could also acquire information from databases, spreadsheets, or sensors and could be used by the expert system to conclude additional information about the problem by using the general knowledge contained in the knowledge base.
3.2.3 Inference Engine The analogy of human reasoning is performed in the expert system with the inference engine. The role of the inference engine is to work with the available information contained in the working memory and the general knowledge contained in the knowledge base to derive new information about the problem. This process is similar to the way a human reasons with available information to arrive at a conclusion. The inference engine, working either in the backward- or forward-chaining mode, will attempt to conclude new information about the problem from available information until some goal is reached or the problem is solved.
3.2.4 User Interface The function of the user interface is to present questions and information to the operator and to supply the operator’s responses to the inference engine. Any values
5323X.indb 36
11/13/07 2:09:28 PM
Development Tools for Expert Systems
37
entered by the user must be received and interpreted by the user interface. Some responses are restricted to a set of possible legal answers; others are not. The user interface checks all responses to ensure that they are of the correct data type. Any responses that are restricted to a legal set of answers are compared against these legal answers. Whenever the user enters an illegal answer, the user interface informs the user that his answer was invalid and prompts him to correct it. As explained in the cross-referenced application, communication between the user interface and the inference engine is performed through the use of a user interface control block, which is passed between the two. Besides providing final results or conclusions, both human experts and expert systems can explain how they arrived at their results. This capability is often important because the types of problems to which expert systems are applied require that a justification of the results be provided to the user. For example, an expert system that recommends some antibiotic treatment for a patient would need to explain to the physician how this recommendation was formulated. Expert systems also often have the capability of explaining why a given question is being asked. When an individual consults with a human expert, the conversation is highly interactive, and on occasion, the individual may ask why a certain line of reasoning is being pursued. The explanation given can make the user feel more comfortable with the line of questioning and also can help to clarify what issues the expert believes are important for the problem. Several programming languages and tools are available for developing an expert system. The most important ones will be described here.
3.3 Imperative versus Declarative Programming Conventional programming is performed typically in the form of sequential statements, each of which changes the state of variables in the program. This method of program development is called imperative programming. A common method of imperative programming is procedural programming, where the code consists of sequential calls to functions, methods, and subroutines, all of which are abstracted as procedures. Another paradigm is functional programming that focuses on the application of mathematical functions rather than the changing states of variables. Figure 3.2 gives an overview of the different programming paradigms and examples of programming languages. A code example of an imperative language (C++) is shown following. This is a function that normalizes a vector according to the Euclidian L2-Norm — that is, it divides each component of a vector by the square root of the sum of products of all vector components. The function is called with two parameters, one of which is a pointer (*) to a multidimensional variable (Vector) that represents the input vector that has to be normalized. The other parameter is the length of the vector (N). The code then declares a variable (Norm) and initializes it, or (assigns a start value of zero). It then defines a loop (for statement) that runs over all components (i) of the vector to summarize all component products. It then calculates the square root (sqrt) of the sum. In a second loop, it again runs over all vector components to divide them by the square
5323X.indb 37
11/13/07 2:09:28 PM
38
Expert Systems in Chemistry Research
Expert Systems Shells
Descriptive Languages
Application-Specific
Macro Languages, VBA
Scripting
JavaScript, VB, Python, Perl
Imperative, Object-Oriented
C++, C#, Java, Python, Perl
Imperative, Procedural Machine
FORTRAN, Common LISP, C Assembler
Computers
Figure 3.2 A schematic model of programming paradigms shows how the complexity of programming techniques increases from the computer on the base to the expert systems on top. Expert systems are designed for extensive user interaction and require descriptive languages that are on a higher level (i.e., closer to the user) than those actually processed by the computer, such as machine languages. Imperative and object-oriented paradigms support the sequential processing and the data and functional modeling of a program. Most scripting languages are based on such a layer, whereas macro languages are typically intended to perform user interactions automatically. The example of visual basic for applications (VBA) combines imperative programming with application-specific functionality. Finally, descriptive languages no longer include any basic business logic but are adapted to the expert using the system.
root (Norm). The if statement ensures that the calculation is performed only if the nominator is not zero: void EuclNormF(double* Vector, int N) { double Norm = 0; for (int i = 0; i < N; i++) { Norm += Vector[i] * Vector[i]; } Norm = (sqrt(Norm)); for (int i = 0; i < N; i++) { if(Vector[i]) Vector[i] /= Norm; } }
5323X.indb 38
11/13/07 2:09:29 PM
Development Tools for Expert Systems
39
This is a procedural algorithm that is performed in a sequential manner and leads to an explicit answer. The program code may be called from another code and may include function calls, such as the square root function sqrt. Other methods of programming paradigms are logical programming and declarative programming. Logical programming languages describe the business logic of a program but do not include the way the computation is performed — that is, any algorithms as imperative programming languages do. A typical example is PROLOG, which is described later. Closely related to the latter is declarative programming that describes states rather than how to achieve them. Additionally, the syntax is modeled after the way humans describe a state and thus is more declarative than other languages. Logic and declarative programs explicitly specify a goal that has to be achieved and leave the implementation of the algorithm to other software. A simple example is Structured Query Language (SQL), which is the most popular computer language for creating, updating, and retrieving data from relational database management systems. An SQL select statement, for instance, specifies what data are to be retrieved from a database but does not describe the process of how this is actually done; this task is left to the database management software. An example is shown here: SELECT name FROM compounds WHERE compounds.mol_wt < 250 ORDER BY name ASC; This is a statement that retrieves names (name) from a table of compounds in a database, restricting the results to those compounds that have a molecular weight (mol_wt) less than 250, and presents the results in ascending order of the names. These statements do not include any algorithm or function that defines how the search is actually done; the database server incorporates the business logic to do this. In addition, this is a declarative statement in the sense that it is more or less self-explanatory. A clear classification of programming languages according to these or other programming paradigms is often not possible. This is on one hand due to small differences in the definition of the paradigms. For instance, a different definition of a declarative program includes just functional and logic programming languages, and often the term declarative is used to distinguish languages from imperative ones. On the other hand, most logic programming languages are able to describe algorithms and implementation details. Similarly, it is possible to write programs in a declarative style even in an imperative programming language by using declarative names for variables and excluding nondeclarative details from the main code. However, a declarative language, like all natural languages, has a syntax describing how the words in the language may be combined and a semantics describing how sentences in the language correspond to the result of a program. In the following sections, we will examine some of the programming languages that are of importance for the development of expert systems.
5323X.indb 39
11/13/07 2:09:29 PM
40
Expert Systems in Chemistry Research
3.4 List Processing (LISP) LISP is a programming language that was developed in 1959 at the Massachusetts Institute of Technology [1]. Its origin goes back to the development of formula translator programming language (Fortran) subprograms for symbolic calculations. At that time, John McCarthy had the idea of developing an interpreter for symbolic calculations and formed the basis for LISP. It uses two data structures — single scalar values, or atoms; and single or nested lists — that form an associative array. These data structures are created dynamically without the need for reserving computer memory. The dynamic approach makes declarations unnecessary and allows variables to represent arbitrary objects, independent of their data type or structure. LISP programs can run using interpreters or compilers, which allows the programmer to combine the advantages of both worlds into a single program. LISP was used widely in the 1970s for developing expert systems. However, the low availability of LISP on a wide variety of conventional computers, the high cost of state-of-the-art LISP tools, and the poor integration of LISP with other languages, led National Aeronautics and Space Administration’s (NASA’s) computing department to the development of another expert system shell: C Language Integrated Production System (CLIPS) (see Section 3.6). Since LISP has been available in different variants, American National Standards Institute (ANSI) standardized a dialect of the LISP programming language in 1994 called Common Lisp (CL) [2]. Common Lisp is a language specification rather than an implementation. An example follows: (defun square (x) (* x x)) (let ((x 3) (y 7)) (+ (square(x) y)) The code defines a function (square) that returns the square of a variable (x), assigns (let) values to the variables x and y, and performs a calculation using the square of x and adding it to y. The results in this case would be 16. As the name list processing suggests, the strength of this language lies in the definition, manipulation, and evaluation of lists that are defined in a simple statement: (:benzene “aromatic” :cyclohexane “cycloaliphatic”) This is a property list, where keywords (those preceded by a colon) have values assigned that can be retrieved by using a retrieval function (getf): (getf :benzene) “aromatic” Few chemistry specific dialects have been developed, one of which is the ChemLisp language interpreter in Apex-3D, an expert system for investigating structure– activity relationships. ChemLisp represents a special dialect of the LISP language. ChemLisp accesses main data structures and modules containing basic algorithmic functions.
5323X.indb 40
11/13/07 2:09:29 PM
Development Tools for Expert Systems
41
3.5 Programming Logic (PROLOG) Logic programming and one of its representatives, programation et logique (or programming and logic; PROLOG), make a declarative approach to writing computer programs. PROLOG was designed by a team around Alain Colmerauer, a French computer scientist, in the 1970s with the goal of creating a tool for communicating with computer software in a natural language [3]. The development was divided into two parts: a system for automatic deduction, developed by Jean Trudel and Phillipe Roussel, and a natural language interface, taken over by Colmerauer and Robert Pasero [4]. A PROLOG program consists of a database including facts and rules, which are evaluated systematically when a query is defined by a user. The result of this evaluation can be either positive or negative. The outcome is positive if a logical deduction can be made; otherwise it is negative if the current database does not allow a deduction. PROLOG, as a declarative programming language, allows communication with a PROLOG program in some kind of natural language. In contrast to imperative programming languages, like C++, C#, or Java, the rule-based approach does not require a fixed-path solution but creates the solution by deduction from rules. In general, logic programs use two abstract concepts: (1) truth; and (2) logical deduction. A PROLOG program can be asked whether a fact is true or whether a logical statement is consistent with the rules stored in the program. These questions can be answered independently of a concrete hard-coded code. Facts consist of a name (or functor) similar to a variable name in other programming languages and one or more arguments that define the corresponding values.
3.5.1 PROLOG Facts PROLOG facts might be the following: compounds(benzene, aromatic). compounds(toluene, aromatic). compounds(xylene, aromatic). compounds(hexane, aliphatic). The first fact indicates that benzene is an aromatic member of a series of compounds. Another example is the following: boilingPoint(benzene, 353.2) where benzene has a value in the list of boiling points. Note that the facts can be arranged in any appropriate way. For instance, aromaticCompounds(benzene) would be another way of declaring that benzene is an aromatic compound.
5323X.indb 41
11/13/07 2:09:29 PM
42
Expert Systems in Chemistry Research
3.5.2 PROLOG Rules A PROLOG rule is an if–then construction; in PROLOG notation this would look like the following: aromaticCompound(X) : coplanar(X), cyclicConjugated(X), hueckelElectrons(X) The rule is interpreted as X X X X
is an aromatic compound if is coplanar AND is cyclic conjugated AND follows the Hueckel rule (4n+2 pi electrons).
Here X is a variable, where the notation requires an uppercase character in PROLOG — variables are not similar to those in conventional programming languages. If a set of facts and rules have been defined, a PROLOG program can be executed, which leads to a question-and-answer scenario. PROLOG tries to interpret the facts and rules to answers with either yes, no, or a predefined answer. For example, the following could be asked: ?- compounds(benzene, aromatic) which would result in the answer yes, since this has been stored as a fact before. Questions can have variables in them, which may get bound to a particular value when PROLOG tries to answer the question. PROLOG will display the resulting bindings of all the variables in the question. So for a variable CompoundType (note the uppercase notation for variables), we might have the following: ?- compounds(toluene, CompoundType). CompoundType = aromatic We can also ask the question the other way around: ?- compounds(Compound, aromatic). Compound = benzene Compound = toluene Compound = xylene Or we can ask for all compounds ?- compounds(Compound, CompoundType). Compound = benzene CompoundType = aromatic Compound = toluene
5323X.indb 42
11/13/07 2:09:29 PM
Development Tools for Expert Systems
43
CompoundType = aromatic Compound = xylene CompoundType = aromatic Compound = hexane CompoundType = aliphatic PROLOG systematically goes through all its facts and rules and tries to find all the ways it can associate variables with particular values so that the initial query is satisfied. As an example of how rules are used, suppose we ask the following question: ?- aromaticCompound(A) and we have defined the facts coplanar(cyclopentadiene) coplanar(benzene) hueckelElectrons(benzene) hueckelElectrons(cyclohexene) cyclicConjugated(benzene) and the rule stated previously. In this case, PROLOG will respond to A = benzene PROLOG matches aromaticCompound(A) against the head of the rule aromaticCompound(X). The first condition in the rule coplanar(X) can be satisfied with cyclopentadiene; that is, cyclopentadiene is bound to A. The next condition cyclicConjugated(X) does not apply, so PROLOG goes back to the first condition, where it now binds benzene to A. PROLOG checks all facts that match the rule (the three conditions) until it finds an answer; the only fact matching all three rules for aromaticity is benzene. The concept of other programming languages for expert systems is very similar to the one just introduced; however, the syntax and the individual functionalities differ.
3.6 National Aeronautics and Space Administration’s (NASA’s) Alternative — C Language Integrated Production System (CLIPS) CLIPS is a programming language shell developed in 1985 at the NASA Johnson Space Center [5]. The original intent for CLIPS was to improve knowledge about the construction of expert system tools to form the basis for replacing the existing commercial tools. Just one year later it became apparent that CLIPS, as a low-cost expert system tool, would be an ideal replacement for existing systems based on LISP. The first version made available to groups outside of NASA was version 3.0. Originally, the primary representation methodology in CLIPS was a forward-chaining rule language based on the Rete algorithm. In version 5.0 CLIPS procedural programming,
5323X.indb 43
11/13/07 2:09:30 PM
44
Expert Systems in Chemistry Research
as found in C, and object-oriented programming, like in C++, were introduced. The latter is implemented in the CLIPS Object-Oriented Language (COOL). Version 6.0 of CLIPS further supported the development of modular programs as well as improved integration between object-oriented and rule-based programming capabilities. Now, CLIPS is available in version 6.24 and is maintained independently from NASA as public domain software [6]. CLIPS allows for handling of a wide variety of knowledge and supports three programming paradigms: (1) rule based; (2) object oriented; and (3) procedural. Rule-based programming represents knowledge as heuristics, which specify a set of actions to be performed for a given situation. The object-oriented approach allows dividing complex systems to be modeled as modular components that can be reused for other purposes. Procedural programming is supported in a similar manner like in conventional programming languages. CLIPS program code can be embedded within procedural code written in C, Java, Fortran, and ADA and provides a series of protocols for integration with software written in other programming languages. Although CLIPS is written in C, it can be installed on different operating systems, like Windows 95/98/NT/2000/XP, MacOS X, and UNIX, without code changes. It can be ported to systems with ANSIcompliant C or C++ compilers. CLIPS can be easily extended, ported, and integrated with existing systems and databases. These are some of the reasons why CLIPS has been well accepted in the scientific community throughout government, industry, and academia. The development of CLIPS has strongly supported the acceptance of expert system technology for a wide range of applications and diverse computing environments. NASA uses these tools extensively to solve problems in scientific and military areas, and several governmental institutions, universities, and private companies are taking advantage of this language. Additional software tools have been created to support development of expert systems. A good example is the parallel production system (PPS), developed by Frank Lopez during his thesis at the University of Illinois [7]. PPS is designed for creating modular expert systems on single or multiprocessor architectures. The resulting modules, called expert objects, are knowledge sources that include communication features for the expert. When communicating with a module, the expert does actually not see any physical architecture of the computer since PPS adapts automatically to it. Users may create a graph of how the expert objects are to communicate on a blackboard structure. In addition, the language of PPS is simple but powerful. Lopez, who later became the founder of Silicon Valley One (http://www. siliconvalleyone.com), also developed OPS-2000. This is an interactive, rule- and object-based software development environment and is the first knowledge engineering tool designed for the C++ programming language. Let us have a closer look at a simple interaction with the CLIPS command interface.
3.6.1 CLIPS Facts The (assert) command can be used to add facts to the system: CLIPS> (assert (aromatic))
5323X.indb 44
11/13/07 2:09:30 PM
Development Tools for Expert Systems
45
The user interface responds with a statement giving the number of facts (starting with zero) currently available in the systems. The facts can be listed by using the (facts) command: CLIPS> (facts) f-0 (aromatic) For a total of 1 fact. Adding a list of facts can also be created in a single statement: CLIPS> (assert (aliphatic) (polycyclic) (linear)) CLIPS> (facts) f-0 (aromatic) f-1 (aliphatic) f-2 (polycyclic) f-3 (linear) For a total of 4 facts. In this case, the facts such as (aromatic) and (aliphatic) have no names; they are single-field facts. To name the fact, the following syntax can be used: CLIPS> (assert (compoundType aromatic aliphatic polycyclic linear)) CLIPS> (facts) f-0 (aromatic) f-1 (aliphatic) f-2 (polycyclic) f-3 (linear) f-4 (compoundTypes aromatic aliphatic polycyclic linear) For a total of 5 facts.
3.6.2 CLIPS Rules A rule in CLIPS uses the following syntax: CLIPS> (defrule stability (compoundType aromatic) => (assert (stability high))) Here (defrule) is the command for defining a new rule, which has the header stability. The next line is the left-hand side of the rule, here consisting of a single condition. The => notation is similar to a then statement in a conventional programming language. The result of a successful match is to perform the action described in the fourth line, which is the creation of a new fact. The system interprets this rule as follows:
5323X.indb 45
11/13/07 2:09:30 PM
46
Expert Systems in Chemistry Research
If compound type is aromatic, then stability is high. The user interaction scenario is similar to the one described previously. We can have a look at the rule by using the (ppdefrule) command: CLIPS> (ppdefrule stability) (defrule MAIN::stability (compoundTypes aromatic) => (assert (stability high))) Looking at the facts currently stored, we will get the following: CLIPS> (facts) f-0 (aromatic) f-1 (aliphatic) f-2 (polycyclic) f-3 (linear) f-4 (compoundTypes aromatic aliphatic polycyclic linear) For a total of 5 facts. To execute our program, we just need to enter the (run) command and to check the facts: CLIPS> (run) CLIPS> (facts) f-0 (aromatic) f-1 (aliphatic) f-2 (polycyclic) f-3 (linear) f-4 (compoundTypes aromatic aliphatic polycyclic linear) f-5 (stability high) For a total of 6 facts. The program used the given facts and matched them to the given rules and created a new fact. Basically any programmable action can take place as a result of a positive rule match. For instance, another set of rules could be evaluated; facts could be added, deleted, or changed; or even an external program could be called to trigger any other external action, like a data analysis. CLIPS has a series of extensions to integrate additional technologies, many of them also developed by NASA staff. Also, several scientific institutions have worked on extensions. FuzzyCLIPS, for instance, was developed by the Integrated Reasoning Group at the NRC Institute for Information Technology. FuzzyCLIPS enhances CLIPS by providing a fuzzy reasoning capability that can integrate with CLIPS facts and the CLIPS inference engine. It allows representing and manipulating fuzzy facts
5323X.indb 46
11/13/07 2:09:30 PM
Development Tools for Expert Systems
47
and rules. It handles fuzzy reasoning and supports uncertainty concepts and allows combining nonfuzzy and fuzzy algorithms in the rules and facts of an expert system. DYNAmic CLIPS Utilities (DYNACLIPS) is another set of tools that can be linked with CLIPS providing dynamic knowledge exchange and agent tools implemented as a set of libraries. Expert systems developed with CLIPS may be executed as follows: • Interactively using a text-oriented command line interface or a Windows interface with mouse interaction. • Batch-oriented using a text file that contains a series of commands that can be automatically read from a file when CLIPS is started. • Embedded in an external program, where calls can take advantage of the CLIPS program. In this case, the user interfaces are provided by the external program. The generic CLIPS interface works similar to the one previously described for PROLOG. The knowledge base is usually created with a standard text editor and is saved as one or more text files. Executing CLIPS then loads the knowledge base and begins with the question–answer user interaction scenario. The CLIPS user interface allows for changing rules and facts on the fly, for viewing the current state of the system, and for tracing the execution. A window interface is available for the Macintosh, Windows, and X Window environments.
3.7 Java-Based Expert Systems — JESS Java Expert System Shell (JESS) is a superset of the CLIPS programming language for the Java platform developed by Ernest Friedman-Hill of Sandia National Laboratories, a National Nuclear Security Administration laboratory, in late 1995 [8]. JESS provides rule-based programming for automating an expert system and is often referred to as an expert system shell. Rather than a procedural paradigm, where a single program has a loop that is activated only one time, the declarative paradigm used by JESS continuously applies a collection of rules to a collection of facts by a process called pattern matching. Rules can modify the collection of facts, or they can execute any Java code. Each JESS rule engine can incorporate a series of facts, which are stored in the working memory. Rules react to additions, deletions, and changes to the working memory. JESS distinguishes between pure facts, which are defined by JESS, and shadow facts, which are connected to Java objects. Facts are based on templates, similar to Java classes, which consist of a name and a series of slots that represent the properties of the class. A definition of a template might look like this: (deftemplate molecule “A generic molecule.” (slot name) (slot formula)
5323X.indb 47
11/13/07 2:09:30 PM
48
Expert Systems in Chemistry Research
(slot nAtoms (type INTEGER)) (slot state (default liquid))) A fact is then defined by using the (assert) command: (assert (molecule (name Benzene) (formula C6H6) (nAtoms 12))) Using a shadow fact that derives from a Java class called Molecules requires a reference to this class in the template: (deftemplate Molecule (declare (from-class Molecule))), where (Molecule) is the template name as well as the name of the corresponding Java class. If both are identical, the template automatically generates slots that correspond to the properties of the Java class. If the Java class provides a method getFormula(), the corresponding slot (formula) is automatically generated in the template. JESS uses a special Java class (java.beans.Introspector) to find the property names. A JESS rule is declared similarly to the CLIPS notation (see previous example). JESS can be used to build Java servlets, Enterprise JavaBeans (EJBs), applets, and full applications that use knowledge in the form of declarative rules to draw conclusions and to make inferences. Since many rules may match many inputs, there are few effective general-purpose matching algorithms. The JESS rules engine uses the Rete algorithm for pattern matching. The most current version, JESS 7.0, includes a series of new tools, like an integrated development environment for rules that features tools for creating, editing, visualizing, monitoring, and debugging rules. JESS 7.0 executes rules written both in its own expressive rule language and in eXtensible Markup Language (XML).
3.8 Rule Engines — JBoss Rules JBoss Rules (also referred to as Drools) is an open-source business rules engine designed for implementing business rules according to business policies [9]. It consists basically of a rules engine that allows for viewing and managing business rules encoded in a software application as well as validating the implemented rules against documented business rules. JBoss Rules is more correctly classified as a production rule system, which works with a memory of current states and details of a problem, a rule base, and an interpreter (i.e., inference engine), which applies the rules to each fact entering the memory. The system implements both Rete and leaps pattern-matching algorithms. Rules are stored in the production memory, and the facts that the inference engine matches against are stored in the working memory. A typical syntax is as follows: rule “Strength of Ternary Acids” when
5323X.indb 48
11/13/07 2:09:31 PM
Development Tools for Expert Systems
49
Molecule(nOxygen) && Molecule(nOxygen > Molecule(nHydrogen) + 2) then Strength = “strong” else Strength = “weak”; end The strength of a ternary acid is estimated from the difference between the number of oxygen atoms and hydrogen atoms. Facts are asserted by a simple assignment statement, similar to object-oriented programming languages: Molecule aspirin = new Molecule(“Acetylsalicylic acid”); aspirin.setOxygen(4); aspirin.setHydrogen(8); Here (aspirin) is an instance of a molecule and receives two values for the number of oxygen and hydrogen atoms. A fact that is asserted is directly examined for matches against the rules; consequently, aspirin would be interpreted as weak acid. Facts are asserted into the working memory where they may then be modified or retracted. A system with a large number of rules and facts may result in many rules being true for the same fact assertion; these rules are said to be in conflict. The execution order of these conflicting rules is managed using a conflict resolution strategy based on, for instance, the time when the fact has been asserted. The most current version 3.0 of JBoss Rules is available on the World Wide Web.
3.9 Languages for Knowledge Representation From a formal point of view, a knowledge representation language (KRL) incorporates syntactic (i.e., notational) and inferential aspects. The syntactic aspect covers the explicit format used for storing knowledge, whereas the inferential part deals with the use of knowledge for reasoning. Knowledge representation languages typically encompass business logic, production rules, semantic networks, and frames. One of the examples is KRL, developed by Bobrow and Winograd from Stanford University [10]. KRL supports knowledge representation in different declarative forms based on conceptual objects that form a network. KRL is basically an interpreter that allows LISP expressions to be formulated in a more expressive fashion. It includes an inference engine with a backward-chaining mechanism. KL-ONE is a frame-like family of knowledge representation approaches in the tradition of semantic networks [11]. It was developed to overcome some of the drawbacks of semantic networks and represents conceptual information in a structured network for inheritance. The frames — in the KL-ONE approach called concepts — are arranged in a class-like hierarchy that includes the relations between frames. Frames are typically inherited from super classes.
5323X.indb 49
11/13/07 2:09:31 PM
50
Expert Systems in Chemistry Research
3.9.1 Classification of Individuals and Concepts (CLASSIC) CLASSIC is a successor of the KL-ONE approach. The framework allows representing concepts, attributes, objects, and rules in either primitive frames or in object-oriented manner [12]. Concepts underlie an automatic generalization, and objects are automatically made instances of all concepts for which they pass a membership test. CLASSIC notation uses the following terminology to represents concepts, instances, and attributes. • Roles are attributes, such as author, color, or material, and are stored in the frames’ slots. • Individuals are instances or objects that have roles (i.e., slots) assigned to them, such as molecule, container, or matrix. • Concepts are combining individuals and attributes to describe an item. • Rules provide the guiding principles for computing individuals based on their roles. A series of operators — such as AND, FILLS, ONE-OF, ALL, and AT-LEAST — allow for comparison and for defining restrictions on attributes. Individuals are specified in the following manner: Wasp -> SPECIES Philanthotoxin -> TOXIN-VENOM Wasp-Philanthotoxin -> (AND TOXIN (FILLS insect Wasp) (FILLS toxin Philanthotoxin) (FILLS LD50 2.4) (FILLS species Vespidae) (FILLS type Venom)) The first two statements define instances of an object: “Wasp is a species”; “Philanthotoxin is a toxin of the class of venoms.” The third line defines a specific instance of the wasp philanthotoxin and adds several attributes in terms of slots. A concept may be defined as follows: VENOM-SPECIES <=> (AND VENOM-PROPERTY (ONE-OF Apidae Mutillidae Vespidae Formicidae)) VENOM-TOXICITY <=> (AND VENOM-PROPERTY (ONE-OF High Medium Low)) TOXIN => (AND MOLECULE (AT-LEAST 1 toxicity) (ALL toxicity TOXIN-TOXICITY) (AT-LEAST 1 species) (ALL species VENOM-SPECIES) WASP-TOXINE <=> (AND TOXIN (FILLS species Vespidae))
5323X.indb 50
11/13/07 2:09:31 PM
Development Tools for Expert Systems
51
Here, two instances of venom — species and toxicity — are assigned a list of possible values. The concept of toxin is finally derived from the concept of molecules and includes certain constraints for toxicity and species. Finally, a specific instance of the wasp toxine receives a constraint for the corresponding species. This kind of knowledge representation is designed for applications where rapid responses to questions are essential rather than expressive power. The CLASSIC is available in the original LISP version that was developed for research purposes, as well as in C and C++, the latter of which is called NeoCLASSIC.
3.9.2 Knowledge Machine Knowledge machine (KM) is a frame-based knowledge representation language similar to KRL and other KL-ONE representation languages such as Loom and CLASSIC [13–15]. In KM, a frame denotes either a class (i.e., type) or an instance (i.e., individual). Frames have slots, or binary predicates, in which the fillers are axioms about the slot’s value. These axioms have both declarative and procedural semantics, allowing for procedural inference. KM, besides the knowledge representation aspects, also features for reasoning with a variety of representations, such as frames, patterns, constraints, and metaclasses. It provides support for a series of representations such as patterns, contexts, constraints, situations, and metaclasses. A main difference between it and other approaches is the ability of handling value sets instead of single values for search and assignment of slots. This enables coreferences between statements at different levels of abstraction to be correctly determined. KM includes a mechanism for producing justifications for the reasoning process based on a proof tree. Similar to object-oriented programming, KM provides classes and instances thereof. Properties of members of a class are described by the following: (every has (<slot1> (<expr11> <expr12> ...)) (<slot2> (<expr21> <expr22> ...)) ... ) For example, the class Steroid could look as follows: (every Steroid has (fusedRing-count (4)) (stereocenters-count ((a Number))) (properties ((a Spectrum) (a Descriptor)))) defining a fixed value for the number of fused rings, a number of stereocenters, which is itself a class of type number, and the two properties, derived from the classes Spectrum and Descriptor. We can now define the class Spectrum in the following manner: (every Spectrum has (type (a SpectrumType)) (nucleus (a Nucleus))
5323X.indb 51
11/13/07 2:09:31 PM
52
Expert Systems in Chemistry Research
A specific instance of descriptor can look as follows: (*HNMR has (instance-of (Spectrum)) (type *NMR) (nucleus *H) (frequency (400)) (solvent (*CDCL3))) We can then generate a steroid instance and query the slots by using the following syntax: > (a Steroid) (_Steroid0) > (the properties of _Steroid0) (_Spectrum1 _Descriptor7) The first expression creates an anonymous instance, or Skolem constant, from the class Steroid at run time, denoted by the underscore and an instance number. These anonymous instances are also created for the Spectrum and the Descriptor. A named instance can be created by (*Cholesterol has (instance-of Steroid) (stereocenters-count (8)) (Spectrum (*HNMR)))) A query is expressed in the form the slot of instance: >(the stereocenters-count of ((*Cholesterol))) (8) This type of query has a single access path, since it derives its answer with a single step. A query with an access path of 2 would look as follows: >(the type of (Spectrum of (*Cholesterol))) (*HNMR) These access paths are already a simple form of a rule. Additional conditional expressions may be used and inserted in a slot, forming a more complex rule: (every Steroid has (diffusion ( (if ((the value of (the lipophilicity of (Self)) > 3) and (the polarizability of (Self)) = *High))
5323X.indb 52
11/13/07 2:09:31 PM
Development Tools for Expert Systems
53
then *Medium else *Low)))) In this example a steroid tends to diffuse through a membrane if it the lipophilicity exceeds a certain value and the polarizability is high; otherwise, the diffusion would be described as low. An instance may inherit information from multiple generalizations. The semantics of KM dictate that the instance will acquire all the properties of all its generalizations; that is, all axioms applying to its generalizations also apply to the instance. Consequently, KM will merge information from all generalizations when computing the slot values of the instance. This process, called unification, is one of the important differences between KM and object-oriented programming languages. Unification of inherited information is fundamental for building a knowledge base in a modular and reusable fashion. Instead of incorporating all information in a single frame, unification distributes information and allows it to be applied to concepts outside the original context.
3.10 Advanced Development Tools Several noncommercial solutions have been in place for several decades. Among them, BABYLON is a modular development environment for expert systems, created at GMD (Forschungszentrum Informationstechnik GmbH) in Germany, which is now integrated in Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. [16]. BABYLON is composed of frames, constraints, a PROLOG-like logic formalism, and a description language for diagnostic applications. It is implemented in Common LISP and runs on Mac and UNIX. The Generic Expert System Tool (GEST) was a blackboard system approach internationally licensed by Georgia Polytechnic Institute for expert system applications [17]. GEST can be used in a variety of problem domains and supports backward and forward chaining. Its knowledge representation schemes include frames, rules, and procedures. Support is also present for fuzzy logic and certainty factors. The user interface uses the Symbolics mouse and menu-driven user interface and runs on Symbolics Lisp Machines, Genera 7.2, and Sun platforms. Most of the noncommercial solutions have not been maintained over time, but even commercial solutions disappeared from time to time due to company mergers. An example is Aion Corporation, the developer of the Aion Development System (ADS). ADS was an expert system shell that supported forward and backward chaining, an object-oriented knowledge representation, and graphics and integrated with other programming languages, like C and Pascal. In 1992 Aion Corporation merged with AICorp Inc. to form Trinzic Corporation. Just three years later, Platinum Technology Inc. acquired Trinzic Corporation, which operated for approximately four years as a unit of Platinum Technology. Finally, Platinum Technology International Inc. was acquired by Computer Associates International Inc. in May 1999. Computer Associates has a series of products in the market that support expert system development and knowledge management. A remainder of the original Aion development is CleverPath Aion Business Rules Expert (BRE), which is an advanced
5323X.indb 53
11/13/07 2:09:32 PM
54
Expert Systems in Chemistry Research
rules engine to capture knowledge in manageable rules for business processes [18]. Equipped with a powerful inference engine, BRE supports dynamic rules that allow a fast adaptation to changing business scenarios and comes in component-based development design. This proven solution reduces development and maintenance efforts through easy-to-maintain decision tables, dynamic rules, and a sophisticated inference engine. Particularly interesting with BRE is the rule manager, which enables both business and information technology (IT) users to build and maintain rules without programming. Rules are presented in intuitive formats and can be created and adapted in a Windows-like user interface. The vocabulary can be adapted to the actual business approach. BRE allows organization of rule projects — entities that allow the user to organize rules in a hierarchical manner that looks similar to file-and-folder view. A rule definition wizard guides users through the steps of rule definition. BRE also provides features for rule validation using test cases as well as an administrative approval process including reviewing and deployment of rules. All of these features are covered by a user management system that ensures that individual users have the right access, create, modify, and approve rules in their domain. BRE thus provides a typical system for business applications. The most current version, 10.2, has an extended database and platform support. EXSYS Inc. in Albuquerque, New Mexico, provides several interesting tools for expert system development [19]. For instance, EXSYS Professional is a rulebased expert system shell and educational tool that comes with many examples and a good tutorial on developing expert systems. It features backward chaining, forward chaining, and fuzzy logic. It provides a SQL interface to link to databases and integrates with spreadsheet programs and runs on DOS, Windows, Macintosh, UNIX, and VAX platforms. EXSYS CORVID is a knowledge automation expert system development tool where decision steps are described by building logic diagrams. It provides object-structured knowledge representation, backward and forward chaining, and fuzzy logic. It can run either stand alone or via Java applet or servlet. It supports database integration and help desk systems and runs on Microsoft (MS) Windows 95/98/NT/2000/ME/XP; the servlet runs on servers with Apache Tomcat or compatible servlet engines with Java Runtime v1.3 or higher. Other companies, like Intellicorp located in California, did not continue to develop their tools. Intellicorp meanwhile markets diagnostic and system management software for organizations using software from SAP AG [20]. IntelliCorp continued to market the LISP-based knowledge engineering environment (KEE), which was originally introduced in August 1983, until 1999 when it was announced that they maintain KEE but do no further development on the product [21]. Knowledge engineering environment is a frame-based expert system supporting dynamic inheritance, multiple inheritance, and polymorphism. It provides a knowledge base development tool, utilities for the interface with the user, and graph drawing tools for the knowledge base and execution. The methods are written in LISP, and the inference engine provides both forward and backward chaining. The system allows for linking to external databases and provides an extensive graphical user interface. NEXPERT OBJECT, a system originally developed by Neuron Data (later renamed to Blaze software), was one of the first systems written in Pascal and
5323X.indb 54
11/13/07 2:09:32 PM
Development Tools for Expert Systems
55
introduced on an Apple Macintosh personal computer in 1985, at a time when most available products were coded in LISP and only available on large computers [22]. In 1992, a major version, 3.0, was released, and 25,000 installations were applied mainly for modeling business tasks, corporate decision making, and building knowledge repositories. NEXPERT OBJEXT represented domain specific knowledge explicitly as networks of rules and class-object hierarchies. The rule-based and object-based representations are tightly integrated in a common design environment and are supported by visual editors and graphical browsers. The system’s inference engine offers one of the largest sets of reasoning techniques. Besides forward and backward chaining, examples are as follows: • Nonmonotonic reasoning: The ability to infer alternate conclusions from new facts. • Defeasible reasoning: The ability to suspend or retract deductions. • Multiple inheritance: To propagate information through class-object hierarchies and exception and uncertainty handlers. NEXPERT OBJECT was probably one of the most mature systems concerning the integration capabilities with software on a corporate level. It offered an application programming interface (API) that supports industry standard programming languages, such as C, C++, Smalltalk, or LISP. Neuron Data was acquired by Brokat in 2000 and was subsequently sold to HNC Inc., which merged in 2002 with the Fair Isaac Corporation.
3.10.1 XpertRule A particularly interesting software package available on the market is XpertRule, a series of Windows-based expert system development tools that use genetic algorithms for optimization [23]. XpertRule Knowledge Builder, for instance, is an environment for developing and deploying knowledge-based applications and components. Knowledge Builder supports a broad range of knowledge representations and graphical knowledge building blocks to define them. In particular, decision making (Decisioning) is supported for diagnostic applications, selection processes, assessment, monitoring, and workflow applications. The knowledge in such applications is represented by rules or decision trees. The decisions are derived from attributes, which are captured from the user through question–answer dialogs or covered by calculations. Decision trees relate a decision to a number of attributes; a table of cases contains a list of examples or rules, each showing how an outcome or decision relates to a combination of attribute values. The system uses case-based reasoning to be used in diagnostic applications, as well as fuzzy logic for applications like performance assessment and diagnostics. Knowledge Builder uses graphical knowledge representation to simplify the process of capturing business knowledge. Several additional software packages are available in Xpert Rule; among them the XpertRule Miner is a dedicated data mining software product, which allows deriving decision-tree rules or patterns from data files using data-mining technologies.
5323X.indb 55
11/13/07 2:09:32 PM
56
Expert Systems in Chemistry Research
Analyser is a machine-learning add-on to XpertRule that uses genetic algorithms to optimize solutions (http://www.attar.com). Xpert Rule is widely used in industrial and scientific applications, one of them performed in cooperation with NASA and Rockwell Aerospace. It is called NASA’s Contamination Control Engineering Design Guidelines Expert System and was developed by Rockwell International’s Space Systems Division. It was developed for education in contamination control processes and is designed as an interactive guide to assist with quantifying contamination for sensitive surfaces. The tool enables the user to quantify molecular and particulate contamination requirements for solar arrays, thermal control surfaces, or optical sensors [24].
3.10.2 Rule Interpreter (RI) RI is a program that supports the development and execution of rule scripts, written in rule description language (RDL) [25]. RDL combines substructure search with descriptor-oriented selection, incorporates Boolean logic, and allows the generation of a tree-like decision structure. RI was written in Delphi code under MS Windows and derived from the OASIS SAR system [26,27]. Rule scripts operate on substances defined in a data file in either SMILES (simplified molecular input line entry specification) or CMP (compound) format. The conventional SMILES notation as developed by Weininger [28] provides a basic description of molecules in terms of two-dimensional chemical graphs. The CMP file format developed with the OASIS system [29] provides separate logical records for information about connectivity, three-dimensional structure, electronic structure from quantum-chemical molecular–orbital computations, as well as physicochemical and experimental toxicological data. The output of RI is files in SMILES binary CMP files, including toxicochemical classes based on descriptors and generates toxic potency estimates. The RDL uses SMILES as a basic format and includes extensions. An RDL script consists of two definition sections and an application section. The Define Section contains definitions of screens in an extended SMILES format that includes qualifiers. Qualifiers are enclosed in braces in the structure definition, succeeding the atom or bond entries to which they belong. They may denote ionic state, hybridization, participation in rings, number of adjacent protons, and chiral parity for an atom. An example is as follows: c1c{H}c(C{sp3})c{H}cc1cl; SMILES: c1cc(C)ccc1cl It specifies the presence of a para-chlorotoluene moiety and excludes any orthosubstituents. Bond qualifiers define a particular bond configuration, cis or trans for a double bond. Descriptor qualifiers can be used to restrict values of, for instance, physicochemical descriptors of an atom: a qualifier like (–0.2 < q) identifies a charge descriptor with a partial atomic charge less than 0.2. In addition to predefined descriptors, reserved key can be used for dynamic descriptors that depend on the screen context. Examples are the reserved key enumerate, which denotes the frequency of the preceding substructure occurring in a molecule, and distance, which denotes the distance between the geometric centers of two substructures.
5323X.indb 56
11/13/07 2:09:32 PM
Development Tools for Expert Systems
57
The Rule Section contains the definitions of rules that are applied to a substructure. The rule is enclosed in quotation marks and uses the logical operators AND, OR, and NOT to return either TRUE or FALSE as result. Rules are organized in expressions, and their priority of execution is determined by the enclosing brackets. A rule identifier ensures the unique identification and reuse of rules in other expressions. An example is as follows: Rbenz: ‘Benz’ and not ‘Polyarom’ where Rbenz is the identifier for the rule, and Benz and Polyarom are definitions for benzene and polyaromtic substructures. The Apply Section represents the business logic and uses the definitions and rules described in the previous section. The decision scheme is constructed of assignment, conditional, or compound statements. An assignment statement assigns either a numerical value or results of a screening definition to a molecular descriptor, similar to conventional programming languages. Conditional statements represent the nodes in a decision tree in the form of if–then–else statements and can be combined as a compound statement. A text editor provides special functionality for creating and editing rule script files, as well as for syntax checking and script application. The software incorporates a two-dimensional chemical model builder that uses SMILES as input. RI has been applied in different areas, such as for determining acute fish toxicity and for determining androgen receptor binding affinity. Though RI is a script-based system requiring some expertise in writing and evaluating scripts and rules, it is a good example of a robust and flexible tool for screening of chemical databases and inventories.
3.11 Concise Summary Action is one of the possible results of the reasoning process in an expert system. In contrast to assignments, actions are triggered processes that lead to a change in the software environment. C Language Integrated Production System (CLIPS) is a programming language shell designed for rule-based, object-oriented, and procedural software development. CLIPS Facts are assignment statements in CLIPS representing an actual state or property in the form: assert(X, value1, value2, …). CLIPS Rules are conditional statements in CLIPS used in the inference process in the form: defrule rulename (if-statement condition) => (then-statement). Declarative Programming is a programming paradigm that describes computations by specifying explicit objectives rather than the procedure on how to achieve them. Examples are SQL, PROLOG XSLT (see also Imperative Programming). Expert System Shell is a suite of software that allows construction of a knowledge base and interaction through use of an inference engine. Imperative Programming is a programming paradigm that describes computation as sequential statements that change a program state. Examples are FORTRAN, Pascal, C, and Visual Basic (see also Declarative Programming).
5323X.indb 57
11/13/07 2:09:33 PM
58
Expert Systems in Chemistry Research
Inference Engine is a module of an expert system that performs reasoning using rules from the knowledge base and facts from the working memory to process them according to specified business rules. JBoss Rules is an open-source production rule system designed for implementing business rules according to business policies. JESS is a superset of the CLIPS programming language for the Java platform using pattern matching to continuously apply rules to facts. Knowledge Base is a part of the expert systems memory that stores domain-specific knowledge in rules, frames, or other types of knowledge representations. List Processing (LISP) is a programming language that uses an interpreter for symbolic calculations based on single-scalar values (atoms) and associative arrays (lists). Production Rule System is a computer program using productions, which include a basic representation of knowledge and rules. Productions can be executed and triggered to perform an action. Programation et Logique, or Programming and Logic (PROLOG) is a declarative programming language based on facts and rules that are evaluated systematically to form a logical deduction. PROLOG Facts are assignment statements in PROLOG representing an actual state or property in the form: property(X, value). PROLOG Rules are conditional statements in PROLOG used in the inference process in the form: Then-Statement(X) :- if-condition1(X), if-condition2(X), …. Rule Description Language (RDL) is a language developed for the Rule Interpreter (RI) software that combines substructure search with descriptor-oriented selection. Rule Interpreter (RI) is a program that supports the development and execution of rule scripts, written in the rule description language (RDL). Working Memory is a part of the expert systems memory that stores specific information on a current problem represented as facts. XpertRule is a series of Windows-based expert system development tools, which use genetic algorithms for optimization.
References
5323X.indb 58
1. Winston, P.H. and Horn, B.K., Lisp, Addison-Wesley, Reading, MA, 1989. 2. Steele, G.L., Common Lisp: The Language, 2nd ed., Digital Press, Bedford, MA, 1990. 3. Colmerauer, A., et al., Un systeme de communication homme-machine en francais, Internal report, Croupe d’Intellengence Artificielle, Universite Aix-Marseillo II., June 1973. 4. Colmerauer, A., Metamorphosis Grammars, in Natural Language Communication with Computer, Bolc, L., Ed., Springer Verlag, Berlin, 1978. 5. Giarratano, J.C. and Riley, G.D., Expert Systems, Principles and Programming, 4th ed., Thomson Course Technology, Boston, 2005. 6. Riley, G., CLIPS — A Tool for Building Expert Systems, http://www.ghg.net/clips/CLIPS. html. 7. Lopez, F., The Parallel Production System, Ph.D. thesis, University of Illinois at Urbana-Champaign, 1987.
11/13/07 2:09:33 PM
Development Tools for Expert Systems
5323X.indb 59
59
8. Friedman-Hill, E., Jess in Action — Java Rule-Based Systems, 1st ed., Manning Publications Co., New York, 2003. 9. JBoss, JBoss Rules, http://www.jboss.com/products/rules. 10. Bobrow, D. and Winograd, T., An overview of KRL, a Knowledge Representation Language, Cognitive Sci., 1, 1, 1977. 11. Brachman, R.J. and Schmolze, J., An Overview of the KL-ONE Knowledge Representation System, Cognitive Sci., 9, 171, 1985. 12. Brachman, R.J., et al., Living with CLASSIC: When and How to Use a KL-ONE-Like Language, in Principles of Semantic Networks: Explorations in the Representation of Knowledge, Sowa, J.F., Ed., Morgan Kaufmann Publishers, San Mateo, CA, 1991, 401. 13. MacGregor, R. and Bates, R., The LOOM Knowledge Representation Language, Tech. Report RS-87-188, Information Sciences Institute, University of Southern California, Los Angeles, CA, 1987. 14. Clark, P. and Porter, B., Building Concept Representations from Reusable Components, in Proceedings of the 14th National Conference on Artificial Intelligence, Providence, RI, 1997, 369. 15. Clark, P. and Porter, B., KM — The Knowledge Machine: Users Manual, http://www. cs.utexas.edu/users/mfkb/km/userman.pdf. 16. Christaller, T., Di Primio, F., Schnepf, U., and Voss, A., The AI-Workbench BABYLON — An Open and Portable Development Environment for Expert Systems, Academic Press, New York, 1992. 17. Gilmore, J.F., Roth, S.P., and Tynor, S.D., A Blackboard System for Distributed Problem Solving: Blackboard Architectures and Applications, Academic Press, San Diego, 1989. 18. Computer Associates International, Inc., New York, CA Aion Business Rules Expert, http://www.ca.com. 19. EXSYS Inc., Albuquerque, New Mexico, http://www.exsys.com/. 20. Intellicorp, Menlo Park, CA, http://www.intellicorp.com. 21. Kehler, T. and Clemenson, G., KEE: The Knowledge Engineering Environment for Industry, Systems and Software, 3(1), 212, 1984. 22. Turner, S.R., Nexpert Object, IEEE Expert: Intelligent Systems and Their Applications, 6, 72, 1991. 23. XpertRule Software Ltd., Leigh, UK, http://www.xpertrule.com. 24. Tribble, A.C. and Boyadjian, B.V., An Expert System for Aerospace, PC AI Magazine, 11(3), 16, 1997. 25. Karabunarliev, S., et al., Rule Interpreter: A Chemical Language for Structure-Based Screening, J. Mol. Struct. (Theochem), 622, 53, 2003. 26. Mekenyan, O., Karabunarliev, S., and Bonchev, D., The OASIS Concept for Predicting the Biological Activity of Chemical Compounds, J. Math. Chem., 4, 207, 1990. 27. Mekenyan, O., et al., A New Development of the OASIS Computer System For Modeling Molecular Properties, Comp. Chem., 18, 173, 1994. 28. Weininger, D., SMILES — A Chemical Language and Information System. 1. Introduction to Methodology and Encoding Rules, J. Chem. Inf. Comput. Sci., 28, 31, 1988. 29. Prickett, S.E. and Mavrovouniotis, M.L., Construction of Complex-Reaction Systems .1. Reaction Description Language, Comp. Chem. Engin., 21, 1219, 1997.
11/13/07 2:09:33 PM
5323X.indb 60
11/13/07 2:09:33 PM
4
Dealing with Chemical Information
4.1 Introduction Artificial intelligence, or soft computing, plays an increasingly important role in evaluating scientific data. In particular, analytical chemistry, cheminformatics, and bioinformatics depend on the power of artificial intelligence approaches to store and use the experience and knowledge of the experts in a company efficiently. The extensive use of computational methods leads to a steady increase of data that are barely manageable, even with a team of scientists. Consequently, automated intelligent analysis techniques for primary data are becoming essential, particularly in the research areas of combinatorial chemistry, chemometrics, cheminformatics, and bioinformatics. This chapter provides an overview of techniques used in combination with knowledge- or rule-based approaches.
4.2 Structure Representation Databases for chemical structures store machine-readable representations of the two-dimensional (2D) or three-dimensional (3D) structure models of chemical compounds. Most of the research in the field of representing and searching 2D and 3D chemical structures is based on the Cambridge Structural Database, which is maintained by the Cambridge Crystallographic Data Centre. The Cambridge Structural Database contains structures of small organic and organometallic compounds. The Chemical Abstracts Service (CAS) initiated the first investigations on processing chemical structures. The two-dimensional structure of a chemical compound defines the topology of a molecule — that is, the location of atoms and bonds. To represent 2D chemical structures in machine-readable form, the topology has to be defined in a unique manner.
4.2.1 Connection Tables (CTs) Early approaches used fragmentation codes, which consist of a list of substructures (fragments) defining the molecule [1]. This approach did not cover any information about the connectivity of the substructures, and the assignment of substructures was done manually. A more thorough approach was the connection table (CT), which consists of two arrays: (1) the atom list, which defines the atom symbols or atomic numbers of a molecule; and (2) the bond list, which contains the bonded atoms and the bond order. A connection table is a comprehensive description of the topology of a molecular graph [2]. It can either be redundant, listing each bond in the molecule twice, or nonredundant, where each bond is listed once. To ensure that connection 61
5323X.indb 61
11/13/07 2:09:34 PM
62
Expert Systems in Chemistry Research
tables are unique representations of molecules, the numbering of atoms follows a set scheme. Connection tables are the most common representation and are supported by most structure editor software.
4.2.2 Connectivity Matrices Atom and bond matrices can be used for calculation. A special form of a connection table is the adjacency matrix, which contains information about the connections between atoms without atom types and bond order. The adjacency matrix, M, of a chemical structure is defined by the elements, Mij, where Mij is 1 if atoms i and j are bonded and zero otherwise (Figure 4.1a). In addition, a distance matrix contains the bond length information. The distance matrix, D, of a chemical structure is usually defined by the elements Dij, where Dij is the length of the shortest path from atoms i to j; zero is used if atoms i and j are not part of the same connected component (Figure 4.1b). Distance matrices may also be calculated for real three-dimensional (Euclidean) distances between atoms using the Cartesian coordinates of the atom positions (Figure 4.1c). These matrices allow the calculation of descriptors that account for the shape and conformation of atoms. If conformation is not required or desired, the bond O S Cl Cl (a) Adjacency O1 Cl2 S3 Cl4
O1 — 0 1 0
Cl2 0 — 1 0
(b) Distance S3 1 1 — 1
Cl4 0 0 1 —
(c) Cartesian Distance O1 Cl2 S3 Cl4
Cl4 O1 S3 Cl2 — 2.664 1.421 2.664 2.664 — 2.020 3.065 1.421 2.020 — 2.020 2.664 3.095 2.020 —
O1 Cl2 S3 Cl4
O1 — 2 1 2
Cl2 2 — 1 2
S3 1 1 — 1
Cl4 2 2 1 —
(d) Bond Path Distance O1 Cl2 S3 Cl4
Cl4 O1 S3 Cl2 — 3.441 1.421 3.441 3.441 — 2.020 4.040 1.421 2.020 — 2.020 3.441 4.040 2.020 —
Figure 4.1 Matrices can be used for describing chemical structures in different fashions. The adjacency matrix (a) of thionyl chloride shows whether are bonded to each other. The distance matrix (b) describes the number of bonds between two elements of a structure. The Cartesian distance matrix (c) contains the real three-dimensional (Euclidean) distances between atoms calculated from Cartesian coordinates of the atom positions. The bond path distance matrix (d) contains the sum of bond length between two atoms and is, in contrast to the Cartesian matrix, independent of the conformation of the molecule.
5323X.indb 62
11/13/07 2:09:34 PM
Dealing with Chemical Information
63
path distance matrix is an alternative, which sums up the bond length between the atoms (Figure 4.1d). We will investigate applications of these matrices in the next chapter.
4.2.3 Linear Notations Another approach for representing 2D chemical structures is the linear notation. Linear notations are strings that represent the 2D structure as a more or less complex set of characters and symbols. Characters represent the atoms in a linear manner, whereas symbols are used to describe information about the connectivity [3]. The most commonly used notations are the Wiswesser line notation (WLN) and the simplified molecular input line entry specification (SMILES) [2]. The WLN, invented by William J. Wiswesser in the 1949, was the first line notation capable of precisely describing complex molecules [4]. It consists of a series of uppercase characters (A–Z), numerals (0–9), the ampersand (&), the hyphen (-), the oblique stroke (/), and a blank space.
4.2.4 Simplified Molecular Input Line Entry Specification (SMILES) SMILES was developed by David Weininger at the U.S. Environmental Research Laboratory in Duluth, Minnesota, in 1986 [5]. The simplistic approach of this notation made it to a readily used exchange format for chemical structures. The coding starts with an arbitrary atom and follows the bonds along the molecule. Whereas single bonds are implicit, double and triple bonds are described by the symbols = and #. Atoms are described with their atomic symbol starting with an uppercase character; lowercase characters are reserved for aromatic atoms, and hydrogen atoms are omitted. Chain branches are represented with brackets and closed-ring systems by numbers, and stereochemistry is described by the symbols /, \, and @. Some examples are shown in Table 4.1. Table 4.1 Examples of Chemical Compounds and Their SMILES Notation
5323X.indb 63
Trivial Name
SMILES Notation
Ethane Carbon dioxide Hydrogen cyanide
CC O=C=O C#N
Triethylamine Acetic acid Cyclohexane Benzene Hydronium ion E-difluoroethene L-alanine D-alanine Nicotine Vitamin A
CCN(CC)CC CC(=O)O C1CCCCC1 c1ccccc1 [OH3+] F/C=C/F N[C@@H](C)C(=O)O N[C@H](C)C(=O)O CN1CCC[C@H]1c2cccnc2 C/C(=C\CO)/C=C/C=C(/C)\C=C\C1=C(C)CCCC1(C)C
11/13/07 2:09:35 PM
64
Expert Systems in Chemistry Research
4.2.5 SMILES Arbitrary Target Specification (SMARTS) SMARTS is an extension to SMILES specifically developed for substructure searching [6]. It is notation that allows specifying substructures using rules that are extensions of SMILES. In the SMILES language, there are two fundamental types of symbols — atoms and bonds — that allow specifying a molecular graph. In SMARTS, the labels for the atoms and bonds are extended to include logical operators and special atomic and bond symbols; these allow atoms and bonds to be more general. For example, the atomic symbol [C,N,O] in SMARTS is an atom list for aliphatic C, N, or O atoms; the bond symbol ~ (tilde) matches any bond.
4.3 Searching for Chemical Structures Structure search, also known as identity search, can be used to find out whether the substance is already known or to find references and information on that structure to discover what other work has already been performed on that structure. The advantage of structure search over text searching is that it will find any occurrence of the structure in the database regardless of whether the substance has numerous different names. Whereas fragmentation codes and line notations are excellently suited for data transfer over the Internet — they are just strings — they are not unique in every case and, thus, are unfavorable for structure searches. Connection tables are unique representations for structure and substructure searching ensuring both precise indexing and retrieval. One of the first approaches for substructure search was published by Feldmann et al. [7]. They used connection table representations of 2D chemical structures to search for particular substructures within these larger structures. The resulting system — the National Institutes of Health–Environmental Protection Agency (NIH– EPA) Chemical Information System — started in 1973 as a joint project in mass spectrometry and structure searching between the NIH and the EPA [8]. In many cases scientists search for substructures rather than structures to find candidates with a similar chemical structure or compounds with a common partial structure.
4.3.1 Identity Search versus Substructure Search The difference between algorithms for identity search and substructure search is significant. Finding a structure in a database can be performed by comparing two molecules with a unique enumeration of atoms. Several techniques have been published for unique enumeration of atoms in a molecule, of which the Morgan algorithm [9] is one of the most well known. To prove the identity of consistently enumerated molecules, a direct superimposition by atom-by-atom matching algorithms is performed. However, a unique enumeration does not solve the problem of substructure search. Superimposition of substructures on structures would require the mapping of any combination of molecular graphs to find a graph isomorphism; this is a tedious and time-consuming process. Because the rate of search is always one of the most important limitations for database applications, substructure search should incorporate additional preprocessing steps that restrict the number of molecules to be compared in an atom-by-atom matching algorithm.
5323X.indb 64
11/13/07 2:09:35 PM
Dealing with Chemical Information
65
A substructure is simply a collection of atoms with no order of atoms or bonds being implied; each atom and bond of a molecule occurs only once in the substructure. We normally think of a substructure as a set of atoms interconnected by bonds in a chemically reasonable way. However, a substructure object can be used to represent much more or less than an ordinary substructure; it can also be used to represent less conventional collections, like the number of double bonds in a structure, all of the atoms with an odd number of protons, and so forth. In other words, a substructure object is just an arbitrary set of atoms and bonds; it is up to the programmer using a substructure object to determine the relevance of the set. Finding a substructure, or molecular fragment, in one or more structures in a chemical database corresponds to finding a subgraph isomorphism. The number of structures in the database that needs to be examined can be reduced by preprocessing the database either by clustering the database entries with a common substructure or with a given hyperstructure (e.g., substructure screening) or by using special key codes (e.g., hash codes) to describe the database information relevant to substructures. Finally, the molecular fragment must be superimposed on the database structure at various positions, a process typically performed by the previously mentioned atom-by-atom matching algorithms.
4.3.2 Isomorphism Algorithms A connection table is a simple representation of a labeled graph where the nodes and edges of the graph are the atoms and the bonds of the structure, respectively. In 1976 graph theory was used for describing 2D chemical structures leading finally to the use of isomorphism algorithms for searching chemical structures [10]. An isomorphism algorithm is used in graph theory to determine the extent to which two graphs can be mapped onto each other; this is done by permutation through the vertices of the graph. Three methods are typical. • Graph isomorphism algorithms can be used for mapping a (query) full structure onto a structure to determine identity between two structures. • Subgraph isomorphism algorithms can find a substructure within a structure. • Maximum common subgraph isomorphism algorithms are used to locate the largest common part that two structures have in common. These algorithms are used to find similar structures. A subgraph isomorphism algorithm starts at an initial atom of the query and tries to match it with a database structure. When two atoms are successfully matched, it continues on adjacent atoms; when a mismatch with the query is found, the algorithm traces back to the last matching atom. This method is called atom-by-atom mapping and continues until the query substructure is completely found in the database structure. These and similar approaches for subgraph isomorphism algorithms have been published [11–13]. Due to the nature of this approach, subgraph isomorphism algorithms are time consuming; isomorphism is a combinatorial problem belonging to the nondeterministic polynomial time complete (NP complete) class of problems, which are widely believed to be unsolvable. Several authors suggested improvements to reduce the
5323X.indb 65
11/13/07 2:09:35 PM
66
Expert Systems in Chemistry Research
computational effort, like the concept of parallelism published by Wipke and Rogers in 1984. Parallelism eliminates backtracking by performing individual computational tasks for each atom in a query substructure [14]. Tasks that contain a mismatch end, whereas the other tasks continue. This approach is strongly supported by computers that support parallel processing.
4.3.3 Prescreening One of the most successful attempts may be prescreening, a method that reduces the number of molecules that require the full subgraph search [3]. Prescreening uses bit strings encoding the presence or absence of a fragment in the query and the database molecules. If the database structure bit string contains all the bits set in the query bit string, then that molecule undergoes a subgraph isomorphism algorithm. The screen assignment for database structures is performed on uploading the structure and can be done automatically; therefore, the screen searching part of the substructure searching method is very efficient. However, the definition of the fragments to be encoded may vary from task to task. The substructure search problem can be reduced to a search for all chemical structures containing fragments isomorphic to a query fragment. Testing whether or not a chemical structure contains an isomorphic substructure can be formalized as a subgraph isomorphism problem. Several practical algorithms have been developed for chemical structures. However, substructure search will take a very long time if such an algorithm is applied to all chemical structures in a database. To avoid testing all chemical structures, substructure screens are used. Substructure screens are defined and used as follows. For each chemical structure in a database, a bit vector is associated; each bit represents the presence (1) or absence (0) of a predefined chemical fragment. These screen fragments are chosen according to the structural relevance for a given data set of molecules. The query structure is analyzed for screen structures, and an equivalent screen vector is produced. If a database structure contains a fragment isomorphic to one of the screen structures, the corresponding bit is set to 1. Consequently, every bit set in the query molecule must appear in the database molecule bit vector to ensure the presence of the corresponding screen fragment. Comparing the screen vectors of query and database molecules can reduce the search effort dramatically if the screen fragments have been selected adequately. Although substructure screens have been used effectively in existing database systems, no rules exist for compiling a set of effective screen fragments for multipurpose applications. At that point, the chemist is in demand to select screen fragments for a given data set of chemical structures in the most efficient way — that is, by minimizing the number of fragments used as a substructure screen and by optimizing a set of fragments. As a matter of fact, substructure screens can actually be adapted to specific molecule databases.
4.3.4 Hash Coding Hash coding is a scheme for providing rapid access to data items that are distinguished by some key. Each data item is associated with a key (e.g., a substructure). A
5323X.indb 66
11/13/07 2:09:35 PM
Dealing with Chemical Information
67
hash function is applied to the key of the item, and the resulting hash value is used as an index to select one out of a number of hash buckets in a hash table. The table contains pointers to the original items. If, when adding a new item, the hash table already has an entry at the indicated location, the key of the entry must be compared with the given key. If two keys hash to the same value (hash collision), an alternative location is used — usually the next free location cyclically following the indicated one. For optimum performance, the table size and hash function must be tailored to the number of entries and range of keys to be used. The hash function usually depends on the table size; consequently, the table must usually be rebuilt completely when hash codes are recalculated.
4.3.5 Stereospecific Search When stereospecific search options are used, only compounds that include stereochemical information are regarded. The stereo property of atoms is defined by specifying wedged and dashed bonds. The resulting stereo descriptor is calculated internally. If no stereo center has been specified, the stereo configuration is usually presumed as undefined. Stereo bonds can be defined in several ways: • • • •
Undefined — configuration is unknown. Relative S configuration defined. Relative R configuration defined. Either stereo configuration (mixture of enantiomers).
The latter case can be used to specify racemic mixtures with a stereo either bond. Stereo either bonds are interpreted as a racemic mixture of R and S enantiomers and are also used for epimers. An atom is considered as a potential stereo center if it binds to at least three substituents; the fourth may be an implicit hydrogen atom. The atom is handled as an explicit stereo center if at least three different nonhydrogen atoms are bonded. Recognition of stereochemical information can be restricted to several search modes, such as • Stereo inselective: Disables the stereo check. • Isomers selective: Checks only if the database molecule has stereo information at the same positions as the query (independent of the stereo descriptors). • Relative configuration: Finds all molecules on the database with the same stereochemistry and all its enantiomers, but no mixtures. • Absolute configuration: In this mode, only molecules with the same stereochemistry are found.
4.3.6 Tautomer Search In contrast to resonance effects where delocalization of π-electrons occur, tautomeric effects lead to isomeric structures, which differ significantly in the relative positions of their atoms. Tautomerism always involves formation and dissection σ-bonds in the course of a transformation together with a change in geometry. A
5323X.indb 67
11/13/07 2:09:36 PM
68
Expert Systems in Chemistry Research
tautomer search algorithm enables the recognition of molecules capable of proton tautomerism, where a proton changes its connection while a double bond moves to an adjacent position. To identify potential tautomeric structures, an additional hash code is stored on the database for each structure. This hash code represents the molecule transformed to a canonical structure. Each query is also transformed to the canonical structure, before the atom-by-atom matching algorithms are applied.
4.3.7 Specifying a Query Structure Commercially available structure editors provide several tools that allow a query to be defined with specific or general components. The back-end software has to transfer this information to the substructure search module. Transferring the structure and special information is done via standardized molecule file formats that include or are enhanced by special code conventions for the additional information. The substructure search module interprets the special conventions and translates them into an internal substructure representation. Examples for special query information are as follows: • • • • • • • • • •
A positive or negative charge on a node of the structure A particular atom at a site or a generic atom The bond type, or left unspecified Free sites Stereochemistry Generic atom lists Stereoisomers Tautomers Charged compounds Isotopes
Besides specific search parameters, the substructure search modules often include formula or formula range search as well as mass and mass-range-search algorithms. Many structures can be drawn in various manners, even if they represent the same molecule. To achieve a unique representation of structures in a database, a normalization algorithm is required to convert ambiguous structures into a unique form. For identity searches and substructure searches, all structures are usually normalized. Normalization is performed for each database entry just before uploading the structure, as well as for queries before sending the query to the search module. Stereoselective substructure search can be performed in various levels of detail. The stereo structure query is created by using special conventions, like wedge and dashed bonds, to indicate the chirality of atoms. By default, a stereochemistry in the query is relative. Advanced features are necessary for stereospecific substructure search requests to search for a chemical fragment while controlling, for example, the degree of substitution of atoms, the coordination number (i.e., number of substituents) of atoms, the cyclicity of atoms and bonds (i.e., whether or not they are part of a ring), and the chemical environment of the substructure.
5323X.indb 68
11/13/07 2:09:36 PM
Dealing with Chemical Information
69
If the structure search retrieves too many results, then there are a number of ways to refine or limit the result: • Change the drawn structure to be more specific. • Add another chemical structure to the search strategy. • Combine structure search with a text search or other searches. Query atoms can be defined either as single atoms or as atom lists. Atom lists are represented in brackets, such as [C,N,O,P]. In this case, the atoms in the list are allowed at the corresponding position. Query atom lists can be negated (NOT [N,O]) to exclude atom types at certain positions. It is possible to define multiple query atom lists at several positions. Several other options are provided by most of the commercially available structure editors: • The number of substituents for variable bonded atoms can be restricted by using a substitution count atom property. • The number of bonds belonging to a ring can be specified for an atom. • Bond types allow a specification whether the bond is belonging to a ring or a chain system. • In substructure queries, each atom that is not explicitly defined is interpreted as any-atom (A). In most cases, it is not necessary to define anyatoms because open valences are interpreted automatically as any-atom (including hydrogen). However, because the bond type for an implicit anyatom is a single bond by default, any-atoms are necessary for specifying another bond type that connects the any-atom. The explicit specification of the any-atom includes all elements except hydrogen. • Molecular formulas can be used as additional search criteria. Either complete formulas — similar to sum formula identity search — or partial formulas and atomic ranges can be specified, such as C5-8H8-12O2. • Molecular masses or ranges of masses can be defined to restrict the search results to molecules of a certain mass. • Charge selective searches enable one to distinguish between charged and uncharged compounds. • For semipolar bonds, nitro and nitroso groups are sometimes drawn with double bonds instead of a single bond with formal charges. To enable finding groups with two double bonds as well as those with formal charges, nitro and nitroso groups are normalized.
4.4 Describing Molecules The investigation of molecular structures and their properties is one of most fascinating topics in chemistry. Since the first alchemy experiments, scientists have created a language consisting of symbols, terms, and notations to describe compounds, molecules, and their properties. This language was refined to give a unique notation known today by scientists all over the world. The increasing use of computational
5323X.indb 69
11/13/07 2:09:36 PM
70
Expert Systems in Chemistry Research
methods made it necessary to implement this language into computer software. Although a trained scientist understands the conventional language easily, new approaches have become necessary for describing molecular information in computer software. A new kind of language for computational chemistry has evolved: the language of molecular descriptors. Let us start with a basic definition of a molecular descriptor: A molecular descriptor represents a certain property or a set of properties of a olecule in a way that is suitable for computational processing. m
It is important to note that a molecular descriptor does not necessarily represent an entire chemical structure; it does not even necessarily describe structural features at all. As scientists look at a structural drawing, they might have different perceptions depending on the task they have to solve. A synthesis chemist will have a special perception for the reactive centers, whereas a spectroscopist might be interested in functional groups or mass fragments. This difference in perceiving a structural drawing has to be reflected in a molecular descriptor. Consequently, molecular descriptors are closely related to the task to be solved. Let us stay a bit with the example of the spectroscopist. Let us assume he is a specialist in infrared spectroscopy. One of the features of a structural drawing he would pay special attention to is hydroxy groups. The reason he does this is because his experience tells him that hydroxyl groups can usually be seen as a broad band at the beginning of an infrared spectrum. He translates this information almost automatically into a hydroxyl band in the spectrum. In fact, the infrared spectrum is nothing else than a molecular descriptor that represents a certain property of the molecule: its vibrational behavior under infrared radiation. If we look from this standpoint at molecular descriptors, several well-known structural descriptors are already used in the day-to-day laboratory of a scientist. In contrast to that, molecular descriptors in the context of computational chemistry are values, vectors, or matrices that are calculated from one or more measured or calculated properties of a molecule. For ease of understanding, let us define those descriptors recorded as a result of an analytical technique as experimental descriptors. In contrast to that, we will talk about artificial descriptors when we refer to those that are calculated. Experimental descriptors emerge from a fixed experimental design, and their appearance is subject to the physical or chemical limitations of the measurement technique. The advantage of artificial descriptors is that they can be adjusted and fine-tuned easily to fit to a task due to their pure mathematical nature. The only limitation to this approach is the scientist’s imagination. However, there are several constraints to be taken into account when selecting or constructing a molecular descriptor. Todeschini and Consonni pointed these out in their book Handbook of Molecular Descriptors [15]. Let us have a closer look at these constraints.
4.4.1 Basic Requirements for Molecular Descriptors There are four basic requirements for a molecular descriptor property.
5323X.indb 70
11/13/07 2:09:37 PM
Dealing with Chemical Information
71
4.4.1.1 Independency of Atom Labeling Labels or numbers assigned to atoms are not related at all to a molecular property and, thus, should not have an effect on the descriptor. This applies particularly to the sequence of atoms, which is most often defined by the sequence of drawing atoms in the structure editor software. Unless a special algorithm is used to unique enumeration, like the Morgan algorithm [9] or the Jochum-Gasteiger canonical renumbering [16], the numbering of atoms is purely arbitrary. The Morgan algorithm assigns an integer label to every atom in the molecule and updates the atom labels iteratively so that topologically equivalent atoms have the same labels. In the initial phase, every atom obtains a label for the number of connected bonds. In subsequent iterative steps, every atom label is updated as the sum of its current label and the labels of adjacent atoms. After the n-th iteration, each atom obtains a label that contains the information of nodes that are within n bonds around it. The computed labels are used as a necessary condition to find topologically equivalent nodes in the graph structure. The algorithm does not work for structures whose atoms are connected by the same number of bonds (regular graphs). However, except for a few special cases, like fullerenes, such regular graphs are rarely found in chemical compounds. 4.4.1.2 Rotational/Translational Invariance A descriptor has to be unaffected by changes in the absolute coordinates of atoms. There are two cases to distinguish here: • Conformational changes (rotation of parts of a molecule around a bond axis): A descriptor might be independent of conformational changes; however, it may be desired that a descriptor reflects rotations around single bonds. • Entire molecule rotation: A descriptor should definitely not be affected by rotation of the entire molecule. The way atom coordinates are calculated is more or less arbitrary, since it depends on a starting point for calculation. 4.4.1.3 Unambiguous Algorithmically Computable Definition Molecular descriptors can only be managed by computer software in an effective manner. Consequently, the calculation shall be computationally unambiguous and reasonably fast. The same algorithm shall result in the same descriptor as long as the molecule properties of interest do not change. 4.4.1.4 Range of Values Values of a descriptor shall be in a suitable numerical range. It is always hard to work with data that may appear in any arbitrary range or may be infinite. It is a good approach to allow minimum and maximum values for a descriptor in advance, which makes comparison easier and avoids calculating numbers in an unpredictable range when further processing is performed.
5323X.indb 71
11/13/07 2:09:37 PM
72
Expert Systems in Chemistry Research
Another important fact is a fixed size of a descriptor. Descriptors shall be comparable by using statistical methods, artificial neural networks, or other mathematical approaches that rely on vectors or matrices of the same dimension.
4.4.2 Desired Properties of Molecular Descriptors Todeschini and Consonni also pointed out a series of important desired characteristics [15]. Further required characteristics of molecular descriptors were discussed by Randic [17]. Let us have a closer look at some of these. • Structural Interpretation — It is helpful if a descriptor can be interpreted in a way that allows features of the chemical structure or even the entire chemical structure to be derived. This is particularly interesting, if a molecular descriptor is applied to the task like a spectrum; in this case we could talk about an artificial spectrum. Several approaches exist that use such artificial spectra to compare them with experimental spectra — for instance, to derive experimental spectra that do not exist in a database. • Descriptor/Property Correlation — This is one of the basic tasks for a descriptor; descriptors are usually helpful for predicting properties or comparing properties of chemical compounds. • Correlation with Other Molecular Descriptors — This applies mainly to artificial descriptors. A correlation with experimental descriptors is helpful to produce missing experimental data. • Reflection of Structural Changes — This applies if the molecular descriptor is intended to represent a structural feature. Most of the properties of a molecule are, in fact, related to the structure. However, it depends on the definition of a chemical structure; two- or three-dimensional representations, molecular surface, charge distribution, and polarizability are all different features of a structure, and the extent of their effects on, for instance, chemical binding is quite different. For instance, a 2D structural drawing is of minor importance for the biological activity of a compound, which is mainly affected by the surface and the charge distribution of a structure. Inner structural changes in a large protein might not affect the shape and charge distribution at the surface; thus, a descriptor for biological activity should not react on these changes. • Restriction to Molecule Classes — A molecular descriptor shall be applicable to all potential molecules. This is the ideal case; actually, this is barely achievable when working with vectors or matrices. A typical example is the size of a molecule: The larger the molecule, the fuzzier is the influence of detail features of a molecule on the desired property. To calculate vectors or matrices of fixed dimension — which is a requirement described already — the size of the molecule has to be independent of the resulting descriptor. Consequently, a descriptor that does a fine representation of a small or medium structure will usually fail to represent a protein or a gene. It is like looking at a painting from 200 feet; details get fuzzy, and it gets hard to compare the view with the one standing in front of the painting.
5323X.indb 72
11/13/07 2:09:37 PM
Dealing with Chemical Information
73
• Discrimination among Isomers — Isomerism and, in particular, stereochemistry have a profound influence on macroscopic properties of a compound. Whereas rotation around a single bond might not be a desired feature for a descriptor used for structure searching, stereochemical hindrance might affect the reactivity dramatically. The ideal algorithm for molecular descriptors will allow these properties to be switched on or off as desired. 4.4.2.1 Reversible Encoding The ideal descriptor can be decoded to obtain the original chemical structure or the properties that have been used to calculate the descriptor. Although this is definitely desired, the real world shows that the information used to calculate the descriptor is usually too complex to use it at its full size. A descriptor shall have a reasonable size for effective computation, and this is mostly achieved by reducing information to the facts that are of major importance for the task. The need for a fixed descriptor dimension also contradicts this requirement.
4.4.3 Approaches for Molecular Descriptors A molecular or compound property consisting of a single value can already be used as a descriptor. Molecular weight, sum of atomic polarizabilities, mean atomic van der Waals volume, mean atomic polarizability, and number of atoms are typical instances for a descriptor. These properties can be used in the form of a single component, or they can be combined in a vector consisting of multiple different properties to fit the task in the most appropriate manner. An example is an application published by Wagener et al. for the prediction of the effective concentration of carboquinones as anticarcinogenic drugs [18]. For the prediction they used four physicochemical properties: (1) the molar refractivity index; (2) hydrophobicity; (3) field effect; and (4) resonance effect. By calculating the contributions of different residues on para-benzochinone derivatives and feeding them into a back-propagation neural network, they were able to predict the concentration of the drug that still showed a carcinogenic effect.
4.4.4 Constitutional Descriptors Constitutional descriptors represent the chemical composition of a molecule in a generic way. They are independent of molecular connectivity and geometry. Typical constitutional descriptors are number, sum, mean, or average of constitutional properties. Examples are number of a particular atom type or bond type, number of particular ring systems, sum of electronegativities, mean atomic van der Waals volumes, and average molecular weight. Constitutional descriptors are particularly attractive because of their simplicity; it is easy to understand their meaning in the context, and they can be easily calculated from atom lists or textual representations. One of the first generic approaches for calculating constitutional descriptors comes from Free and Wilson in 1964 [19]:
5323X.indb 73
11/13/07 2:09:38 PM
74
Expert Systems in Chemistry Research
P = P0 +
N
∑c I
k k
(4.1)
k
Here, P is a property calculated from the sum of k structural features I multiplied by a dynamic or constant factor c. Constitutional descriptors are widely used for investigating quantitative structure/activity relationships (QSAR) for drug discovery.
4.4.5 Topological Descriptors Topological indices are calculated using data on the connectivity of atoms within a molecule. Consequently, these descriptors contain information about the constitution, size, shape, branching, and bond type of a chemical structure, whereas bond length, bond angles, and torsion angles are neglected. The first published topological descriptor is the Wiener Index, named after Harry Wiener [20] — is the first topological descriptor. It calculates the sum of the number of minimum bonds n between all nonhydrogen atoms i and j:
W=
1 2
N
∑n
ij
(4.2)
i, j
The factor in front of the sum term ensures that bond paths are only counted once. The Wiener index is particularly useful to describe quantitative structure–property relationships (QSPR). An example is the correlation of saturation vapor pressure with the chemical structure. Since the Wiener index does not distinguish atom types, those correlations can only be achieved for homologues series of compounds. Another series of successfully applied topological descriptors is derived from graph theory using atom connectivity information of a molecule. An example is the connectivity index developed by Randic [21]. In the simple form, N
χ=
∑( D D ) i
j
−1 2
(4.3)
i, j
D denotes the degrees of the atoms i and j of a molecular graph, and the sum goes over all adjacent atoms. The exponent was originally chosen by Randic to be –1/2; however, it has been used to optimize the correlation between the descriptor and particular classes of organic compounds. The connectivity index was later extended by Kier and Hall to applications with connected subgraphs [22].
4.4.6 Topological Autocorrelation Vectors Topological autocorrelation vectors are calculated from the two-dimensional structure of a molecule that can be expressed as molecular graph. One of the original
5323X.indb 74
11/13/07 2:09:39 PM
75
Dealing with Chemical Information
approaches was developed by Moreau and Broto, called the autocorrelation of a topological structure (ATS) [23]. ATS describes how a property is distributed along the topological structure:
Ad
N
∑δ ( p p ) ij
i, j
i
j d
(4.4)
The properties p of the atoms i and j are considered for a particular topological distance d. δij is a Kronecker delta that represents additional constraints or conditions. The topological distance may also be replaced by the Euclidean distance, thus accounting for two- or three-dimensional arrangement of atoms. Three-dimensional spatial autocorrelation of physicochemical properties has been used to model the biological activity of compound classes [24]. In this case, a set of randomly distributed points is selected on the molecular surface, and all distances between the surface points are calculated and sorted into preset intervals. These points are used to calculate the spatial autocorrelation coefficient for particular molecular properties, such as the molecular electrostatic potential (MEP). The resulting descriptor is a condensed representation of the distribution of the property on the molecular surface.
4.4.7 Fragment-Based Coding A widely used method for calculating a molecular descriptor is substructure- or fragment-based coding. The molecule to be encoded is divided into several substructures that represent the typical information necessary for the task. Many authors have used this method for the automated interpretation of spectra with artificial neural networks [25], in expert systems for structure elucidation [26], with pattern recognition methods [27], and with semiempirical calculations [28]. A binary vector is typically used to define simply the presence or absence of functional groups that exhibit important spectral features in the corresponding spectrum. The main disadvantage of this method is that the number of substructures cannot be generally restricted. In various publications, the number of substructures differs from 40 to 229, depending on the more or less subjective point of view of the problem. Affolter et al. showed that a simple assignment of infrared-relevant substructures and corresponding infrared signals does not describe spectrum–structure correlation with adequate accuracy [29]. This is mainly due to the effect of the chemical environment on the shape and position of absorption signals. An improvement of the simple substructure approach is the method fragment reduced to an environment that is limited (FREL) introduced by Dubois et al. [30]. Several centers of the molecule are described, including their chemical environment. By taking the elements H, C, N and O and halogens into account and combining all bond types — single, double, triple, aromatic — the authors found descriptors for 43 different FREL centers that can be used to characterize a molecule. To characterize the arrangement of all atoms in a molecule, the entire molecule can be regarded as a connectivity graph where the edges represent the bonds and
5323X.indb 75
11/13/07 2:09:40 PM
76
Expert Systems in Chemistry Research
the nodes represent the atoms. It is possible to calculate a descriptor that defines the constitution of a molecule independently of conformational changes by adding the number of bonds or the sum of bond lengths between all pairs of atoms. The resulting descriptor is not restricted regarding the number of atoms. Clerc and Terkovics [31] used this method based on the number of bonds for the investigation of Quantitative Structure/Property Relationships (QSPR). Using the sum of bonds (or sum of distances) eliminates redundant information, but also certain molecular features. Such a descriptor no longer characterizes complex relationships, such as the correlation between structure and spectrum. Further information about molecular descriptors can be found at the free on-line resource “Molecular Descriptors” [32].
4.4.8 3D Molecular Descriptors Molecules are usually represented as 2D graphs or 3D molecular models. Although two-dimensional representations are extremely helpful for communicating and reporting, most of the compound properties depend on the spatial arrangement of the atoms in a molecule. Three-dimensional coordinates of atoms in a molecule are sufficient to describe the spatial arrangement of atoms; however, they exhibit two major disadvantages: (1) They depend on the size of a molecule; and (2) they do not describe any additional properties, like partial charges or polarizability. The first attribute is important for computational analysis of data. Even a simple statistical function (e.g., the correlation) requires the information to be represented in equally sized vectors of a fixed dimension. The solution to this problem is a mathematical transformation of the Cartesian coordinates of a molecule to a vector of fixed length. The second point can be overcome by including the desired properties in the transformation algorithm. A descriptor for the three-dimensional arrangement of a molecule can be derived from the Cartesian coordinates of the atoms. Reliable coordinates can be calculated quite easily by semiempirical or molecular mechanics (i.e., force-field) methods by using molecular modeling software. Fast 3D structure generators are available that combine rules and force-field methods to calculate Cartesian coordinates from the connection table of a molecule (e.g., CORINA [33]). Let us summarize the three important prerequisites for a 3D structure descriptor: It should be (1) independent of the number of atoms, that is, the size of a molecule; (2) unambiguous regarding the three-dimensional arrangement of the atoms; and (3) invariant against translation and rotation of the entire molecule. Further prerequisites depend on the chemical problem to be solved. Some chemical effects may have an undesired influence on the structure descriptor if the experimental data to be processed do not account for them. A typical example is the conformational flexibility of a molecule, which has a profound influence on a 3D descriptor based on Cartesian coordinates. The application in the field of structure–spectrum correlation problems in vibrational spectroscopy requires that a descriptor contains physicochemical information related to vibration states. In addition, it would be helpful to gain the complete 3D structure from the descriptor or at least structural information (descriptor decoding).
5323X.indb 76
11/13/07 2:09:40 PM
77
Dealing with Chemical Information
4.4.9 3D Molecular Representation Based on Electron Diffraction One way to 3D molecular descriptors is deriving them from molecular transforms, which are generalized scattering functions used to create theoretical scattering curves in electron diffraction studies. A molecular transform serves as the functional basis for deriving the relationship between a known molecular structure and x-ray and electron diffraction data. The general molecular transform is
I (s) =
N
∑f
j•
e2πi rj s •
(4.5)
j
where I(s) represents the intensity of the scattered radiation and s the scattering in various directions by a collection of N atoms located at points rj. fj is a form factor taking into account the directional dependence of scattering from a spherical body of finite size. The value s measures the wavelength-dependent scattering s = 4π·sin(ϑ/2)/λ, where ϑ is the scattering angle and λ is the wavelength of the electron beam. Wierl suggested in 1931 to use this equation in a modified form [34]. Soltzberg and Wilkins introduced a number of simplifications to obtain a binary code [35]. They considered only the zero crossings of I(s) = 0 in the range 0 – 31 Å–1 and divided the s range into 100 equal intervals, each described by a binary value equal to 1 if the interval contains a zero crossing, 0 otherwise. The resulting code is a 100-dimensional binary vector. Considering a rigid molecule, neglecting the instrumental conditions, and replacing the form factors by the atomic number, A, they obtained
I (s) =
N −1
N
∑∑ A A i
i
j >i
j
sin ( s • rij ) s • rij
(4.6)
where I(s) is the scattered electron intensity, Ai and Aj are the atomic numbers, rij are the interatomic distances between the ith and jth atoms, respectively, and N is the number of atoms. Gasteiger et al. returned to the initial I(s) curve and maintained the explicit form of the curve [36]. For A they substituted various physicochemical properties such as atomic mass, partial atomic charges, and atomic polarizability. To obtain uniform length descriptors, the intensity distribution I(s) is made discrete, calculating its value at a sequence of evenly distributed values of, for example, 32 or 64 values in the range of 0 – 31Å–1. The resolution of the molecule representation increases with higher number of values. The resulting descriptor is the 3D MoRSE (Molecular Representation of Structures based on Electron diffraction) Code.
4.4.10 Radial Distribution Functions 3D MoRSE codes are valuable for conserving molecular features, but it is not possible to interpret them directly. This drawback leads to investigations of related types of descriptors. Steinhauer and Gasteiger picked up the idea of radial distribution
5323X.indb 77
11/13/07 2:09:42 PM
78
Expert Systems in Chemistry Research
functions (RDF) used in x-ray scattering investigations [38,39]. Radial distribution functions, well known in physicochemistry and physics, describe the distance distribution of points in three-dimensional space. This suggests applying this function to a three-dimensional molecular model. The radial distribution function
g (r ) =
N −1
N
i
j >i
∑∑
2 − B r−r e ( ij )
(4.7)
describes an ensemble of N atoms in 3D space in a spherical volume of radius r. g(r) is the probability distribution of atom distances rij between atoms i and j. The exponential term contains the smoothing parameter B, which determines the width of the individual peaks in the probability distribution. B can be interpreted as a temperature factor defining the movement of atoms. A number of discrete intervals is usually used to calculate g(r). Equation 4.3 meets all the requirements for a 3D structure descriptor: Within a predefined distance range, it is independent of the number of atoms (the size of a molecule), unique concerning the three-dimensional arrangement, and invariant against translation and rotation of the entire molecule. A slight modification of the general form of an RDF leads to a molecular descriptor, the radial distribution function (RDF) code, which includes atom properties that address characteristic atom features in the molecular environment. Fields of application for RDF codes are the simulation of infrared spectra and deriving molecular structure information from infrared spectra [40,41].
4.4.11 Finding the Appropriate Descriptor According to the previous definition, molecular descriptors represent properties of a molecule. The most important issue is to eliminate properties that are not relevant for the task to be solved. This is similar to analytical methodology, where unwanted effects are eliminated or suppressed by using chemical or physical methods, like chemical modifiers in chromatography or background compensation in spectroscopy. Finding the appropriate descriptor for the representation of chemical structures is one of the basic problems in chemical data analysis. Several methods have been developed for the description of molecules, including their chemical or physicochemical properties [42]. The wide variety of molecular structures usually requires a reduction of information together with the encoding process. Additionally, structural features should be encoded including properties that have a profound influence on the features to be investigated, such as molecular symmetry, physicochemical bond, or atomic or molecular properties like charge distribution, electronegativity, and polarizability of the compounds. Radial distribution functions and their derivatives have been found
The term radial distribution functions should not be confused with radial basis functions (RBF), a term introduced by Broomhead and Lowe in 1988 and which represents a type of function used for neural networks employing a hidden layer of radial units [37].
5323X.indb 78
11/13/07 2:09:43 PM
79
Dealing with Chemical Information
to be particularly useful due to their flexibility. There are several ways of generating descriptors based on the concept of radial distribution functions. • RDF descriptors can be restricted to specific atom types or distance ranges to represent complete or partial information in a certain chemical environment, for instance, to describe steric hindrance in a reaction or structure– activity relationships of a molecule. • RDF descriptors are interpretable by using simple rules. Consequently, they provide a possibility to convert the vector into the corresponding 3D structure. • RDF descriptors provide further valuable information, such as about bond distances, ring types, planar and nonplanar systems, and atom types. This is a valuable consideration for computer-assisted descriptor elucidation. The next chapter deals with the development and modifications of RDF descriptors.
4.5 Descriptive Statistics Application of statistics in expert systems is a topic that fills more than a single book. However, some of the investigations presented in the next chapters are based on methods of descriptive statistics. The terms and basic concepts of importance for the interpretation of these methods should be introduced first. Algorithms and detailed descriptions can be found in several textbooks [43–47].
4.5.1 Basic Terms Before we look at some details of the analysis process, let us recall some basic important statistical terms. 4.5.1.1 Standard Deviation (SD) SD of a data set is a measure of how spread out the data is; it is the average distance from the mean of the data set to a point, calculated by the sum of squares of the distances from each data point to the mean of the set divided by a subset of the population, expressed as n – 1:
σ=
1 n −1
n
∑(x − x ) i
2
(4.8)
i =1
4.5.1.2 Variance This is another measure of the spread of data in a data set. It is in fact almost identical to the standard deviation; it is simply the standard deviation squared.
5323X.indb 79
var ( x ) =
1 n −1
n
∑(x − x ) i
2
= σ2
(4.9)
i =1
11/13/07 2:09:44 PM
80
Expert Systems in Chemistry Research
4.5.1.3 Covariance Whereas standard deviation and variance are purely one dimensional, covariance is measured between two dimensions. Instead of taking squares of the distances of the one-dimensional data x, we take the product of distances for two variables, x and y. For each data item, multiply the difference of the x value and the mean of x by the difference between the y value and the mean of y.
cov( x, y) =
1 n −1
n
∑ ( x − x )( y − y ) i
i
(4.10)
i =1
A positive covariance indicates that both dimensions increase together, whereas if the covariance is negative, then as one dimension increases, the other decreases. If the covariance is zero, it indicates that the two dimensions are independent of each other. 4.5.1.4 Covariance Matrix This is an extension to the covariance concept for multidimensional data. It is a matrix that shows all covariance values calculated between the individual dimensions. A covariance matrix for three dimensions, x, y, and z, would look as follows:
cov( x, x ) cov( y, x ) cov( z, x )
cov( x, y) cov( y, y) cov( z, y)
cov( x, z ) cov( y, z ) cov( z, z )
(4.11)
Since the sequence of dimensional terms in the covariance does not affect the results, that is, cov(x,y) = cov(y,x), the covariance matrix is a symmetric one. The main diagonal shows the self-products and, thus, the variances for the dimension. 4.5.1.5 Eigenvalues and Eigenvectors Multiplying a matrix A with a vector v results in the transformation of this vector from its original position. When multiplying matrices with vectors, we have to consider a special case: If the resulting vector is an integer multiple of the original vector, the factor λ is called an eigenvalue and the vector is an eigenvector of matrix A; in mathematical terms,
Av = λv
(4.12)
In this case the length of the vector changes by λ, but not its orientation. In other words, the eigenvectors of a matrix are the nonzero vectors that when multiplied by the matrix give integer multiples of the eigenvectors, and the corresponding integer factors are the eigenvalues of the matrix. Eigenvectors arise from transformations and can only be found for square matrices, and if they are found, exactly n eigenvectors exist for an n times n matrix. All eigenvectors of a matrix are orthogonal; that is, data can be represented in terms of
5323X.indb 80
11/13/07 2:09:46 PM
81
Dealing with Chemical Information
these orthogonal vectors instead of expressing them in terms of the x and y axes. Due to the fact that scaling does not change the direction of a vector, scaling of eigenvectors still leads to the same eigenvalue. Consequently, eigenvectors are usually scaled to have a length of 1 to be comparable, resulting in the unit eigenvectors. If we consider all n eigenvectors, we can write an eigenvector matrix V
V = v (1)
v (2 )
v1(1) v(1) 2 v ( n ) = (1) vn
v1( 2 ) v2( 2 ) (2 ) vn
v1( n ) v2( n ) . vn( n )
(4.13)
We can combine all of them by using the single matrix equation AV = ΛV ,
(4.14)
where Λ is a diagonal matrix of all eigenvalues:
0 λ2 0
λ1 0 Λ= 0
0 0 λ n
(4.15)
The diagonal eigenvalues in this matrix make up the eigenvalue spectrum. The decomposition A = ΛVV −1 , (4.16) helps in characterizing many of the behaviors of A in various applications.
4.5.2 Measures of Similarity It is useful to characterize a set of values that has a tendency to cluster around a centered value by its centered moments of distribution, Mn (the sums of the nth integer powers of the values). Although most descriptors exhibit no centered distribution, the similarity in the general trend of distribution makes it comparable to other descriptors. Several general measures for describing the similarity of descriptors are valuable. The root mean square (RMS) error is calculated by the mean squared individual differences of the components of two descriptors, gAj := [gA1...gAn] and gBj := [gB1... gBn] (n is the number of components) and is used as the default measure of similarity for descriptor database searches:
5323X.indb 81
RMS =
1 n
n
∑(g j =1
Aj
2
− gBj )
(4.17)
11/13/07 2:09:49 PM
82
Expert Systems in Chemistry Research
Many investigations rely on descriptor sets rather than individual descriptors. An obvious way to compare an individual descriptor with a descriptor set is to calculate an average set descriptor g (r ) from the sum of all vector components of the descriptor gi(r) divided by the number of descriptors L:
g (r ) =
L
1 L
∑ g (r )
(4.18)
i
i =1
In other words, each component of a descriptor gi(r) in a data set is summarized and divided by the number of molecules L. g (r ) represents the mean descriptor and, thus, a kind of mean molecule of a data set. The next step is to calculate the difference between each descriptor gi(r) and the average descriptor g (r ) . This value is defined by the average descriptor deviation δgi for an individual descriptor i with the components j,
δgi =
1 n
n
∑g
ij
− gj
(4.19)
j =1
and represents a robust estimator for the variability of a descriptor within its data set. The sum of all average descriptor deviations divided by the number of descriptors L leads to the average diversity ∆g: 1 ∆g = L
L
∑ δg
(4.20)
i
i =1
The average diversity represents an estimator for the molecular diversity of a data set and is calculated for each descriptor in a data set. The correlation measures the relation between two or more variables and goes back to works performed in the late nineteenth century [48]. The most frequently used type of correlation is the product–moment correlation according to Pearson [49]. The Pearson correlation determines the extent to which values of two variables are linearly related to each other. The value of the correlation (i.e., the correlation coefficient) does not depend on the specific measurement units used. A high correlation can be approximated by a straight regression line — sloped upward or downward — that is determined by the minimum sum of the squared distances of all the data points. One of the descriptors, gA or gB, can be regarded as the dependent variable. Consequently, two regression lines with different slopes can exist. By defining
aAB =
n
∑(g j =1
5323X.indb 82
g
Aj Bj
) − ngA gB
(4.21)
11/13/07 2:09:52 PM
83
Dealing with Chemical Information
the slopes bAB and bBA can be calculated:
bAB = aAB
n
∑ ( g ) − ng 2 Aj
2 A
(4.22)
∑ ( g ) − ng
2 B
(4.23)
j =1
bBA = aAB
n
2 Bj
j =1
The geometric mean of both regression lines reflects the correlation coefficient R:
R = aAB
bABbBA
(4.24)
The squared correlation coefficient R2, the coefficient of determination, represents the proportion of common variation in the two variables (i.e., the magnitude of the relationship).
4.5.3 Skewness and Kurtosis An important prerequisite for the use of descriptive statistics is the shape of the distribution of variables, that is, the frequency of values from different ranges of the variable. It is assumed in multiple regression analysis that the residuals — predicted minus observed values — are normally distributed. The significance of correlation coefficients of a particular magnitude will change depending on the size of the sample from which it was computed. Because a regression line is determined by minimizing the sum of squares of distances, outliers have a profound influence on the slope of the regression line and consequently on the value of the correlation coefficient. A single outlier is capable of considerably changing the slope of the regression line and, thus, the value of the correlation. This effect has been described as the King Kong Effect [50]. Let us assume we want to evaluate the relationship between the height and the weight of individuals in a horde of gorillas by using simple correlation. For most of the adult animals we will find a height between 150 and 170 centimeters, whereas the weight ranges between 130 and 140 kilograms. A typical correlation coefficient results in 0.85, which shows a trend in the relationship. Now we find King Kong, who is the chief of the horde, with a remarkable height of 190 centimeters and 188 kilograms; including these data into our correlation chart results in a correlation coefficient of 0.95 (Figure 4.2a). The reason for this increase in correlation by including a value that is obviously an outlier lies in the distribution of values. Figure 4.2b shows that a standard distribution no longer exists, and thus, the basic assumptions for the correlation coefficient do not apply. Descriptive statistics can provide information about how well the distribution can be approximated by the normal distribution with skewness and kurtosis (the term was first used by Pearson) [49].
5323X.indb 83
11/13/07 2:09:54 PM
84
Expert Systems in Chemistry Research 190
King Kong
Weight/kg
180
r = 0.95
170 160 150
r = 0.85
140 130
150
160
170
(a)
180
190
Height/cm
12
Frequency
10
.
Height Weight King Kong
8 6 4 2 0
150
155
160
(b)
165
170
175
180
185
Height/cm
Figure 4.2 The King Kong effect in unweighted linear regression. The inclusion of a significant outlier can lead to an increase in the correlation coefficient; even the graphical representation shows that the bulk of data in the lower right is no longer well represented by the regression line when the outlier is included. The reason is the distance of the bulk data to the outlier. (b) The distribution of data shown in (a) no longer shows Gaussian behavior due to the outlier included in the data set. Without weighing the individual data points, one of the basic requirements for applying linear regression — standard distribution of data points — is no longer given.
The skewness, S, of a descriptor reflects symmetry of distribution of its components related to the Gaussian distribution
S=
n⋅ M3 n − 1 ( ) ⋅ (n − 2) ⋅ σ 3
(4.25)
with
5323X.indb 84
M k = ∑i ( xi − x )k
(4.26)
11/13/07 2:09:56 PM
85
Dealing with Chemical Information
Gaussian
Leptokurtic
Platykurtic
Kurtosis
Positive
Negative Skewness
Figure 4.3 Graphical representation of kurtosis, K, and skewness, S, in comparison to the Gaussian (standard) distribution (upper left). The right-hand side shows leptokurtic (peaked) or platykurtic (flatted) distribution as well as positive skewed distribution (fronting) and negative skewed distribution (tailing).
as the kth moment and σ = M 2 (n − 1)
(4.27)
as the unbiased sample estimate of a population variance. S is clearly different from zero when the distribution is asymmetric — whereas normal distributions should be perfectly symmetric. A positive skewness indicates a fronting, whereas a negative skewness indicates a tailing in the distribution. The kurtosis, K, measures the flatness of a distribution related to the Gaussian distribution
K=
n ⋅ ( n + 1) ⋅ M 4 − 3 ⋅ ( M 2 )2 ⋅ ( n − 1) ( n − 1) ⋅ ( n − 2 ) ⋅ ( n − 3) ⋅ σ 4
(4.28)
If the kurtosis is substantially different from zero, then the distribution is either flatter (platykurtic, K < 0) or more peaked (leptokurtic, K > 0) than the Gaussian distribution that should have a kurtosis of zero (Figure 4.3).
4.5.4 Limitations of Regression The major conceptual limitation of regression techniques is that one can only ascertain relationships but can never be sure about the underlying causal mechanism. As mentioned already, multiple linear regression analysis assumes that the relationship between variables is linear. In practice, this assumption is rarely met. Fortunately, multiple regression procedures are not greatly affected by minor deviations from
5323X.indb 85
11/13/07 2:09:57 PM
86
Expert Systems in Chemistry Research
this assumption. If curvature in the relationships is evident — determined by skewness and kurtosis — one may consider either transforming the variables or explicitly allowing for nonlinear components. The skewness as well as the kurtosis of measured or calculated values will be non-zero in many cases, even if the underlying distribution is in fact more or less symmetrical. The meaning of S could be estimated by the standard deviation of S itself. Unfortunately, this depends on the shape of the distribution. As an approximation, the standard deviation for Equation 4.22 in the idealized case of a normal distribution
σ ≅ ± 15/n
(4.29)
is used as a lower limit for S. A similar approach can be applied to the kurtosis with a Gaussian standard deviation
σ ≅ ± 96 / n
(4.30)
Thus, for a vector consisting of 128 components the skewness limit is about 0.35, and the kurtosis limit is about 0.85. Whereas the mean is a measure of the tendency of a variable within its confidence intervals, the width of the confidence interval itself depends on the sample size and on the variation of data values. On one hand, the larger the sample size, the more reliable is the mean. On the other hand, the larger the variation, the less reliable is the mean. The calculation of confidence intervals is based on the assumption that the variable is normally — or at least equally — distributed in the population. The estimate may not be valid if this assumption is not met, unless the sample size is large. The concept of squared distances has important functional consequences on how the value of the correlation coefficient reacts to various specific arrangements of data. The significance of correlation is based on the assumption that the distribution of the residual values (i.e., the deviations from the regression line) for the dependent variable y follows the normal distribution and that the variability of the residual values is the same for all values of the independent variable. However, Monte Carlo studies have shown that meeting these assumptions closely is not crucial if the sample size is very large. Serious biases are unlikely if the sample size is 50 or more; normality can be assumed if the sample size exceeds 100.
4.5.5 Conclusions for Investigations of Descriptors Consequently, statistical investigations of descriptors should be performed on vectors containing at least 128 components and training sets containing at least 50 compounds. Similar conditions related to the sample size apply to the investigations with neural networks, which are, in fact, nothing more than a more complex statistical algorithm. Since a network minimizes an overall error, the proportion of types of data in the set is critical. A network trained on a data set with 900 good cases and 100 bad ones will bias its decision toward good cases, as this allows the algorithm
5323X.indb 86
11/13/07 2:09:59 PM
Dealing with Chemical Information
87
to lower the overall error, which is much more heavily influenced by the good cases. If the representation of good and bad cases is different in the real population, the network’s decisions may be wrong. In such circumstances, the data set may need to be crafted to take account of the distribution of data (e.g., the less numerous cases can be replicated, or some of the numerous cases can be removed). Often, the best approach is to ensure an even representation of different cases and then to interpret the network’s decisions accordingly. With an insufficient number of training data, the leave-one-out technique was applied. In this procedure all available data are used for the training of the network, except the one for which a prediction or classification has to be performed. This method can be applied iteratively for each object in the data set.
4.6 Capturing Relationships — Principal Components The idea behind statistics is based on the analysis of data sets in terms of the relationships between the individual points in that data set. If we summarize the correlation between two variables in a scatter plot, a regression line would show the linear relationship between the variables. We can now define a variable that approximates the regression line and characterizes the essential relationship between the two variables. In this case, we have reduced the relationship to a single factor, which is actually a linear combination of the two variables. There are two noteworthy techniques for such an approach: (1) principal component analysis (PCA); and (2) factor analysis.
4.6.1 Principal Component Analysis (PCA) PCA is one of the methods for simplifying a data set by reducing the number of dimensions in a data set. It goes back to a work from Pearson in 1901 [51]. In mathematical terms, PCA is an orthogonal linear transformation of a data set to a new coordinate system. The new coordinate system may, for instance, show the greatest variance on the first coordinate (i.e., first principal component), the second greatest variance on the second coordinate, and so on. The graphical representation of this coordinate system then shows relationships between data points by grouping them into particular areas of the coordinate system. PCA can be used to identify patterns in data and to express the data in such a way as to highlight their similarities and differences. Since patterns in data can be hard to find in data of high dimension, where graphical representation is not available, PCA is a powerful tool for analyzing data. Once the patterns are found in the data, they can be compressed by reducing the number of dimensions, without significant loss of information — a technique used for instance in image compression. With the basic concepts of covariance and eigenvectors, a principal component analysis can be performed with the following steps. Suppose we have calculated distance-based molecular descriptors represented as vectors for a data set of n compounds. If we write the components of the descriptor as column vectors, we can create a matrix of compounds, where each column represents a compound and each row contains the individual values of the descriptor:
5323X.indb 87
11/13/07 2:09:59 PM
88
Expert Systems in Chemistry Research
g1(1) g (1) 2 G= (1) gm
g1( 2 ) g2( 2 ) gm( 2 )
g1( n ) g2( n ) , gm( n )
(4.31)
or the individual descriptors
g ( k ) = g1( k )
g2( k )
T
gm( k )
(4.32)
In principal component analysis we regard each vector as representing a single point in an m-dimensional Euclidean space. If we look at the relationship between the first and second component of each descriptor (m = 2), we obtain a graph with n points (one for each descriptor). Obviously g1 and g2 are related to each other. The first objective of PCA (Figure 4.4) is to specify this relationship with a straight line, which is called the principal axis y1 of a new coordinate system for the points. This axis is determined by finding the largest variation in the data points. The second axis of this new coordinate system is perpendicular to the first axis. This example is based on two components; we can extend it to all components of the descriptor by adding axes that are perpendicular to the preceding axis. The third axis will be perpendicular to both the first and second axes and will correspond to the third largest variation in the data, and so on for the remaining m-3 axes. We assumed that the observations along a single row in the descriptor matrix are independent (different compounds) but that some values within a column vector are
Figure 4.4 Construction of a new coordinate system for PCA. The relationship of the first and the second component of two descriptors is displayed in the left-hand diagram. The data points are distributed, but show a trend along a straight line. In PCA, two new axes are introduced, one determined by the largest variation (y1), and the other one (y2) is perpendicular to the first.
5323X.indb 88
11/13/07 2:10:01 PM
89
Dealing with Chemical Information
correlated (distances). The existence of such correlation is a basic requirement for PCA. We can eventually achieve a simplification of the data by discarding dimensions that correspond to small variations in the data. If we work with experimental data, the principal axes corresponding to smaller variations will represent the noise in data. That is, reducing the dimensionality by discarding the small variation will have a denoising effect on the data. In our example of descriptors, the small variations may correspond to underlying physical properties that naturally produce small changes. For instance, if we look at a conformational change of a protein structure, we may find small changes in the double bond length of aromatic rings due to the conformational stress. This effect can be eliminated by discarding the corresponding principal components. If we do this for all components of our descriptors, we will find the first principal component corresponding to the linear combination of the data rows in the descriptor matrix that shows the largest variation in the data. The next principal component is another linear combination of data rows that accounts for the next largest variation in the data set and so forth. The procedure for performing a principal component analysis is described in the following sections. 4.6.1.1 Centering the Data To make the principal components comparable, we center the data to find the first principal component as a direction from the origin of our coordinate system (Figure 4.4). We achieve this by calculating the mean of the variations in distance in each row to get m mean values (one for each descriptor), and subtract them from the descriptor matrix to get
g1(1) − g1 g (1) − g 2 2 G= (1) gm − gm
g1( 2 ) − g1 g2( 2 ) − g2 (2 ) gm − gm
g1( n ) − g1 g2( n ) − g2 gm( n ) − gm
(4.33)
Each row in this matrix will sum now to zero. 4.6.1.2 Calculating the Covariance Matrix The covariance for two components i and j can be calculated as
sij =
1 n −1
n
∑(g
(k ) i
)(
)
− gi g (jk ) − g j
k =1
(4.34)
or since we are using centered data, where gi = g j = 0 , we can simplify to
5323X.indb 89
sij =
1 n −1
n
∑g
g
(k ) (k ) i j
(4.35)
k =1
11/13/07 2:10:04 PM
90
Expert Systems in Chemistry Research
We will retain the calculations for all covariances in a symmetric m by m covariance matrix S with the entry at row i and column j containing si,j while the main diagonal contains the variance for row(i).
s11 s12 S= s 1m
s21 s22 s2 m
s m1 sm 2 smm
(4.36)
The next steps are summarized in the following: • Calculate the eigenvectors and eigenvalues of the covariance matrix. • Scale the eigenvectors to obtain unit eigenvectors. • Select the eigenvector with the highest eigenvalue to define the principle component of the data set, that is, the one showing the most significant relationship between the data dimensions. • Sort the eigenvectors in order of decreasing eigenvalues, which gives the components in order of significance. • Optionally, ignore the components of lesser significance. This will reduce information content the higher the eigenvalues are. However, it will reduce the final data set to fewer dimensions than the original; in fact, the number of dimensions is reduced by the number of eigenvectors left out. • Form a feature vector; this is constructed by taking the eigenvectors that shall be kept from the list of eigenvectors and forming a matrix with these eigenvectors in the columns: FV = (eig1 eig2 … eign). • Derive the new data set by taking the transpose of the vector and multiply it on the left of the original data set, transposed:
ND = RV × AD
(4.37)
where RV is the matrix with the eigenvectors in the columns transposed (eigenvectors are now in the rows), with the most significant eigenvector at the top, and AD is the transposed mean-adjusted descriptor values, where data items are in each column with each row holding a separate dimension. ND is the new data set, with data items in columns and dimensions along rows. This procedure gives us the original data in terms of the eigenvectors we chose. If the original data set had two axes, x and y, they are now represented in terms of eigenvectors. In the case of keeping all eigenvectors for the transformation, we get the original data set rotated so that the eigenvectors are the axes. In conclusion, we have seen that the coefficients generating the linear combinations that transform the original variables into uncorrelated variables are the eigenvectors of the covariance matrix. The fraction of an eigenvalue out of the sum of all eigenvalues of the covariance matrix represents the amount of variation accounted by the corresponding eigenvector. The underlying intuition for this statement is that
5323X.indb 90
11/13/07 2:10:05 PM
91
Dealing with Chemical Information
the magnitude of each eigenvalue represents the length of the corresponding eigenvector. The length of the eigenvector is a measure of the degree of common variation it accounts for in the data or, better yet, the intensity of a trend in the data. Empirically, when PCA works well, the first two eigenvalues usually account for more than 50% of the total variation in the data.
4.6.2 Singular Value Decomposition (SVD) One mathematical procedure that can be used in the context of PCA is singular value decomposition (SVD). SVD decomposes a matrix A created from m compounds each having n descriptor components by A m× n = U m× mS m× n (V T )n× n
(4.38)
where U is the unitary matrix of the principal components sorted from highest to the lowest value; V is the unitary matrix of loadings, containing the loadings of the principal components in the columns; and S is a diagonal matrix containing the singular values s associated to the variance in the direction of each principal component in decreasing order:
0 s2
s1 0 S= 0
0 0 0 0 sm 0 m × n
0
(4.39)
We can discard the principal components with small variance and make a reduced matrix S : s1 0 S= 0
0 s2 0
0 0 sr r × r
(4.40)
For a square, symmetric matrix X, singular value decomposition is equivalent to diagonalization, or solution of the eigenvalue problem. The product of U and S yields the matrix of coefficients Tm× n in the new basis: Tm× n = U m× mS m× n
(4.41)
Inserting this in Equation 4.38 results in
A m× n = Tm× m V Tn× n ,
(4.42)
A m× n Vm× n = Tm× m V Tn× n Vm× n
(4.43)
and multiplying by V yields
5323X.indb 91
11/13/07 2:10:09 PM
92
Expert Systems in Chemistry Research
and A m× n Vm× n = Tm× m
(4.44)
PCA is equivalent to finding the SVD of X and can be used to obtain the regresˆ from sion vector estimate w ˆ − A Ty = 0 A TAw
(4.45)
by replacing A with Equation 4.38:
ˆ − (USV T )T y = 0 (USV T )T USV Tw
(4.46)
ˆ = VSU T y . VSU TUSV Tw
(4.47)
and
Multiplying both sides by V T
ˆ = V T ⋅ VSU T y V T ⋅ VSU TUSV Tw
(4.48)
ˆ = VV T ⋅ SU T y . VV T ⋅ UU T ⋅ S 2 V Tw
(4.49)
we can rearrange to
Eliminating the identity matrices VV T and UU T that do not change the results, we can reduce this equation to or
ˆ = SU T y S 2 V Tw
(4.50)
ˆ = S −1U T y . V Tw
(4.51)
Multiplying both sides by V yields
ˆ = VS −1U T y . w
(4.52)
The regression vector estimate allows us to predict molecular properties from descriptors using a simple linear regression model. This empirical approach assumes a principal relationship between the descriptor and the property. Predicting a molecular property by using descriptors for three-dimensional structures makes the only sense, if the molecular property is somehow related to the arrangement of atoms. However, by including the appropriate physicochemical atomic properties, that is, those that account for the molecular property to be predicted, we are able to easily evaluate any existing structure/property correlation in a simple manner.
5323X.indb 92
11/13/07 2:10:16 PM
Dealing with Chemical Information
93
Linear transforms that utilize orthogonal basis vectors and provide compression — such as Fourier or wavelet transforms — can be used for regression. Any descriptor consisting of n components can be written in terms of n basis vectors. In a matrix definition, we would write A m× n = Tm× n Ψ n× n
(4.53)
with A as the matrix of descriptors and T as the matrix of coefficients. Ψ is a matrix of basis vectors, each of which represents a row in the matrix. The regression term would look like ˆ − A T y = 0 A TAw
(4.54)
Since we assumed orthogonal basis vectors, it applies that Ψ TΨ = I. If we compress information, we can reduce the number of coefficients by discarding some columns to obtain a reduced matrix T. We can now replace A by TΨ to get:
ˆ = Ψ TT T y Ψ TT TTΨw
(4.55a)
ˆ = ΨΨ TT T y ⇒ ΨΨ TT TTΨw
(4.55b)
ˆ = (T TT )−1 T T y ⇒ Ψw
(4.55c)
ˆ = Ψ T (T TT )−1 T T y T ⇒w
(4.55d)
This matrices of coefficients can be reduced to where
ˆ = Ψ Tw ˆ t , w
(4.56)
ˆ t = (T TT )−1 T T y w
(4.57)
is a regression vector in the reduced matrix of coefficients, that is, those that were retained in the transform domain. The transform domain generally allows a more efficient discrimination of descriptor features since collinearity is removed and the variance of reduced coefficients is larger than in the original domain. PCA and SVD are strongly related: whereas SVD provides a factorization of a descriptor matrix, PCA provides a nearly parallel factoring, due to the analysis of eigenvalues of the covariance matrix. Singular value decomposition is a valuable approach to matrix analysis. Compared to the original matrix, the SVD of a descriptor matrix reveals its geometric aspects and is more robust to numerical errors.
5323X.indb 93
11/13/07 2:10:21 PM
94
Expert Systems in Chemistry Research
This procedure transformed the original data so that it is expressed in terms of the patterns between them, where the patterns are the lines that most closely describe the relationships between the data. This is helpful because we have now classified our data point as a combination of the contributions from each of those lines. In contrast to the coordinate system that does not indicate how a data point relates to the rest of the data, PCA tells us exactly whether a data point sits above or below the trend lines. Working with reduced eigenvectors removes the contribution from these vectors and leaves just data of higher significance. Thus, the original data can be backtransformed without losing significant information. PCA can be used, for instance, to select a smaller set of descriptors to be used for multivariate regression and artificial neural networks. A set of descriptors can be divided into subclasses, such as geometrical, topological, or constitutional. A principal component analysis including the required output can then be performed for each subclass and the descriptor nearest to the output can be selected, checking if it improved the predictive power of a model.
4.6.3 Factor Analysis Principal component analysis is used to reduce the information in many variables into a set of weighted linear combinations of those variables; it does not differentiate between common and unique variance. If latent variables have to be determined, which contribute to the common variance in a set of measured variables, factor analysis (FA) is a valuable statistical method, since it attempts to exclude unique variance from the analysis. FA is a statistical approach that can be used to analyze interrelationships among a large number of variables and to explain these variables in terms of their common underlying dimensions, or factors. This statistical approach involves finding a way of condensing the information contained in a number of original variables into a smaller set of dimensions, or factors, with a minimum loss of information [52]. There are three stages in factor analysis: (1) generation of a correlation matrix for all the variables consisting of an array of correlation coefficients of the variables; (2) extraction of factors from the correlation matrix based on the correlation coefficients of the variables; and (3) rotation of factors to maximize the relationship between the variables and some of the factors. FA requires a set of data points in matrix form, and the data must be bilinear — that is, the rows and columns must be independent of each other. The result of FA reveals the factors that are of highest significance for the data set. A general rule for determining the number of factors is the eigenvalue criterion. Factors that exhibit an eigenvalue greater than 1 usually account for at least the variance of one of the variables used in the analysis. If the number of variables is small, the analysis may result in fewer factors than really exist in the data, whereas a large number of variables may produce more factors meeting the criteria than are meaningful.
5323X.indb 94
11/13/07 2:10:21 PM
95
Dealing with Chemical Information
4.7 Transforming Descriptors Mathematical transforms are widely used in nearly all application areas where mathematical functions are involved. The basic idea behind transforms is rather simple: A transform represents any function as a summation of a series of terms of increasing frequency. In other words, data varying in space or time can be transformed into a different domain called the frequency space. In the original physical context, the term frequency describes a variation in time of periodic motion or behavior. Frequency in another context might have a different meaning. In image processing, frequency may be related to variation in brightness or color across an image; that is, it is a function of spatial coordinates rather than time. However, the basic idea of frequency is the variation of a value or quantity along a certain continuous or discrete variable. Transform techniques in computational chemistry allow separating important aspects of a signal from unimportant ones. For instance, some typical applications of Fourier transforms in spectroscopy are smoothing or filtering to enhance signalto-noise ratio, resolution enhancement, changing spectral line