ADDITIVE GmbH
Abt. Engineering & Science
Rohrwiesenstraße 2
D-61381 Friedrichsdorf/Ts.
  Tel.:  +49 6172 5905 30
  Fax:  +49 6172 77613
  email: mathematica@additive-net.de
  www:  http://www.additive.net.de

Die Mathematica Applications Library

Mathematica hat sich in den vergangenen Jahren als unentbehrliches Werkzeug im Ingenieurwesen etabliert. Es bietet dem professionellen Anwender, durch seinen mächtigen und fortschrittlichen Funktionsumfang, eine unerreichte Flexibilität bei der Analyse, Modellierung und Visualisierung der während der täglichen Ingenieurarbeit anfallenden Daten. Im Sommer 1994 startete Wolfram Research die Veröffentlichung der Mathematica Anwendungs-Pakete, die dem Benutzer sofort einsetzbare Funktionen aus speziellen Anwendungsbereichen bieten. Dem Anwender wird durch diese Pakete der schrittweise Einstieg in die Thematik bis zum Expertenwissen leicht gemacht. Die Zahl der Zusatzpakete wurde seit dieser Zeit auf über 30 Zusatzpakete erweitert.

Ein Übersicht zu den Zusatzpaketen ist aktuell im INTERNET zu finden.

http://www.additive-net.de/software/mathematica/packages/mma.packages.shtml

Experimental Data Analyst
Table of Contents

Chapter 1
Introduction to Experimental Data Analyst        
1.1 Summary and Use of EDA                                     2                         
1.1.1 Summary of the Chapters of EDA                    2                                                  
1.1.2 Using EDA                                                       3                                                                                   
1.2 The EDA Data Format                                        5
1.3 EDA Utilities and Supplied Data Sets                  8     
1.3.1 Contents of the EDA `Data` Directory            16

Chapter 2
Importing and Exporting Data
2.1 Importing Data from an ASCII File                      23
2.1.1 Introduction                                                     23
2.1.2 A Real-World Example                                    29
2.1.3 Summary of ImportData                                   34
2.2 Exporting Data to an ASCII File                          37
2.2.1 Introduction                                                     37
2.2.2 Summary of ExportData                                   41
2.3 Importing Data from a Binary File                       43
2.4 Getting Data from a Scanned Plot                      46

Chapter 3
Experimental Errors and Error Analysis
3.1 Introduction                                                          51
3.1.1 The Purpose of Error Analysis                          51
3.1.2 Different Types of Errors                                  53
3.1.3 References                                                       53
3.2 Determining the Precision                                    54
3.2.1 The Standard Deviation                                    54
3.2.2 The Reading Error                                            59
3.2.3 "THE" Error                                                       59
3.2.4 Rejection of Measurements                              60
3.3 Propagation of Errors of Precision                               61
3.3.1 Discussion and Examples                                  61
3.3.2 Finding the Error in an Average                        72
3.4 Calibration, Accuracy, and Systematic Errors      74
3.5 Summary of the Error Propagation Routines       77

Chapter 4
Fitting Data to Linear Models
by Least-Squares Techniques
4.1 Background Discussion                                        81
4.1.1 Linear Fits                                                         81
4.1.2 Least-Squares Techniques                               83
4.1.3 Fitting to Data with Experimental Errors            85
4.1.4 Evaluating the Goodness of a Fit                      87
4.1.5 References                                                       94
4.2  Curve Fitting When the Data Have No Explicit Errors  95    
4.2.1 Unwinding the Residual Calculation                         103
4.3 Curve Fitting When the Data Have Explicit Errors       105
4.4 Options, Utilities, and Details                                      113      
4.4.1 Options to LinearFit                                                 113
4.4.2 Other Routines in the LinearFit Package                 146
4.5 Summary of the LinearFit Package                             153
4.5.1 The LinearFit Routine                                              153
4.5.2 The ShowLinearFit Routine                                      159
4.5.3 The ToLinearFunction Routine                                 160

Chapter 5
Fitting Data to Nonlinear Models
5.1 Introduction                                                                  163
5.1.1 Overview of FindFit                                                    163
5.1.2 Providing Initial Parameter Values to FindFit             166
5.1.3 Comparing LinearFit and FindFit                               172
5.1.4 References                                                                181
5.2 Examples                                                                      182
5.2.1 Fitting to a Single Peak with a Background               182
5.2.2 Fitting to Two Peaks with No Background                 196
5.3 Options, Utilities, and Details                                        198
5.3.1 Options to FindFit                                                      198
5.3.2 Other Routines in the FindFit Package                      206
5.3.3 Peak Shape Routines                                                211
5.4 Summary of the FindFit Package                                  212

Chapter 6
Smoothing Data, Filling Missing Data,
and Nonparametric Fitting
6.1 Introduction                                                                  219
6.1.1 Smoothing with Averaging Techniques                      219
6.1.2 Fourier Filters                                                           228
6.1.3 Loess Fitting                                                             232
6.1.4 Using Mathematica Built-in Functions                       239
6.1.5 Smoothing an Interpolation                                       243
6.1.6 Filling Missing Data                                                   248
6.1.7 References                                                                263
6.2 Details                                                                          263
6.2.1 Nonmonotonic Data and Related Assumptions         263
6.2.2 Comparing Various Smoothing Methods                   264
6.2.3 How LoessFit Works                                                 275
6.3 Summary of the SmoothData Package                        278

Chapter 7
Robust Fitting
7.1 Introduction                                                                 283
7.1.1 Using the Median                                                     283
7.1.2 Using Weighting Techniques                                   289
7.2 Details                                                                         293
7.2.1 RobustCurveFit                                                        293
7.2.2 RobustLineFit                                                           296
7.2.3 Comparing RobustCurveFit and RobustLineFit        300
7.3 Summary of the RobustFit Package                            301

Chapter 8
Exploration and Transformation of Data

8.1 Graphical Exploration                                                                    305
8.1.1 EDAListPlot                                                                                  306
8.1.2 Histogram                                                                                     309
8.1.3 BoxPlot                                                                                          313
8.1.4 Summary of General Purpose EDA Graphics Routines       319
8.2 Transforming Data                                                                          323
8.2.1 The Data Itself Suggests a Transformation                              323
8.2.2 Transforming to Match a Model                                                 340
8.2.3 Finding Hidden Features of the Data                                        344
8.3 Estimating Parameters for Peaks                                                 349
8.3.1 Summary of FindPeaks                                                              360
8.4 References                                                                                       362
8.4.1 References to Mathematica                                                       362
8.4.2 References to Exploratory Data Analysis and Transformation        363

Chapter 1
Introduction to
Experimental Data Analyst

Experimental Data Analyst (EDA) is a collection of tools and tutorials designed specifically for the needs of the physical scientist and engineer and the students of science and engineering. Included are over 30 functions, tutorials in using the functions, tutorials in using Mathematica built-in commands and standard Mathematica packages, and discussions of the underlying theory of data analysis at a variety of levels from beginning undergraduate to practicing researcher. The tutorials make use of a variety of real experimental data drawn from many different fields, and the data is also included in the package.

The material is divided into eight chapters, beginning with this chapter which provides an overview of the contents. The chapters can be accessed as the hard-copy version, or read on line as Mathematica notebooks. The notebooks can be read with the front end included with Mathematica or with the free MathReader program available from the MathSource archive; in the former case, the notebooks can be interactive, and the reader is encouraged to experiment with the commands and supplied data sets.

If you are reading this chapter with the Mathematica front end, when you open the notebook a dialog box inquires if you wish to initialize the notebook. If you reply "Yes", the following initialization command is executed.

Needs["EDA`Master`"]

If the command is executed, and produces any error messages, you should consult your installation document for the package or see Section 1.1.2 "Using EDA"; if no error messages are produced, then this notebook is "live" and all Mathematica commands in the notebook may be executed, changed, reexecuted, etc.

The chapters do not all have to be read in order. In general, we recommend that you look over this chapter in its entirety before going on to the others. Section 1.1.1 gives further recommendations of which chapters and sections are probably prerequisites for succeeding ones.

If you are reading this chapter on line, the following sections are "closed". A solid triangle on the bottom of the cell bracket identifies a closed group. To open a closed group, point at the cell bracket with the mouse or trackball and double click; alternatively, point at the bracket, click once, and choose the Open/Close Group command from the Cell menu at the top of this window.

1.1 Summary and Use of EDA

1.1.1 Summary of the Chapters of EDA

Chapter 2 deals with methods of getting data into Mathematica so that it may be analyzed. This chapter is independent of all other chapters supplied with EDA. It discusses techniques to read and write files containing data sets; and also introduces the EDA programs ImportData and ExportData.

Chapter 3 discusses a topic which pervades data analysis in the physical sciences and engineering: error analysis. Here a tutorial is supplied. Although the level is suitable for an undergraduate student in the sciences, we are also aware of many professional researchers whose education and training have managed to miss some of the material discussed here. In addition, EDA supplies programs and constructs to simplify error analysis, and they are introduced in this chapter. The EDA functions discussed are AdjustSignificantFigures, CombineWithError, DivideWithError, PlusWithError, PowerWithError, Quadrature, SubtractWithError, and TimesWithError. Also, EDA defines Data and Datum constructs for doing propagation of errors; these constructs are introduced in this chapter. The functions and constructs discussed here are used in later chapters.

Chapters 4 through 8 are the "heart" of the analysis tools provided by EDA.

Chapter 4 introduces one of the most commonly performed tasks in data analysis: fitting data to linear models, especially straight lines and curves. Section 4.1 provides background discussions of a linear fit, least-squares techniques, evaluation of the quality of the fit, etc.; some familiarity with this material is assumed in all later sections. The remainder of Chapter 4 discusses the EDA function LinearFit that performs linear least-squares fits to data; included is a discussion of the related EDA functions ShowLinearFit and ToLinearFunction. The LinearFit function is used in all subsequent chapters. Also, many other EDA functions have similar syntax. Thus, some familiarity with its syntax and options is recommended.

Chapter 5 extends the materials of Chapter 4 to arbitrary models. It introduces the EDA function FindFit and the related functions ShowFitFunction and ToFitFunction. Also discussed are some convenience functions that define peak shapes; these are BreitWigner, Galatry, Gaussian, Lorentzian, PearsonVII, RelativisticBrietWigner, and Voigt. Little in the chapter is used in later discussions, although the FindPeaks function discussed in Chapter 8 is primarily intended as a companion to FindFit.

Chapter 6 discusses techniques to eliminate noise in data and to fill in missing values. It also discusses the related topic of fitting data when a model is not available or

appropriate. The chapter introduces the EDA functions SmoothData, LoessFit, and FillData. There is also a tutorial on using built-in Mathematica functions to smooth data, and the algorithm used by the FindPeaks program discussed in Chapter 8 is introduced here. With the exception of the algorithm of FindPeaks, nothing in later chapters depends on anything appearing here.

Chapter 7 discusses techniques to fit data to lines and curves when one or more of the data points may be "wild", i.e.,the data contains "outliers."  Alternatives to least-squares techniques should be considered in this case since an outlier can have a significant effect on the least-squares fit. The chapter introduces the EDA functions RobustCurveFit and RobustLineFit. Nothing in the chapter is required for Chapter 8.

Chapter 8 discusses what to do when the relations in a data set are not known. Graphical techniques are explored in Section 8.1. The discussion includes the EDA function EDAListPlot, which is briefly introduced in Section 1.3; the EDA functions Histogram and BoxPlot are also discussed. Section 8.2 is a tutorial on using Mathematica built-in functions to transform the data; the discussion is one of the more advanced in EDA. Nonetheless, Section 8.2 only "scratches the surface" of this topic. Finally, Section 8.3 gives a full discussion of the FindPeaks function; this function was briefly introduced in Chapter 5, and its algorithm was discussed in Section 6.1.5.

1.1.2 Using EDA

Although EDA notebooks include general purpose tutorials, EDA is also a collection of software tools, and this subsection discusses using the software. The software consists of 12 Mathematica packages, which are files written in the Mathematica language. These are the files that define EDA's functions, options, etc.

The easiest way to access an EDA function is to load the EDA`Master` file. This file knows which package file contains which programs, symbols, etc. Then when you invoke a specific function, the Mathematica kernel loads the appropriate package. The Master file can be loaded with the following.

Needs["EDA`Master`"]

If Mathematica cannot find the Master package, verify that the $Path variable includes the directory where applications such as Experimental Data Analyst are placed.

Once the EDA`Master` file has successfully loaded, you can use Mathematica just as if all the packages of EDA were loaded, except for a few differences. The three main differences are the following.

First, since all of the packages are loaded only when needed, the size of the Mathematica kernel is over a megabyte smaller than if they were actually loaded.

Second, the first time you invoke an EDA function, the package containing the function has to be loaded. Often, other packages are used by the package containing the definition of the function you have invoked, and they must be loaded, too. For example, the package containing the definition of LinearFit loads six other packages; some of these load yet other packages. For this example, invoking LinearFit loads 10 other packages besides the one containing the definition for LinearFit itself. Therefore, the first time you invoke some EDA functions, it may take a few moments before everything is loaded and the program begins its work. Second and subsequent invocations will begin their work almost immediately.