PSPP

The authors wish to thank Network Theory Ltd http://www.network-theory.co.uk for their financial support in the production of this manual.

Table of Contents

GNU PSPP

This manual is for GNU PSPP version 0.8.5, software for statistical analysis.

Copyright © 1997, 1998, 2004, 2005, 2009, 2012, 2013, 2014 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

1 Introduction

PSPP is a tool for statistical analysis of sampled data. It reads the data, analyzes the data according to commands provided, and writes the results to a listing file, to the standard output or to a window of the graphical display.

The language accepted by PSPP is similar to those accepted by SPSS statistical products. The details of PSPP’s language are given later in this manual.

PSPP produces tables and charts as output, which it can produce in several formats; currently, ASCII, PostScript, PDF, HTML, and DocBook are supported.

The current version of PSPP, 0.8.5, is incomplete in terms of its statistical procedure support. PSPP is a work in progress. The authors hope to fully support all features in the products that PSPP replaces, eventually. The authors welcome questions, comments, donations, and code submissions. See Submitting Bug Reports, for instructions on contacting the authors.

2 Your rights and obligations

PSPP is not in the public domain. It is copyrighted and there are restrictions on its distribution, but these restrictions are designed to permit everything that a good cooperating citizen would want to do. What is not allowed is to try to prevent others from further sharing any version of this program that they might get from you.

Specifically, we want to make sure that you have the right to give away copies of PSPP, that you receive source code or else can get it if you want it, that you can change these programs or use pieces of them in new free programs, and that you know you can do these things.

To make sure that everyone has such rights, we have to forbid you to deprive anyone else of these rights. For example, if you distribute copies of PSPP, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must tell them their rights.

Also, for our own protection, we must make certain that everyone finds out that there is no warranty for PSPP. If these programs are modified by someone else and passed on, we want their recipients to know that what they have is not what we distributed, so that any problems introduced by others will not reflect on our reputation.

Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone’s free use or not licensed at all.

The precise conditions of the license for PSPP are found in the GNU General Public License. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. This manual specifically is covered by the GNU Free Documentation License (see GNU Free Documentation License).

3 Invoking pspp

PSPP has two separate user interfaces. This chapter describes pspp, PSPP’s command-line driven text-based user interface. The following chapter briefly describes PSPPIRE, the graphical user interface to PSPP.

The sections below describe the pspp program’s command-line interface.

3.1 Main Options

Here is a summary of all the options, grouped by type, followed by explanations in the same order.

In the table, arguments to long options also apply to any corresponding short options.

Non-option arguments

syntax-file

Output options

-o, --output=output-file
-O option=value
-O format=format
-O device={terminal|listing}
--no-output
-e, --error-file=error-file

Language options

-I, --include=dir
-I-, --no-include
-b, --batch
-i, --interactive
-r, --no-statrc
-a, --algorithm={compatible|enhanced}
-x, --syntax={compatible|enhanced}
--syntax-encoding=encoding

Informational options

-h, --help
-V, --version

Other options

-s, --safer
--testing-mode

syntax-file

Read and execute the named syntax file. If no syntax files are specified, PSPP prompts for commands. If any syntax files are specified, PSPP by default exits after it runs them, but you may make it prompt for commands by specifying ‘-’ as an additional syntax file.

-o output-file

Write output to output-file. PSPP has several different output drivers that support output in various formats (use --help to list the available formats). Specify this option more than once to produce multiple output files, presumably in different formats.

Use ‘-’ as output-file to write output to standard output.

If no -o option is used, then PSPP writes text and CSV output to standard output and other kinds of output to whose name is based on the format, e.g. pspp.pdf for PDF output.

-O option=value

Sets an option for the output file configured by a preceding -o. Most options are specific to particular output formats. A few options that apply generically are listed below.

-O format=format

PSPP uses the extension of the file name given on -o to select an output format. Use this option to override this choice by specifying an alternate format, e.g. -o pspp.out -O html to write HTML to a file named pspp.out. Use --help to list the available formats.

-O device={terminal|listing}

Sets whether PSPP considers the output device configured by the preceding -o to be a terminal or a listing device. This affects what output will be sent to the device, as configured by the SET command’s output routing subcommands (see SET). By default, output written to standard output is considered a terminal device and other output is considered a listing device.

--no-output

Disables output entirely, if neither -o nor -O is also used. If one of those options is used, --no-output has no effect.

-e error-file

--error-file=error-file

Configures a file to receive PSPP error, warning, and note messages in plain text format. Use ‘-’ as error-file to write messages to standard output. The default error file is standard output in the absence of these options, but this is suppressed if an output device writes to standard output (or another terminal), to avoid printing every message twice. Use ‘none’ as error-file to explicitly suppress the default.

-I dir

--include=dir

Appends dir to the set of directories searched by the INCLUDE (see INCLUDE) and INSERT (see INSERT) commands.

-I-

--no-include

Clears all directories from the include path, including directories inserted in the include path by default. The default include path is . (the current directory), followed by .pspp in the user’s home directory, followed by PSPP’s system configuration directory (usually /etc/pspp or /usr/local/etc/pspp).

-b

--batch

-i

--interactive

These options forces syntax files to be interpreted in batch mode or interactive mode, respectively, rather than the default “auto” mode. See Syntax Variants, for a description of the differences.

-r

--no-statrc

Disables running rc at PSPP startup time.

-a {enhanced|compatible}

--algorithm={enhanced|compatible}

With enhanced, the default, PSPP uses the best implemented algorithms for statistical procedures. With compatible, however, PSPP will in some cases use inferior algorithms to produce the same results as the proprietary program SPSS.

Some commands have subcommands that override this setting on a per command basis.

-x {enhanced|compatible}

--syntax={enhanced|compatible}

With enhanced, the default, PSPP accepts its own extensions beyond those compatible with the proprietary program SPSS. With compatible, PSPP rejects syntax that uses these extensions.

--syntax-encoding=encoding

Specifies encoding as the encoding for syntax files named on the command line. The encoding also becomes the default encoding for other syntax files read during the PSPP session by the INCLUDE and INSERT commands. See INSERT, for the accepted forms of encoding.

--help

Prints a message describing PSPP command-line syntax and the available device formats, then exits.

-V

--version

Prints a brief message listing PSPP’s version, warranties you don’t have, copying conditions and copyright, and e-mail address for bug reports, then exits.

-s

--safer

Disables certain unsafe operations. This includes the ERASE and HOST commands, as well as use of pipes as input and output files.

--testing-mode

Invoke heuristics to assist with testing PSPP. For use by make check and similar scripts.

3.2 PDF, PostScript, and SVG Output Options

To produce output in PDF, PostScript, and SVG formats, specify -o file on the PSPP command line, optionally followed by any of the options shown in the table below to customize the output format.

PDF, PostScript, and SVG output is only available if your installation of PSPP was compiled with the Cairo library.

-O format={pdf|ps|svg}

Specify the output format. This is only necessary if the file name given on -o does not end in .pdf, .ps, or .svg.

-O paper-size=paper-size

Paper size, as a name (e.g. a4, letter) or measurements (e.g. 210x297, 8.5x11in).

The default paper size is taken from the PAPERSIZE environment variable or the file indicated by the PAPERCONF environment variable, if either variable is set. If not, and your system supports the LC_PAPER locale category, then the default paper size is taken from the locale. Otherwise, if /etc/papersize exists, the default paper size is read from it. As a last resort, A4 paper is assumed.

-O foreground-color=color

-O background-color=color

Sets color as the color to be used for the background or foreground. Color should be given in the format #RRRRGGGGBBBB, where RRRR, GGGG and BBBB are 4 character hexadecimal representations of the red, green and blue components respectively.

-O orientation=orientation

Either portrait or landscape. Default: portrait.

-O left-margin=dimension

-O right-margin=dimension

-O top-margin=dimension

-O bottom-margin=dimension

Sets the margins around the page. See below for the allowed forms of dimension Default: 0.5in.

-O prop-font=font-name

-O emph-font=font-name

-O fixed-font=font-name

Sets the font used for proportional, emphasized, or fixed-pitch text. Most systems support CSS-like font names such as “serif” and “monospace”, but a wide range of system-specific font are likely to be supported as well.

Default: proportional font serif, emphasis font serif italic, fixed-pitch font monospace.

-O font-size=font-size

Sets the size of the default fonts, in thousandths of a point. Default: 10000 (10 point).

-O line-gutter=dimension

Sets the width of white space on either side of lines that border text or graphics objects. Default: 1pt.

-O line-spacing=dimension

Sets the spacing between the lines in a double line in a table. Default: 1pt.

-O line-width=dimension

Sets the width of the lines used in tables. Default: 0.5pt.

Each dimension value above may be specified in various units based on its suffix: ‘mm’ for millimeters, ‘in’ for inches, or ‘pt’ for points. Lacking a suffix, numbers below 50 are assumed to be in inches and those about 50 are assumed to be in millimeters.

3.3 Plain Text Output Options

PSPP can produce plain text output, drawing boxes using ASCII or Unicode line drawing characters. To produce plain text output, specify -o file on the PSPP command line, optionally followed by options from the table below to customize the output format.

Plain text output is encoded in UTF-8.

-O format=txt

Specify the output format. This is only necessary if the file name given on -o does not end in .txt or .list.

-O charts={template.png|none}

Name for chart files included in output. The value should be a file name that includes a single ‘#’ and ends in png. When a chart is output, the ‘#’ is replaced by the chart number. The default is the file name specified on -o with the extension stripped off and replaced by -#.png.

Specify none to disable chart output. Charts are always disabled if your installation of PSPP was compiled without the Cairo library.

-O foreground-color=color

-O background-color=color

Sets color as the color to be used for the background or foreground to be used for charts. Color should be given in the format #RRRRGGGGBBBB, where RRRR, GGGG and BBBB are 4 character hexadecimal representations of the red, green and blue components respectively. If charts are disabled, this option has no effect.

-O paginate=boolean

If set, PSPP writes an ASCII formfeed the end of every page. Default: off.

-O headers=boolean

If enabled, PSPP prints two lines of header information giving title and subtitle, page number, date and time, and PSPP version are printed at the top of every page. These two lines are in addition to any top margin requested. Default: off.

-O length=line-count

Physical length of a page. Headers and margins are subtracted from this value. You may specify the number of lines as a number, or for screen output you may specify auto to track the height of the terminal as it changes. Default: 66.

-O width=character-count

Width of a page, in characters. Margins are subtracted from this value. For screen output you may specify auto in place of a number to track the width of the terminal as it changes. Default: 79.

-O top-margin=top-margin-lines

Length of the top margin, in lines. PSPP subtracts this value from the page length. Default: 0.

-O bottom-margin=bottom-margin-lines

Length of the bottom margin, in lines. PSPP subtracts this value from the page length. Default: 0.

-O box={ascii|unicode}

Sets the characters used for lines in tables. If set to ascii the characters ‘-’, ‘|’, and ‘+’ for single-width lines and ‘=’ and ‘#’ for double-width lines are used. If set to unicode then Unicode box drawing characters will be used. The default is unicode if the locale’s character encoding is "UTF-8" or ascii otherwise.

-O emphasis={none|bold|underline}

How to emphasize text. Bold and underline emphasis are achieved with overstriking, which may not be supported by all the software to which you might pass the output. Default: none.

3.4 HTML Output Options

To produce output in HTML format, specify -o file on the PSPP command line, optionally followed by any of the options shown in the table below to customize the output format.

-O format=html

Specify the output format. This is only necessary if the file name given on -o does not end in .html.

-O charts={template.png|none}

Sets the name used for chart files. See Plain Text Output Options, for details.

-O borders=boolean

Decorate the tables with borders. If set to false, the tables produced will have no borders. The default value is true.

-O css=boolean

Use cascading style sheets. Cascading style sheets give an improved appearance and can be used to produce pages which fit a certain web site’s style. The default value is true.

3.5 OpenDocument Output Options

To produce output as an OpenDocument text (ODT) document, specify -o file on the PSPP command line. If file does not end in .odt, you must also specify -O format=odt.

ODT support is only available if your installation of PSPP was compiled with the libxml2 library.

The OpenDocument output format does not have any configurable options.

3.6 Comma-Separated Value Output Options

To produce output in comma-separated value (CSV) format, specify -o file on the PSPP command line, optionally followed by any of the options shown in the table below to customize the output format.

-O format=csv

Specify the output format. This is only necessary if the file name given on -o does not end in .csv.

-O separator=field-separator

Sets the character used to separate fields. Default: a comma (‘,’).

-O quote=qualifier

Sets qualifier as the character used to quote fields that contain white space, the separator (or any of the characters in the separator, if it contains more than one character), or the quote character itself. If qualifier is longer than one character, only the first character is used; if qualifier is the empty string, then fields are never quoted.

-O titles=boolean

Whether table titles (brief descriptions) should be printed. Default: on.

-O captions=boolean

Whether table captions (more extensive descriptions) should be printed. Default: on.

The CSV format used is an extension to that specified in RFC 4180:

Tables

Each table row is output on a separate line, and each column is output as a field. The contents of a cell that spans multiple rows or columns is output only for the top-left row and column; the rest are output as empty fields.

Titles

When a table has a title and titles are enabled, the title is output just above the table as a single field prefixed by ‘Table:’.

Captions

When a table has a caption and captions are enabled, the caption is output just below the table as a single field prefixed by ‘Caption:’.

Footnotes

Within a table, footnote markers are output as bracketed letters following the cell’s contents, e.g. ‘[a]’, ‘[b]’, ... The footnotes themselves are output following the body of the table, as a separate two-column table introduced with a line that says ‘Footnotes:’. Each row in the table represent one footnote: the first column is the marker, the second column is the text.

Text

Text in output is printed as a field on a line by itself. The TITLE and SUBTITLE produce similar output, prefixed by ‘Title:’ or ‘Subtitle:’, respectively.

Messages

Errors, warnings, and notes are printed the same way as text.

Charts

Charts are not included in CSV output.

Successive output items are separated by a blank line.

4 Invoking psppire

4.1 The graphic user interface

The PSPPIRE graphic user interface for PSPP can perform all functionality of the command line interface. In addition it gives an instantaneous view of the data, variables and statistical output.

The graphic user interface can be started by typing psppire at a command prompt. Alternatively many systems have a system of interactive menus or buttons from which psppire can be started by a series of mouse clicks.

Once the principles of the PSPP system are understood, the graphic user interface is designed to be largely intuitive, and for this reason is covered only very briefly by this manual.

5 Using PSPP

PSPP is a tool for the statistical analysis of sampled data. You can use it to discover patterns in the data, to explain differences in one subset of data in terms of another subset and to find out whether certain beliefs about the data are justified. This chapter does not attempt to introduce the theory behind the statistical analysis, but it shows how such analysis can be performed using PSPP.

For the purposes of this tutorial, it is assumed that you are using PSPP in its interactive mode from the command line. However, the example commands can also be typed into a file and executed in a post-hoc mode by typing ‘pspp filename’ at a shell prompt, where filename is the name of the file containing the commands. Alternatively, from the graphical interface, you can select File → New → Syntax to open a new syntax window and use the Run menu when a syntax fragment is ready to be executed. Whichever method you choose, the syntax is identical.

When using the interactive method, PSPP tells you that it’s waiting for your data with a string like PSPP> or data>. In the examples of this chapter, whenever you see text like this, it indicates the prompt displayed by PSPP, not something that you should type.

Throughout this chapter reference is made to a number of sample data files. So that you can try the examples for yourself, you should have received these files along with your copy of PSPP.¹

Please note: Normally these files are installed in the directory /usr/local/share/pspp/examples. If however your system administrator or operating system vendor has chosen to install them in a different location, you will have to adjust the examples accordingly.

5.1 Preparation of Data Files

Before analysis can commence, the data must be loaded into PSPP and arranged such that both PSPP and humans can understand what the data represents. There are two aspects of data:

• The variables — these are the parameters of a quantity which has been measured or estimated in some way. For example height, weight and geographic location are all variables.
• The observations (also called ‘cases’) of the variables — each observation represents an instance when the variables were measured or observed.

For example, a data set which has the variables height, weight, and name, might have the observations:

1881 89.2 Ahmed
1192 107.01 Frank
1230 67 Julie

The following sections explain how to define a dataset.

5.1.1 Defining Variables

Variables come in two basic types, viz: numeric and string. Variables such as age, height and satisfaction are numeric, whereas name is a string variable. String variables are best reserved for commentary data to assist the human observer. However they can also be used for nominal or categorical data.

Example 5.1 defines two variables forename and height, and reads data into them by manual input.

PSPP> data list list /forename (A12) height.
PSPP> begin data.
data> Ahmed 188
data> Bertram 167
data> Catherine 134.231
data> David 109.1
data> end data
PSPP>

There are several things to note about this example.

• The words ‘data list list’ are an example of the DATA LIST command. See DATA LIST. It tells PSPP to prepare for reading data. The word ‘list’ intentionally appears twice. The first occurrence is part of the DATA LIST call, whilst the second tells PSPP that the data is to be read as free format data with one record per line.
• The ‘/’ character is important. It marks the start of the list of variables which you wish to define.
• The text ‘forename’ is the name of the first variable, and ‘(A12)’ says that the variable forename is a string variable and that its maximum length is 12 bytes. The second variable’s name is specified by the text ‘height’. Since no format is given, this variable has the default format. Normally the default format expects numeric data, which should be entered in the locale of the operating system. Thus, the example is correct for English locales and other locales which use a period (‘.’) as the decimal separator. However if you are using a system with a locale which uses the comma (‘,’) as the decimal separator, then you should in the subsequent lines substitute ‘.’ with ‘,’. Alternatively, you could explicitly tell PSPP that the height variable is to be read using a period as its decimal separator by appending the text ‘DOT8.3’ after the word ‘height’. For more information on data formats, see Input and Output Formats.
• Normally, PSPP displays the prompt PSPP> whenever it’s expecting a command. However, when it’s expecting data, the prompt changes to data> so that you know to enter data and not a command.
• At the end of every command there is a terminating ‘.’ which tells PSPP that the end of a command has been encountered. You should not enter ‘.’ when data is expected (ie. when the data> prompt is current) since it is appropriate only for terminating commands.

5.1.2 Listing the data

Once the data has been entered, you could type

PSPP> list /format=numbered.

to list the data. The optional text ‘/format=numbered’ requests the case numbers to be shown along with the data. It should show the following output:

Case#     forename   height
----- ------------ --------
    1 Ahmed          188.00 
    2 Bertram        167.00 
    3 Catherine      134.23 
    4 David          109.10 

Note that the numeric variable height is displayed to 2 decimal places, because the format for that variable is ‘F8.2’. For a complete description of the LIST command, see LIST.

5.1.3 Reading data from a text file

The previous example showed how to define a set of variables and to manually enter the data for those variables. Manual entering of data is tedious work, and often a file containing the data will be have been previously prepared. Let us assume that you have a file called mydata.dat containing the ascii encoded data:

Ahmed          188.00 
Bertram        167.00 
Catherine      134.23 
David          109.10 
              .
              .
              .
Zachariah      113.02

You can can tell the DATA LIST command to read the data directly from this file instead of by manual entry, with a command like:

PSPP> data list file='mydata.dat' list /forename (A12) height.

Notice however, that it is still necessary to specify the names of the variables and their formats, since this information is not contained in the file. It is also possible to specify the file’s character encoding and other parameters. For full details refer to see DATA LIST.

5.1.4 Reading data from a pre-prepared PSPP file

When working with other PSPP users, or users of other software which uses the PSPP data format, you may be given the data in a pre-prepared PSPP file. Such files contain not only the data, but the variable definitions, along with their formats, labels and other meta-data. Conventionally, these files (sometimes called “system” files) have the suffix .sav, but that is not mandatory. The following syntax loads a file called my-file.sav.

PSPP> get file='my-file.sav'.

You will encounter several instances of this in future examples.

5.1.5 Saving data to a PSPP file.

If you want to save your data, along with the variable definitions so that you or other PSPP users can use it later, you can do this with the SAVE command.

The following syntax will save the existing data and variables to a file called my-new-file.sav.

PSPP> save outfile='my-new-file.sav'.

If my-new-file.sav already exists, then it will be overwritten. Otherwise it will be created.

5.1.6 Reading data from other sources

Sometimes it’s useful to be able to read data from comma separated text, from spreadsheets, databases or other sources. In these instances you should use the GET DATA command (see GET DATA).

5.1.7 Exiting PSPP

Use the FINISH command to exit PSPP:

PSPP> finish.

5.2 Data Screening and Transformation

Once data has been entered, it is often desirable, or even necessary, to transform it in some way before performing analysis upon it. At the very least, it’s good practice to check for errors.

5.2.1 Identifying incorrect data

Data from real sources is rarely error free. PSPP has a number of procedures which can be used to help identify data which might be incorrect.

The DESCRIPTIVES command (see DESCRIPTIVES) is used to generate simple linear statistics for a dataset. It is also useful for identifying potential problems in the data. The example file physiology.sav contains a number of physiological measurements of a sample of healthy adults selected at random. However, the data entry clerk made a number of mistakes when entering the data. Example 5.2 illustrates the use of DESCRIPTIVES to screen this data and identify the erroneous values.

PSPP> get file='/usr/local/share/pspp/examples/physiology.sav'.
PSPP> descriptives sex, weight, height.

DESCRIPTIVES.  Valid cases = 40; cases with missing value(s) = 0.
+--------#--+-------+-------+-------+-------+
|Variable# N|  Mean |Std Dev|Minimum|Maximum|
#========#==#=======#=======#=======#=======#
|sex     #40|    .45|    .50|    .00|   1.00|
|height  #40|1677.12| 262.87| 179.00|1903.00|
|weight  #40|  72.12|  26.70| -55.60|  92.07|
+--------#--+-------+-------+-------+-------+

In the output of Example 5.2, the most interesting column is the minimum value. The weight variable has a minimum value of less than zero, which is clearly erroneous. Similarly, the height variable’s minimum value seems to be very low. In fact, it is more than 5 standard deviations from the mean, and is a seemingly bizarre height for an adult person. We can examine the data in more detail with the EXAMINE command (see EXAMINE):

In Example 5.3 you can see that the lowest value of height is 179 (which we suspect to be erroneous), but the second lowest is 1598 which we know from the DESCRIPTIVES command is within 1 standard deviation from the mean. Similarly the weight variable has a lowest value which is negative but a plausible value for the second lowest value. This suggests that the two extreme values are outliers and probably represent data entry errors.

PSPP> examine height, weight /statistics=extreme(3).    

#===============================#===========#=======#
#                               #Case Number| Value #
#===============================#===========#=======#
#Height in millimetres Highest 1#         14|1903.00#
#                              2#         15|1884.00#
#                              3#         12|1801.65#
#                     ----------#-----------+-------#
#                       Lowest 1#         30| 179.00#
#                              2#         31|1598.00#
#                              3#         28|1601.00#
#                     ----------#-----------+-------#
#Weight in kilograms   Highest 1#         13|  92.07#
#                              2#          5|  92.07#
#                              3#         17|  91.74#
#                     ----------#-----------+-------#
#                       Lowest 1#         38| -55.60#
#                              2#         39|  54.48#
#                              3#         33|  55.45#
#===============================#===========#=======#

5.2.2 Dealing with suspicious data

If possible, suspect data should be checked and re-measured. However, this may not always be feasible, in which case the researcher may decide to disregard these values. PSPP has a feature whereby data can assume the special value ‘SYSMIS’, and will be disregarded in future analysis. See Missing Observations. You can set the two suspect values to the ‘SYSMIS’ value using the RECODE command.

PSPP> recode height (179 = SYSMIS).
PSPP> recode weight (LOWEST THRU 0 = SYSMIS).

The first command says that for any observation which has a height value of 179, that value should be changed to the SYSMIS value. The second command says that any weight values of zero or less should be changed to SYSMIS. From now on, they will be ignored in analysis. For detailed information about the RECODE command see RECODE.

If you now re-run the DESCRIPTIVES or EXAMINE commands in Example 5.2 and Example 5.3 you will see a data summary with more plausible parameters. You will also notice that the data summaries indicate the two missing values.

5.2.3 Inverting negatively coded variables

Data entry errors are not the only reason for wanting to recode data. The sample file hotel.sav comprises data gathered from a customer satisfaction survey of clients at a particular hotel. In Example 5.4, this file is loaded for analysis. The line display dictionary. tells PSPP to display the variables and associated data. The output from this command has been omitted from the example for the sake of clarity, but you will notice that each of the variables v1, v2 … v5 are measured on a 5 point Likert scale, with 1 meaning “Strongly disagree” and 5 meaning “Strongly agree”. Whilst variables v1, v2 and v4 record responses to a positively posed question, variables v3 and v5 are responses to negatively worded questions. In order to perform meaningful analysis, we need to recode the variables so that they all measure in the same direction. We could use the RECODE command, with syntax such as:

recode v3 (1 = 5) (2 = 4) (4 = 2) (5 = 1).

However an easier and more elegant way uses the COMPUTE command (see COMPUTE). Since the variables are Likert variables in the range (1 … 5), subtracting their value from 6 has the effect of inverting them:

compute var = 6 - var.

Example 5.4 uses this technique to recode the variables v3 and v5. After applying COMPUTE for both variables, all subsequent commands will use the inverted values.

5.2.4 Testing data consistency

A sensible check to perform on survey data is the calculation of reliability. This gives the statistician some confidence that the questionnaires have been completed thoughtfully. If you examine the labels of variables v1, v3 and v4, you will notice that they ask very similar questions. One would therefore expect the values of these variables (after recoding) to closely follow one another, and we can test that with the RELIABILITY command (see RELIABILITY). Example 5.4 shows a PSPP session where the user (after recoding negatively scaled variables) requests reliability statistics for v1, v3 and v4.

PSPP> get file='/usr/local/share/pspp/examples/hotel.sav'.
PSPP> display dictionary.
PSPP> * recode negatively worded questions.
PSPP> compute v3 = 6 - v3.
PSPP> compute v5 = 6 - v5.
PSPP> reliability v1, v3, v4.

1.1 RELIABILITY.  Case Processing Summary
#==============#==#======#
#              # N|   %  #
#==============#==#======#
#Cases Valid   #17|100.00#
#      Excluded# 0|   .00#
#      Total   #17|100.00#
#==============#==#======#

1.2 RELIABILITY.  Reliability Statistics
#================#==========#
#Cronbach's Alpha#N of Items#
#================#==========#
#             .81#         3#
#================#==========#

As a rule of thumb, many statisticians consider a value of Cronbach’s Alpha of 0.7 or higher to indicate reliable data. Here, the value is 0.81 so the data and the recoding that we performed are vindicated.

5.2.5 Testing for normality

Many statistical tests rely upon certain properties of the data. One common property, upon which many linear tests depend, is that of normality — the data must have been drawn from a normal distribution. It is necessary then to ensure normality before deciding upon the test procedure to use. One way to do this uses the EXAMINE command.

In Example 5.5, a researcher was examining the failure rates of equipment produced by an engineering company. The file repairs.sav contains the mean time between failures (mtbf) of some items of equipment subject to the study. Before performing linear analysis on the data, the researcher wanted to ascertain that the data is normally distributed.

A normal distribution has a skewness and kurtosis of zero. Looking at the skewness of mtbf in Example 5.5 it is clear that the mtbf figures have a lot of positive skew and are therefore not drawn from a normally distributed variable. Positive skew can often be compensated for by applying a logarithmic transformation. This is done with the COMPUTE command in the line

compute mtbf_ln = ln (mtbf).

Rather than redefining the existing variable, this use of COMPUTE defines a new variable mtbf_ln which is the natural logarithm of mtbf. The final command in this example calls EXAMINE on this new variable, and it can be seen from the results that both the skewness and kurtosis for mtbf_ln are very close to zero. This provides some confidence that the mtbf_ln variable is normally distributed and thus safe for linear analysis. In the event that no suitable transformation can be found, then it would be worth considering an appropriate non-parametric test instead of a linear one. See NPAR TESTS, for information about non-parametric tests.

PSPP> get file='/usr/local/share/pspp/examples/repairs.sav'.
PSPP> examine mtbf 
                /statistics=descriptives.
PSPP> compute mtbf_ln = ln (mtbf).
PSPP> examine mtbf_ln
                /statistics=descriptives.

1.2 EXAMINE.  Descriptives
#====================================================#=========#==========#
#                                                    #Statistic|Std. Error#
#====================================================#=========#==========#
#mtbf    Mean                                        #   8.32  |   1.62   #
#        95% Confidence Interval for Mean Lower Bound#   4.85  |          #
#                                         Upper Bound#  11.79  |          #
#        5% Trimmed Mean                             #   7.69  |          #
#        Median                                      #   8.12  |          #
#        Variance                                    #  39.21  |          #
#        Std. Deviation                              #   6.26  |          #
#        Minimum                                     #   1.63  |          #
#        Maximum                                     #  26.47  |          #
#        Range                                       #  24.84  |          #
#        Interquartile Range                         #   5.83  |          #
#        Skewness                                    #   1.85  |    .58   #
#        Kurtosis                                    #   4.49  |   1.12   #
#====================================================#=========#==========#

2.2 EXAMINE.  Descriptives
#====================================================#=========#==========#
#                                                    #Statistic|Std. Error#
#====================================================#=========#==========#
#mtbf_ln Mean                                        #   1.88  |    .19   #
#        95% Confidence Interval for Mean Lower Bound#   1.47  |          #
#                                         Upper Bound#   2.29  |          #
#        5% Trimmed Mean                             #   1.88  |          #
#        Median                                      #   2.09  |          #
#        Variance                                    #   .54   |          #
#        Std. Deviation                              #   .74   |          #
#        Minimum                                     #   .49   |          #
#        Maximum                                     #   3.28  |          #
#        Range                                       #   2.79  |          #
#        Interquartile Range                         #   .92   |          #
#        Skewness                                    #   -.16  |    .58   #
#        Kurtosis                                    #   -.09  |   1.12   #
#====================================================#=========#==========#

5.3 Hypothesis Testing

One of the most fundamental purposes of statistical analysis is hypothesis testing. Researchers commonly need to test hypotheses about a set of data. For example, she might want to test whether one set of data comes from the same distribution as another, or whether the mean of a dataset significantly differs from a particular value. This section presents just some of the possible tests that PSPP offers.

The researcher starts by making a null hypothesis. Often this is a hypothesis which he suspects to be false. For example, if he suspects that A is greater than B he will state the null hypothesis as A = B.²

The p-value is a recurring concept in hypothesis testing. It is the highest acceptable probability that the evidence implying a null hypothesis is false, could have been obtained when the null hypothesis is in fact true. Note that this is not the same as “the probability of making an error” nor is it the same as “the probability of rejecting a hypothesis when it is true”.

5.3.1 Testing for differences of means

A common statistical test involves hypotheses about means. The T-TEST command is used to find out whether or not two separate subsets have the same mean.

Example 5.6 uses the file physiology.sav previously encountered. A researcher suspected that the heights and core body temperature of persons might be different depending upon their sex. To investigate this, he posed two null hypotheses:

• The mean heights of males and females in the population are equal.
• The mean body temperature of males and females in the population are equal.

For the purposes of the investigation the researcher decided to use a p-value of 0.05.

In addition to the T-test, the T-TEST command also performs the Levene test for equal variances. If the variances are equal, then a more powerful form of the T-test can be used. However if it is unsafe to assume equal variances, then an alternative calculation is necessary. PSPP performs both calculations.

For the height variable, the output shows the significance of the Levene test to be 0.33 which means there is a 33% probability that the Levene test produces this outcome when the variances are equal. Had the significance been less than 0.05, then it would have been unsafe to assume that the variances were equal. However, because the value is higher than 0.05 the homogeneity of variances assumption is safe and the “Equal Variances” row (the more powerful test) can be used. Examining this row, the two tailed significance for the height t-test is less than 0.05, so it is safe to reject the null hypothesis and conclude that the mean heights of males and females are unequal.

For the temperature variable, the significance of the Levene test is 0.58 so again, it is safe to use the row for equal variances. The equal variances row indicates that the two tailed significance for temperature is 0.20. Since this is greater than 0.05 we must reject the null hypothesis and conclude that there is insufficient evidence to suggest that the body temperature of male and female persons are different.

PSPP> get file='/usr/local/share/pspp/examples/physiology.sav'.
PSPP> recode height (179 = SYSMIS).
PSPP> t-test group=sex(0,1) /variables = height temperature.

1.1 T-TEST.  Group Statistics
#==================#==#=======#==============#========#
#              sex | N|  Mean |Std. Deviation|SE. Mean#
#==================#==#=======#==============#========#
#height      Male  |22|1796.49|         49.71|   10.60#
#            Female|17|1610.77|         25.43|    6.17#
#temperature Male  |22|  36.68|          1.95|     .42#
#            Female|18|  37.43|          1.61|     .38#
#==================#==#=======#==============#========#
1.2 T-TEST.  Independent Samples Test
#===========================#=========#===============================   =#
#                           # Levene's| t-test for Equality of Means      #
#                           #----+----+------+-----+------+---------+-   -#
#                           #    |    |      |     |      |         |     #
#                           #    |    |      |     |Sig. 2|         |     #
#                           #  F |Sig.|   t  |  df |tailed|Mean Diff|     #
#===========================#====#====#======#=====#======#=========#=   =#
#height      Equal variances# .97| .33| 14.02|37.00|   .00|   185.72| ... #
#          Unequal variances#    |    | 15.15|32.71|   .00|   185.72| ... #
#temperature Equal variances# .31| .58| -1.31|38.00|   .20|     -.75| ... #
#          Unequal variances#    |    | -1.33|37.99|   .19|     -.75| ... #
#===========================#====#====#======#=====#======#=========#=   =#

5.3.2 Linear Regression

Linear regression is a technique used to investigate if and how a variable is linearly related to others. If a variable is found to be linearly related, then this can be used to predict future values of that variable.

In example Example 5.7, the service department of the company wanted to be able to predict the time to repair equipment, in order to improve the accuracy of their quotations. It was suggested that the time to repair might be related to the time between failures and the duty cycle of the equipment. The p-value of 0.1 was chosen for this investigation. In order to investigate this hypothesis, the REGRESSION command was used. This command not only tests if the variables are related, but also identifies the potential linear relationship. See REGRESSION.

PSPP> get file='/usr/local/share/pspp/examples/repairs.sav'.
PSPP> regression /variables = mtbf duty_cycle /dependent = mttr.
PSPP> regression /variables = mtbf /dependent = mttr.

1.3(1) REGRESSION.  Coefficients
#=============================================#====#==========#====#=====#
#                                             #  B |Std. Error|Beta|  t  #
#========#====================================#====#==========#====#=====#
#        |(Constant)                          #9.81|      1.50| .00| 6.54#
#        |Mean time between failures (months) #3.10|       .10| .99|32.43#
#        |Ratio of working to non-working time#1.09|      1.78| .02|  .61#
#        |                                    #    |          |    |     #
#========#====================================#====#==========#====#=====#

1.3(2) REGRESSION.  Coefficients
#=============================================#============#
#                                             #Significance#
#========#====================================#============#
#        |(Constant)                          #         .10#
#        |Mean time between failures (months) #         .00#
#        |Ratio of working to non-working time#         .55#
#        |                                    #            #
#========#====================================#============#
2.3(1) REGRESSION.  Coefficients
#============================================#=====#==========#====#=====#
#                                            #  B  |Std. Error|Beta|  t  #
#========#===================================#=====#==========#====#=====#
#        |(Constant)                         #10.50|       .96| .00|10.96#
#        |Mean time between failures (months)# 3.11|       .09| .99|33.39#
#        |                                   #     |          |    |     #
#========#===================================#=====#==========#====#=====#

2.3(2) REGRESSION.  Coefficients
#============================================#============#
#                                            #Significance#
#========#===================================#============#
#        |(Constant)                         #         .06#
#        |Mean time between failures (months)#         .00#
#        |                                   #            #
#========#===================================#============#

The coefficients in the first table suggest that the formula mttr = 9.81 + 3.1 \times mtbf + 1.09 \times duty_cycle can be used to predict the time to repair. However, the significance value for the duty_cycle coefficient is very high, which would make this an unsafe predictor. For this reason, the test was repeated, but omitting the duty_cycle variable. This time, the significance of all coefficients no higher than 0.06, suggesting that at the 0.06 level, the formula mttr = 10.5 + 3.11 \times mtbf is a reliable predictor of the time to repair.

6 The PSPP language

This chapter discusses elements common to many PSPP commands. Later chapters will describe individual commands in detail.

6.1 Tokens

PSPP divides most syntax file lines into series of short chunks called tokens. Tokens are then grouped to form commands, each of which tells PSPP to take some action—read in data, write out data, perform a statistical procedure, etc. Each type of token is described below.

Identifiers

Identifiers are names that typically specify variables, commands, or subcommands. The first character in an identifier must be a letter, ‘#’, or ‘@’. The remaining characters in the identifier must be letters, digits, or one of the following special characters:

. _ $ # @

Identifiers may be any length, but only the first 64 bytes are significant. Identifiers are not case-sensitive: foobar, Foobar, FooBar, FOOBAR, and FoObaR are different representations of the same identifier.

Some identifiers are reserved. Reserved identifiers may not be used in any context besides those explicitly described in this manual. The reserved identifiers are:

ALL AND BY EQ GE GT LE LT NE NOT OR TO WITH

Keywords

Keywords are a subclass of identifiers that form a fixed part of command syntax. For example, command and subcommand names are keywords. Keywords may be abbreviated to their first 3 characters if this abbreviation is unambiguous. (Unique abbreviations of 3 or more characters are also accepted: ‘FRE’, ‘FREQ’, and ‘FREQUENCIES’ are equivalent when the last is a keyword.)

Reserved identifiers are always used as keywords. Other identifiers may be used both as keywords and as user-defined identifiers, such as variable names.

Numbers

Numbers are expressed in decimal. A decimal point is optional. Numbers may be expressed in scientific notation by adding ‘e’ and a base-10 exponent, so that ‘1.234e3’ has the value 1234. Here are some more examples of valid numbers:

-5  3.14159265359  1e100  -.707  8945.

Negative numbers are expressed with a ‘-’ prefix. However, in situations where a literal ‘-’ token is expected, what appears to be a negative number is treated as ‘-’ followed by a positive number.

No white space is allowed within a number token, except for horizontal white space between ‘-’ and the rest of the number.

The last example above, ‘8945.’ will be interpreted as two tokens, ‘8945’ and ‘.’, if it is the last token on a line. See Forming commands of tokens.

Strings

Strings are literal sequences of characters enclosed in pairs of single quotes (‘'’) or double quotes (‘"’). To include the character used for quoting in the string, double it, e.g. ‘'it''s an apostrophe'’. White space and case of letters are significant inside strings.

Strings can be concatenated using ‘+’, so that ‘"a" + 'b' + 'c'’ is equivalent to ‘'abc'’. So that a long string may be broken across lines, a line break may precede or follow, or both precede and follow, the ‘+’. (However, an entirely blank line preceding or following the ‘+’ is interpreted as ending the current command.)

Strings may also be expressed as hexadecimal character values by prefixing the initial quote character by ‘x’ or ‘X’. Regardless of the syntax file or active dataset’s encoding, the hexadecimal digits in the string are interpreted as Unicode characters in UTF-8 encoding.

Individual Unicode code points may also be expressed by specifying the hexadecimal code point number in single or double quotes preceded by ‘u’ or ‘U’. For example, Unicode code point U+1D11E, the musical G clef character, could be expressed as U'1D11E'. Invalid Unicode code points (above U+10FFFF or in between U+D800 and U+DFFF) are not allowed.

When strings are concatenated with ‘+’, each segment’s prefix is considered individually. For example, 'The G clef symbol is:' + u"1d11e" + "." inserts a G clef symbol in the middle of an otherwise plain text string.

Punctuators and Operators

These tokens are the punctuators and operators:

, / = ( ) + - * / ** < <= <> > >= ~= & | .

Most of these appear within the syntax of commands, but the period (‘.’) punctuator is used only at the end of a command. It is a punctuator only as the last character on a line (except white space). When it is the last non-space character on a line, a period is not treated as part of another token, even if it would otherwise be part of, e.g., an identifier or a floating-point number.

6.2 Forming commands of tokens

Most PSPP commands share a common structure. A command begins with a command name, such as FREQUENCIES, DATA LIST, or N OF CASES. The command name may be abbreviated to its first word, and each word in the command name may be abbreviated to its first three or more characters, where these abbreviations are unambiguous.

The command name may be followed by one or more subcommands. Each subcommand begins with a subcommand name, which may be abbreviated to its first three letters. Some subcommands accept a series of one or more specifications, which follow the subcommand name, optionally separated from it by an equals sign (‘=’). Specifications may be separated from each other by commas or spaces. Each subcommand must be separated from the next (if any) by a forward slash (‘/’).

There are multiple ways to mark the end of a command. The most common way is to end the last line of the command with a period (‘.’) as described in the previous section (see Tokens). A blank line, or one that consists only of white space or comments, also ends a command.

6.3 Syntax Variants

There are three variants of command syntax, which vary only in how they detect the end of one command and the start of the next.

In interactive mode, which is the default for syntax typed at a command prompt, a period as the last non-blank character on a line ends a command. A blank line also ends a command.

In batch mode, an end-of-line period or a blank line also ends a command. Additionally, it treats any line that has a non-blank character in the leftmost column as beginning a new command. Thus, in batch mode the second and subsequent lines in a command must be indented.

Regardless of the syntax mode, a plus sign, minus sign, or period in the leftmost column of a line is ignored and causes that line to begin a new command. This is most useful in batch mode, in which the first line of a new command could not otherwise be indented, but it is accepted regardless of syntax mode.

The default mode for reading commands from a file is auto mode. It is the same as batch mode, except that a line with a non-blank in the leftmost column only starts a new command if that line begins with the name of a PSPP command. This correctly interprets most valid PSPP syntax files regardless of the syntax mode for which they are intended.

The --interactive (or -i) or --batch (or -b) options set the syntax mode for files listed on the PSPP command line. See Main Options, for more details.

6.4 Types of Commands

Commands in PSPP are divided roughly into six categories:

Utility commands

Set or display various global options that affect PSPP operations. May appear anywhere in a syntax file. See Utility commands.

File definition commands

Give instructions for reading data from text files or from special binary “system files”. Most of these commands replace any previous data or variables with new data or variables. At least one file definition command must appear before the first command in any of the categories below. See Data Input and Output.

Input program commands

Though rarely used, these provide tools for reading data files in arbitrary textual or binary formats. See INPUT PROGRAM.

Transformations

Perform operations on data and write data to output files. Transformations are not carried out until a procedure is executed.

Restricted transformations

Transformations that cannot appear in certain contexts. See Order of Commands, for details.

Procedures

Analyze data, writing results of analyses to the listing file. Cause transformations specified earlier in the file to be performed. In a more general sense, a procedure is any command that causes the active dataset (the data) to be read.

6.5 Order of Commands

PSPP does not place many restrictions on ordering of commands. The main restriction is that variables must be defined before they are otherwise referenced. This section describes the details of command ordering, but most users will have no need to refer to them.

PSPP possesses five internal states, called initial, input-program file-type, transformation, and procedure states. (Please note the distinction between the INPUT PROGRAM and FILE TYPE commands and the input-program and file-type states.)

PSPP starts in the initial state. Each successful completion of a command may cause a state transition. Each type of command has its own rules for state transitions:

Utility commands

• Valid in any state.
• Do not cause state transitions. Exception: when N OF CASES is executed in the procedure state, it causes a transition to the transformation state.

DATA LIST

• Valid in any state.
• When executed in the initial or procedure state, causes a transition to the transformation state.
• Clears the active dataset if executed in the procedure or transformation state.

INPUT PROGRAM

• Invalid in input-program and file-type states.
• Causes a transition to the intput-program state.
• Clears the active dataset.

FILE TYPE

• Invalid in intput-program and file-type states.
• Causes a transition to the file-type state.
• Clears the active dataset.

Other file definition commands

• Invalid in input-program and file-type states.
• Cause a transition to the transformation state.
• Clear the active dataset, except for ADD FILES, MATCH FILES, and UPDATE.

Transformations

• Invalid in initial and file-type states.
• Cause a transition to the transformation state.

Restricted transformations

• Invalid in initial, input-program, and file-type states.
• Cause a transition to the transformation state.

Procedures

• Invalid in initial, input-program, and file-type states.
• Cause a transition to the procedure state.

6.6 Handling missing observations

PSPP includes special support for unknown numeric data values. Missing observations are assigned a special value, called the system-missing value. This “value” actually indicates the absence of a value; it means that the actual value is unknown. Procedures automatically exclude from analyses those observations or cases that have missing values. Details of missing value exclusion depend on the procedure and can often be controlled by the user; refer to descriptions of individual procedures for details.

The system-missing value exists only for numeric variables. String variables always have a defined value, even if it is only a string of spaces.

Variables, whether numeric or string, can have designated user-missing values. Every user-missing value is an actual value for that variable. However, most of the time user-missing values are treated in the same way as the system-missing value.

For more information on missing values, see the following sections: Datasets, MISSING VALUES, Expressions. See also the documentation on individual procedures for information on how they handle missing values.

6.7 Datasets

PSPP works with data organized into datasets. A dataset consists of a set of variables, which taken together are said to form a dictionary, and one or more cases, each of which has one value for each variable.

At any given time PSPP has exactly one distinguished dataset, called the active dataset. Most PSPP commands work only with the active dataset. In addition to the active dataset, PSPP also supports any number of additional open datasets. The DATASET commands can choose a new active dataset from among those that are open, as well as create and destroy datasets (see DATASET).

The sections below describe variables in more detail.

6.7.1 Attributes of Variables

Each variable has a number of attributes, including:

Name

An identifier, up to 64 bytes long. Each variable must have a different name. See Tokens.

Some system variable names begin with ‘$’, but user-defined variables’ names may not begin with ‘$’.

The final character in a variable name should not be ‘.’, because such an identifier will be misinterpreted when it is the final token on a line: FOO. will be divided into two separate tokens, ‘FOO’ and ‘.’, indicating end-of-command. See Tokens.

The final character in a variable name should not be ‘_’, because some such identifiers are used for special purposes by PSPP procedures.

As with all PSPP identifiers, variable names are not case-sensitive. PSPP capitalizes variable names on output the same way they were capitalized at their point of definition in the input.

Type

Numeric or string.

Width

(string variables only) String variables with a width of 8 characters or fewer are called short string variables. Short string variables may be used in a few contexts where long string variables (those with widths greater than 8) are not allowed.

Position

Variables in the dictionary are arranged in a specific order. DISPLAY can be used to show this order: see DISPLAY.

Initialization

Either reinitialized to 0 or spaces for each case, or left at its existing value. See LEAVE.

Missing values

Optionally, up to three values, or a range of values, or a specific value plus a range, can be specified as user-missing values. There is also a system-missing value that is assigned to an observation when there is no other obvious value for that observation. Observations with missing values are automatically excluded from analyses. User-missing values are actual data values, while the system-missing value is not a value at all. See Missing Observations.

Variable label

A string that describes the variable. See VARIABLE LABELS.

Value label

Optionally, these associate each possible value of the variable with a string. See VALUE LABELS.

Print format

Display width, format, and (for numeric variables) number of decimal places. This attribute does not affect how data are stored, just how they are displayed. Example: a width of 8, with 2 decimal places. See Input and Output Formats.

Write format

Similar to print format, but used by the WRITE command (see WRITE).

Custom attributes

User-defined associations between names and values. See VARIABLE ATTRIBUTE.

Role

The intended role of a variable for use in dialog boxes in graphical user interfaces. See VARIABLE ROLE.

6.7.2 Variables Automatically Defined by PSPP

There are seven system variables. These are not like ordinary variables because system variables are not always stored. They can be used only in expressions. These system variables, whose values and output formats cannot be modified, are described below.

$CASENUM

Case number of the case at the moment. This changes as cases are shuffled around.

$DATE

Date the PSPP process was started, in format A9, following the pattern DD MMM YY.

$JDATE

Number of days between 15 Oct 1582 and the time the PSPP process was started.

$LENGTH

Page length, in lines, in format F11.

$SYSMIS

System missing value, in format F1.

$TIME

Number of seconds between midnight 14 Oct 1582 and the time the active dataset was read, in format F20.

$WIDTH

Page width, in characters, in format F3.

6.7.3 Lists of variable names

To refer to a set of variables, list their names one after another. Optionally, their names may be separated by commas. To include a range of variables from the dictionary in the list, write the name of the first and last variable in the range, separated by TO. For instance, if the dictionary contains six variables with the names ID, X1, X2, GOAL, MET, and NEXTGOAL, in that order, then X2 TO MET would include variables X2, GOAL, and MET.

Commands that define variables, such as DATA LIST, give TO an alternate meaning. With these commands, TO define sequences of variables whose names end in consecutive integers. The syntax is two identifiers that begin with the same root and end with numbers, separated by TO. The syntax X1 TO X5 defines 5 variables, named X1, X2, X3, X4, and X5. The syntax ITEM0008 TO ITEM0013 defines 6 variables, named ITEM0008, ITEM0009, ITEM0010, ITEM0011, ITEM0012, and ITEM00013. The syntaxes QUES001 TO QUES9 and QUES6 TO QUES3 are invalid.

After a set of variables has been defined with DATA LIST or another command with this method, the same set can be referenced on later commands using the same syntax.

6.7.4 Input and Output Formats

An input format describes how to interpret the contents of an input field as a number or a string. It might specify that the field contains an ordinary decimal number, a time or date, a number in binary or hexadecimal notation, or one of several other notations. Input formats are used by commands such as DATA LIST that read data or syntax files into the PSPP active dataset.

Every input format corresponds to a default output format that specifies the formatting used when the value is output later. It is always possible to explicitly specify an output format that resembles the input format. Usually, this is the default, but in cases where the input format is unfriendly to human readability, such as binary or hexadecimal formats, the default output format is an easier-to-read decimal format.

Every variable has two output formats, called its print format and write format. Print formats are used in most output contexts; write formats are used only by WRITE (see WRITE). Newly created variables have identical print and write formats, and FORMATS, the most commonly used command for changing formats (see FORMATS), sets both of them to the same value as well. Thus, most of the time, the distinction between print and write formats is unimportant.

Input and output formats are specified to PSPP with a format specification of the form TYPEw or TYPEw.d, where TYPE is one of the format types described later, w is a field width measured in columns, and d is an optional number of decimal places. If d is omitted, a value of 0 is assumed. Some formats do not allow a nonzero d to be specified.

The following sections describe the input and output formats supported by PSPP.

6.7.4.1 Basic Numeric Formats

The basic numeric formats are used for input and output of real numbers in standard or scientific notation. The following table shows an example of how each format displays positive and negative numbers with the default decimal point setting:

On output, numbers in F format are expressed in standard decimal notation with the requested number of decimal places. The other formats output some variation on this style:

• Numbers in COMMA format are additionally grouped every three digits by inserting a grouping character. The grouping character is ordinarily a comma, but it can be changed to a period (see SET DECIMAL).
• DOT format is like COMMA format, but it interchanges the role of the decimal point and grouping characters. That is, the current grouping character is used as a decimal point and vice versa.
• DOLLAR format is like COMMA format, but it prefixes the number with ‘$’.
• PCT format is like F format, but adds ‘%’ after the number.
• The E format always produces output in scientific notation.

On input, the basic numeric formats accept positive and numbers in standard decimal notation or scientific notation. Leading and trailing spaces are allowed. An empty or all-spaces field, or one that contains only a single period, is treated as the system missing value.

In scientific notation, the exponent may be introduced by a sign (‘+’ or ‘-’), or by one of the letters ‘e’ or ‘d’ (in uppercase or lowercase), or by a letter followed by a sign. A single space may follow the letter or the sign or both.

On fixed-format DATA LIST (see DATA LIST FIXED) and in a few other contexts, decimals are implied when the field does not contain a decimal point. In F6.5 format, for example, the field 314159 is taken as the value 3.14159 with implied decimals. Decimals are never implied if an explicit decimal point is present or if scientific notation is used.

E and F formats accept the basic syntax already described. The other formats allow some additional variations:

• COMMA, DOLLAR, and DOT formats ignore grouping characters within the integer part of the input field. The identity of the grouping character depends on the format.
• DOLLAR format allows a dollar sign to precede the number. In a negative number, the dollar sign may precede or follow the minus sign.
• PCT format allows a percent sign to follow the number.

All of the basic number formats have a maximum field width of 40 and accept no more than 16 decimal places, on both input and output. Some additional restrictions apply:

• As input formats, the basic numeric formats allow no more decimal places than the field width. As output formats, the field width must be greater than the number of decimal places; that is, large enough to allow for a decimal point and the number of requested decimal places. DOLLAR and PCT formats must allow an additional column for ‘$’ or ‘%’.
• The default output format for a given input format increases the field width enough to make room for optional input characters. If an input format calls for decimal places, the width is increased by 1 to make room for an implied decimal point. COMMA, DOT, and DOLLAR formats also increase the output width to make room for grouping characters. DOLLAR and PCT further increase the output field width by 1 to make room for ‘$’ or ‘%’. The increased output width is capped at 40, the maximum field width.
• The E format is exceptional. For output, E format has a minimum width of 7 plus the number of decimal places. The default output format for an E input format is an E format with at least 3 decimal places and thus a minimum width of 10.

More details of basic numeric output formatting are given below:

• Output rounds to nearest, with ties rounded away from zero. Thus, 2.5 is output as 3 in F1.0 format, and -1.125 as -1.13 in F5.1 format.
• The system-missing value is output as a period in a field of spaces, placed in the decimal point’s position, or in the rightmost column if no decimal places are requested. A period is used even if the decimal point character is a comma.
• A number that does not fill its field is right-justified within the field.
• A number is too large for its field causes decimal places to be dropped to make room. If dropping decimals does not make enough room, scientific notation is used if the field is wide enough. If a number does not fit in the field, even in scientific notation, the overflow is indicated by filling the field with asterisks (‘*’).
• COMMA, DOT, and DOLLAR formats insert grouping characters only if space is available for all of them. Grouping characters are never inserted when all decimal places must be dropped. Thus, 1234.56 in COMMA5.2 format is output as ‘ 1235’ without a comma, even though there is room for one, because all decimal places were dropped.
• DOLLAR or PCT format drop the ‘$’ or ‘%’ only if the number would not fit at all without it. Scientific notation with ‘$’ or ‘%’ is preferred to ordinary decimal notation without it.
• Except in scientific notation, a decimal point is included only when it is followed by a digit. If the integer part of the number being output is 0, and a decimal point is included, then the zero before the decimal point is dropped.

  In scientific notation, the number always includes a decimal point, even if it is not followed by a digit.

• A negative number includes a minus sign only in the presence of a nonzero digit: -0.01 is output as ‘-.01’ in F4.2 format but as ‘  .0’ in F4.1 format. Thus, a “negative zero” never includes a minus sign.
• In negative numbers output in DOLLAR format, the dollar sign follows the negative sign. Thus, -9.99 in DOLLAR6.2 format is output as -$9.99.
• In scientific notation, the exponent is output as ‘E’ followed by ‘+’ or ‘-’ and exactly three digits. Numbers with magnitude less than 10**-999 or larger than 10**999 are not supported by most computers, but if they are supported then their output is considered to overflow the field and will be output as asterisks.
• On most computers, no more than 15 decimal digits are significant in output, even if more are printed. In any case, output precision cannot be any higher than input precision; few data sets are accurate to 15 digits of precision. Unavoidable loss of precision in intermediate calculations may also reduce precision of output.
• Special values such as infinities and “not a number” values are usually converted to the system-missing value before printing. In a few circumstances, these values are output directly. In fields of width 3 or greater, special values are output as however many characters will fit from +Infinity or -Infinity for infinities, from NaN for “not a number,” or from Unknown for other values (if any are supported by the system). In fields under 3 columns wide, special values are output as asterisks.

6.7.4.2 Custom Currency Formats

The custom currency formats are closely related to the basic numeric formats, but they allow users to customize the output format. The SET command configures custom currency formats, using the syntax

SET CCx="string".

where x is A, B, C, D, or E, and string is no more than 16 characters long.

string must contain exactly three commas or exactly three periods (but not both), except that a single quote character may be used to “escape” a following comma, period, or single quote. If three commas are used, commas will be used for grouping in output, and a period will be used as the decimal point. Uses of periods reverses these roles.

The commas or periods divide string into four fields, called the negative prefix, prefix, suffix, and negative suffix, respectively. The prefix and suffix are added to output whenever space is available. The negative prefix and negative suffix are always added to a negative number when the output includes a nonzero digit.

The following syntax shows how custom currency formats could be used to reproduce basic numeric formats:

SET CCA="-,,,".  /* Same as COMMA.
SET CCB="-...".  /* Same as DOT.
SET CCC="-,$,,". /* Same as DOLLAR.
SET CCD="-,,%,". /* Like PCT, but groups with commas.

Here are some more examples of custom currency formats. The final example shows how to use a single quote to escape a delimiter:

SET CCA=",EUR,,-".   /* Euro.
SET CCB="(,USD ,,)". /* US dollar.
SET CCC="-.R$..".    /* Brazilian real.
SET CCD="-,, NIS,".  /* Israel shekel.
SET CCE="-.Rp'. ..". /* Indonesia Rupiah.

These formats would yield the following output:

The default for all the custom currency formats is ‘-,,,’, equivalent to COMMA format.

6.7.4.3 Legacy Numeric Formats

The N and Z numeric formats provide compatibility with legacy file formats. They have much in common:

• Output is rounded to the nearest representable value, with ties rounded away from zero.
• Numbers too large to display are output as a field filled with asterisks (‘*’).
• The decimal point is always implicitly the specified number of digits from the right edge of the field, except that Z format input allows an explicit decimal point.
• Scientific notation may not be used.
• The system-missing value is output as a period in a field of spaces. The period is placed just to the right of the implied decimal point in Z format, or at the right end in N format or in Z format if no decimal places are requested. A period is used even if the decimal point character is a comma.
• Field width may range from 1 to 40. Decimal places may range from 0 up to the field width, to a maximum of 16.
• When a legacy numeric format used for input is converted to an output format, it is changed into the equivalent F format. The field width is increased by 1 if any decimal places are specified, to make room for a decimal point. For Z format, the field width is increased by 1 more column, to make room for a negative sign. The output field width is capped at 40 columns.

N Format

The N format supports input and output of fields that contain only digits. On input, leading or trailing spaces, a decimal point, or any other non-digit character causes the field to be read as the system-missing value. As a special exception, an N format used on DATA LIST FREE or DATA LIST LIST is treated as the equivalent F format.

On output, N pads the field on the left with zeros. Negative numbers are output like the system-missing value.

Z Format

The Z format is a “zoned decimal” format used on IBM mainframes. Z format encodes the sign as part of the final digit, which must be one of the following:

0123456789
{ABCDEFGHI
}JKLMNOPQR

where the characters in each row represent digits 0 through 9 in order. Characters in the first two rows indicate a positive sign; those in the third indicate a negative sign.

On output, Z fields are padded on the left with spaces. On input, leading and trailing spaces are ignored. Any character in an input field other than spaces, the digit characters above, and ‘.’ causes the field to be read as system-missing.

The decimal point character for input and output is always ‘.’, even if the decimal point character is a comma (see SET DECIMAL).

Nonzero, negative values output in Z format are marked as negative even when no nonzero digits are output. For example, -0.2 is output in Z1.0 format as ‘J’. The “negative zero” value supported by most machines is output as positive.

6.7.4.4 Binary and Hexadecimal Numeric Formats

The binary and hexadecimal formats are primarily designed for compatibility with existing machine formats, not for human readability. All of them therefore have a F format as default output format. Some of these formats are only portable between machines with compatible byte ordering (endianness) or floating-point format.

Binary formats use byte values that in text files are interpreted as special control functions, such as carriage return and line feed. Thus, data in binary formats should not be included in syntax files or read from data files with variable-length records, such as ordinary text files. They may be read from or written to data files with fixed-length records. See FILE HANDLE, for information on working with fixed-length records.

P and PK Formats

These are binary-coded decimal formats, in which every byte (except the last, in P format) represents two decimal digits. The most-significant 4 bits of the first byte is the most-significant decimal digit, the least-significant 4 bits of the first byte is the next decimal digit, and so on.

In P format, the most-significant 4 bits of the last byte are the least-significant decimal digit. The least-significant 4 bits represent the sign: decimal 15 indicates a negative value, decimal 13 indicates a positive value.

Numbers are rounded downward on output. The system-missing value and numbers outside representable range are output as zero.

The maximum field width is 16. Decimal places may range from 0 up to the number of decimal digits represented by the field.

The default output format is an F format with twice the input field width, plus one column for a decimal point (if decimal places were requested).

IB and PIB Formats

These are integer binary formats. IB reads and writes 2’s complement binary integers, and PIB reads and writes unsigned binary integers. The byte ordering is by default the host machine’s, but SET RIB may be used to select a specific byte ordering for reading (see SET RIB) and SET WIB, similarly, for writing (see SET WIB).

The maximum field width is 8. Decimal places may range from 0 up to the number of decimal digits in the largest value representable in the field width.

The default output format is an F format whose width is the number of decimal digits in the largest value representable in the field width, plus 1 if the format has decimal places.

RB Format

This is a binary format for real numbers. By default it reads and writes the host machine’s floating-point format, but SET RRB may be used to select an alternate floating-point format for reading (see SET RRB) and SET WRB, similarly, for writing (see SET WRB).

The recommended field width depends on the floating-point format. NATIVE (the default format), IDL, IDB, VD, VG, and ZL formats should use a field width of 8. ISL, ISB, VF, and ZS formats should use a field width of 4. Other field widths will not produce useful results. The maximum field width is 8. No decimal places may be specified.

The default output format is F8.2.

PIBHEX and RBHEX Formats

These are hexadecimal formats, for reading and writing binary formats where each byte has been recoded as a pair of hexadecimal digits.

A hexadecimal field consists solely of hexadecimal digits ‘0’…‘9’ and ‘A’…‘F’. Uppercase and lowercase are accepted on input; output is in uppercase.

Other than the hexadecimal representation, these formats are equivalent to PIB and RB formats, respectively. However, bytes in PIBHEX format are always ordered with the most-significant byte first (big-endian order), regardless of the host machine’s native byte order or PSPP settings.

Field widths must be even and between 2 and 16. RBHEX format allows no decimal places; PIBHEX allows as many decimal places as a PIB format with half the given width.

6.7.4.5 Time and Date Formats

In PSPP, a time is an interval. The time formats translate between human-friendly descriptions of time intervals and PSPP’s internal representation of time intervals, which is simply the number of seconds in the interval. PSPP has two time formats:

A date is a moment in the past or the future. Internally, PSPP represents a date as the number of seconds since the epoch, midnight, Oct. 14, 1582. The date formats translate between human-readable dates and PSPP’s numeric representation of dates and times. PSPP has several date formats:

The templates in the preceding tables describe how the time and date formats are input and output:

dd

Day of month, from 1 to 31. Always output as two digits.

mm

mmm

Month. In output, mm is output as two digits, mmm as the first three letters of an English month name (January, February, …). In input, both of these formats, plus Roman numerals, are accepted.

yyyy

Year. In output, DATETIME always produces a 4-digit year; other formats can produce a 2- or 4-digit year. The century assumed for 2-digit years depends on the EPOCH setting (see SET EPOCH). In output, a year outside the epoch causes the whole field to be filled with asterisks (‘*’).

jjj

Day of year (Julian day), from 1 to 366. This is exactly three digits giving the count of days from the start of the year. January 1 is considered day 1.

q

Quarter of year, from 1 to 4. Quarters start on January 1, April 1, July 1, and October 1.

ww

Week of year, from 1 to 53. Output as exactly two digits. January 1 is the first day of week 1.

DD

Count of days, which may be positive or negative. Output as at least two digits.

hh

Count of hours, which may be positive or negative. Output as at least two digits.

HH

Hour of day, from 0 to 23. Output as exactly two digits.

MM

Minute of hour, from 0 to 59. Output as exactly two digits.

SS.ss

Seconds within minute, from 0 to 59. The integer part is output as exactly two digits. On output, seconds and fractional seconds may or may not be included, depending on field width and decimal places. On input, seconds and fractional seconds are optional. The DECIMAL setting controls the character accepted and displayed as the decimal point (see SET DECIMAL).

For output, the date and time formats use the delimiters indicated in the table. For input, date components may be separated by spaces or by one of the characters ‘-’, ‘/’, ‘.’, or ‘,’, and time components may be separated by spaces, ‘:’, or ‘.’. On input, the ‘Q’ separating quarter from year and the ‘WK’ separating week from year may be uppercase or lowercase, and the spaces around them are optional.

On input, all time and date formats accept any amount of leading and trailing white space.

The maximum width for time and date formats is 40 columns. Minimum input and output width for each of the time and date formats is shown below:

In the table, “Option” describes what increased output width enables:

4-digit year

A field 2 columns wider than minimum will include a 4-digit year. (DATETIME format always includes a 4-digit year.)

seconds

A field 3 columns wider than minimum will include seconds as well as minutes. A field 5 columns wider than minimum, or more, can also include a decimal point and fractional seconds (but no more than allowed by the format’s decimal places).

For the time and date formats, the default output format is the same as the input format, except that PSPP increases the field width, if necessary, to the minimum allowed for output.

Time or dates narrower than the field width are right-justified within the field.

When a time or date exceeds the field width, characters are trimmed from the end until it fits. This can occur in an unusual situation, e.g. with a year greater than 9999 (which adds an extra digit), or for a negative value on TIME or DTIME (which adds a leading minus sign).

The system-missing value is output as a period at the right end of the field.

6.7.4.6 Date Component Formats

The WKDAY and MONTH formats provide input and output for the names of weekdays and months, respectively.

On output, these formats convert a number between 1 and 7, for WKDAY, or between 1 and 12, for MONTH, into the English name of a day or month, respectively. If the name is longer than the field, it is trimmed to fit. If the name is shorter than the field, it is padded on the right with spaces. Values outside the valid range, and the system-missing value, are output as all spaces.

On input, English weekday or month names (in uppercase or lowercase) are converted back to their corresponding numbers. Weekday and month names may be abbreviated to their first 2 or 3 letters, respectively.

The field width may range from 2 to 40, for WKDAY, or from 3 to 40, for MONTH. No decimal places are allowed.

The default output format is the same as the input format.

6.7.4.7 String Formats

The A and AHEX formats are the only ones that may be assigned to string variables. Neither format allows any decimal places.

In A format, the entire field is treated as a string value. The field width may range from 1 to 32,767, the maximum string width. The default output format is the same as the input format.

In AHEX format, the field is composed of characters in a string encoded as hex digit pairs. On output, hex digits are output in uppercase; on input, uppercase and lowercase are both accepted. The default output format is A format with half the input width.

6.7.5 Scratch Variables

Most of the time, variables don’t retain their values between cases. Instead, either they’re being read from a data file or the active dataset, in which case they assume the value read, or, if created with COMPUTE or another transformation, they’re initialized to the system-missing value or to blanks, depending on type.

However, sometimes it’s useful to have a variable that keeps its value between cases. You can do this with LEAVE (see LEAVE), or you can use a scratch variable. Scratch variables are variables whose names begin with an octothorpe (‘#’).

Scratch variables have the same properties as variables left with LEAVE: they retain their values between cases, and for the first case they are initialized to 0 or blanks. They have the additional property that they are deleted before the execution of any procedure. For this reason, scratch variables can’t be used for analysis. To use a scratch variable in an analysis, use COMPUTE (see COMPUTE) to copy its value into an ordinary variable, then use that ordinary variable in the analysis.

6.8 Files Used by PSPP

PSPP makes use of many files each time it runs. Some of these it reads, some it writes, some it creates. Here is a table listing the most important of these files:

command file

syntax file

These names (synonyms) refer to the file that contains instructions that tell PSPP what to do. The syntax file’s name is specified on the PSPP command line. Syntax files can also be read with INCLUDE (see INCLUDE).

data file

Data files contain raw data in text or binary format. Data can also be embedded in a syntax file with BEGIN DATA and END DATA.

listing file

One or more output files are created by PSPP each time it is run. The output files receive the tables and charts produced by statistical procedures. The output files may be in any number of formats, depending on how PSPP is configured.

system file

System files are binary files that store a dictionary and a set of cases. GET and SAVE read and write system files.

portable file

Portable files are files in a text-based format that store a dictionary and a set of cases. IMPORT and EXPORT read and write portable files.

6.9 File Handles

A file handle is a reference to a data file, system file, or portable file. Most often, a file handle is specified as the name of a file as a string, that is, enclosed within ‘'’ or ‘"’.

A file name string that begins or ends with ‘|’ is treated as the name of a command to pipe data to or from. You can use this feature to read data over the network using a program such as ‘curl’ (e.g. GET '|curl -s -S http://example.com/mydata.sav'), to read compressed data from a file using a program such as ‘zcat’ (e.g. GET '|zcat mydata.sav.gz'), and for many other purposes.

PSPP also supports declaring named file handles with the FILE HANDLE command. This command associates an identifier of your choice (the file handle’s name) with a file. Later, the file handle name can be substituted for the name of the file. When PSPP syntax accesses a file multiple times, declaring a named file handle simplifies updating the syntax later to use a different file. Use of FILE HANDLE is also required to read data files in binary formats. See FILE HANDLE, for more information.

In some circumstances, PSPP must distinguish whether a file handle refers to a system file or a portable file. When this is necessary to read a file, e.g. as an input file for GET or MATCH FILES, PSPP uses the file’s contents to decide. In the context of writing a file, e.g. as an output file for SAVE or AGGREGATE, PSPP decides based on the file’s name: if it ends in ‘.por’ (with any capitalization), then PSPP writes a portable file; otherwise, PSPP writes a system file.

INLINE is reserved as a file handle name. It refers to the “data file” embedded into the syntax file between BEGIN DATA and END DATA. See BEGIN DATA, for more information.

The file to which a file handle refers may be reassigned on a later FILE HANDLE command if it is first closed using CLOSE FILE HANDLE. See CLOSE FILE HANDLE, for more information.

6.10 Backus-Naur Form

The syntax of some parts of the PSPP language is presented in this manual using the formalism known as Backus-Naur Form, or BNF. The following table describes BNF:

• Words in all-uppercase are PSPP keyword tokens. In BNF, these are often called terminals. There are some special terminals, which are written in lowercase for clarity:

  number

  A real number.

  integer

  An integer number.

  string

  A string.

  var-name

  A single variable name.

  =, /, +, -, etc.

  Operators and punctuators.

  .

  The end of the command. This is not necessarily an actual dot in the syntax file: See Commands, for more details.

• Other words in all lowercase refer to BNF definitions, called productions. These productions are also known as nonterminals. Some nonterminals are very common, so they are defined here in English for clarity:

  var-list

  A list of one or more variable names or the keyword ALL.

  expression

  An expression. See Expressions, for details.

• ‘::=’ means “is defined as”. The left side of ‘::=’ gives the name of the nonterminal being defined. The right side of ‘::=’ gives the definition of that nonterminal. If the right side is empty, then one possible expansion of that nonterminal is nothing. A BNF definition is called a production.
• So, the key difference between a terminal and a nonterminal is that a terminal cannot be broken into smaller parts—in fact, every terminal is a single token (see Tokens). On the other hand, nonterminals are composed of a (possibly empty) sequence of terminals and nonterminals. Thus, terminals indicate the deepest level of syntax description. (In parsing theory, terminals are the leaves of the parse tree; nonterminals form the branches.)
• The first nonterminal defined in a set of productions is called the start symbol. The start symbol defines the entire syntax for that command.

7 Mathematical Expressions

Expressions share a common syntax each place they appear in PSPP commands. Expressions are made up of operands, which can be numbers, strings, or variable names, separated by operators. There are five types of operators: grouping, arithmetic, logical, relational, and functions.

Every operator takes one or more operands as input and yields exactly one result as output. Depending on the operator, operands accept strings or numbers as operands. With few exceptions, operands may be full-fledged expressions in themselves.

7.1 Boolean Values

Some PSPP operators and expressions work with Boolean values, which represent true/false conditions. Booleans have only three possible values: 0 (false), 1 (true), and system-missing (unknown). System-missing is neither true nor false and indicates that the true value is unknown.

Boolean-typed operands or function arguments must take on one of these three values. Other values are considered false, but provoke a warning when the expression is evaluated.

Strings and Booleans are not compatible, and neither may be used in place of the other.

7.2 Missing Values in Expressions

Most numeric operators yield system-missing when given any system-missing operand. A string operator given any system-missing operand typically results in the empty string. Exceptions are listed under particular operator descriptions.

String user-missing values are not treated specially in expressions.

User-missing values for numeric variables are always transformed into the system-missing value, except inside the arguments to the VALUE and SYSMIS functions.

The missing-value functions can be used to precisely control how missing values are treated in expressions. See Missing Value Functions, for more details.

7.3 Grouping Operators

Parentheses (‘()’) are the grouping operators. Surround an expression with parentheses to force early evaluation.

Parentheses also surround the arguments to functions, but in that situation they act as punctuators, not as operators.

7.4 Arithmetic Operators

The arithmetic operators take numeric operands and produce numeric results.

a + b

Yields the sum of a and b.

a - b

Subtracts b from a and yields the difference.

a * b

Yields the product of a and b. If either a or b is 0, then the result is 0, even if the other operand is missing.

a / b

Divides a by b and yields the quotient. If a is 0, then the result is 0, even if b is missing. If b is zero, the result is system-missing.

a ** b

Yields the result of raising a to the power b. If a is negative and b is not an integer, the result is system-missing. The result of 0**0 is system-missing as well.

- a

Reverses the sign of a.

7.5 Logical Operators

The logical operators take logical operands and produce logical results, meaning “true or false.” Logical operators are not true Boolean operators because they may also result in a system-missing value. See Boolean Values, for more information.

a AND b

a & b

True if both a and b are true, false otherwise. If one operand is false, the result is false even if the other is missing. If both operands are missing, the result is missing.

a OR b

a | b

True if at least one of a and b is true. If one operand is true, the result is true even if the other operand is missing. If both operands are missing, the result is missing.

NOT a

~ a

True if a is false. If the operand is missing, then the result is missing.

7.6 Relational Operators

The relational operators take numeric or string operands and produce Boolean results.

Strings cannot be compared to numbers. When strings of different lengths are compared, the shorter string is right-padded with spaces to match the length of the longer string.

The results of string comparisons, other than tests for equality or inequality, depend on the character set in use. String comparisons are case-sensitive.

a EQ b

a = b

True if a is equal to b.

a LE b

a <= b

True if a is less than or equal to b.

a LT b

a < b

True if a is less than b.

a GE b

a >= b

True if a is greater than or equal to b.

a GT b

a > b

True if a is greater than b.

a NE b

a ~= b

a <> b

True if a is not equal to b.

7.7 Functions

PSPP functions provide mathematical abilities above and beyond those possible using simple operators. Functions have a common syntax: each is composed of a function name followed by a left parenthesis, one or more arguments, and a right parenthesis.

Function names are not reserved. Their names are specially treated only when followed by a left parenthesis, so that ‘EXP(10)’ refers to the constant value e raised to the 10th power, but ‘EXP’ by itself refers to the value of a variable called EXP.

The sections below describe each function in detail.

7.7.1 Mathematical Functions

Advanced mathematical functions take numeric arguments and produce numeric results.

Function: EXP (exponent)

Returns e (approximately 2.71828) raised to power exponent.

Function: LG10 (number)

Takes the base-10 logarithm of number. If number is not positive, the result is system-missing.

Function: LN (number)

Takes the base-e logarithm of number. If number is not positive, the result is system-missing.

Function: LNGAMMA (number)

Yields the base-e logarithm of the complete gamma of number. If number is a negative integer, the result is system-missing.

Function: SQRT (number)

Takes the square root of number. If number is negative, the result is system-missing.

7.7.2 Miscellaneous Mathematical Functions

Miscellaneous mathematical functions take numeric arguments and produce numeric results.

Function: ABS (number)

Results in the absolute value of number.

Function: MOD (numerator, denominator)

Returns the remainder (modulus) of numerator divided by denominator. If numerator is 0, then the result is 0, even if denominator is missing. If denominator is 0, the result is system-missing.

Function: MOD10 (number)

Returns the remainder when number is divided by 10. If number is negative, MOD10(number) is negative or zero.

Function: RND (number [, mult[, fuzzbits]])

Rounds number and rounds it to a multiple of mult (by default 1). Halves are rounded away from zero, as are values that fall short of halves by less than fuzzbits of errors in the least-significant bits of number. If fuzzbits is not specified then the default is taken from SET FUZZBITS (see SET FUZZBITS), which is 6 unless overridden.

Function: TRUNC (number)

Discards the fractional part of number; that is, rounds number towards zero.

7.7.3 Trigonometric Functions

Trigonometric functions take numeric arguments and produce numeric results.

Function: ARCOS (number)

Function: ACOS (number)

Takes the arccosine, in radians, of number. Results in system-missing if number is not between -1 and 1 inclusive. This function is a PSPP extension.

Function: ARSIN (number)

Function: ASIN (number)

Takes the arcsine, in radians, of number. Results in system-missing if number is not between -1 and 1 inclusive.

Function: ARTAN (number)

Function: ATAN (number)

Takes the arctangent, in radians, of number.

Function: COS (angle)

Takes the cosine of angle which should be in radians.

Function: SIN (angle)

Takes the sine of angle which should be in radians.

Function: TAN (angle)

Takes the tangent of angle which should be in radians. Results in system-missing at values of angle that are too close to odd multiples of \pi/2. Portability: none.

7.7.4 Missing-Value Functions

Missing-value functions take various numeric arguments and yield various types of results. Except where otherwise stated below, the normal rules of evaluation apply within expression arguments to these functions. In particular, user-missing values for numeric variables are converted to system-missing values.

Function: MISSING (expr)

Returns 1 if expr has the system-missing value, 0 otherwise.

Function: NMISS (expr [, expr]…)

Each argument must be a numeric expression. Returns the number of system-missing values in the list, which may include variable ranges using the var1 TO var2 syntax.

Function: NVALID (expr [, expr]…)

Each argument must be a numeric expression. Returns the number of values in the list that are not system-missing. The list may include variable ranges using the var1 TO var2 syntax.

Function: SYSMIS (expr)

When expr is simply the name of a numeric variable, returns 1 if the variable has the system-missing value, 0 if it is user-missing or not missing. If given expr takes another form, results in 1 if the value is system-missing, 0 otherwise.

Function: VALUE (variable)

Prevents the user-missing values of variable from being transformed into system-missing values, and always results in the actual value of variable, whether it is valid, user-missing, or system-missing.

7.7.5 Set-Membership Functions

Set membership functions determine whether a value is a member of a set. They take a set of numeric arguments or a set of string arguments, and produce Boolean results.

String comparisons are performed according to the rules given in Relational Operators.

Function: ANY (value, set [, set]…)

Results in true if value is equal to any of the set values. Otherwise, results in false. If value is system-missing, returns system-missing. System-missing values in set do not cause /NAME/ to return system-missing.

Function: RANGE (value, low, high [, low, high]…)

Results in true if value is in any of the intervals bounded by low and high inclusive. Otherwise, results in false. Each low must be less than or equal to its corresponding high value. low and high must be given in pairs. If value is system-missing, returns system-missing. System-missing values in set do not cause /NAME/ to return system-missing.

7.7.6 Statistical Functions

Statistical functions compute descriptive statistics on a list of values. Some statistics can be computed on numeric or string values; other can only be computed on numeric values. Their results have the same type as their arguments. The current case’s weighting factor (see WEIGHT) has no effect on statistical functions.

These functions’ argument lists may include entire ranges of variables using the var1 TO var2 syntax.

Unlike most functions, statistical functions can return non-missing values even when some of their arguments are missing. Most statistical functions, by default, require only 1 non-missing value to have a non-missing return, but /NAME/, /NAME/, and /NAME/ require 2. These defaults can be increased (but not decreased) by appending a dot and the minimum number of valid arguments to the function name. For example, MEAN.3(X, Y, Z) would only return non-missing if all of ‘X’, ‘Y’, and ‘Z’ were valid.

Function: CFVAR (number, number[, …])

Results in the coefficient of variation of the values of number. (The coefficient of variation is the standard deviation divided by the mean.)

Function: MAX (value, value[, …])

Results in the value of the greatest value. The values may be numeric or string.

Function: MEAN (number, number[, …])

Results in the mean of the values of number.

Function: MIN (number, number[, …])

Results in the value of the least value. The values may be numeric or string.

Function: SD (number, number[, …])

Results in the standard deviation of the values of number.

Function: SUM (number, number[, …])

Results in the sum of the values of number.

Function: VARIANCE (number, number[, …])

Results in the variance of the values of number.

7.7.7 String Functions

String functions take various arguments and return various results.

Function: CONCAT (string, string[, …])

Returns a string consisting of each string in sequence. CONCAT("abc", "def", "ghi") has a value of "abcdefghi". The resultant string is truncated to a maximum of 255 characters.

Function: INDEX (haystack, needle)

Returns a positive integer indicating the position of the first occurrence of needle in haystack. Returns 0 if haystack does not contain needle. Returns system-missing if needle is an empty string.

Function: INDEX (haystack, needles, needle_len)

Divides needles into one or more needles, each with length needle_len. Searches haystack for the first occurrence of each needle, and returns the smallest value. Returns 0 if haystack does not contain any part in needle. It is an error if needle_len does not evenly divide the length of needles. Returns system-missing if needles is an empty string.

Function: LENGTH (string)

Returns the number of characters in string.

Function: LOWER (string)

Returns a string identical to string except that all uppercase letters are changed to lowercase letters. The definitions of “uppercase” and “lowercase” are system-dependent.

Function: LPAD (string, length)

If string is at least length characters in length, returns string unchanged. Otherwise, returns string padded with spaces on the left side to length length. Returns an empty string if length is system-missing, negative, or greater than 255.

Function: LPAD (string, length, padding)

If string is at least length characters in length, returns string unchanged. Otherwise, returns string padded with padding on the left side to length length. Returns an empty string if length is system-missing, negative, or greater than 255, or if padding does not contain exactly one character.

Function: LTRIM (string)

Returns string, after removing leading spaces. Other white space, such as tabs, carriage returns, line feeds, and vertical tabs, is not removed.

Function: LTRIM (string, padding)

Returns string, after removing leading padding characters. If padding does not contain exactly one character, returns an empty string.

Function: NUMBER (string, format)

Returns the number produced when string is interpreted according to format specifier format. If the format width w is less than the length of string, then only the first w characters in string are used, e.g. NUMBER("123", F3.0) and NUMBER("1234", F3.0) both have value 123. If w is greater than string’s length, then it is treated as if it were right-padded with spaces. If string is not in the correct format for format, system-missing is returned.

Function: RINDEX (string, format)

Returns a positive integer indicating the position of the last occurrence of needle in haystack. Returns 0 if haystack does not contain needle. Returns system-missing if needle is an empty string.

Function: RINDEX (haystack, needle, needle_len)

Divides needle into parts, each with length needle_len. Searches haystack for the last occurrence of each part, and returns the largest value. Returns 0 if haystack does not contain any part in needle. It is an error if needle_len does not evenly divide the length of needle. Returns system-missing if needle is an empty string.

Function: RPAD (string, length)

If string is at least length characters in length, returns string unchanged. Otherwise, returns string padded with spaces on the right to length length. Returns an empty string if length is system-missing, negative, or greater than 255.

Function: RPAD (string, length, padding)

If string is at least length characters in length, returns string unchanged. Otherwise, returns string padded with padding on the right to length length. Returns an empty string if length is system-missing, negative, or greater than 255, or if padding does not contain exactly one character.

Function: RTRIM (string)

Returns string, after removing trailing spaces. Other types of white space are not removed.

Function: RTRIM (string, padding)

Returns string, after removing trailing padding characters. If padding does not contain exactly one character, returns an empty string.

Function: STRING (number, format)

Returns a string corresponding to number in the format given by format specifier format. For example, STRING(123.56, F5.1) has the value "123.6".

Function: SUBSTR (string, start)

Returns a string consisting of the value of string from position start onward. Returns an empty string if start is system-missing, less than 1, or greater than the length of string.

Function: SUBSTR (string, start, count)

Returns a string consisting of the first count characters from string beginning at position start. Returns an empty string if start or count is system-missing, if start is less than 1 or greater than the number of characters in string, or if count is less than 1. Returns a string shorter than count characters if start + count - 1 is greater than the number of characters in string. Examples: SUBSTR("abcdefg", 3, 2) has value "cd"; SUBSTR("nonsense", 4, 10) has the value "sense".

Function: UPCASE (string)

Returns string, changing lowercase letters to uppercase letters.

7.7.8 Time & Date Functions

For compatibility, PSPP considers dates before 15 Oct 1582 invalid. Most time and date functions will not accept earlier dates.

7.7.8.1 How times & dates are defined and represented

Times and dates are handled by PSPP as single numbers. A time is an interval. PSPP measures times in seconds. Thus, the following intervals correspond with the numeric values given:

          10 minutes                        600
          1 hour                          3,600
          1 day, 3 hours, 10 seconds     97,210
          40 days                     3,456,000

A date, on the other hand, is a particular instant in the past or the future. PSPP represents a date as a number of seconds since midnight preceding 14 Oct 1582. Because midnight preceding the dates given below correspond with the numeric PSPP dates given:

              15 Oct 1582                86,400
               4 Jul 1776         6,113,318,400
               1 Jan 1900        10,010,390,400
               1 Oct 1978        12,495,427,200
              24 Aug 1995        13,028,601,600

7.7.8.2 Functions that Produce Times

These functions take numeric arguments and return numeric values that represent times.

Function: TIME.DAYS (ndays)

Returns a time corresponding to ndays days.

Function: TIME.HMS (nhours, nmins, nsecs)

Returns a time corresponding to nhours hours, nmins minutes, and nsecs seconds. The arguments may not have mixed signs: if any of them are positive, then none may be negative, and vice versa.

7.7.8.3 Functions that Examine Times

These functions take numeric arguments in PSPP time format and give numeric results.

Function: CTIME.DAYS (time)

Results in the number of days and fractional days in time.

Function: CTIME.HOURS (time)

Results in the number of hours and fractional hours in time.

Function: CTIME.MINUTES (time)

Results in the number of minutes and fractional minutes in time.

Function: CTIME.SECONDS (time)

Results in the number of seconds and fractional seconds in time. (CTIME.SECONDS does nothing; CTIME.SECONDS(x) is equivalent to x.)

7.7.8.4 Functions that Produce Dates

These functions take numeric arguments and give numeric results that represent dates. Arguments taken by these functions are:

day

Refers to a day of the month between 1 and 31. Day 0 is also accepted and refers to the final day of the previous month. Days 29, 30, and 31 are accepted even in months that have fewer days and refer to a day near the beginning of the following month.

month

Refers to a month of the year between 1 and 12. Months 0 and 13 are also accepted and refer to the last month of the preceding year and the first month of the following year, respectively.

quarter

Refers to a quarter of the year between 1 and 4. The quarters of the year begin on the first day of months 1, 4, 7, and 10.

week

Refers to a week of the year between 1 and 53.

yday

Refers to a day of the year between 1 and 366.

year

Refers to a year, 1582 or greater. Years between 0 and 99 are treated according to the epoch set on SET EPOCH, by default beginning 69 years before the current date (see SET EPOCH).

If these functions’ arguments are out-of-range, they are correctly normalized before conversion to date format. Non-integers are rounded toward zero.

Function: DATE.DMY (day, month, year)

Function: DATE.MDY (month, day, year)

Results in a date value corresponding to the midnight before day day of month month of year year.

Function: DATE.MOYR (month, year)

Results in a date value corresponding to the midnight before the first day of month month of year year.

Function: DATE.QYR (quarter, year)

Results in a date value corresponding to the midnight before the first day of quarter quarter of year year.

Function: DATE.WKYR (week, year)

Results in a date value corresponding to the midnight before the first day of week week of year year.

Function: DATE.YRDAY (year, yday)

Results in a date value corresponding to the day yday of year year.

7.7.8.5 Functions that Examine Dates

These functions take numeric arguments in PSPP date or time format and give numeric results. These names are used for arguments:

date

A numeric value in PSPP date format.

time

A numeric value in PSPP time format.

time-or-date

A numeric value in PSPP time or date format.

Function: XDATE.DATE (time-or-date)

For a time, results in the time corresponding to the number of whole days date-or-time includes. For a date, results in the date corresponding to the latest midnight at or before date-or-time; that is, gives the date that date-or-time is in.

Function: XDATE.HOUR (time-or-date)

For a time, results in the number of whole hours beyond the number of whole days represented by date-or-time. For a date, results in the hour (as an integer between 0 and 23) corresponding to date-or-time.

Function: XDATE.JDAY (date)

Results in the day of the year (as an integer between 1 and 366) corresponding to date.

Function: XDATE.MDAY (date)

Results in the day of the month (as an integer between 1 and 31) corresponding to date.

Function: XDATE.MINUTE (time-or-date)

Results in the number of minutes (as an integer between 0 and 59) after the last hour in time-or-date.

Function: XDATE.MONTH (date)

Results in the month of the year (as an integer between 1 and 12) corresponding to date.

Function: XDATE.QUARTER (date)

Results in the quarter of the year (as an integer between 1 and 4) corresponding to date.

Function: XDATE.SECOND (time-or-date)

Results in the number of whole seconds after the last whole minute (as an integer between 0 and 59) in time-or-date.

Function: XDATE.TDAY (date)

Results in the number of whole days from 14 Oct 1582 to date.

Function: XDATE.TIME (date)

Results in the time of day at the instant corresponding to date, as a time value. This is the number of seconds since midnight on the day corresponding to date.

Function: XDATE.WEEK (date)

Results in the week of the year (as an integer between 1 and 53) corresponding to date.

Function: XDATE.WKDAY (date)

Results in the day of week (as an integer between 1 and 7) corresponding to date, where 1 represents Sunday.

Function: XDATE.YEAR (date)

Returns the year (as an integer 1582 or greater) corresponding to date.

7.7.8.6 Time and Date Arithmetic

Ordinary arithmetic operations on dates and times often produce sensible results. Adding a time to, or subtracting one from, a date produces a new date that much earlier or later. The difference of two dates yields the time between those dates. Adding two times produces the combined time. Multiplying a time by a scalar produces a time that many times longer. Since times and dates are just numbers, the ordinary addition and subtraction operators are employed for these purposes.

Adding two dates does not produce a useful result.

Dates and times may have very large values. Thus, it is not a good idea to take powers of these values; also, the accuracy of some procedures may be affected. If necessary, convert times or dates in seconds to some other unit, like days or years, before performing analysis.

PSPP supplies a few functions for date arithmetic:

Function: DATEDIFF (date2, date1, unit)

Returns the span of time from date1 to date2 in terms of unit, which must be a quoted string, one of ‘years’, ‘quarters’, ‘months’, ‘weeks’, ‘days’, ‘hours’, ‘minutes’, and ‘seconds’. The result is an integer, truncated toward zero.

One year is considered to span from a given date to the same month, day, and time of day the next year. Thus, from Jan. 1 of one year to Jan. 1 the next year is considered to be a full year, but Feb. 29 of a leap year to the following Feb. 28 is not. Similarly, one month spans from a given day of the month to the same day of the following month. Thus, there is never a full month from Jan. 31 of a given year to any day in the following February.

Function: DATESUM (date, quantity, unit[, method])

Returns date advanced by the given quantity of the specified unit, which must be one of the strings ‘years’, ‘quarters’, ‘months’, ‘weeks’, ‘days’, ‘hours’, ‘minutes’, and ‘seconds’.

When unit is ‘years’, ‘quarters’, or ‘months’, only the integer part of quantity is considered. Adding one of these units can cause the day of the month to exceed the number of days in the month. In this case, the method comes into play: if it is omitted or specified as ‘closest’ (as a quoted string), then the resulting day is the last day of the month; otherwise, if it is specified as ‘rollover’, then the extra days roll over into the following month.

When unit is ‘weeks’, ‘days’, ‘hours’, ‘minutes’, or ‘seconds’, the quantity is not rounded to an integer and method, if specified, is ignored.

7.7.9 Miscellaneous Functions

Function: LAG (variable[, n])

variable must be a numeric or string variable name. LAG yields the value of that variable for the case n before the current one. Results in system-missing (for numeric variables) or blanks (for string variables) for the first n cases.

LAG obtains values from the cases that become the new active dataset after a procedure executes. Thus, LAG will not return values from cases dropped by transformations such as SELECT IF, and transformations like COMPUTE that modify data will change the values returned by LAG. These are both the case whether these transformations precede or follow the use of LAG.

If LAG is used before TEMPORARY, then the values it returns are those in cases just before TEMPORARY. LAG may not be used after TEMPORARY.

If omitted, ncases defaults to 1. Otherwise, ncases must be a small positive constant integer. There is no explicit limit, but use of a large value will increase memory consumption.

Function: YRMODA (year, month, day)

year is a year, either between 0 and 99 or at least 1582. Unlike other PSPP date functions, years between 0 and 99 always correspond to 1900 through 1999. month is a month between 1 and 13. day is a day between 0 and 31. A day of 0 refers to the last day of the previous month, and a month of 13 refers to the first month of the next year. year must be in range. year, month, and day must all be integers.

YRMODA results in the number of days between 15 Oct 1582 and the date specified, plus one. The date passed to YRMODA must be on or after 15 Oct 1582. 15 Oct 1582 has a value of 1.

Function: VALUELABEL ( variable)

Returns a string matching the label associated with the current value of variable. If the current value of variable has no associated label, then this function returns the empty string. variable may be a numeric or string variable.

7.7.10 Statistical Distribution Functions

PSPP can calculate several functions of standard statistical distributions. These functions are named systematically based on the function and the distribution. The table below describes the statistical distribution functions in general:

PDF.dist (x[, param…])

Probability density function for dist. The domain of x depends on dist. For continuous distributions, the result is the density of the probability function at x, and the range is nonnegative real numbers. For discrete distributions, the result is the probability of x.

CDF.dist (x[, param…])

Cumulative distribution function for dist, that is, the probability that a random variate drawn from the distribution is less than x. The domain of x depends dist. The result is a probability.

SIG.dist (x[, param…)

Tail probability function for dist, that is, the probability that a random variate drawn from the distribution is greater than x. The domain of x depends dist. The result is a probability. Only a few distributions include an /NAME/ function.

IDF.dist (p[, param…])

Inverse distribution function for dist, the value of x for which the CDF would yield p. The value of p is a probability. The range depends on dist and is identical to the domain for the corresponding CDF.

RV.dist ([param…])

Random variate function for dist. The range depends on the distribution.

NPDF.dist (x[, param…])

Noncentral probability density function. The result is the density of the given noncentral distribution at x. The domain of x depends on dist. The range is nonnegative real numbers. Only a few distributions include an /NAME/ function.

NCDF.dist (x[, param…])

Noncentral cumulative distribution function for dist, that is, the probability that a random variate drawn from the given noncentral distribution is less than x. The domain of x depends dist. The result is a probability. Only a few distributions include an NCDF function.

The individual distributions are described individually below.

7.7.10.1 Continuous Distributions

The following continuous distributions are available:

Function: PDF.BETA (x)

Function: CDF.BETA (x, a, b)

Function: IDF.BETA (p, a, b)

Function: RV.BETA (a, b)

Function: NPDF.BETA (x, a, b, lambda)

Function: NCDF.BETA (x, a, b, lambda)

Beta distribution with shape parameters a and b. The noncentral distribution takes an additional parameter lambda. Constraints: a > 0, b > 0, lambda >= 0, 0 <= x <= 1, 0 <= p <= 1.

Function: PDF.BVNOR (x0, x1, rho)

Function: CDF.VBNOR (x0, x1, rho)

Bivariate normal distribution of two standard normal variables with correlation coefficient rho. Two variates x0 and x1 must be provided. Constraints: 0 <= rho <= 1, 0 <= p <= 1.

Function: PDF.CAUCHY (x, a, b)

Function: CDF.CAUCHY (x, a, b)

Function: IDF.CAUCHY (p, a, b)

Function: RV.CAUCHY (a, b)

Cauchy distribution with location parameter a and scale parameter b. Constraints: b > 0, 0 < p < 1.

Function: CDF.CHISQ (x, df)

Function: SIG.CHISQ (x, df)

Function: IDF.CHISQ (p, df)

Function: RV.CHISQ (df)

Function: NCDF.CHISQ (x, df, lambda)

Chi-squared distribution with df degrees of freedom. The noncentral distribution takes an additional parameter lambda. Constraints: df > 0, lambda > 0, x >= 0, 0 <= p < 1.

Function: PDF.EXP (x, a)

Function: CDF.EXP (x, a)

Function: IDF.EXP (p, a)

Function: RV.EXP (a)

Exponential distribution with scale parameter a. The inverse of a represents the rate of decay. Constraints: a > 0, x >= 0, 0 <= p < 1.

Function: PDF.XPOWER (x, a, b)

Function: RV.XPOWER (a, b)

Exponential power distribution with positive scale parameter a and nonnegative power parameter b. Constraints: a > 0, b >= 0, x >= 0, 0 <= p <= 1. This distribution is a PSPP extension.

Function: PDF.F (x, df1, df2)

Function: CDF.F (x, df1, df2)

Function: SIG.F (x, df1, df2)

Function: IDF.F (p, df1, df2)

Function: RV.F (df1, df2)

F-distribution of two chi-squared deviates with df1 and df2 degrees of freedom. The noncentral distribution takes an additional parameter lambda. Constraints: df1 > 0, df2 > 0, lambda >= 0, x >= 0, 0 <= p < 1.

Function: PDF.GAMMA (x, a, b)

Function: CDF.GAMMA (x, a, b)

Function: IDF.GAMMA (p, a, b)

Function: RV.GAMMA (a, b)

Gamma distribution with shape parameter a and scale parameter b. Constraints: a > 0, b > 0, x >= 0, 0 <= p < 1.

Function: PDF.LANDAU (x)

Function: RV.LANDAU ()

Landau distribution.

Function: PDF.LAPLACE (x, a, b)

Function: CDF.LAPLACE (x, a, b)

Function: IDF.LAPLACE (p, a, b)

Function: RV.LAPLACE (a, b)

Laplace distribution with location parameter a and scale parameter b. Constraints: b > 0, 0 < p < 1.

Function: RV.LEVY (c, alpha)

Levy symmetric alpha-stable distribution with scale c and exponent alpha. Constraints: 0 < alpha <= 2.

Function: RV.LVSKEW (c, alpha, beta)

Levy skew alpha-stable distribution with scale c, exponent alpha, and skewness parameter beta. Constraints: 0 < alpha <= 2, -1 <= beta <= 1.

Function: PDF.LOGISTIC (x, a, b)

Function: CDF.LOGISTIC (x, a, b)

Function: IDF.LOGISTIC (p, a, b)

Function: RV.LOGISTIC (a, b)

Logistic distribution with location parameter a and scale parameter b. Constraints: b > 0, 0 < p < 1.

Function: PDF.LNORMAL (x, a, b)

Function: CDF.LNORMAL (x, a, b)

Function: IDF.LNORMAL (p, a, b)

Function: RV.LNORMAL (a, b)

Lognormal distribution with parameters a and b. Constraints: a > 0, b > 0, x >= 0, 0 <= p < 1.

Function: PDF.NORMAL (x, mu, sigma)

Function: CDF.NORMAL (x, mu, sigma)

Function: IDF.NORMAL (p, mu, sigma)

Function: RV.NORMAL (mu, sigma)

Normal distribution with mean mu and standard deviation sigma. Constraints: b > 0, 0 < p < 1. Three additional functions are available as shorthand:

Function: CDFNORM (x)

Equivalent to CDF.NORMAL(x, 0, 1).

Function: PROBIT (p)

Equivalent to IDF.NORMAL(p, 0, 1).

Function: NORMAL (sigma)

Equivalent to RV.NORMAL(0, sigma).

Function: PDF.NTAIL (x, a, sigma)

Function: RV.NTAIL (a, sigma)

Normal tail distribution with lower limit a and standard deviation sigma. This distribution is a PSPP extension. Constraints: a > 0, x > a, 0 < p < 1.

Function: PDF.PARETO (x, a, b)

Function: CDF.PARETO (x, a, b)

Function: IDF.PARETO (p, a, b)

Function: RV.PARETO (a, b)

Pareto distribution with threshold parameter a and shape parameter b. Constraints: a > 0, b > 0, x >= a, 0 <= p < 1.

Function: PDF.RAYLEIGH (x, sigma)

Function: CDF.RAYLEIGH (x, sigma)

Function: IDF.RAYLEIGH (p, sigma)

Function: RV.RAYLEIGH (sigma)

Rayleigh distribution with scale parameter sigma. This distribution is a PSPP extension. Constraints: sigma > 0, x > 0.

Function: PDF.RTAIL (x, a, sigma)

Function: RV.RTAIL (a, sigma)

Rayleigh tail distribution with lower limit a and scale parameter sigma. This distribution is a PSPP extension. Constraints: a > 0, sigma > 0, x > a.

Function: PDF.T (x, df)

Function: CDF.T (x, df)

Function: IDF.T (p, df)

Function: RV.T (df)

T-distribution with df degrees of freedom. The noncentral distribution takes an additional parameter lambda. Constraints: df > 0, 0 < p < 1.

Function: PDF.T1G (x, a, b)

Function: CDF.T1G (x, a, b)

Function: IDF.T1G (p, a, b)

Type-1 Gumbel distribution with parameters a and b. This distribution is a PSPP extension. Constraints: 0 < p < 1.

Function: PDF.T2G (x, a, b)

Function: CDF.T2G (x, a, b)

Function: IDF.T2G (p, a, b)

Type-2 Gumbel distribution with parameters a and b. This distribution is a PSPP extension. Constraints: x > 0, 0 < p < 1.

Function: PDF.UNIFORM (x, a, b)

Function: CDF.UNIFORM (x, a, b)

Function: IDF.UNIFORM (p, a, b)

Function: RV.UNIFORM (a, b)

Uniform distribution with parameters a and b. Constraints: a <= x <= b, 0 <= p <= 1. An additional function is available as shorthand:

Function: UNIFORM (b)

Equivalent to RV.UNIFORM(0, b).

Function: PDF.WEIBULL (x, a, b)

Function: CDF.WEIBULL (x, a, b)

Function: IDF.WEIBULL (p, a, b)

Function: RV.WEIBULL (a, b)

Weibull distribution with parameters a and b. Constraints: a > 0, b > 0, x >= 0, 0 <= p < 1.

7.7.10.2 Discrete Distributions

The following discrete distributions are available:

Function: PDF.BERNOULLI (x)

Function: CDF.BERNOULLI (x, p)

Function: RV.BERNOULLI (p)

Bernoulli distribution with probability of success p. Constraints: x = 0 or 1, 0 <= p <= 1.

Function: PDF.BINOM (x, n, p)

Function: CDF.BINOM (x, n, p)

Function: RV.BINOM (n, p)

Binomial distribution with n trials and probability of success p. Constraints: integer n > 0, 0 <= p <= 1, integer x <= n.

Function: PDF.GEOM (x, n, p)

Function: CDF.GEOM (x, n, p)

Function: RV.GEOM (n, p)

Geometric distribution with probability of success p. Constraints: 0 <= p <= 1, integer x > 0.

Function: PDF.HYPER (x, a, b, c)

Function: CDF.HYPER (x, a, b, c)

Function: RV.HYPER (a, b, c)

Hypergeometric distribution when b objects out of a are drawn and c of the available objects are distinctive. Constraints: integer a > 0, integer b <= a, integer c <= a, integer x >= 0.

Function: PDF.LOG (x, p)

Function: RV.LOG (p)

Logarithmic distribution with probability parameter p. Constraints: 0 <= p < 1, x >= 1.

Function: PDF.NEGBIN (x, n, p)

Function: CDF.NEGBIN (x, n, p)

Function: RV.NEGBIN (n, p)

Negative binomial distribution with number of successes parameter n and probability of success parameter p. Constraints: integer n >= 0, 0 < p <= 1, integer x >= 1.

Function: PDF.POISSON (x, mu)

Function: CDF.POISSON (x, mu)

Function: RV.POISSON (mu)

Poisson distribution with mean mu. Constraints: mu > 0, integer x >= 0.

7.8 Operator Precedence

The following table describes operator precedence. Smaller-numbered levels in the table have higher precedence. Within a level, operations are always performed from left to right. The first occurrence of ‘-’ represents unary negation, the second binary subtraction.

1. ( )
2. **
3. -
4. * /
5. + -
6. EQ GE GT LE LT NE
7. AND NOT OR

8 Data Input and Output

Data are the focus of the PSPP language. Each datum belongs to a case (also called an observation). Each case represents an individual or “experimental unit”. For example, in the results of a survey, the names of the respondents, their sex, age, etc. and their responses are all data and the data pertaining to single respondent is a case. This chapter examines the PSPP commands for defining variables and reading and writing data. There are alternative commands to read data from predefined sources such as system files or databases (See GET DATA.)

Note: These commands tell PSPP how to read data, but the data will not actually be read until a procedure is executed.

8.1 BEGIN DATA

BEGIN DATA.
…
END DATA.

BEGIN DATA and END DATA can be used to embed raw ASCII data in a PSPP syntax file. DATA LIST or another input procedure must be used before BEGIN DATA (see DATA LIST). BEGIN DATA and END DATA must be used together. END DATA must appear by itself on a single line, with no leading white space and exactly one space between the words END and DATA, like this:

END DATA.

8.2 CLOSE FILE HANDLE

CLOSE FILE HANDLE handle_name.

CLOSE FILE HANDLE disassociates the name of a file handle with a given file. The only specification is the name of the handle to close. Afterward FILE HANDLE.

The file named INLINE, which represents data entered between BEGIN DATA and END DATA, cannot be closed. Attempts to close it with CLOSE FILE HANDLE have no effect.

CLOSE FILE HANDLE is a PSPP extension.

8.3 DATAFILE ATTRIBUTE

DATAFILE ATTRIBUTE
         ATTRIBUTE=name(’value’) [name(’value’)]…
         ATTRIBUTE=name[index](’value’) [name[index](’value’)]…
         DELETE=name [name]…
         DELETE=name[index] [name[index]]…

DATAFILE ATTRIBUTE adds, modifies, or removes user-defined attributes associated with the active dataset. Custom data file attributes are not interpreted by PSPP, but they are saved as part of system files and may be used by other software that reads them.

Use the ATTRIBUTE subcommand to add or modify a custom data file attribute. Specify the name of the attribute as an identifier (see Tokens), followed by the desired value, in parentheses, as a quoted string. Attribute names that begin with $ are reserved for PSPP’s internal use, and attribute names that begin with @ or $@ are not displayed by most PSPP commands that display other attributes. Other attribute names are not treated specially.

Attributes may also be organized into arrays. To assign to an array element, add an integer array index enclosed in square brackets ([ and ]) between the attribute name and value. Array indexes start at 1, not 0. An attribute array that has a single element (number 1) is not distinguished from a non-array attribute.

Use the DELETE subcommand to delete an attribute. Specify an attribute name by itself to delete an entire attribute, including all array elements for attribute arrays. Specify an attribute name followed by an array index in square brackets to delete a single element of an attribute array. In the latter case, all the array elements numbered higher than the deleted element are shifted down, filling the vacated position.

To associate custom attributes with particular variables, instead of with the entire active dataset, use VARIABLE ATTRIBUTE (see VARIABLE ATTRIBUTE) instead.

DATAFILE ATTRIBUTE takes effect immediately. It is not affected by conditional and looping structures such as DO IF or LOOP.

8.4 DATASET commands

DATASET NAME name [WINDOW={ASIS,FRONT}].
DATASET ACTIVATE name [WINDOW={ASIS,FRONT}].
DATASET COPY name [WINDOW={MINIMIZED,HIDDEN,FRONT}].
DATASET DECLARE name [WINDOW={MINIMIZED,HIDDEN,FRONT}].
DATASET CLOSE {name,*,ALL}.
DATASET DISPLAY.

The DATASET commands simplify use of multiple datasets within a PSPP session. They allow datasets to be created and destroyed. At any given time, most PSPP commands work with a single dataset, called the active dataset.

The DATASET NAME command gives the active dataset the specified name, or if it already had a name, it renames it. If another dataset already had the given name, that dataset is deleted.

The DATASET ACTIVATE command selects the named dataset, which must already exist, as the active dataset. Before switching the active dataset, any pending transformations are executed, as if EXECUTE had been specified. If the active dataset is unnamed before switching, then it is deleted and becomes unavailable after switching.

The DATASET COPY command creates a new dataset with the specified name, whose contents are a copy of the active dataset. Any pending transformations are executed, as if EXECUTE had been specified, before making the copy. If a dataset with the given name already exists, it is replaced. If the name is the name of the active dataset, then the active dataset becomes unnamed.

The DATASET DECLARE command creates a new dataset that is initially “empty,” that is, it has no dictionary or data. If a dataset with the given name already exists, this has no effect. The new dataset can be used with commands that support output to a dataset, e.g. AGGREGATE (see AGGREGATE).

The DATASET CLOSE command deletes a dataset. If the active dataset is specified by name, or if ‘*’ is specified, then the active dataset becomes unnamed. If a different dataset is specified by name, then it is deleted and becomes unavailable. Specifying ALL deletes all datasets except for the active dataset, which becomes unnamed.

The DATASET DISPLAY command lists all the currently defined datasets.

Many DATASET commands accept an optional WINDOW subcommand. In the PSPPIRE GUI, the value given for this subcommand influences how the dataset’s window is displayed. Outside the GUI, the WINDOW subcommand has no effect. The valid values are:

ASIS

Do not change how the window is displayed. This is the default for DATASET NAME and DATASET ACTIVATE.

FRONT

Raise the dataset’s window to the top. Make it the default dataset for running syntax.

MINIMIZED

Display the window “minimized” to an icon. Prefer other datasets for running syntax. This is the default for DATASET COPY and DATASET DECLARE.

HIDDEN

Hide the dataset’s window. Prefer other datasets for running syntax.

8.5 DATA LIST

Used to read text or binary data, DATA LIST is the most fundamental data-reading command. Even the more sophisticated input methods use DATA LIST commands as a building block. Understanding DATA LIST is important to understanding how to use PSPP to read your data files.

There are two major variants of DATA LIST, which are fixed format and free format. In addition, free format has a minor variant, list format, which is discussed in terms of its differences from vanilla free format.

Each form of DATA LIST is described in detail below.

See GET DATA, for a command that offers a few enhancements over DATA LIST and that may be substituted for DATA LIST in many situations.

8.5.1 DATA LIST FIXED

DATA LIST [FIXED]
        {TABLE,NOTABLE}
        [FILE=’file_name’ [ENCODING=’encoding’]]
        [RECORDS=record_count]
        [END=end_var]
        [SKIP=record_count]
        /[line_no] var_spec…

where each var_spec takes one of the forms
        var_list start-end [type_spec]
        var_list (fortran_spec)

DATA LIST FIXED is used to read data files that have values at fixed positions on each line of single-line or multiline records. The keyword FIXED is optional.

The FILE subcommand must be used if input is to be taken from an external file. It may be used to specify a file name as a string or a file handle (see File Handles). If the FILE subcommand is not used, then input is assumed to be specified within the command file using BEGIN DATA…END DATA (see BEGIN DATA). The ENCODING subcommand may only be used if the FILE subcommand is also used. It specifies the character encoding of the file. See INSERT, for information on supported encodings.

The optional RECORDS subcommand, which takes a single integer as an argument, is used to specify the number of lines per record. If RECORDS is not specified, then the number of lines per record is calculated from the list of variable specifications later in DATA LIST.

The END subcommand is only useful in conjunction with INPUT PROGRAM. See INPUT PROGRAM, for details.

The optional SKIP subcommand specifies a number of records to skip at the beginning of an input file. It can be used to skip over a row that contains variable names, for example.

DATA LIST can optionally output a table describing how the data file will be read. The TABLE subcommand enables this output, and NOTABLE disables it. The default is to output the table.

The list of variables to be read from the data list must come last. Each line in the data record is introduced by a slash (‘/’). Optionally, a line number may follow the slash. Following, any number of variable specifications may be present.

Each variable specification consists of a list of variable names followed by a description of their location on the input line. Sets of variables may be specified using the DATA LIST TO convention (see Sets of Variables). There are two ways to specify the location of the variable on the line: columnar style and FORTRAN style.

In columnar style, the starting column and ending column for the field are specified after the variable name, separated by a dash (‘-’). For instance, the third through fifth columns on a line would be specified ‘3-5’. By default, variables are considered to be in ‘F’ format (see Input and Output Formats). (This default can be changed; see SET for more information.)

In columnar style, to use a variable format other than the default, specify the format type in parentheses after the column numbers. For instance, for alphanumeric ‘A’ format, use ‘(A)’.

In addition, implied decimal places can be specified in parentheses after the column numbers. As an example, suppose that a data file has a field in which the characters ‘1234’ should be interpreted as having the value 12.34. Then this field has two implied decimal places, and the corresponding specification would be ‘(2)’. If a field that has implied decimal places contains a decimal point, then the implied decimal places are not applied.

Changing the variable format and adding implied decimal places can be done together; for instance, ‘(N,5)’.

When using columnar style, the input and output width of each variable is computed from the field width. The field width must be evenly divisible into the number of variables specified.

FORTRAN style is an altogether different approach to specifying field locations. With this approach, a list of variable input format specifications, separated by commas, are placed after the variable names inside parentheses. Each format specifier advances as many characters into the input line as it uses.

Implied decimal places also exist in FORTRAN style. A format specification with d decimal places also has d implied decimal places.

In addition to the standard format specifiers (see Input and Output Formats), FORTRAN style defines some extensions:

X

Advance the current column on this line by one character position.

Tx

Set the current column on this line to column x, with column numbers considered to begin with 1 at the left margin.

NEWRECx

Skip forward x lines in the current record, resetting the active column to the left margin.

Repeat count

Any format specifier may be preceded by a number. This causes the action of that format specifier to be repeated the specified number of times.

(spec1, …, specN)

Group the given specifiers together. This is most useful when preceded by a repeat count. Groups may be nested arbitrarily.

FORTRAN and columnar styles may be freely intermixed. Columnar style leaves the active column immediately after the ending column specified. Record motion using NEWREC in FORTRAN style also applies to later FORTRAN and columnar specifiers.

Examples

1. DATA LIST TABLE /NAME 1-10 (A) INFO1 TO INFO3 12-17 (1).
   
   BEGIN DATA.
   John Smith 102311
   Bob Arnold 122015
   Bill Yates  918 6
   END DATA.
   

   Defines the following variables:
   • NAME, a 10-character-wide string variable, in columns 1 through 10.
   • INFO1, a numeric variable, in columns 12 through 13.
   • INFO2, a numeric variable, in columns 14 through 15.
   • INFO3, a numeric variable, in columns 16 through 17.

   The BEGIN DATA/END DATA commands cause three cases to be defined:

   Case   NAME         INFO1   INFO2   INFO3
      1   John Smith     10      23      11
      2   Bob Arnold     12      20      15
      3   Bill Yates      9      18       6
   

   The TABLE keyword causes PSPP to print out a table describing the four variables defined.

2. DAT LIS FIL="survey.dat"
           /ID 1-5 NAME 7-36 (A) SURNAME 38-67 (A) MINITIAL 69 (A)
           /Q01 TO Q50 7-56
           /.
   

   Defines the following variables:
   • ID, a numeric variable, in columns 1-5 of the first record.
   • NAME, a 30-character string variable, in columns 7-36 of the first record.
   • SURNAME, a 30-character string variable, in columns 38-67 of the first record.
   • MINITIAL, a 1-character string variable, in column 69 of the first record.
   • Fifty variables Q01, Q02, Q03, …, Q49, Q50, all numeric, Q01 in column 7, Q02 in column 8, …, Q49 in column 55, Q50 in column 56, all in the second record.

   Cases are separated by a blank record.

   Data is read from file survey.dat in the current directory.

   This example shows keywords abbreviated to their first 3 letters.

8.5.2 DATA LIST FREE

DATA LIST FREE
        [({TAB,’c’}, …)]
        [{NOTABLE,TABLE}]
        [FILE=’file_name’ [ENCODING=’encoding’]]
        [SKIP=record_cnt]
        /var_spec…

where each var_spec takes one of the forms
        var_list [(type_spec)]
        var_list *

In free format, the input data is, by default, structured as a series of fields separated by spaces, tabs, commas, or line breaks. Each field’s content may be unquoted, or it may be quoted with a pairs of apostrophes (‘'’) or double quotes (‘"’). Unquoted white space separates fields but is not part of any field. Any mix of spaces, tabs, and line breaks is equivalent to a single space for the purpose of separating fields, but consecutive commas will skip a field.

Alternatively, delimiters can be specified explicitly, as a parenthesized, comma-separated list of single-character strings immediately following FREE. The word TAB may also be used to specify a tab character as a delimiter. When delimiters are specified explicitly, only the given characters, plus line breaks, separate fields. Furthermore, leading spaces at the beginnings of fields are not trimmed, consecutive delimiters define empty fields, and no form of quoting is allowed.

The NOTABLE and TABLE subcommands are as in DATA LIST FIXED above. NOTABLE is the default.

The FILE, SKIP, and ENCODING subcommands are as in DATA LIST FIXED above.

The variables to be parsed are given as a single list of variable names. This list must be introduced by a single slash (‘/’). The set of variable names may contain format specifications in parentheses (see Input and Output Formats). Format specifications apply to all variables back to the previous parenthesized format specification.

In addition, an asterisk may be used to indicate that all variables preceding it are to have input/output format ‘F8.0’.

Specified field widths are ignored on input, although all normal limits on field width apply, but they are honored on output.

8.5.3 DATA LIST LIST

DATA LIST LIST
        [({TAB,’c’}, …)]
        [{NOTABLE,TABLE}]
        [FILE=’file_name’ [ENCODING=’encoding’]]
        [SKIP=record_count]
        /var_spec…

where each var_spec takes one of the forms
        var_list [(type_spec)]
        var_list *

With one exception, DATA LIST LIST is syntactically and semantically equivalent to DATA LIST FREE. The exception is that each input line is expected to correspond to exactly one input record. If more or fewer fields are found on an input line than expected, an appropriate diagnostic is issued.

8.6 END CASE

END CASE.

END CASE is used only within INPUT PROGRAM to output the current case. See INPUT PROGRAM, for details.

8.7 END FILE

END FILE.

END FILE is used only within INPUT PROGRAM to terminate the current input program. See INPUT PROGRAM.

8.8 FILE HANDLE

For text files:
        FILE HANDLE handle_name
                /NAME=’file_name
                [/MODE=CHARACTER]
                [/ENDS={CR,CRLF}]
                /TABWIDTH=tab_width
                [ENCODING=’encoding’]

For binary files in native encoding with fixed-length records:
        FILE HANDLE handle_name
                /NAME=’file_name’
                /MODE=IMAGE
                [/LRECL=rec_len]
                [ENCODING=’encoding’]

For binary files in native encoding with variable-length records:
        FILE HANDLE handle_name
                /NAME=’file_name’
                /MODE=BINARY
                [/LRECL=rec_len]
                [ENCODING=’encoding’]

For binary files encoded in EBCDIC:
        FILE HANDLE handle_name
                /NAME=’file_name’
                /MODE=360
                /RECFORM={FIXED,VARIABLE,SPANNED}
                [/LRECL=rec_len]
                [ENCODING=’encoding’]

Use FILE HANDLE to associate a file handle name with a file and its attributes, so that later commands can refer to the file by its handle name. Names of text files can be specified directly on commands that access files, so that FILE HANDLE is only needed when a file is not an ordinary file containing lines of text. However, FILE HANDLE may be used even for text files, and it may be easier to specify a file’s name once and later refer to it by an abstract handle.

Specify the file handle name as the identifier immediately following the FILE HANDLE command name. The identifier INLINE is reserved for representing data embedded in the syntax file (see BEGIN DATA) The file handle name must not already have been used in a previous invocation of FILE HANDLE, unless it has been closed by an intervening command (see CLOSE FILE HANDLE).

The effect and syntax of FILE HANDLE depends on the selected MODE:

• In CHARACTER mode, the default, the data file is read as a text file. Each text line is read as one record.

  In CHARACTER mode only, tabs are expanded to spaces by input programs, except by DATA LIST FREE with explicitly specified delimiters. Each tab is 4 characters wide by default, but TABWIDTH (a PSPP extension) may be used to specify an alternate width. Use a TABWIDTH of 0 to suppress tab expansion.

  A file written in CHARACTER mode by default uses the line ends of the system on which PSPP is running, that is, on Windows, the default is CR LF line ends, and on other systems the default is LF only. Specify ENDS as CR or CRLF to override the default. PSPP reads files using either convention on any kind of system, regardless of ENDS.

• In IMAGE mode, the data file is treated as a series of fixed-length binary records. LRECL should be used to specify the record length in bytes, with a default of 1024. On input, it is an error if an IMAGE file’s length is not a integer multiple of the record length. On output, each record is padded with spaces or truncated, if necessary, to make it exactly the correct length.
• In BINARY mode, the data file is treated as a series of variable-length binary records. LRECL may be specified, but its value is ignored. The data for each record is both preceded and followed by a 32-bit signed integer in little-endian byte order that specifies the length of the record. (This redundancy permits records in these files to be efficiently read in reverse order, although PSPP always reads them in forward order.) The length does not include either integer.
• Mode 360 reads and writes files in formats first used for tapes in the 1960s on IBM mainframe operating systems and still supported today by the modern successors of those operating systems. For more information, see OS/400 Tape and Diskette Device Programming, available on IBM’s website.

  Alphanumeric data in mode 360 files are encoded in EBCDIC. PSPP translates EBCDIC to or from the host’s native format as necessary on input or output, using an ASCII/EBCDIC translation that is one-to-one, so that a “round trip” from ASCII to EBCDIC back to ASCII, or vice versa, always yields exactly the original data.

  The RECFORM subcommand is required in mode 360. The precise file format depends on its setting:

  F

  FIXED

  This record format is equivalent to IMAGE mode, except for EBCDIC translation.

  IBM documentation calls this *F (fixed-length, deblocked) format.

  V

  VARIABLE

  The file comprises a sequence of zero or more variable-length blocks. Each block begins with a 4-byte block descriptor word (BDW). The first two bytes of the BDW are an unsigned integer in big-endian byte order that specifies the length of the block, including the BDW itself. The other two bytes of the BDW are ignored on input and written as zeros on output.

  Following the BDW, the remainder of each block is a sequence of one or more variable-length records, each of which in turn begins with a 4-byte record descriptor word (RDW) that has the same format as the BDW. Following the RDW, the remainder of each record is the record data.

  The maximum length of a record in VARIABLE mode is 65,527 bytes: 65,535 bytes (the maximum value of a 16-bit unsigned integer), minus 4 bytes for the BDW, minus 4 bytes for the RDW.

  In mode VARIABLE, LRECL specifies a maximum, not a fixed, record length, in bytes. The default is 8,192.

  IBM documentation calls this *VB (variable-length, blocked, unspanned) format.

  VS

  SPANNED

  The file format is like that of VARIABLE mode, except that logical records may be split among multiple physical records (called segments) or blocks. In SPANNED mode, the third byte of each RDW is called the segment control character (SCC). Odd SCC values cause the segment to be appended to a record buffer maintained in memory; even values also append the segment and then flush its contents to the input procedure. Canonically, SCC value 0 designates a record not spanned among multiple segments, and values 1 through 3 designate the first segment, the last segment, or an intermediate segment, respectively, within a multi-segment record. The record buffer is also flushed at end of file regardless of the final record’s SCC.

  The maximum length of a logical record in VARIABLE mode is limited only by memory available to PSPP. Segments are limited to 65,527 bytes, as in VARIABLE mode.

  This format is similar to what IBM documentation call *VS (variable-length, deblocked, spanned) format.

  In mode 360, fields of type A that extend beyond the end of a record read from disk are padded with spaces in the host’s native character set, which are then translated from EBCDIC to the native character set. Thus, when the host’s native character set is based on ASCII, these fields are effectively padded with character X'80'. This wart is implemented for compatibility.

The NAME subcommand specifies the name of the file associated with the handle. It is required in all modes but SCRATCH mode, in which its use is forbidden.

The ENCODING subcommand specifies the encoding of text in the file. For reading text files in CHARACTER mode, all of the forms described for ENCODING on the INSERT command are supported (see INSERT). For reading in other file-based modes, encoding autodetection is not supported; if the specified encoding requests autodetection then the default encoding will be used. This is also true when a file handle is used for writing a file in any mode.

8.9 INPUT PROGRAM

INPUT PROGRAM.
… input commands …
END INPUT PROGRAM.

INPUT PROGRAM…END INPUT PROGRAM specifies a complex input program. By placing data input commands within INPUT PROGRAM, PSPP programs can take advantage of more complex file structures than available with only DATA LIST.

The first sort of extended input program is to simply put multiple DATA LIST commands within the INPUT PROGRAM. This will cause all of the data files to be read in parallel. Input will stop when end of file is reached on any of the data files.

Transformations, such as conditional and looping constructs, can also be included within INPUT PROGRAM. These can be used to combine input from several data files in more complex ways. However, input will still stop when end of file is reached on any of the data files.

To prevent INPUT PROGRAM from terminating at the first end of file, use the END subcommand on DATA LIST. This subcommand takes a variable name, which should be a numeric scratch variable (see Scratch Variables). (It need not be a scratch variable but otherwise the results can be surprising.) The value of this variable is set to 0 when reading the data file, or 1 when end of file is encountered.

Two additional commands are useful in conjunction with INPUT PROGRAM. END CASE is the first. Normally each loop through the INPUT PROGRAM structure produces one case. END CASE controls exactly when cases are output. When END CASE is used, looping from the end of INPUT PROGRAM to the beginning does not cause a case to be output.

END FILE is the second. When the END subcommand is used on DATA LIST, there is no way for the INPUT PROGRAM construct to stop looping, so an infinite loop results. END FILE, when executed, stops the flow of input data and passes out of the INPUT PROGRAM structure.

INPUT PROGRAM must contain at least one DATA LIST or END FILE command.

All this is very confusing. A few examples should help to clarify.

INPUT PROGRAM.
        DATA LIST NOTABLE FILE='a.data'/X 1-10.
        DATA LIST NOTABLE FILE='b.data'/Y 1-10.
END INPUT PROGRAM.
LIST.

The example above reads variable X from file a.data and variable Y from file b.data. If one file is shorter than the other then the extra data in the longer file is ignored.

INPUT PROGRAM.
        NUMERIC #A #B.
        
        DO IF NOT #A.
                DATA LIST NOTABLE END=#A FILE='a.data'/X 1-10.
        END IF.
        DO IF NOT #B.
                DATA LIST NOTABLE END=#B FILE='b.data'/Y 1-10.
        END IF.
        DO IF #A AND #B.
                END FILE.
        END IF.
        END CASE.
END INPUT PROGRAM.
LIST.

The above example reads variable X from a.data and variable Y from b.data. If one file is shorter than the other then the missing field is set to the system-missing value alongside the present value for the remaining length of the longer file.

INPUT PROGRAM.
        NUMERIC #A #B.

        DO IF #A.
                DATA LIST NOTABLE END=#B FILE='b.data'/X 1-10.
                DO IF #B.
                        END FILE.
                ELSE.
                        END CASE.
                END IF.
        ELSE.
                DATA LIST NOTABLE END=#A FILE='a.data'/X 1-10.
                DO IF NOT #A.
                        END CASE.
                END IF.
        END IF.
END INPUT PROGRAM.
LIST.

The above example reads data from file a.data, then from b.data, and concatenates them into a single active dataset.

INPUT PROGRAM.
        NUMERIC #EOF.

        LOOP IF NOT #EOF.
                DATA LIST NOTABLE END=#EOF FILE='a.data'/X 1-10.
                DO IF NOT #EOF.
                        END CASE.
                END IF.
        END LOOP.

        COMPUTE #EOF = 0.
        LOOP IF NOT #EOF.
                DATA LIST NOTABLE END=#EOF FILE='b.data'/X 1-10.
                DO IF NOT #EOF.
                        END CASE.
                END IF.
        END LOOP.

        END FILE.
END INPUT PROGRAM.
LIST.

The above example does the same thing as the previous example, in a different way.

INPUT PROGRAM.
        LOOP #I=1 TO 50.
                COMPUTE X=UNIFORM(10).
                END CASE.
        END LOOP.
        END FILE.
END INPUT PROGRAM.
LIST/FORMAT=NUMBERED.

The above example causes an active dataset to be created consisting of 50 random variates between 0 and 10.

8.10 LIST

LIST
        /VARIABLES=var_list
        /CASES=FROM start_index TO end_index BY incr_index
        /FORMAT={UNNUMBERED,NUMBERED} {WRAP,SINGLE}

The LIST procedure prints the values of specified variables to the listing file.

The VARIABLES subcommand specifies the variables whose values are to be printed. Keyword VARIABLES is optional. If VARIABLES subcommand is not specified then all variables in the active dataset are printed.

The CASES subcommand can be used to specify a subset of cases to be printed. Specify FROM and the case number of the first case to print, TO and the case number of the last case to print, and BY and the number of cases to advance between printing cases, or any subset of those settings. If CASES is not specified then all cases are printed.

The FORMAT subcommand can be used to change the output format. NUMBERED will print case numbers along with each case; UNNUMBERED, the default, causes the case numbers to be omitted. The WRAP and SINGLE settings are currently not used.

Case numbers start from 1. They are counted after all transformations have been considered.

LIST attempts to fit all the values on a single line. If needed to make them fit, variable names are displayed vertically. If values cannot fit on a single line, then a multi-line format will be used.

LIST is a procedure. It causes the data to be read.

8.11 NEW FILE

NEW FILE.

NEW FILE command clears the dictionary and data from the current active dataset.

8.12 PRINT

PRINT 
        [OUTFILE=’file_name’]
        [RECORDS=n_lines]
        [{NOTABLE,TABLE}]
        [ENCODING=’encoding’]
        [/[line_no] arg…]

arg takes one of the following forms:
        ’string’ [start]
        var_list start-end [type_spec]
        var_list (fortran_spec)
        var_list *

The PRINT transformation writes variable data to the listing file or an output file. PRINT is executed when a procedure causes the data to be read. Follow PRINT by EXECUTE to print variable data without invoking a procedure (see EXECUTE).

All PRINT subcommands are optional. If no strings or variables are specified, PRINT outputs a single blank line.

The OUTFILE subcommand specifies the file to receive the output. The file may be a file name as a string or a file handle (see File Handles). If OUTFILE is not present then output will be sent to PSPP’s output listing file. When OUTFILE is present, a space is inserted at beginning of each output line, even lines that otherwise would be blank.

The ENCODING subcommand may only be used if the OUTFILE subcommand is also used. It specifies the character encoding of the file. See INSERT, for information on supported encodings.

The RECORDS subcommand specifies the number of lines to be output. The number of lines may optionally be surrounded by parentheses.

TABLE will cause the PRINT command to output a table to the listing file that describes what it will print to the output file. NOTABLE, the default, suppresses this output table.

Introduce the strings and variables to be printed with a slash (‘/’). Optionally, the slash may be followed by a number indicating which output line will be specified. In the absence of this line number, the next line number will be specified. Multiple lines may be specified using multiple slashes with the intended output for a line following its respective slash.

Literal strings may be printed. Specify the string itself. Optionally the string may be followed by a column number, specifying the column on the line where the string should start. Otherwise, the string will be printed at the current position on the line.

Variables to be printed can be specified in the same ways as available for DATA LIST FIXED (see DATA LIST FIXED). In addition, a variable list may be followed by an asterisk (‘*’), which indicates that the variables should be printed in their dictionary print formats, separated by spaces. A variable list followed by a slash or the end of command will be interpreted the same way.

If a FORTRAN type specification is used to move backwards on the current line, then text is written at that point on the line, the line will be truncated to that length, although additional text being added will again extend the line to that length.

8.13 PRINT EJECT

PRINT EJECT 
        OUTFILE=’file_name’
        RECORDS=n_lines
        {NOTABLE,TABLE}
        /[line_no] arg…

arg takes one of the following forms:
        ’string’ [start-end]
        var_list start-end [type_spec]
        var_list (fortran_spec)
        var_list *

PRINT EJECT advances to the beginning of a new output page in the listing file or output file. It can also output data in the same way as PRINT.

All PRINT EJECT subcommands are optional.

Without OUTFILE, PRINT EJECT ejects the current page in the listing file, then it produces other output, if any is specified.

With OUTFILE, PRINT EJECT writes its output to the specified file. The first line of output is written with ‘1’ inserted in the first column. Commonly, this is the only line of output. If additional lines of output are specified, these additional lines are written with a space inserted in the first column, as with PRINT.

See PRINT, for more information on syntax and usage.

8.14 PRINT SPACE

PRINT SPACE [OUTFILE=’file_name’] [ENCODING=’encoding’] [n_lines].

PRINT SPACE prints one or more blank lines to an output file.

The OUTFILE subcommand is optional. It may be used to direct output to a file specified by file name as a string or file handle (see File Handles). If OUTFILE is not specified then output will be directed to the listing file.

The ENCODING subcommand may only be used if OUTFILE is also used. It specifies the character encoding of the file. See INSERT, for information on supported encodings.

n_lines is also optional. If present, it is an expression (see Expressions) specifying the number of blank lines to be printed. The expression must evaluate to a nonnegative value.

8.15 REREAD

REREAD [FILE=handle] [COLUMN=column] [ENCODING=’encoding’].

The REREAD transformation allows the previous input line in a data file already processed by DATA LIST or another input command to be re-read for further processing.

The FILE subcommand, which is optional, is used to specify the file to have its line re-read. The file must be specified as the name of a file handle (see File Handles). If FILE is not specified then the last file specified on DATA LIST will be assumed (last file specified lexically, not in terms of flow-of-control).

By default, the line re-read is re-read in its entirety. With the COLUMN subcommand, a prefix of the line can be exempted from re-reading. Specify an expression (see Expressions) evaluating to the first column that should be included in the re-read line. Columns are numbered from 1 at the left margin.

The ENCODING subcommand may only be used if the FILE subcommand is also used. It specifies the character encoding of the file. See INSERT, for information on supported encodings.

Issuing REREAD multiple times will not back up in the data file. Instead, it will re-read the same line multiple times.

8.16 REPEATING DATA

REPEATING DATA
        /STARTS=start-end
        /OCCURS=n_occurs
        /FILE=’file_name’
        /LENGTH=length
        /CONTINUED[=cont_start-cont_end]
        /ID=id_start-id_end=id_var
        /{TABLE,NOTABLE}
        /DATA=var_spec…

where each var_spec takes one of the forms
        var_list start-end [type_spec]
        var_list (fortran_spec)

REPEATING DATA parses groups of data repeating in a uniform format, possibly with several groups on a single line. Each group of data corresponds with one case. REPEATING DATA may only be used within an INPUT PROGRAM structure (see INPUT PROGRAM). When used with DATA LIST, it can be used to parse groups of cases that share a subset of variables but differ in their other data.

The STARTS subcommand is required. Specify a range of columns, using literal numbers or numeric variable names. This range specifies the columns on the first line that are used to contain groups of data. The ending column is optional. If it is not specified, then the record width of the input file is used. For the inline file (see BEGIN DATA) this is 80 columns; for a file with fixed record widths it is the record width; for other files it is 1024 characters by default.

The OCCURS subcommand is required. It must be a number or the name of a numeric variable. Its value is the number of groups present in the current record.

The DATA subcommand is required. It must be the last subcommand specified. It is used to specify the data present within each repeating group. Column numbers are specified relative to the beginning of a group at column 1. Data is specified in the same way as with DATA LIST FIXED (see DATA LIST FIXED).

All other subcommands are optional.

FILE specifies the file to read, either a file name as a string or a file handle (see File Handles). If FILE is not present then the default is the last file handle used on DATA LIST (lexically, not in terms of flow of control).

By default REPEATING DATA will output a table describing how it will parse the input data. Specifying NOTABLE will disable this behavior; specifying TABLE will explicitly enable it.

The LENGTH subcommand specifies the length in characters of each group. If it is not present then length is inferred from the DATA subcommand. LENGTH can be a number or a variable name.

Normally all the data groups are expected to be present on a single line. Use the CONTINUED command to indicate that data can be continued onto additional lines. If data on continuation lines starts at the left margin and continues through the entire field width, no column specifications are necessary on CONTINUED. Otherwise, specify the possible range of columns in the same way as on STARTS.

When data groups are continued from line to line, it is easy for cases to get out of sync through careless hand editing. The ID subcommand allows a case identifier to be present on each line of repeating data groups. REPEATING DATA will check for the same identifier on each line and report mismatches. Specify the range of columns that the identifier will occupy, followed by an equals sign (‘=’) and the identifier variable name. The variable must already have been declared with NUMERIC or another command.

REPEATING DATA should be the last command given within an INPUT PROGRAM. It should not be enclosed within a LOOP structure (see LOOP). Use DATA LIST before, not after, REPEATING DATA.

8.17 WRITE

WRITE 
        OUTFILE=’file_name’
        RECORDS=n_lines
        {NOTABLE,TABLE}
        /[line_no] arg…

arg takes one of the following forms:
        ’string’ [start-end]
        var_list start-end [type_spec]
        var_list (fortran_spec)
        var_list *

WRITE writes text or binary data to an output file.

See PRINT, for more information on syntax and usage. PRINT and WRITE differ in only a few ways:

• WRITE uses write formats by default, whereas PRINT uses print formats.
• PRINT inserts a space between variables unless a format is explicitly specified, but WRITE never inserts space between variables in output.
• PRINT inserts a space at the beginning of each line that it writes to an output file (and PRINT EJECT inserts ‘1’ at the beginning of each line that should begin a new page), but WRITE does not.
• PRINT outputs the system-missing value according to its specified output format, whereas WRITE outputs the system-missing value as a field filled with spaces. Binary formats are an exception.

9 System and Portable File I/O

The commands in this chapter read, write, and examine system files and portable files.