DataFrames in Statistics
Description
Examples
This help page describes how to use Statistics commands on DataFrame objects, and other spreadsheet-type data in matrices, sometimes called Matrix data sets.
Many of the data sets you might encounter are two-dimensional in nature. They will have information about a number of items or events; for each item or event, the same properties are known. Such data sets can easily be represented in a DataFrame by having each row correspond to an item and each column to one property of all these items. This is how you would typically store such data in a spreadsheet. You can also store such data in a Matrix, as long as you keep track of labels for the rows and columns yourself.
Many commands in the Statistics package can be used with this type of data:
The following computational commands can be run on DataFrame objects (or Matrices). They are computed per column and the results are returned in a DataSeries object. The labels for the DataSeries are the column labels of the DataFrame. Alternatively, they are computed per column of a Matrix and the results are returned in a row Vector.
AbsoluteDeviation
CentralMoment
Count
CountMissing
Cumulant
DataSummary
Decile
Detrend
Difference
ExpectedValue
FivePointSummary
GeometricMean
HarmonicMean
HodgesLehmann
InterquartileRange
Kurtosis
Mean
MeanDeviation
Median
MedianDeviation
Mode
Moment
Percentile
QuadraticMean
Quantile
Quartile
Range
RousseeuwCrouxQn
RousseeuwCrouxSn
Scale
Skewness
StandardDeviation
StandardError
StandardizedMoment
TrimmedMean
Variance
Variation
WinsorizedMean
The following visualization commands, listed on the Statistics Visualization help page, also accept DataFrame objects. Generally, the row and column labels are used to label data points and data sets, respectively, as appropriate.
AgglomeratedPlot
AreaChart
BarChart
Biplot
BoxPlot
BubblePlot
ColumnGraph
CumulativeSumChart
ErrorPlot
FrequencyPlot
GridPlot
LineChart
ParetoChart
PointPlot
ScatterPlot
ScreePlot
ViolinPlot
Statistics[SplitByColumn] and Statistics[Join] split Matrices into submatrices and join them back together.
DataFrame/Aggregate does similar things for DataFrame objects.
Additional examples are found in the Statistics with DataFrames example worksheet.
with⁡Statistics:
We construct a DataFrame with housing data. The first column has number of bedrooms, the second has the area in square feet, the third has price.
bedrooms≔3,4,2,4,3,2,2,3,4,4,2,4,4,3,3
area≔1130,1123,1049,1527,907,580,878,1075,1040,1295,1100,995,908,853,856
price≔114700,125200,81600,127400,88500,59500,96500,113300,104400,136600,80100,128000,115700,94700,89400
HouseSalesData≔DataFrame⁡bedrooms,area,price,columns=Bedrooms,Area,Price
We can determine the average number of bedrooms, average area, and average price with just the Mean command.
Mean⁡HouseSalesData
We can also determine the standard error for this mean.
StandardError⁡Mean,HouseSalesData
Or the 30th percentile for each column.
Percentile⁡HouseSalesData,30
The GridPlot command can display scatter plots of pairs of columns.
GridPlot⁡HouseSalesData
Tabulate
Bedrooms
Area
Price
We can use the lower diagonal entries to display the values for the correlation.
GridPlot⁡HouseSalesData,lower=Correlation
Tabulate0
0.488597380223581068
0.835789649694966386
0.704398937794964319
We can determine the average area and price for subgroups of sales defined by number of bedrooms. (The Aggregate command is part of the DataFrame object, not the Statistics package, so it is not available for Matrices.)
Aggregate⁡HouseSalesData,Bedrooms
To create a box plot of prices for each number of bedrooms requires a little more effort.
split≔SplitByColumn⁡HouseSalesData,Bedrooms
split≔?,?,?
price_split≔map⁡df↦convert⁡dfPrice,Vector,split
price_split≔,,
BoxPlot⁡price_split,datasetlabels=2,3,4
Most of the things mentioned above can be done with a Matrix, too. Consider the following examples.
HSD_Matrix≔convert⁡HouseSalesData,Matrix
Mean⁡HSD_Matrix
Percentile⁡HSD_Matrix,30
Some commands have calling sequences where one of the arguments is compared to the data; this is the case for the second argument of AbsoluteDeviation and for the origin parameter of Moment. In these cases, it typically does not make much sense to use the same value for each column, so Maple supports using a list or Vector of values instead. These commands do not yet work directly with DataFrame objects.
AbsoluteDeviation⁡HSD_Matrix,3,1000,100000
StandardError⁡Moment,HSD_Matrix,1,origin=3,1000,100000
See Also
examples/DataFrame/Statistics
Download Help Document