Here we perform least-squares regression to fit a Fourier series to a set of sample data given by:

$\mathrm{restart}\:$
$\mathrm{Points} ≔ \left[\begin{array}{cc}0.& 2.1\\ 1.& -1.5\\ 2.& -3.1\\ 3.& 6.3\\ 4.& 8.2\\ 5.& 11.5\\ 6.& 12.7\\ 7.& 8.4\end{array}\right]\:$

The general formula for a Fourier series with $2 N\+1$ terms and coefficients $c\,\mathrm{a\_\_1}\,..\.\,\mathrm{a\_\_N}\,\mathrm{b\_\_1}\,..\.\,\mathrm{b\_\_N}$ is given by:

$F≔\left(x\,a\,b\,c\,N\right)\to c \+ \mathrm{add}\left(a\left[n\right] \cos \left(n x\right) \+ b\left[n\right] \sin \left(n x\right)\,n\=1..N\right)$

For this example we take $N\=4$:$N ≔ 4$

We first declare a sequence of variables in the deep learning graph corresponding to each $a\left[n\right]$ and $b\left[n\right]$:

$\mathrm{with}\left(\mathrm{DeepLearning}\right)\:$ 

$\mathrm{SetEagerExecution}\left(\mathrm{false}\right)$

$a ≔ \mathrm{Array}\left(1..N\,i\to \mathrm{Variable}\left(\left[0.5\right]\, \mathrm{datatype}\=\mathrm{float}\left[4\right]\right)\right)\:$

$b ≔ \mathrm{Array}\left(1..N\,i\to \mathrm{Variable}\left(\left[0.5\right]\, \mathrm{datatype}\=\mathrm{float}\left[4\right]\right)\right)\:$

$c ≔ \mathrm{Variable}\left(\left[0.5\right]\, \mathrm{datatype}\=\mathrm{float}\left[4\right]\right)\:$

We can now define placeholders to hold the sample x- and y-values against which we want to fit $F\left(x\,a\,b\,N\right)$:

$x ≔ \mathrm{Placeholder}\left( \mathrm{float}\left[4\right]\right)$

$y ≔ \mathrm{Placeholder}\left( \mathrm{float}\left[4\right]\right)$

We can now define a least-squares distance function which we aim to minimize:

$\mathrm{loss} ≔\mathrm{ReduceSum}\left({\left(F\left(x\,a\,b\,c\,N\right)- y\right)}^2\right)$

$\mathrm{optimizer} ≔ \mathrm{Optimizer}\left(\mathrm{GradientDescent}\left(0.01\right)\right)\;$

$\mathrm{train} ≔ \mathrm{optimizer}:-\mathrm{Minimize}\left(\mathrm{loss}\right)\;$

With the structure of our deep learning graph defined, we can proceed to train it on sample data.  After initializing the session, we run 1000 training cycles using the training data.

$\mathrm{x\_\_training} ≔ \mathrm{convert}\left(\mathrm{Points}\left[..\,1\right]\,\mathrm{list}\right)$

$\mathrm{y\_\_training} ≔ \mathrm{convert}\left(\mathrm{Points}\left[..\,2\right]\,\mathrm{list}\right)$

$\mathrm{init} ≔\mathrm{VariablesInitializer}\left(\right)\:$ 

$\mathrm{sess} ≔ \mathrm{Session}\left(\right)$

$\mathrm{sess}:-\mathrm{Run}\left(\mathrm{init}\right)\:$

$$\begin{gathered} \mathbf{for} i \mathbf{to} 1000 \mathbf{do} \\ \mathrm{sess}:-\mathrm{Run}\left( \mathrm{train}\, \left\{x \mathbf{in} \mathrm{x\_\_training}\, y \mathbf{in} \mathrm{y\_\_training}\right\} \right)\: \\ \mathbf{end} \mathbf{do}\: \end{gathered}$$

We can now query the state of the trained model for the present value of the loss function:

$\mathrm{sess}:-\mathrm{Run}\left(\mathrm{loss}\,\left[x \mathbf{in} \mathrm{x\_\_training}\, y \mathbf{in} \mathrm{y\_\_training}\right]\right)$

As this is a small value, we have a close fit for the data.  We can then obtain the final value of the parameters from the trained model.

$\mathrm{result}≔ \mathrm{sess}:-\mathrm{Run}\left(\left[\mathrm{Concatenate}\left(a\,0\right)\,\mathrm{Concatenate}\left(b\,0\right)\,c\right]\,\left[x \mathbf{in} \mathrm{x\_\_training}\, y \mathbf{in} \mathrm{y\_\_training}\right]\right)$

$A\,B\,C ≔ \mathrm{result}\left[1\right]\,\mathrm{result}\left[2\right]\,\mathrm{result}\left[3\right]$

Finally, we can visualize the result:
$\mathrm{with}\left(\mathrm{plots}\right)\:$

Here we use a deep neural network to classify the famous Iris flower data set collected by Edgar Anderson and made famous by Ronald Fisher.  This data set includes 150 distinct observations of iris flowers, each of which consists of four empirical observations (sepal length, sepal width, petal length, and petal width) along with a classification into one of three known species (I. setosa, I. versicolor, and I. virginica).

We will repeat here the classical task for which this data set is used: attempting prediction of the species based on the four measured quantities

Training and Test Data
We have divided the data in into training and test data: the former is used to build the model, the latter is used to test its predictive accuracy.      $$

$$\begin{gathered} \mathrm{training\_data} ≔ \mathrm{Import}\left(''example/iris\_training.csv''\,\mathrm{base}\=\mathrm{datadir}\right)\; \\ \mathrm{test\_data} ≔ \mathrm{Import}\left(''example/iris\_test.csv''\, \mathrm{base}\=\mathrm{datadir}\right)\; \end{gathered}$$

We see that this data set has 150 samples (120 for training and 30 for testing) and that the Species column has three distinct species:    $\mathrm{NumRows}\left(\mathrm{test\_data}\right)\, \mathrm{NumRows}\left(\mathrm{training\_data}\right)$

$\mathrm{convert}\left(\mathrm{test\_data}\left[\mathrm{Species}\right]\,\mathrm{set}\right) \mathbf{union} \mathrm{convert}\left(\mathrm{training\_data}\left[\mathrm{Species}\right]\,\mathrm{set}\right)$

To simplify things we will replace the strings designating the species classification with the numbers 0,1,2 (corresponding to setosa, versicolor, and virginica, respectively):

  $\mathrm{training\_set} ≔ \mathrm{map}\left(\mathrm{eval}\,\mathrm{training\_data}\,\left[''setosa''\=0\,''versicolor''\=1\,''virginica''\=2\right]\right)\;$

$\mathrm{test\_set} ≔ \mathrm{map}\left(\mathrm{eval}\,\mathrm{test\_data}\,\left[''setosa''\=0\,''versicolor''\=1\,''virginica''\=2\right]\right)\;$

Training the Deep Neural Network Model

With our data prepared, we can now actually define and train the model.

$\mathrm{with}\left(\mathrm{DeepLearning}\right)\:$

Our first step is to define a feature for each of the four observed quantities in the test data minus the final one (species) which we aim to predict:

$\mathrm{cols} ≔ \mathrm{ColumnLabels}\left(\mathrm{test\_set}\right)$

$\mathrm{fc} ≔ \mathrm{map}\left( \mathrm{NumericColumn}\, \mathrm{cols}\left[1..4\right]\, \mathrm{shape}\=\left[1\right] \right)$

We can now define a deep neural network classifier with these features.  It has 3 classes because there are 3 species of iris in the dataset.

$\mathrm{classifier} ≔ \mathrm{DNNClassifier}\left(\mathrm{fc}\, \mathrm{hidden\_units} \= \left[10\,20\,10\right]\, \mathrm{num\_classes} \= 3\right)\;$

We are now ready to train the model.

$\mathrm{classifier}:-\mathrm{Train}\left( \mathrm{training\_set}\left[1..4\right]\, \mathrm{training\_set}\left[5\right]\, \mathrm{steps}\=2000\, \mathrm{num\_epochs}\=\mathrm{none}\, \mathrm{shuffle}\=\mathrm{true} \right)$

Now trained, we can evaluate the classifier on the test set, and we see that we have achieved 96.7% predictive accuracy:

$\mathrm{results} ≔ \mathrm{classifier}:-\mathrm{Evaluate}\left( \mathrm{test\_set}\left[1..4\right]\, \mathrm{test\_set}\left[5\right]\, \mathrm{shuffle}\=\mathrm{false}\, \mathrm{num\_epochs}\=1 \right)$

We can now build a predictor function that takes an arbitrary set of measurements as a DataSeries and returns a prediction:

$\mathrm{predictor} ≔ \mathbf{proc}\left(\mathrm{ds}\right) \mathrm{classifier}:-\mathrm{Predict}\left(\mathrm{Transpose}\left(\mathrm{DataFrame}\left(\mathrm{ds}\right)\right)\, \mathrm{num\_epochs}\=1\, \mathrm{shuffle}\=\mathrm{false}\right)\left[1\right] \mathbf{end} \mathbf{proc}\:$

Using a Trained Model

With this we can take an arbitrary new point, and generate a prediction from the trained model:

$\mathrm{ds} ≔ \mathrm{DataSeries}\left(\left[5.8\, 3.1\, 5.0\, 1.7\right]\,\mathrm{labels}\=\mathrm{cols}\left[1..4\right]\right)$

$\mathrm{predictor}\left(\mathrm{ds}\right)$

The probabilities field in the above result records the estimated probabilities for each class.

In this case, the model predicts with high probability that this particular sample is class 2, and therefore I. virginica.