The requisite extension assigns to the origin the value of the bivariate limit of $f$ at the origin. Hence, what is required is to show that this limit is zero, a computation summarized below.

Since $\left|f-0\right|\le \sqrt{x^2\+y^2}$, it is clear that $\left|f-0\right|\to 0$ as $\left(x\,y\right)\to \left(0\,0\right)$. Hence, the required extension is

$g\left(x\,y\right)\=\{\begin{array}{cc}f\left(x\,y\right)& \left(x\,y\right)\ne \left(0\,0\right)\\ 0& \left(x\,y\right)\=\left(0\,0\right)\end{array}$

Indeed, $\mathrm{limit}\left(\frac{x^{2}y\cos \left(xy\right)}{x^2\+y^2}\,\left\{x\=0\,y\=0\right\}\right)$ = $0$.