The general form of the Van der Pol ODE is given by the following:

Van_der_Pol_ode := diff(y(x),x,x)-mu*(1-y(x)^2)*diff(y(x),x)+y(x)=0;

$\mathrm{Van\_der\_Pol\_ode}≔\frac{{\ⅆ}^2}{\ⅆx^2}y\left(x\right)-\mathrm{\mu }\left(1-{y\left(x\right)}^2\right)\left(\frac{\ⅆ}{\ⅆx}y\left(x\right)\right)+y\left(x\right)=0$

See Birkhoff and Rota, "Ordinary Differential Equations", p. 134.

The second order Van der Pol ODE can be reduced to a first order ODE of Abel type as soon as the system succeeds in finding one polynomial symmetry for it (see symgen):

with(DEtools, odeadvisor, symgen):

odeadvisor(Van_der_Pol_ode);

$\left[\left[\mathrm{\_2nd\_order}\,\mathrm{\_missing\_x}\right]\,\mathrm{\_Van\_der\_Pol}\right]$

symgen(Van_der_Pol_ode, way=3);

$\left[\mathrm{\_\ξ}=1\,\mathrm{\_\η}=0\right]$

From which, giving the same indication directly to dsolve you obtain the reduction of order

ans := dsolve(Van_der_Pol_ode,way=3);

$\mathrm{ans}≔y\left(x\right)=\mathrm{\_a}\mathbf{where}\left[\left\{\left(\frac{\ⅆ}{\ⅆ\mathrm{\_a}}\mathrm{\_b}\left(\mathrm{\_a}\right)\right)\mathrm{\_b}\left(\mathrm{\_a}\right)+\mathrm{\_b}\left(\mathrm{\_a}\right){\mathrm{\_a}}^2\mathrm{\mu }-\mathrm{\mu }\mathrm{\_b}\left(\mathrm{\_a}\right)+\mathrm{\_a}=0\right\}\,\left\{\mathrm{\_a}=y\left(x\right)\,\mathrm{\_b}\left(\mathrm{\_a}\right)=\frac{\ⅆ}{\ⅆx}y\left(x\right)\right\}\,\left\{x=\int \frac{1}{\mathrm{\_b}\left(\mathrm{\_a}\right)}\ⅆ\mathrm{\_a}+\mathrm{c\_\_1}\,y\left(x\right)=\mathrm{\_a}\right\}\right]$

For the structure of the solution above see ODESolStruc. Reductions of order can also be tested with odetest

odetest(ans,Van_der_Pol_ode);

$0$

The reduced ODE is of type Abel, and can be selected using either the mouse, or the following:

reduced_ode := op([2,2,1,1],ans);

$\mathrm{reduced\_ode}≔\left(\frac{\ⅆ}{\ⅆ\mathrm{\_a}}\mathrm{\_b}\left(\mathrm{\_a}\right)\right)\mathrm{\_b}\left(\mathrm{\_a}\right)+\mathrm{\_b}\left(\mathrm{\_a}\right){\mathrm{\_a}}^2\mathrm{\mu }-\mathrm{\mu }\mathrm{\_b}\left(\mathrm{\_a}\right)+\mathrm{\_a}=0$

odeadvisor(reduced_ode);

$\left[\mathrm{\_rational}\,\left[\mathrm{\_Abel}\,\mathrm{2nd\; type}\,\mathrm{class\; A}\right]\right]$