Verified answer

For this, we need to assume that y > 0 for convergence.

Use integration by parts with u = x, dv = xe^(-2yx^2) dx

du = dx, v = (-1/(4y)) e^(-2yx^2).

So, ∫(-∞ to ∞) x^2 e^(-2yx^2) dx

= x * xe^(-2yx^2) {for x→ -∞ to ∞} - ∫(-∞ to ∞) (-1/(4y)) e^(-2yx^2) dx

= x^2 / e^(2yx^2) {for x→ -∞ to ∞} + (1/(4y)) ∫(-∞ to ∞) e^(-2yx^2) dx

= 0 + (1/(4y)) ∫(-∞ to ∞) e^(-2yx^2) dx, by L'Hopital's Rule

= (1/(4y)) ∫(-∞ to ∞) e^(-t^2) dt/√(2y), letting t = x√(2y)

= (1/(4√2 y^(3/2)) * ∫(-∞ to ∞) e^(-t^2) dt

= (1/(4√2 y^(3/2)) * √π, via gaussian integral

= (1/(4y^(3/2)) * √(π/2).

I hope this helps!