Expansion and contraction is taking place. Why would temperature change otherwise in adiabatic process? Consider the brick went to the right, now came back to equilibrium. Now it has same velocity V/2 and is moving towards left. To stop it, work is done by the gas inside the piston (basically pressure inside the piston is decreasing but outside pressure is constant, so the brick experiences force towards right.
What do you exactly want me to show mathematically?