The Nyquist formula quantifies the thermal noise driven fluctuations of voltage across a resistance in equilibrium. We deal here with the case of a resistance driven out of equilibrium by putting it in contact with two thermostats at different temperatures. We reach a non-equilibrium steady state where a heat flux is flowing through the resistance. Our measurements demonstrate anyway that a simple extension of the Nyquist formula to the non uniform temperature field describes with an excellent precision the thermal noise. For a metallic ohmic material, the fluctuations are actually equivalent to those of a resistance in equilibrium with a single thermostat at the mean temperature between the hot and cold sources.