The regs do not require ADS-B out installation in an aircraft that has never been certified with an engine-driven electrical system.
But my take is that if you have ADS-B installed, no matter where the electricity to run it comes from, you are required to have it on at all times. Best to read the entire reg for yourself:
https://www.law.cornell.edu/cfr/text/14/91.225"91.225 (f): Each person operating an aircraft equipped with ADS-B Out must operate this equipment in the transmit mode at all times."
"91.225(e):(e) The requirements of paragraph (b) of this section do not apply to any aircraft that was not originally certificated with an electrical system, or that has not subsequently been certified with such a system installed, including balloons and gliders. These aircraft may conduct operations without ADS-B Out in the airspace specified in paragraphs (d)(2) and (d)(4) of this section. Operations authorized by this section must be conducted -
(1) Outside any Class B or Class C airspace area; and
(2) Below the altitude of the ceiling of a Class B or Class C airspace area designated for an airport, or 10,000 feet MSL, whichever is lower."
FWIW here's the similar transponder exemption for non-electrical aircraft:
91.215(b)(3) Notwithstanding paragraph (b)(2) of this section, any aircraft which was not originally certificated with an engine-driven electrical system or which has not subsequently been certified with such a system installed, balloon or glider may conduct operations in the airspace within 30 nautical miles of an airport listed in appendix D, section 1 of this part provided such operations are conducted -
(i) Outside any Class A, Class B, or Class C airspace area; and
(ii) Below the altitude of the ceiling of a Class B or Class C airspace area designated for an airport or 10,000 feet MSL, whichever is lower"