ALBANY, N.Y.— New York Gov. David Paterson, defying calls from even fellow Democrats to drop out of the race for a full term, said Tuesday that he would leave only if the voters turned him out through the ballot box, or he's carried out "in a box."

Paterson spoke to reporters after several days of rumors sweeping the state Capitol about carousing in the governor's mansion, all of which Paterson strongly denied.

A few months after Paterson took over from his predecessor, who resigned in a prostitution scandal, his popularity plummeted and many Democrats voiced their preference that Attorney General Andrew Cuomo run for governor when Paterson's term is up.

That infighting and the recent rumormongering have further fractured state Democrats and added a decidedly weird edge to the national party's struggle to maintain ground it gained in the last election.

Facing challenges from coast to coast amid voter frustration with incumbents of both parties, Democrats in particular are girding for losses that could see states' chief executives go into Republican hands. New York, a traditionally Democratic-leaning state, would be a devastating blow to a party that just two years ago saw a landslide amid the election of President Barack Obama.

Paterson appeared to take heart from an interview earlier in the day with The New York Times, which had been widely anticipated to be preparing a story dealing with his personal conduct. He said that in the interview he was not asked about drug use in the governor's mansion or partying with women.

"The only way I'm not going to be governor next year is at the ballot box, and the only way that I will be leaving the office before is in a box," Paterson said during a news conference.

Paterson had decried the allegations Monday in an Associated Press interview.

He has been trying to end one of the weirdest travails yet in a weird place, a political "Twilight Zone" he often refers to as Planet Albany. This time, media coverage alternately derided as a feeding frenzy and a circus, involved one of the world's most influential newspapers and became a scandal about a scandal that was yet to be written and which, according to Paterson, won't be.

"Obviously we are not responsible for what other news organizations are reporting," said Times spokeswoman Diane C. McNulty. "It's not coming from The Times."