Boehner: Raising Tax Rates 'Unacceptable'

"I'll let all the political prognosticators figure out how the election went and why it went the way it did because Mitt Romney and Paul Ryan did a very nice job carrying our banner," he said. "But we lost. The other side did a much more effective job in getting their votes out to them, out to the polls. And as I'm fond of saying, 'Polls don't decide elections, voters do,' and more of their voters showed up than ours."

Asked whether he will make another attempt to fully repeal the Affordable Care Act, Boehner said "the election changes that" and "Obamacare is the law of the land."

Still, there are some parts of the law, he said, that should be on the table as lawmakers work toward a balanced budget.

The speaker also revealed that comprehensive, bipartisan immigration overhaul would be a top priority of his agenda during the 113th Congress.

John Boehner: 'Obamacare is the Law of the Land' Watch Video

Boehner: 'We Don't Have a Tea Party Caucus' Watch Video

John Boehner: There's a Mandate to Work Together Watch Video

"This issue has been around far too long," he said. "A comprehensive approach is long overdue, and I'm confident that the president, myself, others can find the common ground to take care of this issue once and for all."

The speaker also downplayed the influence of the Tea Party on his Congress, even though at least 49 members of the Tea Party caucus won re-election.

"This has been the most misreported story of my two years' tenure. We don't have a Tea Party caucus to speak of in the House," Boehner said. "All of us who were elected in 2010 were supported by the Tea Party.

"These are ordinary Americans who've taken a more active role in their government. They want solutions, but we've all come a long way over the last two years. I think we all understand each other a lot better."

With minorities and women comprising of a majority within the House Democratic Caucus during the next session, Sawyer asked Boehner whether the Republican Party is too white, too old and too male. The speaker acknowledged that the GOP has work ahead to appeal to other demographics.

"What Republicans need to learn is how do we speak to all Americans. You know, not just the people who look like us and act like us, but how do we speak to all Americans," Boehner said.

"Listen, we believe in the American dream. We believe in individual freedom, and we believe in empowering all citizens. I think there's a message there that resonates with all Americans, but we need to do a much more effective job in communicating it."