People who were Christian may have been some of the earliest white settlers (if you want to go back before the founding of the country let's go all the way back to the Native Americans and other First Peoples, who were NOT Christian), but the Founding Fathers and early presidents specifically stated this was not founded as a Christian nation.
There are many sites out there discussing the subject and many writings of Jefferson, Franklin and some of the other men of influence, though not lawmakers, like Paine. I would point to our founding documents, though, as the clearest declaration. The Constitution of the United States has NO mention of God, Creator, or religion except to state "no religious Test shall ever be required as a Qualification to any Office or public Trust under the United States".
The Declaration is a little more religious with three references. The first, "the Laws of Nature and of Nature's God" (which if I were being pedantic I could say really sounds much more Pagan than Christian, but I know that's specious as none of the signers of the Declaration were what anyone modern would consider Pagan). Then there's "endowed by their Creator", which again doesn't specify Who that Creator is. Third is "with a firm reliance on the protection of divine Providence", which one could argue is more referential to a Christian philosophy. However, the Declaration was not a statement of how we are to be governed.
The second president of the United States, John Adams - someone who should know, signed the Treaty of Tripoli that states "As the Government of the United States of America is not, in any sense, founded on the Christian religion,—as it has in itself no character of enmity against the laws, religion, or tranquility, of Mussulmen [Muslims],—and as the said States never entered into any war or act of hostility against any Mahometan [Mohammedan] nation, it is declared by the parties that no pretext arising from religious opinions shall ever produce an interruption of the harmony existing between the two countries."
The Bill of Rights has only one reference to religion, and it is "Amendment I: Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof".
None of this gives ANY indication that our country was founded as a religious country in any way. While we may have had a heavy Puritan influence from some of our earliest settlers, don't forget that many of our other (perhaps unwilling) early settlers were prisoners and criminals. We may (for good or ill, and that's a whole other debate) have been heavily influenced by Judeo-Christian ethics and culture, but that's a different matter.
As to Cuomo's comments, I don't know if I'd go as far as calling it hate speech, but I do think it's intolerant. I will say that the extreme conservatives do want to remove the rights of those with whom they don't agree, whereas more liberal folks want everyone to be able to make their own choices according to their own beliefs. But that's my own opinion.