|Nigeria has tightened laws against homosexuals,|
including a ban on shows of same-sex affection
A 12th person arrested - a Christian - would be tried under secular law, a BBC reporter says.
Under Islamic law, a person can be sentenced to death by stoning if convicted of homosexuality.
It is also illegal to have gay sex in Nigeria, according to its secular laws.
Earlier this month, President Goodluck Jonathan signed a parliamentary bill which tightens laws against homosexuals, banning same-sex marriages, gay groups and shows of same-sex public affection.
The new legislation applies across Nigeria, affecting all citizens.
Most states in the predominantly Muslim north of Nigeria have adopted Islamic law, known as Sharia, since the end of military rule in 1999.
Nigeria is a deeply conservative country, with an influential Christian evangelical movement in the south and strong support for Islamic law among many Muslims in the north.
Hostility towards gay people has escalated since parliament debated the Same-Sex Marriage Prohibition Act last year, Dorothy Aken'Ova, a rights activist with the Nigeria-based International Centre for Reproductive Health and Sexual Rights, told the BBC.
She said that she was aware of 38 people being arrested in Bauchi state last month, and was trying to confirm reports of more arrests in both the north and south of Nigeria since the new law was approved.
"What this act is saying is that they [gay people] do not deserve to exist," she told the BBC's Focus on Africa radio programme.
"It is heartbreaking that we have come to this point in Nigeria."
Jibrin Danlami Hassan, the commissioner of Bauchi state's Sharia Commission, said the alleged homosexuals were arrested by residents of Bauchi city.
They were handed to the Islamic police force, which interrogated them, he said.
"They accept that they are doing that dirty game," Mr Hassan said.
Ms Aken'Ova said some of those arrested had been beaten up and tortured, but Mr Hassan denied this.
The BBC's Ishaq Khalid in Bauchi says the Sharia Commission confirmed to him that a Christian had been arrested with the 11 Muslims, and would be tried in a "conventional court".
The Muslims could be sentenced to death by stoning if they are convicted, but the court would decide, Mr Hassan said.
Several stoning sentences have been handed down by Sharia courts in northern Nigeria since 1999; however, none have so far been carried out.
The UK, US and UN human rights chief Navi Pillay have condemned Nigeria's Same-Sex Marriage Prohibition Act as discriminatory and draconian.
Mr Hassan said told the BBC he was "happy" that Mr Jonathan had signed it into law, despite threats by Western powers to cut aid to Nigeria.
"The threat they are doing cannot make us change our religion," he said.
Map showing gay rights in Africa