Fair Value of Financial Instruments (Details 3) (USD $)	3 Months Ended		9 Months Ended
	Sep. 30, 2011	Sep. 30, 2010	Sep. 30, 2011	Sep. 30, 2010	Dec. 31, 2009
Securities available-for-sale.
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	$ 93,075,000	$ 72,773,000	$ 83,097,000	$ 81,288,000
Total gains or losses (realized and unrealized):
Included in earnings	(611,000)	649,000	(356,000)	4,927,000
Included in other comprehensive (loss) income	(8,982,000)	5,279,000	(6,043,000)	6,957,000
Transfers in to Level 3		22,287,000		22,788,000
Transfers out of Level 3	(17,332,000)		(4,346,000)
Purchases	2,761,000		8,903,000
Sales			(1,665,000)
Settlements	(964,000)		(11,643,000)
Purchases, sales, other settlements and issuances, net		11,846,000		(3,126,000)
Balance at the end of the period	67,947,000	112,834,000	67,947,000	112,834,000
Residential mortgage-backed securities
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	88,538,000	107,081,000	93,929,000	47,572,000
Transfers in from deconsolidation				74,366,000
Total gains or losses (realized and unrealized):
Included in earnings	1,428,000	2,208,000	(373,000)	(5,076,000)
Settlements	(1,757,000)		(5,347,000)
Purchases, sales, other settlements and issuances, net		(3,037,000)		(10,610,000)
Balance at the end of the period	88,209,000	106,252,000	88,209,000	106,252,000
The amount of total gains or losses for the period included in earnings attributable to the change in unrealized gains or losses relating to assets still held at the reporting date	10,620,000	7,698,000	17,729,000	5,383,000
Real estate owned (recorded within other assets on the consolidated balance sheets)					11,400,000
Difference between the residential mortgage loans and RMBS Issued					62,900,000
Residential mortgage loans, at estimated fair value					2,100,000,000
RMBS Issued, estimated fair value					2,000,000,000
Equity investments, at estimated fair value
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	156,684,000	26,482,000	84,932,000	18,289,000
Total gains or losses (realized and unrealized):
Included in earnings	(14,417,000)	(5,835,000)	28,471,000	(8,732,000)
Transfers in to Level 3		41,740,000		41,740,000
Transfers out of Level 3	(28,546,000)		(28,546,000)
Purchases	20,252,000		40,344,000
Sales			(286,000)
Settlements			9,058,000
Purchases, sales, other settlements and issuances, net		6,545,000		17,635,000
Balance at the end of the period	133,973,000	68,932,000	133,973,000	68,932,000
The amount of total gains or losses for the period included in earnings attributable to the change in unrealized gains or losses relating to assets still held at the reporting date	(2,622,000)	(5,835,000)	33,952,000	(8,732,000)
Total rate of return swaps
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	163,000	37,000	104,000	11,809,000
Total gains or losses (realized and unrealized):
Included in earnings	(4,000)	98,000	55,000	1,801,000
Purchases, sales, other settlements and issuances, net				(13,475,000)
Balance at the end of the period	159,000	135,000	159,000	135,000
Common stock warrants
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	4,399,000	126,000	3,453,000	2,471,000
Total gains or losses (realized and unrealized):
Included in earnings	(37,000)	84,000	909,000	135,000
Settlements	(2,183,000)		(2,183,000)
Purchases, sales, other settlements and issuances, net		2,716,000		320,000
Balance at the end of the period	2,179,000	2,926,000	2,179,000	2,926,000
The amount of total gains or losses for the period included in earnings attributable to the change in unrealized gains or losses relating to assets still held at the reporting date	(536,000)	84,000	(536,000)	135,000
Foreign exchange options
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	18,952,000		14,791,000
Total gains or losses (realized and unrealized):
Included in earnings	(3,043,000)	15,309,000	1,118,000	15,309,000
Balance at the end of the period	15,909,000	15,309,000	15,909,000	15,309,000
The amount of total gains or losses for the period included in earnings attributable to the change in unrealized gains or losses relating to assets still held at the reporting date	(3,043,000)	15,309,000	1,118,000	15,309,000
Other Assets
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period	1,119,000
Total gains or losses (realized and unrealized):
Included in earnings	1,027,000		1,031,000
Purchases	400,000		400,000
Settlements	(840,000)		275,000
Balance at the end of the period	1,706,000		1,706,000
The amount of total gains or losses for the period included in earnings attributable to the change in unrealized gains or losses relating to assets still held at the reporting date	1,027,000		1,031,000
Interest rate swaps
Reconciliation of assets measured on Level 3 basis
Balance at the beginning of the period		4,764,000		(281,000)
Total gains or losses (realized and unrealized):
Included in earnings		(3,482,000)		1,348,000
Purchases, sales, other settlements and issuances, net		(10,000)		205,000
Balance at the end of the period		1,272,000		1,272,000
The amount of total gains or losses for the period included in earnings attributable to the change in unrealized gains or losses relating to assets still held at the reporting date		$ (3,482,000)		$ 151,000