Fair Value Measurements - Assets and Liabilities Measured at Fair Value on Recurring Basis (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2014		Dec. 31, 2013
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	$ 243,032		$ 280,738
Loans held for sale	680,147		306,915
Mortgage servicing rights	237,221		241,018
Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	89,134		96,221
Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	153,898		184,517
GSE certificates | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	20,692	[1]	25,200
GSE CMOs | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	59,075	[2]	60,819
Private label CMOs | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	9,367		10,202
Municipal bonds | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	1,097		1,026
Capital trust notes | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	11,609		11,798
Preferred stock | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	122,686		116,239
Common stock | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	18,506		55,454
Fair Value, Measurements, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	243,032		280,738
Loans held for sale	680,147		306,915
Mortgage servicing rights	237,221		241,018
Interest rate lock commitments	2,180		258
Derivative assets-other	1,851	[3]	1,574	[4]
Derivative liabilities	(90)		(388)
Fair Value, Measurements, Recurring | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	89,134		96,221
Fair Value, Measurements, Recurring | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	153,898		184,517
Fair Value, Measurements, Recurring | Quoted Prices in Active Markets for Identical Assets (Level 1)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	111,254		142,682
Derivative assets-other	3,184	[3]	1,267	[4]
Derivative liabilities	(1,623)		(590)
Fair Value, Measurements, Recurring | Quoted Prices in Active Markets for Identical Assets (Level 1) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	111,254		142,682
Fair Value, Measurements, Recurring | Significant Other Observable Inputs (Level 2)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	131,778		138,056
Loans held for sale	680,147		306,915
Derivative assets-other	1,269	[3]	5,155	[4]
Derivative liabilities	(1,889)		(7,422)
Fair Value, Measurements, Recurring | Significant Other Observable Inputs (Level 2) | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	89,134		96,221
Fair Value, Measurements, Recurring | Significant Other Observable Inputs (Level 2) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	42,644		41,835
Fair Value, Measurements, Recurring | Significant Unobservable Inputs (Level 3)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights	237,221		241,018
Interest rate lock commitments	2,180		258
Fair Value, Measurements, Recurring | Netting Adjustment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative assets-other	(2,602)	[3],[5]	(4,848)	[4],[5]
Derivative liabilities	3,422	[5]	7,624	[5]
Fair Value, Measurements, Recurring | GSE certificates | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	20,692		25,200
Fair Value, Measurements, Recurring | GSE certificates | Significant Other Observable Inputs (Level 2) | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	20,692		25,200
Fair Value, Measurements, Recurring | GSE CMOs | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	59,075		60,819
Fair Value, Measurements, Recurring | GSE CMOs | Significant Other Observable Inputs (Level 2) | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	59,075		60,819
Fair Value, Measurements, Recurring | Private label CMOs | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	9,367		10,202
Fair Value, Measurements, Recurring | Private label CMOs | Significant Other Observable Inputs (Level 2) | Mortgage-Related Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	9,367		10,202
Fair Value, Measurements, Recurring | Municipal bonds | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	1,097		1,026
Fair Value, Measurements, Recurring | Municipal bonds | Significant Other Observable Inputs (Level 2) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	1,097		1,026
Fair Value, Measurements, Recurring | Capital trust notes | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	11,609		11,798
Fair Value, Measurements, Recurring | Capital trust notes | Significant Other Observable Inputs (Level 2) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	11,609		11,798
Fair Value, Measurements, Recurring | Preferred stock | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	122,686		116,239
Fair Value, Measurements, Recurring | Preferred stock | Quoted Prices in Active Markets for Identical Assets (Level 1) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	94,406		89,942
Fair Value, Measurements, Recurring | Preferred stock | Significant Other Observable Inputs (Level 2) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	28,280		26,297
Fair Value, Measurements, Recurring | Common stock | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	18,506		55,454
Fair Value, Measurements, Recurring | Common stock | Quoted Prices in Active Markets for Identical Assets (Level 1) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	16,848		52,740
Fair Value, Measurements, Recurring | Common stock | Significant Other Observable Inputs (Level 2) | Other Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available for sale	$ 1,658		$ 2,714

[1] Government-sponsored enterprise

[2] Collateralized mortgage obligations

[3] Includes $3.2 million to purchase Treasury options.

[4] Includes $1.3 million to purchase Treasury options.

[5] Includes cash collateral received from, and paid to, counterparties.