Loans and Allowance for Loan Losses - Additional Charge-offs of Problem Loans (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended	12 Months Ended
	Dec. 31, 2012	Dec. 31, 2013	Dec. 31, 2012
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	$ 10,999
Provision	11,000	(2,685)	40,374
Commercial
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	3,908
Provision		1,078	5,419
Consumer and other
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	24
Provision		86	445
Construction Loans | Real Estate Loans
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	1,601
Provision		(3)	12,742
Owner Occupied Commercial | Real Estate Loans
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	1,599
Provision		(1,022)	5,751
Non-owner Occupied | Real Estate Loans
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	2,267
Provision		(1,764)	8,433
One to Four Family | Real Estate Loans
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	291
Provision		(205)	2,762
Multifamily | Real Estate Loans
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	0
Provision		(158)	136
Home Equity Lines of Credit | Real Estate Loans
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans [Line Items]
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans	1,309
Provision		$ (697)	$ 4,686