Covered Assets and FDIC Indemnification Asset (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012
Covered loans [Line Items]
Covered loans	$ 151,955	$ 217,339
Subtotal	167,692	264,515
Fair value discount	(15,737)	(47,176)
Allowance for loan losses	(14,622)	(3,252)
Total covered loans, net	137,333	214,087
Other Real Estate, Covered [Roll Forward]
Balance, beginning of period	13,460	26,622
Additions to covered OREO	12,513	9,716
Dispositions of covered OREO, net	(17,960)	(19,174)
Valuation adjustments	(2,033)	(3,704)
Balance, end of period	5,980	13,460
FDIC Indemnification Asset [Roll Forward]
Balance, beginning of period	34,571	65,586
Change in FDIC indemnification asset	5,735	(9,126)
Reduction due to loans paid in full	(9,325)	(5,584)
Transfers to (due from) FDIC	(16,445)	(16,305)
Balance, end of period	14,536	34,571
City Bank [Member]
Covered loans [Line Items]
Covered loans	90,732	140,777
Subtotal	97,904	169,757
Fair value discount	(7,172)	(28,980)
Allowance for loan losses	(13,026)	(2,727)
Total covered loans, net	77,706	138,050
Certain Loans Acquired in Transfer Accounted for as Debt Securities, Accretable Yield Movement Schedule [Roll Forward]
Balance at beginning of period	49,168	78,004
Accretion to interest income	(13,315)	(21,282)
Disposals	(13,985)	(8,866)
Reclassifications (to) from nonaccretable difference	(3,364)	1,312
Balance at end of period	18,504	49,168
Other Real Estate, Covered [Roll Forward]
Balance, beginning of period	7,399	19,341
Additions to covered OREO	10,065	2,515
Dispositions of covered OREO, net	(12,321)	(11,991)
Valuation adjustments	(1,145)	(2,466)
Balance, end of period	3,998	7,399
FDIC Indemnification Asset [Roll Forward]
Balance, beginning of period	20,390	43,235
Change in FDIC indemnification asset	5,331	(8,704)
Reduction due to loans paid in full	(7,179)	(2,987)
Transfers to (due from) FDIC	(7,674)	(11,154)
Balance, end of period	10,868	20,390
North County Bank [Member]
Covered loans [Line Items]
Covered loans	61,223	76,562
Subtotal	69,788	94,758
Fair value discount	(8,565)	(18,196)
Allowance for loan losses	(1,596)	(525)
Total covered loans, net	59,627	76,037
Certain Loans Acquired in Transfer Accounted for as Debt Securities, Accretable Yield Movement Schedule [Roll Forward]
Balance at beginning of period	19,567	29,574
Accretion to interest income	(10,133)	(14,905)
Disposals	(3,487)	(6,546)
Reclassifications (to) from nonaccretable difference	6,937	11,444
Balance at end of period	12,884	19,567
Other Real Estate, Covered [Roll Forward]
Balance, beginning of period	6,061	7,281
Additions to covered OREO	2,448	7,201
Dispositions of covered OREO, net	(5,639)	(7,183)
Valuation adjustments	(888)	(1,238)
Balance, end of period	1,982	6,061
FDIC Indemnification Asset [Roll Forward]
Balance, beginning of period	14,181	22,351
Change in FDIC indemnification asset	404	(422)
Reduction due to loans paid in full	(2,146)	(2,597)
Transfers to (due from) FDIC	(8,771)	(5,151)
Balance, end of period	3,668	14,181
Commercial [Member]
Covered loans [Line Items]
Covered loans	17,285	29,011
Commercial [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	8,670	13,863
Commercial [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	8,615	15,148
Real Estate Mortgage [Member]
Covered loans [Line Items]
Covered loans	133,041	198,778
Real Estate Mortgage [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	81,968	136,063
Real Estate Mortgage [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	51,073	62,715
Real Estate Mortgage [Member] | One to Four Family Residential [Member]
Covered loans [Line Items]
Covered loans	10,969	14,195
Real Estate Mortgage [Member] | One to Four Family Residential [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	2,630	3,783
Real Estate Mortgage [Member] | One to Four Family Residential [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	8,339	10,412
Real Estate Mortgage [Member] | Multi Family and Commercial [Member]
Covered loans [Line Items]
Covered loans	122,072	184,583
Real Estate Mortgage [Member] | Multi Family and Commercial [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	79,338	132,280
Real Estate Mortgage [Member] | Multi Family and Commercial [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	42,734	52,303
Real Estate Construction [Member]
Covered loans [Line Items]
Covered loans	10,570	26,507
Real Estate Construction [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	5,223	17,133
Real Estate Construction [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	5,347	9,374
Real Estate Construction [Member] | One to Four Family Residential [Member]
Covered loans [Line Items]
Covered loans	1,327	7,764
Real Estate Construction [Member] | One to Four Family Residential [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	0	4,764
Real Estate Construction [Member] | One to Four Family Residential [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	1,327	3,000
Real Estate Construction [Member] | Multi Family and Commercial [Member]
Covered loans [Line Items]
Covered loans	9,243	18,743
Real Estate Construction [Member] | Multi Family and Commercial [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	5,223	12,369
Real Estate Construction [Member] | Multi Family and Commercial [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	4,020	6,374
Consumer [Member] | Direct Consumer Loan [Member]
Covered loans [Line Items]
Covered loans	6,796	10,219
Consumer [Member] | Direct Consumer Loan [Member] | City Bank [Member]
Covered loans [Line Items]
Covered loans	2,043	2,698
Consumer [Member] | Direct Consumer Loan [Member] | North County Bank [Member]
Covered loans [Line Items]
Covered loans	$ 4,753	$ 7,521