Loans Held for Investment (Details 6) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2012	Dec. 31, 2011	Sep. 30, 2011
Other information concerning the credit quality
Current	$ 857,955	$ 733,564	$ 725,118
30-59 Days Past Due	2,565	699	3,152
60-89 Days Past Due	164	731	937
90+ Days Past Due	4,154	4,260	6,669
Total	864,838	739,254	735,876
Non-Accruing	6,280	6,093	9,357
Real estate loans: Multi-family
Other information concerning the credit quality
Current	173,484	193,830	211,514
Total	173,484	193,830	211,514
Non-Accruing	273	293	300
Real estate loans: Commercial non-owner occupied
Other information concerning the credit quality
Current	256,489	162,663	163,110
30-59 Days Past Due	1,708	434	214
60-89 Days Past Due			418
90+ Days Past Due	3,849	1,244	1,055
Total	262,046	164,341	164,797
Non-Accruing	4,528	1,495	2,457
Real estate loans: One-to-four family
Other information concerning the credit quality
Current	62,307	59,503	60,685
30-59 Days Past Due	301	201	615
60-89 Days Past Due	43		320
90+ Days Past Due	120	323	1,018
Total	62,771	60,027	62,638
Non-Accruing	578	323	1,018
Real estate loans: Construction
Other information concerning the credit quality
Current	308
Total	308
Real estate loans: Land
Other information concerning the credit quality
Current	10,776	5,769	8,290
30-59 Days Past Due	229		8
60-89 Days Past Due		617
90+ Days Past Due		52	198
Total	11,005	6,438	8,496
Non-Accruing	417	52	198
Business loans: Commercial owner occupied
Other information concerning the credit quality
Current	147,812	151,380	161,998
30-59 Days Past Due	327		417
90+ Days Past Due		919	1,853
Total	148,139	152,299	164,268
Non-Accruing	218	2,053	2,760
Business loans: Commercial and industrial
Other information concerning the credit quality
Current	87,984	85,615	70,395
30-59 Days Past Due		12	1,898
60-89 Days Past Due	121		158
90+ Days Past Due		1,057	1,983
Total	88,105	86,684	74,434
Non-Accruing		1,177	1,983
Business loans: Warehouse facilities
Other information concerning the credit quality
Current	112,053	67,518	42,644
Total	112,053	67,518	42,644
Business loans: SBA
Other information concerning the credit quality
Current	4,551	3,900	4,267
30-59 Days Past Due		49
60-89 Days Past Due		113	41
90+ Days Past Due	185	665	562
Total	4,736	4,727	4,870
Non-Accruing	266	700	641
Other loans
Other information concerning the credit quality
Current	2,191	3,386	2,215
30-59 Days Past Due		3
60-89 Days Past Due		1
Total	$ 2,191	$ 3,390	$ 2,215