Loans Held for Investment (Details) (USD $)							9 Months Ended
	Sep. 30, 2012	Dec. 31, 2011	Sep. 30, 2011	Dec. 31, 2010	Sep. 30, 2012 Canyon National	Sep. 30, 2012 Palm Desert National	Sep. 30, 2012 Maximum	Sep. 30, 2012 Real estate loans: Multi-family	Dec. 31, 2011 Real estate loans: Multi-family	Sep. 30, 2011 Real estate loans: Multi-family	Dec. 31, 2010 Real estate loans: Multi-family	Sep. 30, 2012 Real estate loans: Commercial non-owner occupied	Dec. 31, 2011 Real estate loans: Commercial non-owner occupied	Sep. 30, 2011 Real estate loans: Commercial non-owner occupied	Dec. 31, 2010 Real estate loans: Commercial non-owner occupied	Sep. 30, 2012 Real estate loans: One-to-four family	Dec. 31, 2011 Real estate loans: One-to-four family	Sep. 30, 2011 Real estate loans: One-to-four family	Dec. 31, 2010 Real estate loans: One-to-four family	Sep. 30, 2012 Real estate loans: Construction	Sep. 30, 2012 Real estate loans: Land	Dec. 31, 2011 Real estate loans: Land	Sep. 30, 2011 Real estate loans: Land	Sep. 30, 2012 Business loans: Commercial owner occupied	Dec. 31, 2011 Business loans: Commercial owner occupied	Sep. 30, 2011 Business loans: Commercial owner occupied	Dec. 31, 2010 Business loans: Commercial owner occupied	Sep. 30, 2012 Business loans: Commercial and industrial	Dec. 31, 2011 Business loans: Commercial and industrial	Sep. 30, 2011 Business loans: Commercial and industrial	Dec. 31, 2010 Business loans: Commercial and industrial	Sep. 30, 2012 Business loans: Warehouse facilities	Dec. 31, 2011 Business loans: Warehouse facilities	Sep. 30, 2011 Business loans: Warehouse facilities	Sep. 30, 2012 Business loans: SBA	Dec. 31, 2011 Business loans: SBA	Sep. 30, 2011 Business loans: SBA	Dec. 31, 2010 Business loans: SBA	Sep. 30, 2012 Other loans	Dec. 31, 2011 Other loans	Sep. 30, 2011 Other loans	Dec. 31, 2010 Other loans
Loans Held for Investment
Total gross loans	$ 864,838,000	$ 739,254,000	$ 735,876,000					$ 173,484,000	$ 193,830,000	$ 211,514,000		$ 262,046,000	$ 164,341,000	$ 164,797,000		$ 62,771,000	$ 60,027,000	$ 62,638,000		$ 308,000	$ 11,005,000	$ 6,438,000	$ 8,496,000	$ 148,139,000	$ 152,299,000	$ 164,268,000		$ 88,105,000	$ 86,684,000	$ 74,434,000		$ 112,053,000	$ 67,518,000	$ 42,644,000	$ 4,736,000	$ 4,727,000	$ 4,870,000		$ 2,191,000	$ 3,390,000	$ 2,215,000
Less loans held for sale, net	4,728,000
Total gross loans held for investment	860,110,000	739,254,000	735,876,000
Less: Deferred loan origination costs (fees) and premiums (discounts), net	(737,000)	(665,000)	(1,402,000)
Less: Allowance for loan losses	(7,658,000)	(8,522,000)	(8,522,000)	(8,879,000)				(2,097,000)	(2,281,000)	(3,113,000)	(2,730,000)	(2,036,000)	(1,287,000)	(1,418,000)	(1,580,000)	(418,000)	(931,000)	(671,000)	(332,000)			(39,000)		(1,151,000)	(1,119,000)	(1,211,000)	(1,687,000)	(824,000)	(1,361,000)	(1,258,000)	(2,356,000)	(1,046,000)	(1,347,000)	(721,000)	(66,000)	(80,000)	(97,000)	(145,000)	(20,000)	(77,000)	(33,000)	(49,000)
Loans held for investment, net	851,715,000	730,067,000	725,952,000
Secured loans limit to one borrower (as a percent)							25.00%
Unsecured loans limit to one borrower (as a percent)							15.00%
Secured loans limit to one borrower	27,000,000
Unsecured loans limit to one borrower	16,200,000
Aggregate outstanding balance of loans to one borrower of secured credit	21,400,000
Mark-to-market discount					$ 3,100,000	$ 6,300,000