SUMMARY OF SIGNIFICANT ACCOUNTING POLICIES (Details) (USD $)	12 Months Ended
	Dec. 31, 2013 Loan	Dec. 31, 2012 Loan	Dec. 31, 2011
Noncontrolling Interest [Line Items]
Payment to acquire additional ownership interest	$ 1,550,000
Amount credit to Additional Paid in Capital	1,550,000
ASSETS
Total cash and cash equivalents	16,844,000	28,802,000	24,506,000
Investment securities AFS, at fair value	308,727,000	349,203,000
Other investment, at cost	2,250,000	2,250,000
FHLB stock, at cost	4,204,000	6,011,000
Loans and leases held for sale	1,446,000	1,572,000
Loans and leases, net	352,810,000	326,904,000
Bank owned life insurance	15,124,000	14,585,000
Accrued interest receivable	7,054,000	10,256,000
OREO, net	9,617,000	13,435,000
Premises and equipment, net	4,475,000	5,232,000
Other assets	9,703,000	11,205,000
Total assets	732,254,000	769,455,000
Liabilities
Total deposits	528,964,000	554,917,000
Long-term borrowings	97,881,000	108,333,000
Subordinated debentures	25,774,000	25,774,000
Accrued interest payable	965,000	3,760,000
Other liabilities	20,865,000	23,103,000
Total liabilities	684,449,000	715,887,000
Shareholders' equity
Preferred stock	29,950,000	29,396,000	28,878,000
Common stock		23,065,000
Additional paid in capital	127,299,000	126,287,000	126,245,000
Accumulated deficit	(120,396,000)	(121,877,000)	(105,600,000)
Accumulated other comprehensive loss	(6,122,000)	(142,000)	800,000
Treasury stock	(6,336,000)	(6,971,000)	(6,971,000)
Noncontrolling interest	271,000	3,810,000	4,815,000
Total shareholders' equity	47,805,000	53,568,000	71,098,000
Total liabilities and shareholders' equity	732,254,000	769,455,000
Other investments, notice period to redeem funds	60 days
Federal Home Loan Bank Stock [Abstract]
Federal Home Loan Bank stock	4,204,000	6,011,000
Loans Held-for-sale [Abstract]
Number of loans held for sale	3	1
Non-accrual loans held for sale	1,400,000	1,600,000
Gain on transfer of loans from LHFI to LHFS recorded in non-interest income	429,000
Allowance For Loan and Lease Losses [Abstract]
Rolling average period used to determine historical loss rates	3 years
Advertising Cost [Abstract]
Advertising costs	209,000	127,000
Benefit Plans [Abstract]
Highest consecutive years of employee compensation used to compute benefit	3 years
Years of employment used in benefit computation	10 years
Employer matching contribution to employee's contributions between 1% and 5% (in hundredths)	100.00%
Employee contribution matched by employer, lower range (in hundredths)	1.00%
Employer matching contribution to employees contribution that are within the specified limit (in hundredths)	5.00%
Maximum annual contribution per employee	2,500
Employer contribution to plan	0	0
Stock Compensation [Abstract]
Compensation expense	22,000	42,000
Income Taxes [Abstract]
Unrecognized tax benefits		0	0
Accrued interest and penalties on unrecognized tax benefits		0	0
Net income (loss) per common share [Abstract]
Rate class B shares may be converted to class A shares	1.15 to 1
Restrictions on Cash and Amounts Due From Banks [Abstract]
Average balances on hand with the Federal Reserve Bank	100,000	100,000
Restatement Adjustment [Member]
ASSETS
Total cash and cash equivalents		0
Investment securities AFS, at fair value		0
Other investment, at cost		0
FHLB stock, at cost		0
Loans and leases held for sale		0
Loans and leases, net		0
Bank owned life insurance		0
Accrued interest receivable		0
OREO, net		0
Premises and equipment, net		0
Other assets		(4,261,000)
Total assets		(4,261,000)
Liabilities
Total deposits		0
Long-term borrowings		0
Subordinated debentures		0
Accrued interest payable		0
Other liabilities		586,000
Total liabilities		586,000
Shareholders' equity
Preferred stock		0	0
Common stock		0
Additional paid in capital		0	0
Accumulated deficit		(4,797,000)	(4,797,000)
Accumulated other comprehensive loss		0	0
Treasury stock		0	0
Noncontrolling interest		(50,000)	(50,000)
Total shareholders' equity		(4,847,000)	(4,847,000)
Total liabilities and shareholders' equity		(4,261,000)
Federal Home Loan Bank Stock [Abstract]
Federal Home Loan Bank stock		0
Scenario, Previously Reported [Member]
ASSETS
Total cash and cash equivalents		28,802,000
Investment securities AFS, at fair value		349,203,000
Other investment, at cost		2,250,000
FHLB stock, at cost		6,011,000
Loans and leases held for sale		1,572,000
Loans and leases, net		326,904,000
Bank owned life insurance		14,585,000
Accrued interest receivable		10,256,000
OREO, net		13,435,000
Premises and equipment, net		5,232,000
Other assets		15,466,000
Total assets		773,716,000
Liabilities
Total deposits		554,917,000
Long-term borrowings		108,333,000
Subordinated debentures		25,774,000
Accrued interest payable		3,760,000
Other liabilities		22,517,000
Total liabilities		715,301,000
Shareholders' equity
Preferred stock		29,396,000	28,878,000
Common stock		23,065,000
Additional paid in capital		126,287,000	126,245,000
Accumulated deficit		(117,080,000)	(100,803,000)
Accumulated other comprehensive loss		(142,000)	800,000
Treasury stock		(6,971,000)	(6,971,000)
Noncontrolling interest		3,860,000	4,865,000
Total shareholders' equity		58,415,000	75,945,000
Total liabilities and shareholders' equity		773,716,000
Federal Home Loan Bank Stock [Abstract]
Federal Home Loan Bank stock		6,011,000
Common Class B [Member]
Shareholders' equity
Common stock	199,000	202,000	208,000
Total shareholders' equity	199,000	202,000	208,000
Common Class B [Member] | Restatement Adjustment [Member]
Shareholders' equity
Common stock			0
Common Class B [Member] | Scenario, Previously Reported [Member]
Shareholders' equity
Common stock			208,000
Common Class A [Member]
Shareholders' equity
Common stock	22,940,000	22,863,000	22,723,000
Total shareholders' equity	22,940,000	22,863,000	22,723,000
Common Class A [Member] | Restatement Adjustment [Member]
Shareholders' equity
Common stock			0
Common Class A [Member] | Scenario, Previously Reported [Member]
Shareholders' equity
Common stock			22,723,000
Commercial Real Estate Loans [Member]
Concentration Risk [Line Items]
Concentration risk, percentage (in hundredths)	40.00%
Commercial [Member]
Concentration Risk [Line Items]
Concentration risk, percentage (in hundredths)	22.00%
Construction and Development Loans [Member]
Concentration Risk [Line Items]
Concentration risk, percentage (in hundredths)	12.00%
Securities Issued By Government Sponsored Entities [Member]
Concentration Risk [Line Items]
Concentration risk, percentage (in hundredths)	92.00%
Crusader Servicing Corporation [Member]
Noncontrolling Interest [Line Items]
Percentage of ownership interest in subsidiary (in hundredths)		60.00%
Payment settlement related to relinquished ownership interest	1,250,000
Percentage of relinquished ownership interest (in hundredths)	20.00%
Total percentage of ownership interest (in hundredths)	80.00%
Royal Tax Lien Services, LLC [Member]
Noncontrolling Interest [Line Items]
Percentage of ownership interest in subsidiary (in hundredths)		60.00%
Percentage of additional ownership interest acquired (in hundredths)	40.00%
Payment to acquire additional ownership interest	850,000
Payment for tax distribution related additional ownership interest	400,000
Amount credit to Additional Paid in Capital	850,000
Total percentage of ownership interest (in hundredths)	100.00%
Royal Bank America Leasing, LP [Member]
Noncontrolling Interest [Line Items]
Percentage of ownership interest in subsidiary (in hundredths)	60.00%
Additional Paid In Capital [Member]
Noncontrolling Interest [Line Items]
Payment to acquire additional ownership interest	1,500,000
Amount credit to Additional Paid in Capital	1,500,000
Royal Bancshares Capital Trust I and II [Member]
Trust Preferred Securities [Abstract]
Aggregate principal balance	774,000
Crusader Servicing Corporation and Royal Tax Lien Services, LLC [Member]
Noncontrolling Interest [Line Items]
Combined value of ownership interest	$ 2,600,000