Note 7 - Fair Value of Financial Instruments - Financial Assets Measured at on a Recurring Basis (Details) - USD ($)	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2015	Jun. 30, 2015	Mar. 31, 2015	Dec. 31, 2014
Collateralized Mortgage Obligations [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	$ 152,866,000		$ 178,145,000
Collateralized Mortgage Obligations [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	152,866,000		178,145,000
Collateralized Mortgage Obligations [Member]
Fair Value	152,866,000		178,145,000
Collateralized Mortgage Backed Securities [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	9,419,000		10,171,000
Collateralized Mortgage Backed Securities [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	9,419,000		10,171,000
Collateralized Mortgage Backed Securities [Member]
Fair Value	9,419,000		10,171,000
US States and Political Subdivisions Debt Securities [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	25,272,000		23,344,000
US States and Political Subdivisions Debt Securities [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	25,272,000		23,344,000
US States and Political Subdivisions Debt Securities [Member]
Fair Value	25,272,000		23,344,000
Corporate Debt Securities [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	44,929,000		51,295,000
Corporate Debt Securities [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value	2,870,000		2,834,000
Corporate Debt Securities [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	47,799,000		54,129,000
Corporate Debt Securities [Member]
Fair Value	47,799,000		54,129,000
Asset-backed Securities [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	16,063,000		17,005,000
Asset-backed Securities [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	16,063,000		17,005,000
Asset-backed Securities [Member]
Fair Value	16,063,000		17,005,000
Collateralized Debt Obligations [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value	1,755,000		1,883,000
Collateralized Debt Obligations [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	1,755,000		1,883,000
Collateralized Debt Obligations [Member]
Fair Value	1,755,000		1,883,000
Other Debt Obligations [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	115,000		118,000
Other Debt Obligations [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value	115,000		118,000
Other Debt Obligations [Member]
Fair Value	115,000		118,000
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value	248,664,000		280,078,000
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value	4,625,000		4,717,000
SBA Servicing Assets	5,118,000		4,886,000
Fair Value, Measurements, Recurring [Member]
Fair Value	253,289,000		284,795,000
SBA Servicing Assets	5,118,000		4,886,000
Fair Value	253,289,000		284,795,000
SBA Servicing Assets	$ 5,118,000	$ 5,058,000	$ 4,886,000	$ 4,319,000	$ 4,267,000	$ 4,099,000