Note 7 - Fair Value of Financial Instruments - Financial Assets Measured at on a Recurring Basis (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Mar. 31, 2018	Dec. 31, 2017	Jun. 30, 2017	Mar. 31, 2017	Dec. 31, 2016
Total, available for sale, fair value	$ 502,021		$ 464,430
SBA Servicing Assets	4,977	$ 5,059	5,243	$ 5,194	$ 5,298	$ 5,352
Collateralized Mortgage Obligations [Member]
Total, available for sale, fair value	346,942		320,241
Agency Mortgage-backed Securities [Member]
Total, available for sale, fair value	69,382		54,866
US States and Political Subdivisions Debt Securities [Member]
Total, available for sale, fair value	18,578		15,100
Corporate Debt Securities [Member]
Total, available for sale, fair value	59,614		60,282
Asset-backed Securities [Member]
Total, available for sale, fair value	6,958		13,452
Collateralized Debt Obligations [Member]
Total, available for sale, fair value	547		489
Fair Value, Measurements, Recurring [Member]
Total, available for sale, fair value	502,021		464,430
Mortgages Held-for-sale, Fair Value Disclosure	36,090		43,375
SBA Servicing Assets	4,977		5,243
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Mortgages Held-for-sale, Fair Value Disclosure
SBA Servicing Assets
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value	498,391		460,855
Mortgages Held-for-sale, Fair Value Disclosure	36,090		43,375
SBA Servicing Assets
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value	3,630		3,575
Mortgages Held-for-sale, Fair Value Disclosure
SBA Servicing Assets	4,977		5,243
Fair Value, Measurements, Recurring [Member] | Collateralized Mortgage Obligations [Member]
Total, available for sale, fair value	346,942		320,241
Fair Value, Measurements, Recurring [Member] | Collateralized Mortgage Obligations [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Collateralized Mortgage Obligations [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value	346,942		320,241
Fair Value, Measurements, Recurring [Member] | Collateralized Mortgage Obligations [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Agency Mortgage-backed Securities [Member]
Total, available for sale, fair value	69,382		54,866
Fair Value, Measurements, Recurring [Member] | Agency Mortgage-backed Securities [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Agency Mortgage-backed Securities [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value	69,382		54,866
Fair Value, Measurements, Recurring [Member] | Agency Mortgage-backed Securities [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | US States and Political Subdivisions Debt Securities [Member]
Total, available for sale, fair value	18,578		15,100
Fair Value, Measurements, Recurring [Member] | US States and Political Subdivisions Debt Securities [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | US States and Political Subdivisions Debt Securities [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value	18,578		15,100
Fair Value, Measurements, Recurring [Member] | US States and Political Subdivisions Debt Securities [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Corporate Debt Securities [Member]
Total, available for sale, fair value	59,614		60,282
Fair Value, Measurements, Recurring [Member] | Corporate Debt Securities [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Corporate Debt Securities [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value	56,531		57,196
Fair Value, Measurements, Recurring [Member] | Corporate Debt Securities [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value	3,083		3,086
Fair Value, Measurements, Recurring [Member] | Asset-backed Securities [Member]
Total, available for sale, fair value	6,958		13,452
Fair Value, Measurements, Recurring [Member] | Asset-backed Securities [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Asset-backed Securities [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value	6,958		13,452
Fair Value, Measurements, Recurring [Member] | Asset-backed Securities [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Collateralized Debt Obligations [Member]
Total, available for sale, fair value	547		489
Fair Value, Measurements, Recurring [Member] | Collateralized Debt Obligations [Member] | Fair Value, Inputs, Level 1 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Collateralized Debt Obligations [Member] | Fair Value, Inputs, Level 2 [Member]
Total, available for sale, fair value
Fair Value, Measurements, Recurring [Member] | Collateralized Debt Obligations [Member] | Fair Value, Inputs, Level 3 [Member]
Total, available for sale, fair value	547		489
Fair Value, Measurements, Recurring [Member] | Interest Rate Lock Commitments [Member]
Derivative Asset	714		363
Derivative Liability	3		1
Fair Value, Measurements, Recurring [Member] | Interest Rate Lock Commitments [Member] | Fair Value, Inputs, Level 1 [Member]
Derivative Asset
Derivative Liability
Fair Value, Measurements, Recurring [Member] | Interest Rate Lock Commitments [Member] | Fair Value, Inputs, Level 2 [Member]
Derivative Asset	714		363
Derivative Liability	3		1
Fair Value, Measurements, Recurring [Member] | Interest Rate Lock Commitments [Member] | Fair Value, Inputs, Level 3 [Member]
Derivative Asset
Derivative Liability
Fair Value, Measurements, Recurring [Member] | Best Efforts Forward Loan sales Commitments [Member]
Derivative Asset	1		5
Derivative Liability	210		93
Fair Value, Measurements, Recurring [Member] | Best Efforts Forward Loan sales Commitments [Member] | Fair Value, Inputs, Level 1 [Member]
Derivative Asset
Derivative Liability
Fair Value, Measurements, Recurring [Member] | Best Efforts Forward Loan sales Commitments [Member] | Fair Value, Inputs, Level 2 [Member]
Derivative Asset	1		5
Derivative Liability	210		93
Fair Value, Measurements, Recurring [Member] | Best Efforts Forward Loan sales Commitments [Member] | Fair Value, Inputs, Level 3 [Member]
Derivative Asset
Derivative Liability
Fair Value, Measurements, Recurring [Member] | Mandatory Forward Loan Sales Commitments [Member]
Derivative Asset	6		19
Derivative Liability	301		195
Fair Value, Measurements, Recurring [Member] | Mandatory Forward Loan Sales Commitments [Member] | Fair Value, Inputs, Level 1 [Member]
Derivative Asset
Derivative Liability
Fair Value, Measurements, Recurring [Member] | Mandatory Forward Loan Sales Commitments [Member] | Fair Value, Inputs, Level 2 [Member]
Derivative Asset	6		19
Derivative Liability	301		195
Fair Value, Measurements, Recurring [Member] | Mandatory Forward Loan Sales Commitments [Member] | Fair Value, Inputs, Level 3 [Member]
Derivative Asset
Derivative Liability