Parent Company Only Condensed Financial Information - Balance Sheets (Details) - USD ($) $ in Thousands	Dec. 31, 2015		Dec. 31, 2014	Dec. 31, 2013
ASSETS
Cash and cash equivalents	$ 27,655		$ 49,167	$ 70,748
Securities available-for-sale	171,440		135,371
Other assets	9,098		9,719
Total assets	961,218		817,089
LIABILITIES AND STOCKHOLDERS' EQUITY
Notes payable	0		10,250
Preferred stock	268		268
Other liabilities	5,815		10,108
Total liabilities	839,937		786,761
Total stockholders' equity	121,281	[1]	30,328	35,871
Total liabilities and stockholders' equity	961,218		817,089
Parent Company
ASSETS
Cash and cash equivalents	741		764	$ 2,363
Securities available-for-sale	27		27
Investment in subsidiary	135,576		64,008
Other assets	6,075		1,804
Total assets	142,419		66,603
LIABILITIES AND STOCKHOLDERS' EQUITY
Notes payable	0		10,250
Other liabilities	250		5,137
Total liabilities	21,138		36,275
Total stockholders' equity	121,281		30,328
Total liabilities and stockholders' equity	142,419		66,603
Parent Company | Mandatory redeemable preferred stock
LIABILITIES AND STOCKHOLDERS' EQUITY
Preferred stock	268		268
Parent Company | Trust Preferred Stock
LIABILITIES AND STOCKHOLDERS' EQUITY
Preferred stock	$ 20,620		$ 20,620

[1] Share and per share amounts have been adjusted to reflect the Company's 1:30 reverse stock split effective May 29, 2015.