Fair Value of Financial Instruments - Carrying Values and Estimated Fair Values of the Company's Financial Instruments (Details) - USD ($) $ in Thousands	6 Months Ended
	Jun. 30, 2016	Dec. 31, 2015	Jun. 30, 2015	Dec. 31, 2014
Carrying values and estimated fair values of financial instruments:
Prepayments of principal amount of loans assumed	$ 0
Financial assets:
Cash and cash equivalents	28,762	$ 29,133	$ 14,014	$ 30,077
Certificates of deposit with other banks	13,150	13,150
Securities available-for-sale	138,844	70,256
Securities held-to-maturity	0	54,470
Financial liabilities:
FHLB and other borrowings	213,008	183,198
Level I
Financial assets:
Cash and cash equivalents	28,762	29,133
Level II
Financial assets:
Certificates of deposit with other banks	13,394	13,270
Securities available-for-sale	138,844	70,256
Securities held-to-maturity		54,470
Loans held for sale	131,671	102,623
Accrued interest receivable	754	723
Financial liabilities:
Deposits	1,081,195	1,015,521
Repurchase agreements	26,048	27,437
FHLB and other borrowings	213,015	183,211
Accrued interest payable	720	474
Subordinated debt	32,172	32,172
Level III
Financial assets:
Loans, net	1,072,011	1,034,832
Mortgage servicing rights	662	956
Accrued interest receivable	3,220	2,633
Carrying Value
Financial assets:
Cash and cash equivalents	28,762	29,133
Certificates of deposit with other banks	13,150	13,150
Securities available-for-sale	138,844	70,256
Securities held-to-maturity		52,859
Loans held for sale	131,671	102,623
Loans, net	1,078,415	1,024,164
Mortgage servicing rights	662	956
Accrued interest receivable	3,974	3,356
Financial liabilities:
Deposits	1,067,242	1,012,314
Repurchase agreements	26,048	27,437
FHLB and other borrowings	213,008	183,198
Accrued interest payable	720	474
Subordinated debt	33,524	33,524
Total
Financial assets:
Cash and cash equivalents	28,762	29,133
Certificates of deposit with other banks	13,394	13,270
Securities available-for-sale	138,844	70,256
Securities held-to-maturity		54,470
Loans held for sale	131,671	102,623
Loans, net	1,072,011	1,034,832
Mortgage servicing rights	662	956
Accrued interest receivable	3,974	3,356
Financial liabilities:
Deposits	1,081,195	1,015,521
Repurchase agreements	26,048	27,437
FHLB and other borrowings	213,015	183,211
Accrued interest payable	720	474
Subordinated debt	32,172	32,172
Interest rate lock commitment | Level III
Financial assets:
Derivative asset	3,625	1,583
Interest rate lock commitment | Carrying Value
Financial assets:
Derivative asset	3,625	1,583
Interest rate lock commitment | Total
Financial assets:
Derivative asset	3,625	1,583
Interest rate cap | Level II
Financial assets:
Derivative asset	64	437
Interest rate cap | Carrying Value
Financial assets:
Derivative asset	64	437
Interest rate cap | Total
Financial assets:
Derivative asset	64	437
Interest rate swap | Level II
Financial assets:
Derivative asset	1,475	405
Financial liabilities:
Derivative Liability	1,475	405
Interest rate swap | Carrying Value
Financial assets:
Derivative asset	1,475	405
Financial liabilities:
Derivative Liability	1,475	405
Interest rate swap | Total
Financial assets:
Derivative asset	1,475	405
Financial liabilities:
Derivative Liability	1,475	$ 405
Forward sales commitments | Level II
Financial liabilities:
Derivative Liability	2,077
Forward sales commitments | Carrying Value
Financial liabilities:
Derivative Liability	2,077
Forward sales commitments | Total
Financial liabilities:
Derivative Liability	$ 2,077