Summary of Significant Accounting Policies - Narrative (Details)		3 Months Ended								6 Months Ended	12 Months Ended			19 Months Ended
	Dec. 01, 2014 USD ($) owner	Dec. 31, 2016 USD ($) trust borrower	Sep. 30, 2016 USD ($)	Jun. 30, 2016 USD ($)	Mar. 31, 2016 USD ($)	Dec. 31, 2015 USD ($)	Sep. 30, 2015 USD ($)	Jun. 30, 2015 USD ($)	Mar. 31, 2015 USD ($)	Dec. 31, 2016 USD ($) trust borrower	Dec. 31, 2016 USD ($) segment trust borrower shares	Dec. 31, 2015 USD ($) shares	Dec. 31, 2014 USD ($) shares	Jun. 30, 2016 USD ($)
Accounting Policy [Line Items]
Number of wholly-owned Trusts | trust		2								2	2
Number of operating segments | segment											2
Maturity period of securities at acquisition											90 days
Threshold for nonperforming loans individually evaluated for impairment											$ 50,000
Loan loss history used											5 years
Average holding of purchased receivables											1 year
Advertising, promotion and marketing expenses											$ 2,449,000	$ 2,728,000	$ 2,059,000
Anti-dilutive shares (in shares) | shares											0	54,903	0
Commercial and construction loans		$ 376,400,000				$ 391,900,000				$ 376,400,000	$ 376,400,000	$ 391,900,000
Large borrowing relationships | borrower		27								27	27
Unfunded commitments to large borrowers		$ 98,700,000								$ 98,700,000	$ 98,700,000
Goodwill		15,017,000				22,334,000				15,017,000	15,017,000	22,334,000
Compensation expense - RML acquisition payments		708,000	$ 3,250,000	$ 687,000	$ 130,000	1,225,000	$ 780,000	$ 587,000	$ 1,502,000		4,775,000	4,094,000	$ 0
Reduction in net income		$ (3,695,000)	$ (3,283,000)	$ (4,506,000)	$ (3,506,000)	$ (4,226,000)	$ (5,532,000)	$ (4,943,000)	$ (3,633,000)		(14,990,000)	(18,334,000)	$ (17,867,000)
RML
Accounting Policy [Line Items]
Number of former equity owners | owner	4
Voting interests acquired	76.50%
Number of former equity owners with continuing employment contracts | owner	3
Number of former equity owners that must continue employment under terms of acquisition agreement | owner	2
Accrued compensation expense related to acquisition payments											186,000
Compensation expense - RML acquisition payments											4,800,000	$ 4,100,000
Accounting for Payments Made to Former Equity Owners | RML
Accounting Policy [Line Items]
Total purchase price	$ 22,200,000
Goodwill	7,500,000
Accounting for Payments Made to Former Equity Owners | Previously Reported | RML
Accounting Policy [Line Items]
Total purchase price	29,500,000
Goodwill	$ 14,800,000
Accounting for Payments Made to Former Equity Owners | Restatement Adjustment | RML
Accounting Policy [Line Items]
Compensation expense - RML acquisition payments										765,000	2,800,000			$ 2,300,000
Reduction in net income										$ 438,000	$ 1,800,000			$ 1,400,000
Lender Concentration Risk
Accounting Policy [Line Items]
Concentration risk, percent											41.00%
Buildings
Accounting Policy [Line Items]
Premises and equipment, useful life											39 years
Minimum | Furniture And Equipment
Accounting Policy [Line Items]
Premises and equipment, useful life											3 years
Minimum | Leasehold Improvements
Accounting Policy [Line Items]
Premises and equipment, useful life											2 years
Maximum | Furniture And Equipment
Accounting Policy [Line Items]
Premises and equipment, useful life											7 years
Maximum | Leasehold Improvements
Accounting Policy [Line Items]
Premises and equipment, useful life											15 years