Investment Securities (Details 3)	9 Months Ended
	Sep. 30, 2016 USD ($) security	Sep. 30, 2015 USD ($)	Dec. 31, 2015 USD ($) security
Investment Securities
Estimated Fair Value	$ 524,295,000		$ 513,201,000
Net unrealized gain (loss)	7,100,000		(4,400,000)
Purchases of investment securities available-for-sale	77,275,000	$ 50,538,000
U.S. Government-Sponsored Enterprises backed explicitly by the full faith and credit of the U.S. Government
Investment Securities
Estimated Fair Value	27,900,000		21,800,000
GSE debentures
Investment Securities
Estimated Fair Value	$ 66,339,000		$ 40,627,000
Number of securities | security	21		13
Net unrealized gain (loss)	$ 1,600,000		$ (31,000)
Number of securities under unrealized loss position | security	2		7
Purchases of investment securities available-for-sale	$ 32,300,000	11,800,000
GSE CMOs
Investment Securities
Estimated Fair Value	$ 172,800,000		$ 193,800,000
Number of securities | security	62		63
Net unrealized gain (loss)	$ 200,000		$ (4,200,000)
Number of securities under unrealized loss position | security	17		45
Purchases of investment securities available-for-sale	$ 3,100,000	0
GSE MBSs
Investment Securities
Estimated Fair Value	$ 233,400,000		$ 229,900,000
Number of securities | security	196		186
Net unrealized gain (loss)	$ 4,400,000		$ (300,000)
Number of securities under unrealized loss position | security	11		56
Purchases of investment securities available-for-sale	$ 36,700,000	29,400,000
SBA commercial loan asset-backed securities
Investment Securities
Estimated Fair Value	$ 111,000		$ 147,000
Number of securities | security	6		7
Number of securities under unrealized loss position | security	5		6
Corporate debt obligations
Investment Securities
Estimated Fair Value	$ 49,304,000		$ 46,486,000
Number of securities | security	16		15
Net unrealized gain (loss)	$ 1,000,000		$ 300,000
Number of securities under unrealized loss position | security	1		2
Purchases of investment securities available-for-sale	$ 5,100,000	$ 9,300,000
Trust preferred securities
Investment Securities
Estimated Fair Value	$ 1,313,000		$ 1,267,000
Number of securities | security	2		2
Net unrealized gain (loss)	$ (200,000)		$ (200,000)
Number of securities under unrealized loss position | security	2		2
Marketable equity securities
Investment Securities
Estimated Fair Value	$ 1,000,000		$ 977,000
Number of securities | security	2		2
Net unrealized gain (loss)	$ 36,000		$ 21,000
Number of securities under unrealized loss position | security	0		0