Investment Securities - Securities Available-for-Sale (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Dec. 31, 2017
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	$ 958,253	$ 719,607
Gross unrealized gains	1,305	1,795
Gross unrealized (losses)	(27,488)	(10,832)
Securities available-for-sale, at fair value	932,070	710,570
U.S. government and agency obligations
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	54,805	39,909
Gross unrealized gains	0	17
Gross unrealized (losses)	(1,243)	(282)
Securities available-for-sale, at fair value	53,562	39,644
Mortgage-backed securities and collateralized mortgage obligations
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	840,040	615,924
Gross unrealized gains	344	383
Gross unrealized (losses)	(24,852)	(9,695)
Securities available-for-sale, at fair value	815,532	606,612
Municipal obligations
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	1,586	1,957
Gross unrealized gains	0	0
Gross unrealized (losses)	0	0
Securities available-for-sale, at fair value	1,586	1,957
Corporate debt securities
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	54,491	54,489
Gross unrealized gains	99	536
Gross unrealized (losses)	(1,023)	(511)
Securities available-for-sale, at fair value	53,567	54,514
Trust preferred securities
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	5,000	5,000
Gross unrealized gains	0	0
Gross unrealized (losses)	(370)	(344)
Securities available-for-sale, at fair value	4,630	4,656
Equity securities
Schedule of Available-for-sale Securities [Line Items]
Amortized cost	2,331	2,328
Gross unrealized gains	862	859
Gross unrealized (losses)	0	0
Securities available-for-sale, at fair value	$ 3,193	$ 3,187