Investment Securities - Securities by Contractual Maturity (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Dec. 31, 2017
Amortized Cost
Available for sale securities, due in one year or less, amortized cost	$ 10,137	$ 211,019
Available for sale securities, due after one year through five years, amortized cost	165,647	126,452
Available for sale securities, due after five years through ten years, amortized cost	16,539	23,147
Available for sale securities, due after ten years, amortized cost	14,210	15,439
Available for sale securities, subtotal, amortized cost	206,533	376,057
Amortized Cost	543,314	692,824
Fair Value
Available for sale securities, due in one year or less, fair value	10,132	211,019
Available for sale securities, due after one year through five years, fair value	161,611	124,797
Available for sale securities, due after five years through ten years, fair value	16,099	22,804
Available for sale securities, due after ten years, fair value	14,036	15,421
Available for sale securities, subtotal, fair value	201,878	374,041
Fair Value	531,075	689,202
Mortgage-backed securities
Amortized Cost
Available for sale securities, no stated maturity, amortized cost	336,781	313,554
Amortized Cost	298,704	275,958
Fair Value
Available for sale securities, no stated maturity, fair value	329,197	311,652
Fair Value	292,563	274,990
Mutual funds with no stated maturity
Amortized Cost
Available for sale securities, no stated maturity, amortized cost	0	3,213
Fair Value
Available for sale securities, no stated maturity, fair value	$ 0	$ 3,509