Other Assets and Other liabilities - Summary of Other Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Other Assets And Other Liabilities [Abstract]
Deferred compensation	$ 6,710	$ 12,343
Accrued payroll	5,987	6,343
Loan proceeds payable		5,730
Accrued contract termination fee		2,278
Mortgage servicing escrows	1,079	2,297
Mortgage buyback reserve	2,659	2,156
Accrued interest	632	581
Derivatives	586	369
Deferred tax liability	4,180	713
Other liabilities	36,535	11,713
Total other liabilities	$ 58,368	$ 44,523