Borrowings - Summary of Short-Term Borrowings (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Short Term Debt [Line Items]
Total short-term borrowings	$ 412,617	$ 425,077
Federal Home Loan Bank Advances [Member]
Short Term Debt [Line Items]
Total short-term borrowings	350,000	356,758
Serviced GNMA Loans Eligible for Repurchase [Member]
Short Term Debt [Line Items]
Total short-term borrowings	36,025	41,014
Other [Member]
Short Term Debt [Line Items]
Total short-term borrowings	$ 26,592	$ 27,305