Borrowings - Summary of Interest Expense on Borrowings (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Borrowings
Total interest expense on borrowings	$ 95,784	$ 98,707	$ 113,911
Reverse repurchase agreements
Borrowings
Total interest expense on borrowings	40,373	42,626	45,272
FHLB-NY advances | Federal Home Loan Bank of New York
Borrowings
Total interest expense on borrowings	40,790	41,911	50,654
Other borrowings, net
Borrowings
Total interest expense on borrowings	13,031	13,031	17,398
Federal funds purchased
Borrowings
Total interest expense on borrowings	$ 1,590	$ 1,139	$ 587