Fair Value, Carrying Value and Estimated Fair Value (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2018	Dec. 31, 2017	Nov. 30, 2017	Jan. 31, 2017	Dec. 31, 2016
Financial assets:
Loans and investments, net - Principal/Notional Amount	$ 2,776,422	$ 2,652,538
Loans and investments, net	2,702,097	2,579,127
Loans held-for-sale, net - Principal/Notional Amount	281,761	292,249
Securities, held-to-maturity, net - Principal/Notional Amount	52,673	40,566
Securities held-to-maturity, net	36,764	27,837
Derivative financial instruments - Principal/Notional Amount	292,411	77,984
Financial liabilities:
Credit and repurchase facilities, Principal/Notional Amount	629,724	530,938
Credit and repurchase facilities, Carrying value	626,063	528,573
Collateralized loan obligations	1,419,838	1,418,422
Debt fund	68,176	68,084
Senior unsecured notes	196,090	95,280
Convertible senior unsecured notes, net	232,577	231,287
Junior subordinated notes	139,760	139,590
Related party financing		50,000
Derivative financial instruments - Principal/Notional Amount	$ 57,850	291,421
Loans held-for-sale, net
Maximum number of days held-for-sale loans are typically sold	60 days
Carrying Value
Financial assets:
Loans and investments, net	$ 2,702,097	2,579,127
Loans held-for-sale, net	286,325	297,443
Capitalized mortgage servicing rights, net	255,732	252,608
Securities held-to-maturity, net	36,764	27,837
Derivative financial instruments	2,763	684
Financial liabilities:
Credit and repurchase facilities, Carrying value	626,063	528,573
Collateralized loan obligations	1,419,838	1,418,422
Debt fund	68,176	68,084
Senior unsecured notes	196,090	95,280
Convertible senior unsecured notes, net	232,577	231,287
Junior subordinated notes	139,760	139,590
Related party financing		50,000
Derivative financial instruments	300	1,306
Fair Value
Financial assets:
Loans and investments, net	2,774,333	2,652,520
Loans held-for-sale, net	292,153	302,883
Capitalized mortgage servicing rights, net	299,392	286,073
Securities held-to-maturity, net	37,273	28,439
Derivative financial instruments	2,763	684
Financial liabilities:
Credit and repurchase facilities, Carrying value	628,510	529,992
Collateralized loan obligations	1,438,567	1,436,871
Debt fund	70,139	70,000
Senior unsecured notes	198,995	99,582
Convertible senior unsecured notes, net	254,727	254,335
Junior subordinated notes	94,638	94,215
Related party financing		49,682
Derivative financial instruments	300	1,306
CLOs
Financial liabilities:
Debt instrument - Principal/Notional Amount	1,436,274	1,436,274
Collateralized loan obligations	1,419,838	1,418,422
Debt Fund
Financial liabilities:
Debt instrument - Principal/Notional Amount	70,000	70,000
Senior Unsecured Notes
Financial liabilities:
Debt instrument - Principal/Notional Amount	197,860	97,860
Convertible Senior Unsecured Notes
Financial liabilities:
Debt instrument - Principal/Notional Amount	243,750	243,750	$ 143,800	$ 100,000	$ 86,300
Junior subordinated notes
Financial liabilities:
Debt instrument - Principal/Notional Amount	154,336	154,336
Junior subordinated notes	$ 139,800	$ 139,600