Segment Information (Details) - USD ($)	3 Months Ended		6 Months Ended
	Jun. 30, 2016	Jun. 30, 2015	Jun. 30, 2016	Jun. 30, 2015
Segment Reporting Information [Line Items]
Interest income	$ 5,187,141	$ 9,561,348	$ 13,892,057	$ 19,248,745
Interest expense	4,113,725	4,727,749	8,911,002	9,473,226
Net interest income (expense)	1,073,416	4,833,599	4,981,055	9,775,519
Non-interest income	11,461,423	17,430,551	17,126,287	26,806,981
Change in unrealized gain or loss	(20,013,381)	(803,311)	(19,324,692)	(1,942,737)
Realized gain	22,450,664	484,450	22,264,630	649,238
(Loss)/gain on derivative instruments	69,673	1,401,457	(606,595)	494,367
Advisory fee - related party	773,357	717,488	1,540,835	1,428,288
Salaries, commissions and benefits	9,505,846	8,090,407	17,471,938	15,489,665
Operating expenses	2,683,898	3,953,156	5,724,782	6,872,804
Other Expenses:
Expenses	2,907,215	745,725	5,278,650	1,653,500
Depreciation and amortization	236,902	230,538	469,332	457,962
Total other expenses	3,144,117	976,263	5,747,982	2,111,462
Net income (loss) before income taxes	(1,065,423)	9,609,432	(6,044,852)	9,881,149
Income tax expense	55,330	2,899,916	(1,547,905)	2,754,387
Segment net income (loss)	(1,120,753)	6,709,516	(4,496,947)	7,126,762
Residential Mortgage Investment [Member]
Segment Reporting Information [Line Items]
Interest income	4,271,081	8,768,124	12,202,034	17,847,289
Interest expense	1,991,437	2,765,823	4,772,683	5,521,268
Net interest income (expense)	2,279,644	6,002,301	7,429,351	12,326,021
Non-interest income	0	0	0	0
Change in unrealized gain or loss	(20,013,381)	(803,311)	(19,324,692)	(1,942,737)
Realized gain	22,450,664	484,450	22,264,630	649,238
(Loss)/gain on derivative instruments	(381,417)	1,401,457	(1,123,106)	494,367
Advisory fee - related party	295,396	367,323	620,335	696,289
Salaries, commissions and benefits	0	0	0	0
Operating expenses	260,487	170,563	618,819	217,958
Other Expenses:
Expenses	910,633	730,397	1,290,794	1,588,849
Depreciation and amortization	0	0	0	0
Total other expenses	910,633	730,397	1,290,794	1,588,849
Net income (loss) before income taxes	2,868,994	5,816,614	6,716,235	9,023,793
Income tax expense	0	0	0	0
Segment net income (loss)	2,868,994	5,816,614	6,716,235	9,023,793
Residential Mortgage Banking [Member]
Segment Reporting Information [Line Items]
Interest income	916,060	793,224	1,690,023	1,401,456
Interest expense	653,639	518,932	1,205,459	1,072,291
Net interest income (expense)	262,421	274,292	484,564	329,165
Non-interest income	11,461,423	17,430,551	17,126,287	26,806,981
Change in unrealized gain or loss	0	0	0	0
Realized gain	0	0	0	0
(Loss)/gain on derivative instruments	0	0	0	0
Advisory fee - related party	88,891	136,428	202,013	265,264
Salaries, commissions and benefits	9,505,846	8,090,407	17,471,938	15,489,665
Operating expenses	1,778,403	2,175,927	3,494,196	4,166,215
Other Expenses:
Expenses	0	15,328	0	64,651
Depreciation and amortization	236,902	230,538	469,332	457,962
Total other expenses	236,902	245,866	469,332	522,613
Net income (loss) before income taxes	113,802	7,056,215	(4,026,628)	6,692,389
Income tax expense	55,330	2,899,916	(1,547,905)	2,754,387
Segment net income (loss)	58,472	4,156,299	(2,478,723)	3,938,002
Corporate/Other [Member]
Segment Reporting Information [Line Items]
Interest income	0	0	0	0
Interest expense	1,468,649	1,442,994	2,932,860	2,879,667
Net interest income (expense)	(1,468,649)	(1,442,994)	(2,932,860)	(2,879,667)
Non-interest income	0	0	0	0
Change in unrealized gain or loss	0	0	0	0
Realized gain	0	0	0	0
(Loss)/gain on derivative instruments	451,090	0	516,511	0
Advisory fee - related party	389,070	213,737	718,487	466,735
Salaries, commissions and benefits	0	0	0	0
Operating expenses	645,008	1,606,666	1,611,767	2,488,631
Other Expenses:
Expenses	1,996,582	0	3,987,856	0
Depreciation and amortization	0	0	0	0
Total other expenses	1,996,582	0	3,987,856	0
Net income (loss) before income taxes	(4,048,219)	(3,263,397)	(8,734,459)	(5,835,033)
Income tax expense	0	0	0	0
Segment net income (loss)	$ (4,048,219)	$ (3,263,397)	$ (8,734,459)	$ (5,835,033)