Supplemental Financial Data (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Balance sheet related
Loans, held-for-investment	$ 4,438,650	$ 4,246,694
Other assets	9,364	11,262
Total Assets	5,372,095	4,977,018
Total Liabilities	4,537,887	4,132,234
Income statement related
Realized gains	31,913	28,958	$ 38,409
Unrealized gains (losses)	(48,101)	(18,790)	4,853
Income (loss) before provision for income taxes	54,453	64,504	62,843
Income (loss) on unconsolidated joint ventures	$ 2,404	6,088	12,148
WFLLA LLC
Supplemental Financial Data
Ownership percentage	50.00%
Girod HoldCo, LLC
Balance sheet related
Cash	$ 9,746	22,585
Loans, held-for-investment	92,053	103,751
Real estate owned	12,701	7,929
Other assets	429	487
Total Assets	114,929	134,752
Accounts payable and accrued liabilities	80	72
Other liabilities	1,606	4,076
Total Liabilities	1,686	4,148
Income statement related
Interest income	2,372	6,922	21,182
Realized gains	(40)	5,218	27,310
Unrealized gains (losses)	(3,456)	15,286	8,191
Servicing expense and other	(4,359)	(5,996)	(7,972)
Income (loss) before provision for income taxes	(5,483)	21,430	48,711
WFLLA LLC
Balance sheet related
Cash	4,947	11,270
Loans, held-for-investment	45,934	51,772
Real estate owned	6,338	3,957
Other assets	214	239
Total Assets	57,433	67,238
Accounts payable and accrued liabilities	52	36
Other liabilities	801	2,034
Total Liabilities	853	2,070
Income statement related
Interest income	1,184	3,454	10,570
Realized gains	(20)	2,604	13,628
Unrealized gains (losses)	(1,725)	7,628	4,087
Servicing expense and other	(2,175)	(2,999)	(3,989)
Income (loss) before provision for income taxes	(2,736)	10,687	24,296
Income (loss) on unconsolidated joint ventures	$ (1,368)	5,344	$ 12,148
WFLLA LLC | Girod HoldCo, LLC
Supplemental Financial Data
Ownership percentage	49.90%
Other Unconsolidated Joint Ventures
Income statement related
Income (loss) on unconsolidated joint ventures	$ 3,772	$ 744