Multifamily Loans - Product Types and Characteristics of Multifamily Loans (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2020 USD ($) loan	Dec. 31, 2019 USD ($) loan
Accounts, Notes, Loans and Financing Receivable [Line Items]
Number of Loans | loan	198	669
Total Principal	$ 172,748	$ 524,928
30-89 Days DQ	3,362	1,650
90+ Days DQ	$ 1,882	$ 747
Multifamily securities
Accounts, Notes, Loans and Financing Receivable [Line Items]
Number of Loans | loan	28	279
Total Principal	$ 462,808	$ 4,195,000
30-89 Days DQ	0	0
90+ Days DQ	0	0
Multifamily securities | $1,000 to $10,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loan Balance, minimum		1,000
Loan Balance, maximum		$ 10,000
Number of Loans | loan		114
Total Principal		$ 674,666
30-89 Days DQ		0
90+ Days DQ		$ 0
Multifamily securities | $1,000 to $10,000 | Minimum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate		3.29%
Multifamily securities | $1,000 to $10,000 | Maximum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate		4.73%
Multifamily securities | $10,001 to $20,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loan Balance, minimum	10,001	$ 10,001
Loan Balance, maximum	$ 20,000	$ 20,000
Number of Loans | loan	24	102
Total Principal	$ 370,934	$ 1,489,118
30-89 Days DQ	0	0
90+ Days DQ	$ 0	$ 0
Multifamily securities | $10,001 to $20,000 | Minimum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate	4.25%	3.54%
Multifamily securities | $10,001 to $20,000 | Maximum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate	4.25%	4.94%
Multifamily securities | $20,001 to $30,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loan Balance, minimum	$ 20,001	$ 20,001
Loan Balance, maximum	$ 30,000	$ 30,000
Number of Loans | loan	4	32
Total Principal	$ 91,874	$ 750,712
30-89 Days DQ	0	0
90+ Days DQ	$ 0	$ 0
Multifamily securities | $20,001 to $30,000 | Minimum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate	4.25%	3.54%
Multifamily securities | $20,001 to $30,000 | Maximum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate	4.25%	4.69%
Multifamily securities | $30,001 to $40,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loan Balance, minimum		$ 30,001
Loan Balance, maximum		$ 40,000
Number of Loans | loan		19
Total Principal		$ 654,729
30-89 Days DQ		0
90+ Days DQ		$ 0
Multifamily securities | $30,001 to $40,000 | Minimum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate		3.52%
Multifamily securities | $30,001 to $40,000 | Maximum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate		4.79%
Multifamily securities | Over $40,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loan Balance, maximum		$ 40,000
Number of Loans | loan		12
Total Principal		$ 625,775
30-89 Days DQ		0
90+ Days DQ		$ 0
Multifamily securities | Over $40,000 | Minimum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate		3.55%
Multifamily securities | Over $40,000 | Maximum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest Rate		4.65%