Summary of Significant Accounting Policies - Consolidation of Variable Interest Entities (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Variable Interest Entity [Line Items]
Loans held for investment, at amortized cost	$ 5,789,391	$ 5,390,977
Notes payable	20,665,020	19,098,648
Farm & Ranch
Variable Interest Entity [Line Items]
Borrower remittances	8,800	15,600
USDA Guarantees
Variable Interest Entity [Line Items]
Unamortized premiums and discounts	100	(200)
On-Balance Sheet: | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Loans held for investment, at amortized cost	1,540,689	1,600,917
Notes payable	1,549,527	1,616,504
Off-Balance Sheet: | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Maximum exposure to loss	468,205	496,538
Farmer Mac Guaranteed Securities | On-Balance Sheet: | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	33,512	32,041
Maximum exposure to loss	33,440	31,887
Debt Securities
Variable Interest Entity [Line Items]
Unamortized premiums and discounts	(4,704)	(1,382)
Debt Securities | On-Balance Sheet: | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	1,291,950	1,117,203
Maximum exposure to loss	1,302,410	1,120,765
Corporate | On-Balance Sheet: | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Loans held for investment, at amortized cost	0	0
Notes payable	0	0
Corporate | Off-Balance Sheet: | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Maximum exposure to loss	0	0
Corporate | Farmer Mac Guaranteed Securities | On-Balance Sheet: | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	0	0
Maximum exposure to loss	0	0
Corporate | Debt Securities | On-Balance Sheet: | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	1,291,950	1,117,203
Maximum exposure to loss	1,302,410	1,120,765
Operating Segments | On-Balance Sheet: | Farm & Ranch | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Loans held for investment, at amortized cost	1,540,689	1,600,917
Notes payable	1,549,527	1,616,504
Operating Segments | On-Balance Sheet: | USDA Guarantees | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Loans held for investment, at amortized cost	0	0
Notes payable	0	0
Operating Segments | Off-Balance Sheet: | Farm & Ranch | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Maximum exposure to loss	97,302	107,322
Operating Segments | Off-Balance Sheet: | USDA Guarantees | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Maximum exposure to loss	370,903	389,216
Operating Segments | Farmer Mac Guaranteed Securities | On-Balance Sheet: | Farm & Ranch | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	0	0
Maximum exposure to loss	0	0
Operating Segments | Farmer Mac Guaranteed Securities | On-Balance Sheet: | USDA Guarantees | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	33,512	32,041
Maximum exposure to loss	33,440	31,887
Operating Segments | Debt Securities | On-Balance Sheet: | Farm & Ranch | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	0	0
Maximum exposure to loss	0	0
Operating Segments | Debt Securities | On-Balance Sheet: | USDA Guarantees | Variable Interest Entity, Not Primary Beneficiary
Variable Interest Entity [Line Items]
Carrying value	0	0
Maximum exposure to loss	$ 0	$ 0