Investment Securities - Schedule of Investment Securities (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Held-to-maturity:
Allowance for losses	$ (640)
Debt Securities
Available-for-sale:
Amount Outstanding	2,954,990	$ 2,960,048
Unamortized Premium/(Discount)	4,704	1,382
Amortized Cost	2,959,694	2,961,430
Allowance for losses	(24)
Unrealized Gains	15,739	3,959
Unrealized Losses	(14,252)	(5,546)
Fair Value	2,961,157	2,959,843
Held-to-maturity:
Unamortized Premium/(Discount)	4,704	1,382
Total investment securities
Amount Outstanding	3,000,022	3,005,080
Unamortized Premium/(Discount)	4,704	1,382
Amortized Cost	3,004,726	3,006,462
Allowance for losses	(24)
Unrealized Gains	15,870	4,912
Unrealized Losses	(14,252)	(5,546)
Fair Value	3,006,320	3,005,828
Accrued interest excluded	8,300
Available-for-Sale Securities \| Debt Securities
Available-for-sale:
Unamortized Premium/(Discount)	4,704	1,382
Held-to-maturity:
Unamortized Premium/(Discount)	4,704	1,382
Total investment securities
Unamortized Premium/(Discount)	4,704	1,382
Floating Interest Rate \| Floating rate auction-rate certificates backed by Government guaranteed student loans
Available-for-sale:
Amount Outstanding	19,700	19,700
Amortized Cost	19,700	19,700
Allowance for losses	(24)
Unrealized Gains	0	0
Unrealized Losses	(2,955)	(788)
Fair Value	16,721	18,912
Floating Interest Rate \| Floating rate asset-backed securities
Available-for-sale:
Amount Outstanding	10,521	11,092
Amortized Cost	10,521	11,092
Allowance for losses	0
Unrealized Gains	0	0
Unrealized Losses	(18)	(7)
Fair Value	10,503	11,085
Floating Interest Rate \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Amount Outstanding	1,783,998	1,633,731
Amortized Cost	1,784,293	1,634,905
Allowance for losses	0
Unrealized Gains	4,985	2,414
Unrealized Losses	(11,279)	(4,736)
Fair Value	1,777,999	1,632,583
Floating Interest Rate \| Available-for-Sale Securities \| Floating rate auction-rate certificates backed by Government guaranteed student loans
Available-for-sale:
Unamortized Premium/(Discount)	0	0
Held-to-maturity:
Unamortized Premium/(Discount)	0	0
Total investment securities
Unamortized Premium/(Discount)	0	0
Floating Interest Rate \| Available-for-Sale Securities \| Floating rate asset-backed securities
Available-for-sale:
Unamortized Premium/(Discount)	0	0
Held-to-maturity:
Unamortized Premium/(Discount)	0	0
Total investment securities
Unamortized Premium/(Discount)	0	0
Floating Interest Rate \| Available-for-Sale Securities \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Unamortized Premium/(Discount)	295	1,174
Held-to-maturity:
Unamortized Premium/(Discount)	295	1,174
Total investment securities
Unamortized Premium/(Discount)	295	1,174
Fixed Interest Rate \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Amount Outstanding	306	315
Amortized Cost	306	315
Allowance for losses	0
Unrealized Gains	29	25
Unrealized Losses	0	0
Fair Value	335	340
Held-to-maturity:
Amount Outstanding	45,032	45,032
Amortized Cost	45,032	45,032
Allowance for losses	0
Unrealized Gains	131	953
Unrealized Losses	0	0
Fair Value	$ 45,163	$ 45,985
Total investment securities
Held-to-maturity investment securities weighted average yield	3.00%	3.30%
Fixed Interest Rate \| Fixed rate U.S. Treasuries
Available-for-sale:
Amount Outstanding	$ 1,140,465	$ 1,295,210
Amortized Cost	1,144,874	1,295,418
Allowance for losses	0
Unrealized Gains	10,725	1,520
Unrealized Losses	0	(15)
Fair Value	1,155,599	1,296,923
Fixed Interest Rate \| Available-for-Sale Securities \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Unamortized Premium/(Discount)	0	0
Held-to-maturity:
Unamortized Premium/(Discount)	0	0
Total investment securities
Unamortized Premium/(Discount)	0	0
Fixed Interest Rate \| Available-for-Sale Securities \| Fixed rate U.S. Treasuries
Available-for-sale:
Unamortized Premium/(Discount)	4,409	208
Held-to-maturity:
Unamortized Premium/(Discount)	4,409	208
Total investment securities
Unamortized Premium/(Discount)	4,409	208
Fixed Interest Rate \| Held-to-Maturity Securities \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Unamortized Premium/(Discount)	0	0
Held-to-maturity:
Unamortized Premium/(Discount)	0	0
Total investment securities
Unamortized Premium/(Discount)	$ 0	$ 0

Debt Securities, Available-for-Sale, Accrued Interest

No definition available.

Name:	agm_DebtSecuritiesAvailableForSaleAccruedInterest
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Debt Securities, Available-for-Sale, Unpaid Principal Balance

+ References

No definition available.

+ Details

Name:	agm_DebtSecuritiesAvailableforSaleUnpaidPrincipalBalance
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Held-to-Maturity Securities, Debt Maturities, Weighted-Average Yield

+ References

No definition available.

+ Details

Name:	agm_HeldtoMaturitySecuritiesDebtMaturitiesWeightedAverageYield
Namespace Prefix:	agm_
Data Type:	num:percentItemType
Balance Type:	na
Period Type:	instant

- Definition

Held-to-Maturity Unpaid Principal Balance

+ References

No definition available.

+ Details

Name:	agm_HeldtoMaturityUnpaidPrincipalBalance
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Marketable Securities, Allowance for Credit Loss

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesAllowanceForCreditLoss
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Marketable Securities, Amortized Cost Basis

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesAmortizedCostBasis
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Marketable Securities, Gross Unrealized Gains

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesGrossUnrealizedGains
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Marketable Securities, Gross Unrealized Losses

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesGrossUnrealizedLosses
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Marketable Securities Unpaid Principal Balance

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesUnpaidPrincipalBalance
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount, before tax, of unrealized gain in accumulated other comprehensive income (AOCI) on investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleDebtSecuritiesAccumulatedGrossUnrealizedGainBeforeTax
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount, before tax, of unrealized loss in accumulated other comprehensive income (AOCI) on investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleDebtSecuritiesAccumulatedGrossUnrealizedLossBeforeTax
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amortized cost of investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27161-111563

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 30
-Section 45
-Paragraph 1
-URI http://asc.fasb.org/extlink&oid=118942338&loc=SL82898722-210454

+ Details

Name:	us-gaap_AvailableForSaleDebtSecuritiesAmortizedCostBasis
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_AvailableForSaleSecuritiesAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Amount of investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 30
-Section 45
-Paragraph 1
-URI http://asc.fasb.org/extlink&oid=118942338&loc=SL82898722-210454

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 45
-Paragraph 1
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=118954083&loc=d3e26610-111562

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (aa)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleSecuritiesDebtSecurities
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount, after accumulated amortization, of debt discount (premium).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 3
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28555-108399

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 2
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28551-108399

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 4
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28567-108399

Reference 4: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 1A
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28541-108399

Reference 5: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 55
-Paragraph 8
-URI http://asc.fasb.org/extlink&oid=114775985&loc=d3e28878-108400

+ Details

Name:	us-gaap_DebtInstrumentUnamortizedDiscountPremiumNet
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount of allowance for credit loss on investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 30
-Section 50
-Paragraph 9
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=118955579&loc=SL82922900-210455

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 30
-Section 50
-Paragraph 9
-Subparagraph (i)
-URI http://asc.fasb.org/extlink&oid=118955579&loc=SL82922900-210455

Reference 3: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (aaa)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27161-111563

Reference 4: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 30
-Section 45
-Paragraph 1
-URI http://asc.fasb.org/extlink&oid=118942338&loc=SL82898722-210454

+ Details

Name:	us-gaap_DebtSecuritiesAvailableForSaleAllowanceForCreditLoss
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount of investment in debt security measured at amortized cost (held-to-maturity) and investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

No definition available.

+ Details

Name:	us-gaap_DebtSecuritiesAvailableForSaleAndHeldToMaturity
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount of allowance for credit loss for debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 50
-Paragraph 13
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=119407570&loc=SL82919249-210447

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (aaa)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27232-111563

Reference 3: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 45
-Paragraph 1
-URI http://asc.fasb.org/extlink&oid=118955202&loc=SL82895884-210446

Reference 4: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 50
-Paragraph 13
-Subparagraph (f)
-URI http://asc.fasb.org/extlink&oid=119407570&loc=SL82919249-210447

+ Details

Name:	us-gaap_DebtSecuritiesHeldToMaturityAllowanceForCreditLoss
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount of accumulated unrecognized gain on investment in debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27232-111563

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesAccumulatedUnrecognizedHoldingGain
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount of accumulated unrealized loss on investment in debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27232-111563

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesAccumulatedUnrecognizedHoldingLoss
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount, before other-than-temporary impairment (OTTI) accretion, of investment in debt securities measured at amortized cost (held-to-maturity).

+ References

No definition available.

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesAmortizedCostBeforeOtherThanTemporaryImpairment
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesClassifiedAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Fair value of investment in debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (aa)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=d3e27232-111563

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesFairValue
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_MarketableSecuritiesAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_DebtSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_InformationByCategoryOfDebtSecurityAxis=us-gaap_AvailableforsaleSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	srt_ProductOrServiceAxis=agm_FloatingInterestRateMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_AuctionRateSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_AssetBackedSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_MortgageBackedSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	srt_ProductOrServiceAxis=agm_FixedInterestRateMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_USTreasurySecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_InformationByCategoryOfDebtSecurityAxis=us-gaap_HeldtomaturitySecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type: