Investment Securities - Schedule of Investment Securities (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Debt Securities
Available-for-sale:
Amount Outstanding	$ 2,960,048	$ 2,226,409
Unamortized Premium/(Discount)	1,382	(3,489)
Amortized Cost	2,961,430	2,222,920
Unrealized Gains	3,959	860
Unrealized Losses	(5,546)	(5,928)
Fair Value	2,959,843	2,217,852
Held-to-maturity:
Unamortized Premium/(Discount)	1,382	(3,489)
Total investment securities
Amount Outstanding	3,005,080	2,271,441
Unamortized Premium/(Discount)	1,382	(3,489)
Amortized Cost	3,006,462	2,267,952
Unrealized Gains	4,912	1,422
Unrealized Losses	(5,546)	(5,928)
Fair Value	3,005,828	2,263,446
Available-for-Sale Securities \| Debt Securities
Available-for-sale:
Unamortized Premium/(Discount)	1,382	(3,489)
Held-to-maturity:
Unamortized Premium/(Discount)	1,382	(3,489)
Total investment securities
Unamortized Premium/(Discount)	1,382	(3,489)
Floating Interest Rate \| Floating rate auction-rate certificates backed by Government guaranteed student loans
Available-for-sale:
Amount Outstanding	19,700	19,700
Amortized Cost	19,700	19,700
Unrealized Gains	0	0
Unrealized Losses	(788)	(985)
Fair Value	18,912	18,715
Floating Interest Rate \| Floating rate asset-backed securities
Available-for-sale:
Amount Outstanding	11,092	28,940
Amortized Cost	11,092	28,804
Unrealized Gains	0	2
Unrealized Losses	(7)	(128)
Fair Value	11,085	28,678
Floating Interest Rate \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Amount Outstanding	1,633,731	1,379,472
Amortized Cost	1,634,905	1,381,000
Unrealized Gains	2,414	721
Unrealized Losses	(4,736)	(4,267)
Fair Value	1,632,583	1,377,454
Floating Interest Rate \| Available-for-Sale Securities \| Floating rate auction-rate certificates backed by Government guaranteed student loans
Available-for-sale:
Unamortized Premium/(Discount)	0	0
Held-to-maturity:
Unamortized Premium/(Discount)	0	0
Total investment securities
Unamortized Premium/(Discount)	0	0
Floating Interest Rate \| Available-for-Sale Securities \| Floating rate asset-backed securities
Available-for-sale:
Unamortized Premium/(Discount)	0	(136)
Held-to-maturity:
Unamortized Premium/(Discount)	0	(136)
Total investment securities
Unamortized Premium/(Discount)	0	(136)
Floating Interest Rate \| Available-for-Sale Securities \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Unamortized Premium/(Discount)	1,174	1,528
Held-to-maturity:
Unamortized Premium/(Discount)	1,174	1,528
Total investment securities
Unamortized Premium/(Discount)	1,174	1,528
Fixed Interest Rate \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Amount Outstanding	315	384
Amortized Cost	315	385
Unrealized Gains	25	18
Unrealized Losses	0	0
Fair Value	340	403
Held-to-maturity:
Amount Outstanding	45,032	45,032
Amortized Cost	45,032	45,032
Unrealized Gains	953	562
Unrealized Losses	0	0
Fair Value	$ 45,985	$ 45,594
Total investment securities
Held-to-maturity investment securities weighted average yield	3.30%	3.50%
Fixed Interest Rate \| Fixed rate U.S. Treasuries
Available-for-sale:
Amount Outstanding	$ 1,295,210	$ 797,913
Amortized Cost	1,295,418	793,031
Unrealized Gains	1,520	119
Unrealized Losses	(15)	(548)
Fair Value	1,296,923	792,602
Fixed Interest Rate \| Available-for-Sale Securities \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Unamortized Premium/(Discount)	0	1
Held-to-maturity:
Unamortized Premium/(Discount)	0	1
Total investment securities
Unamortized Premium/(Discount)	0	1
Fixed Interest Rate \| Available-for-Sale Securities \| Fixed rate U.S. Treasuries
Available-for-sale:
Unamortized Premium/(Discount)	208	(4,882)
Held-to-maturity:
Unamortized Premium/(Discount)	208	(4,882)
Total investment securities
Unamortized Premium/(Discount)	208	(4,882)
Fixed Interest Rate \| Held-to-Maturity Securities \| Government/GSE Guaranteed Mortgage Backed Securities
Available-for-sale:
Unamortized Premium/(Discount)	0	0
Held-to-maturity:
Unamortized Premium/(Discount)	0	0
Total investment securities
Unamortized Premium/(Discount)	$ 0	$ 0

Debt Securities, Available-for-Sale, Unpaid Principal Balance

No definition available.

Name:	agm_DebtSecuritiesAvailableforSaleUnpaidPrincipalBalance
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Held-to-Maturity Securities, Debt Maturities, Weighted-Average Yield

+ References

No definition available.

+ Details

Name:	agm_HeldtoMaturitySecuritiesDebtMaturitiesWeightedAverageYield
Namespace Prefix:	agm_
Data Type:	num:percentItemType
Balance Type:	na
Period Type:	instant

- Definition

Held-to-Maturity Unpaid Principal Balance

+ References

No definition available.

+ Details

Name:	agm_HeldtoMaturityUnpaidPrincipalBalance
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Marketable Securities, Amortized Cost Basis

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesAmortizedCostBasis
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Marketable Securities, Gross Unrealized Gains

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesGrossUnrealizedGains
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Marketable Securities, Gross Unrealized Losses

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesGrossUnrealizedLosses
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Marketable Securities Unpaid Principal Balance

+ References

No definition available.

+ Details

Name:	agm_MarketableSecuritiesUnpaidPrincipalBalance
Namespace Prefix:	agm_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount, before tax, of unrealized gain in accumulated other comprehensive income (AOCI) on investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleDebtSecuritiesAccumulatedGrossUnrealizedGainBeforeTax
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount, before tax, of unrealized loss in accumulated other comprehensive income (AOCI) on investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleDebtSecuritiesAccumulatedGrossUnrealizedLossBeforeTax
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amortized cost of investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleDebtSecuritiesAmortizedCostBasis
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_AvailableForSaleSecuritiesAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Amount of investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 45
-Paragraph 1
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=109979856&loc=d3e26610-111562

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (aa)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27161-111563

+ Details

Name:	us-gaap_AvailableForSaleSecuritiesDebtSecurities
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount, after accumulated amortization, of debt discount (premium).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 3
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28555-108399

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 2
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28551-108399

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 4
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28567-108399

Reference 4: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 45
-Paragraph 1A
-URI http://asc.fasb.org/extlink&oid=114775744&loc=d3e28541-108399

Reference 5: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 835
-SubTopic 30
-Section 55
-Paragraph 8
-URI http://asc.fasb.org/extlink&oid=114775985&loc=d3e28878-108400

+ Details

Name:	us-gaap_DebtInstrumentUnamortizedDiscountPremiumNet
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount of investment in debt security measured at amortized cost (held-to-maturity) and investment in debt security measured at fair value with change in fair value recognized in other comprehensive income (available-for-sale).

+ References

No definition available.

+ Details

Name:	us-gaap_DebtSecuritiesAvailableForSaleAndHeldToMaturity
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount of accumulated unrecognized gain on investment in debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27232-111563

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesAccumulatedUnrecognizedHoldingGain
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount of accumulated unrealized loss on investment in debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27232-111563

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesAccumulatedUnrecognizedHoldingLoss
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amount, before other-than-temporary impairment (OTTI) accretion, of investment in debt securities measured at amortized cost (held-to-maturity).

+ References

No definition available.

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesAmortizedCostBeforeOtherThanTemporaryImpairment
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesClassifiedAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Fair value of investment in debt security measured at amortized cost (held-to-maturity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 5
-Subparagraph (aa)
-URI http://asc.fasb.org/extlink&oid=82897076&loc=d3e27232-111563

+ Details

Name:	us-gaap_HeldToMaturitySecuritiesFairValue
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_MarketableSecuritiesAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_DebtSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_InformationByCategoryOfDebtSecurityAxis=us-gaap_AvailableforsaleSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	srt_ProductOrServiceAxis=agm_FloatingInterestRateMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_AuctionRateSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_AssetBackedSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_MortgageBackedSecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	srt_ProductOrServiceAxis=agm_FixedInterestRateMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_USTreasurySecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_InformationByCategoryOfDebtSecurityAxis=us-gaap_HeldtomaturitySecuritiesMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type: