Business Segment Reporting - (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended								12 Months Ended
	Dec. 31, 2013	Sep. 30, 2013	Jun. 30, 2013	Mar. 31, 2013	Dec. 31, 2012	Sep. 30, 2012	Jun. 30, 2012	Mar. 31, 2012	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
Segment Reporting Information [Line Items]
Total interest income	$ 49,180	$ 62,975	$ 61,745	$ 61,498	$ 56,151	$ 63,810	$ 71,578	$ 73,131	$ 235,398	$ 264,670	$ 274,687
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									0	0	0
Interest Expense	(35,777)	(34,787)	(33,584)	(33,128)	(33,358)	(33,448)	(36,961)	(38,923)	(137,276)	(142,690)	(153,382)
Net interest income	13,403	28,188	28,161	28,370	22,793	30,362	34,617	34,208	98,122	121,980	121,305
Guaranty Fee Income	6,768	6,819	6,759	6,612	6,568	6,401	6,064	5,930	26,958	24,963	24,821
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									38,813	(21,228)	(72,210)
Non-interest Income/(loss)	17,022	9,821	26,483	12,445	8,374	8,464	(27,281)	14,178	65,771	3,735	(47,389)
(Release of/(provision for) loan losses	(117)	499	529	(430)	(4,354)	(137)	1,220	(420)	481	(3,691)	(610)
Provision for/(release of) losses									(929)	1,816	2,957
Noninterest Expense, Excluding Loan, Lease and Other Losses									(32,178)	(32,724)	(31,616)
Noninterest Expense	(7,621)	(8,441)	(7,964)	(9,081)	(6,109)	(7,748)	(9,209)	(7,842)	(33,107)	(30,908)	(28,659)
Income/(loss) before income taxes	22,687	30,067	47,209	31,304	20,704	30,941	(653)	40,124	131,267	91,116	44,647
Income tax (benefit)/expense	(3,774)	(8,226)	(13,036)	(8,716)	(4,837)	(8,294)	2,629	(11,654)	(33,752)	(22,156)	(5,797)
Net income	18,913	21,841	34,173	22,588	15,867	22,647	1,976	28,470	97,515	68,960	38,850
Preferred stock dividends	(882)	(881)	(881)	(851)	(720)	(719)	(720)	(720)	(3,495)	(2,879)	(2,879)
Less: Net Income attributable to non-controlling Interest - preferred stock dividends	(5,546)	(5,547)	(5,547)	(5,547)	(5,546)	(5,547)	(5,547)	(5,547)	(22,187)	(22,187)	(22,187)
Net Income (Loss) Available to Common Stockholders, Basic	12,485	15,413	27,745	16,190	9,601	16,381	(4,291)	22,203	71,833	43,894	13,784
Total assets at carrying value	13,361,780				12,622,201				13,361,780	12,622,201	11,883,508
On and Off Balance Sheet Program Assets, At Principal Balance	13,950,312				13,015,188				13,950,312	13,015,188	11,913,302
Farm & Ranch [Member]
Segment Reporting Information [Line Items]
Total interest income									67,429	70,590	98,266
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									(964)	(1,659)	(3,269)
Interest Expense									(32,062)	(36,390)	(72,967)
Net interest income									34,403	32,541	22,030
Guaranty Fee Income									14,944	14,292	15,178
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									2,244	2,427	3,201
Non-interest Income/(loss)									17,188	16,719	18,379
(Release of/(provision for) loan losses									481	(3,691)	(610)
Provision for/(release of) losses									(929)	1,816	2,957
Noninterest Expense, Excluding Loan, Lease and Other Losses									(14,649)	(14,836)	(15,232)
Noninterest Expense									(15,578)	(13,020)	(12,275)
Segment core earnings before income taxes									36,494	32,549	27,524
Income tax (benefit)/expense									(12,773)	(11,392)	(9,633)
Segment core earnings before preferred stock dividends and attribution of income to non-controlling interest - preferred stock dividends									23,721	21,157	17,891
Preferred stock dividends									0	0	0
Less: Net Income attributable to non-controlling Interest - preferred stock dividends									0	0	0
Segment core earnings									23,721	21,157	17,891
Total assets at carrying value	2,190,224				1,736,391				2,190,224	1,736,391	1,999,388
On and Off Balance Sheet Program Assets, At Principal Balance	5,163,080				4,747,289				5,163,080	4,747,289	4,346,027
USDA Guarantees [Member]
Segment Reporting Information [Line Items]
Total interest income									53,384	56,815	56,726
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									0	0	0
Interest Expense									(35,656)	(38,761)	(40,650)
Net interest income									17,728	18,054	16,076
Guaranty Fee Income									132	163	204
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									791	599	283
Non-interest Income/(loss)									923	762	487
(Release of/(provision for) loan losses									0	0	0
Provision for/(release of) losses									0	0	0
Noninterest Expense, Excluding Loan, Lease and Other Losses									(2,904)	(2,773)	(2,496)
Noninterest Expense									(2,904)	(2,773)	(2,496)
Segment core earnings before income taxes									15,747	16,043	14,067
Income tax (benefit)/expense									(5,511)	(5,615)	(4,923)
Segment core earnings before preferred stock dividends and attribution of income to non-controlling interest - preferred stock dividends									10,236	10,428	9,144
Preferred stock dividends									0	0	0
Less: Net Income attributable to non-controlling Interest - preferred stock dividends									0	0	0
Segment core earnings									10,236	10,428	9,144
Total assets at carrying value	1,656,688				1,641,030				1,656,688	1,641,030	1,552,105
On and Off Balance Sheet Program Assets, At Principal Balance	1,687,117				1,615,579				1,687,117	1,615,579	1,513,177
Rural Utilities [Member]
Segment Reporting Information [Line Items]
Total interest income									35,194	35,566	22,377
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									0	0	0
Interest Expense									(23,601)	(23,301)	(13,860)
Net interest income									11,593	12,265	8,517
Guaranty Fee Income									0	0	0
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									0	466	0
Non-interest Income/(loss)									0	466	0
(Release of/(provision for) loan losses									0	0	0
Provision for/(release of) losses									0	0	0
Noninterest Expense, Excluding Loan, Lease and Other Losses									(3,100)	(3,204)	(2,944)
Noninterest Expense									(3,100)	(3,204)	(2,944)
Segment core earnings before income taxes									8,493	9,527	5,573
Income tax (benefit)/expense									(2,973)	(3,334)	(1,951)
Segment core earnings before preferred stock dividends and attribution of income to non-controlling interest - preferred stock dividends									5,520	6,193	3,622
Preferred stock dividends									0	0	0
Less: Net Income attributable to non-controlling Interest - preferred stock dividends									0	0	0
Segment core earnings									5,520	6,193	3,622
Total assets at carrying value	1,076,298				1,080,045				1,076,298	1,080,045	967,668
On and Off Balance Sheet Program Assets, At Principal Balance	1,052,251				1,031,945				1,052,251	1,031,945	916,027
Institutional Credit [Member]
Segment Reporting Information [Line Items]
Total interest income									77,868	88,801	79,247
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									0	0	0
Interest Expense									(53,613)	(64,942)	(59,710)
Net interest income									24,255	23,859	19,537
Guaranty Fee Income									12,846	12,167	12,708
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									0	0	0
Non-interest Income/(loss)									12,846	12,167	12,708
(Release of/(provision for) loan losses									0	0	0
Provision for/(release of) losses									0	0	0
Noninterest Expense, Excluding Loan, Lease and Other Losses									(1,774)	(1,834)	(1,685)
Noninterest Expense									(1,774)	(1,834)	(1,685)
Segment core earnings before income taxes									35,327	34,192	30,560
Income tax (benefit)/expense									(12,364)	(11,967)	(10,696)
Segment core earnings before preferred stock dividends and attribution of income to non-controlling interest - preferred stock dividends									22,963	22,225	19,864
Preferred stock dividends									0	0	0
Less: Net Income attributable to non-controlling Interest - preferred stock dividends									0	0	0
Segment core earnings									22,963	22,225	19,864
Total assets at carrying value	5,121,666				4,772,509				5,121,666	4,772,509	4,295,367
On and Off Balance Sheet Program Assets, At Principal Balance	6,047,864				5,620,375				6,047,864	5,620,375	5,138,071
Corporate Segment [Member]
Segment Reporting Information [Line Items]
Total interest income									21,940	24,729	28,117
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									0	0	0
Interest Expense									(4,668)	(4,891)	(4,858)
Net interest income									17,272	19,838	23,259
Guaranty Fee Income									0	0	0
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									1,622	(2,113)	(2,272)
Non-interest Income/(loss)									1,622	(2,113)	(2,272)
(Release of/(provision for) loan losses									0	0	0
Provision for/(release of) losses									0	0	0
Noninterest Expense, Excluding Loan, Lease and Other Losses									(9,751)	(10,077)	(9,259)
Noninterest Expense									(9,751)	(10,077)	(9,259)
Segment core earnings before income taxes									9,143	7,648	11,728
Income tax (benefit)/expense									8,991	7,057	5,724
Segment core earnings before preferred stock dividends and attribution of income to non-controlling interest - preferred stock dividends									18,134	14,705	17,452
Preferred stock dividends									(3,495)	(2,879)	(2,879)
Less: Net Income attributable to non-controlling Interest - preferred stock dividends									(22,187)	(22,187)	(22,187)
Segment core earnings									(7,548)	(10,361)	(7,614)
Total assets at carrying value	3,316,904				3,392,226				3,316,904	3,392,226	3,068,980
On and Off Balance Sheet Program Assets, At Principal Balance	0								0		0
Reconciling Adjustments [Member]
Segment Reporting Information [Line Items]
Total interest income									(20,417)	(11,831)	(10,046)
Interest income related to consolidated trusts owned by third parties reclassified to guarantee fee income									964	1,659	3,269
Interest Expense									12,324	25,595	38,663
Net interest income									(7,129)	15,423	31,886
Guaranty Fee Income									(964)	(1,659)	(3,269)
Other Non-Interest Income (Expense), Excluding Guarantee and Commitment Fees									34,156	(22,607)	(73,422)
Non-interest Income/(loss)									33,192	(24,266)	(76,691)
(Release of/(provision for) loan losses									0	0	0
Provision for/(release of) losses									0	0	0
Noninterest Expense, Excluding Loan, Lease and Other Losses									0	0	0
Noninterest Expense									0	0	0
Segment core earnings before income taxes									26,063	(8,843)	(44,805)
Income tax (benefit)/expense									(9,122)	3,095	15,682
Segment core earnings before preferred stock dividends and attribution of income to non-controlling interest - preferred stock dividends									16,941	(5,748)	(29,123)
Preferred stock dividends									0	0	0
Less: Net Income attributable to non-controlling Interest - preferred stock dividends									0	0	0
Segment core earnings									16,941	(5,748)	(29,123)
Total assets at carrying value	0				0				0	0	0
On and Off Balance Sheet Program Assets, At Principal Balance	$ 0				$ 0				$ 0	$ 0	$ 0