Certain Transfers of Financial Assets - Asset Quality of SBA Loans (Details) - SBA $ in Thousands	12 Months Ended
	Dec. 31, 2017 USD ($)
Servicing Assets at Fair Value [Line Items]
Unpaid Principal Balance	$ 421,690
Delinquent 30 to 89 days	13,515
Delinquent 90 plus Days	4,423
Net Charge-offs for the Year Ended December 31, 2017	160
Serviced for others
Servicing Assets at Fair Value [Line Items]
Unpaid Principal Balance	261,425
Delinquent 30 to 89 days	8,210
Delinquent 90 plus Days	1,957
Net Charge-offs for the Year Ended December 31, 2017	0
Held-for-sale
Servicing Assets at Fair Value [Line Items]
Unpaid Principal Balance	13,615
Delinquent 30 to 89 days	0
Delinquent 90 plus Days	0
Net Charge-offs for the Year Ended December 31, 2017	0
Held-for-investment
Servicing Assets at Fair Value [Line Items]
Unpaid Principal Balance	146,650
Delinquent 30 to 89 days	5,305
Delinquent 90 plus Days	2,466
Net Charge-offs for the Year Ended December 31, 2017	$ 160