Fair Value of Financial Instruments - Significant Unobservable Inputs Used in Fair Value Measurement (Details) - USD ($) $ in Thousands	6 Months Ended	12 Months Ended
	Jun. 30, 2016	Dec. 31, 2015
Nonrecurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Impaired loans	$ 16,441	$ 8,447
Other real estate	356	1,847
Residential mortgage servicing rights	64,899	40,713
SBA servicing rights	2,155	2,132
Recurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
IRLCs	8,738	4,022
Quoted Prices in Active Markets for Identical Assets (Level 1) | Nonrecurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Impaired loans	0	0
Other real estate	0	0
Residential mortgage servicing rights	0	0
SBA servicing rights	0	0
Quoted Prices in Active Markets for Identical Assets (Level 1) | Recurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
IRLCs	0	0
Significant Other Observable Inputs (Level 2) | Nonrecurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Impaired loans	0	0
Other real estate	0	0
Residential mortgage servicing rights	0	0
SBA servicing rights	0	0
Significant Other Observable Inputs (Level 2) | Recurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
IRLCs	$ 0	$ 0
Significant Unobservable Inputs (Level 3) | Impaired Loans | Minimum | Discounted Appraisals
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, estimated selling costs	0.00%	0.00%
Significant Unobservable Inputs (Level 3) | Impaired Loans | Maximum | Discounted Appraisals
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, estimated selling costs	10.00%	10.00%
Significant Unobservable Inputs (Level 3) | Nonrecurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Impaired loans	$ 16,441	$ 8,447
Other real estate	356	1,847
Residential mortgage servicing rights	64,899	40,713
SBA servicing rights	2,155	2,132
Significant Unobservable Inputs (Level 3) | Nonrecurring | Impaired Loans
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Impaired loans	$ 16,441	$ 8,447
Significant Unobservable Inputs (Level 3) | Nonrecurring | Impaired Loans | Weighted Average | Discounted Appraisals
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, estimated selling costs	9.48%	9.70%
Significant Unobservable Inputs (Level 3) | Nonrecurring | Other Real Estate
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Other real estate	$ 356	$ 1,847
Significant Unobservable Inputs (Level 3) | Nonrecurring | Other Real Estate | Minimum | Discounted Appraisals
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, estimated selling costs	0.00%	0.00%
Significant Unobservable Inputs (Level 3) | Nonrecurring | Other Real Estate | Maximum | Discounted Appraisals
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, estimated selling costs	10.00%	10.00%
Significant Unobservable Inputs (Level 3) | Nonrecurring | Other Real Estate | Weighted Average | Discounted Appraisals
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, estimated selling costs	8.51%	8.29%
Significant Unobservable Inputs (Level 3) | Nonrecurring | Mortgage Servicing Rights
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Residential mortgage servicing rights	$ 64,899	$ 40,713
Significant Unobservable Inputs (Level 3) | Nonrecurring | Mortgage Servicing Rights | Minimum | Discounted Cash Flows
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, discount rate	9.15%	9.75%
Unobservable inputs, prepayment speeds	13.34%	7.56%
Significant Unobservable Inputs (Level 3) | Nonrecurring | Mortgage Servicing Rights | Maximum | Discounted Cash Flows
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, discount rate	10.63%	12.50%
Unobservable inputs, prepayment speeds	22.54%	15.24%
Significant Unobservable Inputs (Level 3) | Nonrecurring | Mortgage Servicing Rights | Weighted Average | Discounted Cash Flows
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, discount rate	9.39%	10.06%
Unobservable inputs, prepayment speeds	11.48%	8.66%
Significant Unobservable Inputs (Level 3) | Nonrecurring | SBA loan servicing rights
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
SBA servicing rights	$ 2,155	$ 2,132
Significant Unobservable Inputs (Level 3) | Nonrecurring | SBA loan servicing rights | Discounted Cash Flows
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, discount rate	12.62%	13.25%
Unobservable inputs, prepayment speeds	8.37%	9.41%
Significant Unobservable Inputs (Level 3) | Recurring
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
IRLCs	$ 8,738	$ 4,022
Significant Unobservable Inputs (Level 3) | Recurring | IRLCs
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
IRLCs	$ 7,935	$ 3,001
Significant Unobservable Inputs (Level 3) | Recurring | IRLCs | Pricing Model
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, pull-through rate	80.00%	80.00%
Significant Unobservable Inputs (Level 3) | Recurring | Forward Commitments
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Forward commitments	$ (4,238)	$ 369
Significant Unobservable Inputs (Level 3) | Recurring | Forward Commitments | Minimum | Investor Pricing
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, pricing spreads	90.00%	90.00%
Significant Unobservable Inputs (Level 3) | Recurring | Forward Commitments | Maximum | Investor Pricing
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, pricing spreads	106.11%	106.20%
Significant Unobservable Inputs (Level 3) | Recurring | Forward Commitments | Weighted Average | Investor Pricing
Significant unobservable inputs used in the fair value measurement of level 3 assets and liabilities
Unobservable inputs, pricing spreads	103.74%	102.67%