Loans and Leases Receivable and Allowance for Loan and Lease Losses - Schedule of Loan Purchases and Sales (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Accounts, Notes, Loans and Financing Receivable [Line Items]
Payments to acquire loans receivable	$ 1,164,119	$ 398,468	$ 264,090
Value at the time of sale of loans	$ 268,872	$ 107,119	$ 458,192
Loans held-for-investment, purchase price as a percentage of loans outstanding	100.30%	99.90%	99.40%
Gain (loss) on sale of SBA and other loans	$ 2,770	$ 3,294	$ 4,223
Residential real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Payments to acquire loans receivable	105,858	368,402	264,090
Value at the time of sale of loans	230,285	0	191,574
Other consumer
Accounts, Notes, Loans and Financing Receivable [Line Items]
Payments to acquire loans receivable	1,058,261	30,066	0
Multi-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Value at the time of sale of loans	0	54,638	226,831
Commercial and industrial (including owner occupied commercial real estate)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Value at the time of sale of loans	22,267	32,263	19,974
Commercial real estate owner occupied
Accounts, Notes, Loans and Financing Receivable [Line Items]
Value at the time of sale of loans	$ 16,320	$ 20,218	$ 19,813