<?xml version="1.0" encoding="us-ascii"?><InstanceReport xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><Version>2.2.0.25</Version><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><ReportLongName>10201 - Disclosure - Summary of Significant Accounting Policies</ReportLongName><DisplayLabelColumn>true</DisplayLabelColumn><ShowElementNames>false</ShowElementNames><RoundingOption /><HasEmbeddedReports>false</HasEmbeddedReports><Columns><Column><Id>1</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelColumn>false</LabelColumn><CurrencyCode>USD</CurrencyCode><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName>1/1/2011 - 3/31/2011
USD ($) / shares

USD ($)
</KeyName><CurrencySymbol>$</CurrencySymbol><contextRef><ContextID>Duration_1_1_2011_To_3_31_2011</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0001419521</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2011-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2011-03-31T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS><UnitProperty><UnitID>Unit13</UnitID><UnitType>Divide</UnitType><NumeratorMeasure><MeasureSchema>http://www.xbrl.org/2003/iso4217</MeasureSchema><MeasureValue>USD</MeasureValue><MeasureNamespace>iso4217</MeasureNamespace></NumeratorMeasure><DenominatorMeasure><MeasureSchema>http://www.xbrl.org/2003/instance</MeasureSchema><MeasureValue>shares</MeasureValue><MeasureNamespace>xbrli</MeasureNamespace></DenominatorMeasure><Scale>0</Scale></UnitProperty><UnitProperty><UnitID>Unit12</UnitID><UnitType>Standard</UnitType><StandardMeasure><MeasureSchema>http://www.xbrl.org/2003/iso4217</MeasureSchema><MeasureValue>USD</MeasureValue><MeasureNamespace>iso4217</MeasureNamespace></StandardMeasure><Scale>0</Scale></UnitProperty></UPS><CurrencyCode>USD</CurrencyCode><OriginalCurrencyCode>USD</OriginalCurrencyCode></MCU><CurrencySymbol>$</CurrencySymbol><Labels><Label Id="1" Label="3 Months Ended" /><Label Id="2" Label="Mar. 31, 2011" /></Labels></Column></Columns><Rows><Row><Id>2</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><Level>0</Level><ElementName>us-gaap_GeneralPoliciesAbstract</ElementName><ElementPrefix>us-gaap</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><ShortDefinition>No definition available.</ShortDefinition><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsSubReportEnd>false</IsSubReportEnd><IsCalendarTitle>false</IsCalendarTitle><IsTuple>false</IsTuple><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole /><FootnoteIndexer /><Cells><Cell><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><NonNumericTextHeader /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios></Cell></Cells><OriginalInstanceReportColumns /><Unit>Other</Unit><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><ElementDefenition>No definition available.</ElementDefenition><IsTotalLabel>false</IsTotalLabel><IsEPS>false</IsEPS><Label>Summary of Significant Accounting Policies</Label></Row><Row><Id>3</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><Level>0</Level><ElementName>us-gaap_SignificantAccountingPoliciesTextBlock</ElementName><ElementPrefix>us-gaap</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><ShortDefinition>No definition available.</ShortDefinition><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsSubReportEnd>false</IsSubReportEnd><IsCalendarTitle>false</IsCalendarTitle><IsTuple>false</IsTuple><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terselabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText>&lt;div&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;2. Summary of Significant Accounting Policies &lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Basis of Presentation and Use of Estimates &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The accompanying unaudited financial statements have been prepared in accordance with U.S. generally accepted accounting principles ("GAAP") for interim financial information and with the instructions to Form 10-Q and Rule&amp;nbsp;10-01 of Regulation&amp;nbsp;S-X. Accordingly, they do not include all of the information and footnotes required by GAAP for complete financial statements. In the opinion of management, all adjustments (consisting of normal recurring accruals) considered necessary for a fair presentation have been included. Operating results for the three months ended March&amp;nbsp;31,&amp;nbsp;2011 are not necessarily indicative of the results that may be expected for the calendar year ending December&amp;nbsp;31,&amp;nbsp;2011. These unaudited financial statements should be read in conjunction with the audited financial statements and notes thereto included in the Company's annual report on Form 10-K for the year ended December&amp;nbsp;31, 2010. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 12px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The preparation of financial statements in conformity with GAAP requires management to make estimates and assumptions that affect the reported amounts of assets and liabilities and disclosure of contingent assets and liabilities at the date of the financial statements and the reported amounts of revenues and expenses during the reporting period. Actual results could differ from those estimates. Significant estimates affecting the accompanying financial statements include the valuation of MBS and derivative instruments. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Financial Instruments &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The Company considers its cash and cash equivalents, restricted cash, MBS (settled and unsettled), forward purchase commitments, debt security held to maturity, accrued interest receivable, accounts payable, derivative instruments, repurchase agreements and accrued interest payable to meet the definition of financial instruments. The carrying amount of cash and cash equivalents, restricted cash, accrued interest receivable and accounts payable approximate their fair value due to the short maturities of these instruments. See Note 4 for discussion of the fair value of MBS and forward purchase commitments. See Note 5 for discussion of the fair value of the held to maturity debt security. See Note 7 for discussion of the fair value of derivatives. The carrying amount of the repurchase agreements and accrued interest payable is deemed to approximate fair value since the lines are based upon a variable rate of interest. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 12px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;Since inception through March&amp;nbsp;31, 2011, the Company has limited its exposure to credit losses on its portfolio of securities by purchasing MBS guaranteed by Fannie Mae and Freddie Mac. The portfolio is diversified to avoid undue exposure to loan originator, geographic and other types of concentration. The Company manages the risk of prepayments of the underlying mortgages by creating a diversified portfolio with a variety of prepayment characteristics. See Note 4 for additional information on MBS. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 12px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The Company is engaged in various trading and brokerage activities including derivative interest rate swap agreements in which counterparties primarily include broker-dealers, banks, and other financial institutions. In the event counterparties do not fulfill their obligations, the Company may be exposed to risk of loss. The risk of default depends on the creditworthiness of the counterparty and/or issuer of the instrument. It is the Company's policy to review, as necessary, the credit standing for each counterparty. See Note 7 for additional information on interest rate swap agreements. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Mortgage-Backed Securities &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The Company invests in Agency MBS representing interests in or obligations backed by pools of single-family adjustable-rate mortgage loans. Guidance under the Financial Accounting Standards Board ("FASB") Accounting Standards Codification ("ASC") Topic on Investments requires the Company to classify its investments as either trading, available-for-sale or held-to-maturity securities. Management determines the appropriate classifications of the securities at the time they are acquired and evaluates the appropriateness of such classifications at each balance sheet date. The Company currently classifies all of its Agency MBS as available-for-sale. All assets that are classified as available-for-sale are carried at fair value and unrealized gains and losses are included in other comprehensive income or loss as a component of shareholders' equity. The estimated fair values of MBS are determined by management by obtaining valuations for its MBS from at least three separate and independent sources and averaging these valuations. Security purchase and sale transactions are recorded on the trade date. Gains or losses realized from the sale of securities are included in income and are determined using the specific identification method. Firm purchase commitments to acquire "when issued" or to-be-announced ("TBA") securities are recorded at fair value in accordance with ASC Topic 815, &lt;i&gt;Derivatives and Hedging. &lt;/i&gt;The fair value of these purchase commitments is included in other assets or liabilities in the accompanying balance sheets. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 12px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The Company assesses its investment securities for other-than-temporary impairment on at least a quarterly basis.&amp;nbsp;When the fair value of an investment is less than its amortized cost at the balance sheet date of the reporting period for which impairment is assessed, the impairment is designated as either "temporary" or "other-than-temporary."&amp;nbsp;In deciding on whether or not a security is other than temporarily impaired, the Company uses a two step evaluation process. First, the Company determines whether it has made any decision to sell a security that is in an unrealized loss position, or, if not the Company determines whether it is more likely than not that the Company will be required to sell the security prior to recovering its amortized cost basis. If the answer to either of these questions is "yes" then the security is considered other-than-temporarily impaired. There were no such impairment losses recognized during the periods presented. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Interest Income &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;Interest income is earned and recognized based on the outstanding principal amount of the investment securities and their contractual terms. Premiums and discounts associated with the purchase of the investment securities are amortized or accreted into interest income over the actual lives of the securities using the effective interest method. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Income Taxes &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;The Company has elected to be taxed as a REIT under the Code. The Company will generally not be subject to Federal income tax to the extent that it distributes 100% of its taxable income, after application of available tax attributes, within the time limits prescribed by the Code and as long as it satisfies the ongoing REIT requirements including meeting certain asset, income and stock ownership tests. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Share-Based Compensation &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 6px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;Share-based compensation is accounted for under the guidance included in the ASC Topic on Stock Compensation. For share and share-based awards issued to employees, a compensation charge is recorded in earnings based on the fair value of the award. For transactions with non-employees in which services are performed in exchange for the Company's common stock or other equity instruments, the transactions are recorded on the basis of the fair value of the service received or the fair value of the equity instruments issued, whichever is more readily measurable at the date of issuance. The Company's share-based compensation transactions resulted in compensation expense of $207 and $342 for the three months ended March&amp;nbsp;31, 2011 and 2010, respectively. &lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 18px; margin-bottom: 0px; margin-left: 2%;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;&lt;b&gt;&lt;i&gt;Earnings Per Common Share (EPS) &lt;/i&gt;&lt;/b&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p style="margin-top: 12px; margin-bottom: 0px; font-size: 1px;"&gt;&amp;nbsp;&lt;/p&gt;

&lt;p style="margin-top: 0px; margin-bottom: 0px;"&gt;&lt;font style="font-family: Times New Roman;" class="_mt" size="2"&gt;Basic EPS is computed by dividing net income available to holders of common stock by the weighted average number of shares of common stock outstanding during the period. Diluted EPS is computed using the two class method, as described in the ASC Topic on Earnings Per Share, which takes into account certain adjustments related to participating securities. Net income available to holders of common stock after deducting dividends on unvested participating securities if antidilutive, is divided by the weighted average shares of common stock and common equivalent shares outstanding during the period. For the diluted EPS calculation, common equivalent shares outstanding includes the weighted average number of shares of common stock outstanding adjusted for the effect of dilutive unexercised stock options. &lt;/font&gt;&lt;/p&gt; &lt;/div&gt;</NonNumbericText><NonNumericTextHeader>2. Summary of Significant Accounting Policies

Basis of Presentation and Use of Estimates

The accompanying unaudited financial statements have been prepared</NonNumericTextHeader><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios></Cell></Cells><OriginalInstanceReportColumns /><Unit>Other</Unit><ElementDataType>us-types:textBlockItemType</ElementDataType><SimpleDataType>string</SimpleDataType><ElementDefenition>This element may be used to describe all significant accounting policies of the reporting entity.</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef
 -Publisher AICPA
 -Name Accounting Principles Board Opinion (APB)
 -Number 22
 -Paragraph 8

</ElementReferences><IsTotalLabel>false</IsTotalLabel><IsEPS>false</IsEPS><Label>Summary of Significant Accounting Policies</Label></Row></Rows><Footnotes /><NumberOfCols>1</NumberOfCols><NumberOfRows>2</NumberOfRows><ReportName>Summary of Significant Accounting Policies</ReportName><MonetaryRoundingLevel>UnKnown</MonetaryRoundingLevel><SharesRoundingLevel>UnKnown</SharesRoundingLevel><PerShareRoundingLevel>UnKnown</PerShareRoundingLevel><ExchangeRateRoundingLevel>UnKnown</ExchangeRateRoundingLevel><HasCustomUnits>false</HasCustomUnits><SharesShouldBeRounded>true</SharesShouldBeRounded></InstanceReport>
