Other sectors were hit as well, most notably energy and utilities. The S&P 500 energy sector index .GSPE fell 0.8 percent and the utilities sector .GSPU was off 1.1 percent.
The day's decline came a session after the S&P 500 finished at a five-year high, boosted by a budget deal and strong economic data. The S&P 500 rose 4.6 percent last week, the best weekly gain in more than a year.
"It's a little bit of taking some risk off the table ahead of profit season, you're not going to see anything all that great" on earnings, said Larry Peruzzi, senior equity trader at Cabrera Capital Markets Inc in Boston.
Earnings are expected to be only slightly better than the third-quarter's lackluster results, and analysts' current estimates are down sharply from where they were in October. Fourth-quarter earnings growth is expected to come in at 2.8 percent, according to Thomson Reuters data.
Aluminum company Alcoa Inc (AA.N) begins the reporting season by announcing its results after Tuesday's market close. Alcoa shares fell 1.7 percent at $9.10.
The Dow Jones industrial average finance/markets/index?symbol=us%21dji">.DJI dropped 50.92 points, or 0.38 percent, to 13,384.29. The Standard & Poor's 500 Index .SPX fell 4.58 points, or 0.31 percent, to 1,461.89. The Nasdaq Composite Index .IXIC lost 2.84 points, or 0.09 percent, to 3,098.81.
Ten mortgage servicers - including Bank of America (BAC.N), Citigroup (C.N), JPMorgan (JPM.N), and Wells Fargo (WFC.N) - agreed on Monday to pay $8.5 billion to end a case-by-case review of foreclosures required by U.S. regulators.
In a separate case, Bank of America also announced roughly $11.6 billion of settlements with mortgage finance company Fannie Mae and a $1.8 billion sale of collection rights on home loans.
The bank also entered into agreements with Nationstar Mortgage Holdings (NSM.N) and Walter Investment Management (WAC.A) to sell about $306 billion of residential mortgage servicing rights.
Bank of America shares lost 0.2 percent at $12.09 while Nationstar Mortgage Holdings (NSM.N) jumped 16.8 percent to $38.83.
Citigroup shares were up 0.09 percent to $42.47, and Wells Fargo shares fell 0.5 percent to $34.77.
"The financials probably have the wind behind them now with a lot of the regulations coming out ... the market has to absorb a lot of the gains, and for that reason there's a pullback from this level," said Warren West, principal at Greentree Brokerage Services in Philadelphia.
Shares of U.S. jet maker Boeing Co (BA.N) dropped 2 percent after a Boeing 787 Dreamliner aircraft with no passengers on board caught fire at Boston's Logan International Airport on Monday morning.
Amazon.com (AMZN.O) shares hit their highest price ever at $269.22 after Morgan Stanley raised is rating on the stock. Shares were up 3.6 percent at $268.46.
Video-streaming service Netflix Inc (NFLX.O) shares gained 3.4 percent to $99.20 after it said it will carry previous seasons of some popular shows produced by Time Warner's (TWX.N) Warner Bros Television.
Walt Disney Co (DIS.N) stock fell 2.3 percent to $50.97. The company started an internal cost-cutting review several weeks ago that may include layoffs at its studio and other units, three people with knowledge of the effort told Reuters.
Volume was lower than average, as 4.78 billion shares were traded on the New York Stock Exchange, NYSE MKT and Nasdaq. This is well below the 2012 average of 6.42 billion per session.
Declining stocks outnumbered advancing ones on the NYSE by 1,629 to 1,363, while on the Nasdaq decliners beat advancers 1,438 to 1,066.
