At the end of the film, after the marshals have captured Sheridan and he is in police custody in the hospital, Gerard realizes that John Royce (Robert Downey Jr.) has been a double agent due to the serial number being scratched off Sheridan's gun. However, exactly how Gerard comes to this conclusion is a little complex, and is explained very quickly in the film.
When Gerard first meets Royce, he asks him if he carries a weapon. Royce says he does, and Gerard asks to see it. Royce produces a Taurus PT945, to which Gerard responds, "Get yourself a Glock. Lose that nickel plated sissy pistol."
The gun is next seen in the standoff in the swamp. Using a Ruger SP-101, Sheridan surprises Royce and takes him hostage, also taking possession of Royce's Taurus, which he uses to shoot Gerard. Royce then manages to retake his Taurus whilst Sheridan escapes.
The gun is next seen in the nursing home, where Royce is about to use it to kill Sheridan, before being interrupted by Newman, who he then shoots with the Taurus. At this point, presumably, Royce gives Gerard the gun as a piece of evidence, telling him that the gun belonged to Sheridan.
Then, later, when Gerard and Royce are sitting outside the hospital room, Gerard is examining the Taurus (which Royce again refers to as "the gun that shot Newman") when he notices that the serial number has been filed off. He then thinks back to his initial meeting with Royce, where he told him to get rid of the Taurus; obviously Gerard has now realized that the gun which shot Newman is in fact Royce's gun. Sheridan only ever had a Taurus during the brief encounter in the swamp. When Gerard interrupts Royce about to kill Sheridan, Gerard asks him "Do you want to use your old gun," indicating the Taurus in the evidence bag. Royce says that the gun isn't his, it's Sheridan's, to which Gerard responds, "No, it's yours. You just filed off the serial number. It's the gun he took away from you in the swamp. The gun he shot me with, the one which you've been carrying since that day. What do you want to bet that bullets pulled out of my vest match the ones that killed Newman?"
There were never two Taurus' in play - there was only ever one: Royce's. In an attempt to ensure no one would realize that the gun used to shoot Newman was his gun, Royce filed off the serial number and presented it as evidence, however it is this very precaution which ultimately causes his failure, as it is when Gerard sees the filed off number that he realizes there was only ever one Taurus.
Some fans have speculated that perhaps Gerard didn't realize Royce got his gun back in the swamp, thinking that he only retook possession of it in the nursing home. However, this cannot be so, as if it were, Royce would not need to file off the serial number, as everyone would know that it was his gun that was used to shoot Newman, as Sheridan had stolen it from him,
The IMFDb has plenty of information on this subject, and some excellent stills from the film; go here for more information.