The GAO notes that public records with SSNs appear in many formats but mostly in paper records filed at agency offices. "Few state agencies make records with SSNs available on the Internet; however, 15 to 28 percent of the nation’s 3,141 counties do place them on the Internet and this could affect millions of people." [Emphasis is mine.]
The GAO survey of counties revealed "that more than three-quarters of U.S. counties hold at least one type of record that displays SSNs, which has implications for the 94 percent of the U.S. population that we estimate live in those counties." This is good news for information gatherers who need to locate people or verify their identity for employment background or litigation.
Interestingly, agencies collect and retain SSNs for some of the same reasons investigators rely on them: to verify identity and to cross reference with other records. The federal government has taken steps to remove SSNs from various identity cards and financial statements, which undoubtedly does reduce access to personal information that can be used by identity thieves. However, a simple utility bill that ends up in the trash, unshreaded, may provide all the leads needed for a nefarious purpose.
Additionally, it's doubtful that identity theft comes about through mining public records. The GAO concludes as much:
The continued visibility of SSNs in public records in virtually every corner of the country presents continued risk of widespread, albeit small-scale, identity theft. Since the public usually obtains such records in individual hard copies, the risk of SSN theft in large volume from public records may be small. [Emphasis is mine.]
I'd be curious to know if there are any legal cases citing an identified criminal conspiracy, in the form of an identity theft gang. In any case, like gun ownership, there are people who should be allowed to have them and others who should not. The public would benefit if privacy advocates and law makers recognized that private investigators are a resource in preventing and ameliorating identity theft, not a threat to individual privacy and security.