Draft as submitted.
Published as Chapter 15 of
G. David Garson (ed) Handbook of Public Information Systems.
(NY: Marcel Dekker, 2000.)
By Jeremy R. T. Lewis
Huntingdon College, Montgomery Alabama
I. INTRODUCTION TO FEDERAL INFORMATION POLICY BEFORE 1996.
Federal information policy rests on legal foundations of the First Amendment's guarantee of unregulated speech, the Copyright Act's section 105 (which bans official copyright), the Freedom of Information Act (FOIA), and perhaps the 1995 Paperwork Reduction Act (which acknowledges that government publication is an essential function).
Codified at 5 USC 552, the FOIA (with its minor cousins the 1974 Privacy Act and 1976 Government in the Sunshine Act, plus the Federal Advisory Committee Act and 1978 Presidential Records Act) modifies section 3 of the 1946 Administrative Procedures Act. The FOIA--copied to some degree by all 50 states --applies to all federal agencies and requires in section (a) that (subject to judicial review de novo) they publish agency procedures and manuals, and release to the public upon request all agency records except those listed in its section (b). The main exemptions are for properly classified national security material, specifically conflicting statutes, confidential commercial material, some internal memos, personal privacy, and (under listed circumstances) law enforcement files.
Generally the FOIA has proved much more important and successful than expected--while also stimulating far more administrative problems and lawsuits than anticipated. Although half a dozen agencies have massive FOIA programs with tens of thousands of requests and year-long backlogs, about 30 of the 100 agencies have only a minor workload under the FOIA. In particular, where documents can be released without line-by-line review and redaction, the workload per request may be quite light. At eight cabinet departments, one U.S. House study (1986a p. 6 n.2; Gellman, 1997, p. 9 n. 3) found over 90% of requesters received all they requested. (The Department of State's figure, however, was far below that of the other seven agencies reviewed.)
The working of the FOIA prior to 1996 was limited (Gellman, 1997, p. 2) by official attitudes, lack of staff, and budget. Amending regulations after the 1986 FOIA took up to 6 years. To these we might add agency priority: FOIA processing is considered a poor career track compared to work focused on the agency's central mission. Time limits have not been enforced and sanctions left unused.
Claims that classified material would be released to foreign powers have not been borne out: in fact, courts have never required that any material claimed to be classified must be released. At trial, although they have complained often about agency practices, they have consistently deferred to agency judgment about protecting national security material.
In the mid-1980s, the issue of electronic records was explored by subunits of the U.S. House Government Operations Committee and the U.S. Office of Management and Budget. (see the chapter on the emergence of federal information policy). As in previous decade-long cycles of information policy, proposals were formulated and investigated from 1990 to 1995 and enacted suddenly in 1996.
II. THE POLITICAL ENVIRONMENT OF THE 1990S
The beginning of the decade was marked by concerns for the federal budget deficit, with much administration attention diverted abroad to the end of the Cold War, the collapse of the Soviet empire, and the development of a golden era of emergent democracies in Eastern Europe and Latin America. President Bush began the decade with high approval ratings during the Gulf War, and consistent with his pledge to avoid new taxes, only to lose office in the 1992 election when a sluggish economy, an agreement to raise taxes, and a lack of domestic policy caught up with him. The Clinton administration presented slightly better opportunities for those favoring open government; on the other hand, the "Republican revolution" in the House in 1995-1999 made the legislative climate for FOIA volatile and unpredictable.
III. PREPARING THE GROUND FOR EFOIA
Like its predecessors, the 1996 EFOIA emerged only after extensive investigation, preparation, and compromise. In 1980-1986 the Senate's policy on public records depended on tension between a subcommittee (of judiciary) on the Constitution chaired by Sen. Orrin Hatch (R-Utah) and that on Technology and the Law chaired by Sen. Patrick Leahy (D-VT).
Until 1986, the FOIA had been debated quite separately from the automation of federal offices. Then under Rep. Glenn English, (D-OK) a set of 1985 U.S. House hearings and report combined with the OMB Circular A-130 to raise the central questions of electronic information policy for the 1990s. The House Subcommittee on Government Information (of the Government Operations Committee) held hearings, in 1985, on electronic collection and dissemination of information by federal agencies, limited (Report, 1986a, n. 1, p. 2) to databases normally open to the public under access laws. Fearing that electronic data systems would increase agencies' control of government information, the 1986 House report based on those hearings sought to maintain a role for the private sector. The Report (1986a, pp. 1-2) also found, "There is a risk that agencies may be able to exert greater control over information in electronic information systems than is possible with data maintained in traditional, hard-copy formats.... Legal ambiguities, practical limitations, and economic constraints may allow federal agencies to restrict unduly the public availability of government data maintained electronically."
Under the subsequent chair, Rep. Bob Wise (D-WV), the recommendations (U.S. House, 1990b) joined the Paperwork Reduction Bill, which passed the House in 1990 only to fail in the Senate on other grounds, until passage of the provisions in similar form in 1995. The counsel drafting the language (Gellman, 1997, p. 4) later commented ruefully on the technological obsolescence of the Act: it required public notice before initiation of a new information dissemination product--unfeasible in the era of daily changes of Web home pages.
OMB's Office of Information and Regulatory Affairs (OIRA) was granted extraordinary control over executive regulations through the 1980 Paperwork Reduction Act and a Reagan administration mandate to review all proposed regulations. Then in 1985, OMB's Circular A-130, "The Management of Federal Information Resources," raised adaptation to the digital information age, and established terms of debate about the public and private sectors in digital publication and database control. It argued (A-130; also Sprehe, interview) for avoiding monopolies of information, for giving value-added resellers a role in the private sector, for the contracting out of databases, and for continuance of inexpensive public access to official records.
While the judicial record on requiring electronic records to be released under the FOIA was mixed, there was some prospect of progress without amending the law (Lewis, 1995, pp. 434-435). The Supreme Court had let stand a decision in Long v. IRS (1979) that computer-stored records are still records under the FOIA. Then in 1980 the Supreme Court had found in Forsham v. Harris that records include machine readable materials. Another court in Yeager v. DEA (1982) had maintained no distinction between manual and computer storage systems. However, there was no major ruling requiring the generation of a report from a database, which could still be interpreted as creating a new record, hence not required by the FOIA. On the other hand, a much criticized decision had upheld the right of the agency to choose to release microfiche in place of the requested computer tape. In 1989, a district court in Armstrong v. Bush decided that National Security Council e-mail records were distinct from telephone conversations, and on remand (sub nom. Armstrong v. EOP) it found that digital files contained information not held by hard copy.
Rep. Bob Wise (D-WV) proposed an electronic version of the FOIA during his chairmanship of the House Government Information Subcommittee in the late 1980s, and Senator Patrick Leahy (D-VT) of the Subcommittee on Technology and the Law sponsored an Electronic FOI Improvement Act (S. 1940, and later S. 1782) to reduce delays in processing. A landmark bill was the EFOIA, S. 1940 introduced by Senators Leahy (D-VT) and Hank Brown (R-CO) on November 7, 1991. (Its prospects were--correctly--rated poorly by Westlaw: a 33% chance of passing the Senate Judiciary Committee and 29% the Floor; 8% for passing the House committee and only 7% the House Floor) (Marke, 1992, p. 1). Leahy's early draft proposed that agencies complying with time limits should retain half the FOIA fees they collected. That draft proposed penalties for undue agency delays: courts could order payment of out-of-pocket expenses up to $75 per day; agencies objections, however, led to this being dropped in later versions. The bill made it explicit that agency records included electronic records and defined a search as "a manual or automated examination to locate records," (Documents from the Senate Subcommittee on Technology and the Law. See also Minahan, 1994.) It proposed that computer capabilities be rated and that agencies able to deliver data in computer formats do so when requested. Other proposals were considered to build a record in the Senate in the hope that in the future conditions in both chambers would favor further legislation.
Agencies responded to the Office of Management and Budget with their usual opposition to greater administrative burdens, except for supporting the proposed partial retention of the fees collected from requesters. Requiring them to supply records in a reasonably convenient format would, they argued (Hammitt, 1994), impose delays and reduce the economies of scale found in delivering frequently requested material in a standard format. (Interviewed: Randall Rader, counsel to Sen. Hatch; Ann Harkins, Marc Rotenberg, and Beryl Howell, counsels to Sen. Leahy; Alan Adler of the ACLU and Patti Goldman of Public Citizen FOI Clearinghouse; staff of OIRA at OMB and of the House FOI subcommittee.)
The House bill, HR 4917, was sponsored by Rep. Maria Cantwell (D-WA) and Gary Condit (D-CA), but even passage in the Senate in 1994 could not move the House to action before its election adjournment. The 1994 election victory for House Republicans caused the dispersal of Democratic staff, breaking a long pattern of continuity on the FOIA.
IV. TRANSITION IN THE WHITE HOUSE, 1992-1994
During the Bush administration electronic FOIA bills languished. The president's veto threat was conveyed to the 1992 Senate Hearings (p. 99) by Steven R. Schlesinger, Director of the Office of Policy Development, Department of Justice.
In contrast, the Clinton administration reasserted the principle of open government being tied to new access technology. On February 22, 1993, the Vice President announced creation of the Government Information Locator System (GILS), an Internet index to materials publicly available at agencies. On June 25, 1993, Sally Katzen, of OMB's OIRA, in a revised Circular A-130, announced a policy reversing the Reagan administration's and favoring agency release of electronic information over having private sector contractors do the job. Agencies were requested to integrate CD-ROM and online indexing and publication with hard-copy processes, and to charge only for dissemination, not acquisition cost.
In an October 4, 1993 memo to agency heads, the President said: "This is an appropriate time for all agencies to take a fresh look at their administration of the Act, to reduce backlogs of Freedom of Information Act requests, and to conform agency practice to the new litigation guidance issued by the Attorney General ... Each agency has a responsibility to distribute information on its own initiative, and to enhance public access through the use of electronic information systems."
The Justice Department, surprisingly, also revised its guidance toward a more open policy. Janet Reno, Attorney General, in an October 4, 1993 circular rescinding the 1981 Justice guidelines, called for agencies to use broader discretion in releasing records, particularly those falling under the FOIA's exemption for internal memoranda.
OMB's draft guidelines of November 1994 set forth 66 principles, which were weaker than the Senate bill in requiring agencies to release digital records. They did abandon the Dismukes standard by which agencies could decide upon a release format, although many agencies by then had been voluntarily releasing data in a variety of formats. Unlike the Senate, which opted for requiring "reasonable" efforts to convert data into a previously unavailable but requested format, OMB suggested limiting format conversion to a meager 2 hours of labor time and to an overall agency resource limit.
However, the Clinton administration--following Armstrong v. EOP, a FOIA lawsuit seeking the Iran-contra affair e-mail records--also terminated counting the National Security Council an agency subject to the FOIA, until in 1995, Judge Charles Richey of the D.C. federal district court rejected the claim as spurious: the NSC was clearly an agency of government.
V. LEGISLATIVE VERSUS ADMINISTRATIVE STRATEGY
It was an open question whether agencies would gradually adopt EFOIA practices regardless of the passage of the bill. This author's survey (Lewis, 1995) of the early online redaction and dissemination systems in 1994-1995 certainly found rapid progress being triggered by the sudden development of the World Wide Web and the need to reduce agency FOIA processing costs. Mitre corporation (Cooley, 1994a and 1994b; Klopfenstein, 1993-1994) was planning a major interagency declassification online system (p.442); in 1993 Congress had mandated the Federal Register be published online; the National Library of Medicine had long made available its MEDLARS system; the EPA published online its Toxic Resource Inventory; the SEC had opened its massive online EDGAR database of corporate reports; the State Department had begun redacting text online for declassification and tracking documents online; the FDA was scanning backlogged documents into a database; and even Social Security with its low-tech, elderly clientele was scanning and tracking correspondence. Consequently, it was not clear that the Senate bill if passed would arrive in advance of administrative developments where resource scarcity and improved technology were the mothers of invention.
VI. JUNE 1996 HOUSE HEARINGS ON THE SENATE EFOIA BILL (S.1090)
Commentators on the EFOIA proposals continued to be divided on the wisdom of legislative rather than administrative and judicial strategies to reform. Less confident than before of solid congressional support for open government, Public Citizen counsel (Goldman, 1988) had long argued in favor of having bureaucracies and courts simply adapt the FOIA to newer technology.
Others argued that some provisions were already out of date: for example (Gellman, 1997, p. 4), the Federal Register was published online before the bill was passed. The bill's exemption of "stocks of documents" from FOIA was a poor choice of words in an era of networked data storage. As for e-mail, it was not treated comprehensively in the bill; as for online redaction of records (p. 5), the deletion and annotation software was not yet developed. The requirement to publish in "available" formats was not sharp enough for practical use.
The bill's requirement that agencies publish records "likely to become the subject of subsequent requests" (Gellman, 1997, pp. 5-6) could be misadministered and endlessly litigated. Section 6's requirement that agencies keep some FOIA fees where the comptroller general determined substantial compliance with time limits was unfortunately dependent on a reduced GAO staff and budget that might delay audits. Although the 1974 amendments "painted us into a corner" with one-size-fits-all time limits, the increase in these time limits would leave nobody better off. The provision that agencies not maintain routine backlogs--without additional funding--would oblige them to transfer resources from other areas.
Section 6(f)'s requirement that agencies establish multitrack processing for large and small, urgent and non-urgent requests (Gellman, 1997, p. 7) was criticized as reducing administrative discretion--and that author added, "I estimate that at least half of all requesters would ask for expedited access once allowed by law. There is no extra cost for seeking faster access." The qualification for expedited access--that the topic be of widespread media coverage and of the propriety of alleged governmental actions--was seen as so vague as to be meaningless. The provision might actually slow down FOIA responses.
Section 8 excluded library materials acquired solely for reference or exhibition purposes, language criticized (Gellman, 1997, pp. 8-9) as an improper import from the Records Disposal Act, wrongly relied on in the electronic data court decision SDC v. Mathews. Instead of legislative amendment, better implementation by the Justice Department's Office of Information and Privacy (OIP) was called for.
Americans for Tax Reform argued, "The Electronic Freedom of Information Improvement Act which passed the Senate on a voice vote last year is really about the minimum step one can make to update FOIA in light of a single fact: we really don't do much on paper anymore" (Lucier, 1996, p. 2).
Anyone who makes the improbable case that EFOIA will be too costly or too difficult for the agencies to implement misses the basic point that every day we go without this necessary update to the original 1966 legislation, we let federal agencies molder in a management environment that is decades behind best practices in the private sector. It costs the taxpayer and the economy at large huge amounts of money to maintain a bureaucracy that is ineffective and overly complex due to antiquated information practices. It also costs the government greatly in public confidence and perceived unresponsiveness, if not outright irrelevance, to the basic problems that trouble many Americans (Lucier, 1996, pp. 2-3).
In the old days, FOIA was supposed to be a citizen's defensive tool to keep all-powerful, bureaucratic, unaccountable government from doing things wrong. We should think of it more as a citizens' enabling tool. In the information age, EFOIA will be one tiny step [to] allow a limited, effective, responsive, participatory government to do what's right (Lucier, 1996, p. 3).
A former FOIA counsel for the ACLU (Adler, 1996, p. 2) also supported the EFOIA bill, while lamenting the tepid support from the Clinton administration: "In the present Congress, unsuccessful attempts to engage the Clinton administration in consideration of S. 1090 substantially delayed action on the bill in the Senate Judiciary Committee. Only after the bill was reported by the committee and pending for consideration by the Senate did the OMB begin to urge revisions in a number of the bill's provisions. The post-Committee negotiations with OMB have resulted in substantial revisions to S.1090 as reported by the Committee."
Adler (1996, p. 2) pointed to compromises in the EFOIA bill from 1994 to 1996 that should pacify critics of the draft.
For example, since Senate passage of an earlier version in 1994, provisions in the pending EFOIA bill that would require agencies to make certain materials and indices available to the public through computer telecommunications and other electronic means have been revised to ensure that agencies have reasonable transition periods for compliance. Similarly, provisions requiring agencies to honor requester format choices and identify redactions in computer-based records have been revised to ensure that compliance efforts will be reasonable and feasible.
The bill would take advantage of efficiencies in agency record keeping and FOIA processing that are expected to result from the greater use of technology in handling FOIA requests. Certain somewhat controversial provisions, such as the fee-related provisions which were included in previous versions of the bill, have been dropped from S. 1090. Others, including the various provisions addressing time limits for agency responses to FOIA requests have been greatly modified in an effort to ensure that the agencies would not be unduly burdened by the new requirements (Adler, 1996, p. 2).
VIII. CONGRESSIONAL ACTION
After some years of sporadic Senate consideration, most recently on August 25, 1996, with modification and passage through House committee in early September, the House suddenly passed its version on September 17, 1996, prompting the Senate to pass its own the same evening and the House version the next day. President Clinton signed the bill (PL 104-231, codified at 101 Stat 2422) on October 2, 1996. Most sections took effect 180 days after enactment, except section 7 (delays) and section 8 (expedited processing) which had one year of lead time.
While on the Senate side Sen. Leahy (D-VT) was still active in promoting EFOIA, the new House Republican majority of 1995 placed Rep. Steven Horn (R-CA) in the chair of the new House Government Reform and Oversight Subcommittee; his staff shared his interest in EFOIA. Speaker Gingrich (from Marietta, GA, a high-tech suburb of Atlanta) shared an interest in information technology and wanted to assist freshmen in creating achievements for their reelection in November 1996. The press maintained its longtime interest in FOIA legislation.
The EFOIA's House sponsor, Rep. Randy Tate (R-WA), had both a high-tech constituency and a difficult reelection campaign. The only opposition to the bill came from OMB, whose changes were rejected as unhelpful. But the bill was criticized by one former House staff director for being more a legislative "trophy" than a substantive bill. He felt, "The requester community agreed to some changes that are as bad as some Reagan administration proposals," (Gellman, 1996a, p. 4).
The legislative history of the 1966 FOIA was for a time compromised by a deal with the White House in which the Justice Department amended the House Report; eventually courts recognized the Senate Report as being a more accurate history of congressional intent. Similarly, the 1996 EFOIA amendments--despite years of consideration of the Leahy Senate bill--left a poor legislative record that might be exploited by the bill's opponents via the courts. The House adopted the Senate bill only after years of inaction, then rewrote it extensively. The Senate staff in turn adopted the House version in order to avoid a Joint Conference (Hammitt, 1996a, p. 2). Thus for much of the bill, the House report is more informative, yet for Senate provisions the Senate report is controlling.
Both chambers united on the purpose of the FOIA "for any public or private use." Senator Leahy (D-VT) explained that the objective was to overturn the Supreme Court's rulings in Reporters Committee (1989) and Defense v. FLRA. "The purpose of the FOIA is not limited to making agency records an information open to the public only in cases where such material would shed light on the activities and operations of Government.... [this] distorts the broader import of the Act in effectuating Government openness." Leahy proposed to return the purpose of the requester issue to the previous understanding: that it was only to be used in balancing disclosure against a privacy interest in withholding under section (b)(6) (Hammitt, 1996, pp. 2-3.) It was not clear, however, how this intent would be accepted by the courts.
VIII. PROVISIONS OF THE 1996 EFOIA
The 1996 EFOIA (Pub. L. No. 104-231, 110 Stat. 3048) recognized best agency practices on handling electronic records, adding useful reporting. It required reasonable efforts to search for and publish electronic records subject to technical limitations. In 5 USC 552 (a)(2) and (a)(3), new or frequently requested records are published on the Internet, with deletions marked, and indices to information systems. Agencies are to search for and supply records in the format requested, unless technically unfeasible under (a)(4)(B); or significantly interfering with operations under (C). Section (a)(6)(A) doubles the time limit, though (6)(B) elaborates negotiating with the requester for limited scope. (6)(C) limits delays for "exceptional circumstances" and permits multitrack processing of urgent or simple requests. (6)(E) orders expedited processing for those disseminating information urgently about alleged federal activity (the press). (6)(F) orders agency denials involve estimates of denied material. Under section (b), deletions are to be marked where technically feasible. In section (e), the Attorney General compiles an expanded report on agency practices and case law. Section (f) adds a definition of record including electronic format, and (g) requires public indices and guides.
IX. PRESIDENTIAL SIGNATURE
When President Clinton signed the EFOIA amendments into law on October 2, 1996, he noted, "The legislation I sign today brings FOIA into the information and electronic age by clarifying that it applies to records maintained in electronic format." He opined, "As the Government actively disseminates more information, I hope that there will be less need to use FOIA to obtain government information." He concluded (Hammitt, 1996a, p. 1), "our country was founded on democratic principles of openness and accountability, and for 30 years, FOIA has supported these principles. Today, the 'Electronic Freedom of Information Act Amendments of 1996' reforge an important link between the United States Government and the American people."
X. CRITICISM OF THE EFOIA PROVISIONS
Not all the FOIA lobby was satisfied that the 1996 EFOIA was an improvement. The former (majority Democrat) House FOI subcommittee staff director felt the issue should have been left until the following congress. (Gellman, 1996a, p. 3). He argued that the administrative provisions, although welcome, would make little difference in practice. Of the new emphasis on electronic records, he argued, "The spin on the law is considerably overblown. The truth is that agencies have been doing electronic searches and providing information on computer tape and disk for years. Most horror stories about denials of access to electronic records are old." Even so, he admitted (p. 5) that some agencies that were still relying on older case law such as SDC Development Corp v Mathews (1976) and Dismukes v Department of the Interior (1984) might be brought to heel. This he termed "the single most important part of the FOIA amendments."
However, he found the need to make reasonable efforts to search for electronic records "except where such efforts would significantly interfere with the operation of the agency's automated information system" was (Gellman, 1996a, p. 5) "a potential loophole of enormous significance ... unfortunate and inappropriate" and "likely to spark years of litigation." He called for OMB to require that "a senior official personally certif[y]" the interference as significant.
Fearing that multitrack processing would permit agencies to consign problematic requests to a dead track, he argued (Gellman, 1996a, p. 6) that it should better have been left to agency discretion, but called in the circumstances for OMB to exert leadership on standards.
The language on expedited access provision he found "more troublesome ... filled with vague concepts, undefined words and muddled objectives." The press, supposed to benefit but resisting definition by government, emerged under the rubric of "a person primarily engaged in disseminating information" and able to demonstrate "urgency to inform the public concerning actual or alleged Federal Government activity." He argued (Gellman, 1996, p. 7) that no one would benefit from the change in time limits, while--without new resources--the expedited process might draw scarce FOIA resources away from other requests. Most worrying, a person seeking expedited access would have to sign a supporting statement that (if suspected of falsehood) could lead to criminal charges under 18 USC 1001. It was scant comfort that the Act had toned down the Senate bill's mention of perjury.
Overall, he felt (Gellman, 1996a, p. 8) "Historically, the FOIA has been amended in a major way about once every ten years. By pushing for a mediocre and limited set of changes, the FOIA community has lost the opportunity for broader reform."
Practical advice for EFOIA requesters includes the following: check online services and look for the agency's list of record locator systems; state in the request the electronic format desired; consider whether your request can qualify for a faster track of processing; if the agency cannot meet the 20-day deadline, document your discussions; realize that if you decline to limit your request, you may be subject to greater delay (Tankersley, 1997a, p. 4.) Remarkably, this advice is very similar to advice given to requesters before the EFOIA; therefore, it tends to support the conclusion that the EFOIA--far from being revolutionary--merely codifies better agency practice.
Generally, there was lukewarm response (Tankersley, 1997b, p. 3) to the EFOIA amendments from the FOIA community. "The amendments fall far short of correcting FOIA's most crippling flaws. Congress has not provided additional revenue resources for processing requests or enhanced enforcement. Instead, the amendments revise agency processing procedures in ways unlikely to improve response time significantly, but that will certainly create complications as requesters and agencies adjust to the new law."
However, support at least for the multitrack processing principle came from Carl Stern, the reporter who had broken the COINTELPRO counterintelligence program story, who was currently working for the Justice Department. He argued (Hammitt, 1996b, pp. 5-6) that with 770 FOIA staff (a doubling under the Clinton administration) there was still a need to streamline processing, via compromise, flexibility, and the end of the rigid first-in-first-out processing system.
XI. BUDGET AND STAFF REQUESTS FOR EFOIA IMPLEMENTATION (1997)
There was some evidence that agencies were ramping up EFOIA efforts pursuant to the passage of EFOIA, based on their budget requests in March 1997. In a House appropriation subcommittee's hearings the Justice Department found (Warren, 1996, p. 6): "Criminal division also is requesting 22 positions, 11 work-years, and $1,393,000 to increase its ability to implement the requirements of the recently enacted EFOIA amendments of 1996; address the growing number of requests for access to Criminal Division records; and reduce the backlog of requests in the Division."
XII. IMPLEMENTATION OF THE 1996 EFOIA: HEARINGS (1998)
In June 1998, the House Government Reform and Oversight Committee's Subcommittee on Government Management, Information, and Technology conducted hearings on Implementation of the Electronic Freedom of Information Improvement Act (EFOIA), S. 1090, under the question: is access to government information improving? Testifying were speakers for public interest groups (OMB Watch, Public Citizen, Critical Mass Energy Project) followed by those for agencies (Justice, the FBI, NASA and the Department of Energy.) In a contrast with numerous previous hearings on agency FOIA practices from 1955 onward, agency successes were the focus of the official testimony.
The chair's opening statement (Horn, 1998), after tying the FOIA to Madison's oft-quoted argument for citizen information, and to the general purpose "to promote open, accessible government," pointed out two major changes in the Act. First, "an increase in affirmative disclosures ... without waiting for individual FOIA requests." Second, "changes to the FOI requesting process itself in an effort to make FOIA a more effective tool for accessing Government information." The implementation hearing would proceed initially with a panel of public interest groups, a journalist and a researcher, followed by agency officials.
OMB Watch summarized its report, "Arming the People ..." (Henderson and McDermott, 1998) based on a survey of 135 agency websites in late 1997. Four EFOIA-required categories of information were sought: official notices and adjudication opinions; an index of major information systems; a FOIA requester's guide; and an electronic reading room. A survey letter requesting the location of the EFOIA website material was sent to 84 agency FOIA officers listed on the Department of Justice FOIA website; two thirds of the agencies treated this letter as a formal request. The researcher revisited websites where further information was revealed in the response to the letter (McDermott, 1998).
At that point the study found that EFOIA information was usually difficult to locate online, not being mentioned on the agency's home page. (The EFOIA amendments did not mandate accessibility of the required material.) Of the 57 agencies examined, 73% had some degree of compliance, but only one (NASA) came close to full compliance. Even NASA had limited its indexing to that required by OMB. OMB Watch found some features of websites attractive: the Department of Defense and the FCC had good search engines; and the SBA and the NSF provided request forms online; the VA accommodated low tech as well as high tech requesters. There was divergence of practice where agencies had delegated authority to satellite units, but overall, agencies were moving to put material online with great speed, if not with convenient locators for the user. (Henderson and McDermott, 1998; McDermott, 1998).
OMB came in for particular criticism: it was argued that OMB's advice (April 1997, rescinded April 1998) that agencies should establish a Government Information Locator Service (GILS) connection to their available data would not truly bring them into compliance with EFOIA. GILS (see OMB Bulletin 95-1) is an index of electronic material only, not including the index of all major information systems also required by EFOIA. In the first year of EFOIA, 18 agencies had followed the rescinded OMB advice. Eighteen months after enactment, only a handful of agencies had promulgated the amendments to agency regulations required by EFOIA. (Henderson, 1998; McDermott, 1998).
However, OMB Watch hoped that online information would render FOIA requests an avenue of last resort; it noted that OMB's advice to agencies had improved; it called for congressional appropriation of funds for EFOIA; it warned against uneven implementation at agency components and called for coordination; and it recommended that agencies define information likely to enjoy repeated requests, and that they bring their practices into line with the best agencies cited. (Henderson and McDermott, 1998; McDermott, 1998).
OMB Watch's call for specific appropriations to fund the EFOIA processing was, however, disowned by at least one FOIA advocate (Gellman, 1998): "A specific line item for FOIA could actually work against the law.... Once the funding ran out, the agency could ... shut down its operations for the rest of the year. Even worse, if Congress eliminated the line item entirely in a subsequent budget, the agency might claim that it was not given any funds to comply with FOIA."
XIII. REDACTION SOFTWARE FOR EFOIA PROCESSING
Dedicated FOIA software is suggested as a solution to the problems of scanned image documents, deletion in context, compound documents, and multiple originators. Graphical images of hard copy documents, once scanned, sometimes hold marginal notes that are part of the record considered for FOIA purposes, thereby posing a problem for electronic redaction. Since some word processing software does not fully delete text on demand unless the action is followed by a full save, and since agencies commonly have several pieces of document software in use--sometimes for the same compound document--there has been increasing interest in dedicated redaction software for EFOIA purposes.
In addition, with documents often altered by multiple staff, perhaps at multiple offices, and redacted by officials unfamiliar with the originators and their software, FOIA officers may be particularly cautious about publishing materials on the Web. For them, FOIA tracking software may be a partial solution.
Houser and Hart (1997) briefly examined the abilities of six redaction software packages to redact passages, indicate the length of material removed, show the reason for redaction, the authority redacting, incorporate digital signatures, import and export formats, and handle multiple pages. Generally they recommended Eastman Software's Imaging for Windows and Imaging Professional but not Imagine's limited Image Viewer. Xerox's Pagis Pro was solid but unable permanently to export formats. Among freeware, for MS Word only, Veterans Affairs' Word Dot could be adapted, but Imagine's Image Viewer was flawed. Digital Applications' Redax (for PDF documents only) was still awaited.
XIV. EFOIA INITIAL COMPLIANCE
The Public Citizen Litigation Group (Tankersley, 1998), having litigated some 300 lawsuits for requesters over 27 years, found electronic data at stake in much recent work, including the landmark case Armstrong v. Executive Office of the President, and Public Citizen v. Carlin. It argued, "FOIA has been crippled by two significant failures in federal information policy: the lack of useful, comprehensive directories of government records, and the chronic delays in making records available to the public." It found many agencies slow to implement the EFOIA provisions and OMB lacking in leadership on the issue. The newly required handbooks for requesting records in particular were delayed and of little use to requesters. Where GILS was the indexing source, as at the USTR, only selected information systems were listed and the index was "woefully incomplete." Consequently, Public Citizen (Tankersley, 1998) "brought suit against seven agencies for failing to prepare inventories and descriptions of major information systems."
Part of the problem with implementing EFOIA is a deeper issue of implementing the Paperwork Reduction Act (44 USC 3506 (f)(4)): by an anecdotal estimate given by NARA, agencies even for their own purposes have failed to index and archive even 35-45% of records within 2 years of their creation. OMB appeared to have orphaned its online index to agencies, GILS. As for the OMB Watch finding of inadequate implementation, Public Citizen concluded (Tankersley, 1998), "A mixture of [lack of] funding and [unwilling] attitude appear to be at the root of this problem." It cited the NRC for having pioneered online access to information before the 1996 enactment of EFOIA; NRC claimed the added cost of an electronic reading room was minimal.
The Department of Justice (1998) advised agencies that records required to be made available online were limited to those created by the agency rather than including those (voluminous materials) received by the agency from outside sources. Although only some agencies had taken that advice, Public Citizen found (Tankersley, 1998) the Justice position "a major threat to full implementation of the statute."
Public Citizen concluded (Tankersley, 1998), "The future of this legislation will depend on real leadership from OMB, funding to promote the efficient management and dissemination of electronic records, and enlightened policies on electronic records from the National Archives."
Public Citizen's Critical Mass Energy Project found (Riccio, 1998) more optimistically that "the U.S. Nuclear Regulatory Commission ha[d] improved both the timeliness and accessibility of information." Formerly, "[m]erely receiving a transcript of [an NRC] meeting would usually take weeks and cost as much as a hundred dollars to reproduce.... Now most transcripts ... are available within a few days and can be downloaded ... in a few minutes." Pointing to the importance of timeliness, it noted, "Since many comment periods on proposed rules ... are only for 30 days, instantaneous access to the proposed rule or amendment is imperative...."
During the transition from hard copy to digital files, there are risks to data integrity. "In the move from paper to computer, the government must be vigilant that information is not lost in the transition." There were "errors in the data base used to track the number and severity of violations of NRC regulation ... one third of the data was missing and half of the data concerning severity of the violation was incorrect. When repairing this database (Riccio, 1998), if the NRC could not match a paper copy of the violation to the computer record, the computer record would be deleted. This undoubtedly resulted in a loss of information."
While welcoming the provisions of the EFOIA, and noting "from the early innovators in Government FOI offices that there is great promise in what agencies can do," the Reporters Committee for Freedom of the Press expressed disappointment that the provisions were "not yet fully operational." It found, "Our greatest disappointment in the Act has been the refusal of the Executive Branch and the Courts to recognize the Act's first finding, that the FOIA is meant to permit responses to requests that serve any purpose, not just the public's interest in government operations and activities." It cited as problems (Kirtley, 1998): the delay in promulgating regulations, delays in establishing leadership at the Department of Justice, the continuing deletion of agencies' e-mail as though it were not a federal record, and the practice of printing out rather than transmitting electronic records. Reading rooms continued to be nonexistent or "information poor," while backlogs were the rule rather than the exception. The new 20-day time limit to respond to requests had produced no improvement.
Reporters found that the best agency websites were set up by agencies (such as Transportation and the EPA) that already had good FOIA processing before 1996, and conversely, laggard EFOIA agencies had been poor FOIA processors before 1996. Reporters Committee (Kirtley, 1998) welcomed the provision to release electronic records specifically, remembering "horror stories of reporters ... receiving boxes and boxes of unedited, unsorted printed materials that were virtually useless to them." It gave the example of a Pulitzer Prize-winning report written partly from an 80-pound printout that cost the reporter countless hours of manual retrieval of information.
Reporters Committee could not find any acceleration of processing requests under the new time limits or expedited review, and found that few reporters were taking advantage of the multitrack processing provision. The faulty limitation of the FOIA to a core purpose--invented by the Supreme Court in Department of Justice v. Reporters Committee (1989)--had "locked up information in any government files that happen to contain the names of identifiable individuals." Although the EFOIA specifically added the phrase, "for any public or private purpose," a district court in O'Kane v. U.S. Customs Service (1997) had already discarded that (Kirtley, 1998) in favor of the Supreme Court's version.
The Department of Justice claimed (Huff, 1998) that, under the leadership of Attorney General Reno, it had "placed a sustained priority on ... making available as much government information as possible." It cited the search tools, links, and lists of contacts on the department's home page. As for reduction of backlogs, it cited the FBI's progress; and the near-elimination of backlogs by the DEA, Bureau of Prisons, U.S. Attorneys Office, and U.S. Marshals Service. Even the INS, subject to 100,000 requests per year, had reduced its larger backlog by 16% and launched an automated FOIA system.
Although criticized by public interest groups for lack of leadership, the Department of Justice cited (Huff, 1998) its efforts to lead via the FOIA telephone counsellor service, guidance on frequently asked questions on its website, training sessions, and a revised guide to the FOIA. With OMB, it had issued guidelines on the enhanced annual report requirements. While it accepted that agency websites varied greatly in their effectiveness, it was shortly to hold a conference to improve compliance.
Not all agencies view the FOIA as peripheral to their mission: NASA testified that its charter mandated wide dissemination of agency activities. Consequently (Riep-Dice, 1998; NASA CIO Executive Notice 21-97), it established an EFOIA reading room in April 1997 with a guide for requesters on the entrance level of the NASA home page (introduced in 1996), and then encouraged components to establish Center home pages, balanced by appropriate security measures against improper access to sensitive systems.
In the first half of 1998, NASA headquarters (HQ) received 500 electronic requests under FOIA, many for documents already publicly available. During May 1998 alone (Riep-Dice, 1998), the HQ FOIA home page received 163,000 visitors.
The establishment of the Electronic Reading Room has reduced the number of FOIA requests for documents that are consistently requested through FOIA ... because they are directly accessible by the public. While it is too early to tell whether the overall number of FOIA requests will be reduced as a result ... this is a promising trend. Although the FOIA Home page has reduced the number of requests for certain frequently requested documents, it has increased the number of electronic requests. NASA believes that with the passage of time and the electronic medium replacing paper, this could create cost savings in the future while improving public access to NASA information.
The Secretary of Energy's Openness Advisory Panel had recommended in July 1996 improving the Declassification process, the accessibility of information, and the culture of secrecy. The department took practical steps such as redesigning its home page to provide better pointers to online documents. It reported "dramatic improvements" (Riep-Dice, 1998): "In 1994 our backlog at headquarters was 658 pending cases ... In addition, we could not rely on our data concerning median case processing times ... requesters sometimes gave up expecting that a response would ever be made. This situation has been reversed. Now, we can accurately report a reduction of more than one third in the ... backlog from 1994 [and the near-elimination of prior request backlogs]."
While the Energy Department had been supplying records in any available format since 1990, and had processed its reading room records via a scanner by mid-1998, it was still working (Riep-Dice, 1998) on extending this to a department-wide information management system, to include field offices.
In conjunction with the Library of Congress and a couple of research organizations, the Copyrights Office is developing an electronic recording and deposit system (CORDS). Eventually the entire process of submitting, examining, recording, and disseminating copyright material will be performed online. From early 1996 to 1998 the system grew to cover several major research universities and several categories of works via multiple combinations of hardware and software. In 1998, a batch-mode of submission was developed for large volumes of submissions. Taking advantage of the flow of 20,000 consistent claims to copyright per year from University Microfilms (Peters, 1998, pp. 3-4), the Copyright Office has been testing processing those submissions electronically, using technicians only, freeing examiners for other tasks.
In improving its electronic services to customers, the Copyright Office cited (Peters, 1998, p. 4) an electronic newsletter which since November 1997 had risen to 2000 subscribers; a website for post-1977 records, accessed over 1.3 million times in its first year; an e-mail inquiry service that responded to over 5000 e-mail requests; and a fax-back service that responded to 5000 documents per year. These practical and affordable measures were stimulated by a Pitney Bowes Ergonomic Solutions report whose recommendations were, alas, "cost-prohibitive."
XV. JUSTICE DEPARTMENT FOIA GUIDELINES
Government webmasters were alerted (Houser, 1998) to Justice Department requirements that websites contain certain wording for the EFOIA link on the agency's home page, that certain minimal links be constructed, and that regular maintenance of links be conducted. Although the term "home page" was not defined, for practical reasons it might be the default page of the root directory of the relevant web service, or simply the advertised entry point to an agency's website.
XVI. AGENCIES' BUDGET REQUESTS FOR AUGMENTED RESOURCES FOR THE EFOIA, POST 1996
The Copyright Office examiner staff of 64 were examining 620,000 claims per year by 1993, but this staff had fallen by 38% between 1993 and 1996. In 1997 (Peters, 1998, p. 6) the Office restored 13 positions with trainees but was still short of eight examiners to be hired in 1998-1999. In addition to asking any retired examiners to return to work, it switched from external microfilming of records to in-house digital imaging. To further cope with the workload, it processed claims at a lower level (with technicians) and adopted (only for solo, living authors) a simplified form with minimal information boxes. One examiner was assigned to each high-volume applicant to enhance service and expertise. While processing enhancements are relatively consensual (Peters, 1998, p. 7), and an assessment of costs and fee structure was underway, the failure among copyright methods reforms was the cumbersome royalty negotiation system, which was still under review.
Within the U.S. Department of Justice, the U.S. Attorneys Office (Dibattiste, 1997, p. 5) requested eight positions for FOIA and Privacy Act (PA) processing at its newly combined executive office and D.C. office, specifically to comply with the EFOIA amendments and reduce existing combined backlogs of requests and litigation. Although the present staff (p. 6) were fulfilling over 3000 requests per year, requests in 1996 increased substantially to 5000 and in 1997 to over 6000. It also requested paralegals to assist in reducing the backlog from the thousands to the hundreds. Experience with contract employees had shown that a decrease in backlogs came at a steep price in training time.
Given universal jurisdiction for FOIA litigation in the D.C. courts, the U.S. Attorneys office there (Dibattiste, 1997, p. 6) found itself responsible for cases increasing from 100 in 1995 to 300 in 1997; with the increase in the time limit for response to 20 days, the office anticipated courts would no longer be inclined to grant extensions.
As in previous FOIA policy cycles, the 1996 EFOIA built upon an extended congressional record of hearings and staff reports, and emerged suddenly after leadership from the Democratic side with a bipartisan, bicameral consensus. Unlike the 1974 amendments, however, the EFOIA may have been oversold: it was debatable whether the 1996 Act extended the FOIA much beyond the current practices of the better agencies' FOIA programs. At least the EFOIA made explicit the wishes of Congress developed since 1985 that electronic records be made public. Preliminary evidence indicates pressure within the administration to introduce online processing and reduce backlogs, modest but welcome improvements in themselves.