June 10, 2002 Memo - Agency Draft Information Quality Guidelines

The U.S. Equal Employment Opportunity Commission

June 10, 2002

MEMORANDUM FOR PRESIDENT'S MANAGEMENT COUNCIL

FROM: John D. Graham

SUBJECT: Agency Draft Information Quality Guidelines

The quality of information disseminated to the public by the Federal Government needs to be improved.

Reflecting this need, Congress recently directed OMB to issue government-wide guidelines that "provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies."The Administration is committed to vigorous implementation of this information quality law.

OMB issued government-wide information quality guidelines on September 28 last year. Each Federal agency is now required to issue its own guidelines that will ensure the quality of information that it disseminates. These guidelines must include mechanisms to allow the public to seek correction of disseminated information that does not comply with the information quality standards in the OMB or agency guidelines. To permit public participation and comment, and to facilitate interagency coordination, agencies are expected to make their draft guidelines available for public comment.

My staff and I have completed a preliminary review of the draft agency guidelines currently available for public comment. We want to thank you for the substantial effort and careful deliberation reflected in the agency drafts. Agencies, with highly diverse program responsibilities, disseminate a wide variety of kinds of information to serve many different purposes. The agency drafts properly reflect this variety.

Some agencies have developed particularly noteworthy provisions that I would suggest for consideration by other agencies in reviewing and revising their own draft guidance. I would also like to point out some provisions in agency drafts that do not appear consistent with the text and intent of the OMB guidelines or are otherwise contrary to Administration policy.

Based on our review, I have attached a discussion of important issues, identified noteworthy approaches for consideration, and provided guidance on those provisions that need to be adopted uniformly in all agency guidance. I request that you send this attachment to the appropriate officials who are responsible for developing your agency's information quality guidelines.

We have asked agencies to submit draft final guidelines to us for review by August 1 (which we have extended from an original July 1 deadline). We encourage you to use this extra time to extend your public comment period. In light of the recent decision to allow additional time for agencies to extend the period for public comment on agency guidelines (and thus compress the time available for final OMB review), it is my intention to have these OIRA comments considered in conjunction with public comments as agencies shape their final guidelines.

As a related matter, I should note that Mark Forman of OMB is leading work on a content model for presenting information on the web. It will include guidelines on how to present web content, how agencies should identify web-based material, and general guidelines for what should go on the public internet.

Attachment

June 10, 2002

OIRA REVIEW OF INFORMATION QUALITY GUIDELINES DRAFTED BY AGENCIES

By October 1, 2002, agencies must publish in the Federal Register a notice that the agency's final guidelines are available on the Internet. Agencies must also provide OMB an opportunity to review each agency's draft final guidelines before they are issued. Drafts must be submitted to OMB no later than August 1.

The underlying legislation is Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554; H.R. 5658). The OMB Information Quality Guidelines can be found in the Federal Register for September 28, 2001 (66 FR 49718), and, as amended, for February 22, 2002 (67 FR 8452).

This attachment discusses important policy issues raised by the agency drafts, identifies noteworthy approaches for consideration, and provides guidance on those provisions that need to be uniformly adopted in all agency information quality guidelines. We urge that draft guidelines submitted for OMB review reflect consideration of this guidance as well as the public comments.

I. SCOPE OF AGENCY GUIDELINES.

In this topic, we discuss a number of constructive approaches agencies used to define the kinds of information that are covered by their guidelines. In some cases, we refer to provisions from agency drafts. These examples are quoted at the end of this attachment.

We cite these agency draft provisions as useful constructive approaches. We caution, however, that these examples are only agency proposals. Based on public comment and other review, the agencies may further refine these examples.

The OMB definitions of "information"and "dissemination"establish the scope of these guidelines. Both definitions contain exceptions. Agencies have elaborated upon the definitions of information and dissemination, and the exceptions thereto, to both broaden and narrow their scope. The specific examples discussed below include modifications that appear reasonable and consistent with the approach OMB takes in its guidelines, as well as suggestions for improvement and greater consistency with the OMB guidelines. We suggest that agencies consider these approaches for their own use.

Use of Statements of "Intent"to Define Scope. Some agencies used statements of intent or purpose to limit the scope of these guidelines. Such use of "intent"clarifies the nature of the inclusion or exclusion in a way to avoid having incidental or inadvertent public disclosure undermine the practical administration of the definition or exclusion. For example, some agencies insert the concept of "intent"into the exemption for intra- or inter-agency use of sharing of information, e.g., exempted is "information ... not disseminated to the public, including documents intended only for inter-agency and intra-agency communications"(ED, 1 & 4). On the other hand, some agencies quote this definition as stated in the OMB guidelines literally, and do not insert a concept of intent. They may wish to include a concept of "intent"to avoid inadvertent public disclosure from undermining practical administration of the guidelines.

Exemption for Press Releases. Some agencies narrowed the exemption in the OMB definition to provide that the agency should already have disseminated the information discussed in the press release in another way. For example, EPA states "These guidelines do not apply to press releases, fact sheets, press conferences or similar communications in any medium that announce, support the announcement or give public notice of information EPA has disseminated elsewhere"(EPA, 15). This limitation avoids creating an incentive to misuse press releases to circumvent information quality standards.

Exemption for Public Filings. Some agencies refined the exemption for public filings to permit agencies to "pass through"information not subject to the guidelines while properly applying the agency and OMB guidelines to third-party information that the agency disseminates. Agencies need to qualify the public filing exemption to ensure that the agency guidelines continue to apply to third-party information that the agency disseminates, as we discuss below under II, "Coverage of 'Third-Party'Information under the Guidelines."

Exclusion For Agency Employed Scientist, Grantee, or Contractor. The preamble to the OMB guidelines discusses situations in which the dissemination of information by an agency-employed scientist, grantee, or contractor is not subject to the guidelines, namely those situations in which they "publish and communicate their research findings in the same manner as their academic colleagues"and thus do not imply official agency endorsement of their views or findings (67 FR 8453-54, February 22, 2002). On the other hand, an agency disseminates information "where an agency has directed a third-party to disseminate information, or where the agency has the authority to review and approve the information before release"(67 FR 8454, February 22, 2002). Agencies that did not explicitly include such an exemption may wish to consider doing so, but need to do so in the carefully balanced ways quoted at the end of this attachment.

Exclusion for Testimony and Other Submissions to Congress. Some agencies exclude "information presented to Congress (as part of the legislative or oversight processes, e.g., testimony of officials, information or drafting assistance provided to Congress in connection with pending or proposed legislation) that is not simultaneously disseminated to the public" (Justice, 3; DOT, 9). As with the exemption for press releases, we think it would be better for agencies to narrow this exemption to provide that the agency should already have disseminated the information discussed in the testimony in another way. This limitation would avoid creating an incentive to misuse testimony and other submissions to Congress to circumvent information quality standards.

Exemption for Subpoenas or Adjudicative Processes. The preamble to the OMB guidelines states that "The exemption from the definition of 'dissemination'for 'adjudicative processes'is intended to exclude ... the findings and determinations that an agency makes in the course of adjudications involving specific parties. There are well-established procedural safeguards and rights to address the quality of adjudicatory decisions and to provide persons with an opportunity to contest decisions. These guidelines do not impose any additional requirements on agencies during adjudicative proceedings and do not provide parties to such adjudicative proceedings any additional rights of challenge or appeal"(67 FR 8454, February 22, 2002). Some agencies adapted the OMB exception very carefully. Other agencies may have broadened this exemption beyond OMB's intent; they need to limit this exemption carefully to be consistent with OMB's intent both as to the adjudicative procedures that are included and the scope of the information covered.

Effective Date. The OMB guidelines establish two somewhat different effective dates (III.4). An agency's obligation to conduct a pre-dissemination review of information quality starts only on October 1: "The agency's pre-dissemination review, under paragraph III.2, shall apply to information that the agency first disseminates on or after October 1, 2002."An agency's obligation to allow the public to seek the correction of information that does not comply with the information quality standards in OMB or agency guidelines starts on October 1, 2002, for information that the agency disseminates on or after October 1, 2002, even if the agency first disseminated that information before October 1: "The agency's administrative mechanisms, under paragraph III.3, shall apply to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information."

Some agencies followed the OMB guidelines carefully in describing when the information quality guidelines will take effect: "The DOJ information quality guidelines will become effective on October 1, 2002. These guidelines will cover information disseminated on or after October 1, 2002, regardless of when the information was first disseminated"(Justice, 2). Other agencies need to be careful to track accurately the OMB guidelines in this regard (III.4).

The effective date for the agency's administrative mechanisms raises the issue of what constitutes agency dissemination of information after October 1, 2002, if the agency first disseminated this information earlier.

DOT defines dissemination after October 1 to exclude archived information that had been disseminated previously. "As provided in OMB's guidelines, these guidelines apply only to information disseminated on or after October 1, 2002. The fact that an information product that was disseminated by DOT before this date is still maintained by the Department (e.g., in DOT's files, in publications that DOT continues to distribute on a website) does not make the information subject to these guidelines or to the request for correction process"(DOT, 23). This interpretation is consistent with OMB's intent, and equivalent to the "archival records" exemption.

Still to be considered is how a complainant demonstrates that an agency disseminates information after October 1, 2002, if the agency first disseminated that information before October 1, 2002. For example, existing official agency data bases, publicly available through agency websites or other means, that serve agency program responsibilities and/or are relied upon by the public as official government data, need to be subject to the Section 515 administrative mechanisms to address public complaints because they are, in effect, constantly being redisseminated.

II. COVERAGE OF "THIRD-PARTY"INFORMATION UNDER THE GUIDELINES.

The preamble to the OMB guidelines states, "If an agency, as an institution, disseminates information prepared by an outside party in a manner that reasonably suggests that the agency agrees with the information, this appearance of having the information represent agency views makes agency dissemination of the information subject to these guidelines"(67 FR 8454, February 22, 2002). Reinforcing this statement of policy, OMB also provided an example in its preamble concerning the applicability of the OMB and agency information quality standards to third-party studies relied upon by an agency as support for a proposed rulemaking, even if the third-party studies had been published before the agency's use of them (67 FR 8457, February 22, 2002).

DOT incorporated these principles from the OMB guidelines by stating that an agency disseminates information if it relies on information in support of a rulemaking. "If the Department is to rely on technical, scientific, or economic information submitted by, for example, a commenter to a proposed rule, that information would need to meet appropriate standards of objectivity and utility"(DOT, 3). "The standards of these guidelines apply not only to information that DOT generates, but also to information that other parties provide to DOT, if the other parties seek to have the Department rely upon or disseminate this information or the Department decides to do so"(DOT, 8).

EPA explicitly includes a provision embodying the OMB example: "If a particular distribution of information is not covered by these guidelines, the guidelines may still apply to a subsequent distribution of the information in which EPA adopts, endorses or uses the information to formulate or support a regulation, guidance, or other Agency decision or position"(EPA, 17). Other agencies - particularly those likely to be involved with using and/or disseminating "influential" information - must include similar provisions in their guidelines.

III. AGENCY COMMITMENT TO INFORMATION QUALITY STANDARDS.

In this topic, we discuss (1) ways in which agencies need to commit to information quality standards, and (2) aspects of how those standards should be defined.

Performance Standards. The OMB guidelines state that, "Overall, agencies shall adopt a basic standard of quality (including objectivity, utility, and integrity) as a performance goal and should take appropriate steps to incorporate information quality criteria into agency information dissemination practices"(III.1). The "information quality criteria"are set forth in the definitions of "Quality," "Utility," "Objectivity," and "Integrity" (V.1-4). Closely related definitions are those for "influential"information, when used in the phrase "influential scientific, financial, or statistical information,"and for "reproducibility"(V.9-10).

Each agency, in structuring its information quality guidelines, must state the agency's information quality criteria (as defined in the OMB and agency guidelines) as performance goals that the agency seeks to attain. Each agency needs to adopt explicitly each aspect of each definition of quality, utility, objectivity, and integrity as an agency information quality standard. Each agency also must explicitly state that it intends to achieve each standard. Otherwise, there will be no benchmark against which a public complainant will be able to suggest non-attainment.

The OMB guidelines also state that, "As a matter of good and effective agency information resources management, agencies shall develop a process for reviewing the quality (including the objectivity, utility, and integrity) of information before it is disseminated"(III.2). Given that guideline, many agencies describe in considerable detail the kinds of activities they now undertake to assure information quality. Regardless, we stress that a mere description of current practices - however good - is not a substitute for explicit performance goals. At a minimum, each agency must embrace the OMB quality definitions as information quality standards they are seeking to attain. Examples of constructive agency statements are quoted at the end of this attachment.

In addition, some agencies and agency components do not appear to have adopted any standards for information quality (utility, objectivity, integrity) and/or defined "influential"or "reproducibility"in ways applicable to them. Each agency must either define its standards in ways applicable to it and consistent with the standards in the OMB guidelines, or explicitly adopt the standards from the OMB guidelines as the agency or component standards. For an agency that does not anticipate disseminating much information that is defined as "influential", we suggest that the agency simply adopt the standards from the OMB guidelines as its own.

Core Definition of "Objectivity". The OMB definition of "objectivity" is the most detailed and complex. This definition has different aspects, some that apply to all information covered by the OMB guidelines, others that apply only to "influential" information.

The first issue relates to all covered information. According to the OMB guidelines, 'Objectivity' has two distinct elements, presentation and substance.

'Objectivity' includes whether disseminated information is being presented in an accurate, clear, complete and unbiased manner [ -- as well as "within a proper context"]. ...
In addition, 'objectivity' involves a focus on ensuring accurate, reliable, and unbiased information(V.3.).

Some agencies have summarized this aspect of the definition of "objectivity" accurately. Other agencies, in summarizing the OMB standard, appear to have left out some of the important standards; those agencies need to summarize the OMB standard accurately.

Peer Review. The discussion of peer review in the definition of "objectivity"relates to all covered information. "If data and analytic results have been subject to formal, independent, external peer review, the information may generally be presumed to be of acceptable objectivity [if the peer review satisfies 'the general criteria for competent and credible peer review'cited in the definition]. However, this presumption is rebuttable based on a persuasive showing by the petitioner in a particular instance"(V.3.b.i).

If an agency or component engages in peer review, it needs to discuss the ways in which it will adhere to the OMB standard in its guidelines. These peer review standards are not limited to information defined as "influential". These OMB peer review standards apply to all information covered by these guidelines, and need to be integrated into existing agency peer review standards applicable to covered information. In addition, agencies must point out - to be consistent with the OMB standard - that the presumption of objectivity afforded to formal, independent, external peer review is rebuttable, although the burden of proof, as explained more fully below, is on the complainant.

"Influential" and "Reproducibility". The next issue relates to agency treatment of influential information. "If an agency is responsible for disseminating influential scientific, financial, or statistical information, agency guidelines shall include a high degree of transparency about data and methods to facilitate the reproducibility of such information by qualified third parties"@ (V.3.b.ii; see V.9 for definition of "influential").

Several agencies provided a carefully considered discussion of the meaning of "influential" in their drafts. See provisions quoted at the end of this attachment.

"Original and supporting data" and "analytic results". With regard to influential information, the OMB guidelines further distinguish between "original and supporting data"and "analytic results"@.

With regard to original and supporting data related thereto, agency guidelines shall not require that all disseminated data be subjected to a reproducibility requirement. Agencies may identify, in consultation with the relevant scientific and technical communities, those particular types of data that can practicably be subjected to a reproducibility requirement (V.3.b.ii.A).

With regard to analytic results related thereto, agency guidelines shall generally require sufficient transparency about data and methods that an independent reanalysis could be undertaken by a qualified member of the public. ...

... Making the data and methods publicly available will assist in determining whether analytic results are reproducible. However, the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections.

...In situations where public access ... will not occur ..., agencies shall apply especially rigorous robustness checks to analytic results and document what checks were undertaken. Agency guidelines shall, in all cases, require a disclosure of the specific data sources ... used and the specific quantitative methods and assumptions ... employed (V.3.b.ii.B).

In draft agency guidelines, it does not appear that any agency undertook to delineate when "original and supporting data"would be subject to a reproducibility requirement. Presumably, the public comment period is being used to seek views from the relevant scientific and technical communities. If, at the end of the public comment period, an agency is not prepared to identify what kinds of original and supporting data will be subject to the reproducibility standard, then the agency must include in its guidelines a statement to the effect that the agency shall assure reproducibility for those kinds of original and supporting data according to "commonly accepted scientific, financial, or statistical standards"(suggested language).

As to "analytic results,"it appears that a number of agencies anticipate that reproducibility will sometimes not be achievable through public access because of confidentiality protections or other compelling interests. In such cases, some agencies do not mention the need to "apply especially rigorous robustness checks."Instead, they describe their intent to disclose specific data sources and specific quantitative methods and assumptions. In such situations, agencies need to state explicitly their commitment to the standards stated in the OMB guidelines to applying "especially rigorous robustness checks"to analytic results and document what checks were undertaken. In addition, agency guidelines must, in all cases, explicitly require a disclosure of the specific data sources, quantitative methods, and assumptions used. We also recommend that agencies, in generating (or contracting to generate) influential information for dissemination, encourage arrangements that will permit appropriate public access to the related original and supporting data and analytic results.

Analysis of Risks to Human Health, Safety and the Environment. With regard to influential information, the OMB guidelines also state that, "With regard to analysis of risks to human health, safety and the environment ..., agencies shall either adopt or adapt the quality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (42 U.S.C. 300g-1(b)(3)(A) & (B))"@ (V.3.b.ii.C). Some agencies discussed these Congressional risk information quality standards; some agencies discussed these in a limited context; and other agencies failed to mention these standards at all. Those agencies that are likely to use and/or disseminate influential information in their analysis of "risks to human health, safety, and the environment"need to clearly state that they are adopting the SDWA standards, or justify in what ways and for what kinds of information the agency is adapting the SDWA standards. FDA adapts the SDWA standards in a carefully considered, practical way (HHS/FDA, 18- 20). We note that FDA read the SDWA standards as applicable to a risk assessment document made available to the public and did not limit their applicability only to documents related to a rulemaking; that is the proper approach.

IV. QUALITY INTEGRAL TO CREATION AND COLLECTION OF INFORMATION.

The OMB guidelines state that "As a matter of good and effective agency information resources management, agencies shall treat information quality as integral to every step of an agency's development of information, including creation, collection, maintenance, and dissemination. This process shall enable the agency to substantiate the quality of the information it has disseminated through documentation or other means appropriate to the information"(III.2). Consistent with the OMB guidelines, the Small Business Administration explicitly included "information development"@, "information acquisition"@, and "information maintenance"within the scope of its information quality guidelines, as quoted at the end of this attachment.

In this light, we note that each agency is already required to demonstrate the "practical utility"of a proposed collection of information in its PRA submission, i.e., for draft information collections designed to gather information that the agency plans to disseminate. Thus, we think it important that each agency should declare in its guidelines that it will demonstrate in its PRA clearance packages that each such draft information collection will result in information that will be collected, maintained, and used in a way consistent with the OMB and agency information quality standards. It is important that we make use of the PRA clearance process to help improve the quality of information that agencies collect and disseminate. Thus, OMB will approve only those information collections that are likely to obtain data that will comply with the OMB and agency information quality guidelines.

V. ADMINISTRATIVE MECHANISM TO ADDRESS PUBLIC COMPLAINTS.

Applicable Standards. The OMB guidelines state, "To facilitate public review, agencies shall establish administrative mechanisms allowing affected persons to seek and obtain, where appropriate, timely correction of information maintained and disseminated by the agency that does not comply with OMB or agency guidelines"(III.3).

Some agencies discuss compliance with both the OMB and agency information quality standards in their discussion of the complaint mechanism. Others discuss compliance only with the agency information quality standards. To be consistent with the OMB guidelines, each agency should explicitly refer complainants to all of the applicable guidelines - the OMB, department, and departmental component's guidelines - as the applicable information quality standards.

"Affected Person". Some agencies defined "affected person" quite broadly. For example, "The term 'affected person' means anyone who may benefit or be harmed by the disseminated information. This includes persons who are seeking to address information about themselves as well as persons who use information" (OFHEO, 5). HHS took an even more open approach. Rather than defining "affected person," HHS just asks the complainant to "describe how the person submitting the complaint is affected by the information error" (HHS, 13). This invites the complainant to describe how he/she is affected, but specifically avoids any provision that would use this answer to limit or restrict who can point out an error in an agency's dissemination of information.

We prefer the HHS approach because it best ensures full public access to the complaint process, a goal of Section 515 and the OMB guidelines. The focus of the complaint process should be on the merits of the complaint, not on the possible interests or qualifications of the complainant. Other agencies need to adopt a similar approach.

Decision Criteria and Burden of Proof for Resolving Complaints. Several agencies state that: "Requesters should be aware that they bear the 'burden of proof' with respect to the necessity for correction as well as with respect to the type of correction they seek" (Justice, 6). Having the burden of proof on the complainant is consistent with the OMB guidelines and will be helpful in permitting agencies to dismiss frivolous or speculative complaints. All agencies should make this clear in describing their complaint mechanism to the public. We quote at the end of this attachment carefully presented statements of the decision criteria and approaches that several agencies plan to follow in resolving complaints.

Time Periods for Resolving Complaints and Any Appeals. The OMB guidelines state, "Agencies shall specify appropriate time periods for agency decisions on whether and how to correct the information, and agencies shall notify the affected persons of the corrections made ... The agency shall establish an administrative appeal process to review the agency's initial decision, and specify appropriate time limits in which to resolve ... requests for reconsideration" (III.3.i & ii).

Each agency must state in its guidelines the time periods for making decisions on both complaints and also on any appeals. Exceptions for unusual cases are appropriate.

Some agencies set a time limit within which, after receiving notice of an initial decision, the complainant could file an appeal, generally 30 days. Setting a time limit for filing appeals appears reasonable.

Some agencies also seek to set time limits for submission of original complaints (in effect, a form of a statute of limitations). OMB has concerns about the potential unintended effects of such limits and will be reviewing them carefully. Sometimes agencies continue, long after the agencies'initial dissemination, to adopt, endorse, or use information, and thus, in effect, continue to disseminate it. Similarly, agencies may continue to maintain ongoing official agency data bases, publicly available through agency websites or other means, that serve agency program responsibilities and/or are relied upon by the public, that are, in effect, constantly being redisseminated. The damaging effects of poor quality information may not occur or be perceived to have occurred until well after the information was originally disseminated.

An Objective Appeals Mechanism. The preamble to the OMB guidelines discusses our intent that agencies establish an objective appeals mechanism. "Recognizing that many agencies already have a process in place to respond to public concerns, it is not necessarily OMB's intent to require these agencies to establish a new or different process. Rather, our intent is to ensure that agency guidelines specify an objective administrative appeal process that, upon further complaint by the affected person, reviews an agency's decision to disagree with the correction request. An objective process will ensure that the office that originally disseminates the information does not have a responsibility for both the initial response and resolution of a disagreement" (67 FR 8458, February 22, 2002).

Some agencies discuss how they plan institutionally to structure their complaint and appeal procedures. Others do not. We strongly suggest that agencies describe to the public how they plan to resolve any complaints and appeals in order to build public confidence in both the reality and appearance of a neutral, fair decision mechanism.

To enhance transparency, we also suggest that agencies provide the public with timely notice of what information the agency intends to correct after it makes a decision to correct it. In the annual report to OMB, agencies should also this information as well as a status report on the numbers and kinds of petitions for corrections, appeals, and any denials or grants of petitions for reconsideration or appeals. Agencies are encouraged, to the extent they practicably can, to give more timely disclosure of this information through, e.g., the use of electronic dockets or agency websites, they are encouraged to do so.

We note, in this regard, that a number of agencies emphasize that their guidelines are not intended to provide any right to judicial review. A few agencies even stress that their guidelines may not be applicable based on unspecified circumstances and that the agency may be free to differ from the guidelines where the agency considers such action appropriate.

Regardless of what kinds of litigation-oriented disclaimers the agencies may include, agency guidelines should not suggest that agencies are free to disregard their own guidelines. Therefore, if you believe it is important to make statements that your agency's guidelines are not intended to provide rights of judicial review, we ask that you not include extraneous assertions that appear to suggest that the OMB and agency information quality standards are not statements of government-wide policy, i.e., government-wide quality standards which an agency is free to ignore based on unspecified circumstances. In addition, agencies should be aware that their statements regarding judicial enforceability might not be controlling in the event of litigation.

VI. MELDING THE STATUTORY REQUIREMENTS OF SECTION 515 INTO THE PROCEDURAL REQUIREMENTS OF OTHER STATUTES.

The agencies take a uniform approach to complaints filed concerning information disseminated in the course of conducting a rulemaking under the Administrative Procedure Act (providing public notice to obtain public comment, then issuing the regulation in final form). The agencies meld the requirement to establish a Section 515 administrative mechanism to address public complaints into the procedures of the APA, NEPA, and other more specific public-comment statutes. This melding of Section 515 complaint procedures into the structure of existing statutes seems reasonable, and is discussed extremely well by a number of agencies. Of course, the substantive standards of quality, the information quality standards provided in the OMB and agency guidelines, remain applicable to any such dissemination of information. Examples of well-reasoned agency statements are quoted at the end of this attachment.

One of the agency discussions raises an interesting issue:

Requests for Correction Concerning Information on Which DOJ Has Sought Public Comment. Information on which DOJ has sought public comment includes a notice of proposed rulemaking (NPRM), studies cited in an NPRM, a regulatory evaluation or cost-benefit analysis pertaining to an NPRM, a preliminary environmental impact analysis, a notice of availability, and request for comment on a risk assessment.

DOJ's response to the request for correction will normally be incorporated in the next document it issues in the matter concerning which it had sought comment. The response will be provided in this document rather than in a separate communication. DOJ may choose to provide an earlier response, if doing so is appropriate, and will not delay the issuance of the final action in the matter (Justice, 6). - 15 -

We suggest that Justice (and other agencies) explain in a little more detail the circumstances under which "an earlier response"might be "appropriate"@. We are sensitive to the procedures and long history behind the Administrative Procedure Act. However, we would suggest that agencies consider adding as criteria for making an early response a demonstration by a complainant of actual harm from the agency's dissemination of a study relied upon in a Notice of Proposed Rulemaking, or a demonstration by the complainant of substantial uncertainty as to whether the proposed rule will take an unusual length of time to go final.

Another interesting issue arises when an agency disseminates a particular study in a Notice of Proposed Rulemaking (NPRM), i.e., in the context of a particular agency policy decision, and a possible complainant has an interest in the study but not necessarily in the substantive policies embodied in the rulemaking. The possible complainant may only learn that the agency has disseminated the study by reading the NPRM, possibly after the comment period has expired. Agencies need to consider how those not directly interested in the rulemaking need to submit and receive consideration of a complaint about the study.

As a general matter, we urge each agency to carefully articulate the ways in which the APA, NEPA, and other more specific public-comment statutes meld with and thus have the apparent effect of superseding the administrative mechanisms to address public complaints provided by Section 515. For example, an agency may disseminate a risk assessment prior to publication of an NPRM. While the agency may anticipate that this risk assessment may be used in support of the NPRM, the agency should still permit complainants to file complaints under Section 515 unless the publication of the NPRM is imminent. Such a risk assessment may have impacts beyond the scope of the rulemaking.

June 10, 2002

OIRA REVIEW OF AGENCY DRAFT INFORMATION QUALITY GUIDELINES

Additional Quotations of Proposed Agency Provisions Organized by Topic

I. SCOPE OF AGENCY GUIDELINES.

Use of Statements of "Intent" to Delimit Scope. SSA and NSF use "intent"to indicate what is covered: e.g., "statistical or actuarial information prepared for public dissemination; reports, studies and summaries prepared to inform the public"(SSA, 2 of 2; NSF, 1).

Justice uses intent to exempt "procedural, operational, policy and internal manuals prepared for the management and operations of DOJ that are not primarily intended for public dissemination" (Justice, 3).

Exemption for Press Releases. FDA/HHS exempts press releases "unless they contain new substantive information not covered by previous information dissemination" (FDA/HHS, 3). EPA adds a different qualifier: "These guidelines do not apply to press releases, fact sheets, press conferences or similar communications in any medium that announce, support the announcement or give public notice of information EPA has disseminated elsewhere" (EPA, 15).

State also limits the scope of the press release exemption to apply to distributions of information or other materials that are "distributed to the press as a summary of a recent event or Department action" (State, 6).

Exemption for Public Filings.

Distribution of information in public filings: Public filings include information submitted to EPA by any individual or person. ... The guidelines do not apply where EPA distributes this information simply to provide the public with quicker and easier access to materials submitted to EPA that are publicly available. This will generally be the case if EPA has not authored the filings, and is not distributing the information in a manner that suggests that EPA endorses or adopts the information, and EPA does not indicate in its distribution that it is using or proposing to use the information to formulate or support a regulation, guidance, or other Agency decision or position (EPA, 16).

Exclusion For Agency Employed Scientist, Grantee, or Contractor.

A Component does not initiate the dissemination of information when a Component-employed scientist or Component grantee or contractor publishes and communicates his research findings in the same manner as his academic colleagues, even if the Component retains ownership or other intellectual property rights because the Component paid for the research. To avoid confusion regarding whether the Component agrees with the information, the researcher should include an appropriate disclaimer ... that the views expressed are his own and do not necessarily reflect the views of the Component. In contrast ..., if the Component has directed a third party to disseminate information or retains the authority to review and approve the information upon release, then the Component has sponsored the dissemination of the information (DOD, 4).

Distribution of information by federal employees and recipients of grants, cooperative agreements, and contracts: These guidelines do not apply to information distributed by recipients of contracts, grants, or cooperative agreements, unless the information is disseminated on EPA's behalf, as when EPA specifically directs or approves the dissemination. These guidelines do not apply to distribution of any type of research by federal employees and recipients of EPA grants, cooperative agreements, or contracts, where the researcher (not EPA) decides whether and how to communicate and publish the research, does so in the same manner as his or her academic colleagues, and distributes the research in a manner that indicates that the research does not represent EPA's official position (for example, by including an appropriate disclaimer). Distribution of research in this manner is not subject to these guidelines even if EPA retains ownership or other intellectual property rights because the Federal government paid for the research (EPA, 15-16).

Exemption for Subpoenas or Adjudicative Processes.

FMC explains this exemption succinctly:

Excluded categories include: ... Subpoenas or adjudicative processes, including Commission orders, opinions, amicus and other briefs. Adjudicative processes also include factual allegations by the staff during the investigative and litigation phases of cases brought by the Commission's Bureau of Enforcement. Because there are well-established procedural safeguards and rights to address the quality of factual allegations and adjudicatory decisions, and to provide persons with an opportunity to contest decisions, these guidelines do not impose any additional requirements on the Commission during adjudicative proceedings and do not provide parties to such adjudicative proceedings any additional rights of challenge or appeal (FMC, 7).

II. COVERAGE OF "THIRD-PARTY"INFORMATION UNDER THE GUIDELINES.

Agencies included "third-party"information under the guidelines in a variety of contexts:

Component dissemination of information prepared by an outside party in a matter that reasonably suggests the Component agrees with the information, renders Component dissemination of the information subject to these guidelines (DOD, 4).

Section III mentions an important concept that may not be immediately obvious to persons reading the OMB guidelines for the first time. As Dr. John Graham, Director [sic: Administrator] of the OMB Office of Information and Regulatory Affairs (OIRA) and others have pointed out in meetings about the information quality guidelines, the standards for data quality that apply directly to Federal agencies also apply, at least indirectly, to outside parties who supply information to the Department. If the Department is to rely on technical, scientific, or economic information submitted by, for example, a commenter to a proposed rule, that information would need to meet appropriate standards of objectivity and utility. Numbers submitted by a commenter as the basis for a regulatory decision - which the Department would necessarily disseminate as part of a rulemaking issuance - should meet data quality standards no less than in the case of information the Department itself generates (DOT, 3). The standards of these guidelines apply not only to information that DOT generates, but also to information that other parties provide to DOT, if the other parties seek to have the Department rely upon or disseminate this information or the Department decides to do so (DOT, 8).

EPA disseminates information to the public for purposes of these guidelines when EPA initiates or sponsors the distribution of information to the public. EPA initiates a distribution of information if EPA prepares the information and distributes it to support or represent EPA's viewpoint, to formulate or support a regulation, guidance, or other Agency decision or position. EPA initiates a distribution of information if EPA distributes information prepared or submitted by an outside party in a manner that reasonably suggests that EPA endorses or agrees with it, if EPA indicates in its distribution that the information supports or represents EPA's viewpoint, or if EPA in its distribution proposes to use or uses the information to formulate or support a regulation, guidance, policy, or other Agency decision or position (EPA, 14).

What happens if information is initially not covered by these guidelines, but EPA subsequently disseminates it to the public? If a particular distribution of information is not covered by these guidelines, the guidelines may still apply to a subsequent distribution of the information in which EPA adopts, endorses or uses the information to formulate or support a regulation, guidance, or other Agency decision or position. For example, if EPA simply makes a public filing (such as facility data required by regulation) available to the public, these guidelines would not apply to that distribution of information. However, if EPA later includes the data in a background document in support of a rulemaking, these guidelines would apply to that later dissemination of the information in that document (EPA, 17).

III. AGENCY COMMITMENT TO INFORMATION QUALITY STANDARDS.

Performance Standards. Some agency guidelines adopted performance standards and a commitment to meeting them. For example, The Office of Special Counsel clearly states information quality standards as performance goals:

Information should adhere to a basic standard of quality ... Information should be objective in substance and presentation ... Information should be responsive to its intended users ... The integrity of information should be protected (67 FR 21318, April 30, 2002). - 20 -

Justice draft guidelines adopt the provision from the OMB guidelines relating to performance standards (III.1).

Overall, agencies shall adopt a basic standard of quality (including objectivity, utility, and integrity) and will take appropriate steps to incorporate information quality criteria into agency information dissemination practices ... A basic standard of quality will be ensured and established for all information prior to its dissemination (Justice 1-2, 3).

Then the Justice draft sets forth a standard and commits the DOJ components to reaching it:

DOJ components will ensure disseminated information [meets the standard of objectivity] ... DOJ components will ensure information [meets the standard of integrity] (Justice, 4).

The Small Business Administration combines the approach taken by Justice:

It is SBA's policy to ensure and maximize the quality, objectivity, utility, and integrity of the information that it disseminates to the public. SBA will take appropriate steps to incorporate information quality criteria into SBA's information dissemination practices, and will ensure the quality of information the agency disseminates in accordance with the standards set forth in these Guidelines. SBA is committed to integrating the principle of information quality into every step of SBA's development of information, including creation, collection, maintenance, and dissemination. SBA will comply with all then-existing legal and policy rules, regulations, directives, and guidance at every step of the process (SBA, 4).

The Federal Energy Regulatory Commission similarly sets standards and commits to reach them:

The Commission strives to present information to the public in an accurate, clear, complete, and unbiased manner. ... The Commission also aims to provide information that is accurate, reliable and unbiased (FERC 5, 6).

Core Definition of "Objectivity". The following are concise, accurate summaries of the heart of the OMB definition of "objectivity":

Objectivity involves two distinct elements; presentation and substance:

(A) Presentation: Disseminate information in an accurate, clear, complete, and unbiased manner. This involves presenting information within a proper context.

(B) Substance: Focus on ensuring accurate, reliable, and unbiased information. In a scientific, financial, or statistical context, generate the original and supporting data, and develop the analytic results, using sound statistical and research methods (Treasury, 3).

Objectivity means ensuring that information is accurate, reliable and unbiased and that information is presented in an accurate, clear, complete and unbiased manner (ERS/USDA, 7).

"Objectivity"focuses on whether the disseminated information is being presented in an accurate, clear, complete and unbiased manner and as a matter of substance, is accurate, reliable, and unbiased (DOD, 3).

"Influential"and "Reproducibility". DOT has a carefully considered discussion of influential. Some highlights are:

DOT emphasizes that to be influential, information must have a clear and substantial impact. A clear and substantial impact, first of all, is one that the agency is firmly convinced has a high probability of occurring ... In rulemaking, influential information is scientific, financial, or statistical information that can reasonably be regarded outcome determinative with respect to one or more key issues in a significant rulemaking, as that term is defined in Executive Order 12886 ... In non-rulemaking contexts, DOT will consider two factors - breadth and intensity - in determining whether information is influential ... DOT organizations should consider whether the information affects a broad range of parties ... DOT organizations will also consider whether the information has an intense impact. Information that has a low cost or modest impact on affected parties is less likely to be influential than information that can have a very costly or crucial impact. In considering whether information has a high-intensity impact, DOT organizations will establish and use as a benchmark the $100 million figure used to determine whether a rule is economically significant (DOT, 20-21).

Justice also has a well-considered definition of influential:

When information is defined as influential there is an added level of scrutiny afforded this information, to include the need to ensure it is reproducible. At DOJ, influential information is that which is expected to have a genuinely clear and substantial impact at the national level, on major public and private policy decisions as they relate to federal justice issues. The accuracy of this information is significant due to the critical nature of these decisions. A clear and substantial impact, first of all, is one that the agency is firmly convinced has a high probability of occurring. If it is merely arguable that an impact will occur, or if it is a close judgment call, then the impact is probably not clear and substantial. To determine that there is a clear and substantial impact, the agency must have greater certainty than would be the case for many ordinary factual determinations. The impact must be on "important" public policy or private sector decisions that are expected to occur. Even if information has a clear and substantial impact, it is not influential if the impact is not on a public or private decision that is important to policy, economic, or other decisions ... The "influential" designation is intended to be applied to information sparingly. DOJ components should not designate information products or types of information as influential on a regular or routine basis. Nor should DOJ components actually place an "influential" label in the title page or text of an information product (Justice, 4).

Both State and DOT, in describing "influential", emphasize the causal link between the information itself, and the effect it may have on the policy position involved:

To be considered influential, information must be based on objective and quantifiable data that constitute a principal basis for substantive policy positions adopted by the Department (State, 6).

It should also be noted that the definition applies to "information" itself, not to decisions that the information may support. Even if a decision or action by DOT is itself very important, a particular piece of information supporting it may or may not be "influential" (DOT, 21).

Analysis of Risks to Human Health, Safety and the Environment. FDA adapts the SDWA standards carefully and practically to the kinds of Risk Assessments that FDA conducts.

Some of the influential information that we disseminate is based on an analysis of the risks to the public of certain actions or exposures to hazardous substances. For purposes of this guidance, we are defining risk as the likelihood that injury or damage is or can be caused by a substance, technology, or activity. We use risk analysis (the integration of risk assessment with risk management and risk communication) as a tool to enhance the scientific basis for all of our regulatory decisions.

The OMB Guidelines provide special considerations that must be taken into account in certain risk assessments, those that provide the basis for the dissemination of influential information. The Guidelines state that "With regard to analysis of risks to human health, safety, and the environment maintained or disseminated by the agencies, agencies shall either adopt or adapt the quality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B))."

The SDWA risk assessment principles are as follows:

To the degree that the agency action is based on science, the agency shall use
1. the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices;
2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data).
In the dissemination of public information about risks, the agency shall ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.
In a document made available to the public in support of a regulation, the agency shall specify, to the extent practicable
1. Each population addressed by any estimate of applicable risk effects;
2. The expected risk or central estimate of risk for the specific populations affected
3. Each appropriate upper-bound or lower-bound estimate of risk;
4. Each significant uncertainty identified in the process of the assessment of risk effects and the studies that would assist in resolving the uncertainty; and
5. Peer-reviewed studies known to the agency that support, are directly relevant to, or fail to support any estimate of risk effects and the methodology used to reconcile the inconsistencies in the scientific data.

Many of our actions are based on scientific experts' judgments using available data, are essentially qualitative, and are generally carried out for non-cancer-causing hazards. Such assessments provide useful answers in most instances that are sufficient for regulatory purposes, and much more elaborate, quantitative estimates extrapolating beyond the data are unnecessary. For example, we may issue regulations on submission requirements for product approval applications, electronic submission of product labeling, or periodic reporting by manufacturers of adverse events from drugs; devices; and biologics, including blood, vaccines, and tissues. Although we analyze the economic costs of the regulations and consider alternatives, regulations like these do not lend themselves to the types of quantitative risk assessments contemplated by the Safe Drinking Water Act principles. Other actions are based on research and supporting data that are generated outside FDA. For example, most product approval actions are based on scientific studies conducted by sponsors seeking marketing approval in accordance with our regulations and guidance documents. Our regulations and guidance documents describe sound scientific practices for conducting human and animal studies of medical products and analyzing the resulting data. Most information in these studies is considered confidential commercial information and is closely held by the sponsors. As a result, formal peer-review of the data is rare. However, for certain drug approval applications, the safety and/or effectiveness information is presented to scientific advisory committees for recommendations. Evaluations of food safety and nutritional data are also presented to scientific advisory committees.

As a result, we have adapted the general principles for risk assessments from the SDWA to fit these situations. The principles we intend to apply to risk assessments involving the dissemination of influential information affecting product approval actions or regulations that do not lend themselves to quantitative risk assessment are as follows:

The Agency will use
1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer reviewed studies when available; and
2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data).
In the dissemination of public information about risks, the Agency will ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.

In situations requiring a quantitative risk assessment, we generally follow basic risk assessment principles in the NAS paradigm of 1983. Our needs for quantitative risk assessments range over a wide variety of hazards including physical hazards encountered during use of a medical device, food chemical residues, and antimicrobial resistance genes in bacteria. Thus, we also ascribe to the statement from NAS when it revisited the risk assessment process in 1994 (Science and Judgment in Risk Assessment, NAS 1994): "Risk assessment is not a single process, but a systematic approach to organizing and analyzing scientific knowledge and information." In each of the areas we regulate, we apply risk assessment practices to the specific task that are widely accepted among relevant domestic and international public health agencies.

For quantitative risk assessments in support of the dissemination of influential information, FDA intends to apply the following principles:

The agency will use
1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices;
2. data collected by accepted methods (if reliability of the method and the nature of the decision justifies use of the data).
In the dissemination of public information about health risks, the agency shall ensure that the presentation of information is comprehensive, informative, and understandable, within the context of its intended purpose.
In a risk assessment document made available to the public, the agency shall specify, to the extent practicable
1. Each population addressed by any estimate of applicable effects;
2. The expected or central estimate of risk for the specific populations affected;
3. Each appropriate upper-bound and/or lower-bound risk estimate;
4. Data gaps and other significant uncertainties identified in the process of the risk assessment and the studies that would assist in reducing the data gaps; and
5. Additional studies not used to produce the risk estimate that support or fail to support the findings of the assessment and the rationale of why they were not used (HHS/FDA, 18-20).

IV. QUALITY INTEGRAL TO CREATION AND COLLECTION OF INFORMATION.

Labor and USDA state the general principle extremely clearly:

The quality assurance process should begin at the inception of the product development process (Labor, 5).

USDA agencies and offices will review the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treat information quality as integral to every step of their development of information, including creation, collection, maintenance, and dissemination (USDA, 3).

The Small Business Administration explicitly included "information development", "information acquisition", and "information maintenance" within the scope of its information quality guidelines:

When SBA develops information, it will use its enterprise architecture as a guide in building the groundwork for the information. This enterprise architecture will help define the goals for the information, information sharing requirements, original and supporting data needs, and all the applications for the information, among other things. SBA will determine and document all requirements for the information (SBA, 4).

At the information acquisition stage, SBA will remain cognizant of potential problems with information quality, including accuracy, currency, and completeness. Wherever possible during the information acquisition process, SBA will verify (assess completeness, accuracy, consistency, currency, timeliness) and validate (assess whether the data are appropriate for the measures it was collected to show) the data it collects, and scrub such data to correct problems. SBA will use lessons learned from this process to improve its information acquisition procedures. SBA also will document limitations on data and other information as a result of problems discovered during the information acquisition stage that SBA could not correct before it disseminates the information (SBA, 4).

SBA will make every effort, within SBA's available resources, to improve the information it maintains, including its data systems or processes. SBA will encourage feedback from both internal and external sources on the quality of SBA's information, and will consider making changes to its information development and acquisition procedures to correct errors and other problems. SBA will conduct information quality assessments, including reviews and inspections of data, comparisons with other sources of similar data, and verification and validation of information and data. SBA also will take steps to ensure that the information SBA maintains remains secure from unauthorized access, revision, falsification, or corruption (SBA, 5).

V. ADMINISTRATIVE MECHANISM TO ADDRESS PUBLIC COMPLAINTS.

Applicable Standards. It is important that the administrative mechanism to address public complaints point out that agency failure to comply with either the OMB or the agency information quality standards can serve as a basis for complaint. For example, "ERS has developed administrative mechanisms to allow affected persons to seek and obtain correction of information disseminated ... that does not comply with OMB, USDA, or ERS Information Quality Guidelines"(ERS/USDA, 14). By citing the OMB, department, and departmental component's guidelines, ERS assures compliance with all of the applicable guidelines and this provision in its guidelines is consistent with the OMB guidelines.

"Affected Persons". HHS and its components ask the complainant to "describe how the person submitting the complaint is affected by the information error"(HHS, 13). SEC invites the complainant to identify the perceived affect "BAan explanation of how the requestor is an affected person with regard to those facts or data"(SEC, 7). - 28 -

SSA and FERC prevent the word "affected" from having any limiting effect by not using it. SSA and FERC make no mention of affected persons in their complaint procedures, and do not require the complainant to explain how he is affected (see, SSA, 1; FERC, 8).

Information Provided to the Agency. HHS encourages complainants to provide needed detail. "Requests for correction that are specific and provide evidence to support the need for correction will enable the agency to provide a satisfactory response"(HHS, 12).

DOT takes the same approach:

DOT may be unable to process, in a timely fashion or at all, requests that omit one or more of the requested elements. DOT will attempt to contact and work with requesters to obtain additional information when warranted (DOT, 15).

Decision Criteria and Burden of Proof for Resolving Complaints. In the preamble to the OMB guidelines, OMB emphasized the discretion agencies had in deciding how to resolve complaints.

Overall, OMB does not envision administrative mechanisms that would burden agencies with frivolous claims. Instead, the correction process should serve to address the genuine and valid needs of the agency and its constituents without disrupting agency processes. Agencies, in making their determination of whether or not to correct information, may reject claims made in bad faith or without justification, and are required to undertake only the degree of correction that they conclude is appropriate for the nature and timeliness of the information involved, and explain such practices in their annual fiscal year reports to OMB (66 FR 49721, September 28, 2001).

Justice emphasizes the limits of its obligation:

After it has completed its review, DOJ will determine whether a correction is warranted, and, if so, what corrective action it will take. Any corrective action will be determined by the nature and timeliness of the information involved and such factors as the significance of the error on the use of the information, the magnitude of the error, and the cost of undertaking a correction. DOJ is not required to change, or in any way alter, the content or status of information simply based on the receipt of a request for correction. The Department need not respond substantively to frivolous or repetitive requests for correction. Nor does the Department have to respond substantively to requests that concern information not covered by the guidelines or from a person whom the information does not affect (Justice, 6).

State articulates the many different ways in which it may respond:

Subject to applicable law, rules and regulations, corrective measures may include, without limitation, personal contacts via letter or telephone, form letters, press releases or postings on the Department website to correct a widely disseminated error or address a frequently raised request. Corrective measures, where appropriate, should be designed to provide reasonable notice to affected persons of any corrections made (State, 5).

Labor stresses practical constraints in correcting errors:

Any structured process would not apply to an agency's archival information or to public filings. Agencies may choose not to respond to complaints about claimed defects that are frivolous or unlikely to have substantial future impact. It may not be in the public interest for agencies to devote significant resources to correcting information where the expenditure of such resources is not, in the agency's view, cost effective in light of the significance of the asserted error, the benefits that are likely to be derived from such a correction, the costs of the correction, and the agency's more pressing priorities and obligations (Labor, 7).

DOT includes economic concerns in its criteria for deciding what and how much to correct:

The costs and benefits of using a higher quality standard or a more extensive review process will be considered in deciding the appropriate level of review and documentation (DOT,13). When the DOT organization determines that a correction of the information is warranted, revisions/corrections to the information in question will begin as quickly as practicable. However, the Department's budget, resources, and priorities, as well as the complexity of the correction task itself, may result in DOT actually taking this corrective action within a reasonable time after the Department has made the determination that a correction is appropriate (DOT, 18).

DOT plans to make both the complaint and subsequent DOT responses available on the web:

In the administrative correction process, DOT will make extensive use of the internet accessible DMS. All requests for correction would come, in the first instance, to the DMS, whether electronically or in hard copy. By docketing requests for correction and subsequent DOT responses in the DMS, the Department will ensure the transparency of the request and response process. The DMS will also electronically notify DOT organizations of pending requests. In addition, filing requests with DMS will allow other interested parties to comment about or make requests with respect to an information issue. For example, suppose DOT publishes a study indicating that 75 percent of a certain kind of accident is caused by a component of a motor vehicle. Manufacturers of that component request correction of the study. Alerted to the request by the DMS posting, vehicle manufacturers could respond within 30 days. The Department seeks comment on this process (DOT, 3).

Time Periods for Resolving Complaints and Any Appeals and Notice to the Public.

EPA takes an indirect approach to setting time limits on the filing of any complaints. EPA exempts what it calls outdated or superseded information from being covered by the EPA guidelines:

The guidelines do not apply to outdated or superseded EPA information that is provided as background information but no longer reflects EPA policy or influences EPA decisions, where EPA indicates (in a disclaimer or otherwise) that the materials are provided as background materials and do not represent EPA's current view (EPA, 15).

An Objective Appeals Mechanism. HHS requires that "the agency official who handles the original complaint will not have responsibility for resolving the appeal"(HHS, 13).

Labor requires that:

The agency should generally provide that the official conducting the second level review is not the same official that responded to the initial request or from the same office that prepared the information in question. Designated agency officials may consult with other agency or Departmental offices, as the agency may deem appropriate to the resolution of the complaint (Labor, 6).

When Interior agrees with an appeal, it also takes steps to notify the public of its decision by withdrawing the information from the public domain.

If at the end of the 45-day period, the bureau or office determines that the complaint is without merit, the complainant will be so notified. If at the end of the 45-day period, the bureau or office determines that the complaint has merit, it shall so notify the complainant, the appropriate program or office, and it shall take reasonable steps to withdraw the information from the public domain and from any decision making process in which it is being used. If the bureau or office determines that it will correct challenged information, it will notify the complainant of its intent and the corrective steps it proposes. The bureau or office may determine the schedule and procedure for correcting the challenged information, but may not disseminate the challenged information in any form until it has been corrected. Upon redisseminating corrected information, the bureau or office will provide the complainant with a copy of the corrected information (Interior, 2-3).

VI. MELDING THE STATUTORY REQUIREMENTS OF SECTION 515 INTO THE PROCEDURAL REQUIREMENTS OF OTHER STATUTES.

Treasury stated its position succinctly:

Certain disseminations of information include a comprehensive public comment process (e.g., notices of proposed rulemaking (NPRM), regulatory analyses, and requests for comment on an information collection subject to the Paperwork Reduction Act). The administrative complaint mechanism described in these guidelines does not apply to such documents. Persons questioning information disseminated in such a document must submit comments as directed in that document. An additional complaint and appeal process for information that is already subject to a public comment process is inappropriate and unfair to other public commenters who submitted timely comments (Treasury, 6-7; Commerce took a similar approach, Commerce, 11).

DOT discusses this issue thoroughly:

[T]here are some circumstances in which there is an existing process to respond to concerns expressed about the DOT's information. The OMB guidelines encourage agencies to make use of existing processes in a flexible way, tailored to their programs. When there is a sound existing process, (such as a process that provides opportunities for public participation in making an agency decision), DOT organizations are asked not to duplicate that process with a separate request response mechanism. For example, when an agency issues a notice of proposed rulemaking (NPRM), it typically describes in the preamble the basis for its proposed regulatory provisions, which may include technical or scientific studies and a regulatory evaluation. In so doing, it disseminates these studies or evaluations, within the meaning of these guidelines. The public comment process can, and often does, generate views from interested persons about the soundness of the underlying information. If someone submits a request for correction pertaining to a document cited in an NPRM, DOT would treat it procedurally like a comment to the rulemaking, responding to it in the preamble of the final rule or a subsequent document such as a Supplemental Notice of Proposed Rulemaking (SNPRM), rather than through the separate request response mechanism of these guidelines. The content of the response would address the issues of the document's compliance with the information quality principles of the OMB and DOT guidelines. (DOT could choose to make an earlier correction, if warranted, assuming so doing would not delay the issuance of the final rule.) This approach would also apply to other processes involving a structured opportunity for public participation on a proposed document before a final document is issued, such as a draft environmental impact statement (EIS), an air quality conformity determination, or a Section 4(f) determination under the Department of Transportation Act ...

On the other hand, with respect to new information appearing for the first time in a final rule or EIS, DOT would consider a request for correction. The Department would not stay the final action involved. However, if it appeared that the information that was the subject of the request did not comply with the guidelines, and that, as a result, the final document was materially flawed, DOT would treat the matter as a request for reconsideration. In such cases, the Department would use any already existing mechanisms and procedures to reconsider corrections, such as the process to petition for a new rule or to request a Supplemental EIS. The submission of a request for correction by itself does not in any way affect the finality of a decision of the Department.

We believe that this approach serves the purposes of the guidelines, affords an opportunity for correction of any material that does not comply with the guidelines, yet does not duplicate effort or interfere with the orderly progress of DOT's work. We seek comment on this approach (DOT, 4-5)

This section concerns requests for correction concerning information on which a DOT organization has sought public comment (e.g., a notice of proposed rulemaking (NPRM), studies cited in an NPRM, a regulatory evaluation or cost-benefit analysis pertaining to the NPRM; a draft environmental impact statement; a proposed policy notice or aviation order on which comment has been sought; a request for comments on an information collection subject to the Paperwork Reduction Act).

The DOT organization's response to the request for correction will normally be incorporated in the next document it issues in the matter concerning which it had sought comment (e.g., in the case of an NPRM, the preamble to the final rule), DOT may choose to provide an earlier response, if doing so is appropriate and will not delay the issuance of the final action in the matter. Once again, the DOT organization will place their response in the DMS. As stated above ... , a DOT organization may reject a request for correction with respect to information in a final document if there was an opportunity for public comment or participation and interested persons could have requested the correction of the information at the proposed stage (DOT, 18).

If there is an existing process for reconsidering a particular sort of information disseminated by DOT, the DOT organization will make use of that process. For example, if the information relates to a final rule a DOT organization has issued, and the DOT organization has an existing process for handling requests for the reconsideration of a final rule, the DOT organization would use that process. If the information relates to a final EIS, the DOT organization may handle the request as though it were a request for a Supplemental EIS (DOT, 19).

Labor included this discussion with its public notice of the complaint mechanism:

This process is not intended to substitute for other legally authorized processes, such as the Privacy Act or the rulemaking processes. Concerns regarding information in a rulemaking must be presented in the rulemaking in accordance with the rulemaking's procedures. ... In deciding how to handle complaints, agencies should be especially mindful of their legal obligations, program priorities, resource constraints, and their duty to use resources efficiently. For example, agencies have important responsibilities to issue rules and provide compliance guidance to the public. Agencies must administer the complaint and appeal process consistent with these obligations and their responsibilities to carry them out in an expeditious manner (Labor, 6-7).

DOT will reject a request for correction of information that could have been raised at the proposed rule stage:

With respect to information in a final rule, final environmental impact statement, or other final document on which there was an opportunity for public comment or participation, could interested persons have requested the correction of the information at the proposed stage? If the DOT organization determines that the answer to [this] Question ... is "yes,"@ DOT will reject your request (DOT, 17).

This page was last modified on October 1, 2002.

Breadcrumb

June 10, 2002 Memo - Agency Draft Information Quality Guidelines