Independent Evaluation of NERC’s Independent... · To evaluate the ROP and SPM processes in place...

Independent Evaluation of NERC’s

Standards Process Manual Requirements

and Standards Applicable to NERC

December 29, 2014

RELIABILITY | ACCOUNTABILITY 2

Table of Contents

• Table of Contents 2

• Executive Summary 3

• Scope and Methodology 6

• SPM/ ROP Section 300 Test Approach 8

Non-Compliance Observations 9

Process Improvement Observations 10

• Standards Applicable to NERC Test Approach 20

Non-Compliance Observations 21

Process Improvement Observations 22

• Appendices 25

Appendix A: NERC Staff 26

Appendix B: Catalog of NERC Documents Reviewed 28

Appendix C: SPM/ROP Section 300 Detailed Observations 32

Appendix D: CCC Audit Observers 42


Executive Summary

As requested by the North American Electric Reliability Corporation (“NERC”) Compliance and Certification Committee (“CCC”) and in accordance with the CCC 2014 Work Plan, program CCCPP-003, an independent evaluation was conducted to evaluate NERC’s Reliability Standards Development processes and procedures that support requirements defined in the NERC Standards Process Manual (“SPM”) and Section 300 of the NERC Rules of Procedure (“ROP”). In addition, the review included NERC-approved Reliability Standards (Standards Applicable to NERC, “SAN”) and the processes NERC implemented to comply with applicable Standards.

Under the direction of Mechelle Thomas, Director of Internal Audit and Corporate Risk Management, and with active participation from CCC Observers, NERC engaged an independent audit firm (“independent auditor”) to conduct a review of ROP Section 300 and SPM procedures as well as review SAN to assess NERC’s compliance with the applicable requirements.

Conclusions As a result of the assessment of NERC’s compliance with the ROP, the SPM and SAN, the independent auditor identified several areas where NERC generally conformed to the ROP, SPM and SAN. Overall, NERC has demonstrated a proactive approach to enhancing the Standards process. Key positive themes identified during the review included:

• Alignment of Standards Development resources with other NERC departments, including Legal, Compliance and Standards to improvethe quality and timeliness of standards development processes;

• Initial development of procedural manuals for conducting standards development processes (e.g. Standard Developer Handbook) thatcan be provided to drafting teams to enhance efficiency and consistency;

• Development of communication plans to identify opportunities for NERC staff and related standards development resources tostreamline communication during the standards development process;

• Development of an enhanced systematic method for storing required documents on the NERC website for Standards Development,with specific emphasis on Standards CIP-014 and PRC-005-2; and

• Development of new technology related to Reliability Standards balloting with enhanced features to address legacy system challenges.


Executive Summary

Conclusions (continued) In addition to identifying areas where NERC generally conformed, the independent auditor also identified non-compliance and process improvement opportunities for NERC to address. To determine whether an observation was an area of non-compliance or an area of process improvement with the ROP/ SPM and SAN requirements, the following criteria were used by the independent auditor:

Non-Compliance: Non-Compliance was based on either the observation of inadequate evidence to demonstrate that NERC complied with the ROP, SPM or SAN or the lack of specific evidence to demonstrate that NERC complied with ROP, SPM or SAN requirements.

Process Improvement Opportunities: Process Improvement was identified where evidence indicated that the ROP, SPM or SAN requirements were achieved; however, additional activities could be implemented to enhance the execution.

The following table highlights the distribution of observations between the ROP/SPM and SAN scope:

In general, independent auditor observation themes include retaining appropriate evidence to demonstrate compliance with the requirements and developing process documentation to reflect the activities NERC conducts to adhere to the SPM.

June 2011 – June 2014 Standards Process Review Observations

Review Area # of Non-Compliance Observations # of Process Improvement Observations

Standards Applicable to NERC (SAN) 1 3

Standards Process Manual (SPM)/ ROP 1 10

Total 2 13


Executive Summary

Recommendations The two specific recommendations for the non-compliance observations included: • Developing procedural documents to assist NERC in conducting non-binding polls for revised VRFs and VSLs associated with a

change to a Requirement in a Standard; and• Developing procedures to ensure a risk-based assessment methodology (“RBAM”) is consistently performed by appropriate

individuals and clearly documented.

A summary of the recommendations for the process improvement observations that NERC should consider include:

ROP/ SPM: • Review the ROP to promote consistency in alignment with requirements in the SPM;• Utilizing existing tools and technology to enhance the efficiency and automation of incoming requests; and• Maintaining and regularly updating internal process and procedure manuals to reflect current NERC activities.

SAN: • Retaining documentation to evidence requirements with Reliability Standards applicable to NERC; and• Timeliness of evidencing compliance with Reliability Standards applicable to NERC.

For each observation, the corresponding recommendations include specific and actionable activities that NERC could implement to enhance overall conformance with Section 300 of the ROP, the SPM and SAN, respectively. While NERC has demonstrated a commitment to further improve its activities supporting the ROP, SPM and SAN, NERC also has opportunities to enhance several of its key processes. Please refer to the detailed observations section of the report for specifics on the observations of non-compliance and process improvement along with corresponding recommendations to support the ROP, SPM and SAN requirements.


Scope and Methodology

Scope and Methodology To satisfy the NERC CCC request, the independent auditor conducted a review of NERC’s conformance with the Standards Development section of the ROP (section 300) , the SPM and SAN to cover the three year period between June 2011 and June 2014. As part of the NERC independent evaluation of the Standards Development process, the independent auditor performed the following procedures between July 2014 and September 2014: • Met with key process owners including Standards, Legal, Information Technology (“IT”) and Compliance, and gained an

understanding of Section 300 of the ROP /SPM and SAN policies, processes and procedures ;• Assessed whether NERC is performing its responsibilities as set forth in the ROP and SPM and whether existing policies, processes

or procedures support the requirements outlined in the NERC ROP with respect to Reliability Standards Development process;• Identified areas of non-compliance and process improvement opportunities, as needed.

The scope of the ROP included in the review was as follows: • Section 300: Reliability Standards Development• Appendix 3A: Standards Process Manual (prior to and post July 2013 SPM modifications)• Appendix 3D: Registered Ballot Body Criteria

Additionally, as part of Section 100, assessed NERC’s compliance with each approved Reliability Standard that identified NERC or the Electric Reliability Organization as a responsible entity.


Scope and Approach

To support conformance with the ROP/SPM and SAN, this report includes the independent auditor’s observations on areas where NERC generally conformed, areas of non-compliance with the ROP/ SPM and SAN and specific process improvement opportunities. The criteria used to determine Non-Compliance was based on either the observation of inadequate evidence to demonstrate that NERC complied with the ROP, SPM or SAN or the lack of specific evidence to demonstrate that NERC complied with ROP, SPM or SAN requirements. The criteria used by the independent auditor to determine a Process Improvement was focused upon evidence that indicated the ROP/SPM or SAN requirements were achieved; however, the independent auditor identified additional activities which could be implemented to enhance execution.

Between July 2014 and September 2014, the independent auditor met with NERC staff and performed detailed testing procedures on processes supporting the requirements of section 300 of the ROP/ SPM and SAN. Please refer to Appendix A for the listing of NERC staff interviewed during the independent evaluation and corresponding processes reviewed within Section 300 of ROP, SPM and SAN. This report summarizes the meetings with key members of NERC staff and testing procedures performed by the independent auditor, as well as the independent auditor's assessment of the areas of general conformance, areas of non-compliance and areas of process improvement within section 300 of the ROP, the SPM and SAN.

The independent auditor’s services were performed in accordance with Standards for Consulting Services established by the American Institute of Certified Public Accountants. The independent auditor's work was limited to the specific procedures and analysis described herein and was based only on the information made available through September 2, 2014. Accordingly, changes in circumstances after this date could affect the findings outlined in this report.


SPM/ ROP Section 300 Test Approach

To evaluate the ROP and SPM processes in place between June 2011 and June 2014, the independent auditor met with seven (7) individuals on NERC staff between July 2014 and September 2014 and collected supporting evidence across the three year review period to ensure that NERC met all requirements as specified within the ROP/SPM1. Please refer to Appendix A for the list of NERC individuals involved in this review and to Appendix B for the evidence collected during the review. Through review of the evidence collected and inquiry of process owners to gain an understanding of the process, the independent auditor performed tests of the ROP/ SPM requirements to determine compliance. During walkthroughs and testing, the independent auditor reviewed section 300 of the ROP and all sections of the SPM and inspected or observed corresponding documentation. Please Appendix C for the sections included in the review and a summary of testing procedures performed.

To cover the full three year period between June 2011 and June 2014, the independent auditor reviewed a sample of Standards Development processes prior to the June 2013 SPM modification and post modification, as applicable.

Throughout the review, NERC staff was very accommodating and responsive to the needs of the independent auditor. NERC worked diligently to provide supplemental information requested by the independent auditor and answered questions in a timely manner.

1 – The Independent auditor identified a total of forty-two (42) ROP/ SPM requirements out of three hundred and forty-five (345) in this independent review, that were not easily testable, due to the nature of the requirement (i.e. non-tangible requirements that were not directed towards NERC staff, Standards Committee, etc.). Therefore, these specific requirements were removed from the scope of the independent review. Please refer to Appendix C for a complete listing of the statements not easily testable.


SPM/ ROP Section 300: Non-Compliance Observations

# Review

Area RoP/SPM Statement Observation Recommendation

1

Priority:

Higher

Standards Process Manual

Section:

SPM 4.9

Paragraph:

3

There is no requirement to conduct a new non-binding poll of the revised VRFs and VSLs if no changes were made to the associated standard; however, if the requirements are modified and conforming changes are made to the associated VRFs and VSLs, another non-binding poll of the revised VRFs and VSLs shall be conducted.

In cases where the requirements in astandard were modified andcorresponding changes were madeto the associated VRF or VSL, theindependent auditor was not able toobserve that another non-binding pollwas conducted.

NERC should develop, retain andannually update procedural documentsfor conducting non-binding polls ofrevised VRFs and VSLs.

NERC should conduct non-binding pollspursuant to the language in the SPM

NE

RC

Res

po

nse

NERC agrees additional education would be beneficial to ensure compliance with this aspect of the SPM. NERC will create an internal procedure to fulfill this SPM requirement. NERC staff will be trained to implement the procedure by the end of Q1 2015.

Targeted completion date – Q1 2015


SPM/ ROP Section 300: Process Improvement Observations

# Review


2

Priority:

Lower


Section:

SPM 4.14/ 4.2

Paragraph:

1

When the drafting team has reached a pointwhere it has made a good faith effort atresolving applicable objections and is notmaking any substantive changes from theprevious ballot, the team shall conduct a“Final Ballot.”

If a SAR for a new Reliability Standard isposted for a formal comment period, theStandards Committee shall appoint adrafting team to work with the NERC Staffcoordinator to give prompt consideration ofthe written views and objections of allparticipants.

The following ambiguousterms are referenced, but not currently defined in the SPM: SubstantivePrompt

NERC should review the uses of thewords substantive and prompt withthe Standards Committee (SC) andtake action to either remove thewords from the SPM or provide asolution to address the ambiguity.

NE

RC

Res

po

nse

Further definition of these terms is not necessary. NERC’s use of the terms has been consistent and are generally accepted terms. However, NERC will consult with the Standards Committee leadership in Q4 2014 on whether they agree with NERC staff’s view regarding the terms.

For “prompt”, while the term suggests a fast response, the focus of the SPM is to help ensure that comments are responded to within a

reasonable period of time, which may differ for each project. Identifying a specific time period may be counterproductive as there may be instances where the SC and NERC agree to postpone or delay project postings, which could run afoul of the specified time period.

For “substantive”, the SPM already provides sufficient information as to the term’s meaning. The term is adequately defined and there is a check in place on how the judgment call was made. For example, Section 4.14 describes the types of changes that would be non-substantive and specifies that if there is a question on whether a change is substantive, the SC has the final decision.

Targeted completion date – TBD with the Standards Committee leadership if further refinement is necessary



# Review


3

Priority:

Lower


Section:

RoP 317/ SPM 13

Paragraph:

1

SPM: All Reliability Standards shall be reviewed at least once every ten years from the effective date of the Reliability Standard or the date of the latest Board of Trustees adoption to a revision of the Reliability Standard, whichever is later.

RoP: NERC shall complete a review of each NERC Reliability Standard at least once every five years, or such longer period as is permitted by the American National Standards Institute, from the effective date of the Reliability Standard or the latest revision to the Reliability Standard, which ever is later.

The language related to the reviewof Reliability Standards in the RoP and SPM may be contradictory.

Section 300 of the RoP states thatNERC shall complete a review at least once every five years from the effective date, or the latest revision, which ever is later. However, Section 13 of the SPM states that all Reliability Standards shall be reviewed at least once every ten years from the effective date or the date of the latest revision, whichever is later.

NERC staff should work with theStandards Committee to initiate anupdate to the RoP to reflect thecurrent periodic review process(as documented in the SPM),which is that all ReliabilityStandards shall be reviewed atleast once every ten years fromthe effective date of the ReliabilityStandard or the date of the latestBoard of trustees (“BOT”) adoptionto a revision of the ReliabilityStandard whichever is later.

NE

RC

Res

po

nse

NERC standards staff will work with NERC legal staff to create updated changes to the ROP to match the 10-year period from the SPM. In this revision process, NERC staff will also consider whether to add a reference to the ANSI process, which requires a five-year review period for any NERC standards that are approved as “American National Standards” by ANSI. NERC currently does not have, and does not have plans to have any standards that are approved as “American National Standard”.

Updates to the ROP are initiated by NERC and do not require SC involvement since the change is to the general ROP.

Targeted start date – Q1 2015 (completion date is open ended due to necessary FERC action and scheduling of ROP changes)



# Review


4

Priority:

Lower


Section:

SPM 4.19 (and various other statements)

Paragraph:

1

Upon identification of a need to retire a Reliability Standard, Variance, Interpretation or definition, where the item will not be superseded by a new or revised version, a SAR containing the proposal to retire a Reliability Standard, Variance, Interpretation or definition will be posted for a comment period and ballot in the same manner as a Reliability Standard.

For Retirements, Remands,

Definitions, Variances andInterpretations, the current SPMrefers to activities beingconducted in the ‘same manner asa Reliability Standard,’ andtherefore, does not clearly statethe manner in which thesedevelopment activities should becarried out.

NERC should develop, retain and annuallyupdate procedural documents for developingRetirements, Remands, Definitions,Variances and Interpretations, beginning withany informal outreach and ending with theapproval of the standard by the FederalEnergy Regulatory Commission (“FERC”),along with the appropriate individual or groupresponsible for completing each activity(Drafting Team, Standards Committee, etc.).

NERC should maintain and develop asystematic method of storing requireddocuments on the NERC website by using aconsistent document retention tool across theorganization.

NE

RC

Res

po

nse

NERC staff agrees with the goal of providing clarity regarding the manner in which the processes listed in the ROP/SPM statement should be

carried out. NERC will create process documents that track the current SPM for each area, including a review and update of documents

where NERC already has a document created. (Targeted completion date – Q1 2015)

NERC supports the goal of improving document tracking and storage. There is currently a company-wide project underway addressing

document storage. The purpose of that project is to create a consistent tool for use across the organization. (Targeted completion date – TBD

pending the company-wide document management project results)



# Review


5

Priority:

Lower


Section:

SPM 4.6

Paragraph:

1

The NERC Reliability Standards Staff shall coordinate a quality review of the Reliability Standard, implementation plan, and VRFs and VSLs in parallel with the development of the Reliability Standard and implementation plan, to assess whether the documents are within the scope of the associated SAR, whether the Reliability Standard is clear and enforceable as written, and whether the Reliability Standard meets the criteria specified in NERC’s Benchmarks for Excellent Standards and criteria for governmental approval of Reliability Standards.

The independent auditor observedthat NERC has an informal and inconsistent process for carrying out quality reviews on Standards Development.

NERC should formalize the processaround the quality review (“QR”) byimplementing and completing thequality review checklist for eachquality review (standard, definition,variance, etc.) for consistency andcompleteness of the review.

NERC should provide training forthe quality reviewers in order topromote consistency of reviews

In addition, guidelines shouldensure that quality reviewersremain independent from thedrafting team.

NE

RC

Res

po

nse

NERC Staff is developing a Quality Review process to ensure consistent consideration of key elements, across standards development projects. This project will be completed and communicated to the Standards Committee by the end of Q1 2015. Training will be provided to NERC staff by early Q2 2015 following submission to the SC.

Targeted completion date – Q1 2015 for development of consistent considerations; early Q2 2015 for training.



# Review


6

Priority:

Lower


Section:

SPM 3.10Paragraph:

1

The drafting team and the Compliance Monitoring and Enforcement Program Staff shall work together during the Reliability Standard development process to ensure an accurate and consistent understanding of the Requirements and their intent, and to ensure that applicable compliance tools accurately reflect that intent.

The independent auditor observed thatNERC has an informal process in place toevidence compliance that the drafting teamand the CMEP staff shall work togetherduring Standard development.

NERC should retain evidence to showcompliance with SPM 3.10

NERC should maintain and develop asystematic method of storing requireddocuments on the NERC website byusing a consistent document retentiontool across the organization.

NE

RC

Res

po

nse

NERC will develop a “compliance/standards interaction tracking sheet” that will be completed for each project to evidence the collaboration between the two departments, thus documenting the fulfillment of this aspect of the SPM. This tracking sheet will be developed and will be used for all new projects that are started in 2015.

NERC currently has a company-wide project underway addressing document storage. The purpose of that project is to create a consistent tool for use across the organization.




# Review


7

Priority:

Lower


Section:

SPM 4.18

Paragraph:

1

SPM Section 4.18: “Withdrawal of a Reliability Standard, interpretation or Definition.”

While the current SPM includes theprocess of withdrawing a ReliabilityStandard, Interpretation or Definition, thecurrent SPM does not include theprocess of withdrawing an appeal.

NERC should work with the StandardsCommittee to initiate an update toSPM Section 4.18 to include theprocess for withdrawing an appeal.

NE

RC

Res

po

nse

NERC staff will include this issue as an item for consideration for the next SPM revision.

Targeted completion date – TBD with the Standards Committee leadership



# Review

Area

RoP/SPM

StatementObservation Recommendation

8

Priority:

Lower


Section 300/ Appendix 3A (SPM)

Multiple As each entity is only permitted one entry into theRegistered Ballot Body (RBB), and memberscannot be substituted in the RBB when jobpositions at the company change, manual votesare entered during a Final Ballot.

Manual entries are submitted to NERC via emailand votes can be manually entered into the NERCBalloting System by System Administrators.There is currently not a formalized process totrack manual votes for voters who were notincluded in the RBB.

NERC should consider implementing a reviewprocess in which an independent review isperformed of all manual votes input by NERCSystem Administrators, in order to ensure thatthe manual votes are submitted as requested.

NE

RC

Res

po

nse

NERC has a process in place and disagrees that a more formal tracking process is necessary. Today, there is an external control in place as, when a manual vote is entered, the balloting system automatically sends a notification of the vote to the entity, thereby providing a timely initial check on the vote. Final votes are also made public, providing an opportunity for validation by the entity.




# Review


9

Priority:

Lower


Section:

SPM 4.8

Paragraph:

N/A

No Registered Ballot Body member may join or withdraw from the ballot pool once the first ballot starts through the point in time where balloting for that Reliability Standard action has ended. The Director of Standards may authorize deviations from this rule for extraordinary circumstances such as the death, retirement, or disability of a ballot pool member that would prevent an entity that had a member in the ballot pool from eligibility to cast a vote during the ballot window. Any approved deviation shall be documented and noted to the Standards Committee.

Although a ballot pool review(“clean up”) was performed toapprove any changes in ballot poolmembers from the first ballotthrough the end of balloting forCOM-002-4, PER-005-2 and CIP-014, the independent auditor werenot able to observe evidence ofdocumented approval for changesin ballot pool members for PER-005-2 and CIP-014.

NERC should work with theStandards Committee and theDirector of Standards to retainevidence of approval for anychanges in ballot pool membersfrom the initial ballot to the endof balloting.

NE

RC

Res

po

nse

NERC staff supports the recommendation and will implement changes to better maintain records in future cases.

Targeted completion date – Q1 2015 (Dependent on SC concurrence)



# Review

Area

RoP/SPM

StatementObservation Recommendation

10

Priority:

Lower


Section 300/ Appendix 3A (SPM)

Multiple Currently, all incoming SARs,interpretation requests, variances, errors, waivers, remands and appeals may not be tracked for approval or rejection.

NERC should implement procedures to track andretain all incoming SARS, interpretation requests,variations, errors, waivers and appeals. This couldbe implemented using existing workflow tools thatwould allow the request to be routed, approved orrejected and permanently tracked.

NE

RC

Res

po

nse

The Manager of Standards Information will review our current processes and recommend internal process improvements to the Director of Standards by Q2 2015.




# Review Area Observation Recommendation

11

Priority:

Lower

Standards Process Manual SPM – General

In response to paragraphs 157, 158 and 159 of FERC Order 693, NERC developed an interim and long term plan and schedule to eliminate” references to the ‘regional reliability organization’ in Reliability Standards…” In currently enforceable standards, references to the RRO in the Applicability section, still remain. Additionally, in the listing of enforceable Standards on NERC”s website (“US Enforcement Status/Functional Applicability” section), there are currently enforceable Reliability Standards where RRO has unique accountability (IRO-001-1 and MOD-016-1.1).

Although NERC is in the process of removing RRO as an applicable entity from Standard IRO-001-3 and retiring Standard MOD-016-1, NERC should work with the Standards Committee to complete this activity and continue to perform a review of Standards and Requirements that have not been reviewed and consider: Reviewing the Standards to assess whether

currently enforceable Standards add value to reliability and at least one Entity is accountable for all currently enforceable Standards.

Reporting to the BOT when Standards no longer add value to reliability or when entities are no longer applicable to that Standard, and consider retiring the Standard.

NE

RC

Res

po

nse

All standards related to the Recommendation are currently being modified in projects. NERC will complete activity in these projects to close

out addressing the issue in FERC Order No. 693.


To evaluate the Standards applicable to NERC between 2011 and 2014, the independent auditor met with 10 (ten) individuals on NERC staff between July 2014 and September 2014 and collected supporting evidence across the three year review period. As part of Section 100, assessed NERC’s compliance with each approved Reliability Standard that identified NERC or the Electric Reliability Organization as a responsible entity. Please refer to Appendix A for the list of NERC individuals involved in this review and to Appendix B for the evidence collected during the review. Through review of the evidence collected and inquiry of process owners to gain an understanding of the processes NERC has in place to assess the Standards applicable to NERC, the independent auditor performed tests of the Reliability Standards that were determined to be applicable to NERC, including:

• Review of NERC's attestation on whether they are an owner operator or user of the BPS, including Cybersecurity Questionnaires,Self Certifications and the Draft Letter of Applicability of NERC’s Reliability Standards to NERC; and

• Review of the Risk Based Audit Methodology for required compliance with CIP-002-3 and CIP-003-3, R2

Throughout the review, NERC staff was very accommodating and responsive to the needs of the independent auditor. NERC worked diligently to provide supplemental information requested by the independent auditor and answered questions in a timely manner.

Refer to the subsequent pages for the areas of non-compliance and process improvement, along with the corresponding recommendations.

Standards Applicable to NERC Test Approach


Standards Applicable to NERC: Non-Compliance Observations


12

Priority:

Higher

Standards Applicable to NERC

According to Reliability Standard, CIP-002-3, R1(in which NERC certified that it is subject to), “theResponsible Entity shall identify and document aRBAM to use to identify its Critical Assets.” Theindependent auditor were unable to observeupdated evidence of the RBAM covering the period1/1/2013-12/31/2013, in order to meet theRequirement of CIP-002-3, R1.

Consistent with NERC’s Standards compliance memo, NERC is required to comply with CIP- 002-3 R1. NERC should develop procedures to ensure an RBAM is consistently performed by appropriate individuals and clearly documented. In order to evidence compliance with CIP-002-3 R1, corresponding documentation of the RBAM should be stored in an appropriate location to evidence the inventory of Critical NERC Assets (or lack thereof) identified during the RBAM process.

NE

RC

Res

po

nse

NERC will work with its CIP staff to develop procedures to ensure the performance of the RBAM is consistently documented by

appropriate individuals.



Standards Applicable to NERC: Process Improvement Observations


13

Priority:

Lower


According to Reliability Standard, CIP-002-3, R1, in which NERC certified is applicable to NERC, “the responsible entity shall identify and document a risk-based assessment methodology to use to identify its Critical Assets.” Although NERC prepared an RBAM for 2011 and 2012 to meet Requirement 3 of CIP-002-3, the independent auditor observed informal documentation evidencing the RBAM performed for these periods.

NERC should work to enhance the evidence and retention of the RBAM documentation in one central location in order to more clearly evidence compliance with Requirement 1 of CIP-002-3.

NE

RC

Res

po

nse

NERC will provide this recommendation to the CIP staff who will assist NERC in improving its documentation related to CIP-002-3.





14

Priority:

Lower


In NERC’s 7/12/2013 self-certification, covering the period 1/1/2012 – 12/31/2012, NERC certified that “NERC, as a NERCnet User Organization, is subject to COM-001-1, Requirement 6, which mandates adherence to Appendix 1, NERCnet Security Policy.” However, as of July 2014, NERC stated that NERC is not a NERCnet User Organization, and therefore, is not subject to COM-001-1, R6.

NERC should continue to review the requirements within Reliability Standard COM-001-1 as well as NERC’s current role with NERCnet to determine if NERC is a user of the tool. Based upon the continuation of this review, NERC should finalize the draft memo as to whether or not NERC is considered a user of NERCnet. If determined to be a user of NERCnet, NERC should evidence compliance with Requirement 6 of the Standard.

NE

RC

Res

po

nse

NERC will finalize the draft memorandum.





15

Priority:

Lower


NERC certified that it is subject to Reliability Standards CIP-002-3 and Requirement 2 of CIP-003-3. Evidence of compliance with these CIP Standards for the periods 1/1/2011 – 12/31/2011 and 1/1/2012 – 12/31/2012 occurred six months after the end of the period, each year: Year Ending 12/31/2011:

• Cybersecurity Questionnaire: Completed July 2012 • Self Certification: Completed June 2012

Year Ending 12/31/2012: • Cybersecurity Questionnaire: Completed July 2013 • Self Certification: Completed July 2013 • Cybersecurity Attestation Review Notes: July 2013

To enhance the performance of the compliance review process, NERC should improve the timeliness of the the review in alignment with the applicable year. NERC should define a standard timeline in which the review should be completed with specific focus on conducting the review more timely.

NE

RC

Res

po

nse

NERC will establish timeframes to ensure the timeliness of NERC’s review.


Appendices


Appendix A: NERC Staff

NERC Staff – Section 300 ROP/SPM

Process NERC Staff Title Initial Interview Date

Standards Process Manual - General

Laura Hussey Director of Standards Development

July 9, 2014

Monica Benson Reliability Standards Analyst

Barbara Nutter Manager of Standards Information

Valerie Agnew Director of Standards

Ryan Stewart Standards Development

Stacey Tyrewala Legal and Regulatory

William Edwards Legal and Regulatory

Shamai Elstein Legal and Regulatory

Between July 2014 and September 2014, the independent auditor met with NERC staff supporting the Section 300 of the ROP and SPM processes. Please refer to the chart below for the staff interviewed as well as the date of the initial meeting. Additional meetings were held with the individuals below as necessary on an ad-hoc basis throughout July and September 2014.


Appendix A: NERC Staff

NERC Staff –SAN

Process NERC Staff Title Initial Interview Date

Standards Applicable to

NERC

Laura Hussey Director of Standards Development

July 16, 2014

Valerie Agnew Director of Standards

Barbara Nutter Manager of Standards Information


Earl Shockley Compliance Analysis and Certification

Marvin Santerfeit Information Technology

Jerry Hedrick Regional Oversight - Compliance

Shamai Elstein Legal and Regulatory


Jeff Hicks Associate Director of Information Technology

Between July 2014 and September 2014, the independent auditor met with NERC staff supporting the Standards Applicable to NERC. Please refer to the chart below for the staff interviewed as well as the date of the initial meeting. Additional meetings were held with the individuals below as necessary on an ad-hoc basis throughout July and September 2014.


Appendix B: Catalogue of NERC Documents Reviewed

Between July 2014 and September 2014, the independent auditor reviewed internal NERC documentation related Section 300 of the RoP/SPM and SAN processes supporting the requirements. Please refer to the following chart for a summary of key documents used as evidence during the review. Evidence used by the independent auditor during the review was either reviewed via hard copy or soft copy or observed on-screen with the assistance of NERC staff, in order to maintain confidentiality.



Section 300 of the ROP/SPM and SAN Data Request Catalog

Ref. # Review Area Document Name

1 SPM Section 2 SPM Section 4 ROP Section 311-312

Final Version of the Reliability Standard for sample of 15 Standards(INT-009-2, PER-005-2, MOD-033, MOD-031, MOD-001-2, EOP-010-1, VAR-001-4, VAR-002-3, CIP-014, COM-002-4, PRC-023-2, PRC-025-1, BAL-001-2, PRC-005-2, IRO-001-3), 1 Regional Reliability Standard (BAL-004-WECC-02)

2 SPM Section 2 SPM Section 4 Final Version of the Implementation Plan for sample of 15 Standards

3 SPM Section 2 SPM Section 4 White Papers and Technical Papers for sample of 15 Standards

4 SPM Section 2 SPM Section 4 VRF and VSL Justification for sample of 15 standards

5 SPM Section 4 Completed Standard Authorization Request (SAR) for sample of 12 SARs (PER-005-2, MOD-033, MOD-031, MOD-001-2, EOP-010-1, VAR-001-4, VAR-002-3, CIP-014, PRC-023-3, PRC-005-2, Cold Weather SAR, Programmable Logic Computers SAR)

6 SPM Section SPM Section 2 Ballot Results page for sample of 15 Standards and evidence of open balloting participation

7

SPM Section 2 SPM Section 4 SPM Section 5 SPM Section 7 SPM Section 13 SPM Section 14 SPM Section 15 ROP Section 311- 312

Project Page for sample of 15 Standards, 7 Interpretations (CIP-003-3, CIP-005-3, CIP-002-4 to CIP-009-4 & EOP-005-2, PRC-006-1, PRC-005-1b), Definitions (BES - Bulk Electric System), 4 Periodic Reviews (FAC-014-2, BAL-004-0, FAC-011-2, IRO-003.2) , Regional Reliability Standards for Regional Entities (RFC, TRE, SERC, NPCC, WECC, FRCC, MRO and SPP), and evidence of public access to Reliability Standards information and the 2013 updates to the SPM

8 SPM Section 2 SPM Section 4 Initial Ballot Results and Final Ballot Results of the Final ballot for sample of 15 standards

9 SPM Section 2 SPM Section 4 SPM Section 7

Consideration of Comments Received on the project page for sample of 15 standards, 7 interpretations and sample of 12 SARs

10 SPM Section 2 SPM Section 4 Consideration of comments for sample of 15 Standards





11 SPM Section 2 SPM Section 4 Standards Announcement for sample of 15 Standards

12 SPM Section 2 SPM Section 4 Drafting Team Roster Page for sample of Standards and sample of 12 SARs

13 SPM Section 2 SPM Section 4 Related Files Page for sample of 15 standards

14 SPM Section 2 SPM Section 5 SPM Section 7

Quality Review emails and evidence for sample of 15 Standards, sample of 1 Definition and sample of 7 Interpretations

15

SPM Section 2 SPM Section 4 SPM Section 16 ROP Section 311-312

Board of Trustees Meeting Minutes for approvals for sample of 15 Standards, sample of 3 Withdrawals (BAL-004-1, MOD-025-RFC-01, PRC-066-SPP-01), sample of 3 Waivers (Project 2014-04, Project 2010-17, Cyber 706 Standard) and 1 sample Regional Reliability Standard

16

SPM Section 4 SPM Section 5 SPM Section 13 SPM Section 15 SPM Section 16 ROP Section 309 & 321

Standards Committee Meeting Minutes for approvals for sample of 15 standards, sample of 3 Waivers, sample of 12 SARs, sample of 1 definition, sample of 4 Periodic Reviews, sample of 2013 Modification to the Standards Process Manual sample of 2 Remands

17 SPM Section 2 SPM Section 4 NERC Standards and Numbering Convention document

18

SPM Section 4 SPM Section 12 SPM Section 16 ROP Section 311-312

US Enforcement Status/Functional Applicability spreadsheet for sample of 3 Withdrawals, sample of 3 Waivers, sample of 1 Regional Reliability Standard and sample of 4 Errata

19 Misc. Staff Organizational Chart for evidence of titles of employees

20 SPM Section 16 Standards Oversight and Technology Committee agenda package for sample of 3 Waivers





21 SPM Section 8 Notice of Appeal for sample of 3 Appeals submitted by Canadian Electricity Association & Midcontinent Independent System Operator, Exelon Corporation and Northwest Power Pool Reserve

22 SPM Section 8 Evidence of response to appellants from NERC for sample of 3 Appeals

23 SPM Section 4 SPM Section 13 ROP Section 310 &321

Reliability Standards Development Plans (RSDP) for sample of 12 SARs, 4 Periodic Reviews and evidence of the filing of the 2011-2014 RSDP

24 SPM Section 5 NERC Glossary of Terms for sample of Definitions

25 SPM Section 13 ROP Section 319 Screenshot of archived folder on a shared drive

26 ROP Section 311-312

Notice of the North American Electric Reliability Corporation of Remand for sample of 1 FERC Remand (TPL-002-0b)

27 ROP Section 311-312

Report of the North American Electric Reliability Corporation of Plan and Timetable for Modification or Development of Reliability Standard for sample of 1 FERC Remand

28 ROP Section 310 &321 NERC 2012 Standards Report Status and Timetable for Addressing Regulatory Directives and Periodic Review of Reliability Standards for the filings of the annual Reliability Standards Development Plan

29 SPM Section 15 Docket No.RR09-6-003 for the 2013 Modification to the SPM

30 ROP Section 311-312 §385.1903 Notice in rulemaking proceedings (Rule 1903) for remand TPL-002-0b for sample of 1 FERC Remand

31 ROP Section 316 Procedure Compliance Form for ANSI Accreditation in 2011- 2014 32 SPM Section 3 Standards Committee Roster for evidence of a non-voting secretary 33 SPM Section 1 NERC Employee Code of Conduct for evidence of ethical requirements

34 ROP Section 304 and 305 Email evidence of review of balloting segment self selection

35 SAN 2011 and 2013 Cyber Security Questionnaire, 2011 and 2012 Self Certification

36 SAN 2013 Cyber Security Attestation Review

37 SAN Draft Letter of Applicability of NERC Reliability Standards to NERC


Appendix C: SPM/ ROP Section 300 Detailed Observations

Review Area Review Area Title Summary of Testing Procedures Performed

Result of Testing

Procedures

Performed

Section 300 of the ROP

Reliability Standards Development

Regional Reliability Standards Through review of the Regional Reliability Standards Project Page and documentation supporting the Standards development process, including BOT minutes and the comment period window, the independent auditor noted that the regional entities devote significant effort in gathering supporting Standards Development documents to one central location on the Project Page, to ensure that Standards are appropriately reviewed and vetted prior to implementation and enforcement, in accordance with the SPM and ROP section 300. Special Rule to Address Certain Regulatory Directives (Remands) Through review of the FERC Orders/Rules addressed to NERC and the NERC response, the independent auditor noted that Applicable Governmental Authorities must be notified of the Remand within five business days and be provided with a plan to implement the Remand within 30 days.

Process Improvement(s): #4

SPM Section 1.0 Introduction Through review of Section 1.0 of the SPM, noted that this section provides a background, scope and overview of the activities related to the NERC Standards development process. Refer to the remaining testing procedures below for testing performed to cover this section.

General Conformance

SPM Section 2.0 Elements of a Reliability Standard

Through review of completed Reliability Standards, the independent auditor noted that there are specific Requirements for each Standard to ensure that each Standard is enforceable and consistently presented (i.e. Standards and Numbering Conventions, geographical applicability, etc.).

General Conformance

SPM Section 3.0 Reliability Standards Program Organization

Through review of the Reliability Standards Project Page and documentation supporting the Standards development process, the independent auditor noted that the Standards Development process consists of organizations with specific tasks and responsibilities for the timely development a of quality Reliability Standard.

Process Improvement #6




Result of Testing

Procedures

Performed

SPM Section 4.0

Process for Developing, Modifying, Withdrawing or Retiring a Reliability Standard

New/Revised Standards Through review of the Reliability Standards Project Page and documentation supporting the Standards development process, including Standards Committee meeting minutes, BOT minutes, Standards Authorization Requests (SARs) and the Project Tracking Schedule, the independent auditor noted that NERC devotes significant effort in gathering supporting Standards Development documents to one central location on the Project Page, to ensure that Standards are appropriately reviewed and vetted prior to implementation and enforcement, in accordance with the SPM and ROP section 300. Further, the independent auditor observed that the Standards development process includes the development of the Standards Drafting team and the Quality Review teams to promote consistency in the Standards development process. Balloting Per review of the Reliability Standards Project Page and consideration for comments section, the independent auditor noted that balloter’s comments and NERC’s responses to comments are publicly posted on NERC’s website. Through review of the SPM and inquiry with NERC staff, noted that there is a balloting process in place to approve or reject a Reliability Standard and the calculation used to count the votes to ensure stakeholder interests are fairly represented, including the weighted Segment votes, affirmative, abstentions, non-responses and negative votes. Through review, the independent auditor noted that the ballots are conducted electronically and the results of the ballot are posted online for public access. Using the balloting results available on NERC’s website, the independent auditor reperformed the balloting calculation for a Reliability Standard prior to the 2013 SPM change and after the 2013 SPM change. Through access to the balloting records online and documentation of the balloting calculation in the SPM and Appendix 3D of the ROP, NERC has provided transparency into the balloting process.

New/Revised Standards Non-Compliance #1 Process Improvement(s): #2, #4, #5, #7 and #10 Balloting Process Improvement(s): #8 and #9




Result of Testing

Procedures

Performed

SPM Section 5.0 Process for Developing a Defined Term

Through review of the Implementation Plans for new Defined Terms, noted that the development of new Definitions are formally documented in the Plan. Further, per review of the Project Page, Standards Committee meeting minutes and BOT minutes, noted that new Defined terms require vetting and appropriate approvals prior to the implementation of the Definition, similar to the development of a Reliability Standard.


SPM Section 6.0 Process for Conducting Field Tests and Collecting and Analyzing Data

During review of the Reliability Standards development process, the independent auditor reviewed the corresponding Field Test page, if applicable, and noted that the results of the Field Test would be included with the SAR on the Standard’s Project page. Additionally, updates on the Field Tests are provided to the Standards Committee. Through this process, noted that if a Field Test is implemented, a waiver could be issued to preclude the entity from compliance with current Requirements of an approved Reliability Standard that is undergoing revision.

General Conformance

SPM Section 7.0 Process for Developing an Interpretation

Per review of the Standard Committee meeting minutes, noted that Interpretations are authorized prior to posting. Further, per review of the Consideration of Comments section of the Reliability Standard Project Page, noted that comment periods are 30 days (first formal) or 45 days (second formal) and that the initial ballot occurs during the final ten days of the second formal comment period, which provides the entities with sufficient time to provide relevant comments and for NERC to respond to all comments, in accordance with the SPM.

Process Improvement #4 Process Improvement #10

SPM Section 8.0 Process for Appealing an Action or Inaction

Through review of the Notice of Appeal for a sample of appeals submitted, noted that an appeal must be submitted within 30 days of the action that provoked the appeal and requires a response from the Director of Standards within 45 days. Further, the independent auditor reviewed the Reliability Standards project page for responses to the appellants and to determine whether supporting documentation on appeals were publically available.





Result of Testing

Procedures

Performed

SPM Section 9.0 Process for Developing a Variance

Refer to testing procedures performed in section 4 of the SPM. Section 9 was tested in conjunction with the previously noted sections.

Process Improvement(s): #4 and #10

SPM Section 10.0

Process for Developing a Reliability Standard Related to a Confidential Issue

Per inquiry of NERC staff (refer to Appendix A) and the Reliability Standards Project Pages, noted that there were no confidential issues (imminent or non-imminent) addressed during the review period of June 2011- June 2014. As there were no instances of confidential issues occurring during the period, the independent auditor did not perform testing of this section.

N/A

SPM Section 11.0 Process for Approving Supporting Documents

Refer to testing procedures performed in section 4 of the SPM. Section 11 was tested in conjunction with the previously noted sections. Although this was testing in conjunction with Section 4, the process improvements corresponding to section 4 are not applicable to the ‘Process for Approving Supporting Documents.’

General Conformance

SPM Section 12.0 Process for Correcting Errata

Per review of Standard Committee meeting minutes and evidence of the date the errata was filed with FERC, noted that errata are thoroughly inspected for material impact on end users prior to final approval.


SPM Section 13.0 Process for Conducting Periodic Reviews of Reliability Standards

Per review of the Reliability Standard Project Page, including review of the previous Reliability Standard periodic review, the independent auditor noted that periodic reviews of Reliability Standards are conducted in accordance with the SPM and that entities are provided a 45 day comment period for the review. Per review of corresponding meeting minutes (Standards Committee and BOT), noted that appropriate approvals for modifications are received prior to implementation. Further, through review of the Reliability Standards Development Plan (RSDP), noted that reviews are completed as planned, per the RSDP. Through review, noted that NERC devotes significant effort for periodic reviews of Reliability Standards.

Process Improvement #3 and #11



Review Area Review Area

Title Summary of Testing Procedures Performed

Result of Testing

Procedures

Performed

SPM Section 14.0 Public Access to Reliability Standards

Through review of the NERC website and inquiry with NERC staff (see Appendix A), noted that NERC is required to maintain an electronic copy of all currently proposed and active Reliability Standards and associated projects. This information is posted on the Reliability Standard’s Project Page of NERC’s public website and contains the formal comment period evidence and associated comments and ballots for each Reliability Standard for the previous five years.

General Conformance

SPM Section 15.0 Process for Updating Standard Processes

Per review of meeting minutes (Standards Committee and BOT), noted that appropriate approvals are received for modifications to Reliability Standards prior to implementation of the modification. Further, reviewed the Reliability Standard’s Project Page and noted that updates to the standard process are posted for a 45 day comment period, to appropriately allow entities to provide input. Through review, noted that NERC devotes significant effort in reviewing all modifications prior to implementation.

General Conformance

SPM Section 16.0 Waiver

Per review of the Reliability Standard Project Page for the waivers sampled, noted that evidence pursuant to the waiver request is posted to the Project Page. Additionally, reviewed communication between the Standards Committee and the individual submitting the waiver to determine whether appropriate timelines were followed as required by the SPM. Per review of meeting minutes, noted that appropriate approvals (Board of Trustees and Standards Oversight and Technology Committee) are received prior to the implementation of a wavier.




Section 300 of the ROP/SPM ‘Not Easily Testable’

Ref. # Review

Area Section Page Paragraph Statement

1 SPM 1.4 4 N/A The process shall be transparent to the public.

2 SPM 2.1 6 N/A The Requirements shall be material to reliability and measurable.

3 SPM 2.2 6 N/A

Each Reliability Standard shall enable or support one or more of the reliability principles, thereby ensuring that each Reliability Standard serves a purpose in support of reliability of the North American Bulk Power Systems. Each Reliability Standard shall also be consistent with all of the reliability principles, thereby ensuring that no Reliability Standard undermines reliability through an unintended consequence.

4 SPM 2.3 6 N/A Recognizing that Bulk Power System reliability and electricity markets are inseparable and mutually interdependent, all Reliability Standards shall be consistent with the market interface principles.

5 SPM 2.5 7 N/A Each Requirement shall be a statement for which compliance is mandatory.

6 SPM 2.5 8 N/A The only mandatory and enforceable components of a Reliability Standard are the: (1) applicability, (2) Requirements, and the (3) effective dates.

7 SPM 3.4 11 N/A [T]he Standards Committee shall not direct a drafting team to change the technical content of a draft Reliability Standard.

8 SPM 4.1 16 N/A When presented with a SAR, the Standards Committee shall determine if the SAR is sufficiently complete to guide Reliability Standard development and whether the SAR is consistent with this manual.

9 SPM 4.1 17 N/A

If the Standards Committee is presented with a SAR that proposes developing a new Reliability Standard or definition but does not have a technical justification upon which the Reliability Standard or definition can be developed, the Standards Committee shall direct the NERC Reliability Standards Staff to post the SAR for a 30-day comment period solely to collect stakeholder feedback on the scope of technical foundation, if any, needed to support the proposed project.




Ref. # Review Area Section Page Paragraph Statement

10 SPM 4.1 17 N/A

If a technical foundation is determined to be necessary, the Standards Committee shall solicit assistance from NERC’s technical committees or other industry experts to provide that foundation before authorizing development of the associated Reliability Standard or definition.

11 SPM 4.14 23 N/A Where there is a question as to whether a proposed modification is “substantive,” the Standards Committee shall make the final determination.

12 SPM 4.16 24 N/A The Board shall adopt or reject a Reliability Standard and its implementation plan, but shall not modify a proposed Reliability Standard.

13 SPM 4.2 17 N/A

If a SAR for a new Reliability Standard is posted for a formal comment period, the Standards Committee shall appoint a drafting team to work with the NERC Staff coordinator to give prompt consideration of the written views and objections of all participants.

14 SPM 4.6 20 N/A The drafting team shall consider the results of the quality review, decide upon appropriate changes, and recommend to the Standards Committee whether the documents are ready for formal posting and balloting.

15 SPM 4.9 21 N/A During a ballot window, NERC shall not sponsor or facilitate public discussion of the Reliability Standard action under ballot.

16 SPM 4.9 21 N/A

There is no requirement to conduct a new non-binding poll of the revised VRFs and VSLs if no changes were made to the associated standard, however if the requirements are modified and conforming changes are made to the associated VRFs and VSLs, another non-binding poll of the revised VRFs and VSLs shall be conducted.

17 SPM 7.0 31 N/A

If, during its deliberations, the Interpretation drafting team identifies a reliability gap in the Reliability Standard that is highlighted by the Interpretation request, the Interpretation drafting team shall notify the Standards Committee of its conclusion and may submit a SAR with the proposed modification to the Reliability Standard at the same time it provides its proposed Interpretation.





18 SPM 7.0 31 N/A

If an Interpretation drafting team proposes a modification to a Reliability Standard as part of its work in developing an Interpretation, the Board of Trustees shall be notified of this proposal at the time the Interpretation is submitted for adoption.

19 SPM 7.0 31 N/A

The Interpretation shall stand until such time as the Interpretation can be incorporated into a future revision of the Reliability Standard or the Interpretation is retired due to a future modification of the applicable Requirement.

20 SPM 8.0 34 N/A

Any entity that has directly and materially affected interests and that has been or will be adversely affected by any procedural action or inaction related to the development, approval, revision, reaffirmation, retirement or withdrawal of a Reliability Standard, definition, Variance, associated implementation plan, or Interpretation shall have the right to appeal.

21 SPM 8.0 34 N/A The burden of proof to show adverse effect shall be on the appellant.

22 SPM 8.2 34 N/A In addition to the appellant, any entity that is directly and materially affected by the procedural action or inaction referenced in the complaint shall be heard by the panel.

23 SPM 10.4 39 N/A The NERC Reliability Standards Staff shall not post or provide the ballot pool with any confidential background information.

24 SPM 11.0 43 N/A Any Requirements that are mandatory shall be incorporated into the Reliability Standard in the Reliability Standard development process.

25 ROP - Section 300 302 3 2.10 Bulk Power Systems shall be planned and operated in a coordinated

manner to perform reliably under normal and abnormal conditions.

26 ROP - Section 300 302 3 2.2

The frequency and voltage of Bulk Power Systems shall be controlled within defined limits through the balancing of real and reactive power supply and demand.





27 ROP - Section 300 302 4 2.3

Information necessary for the planning and operation of reliable Bulk Power Systems shall be made available to those entities responsible for planning and operating Bulk Power Systems.

28 ROP - Section 300 302 4 2.4 Plans for emergency operation and system restoration of Bulk Power

Systems shall be developed, coordinated, maintained, and implemented.

29 ROP - Section 300 302 4 2.5 Facilities for communication, monitoring, and control shall be provided,

used, and maintained for the reliability of Bulk Power Systems.

30 ROP - Section 300 302 4 2.6

Personnel responsible for planning and operating Bulk Power Systems shall be trained and qualified, and shall have the responsibility and authority to implement actions.

31 ROP - Section 300 302 4 2.7 The reliability of the Bulk Power Systems shall be assessed, monitored, and

maintained on a Wide-Area basis.

32 ROP - Section 300 302 4 2.8 Bulk Power Systems shall be protected from malicious physical or cyber

attacks.

33 ROP - Section 300 302 5 8

Each Reliability Standard shall be stated using clear and unambiguous language. Responsible entities, using reasonable judgment and in keeping with good utility practices, are able to arrive at a consistent interpretation of the required performance.

34 ROP - Section 300 303 5 1 Competition — A Reliability Standard shall not give any market participant

an unfair competitive advantage.

35 ROP - Section 300 303 5 2 A Reliability Standard shall neither mandate nor prohibit any specific market

structure.

36 ROP - Section 300 303 5 3 A Reliability Standard shall not preclude market solutions to achieving

compliance with that Reliability Standard.




37 ROP - Section 300 303 5 4 A Reliability Standard shall not require the public disclosure of commercially

sensitive information or other Confidential Information.

38 ROP - Section 300 303 5 4

All market participants shall have equal opportunity to access commercially non-sensitive information that is required for compliance with Reliability Standards.

39 ROP - Section 300 303 5 5 NERC shall not set Reliability Standards defining an adequate amount of, or

requiring expansion of, Bulk Power System resources or delivery capability.

40 ROP - Section 300 303 5 N/A

To ensure Reliability Standards are developed with due consideration of impacts on competition, to ensure Reliability Standards are not unduly discriminatory or preferential, and recognizing that reliability is an essential requirement of a robust North American economy, each Reliability Standard shall meet all of these market-related objectives:

41 ROP - Section 300 306 8 N/A The Standards Committee shall not under any circumstance change the

substance of a draft or approved Reliability Standard.

42 ROP - Section 300 314 16 N/A

If a Bulk Power System owner, operator, or user determines that a NERC or Regional Reliability Standard may conflict with a function, rule, order, tariff, rate schedule, legislative requirement or agreement that has been accepted, approved, or ordered by a governmental authority affecting that entity, the entity shall expeditiously notify the governmental authority, NERC, and the relevant Regional Entity of the conflict.


RELIABILITY | ACCOUNTABILITY42

Appendix D: CCC Audit Observers

CCC Audit Observer Entity

Terry Bilke Midcontinent Independent System Operator, Inc (MISO)

Robert Hoopes PPL Services Corporation

Charles Manning Electric Reliability Council of Texas (ERCOT)

Patti Metro National Rural Electric Cooperative Association (NRECA)

William Temple Northeast Utilities (NU)

Independent Evaluation of NERC’s Independent... · To evaluate the ROP and SPM processes in place...

Documents

Transcript of Independent Evaluation of NERC’s Independent... · To evaluate the ROP and SPM processes in place...