The International Society of Forensic Computer Examiners is responsible for managing and regulating the Certified Computer Examiner (CCE) ® certification process according to the following policy.

1.0 All mechanisms used by the ISFCE to evaluate the knowledge, skills and abilities of CCE candidates will be statistically based, reliable and non-discriminatory in nature.

2.0 An Evaluation Director will be assigned to oversee the process of evaluating and scoring CCE candidate exam results. The Evaluation Director will have suitable qualifications and knowledge to consistently score, monitor and analyze exam results and will also have at least 5 years experience in the field of digital forensics.

3.0 The Evaluation Director will appoint 2 secondary evaluation experts to assist with the evaluation and scoring of Certified Computer Examiner (CCE)® examination results.

4.0 No subcontractors will make the final certification decision, but may be used as an Evaluation Director. When a subcontractor is used, a contract will be executed between the subcontractor and the ISFCE. Such contracts will include statements of conflict avoidance between the subcontractor duties serving the CCE certification process and other work done by the subcontractor outside of the ISFCE.

5.0 Following are the mechanisms and methods that will be used to evaluate the abilities, skills and knowledge of the CCE candidates:

6.0 The CCE Online Exam

6.1 The CCE online exam shall consist of:

6.1.1 One multiple choice examination consisting of at least 75 randomly generated questions from a pool of 250% (per Haladyna, 2004).

6.2 The online test shall include up to 10 experimental questions which are included for validation of those questions prior to addition to the testing pool.

6.3 The 10 experimental questions will not be applied to the final online scoring grade for each candidate.

6.4 The online examination shall be completed without the use of books, notes, outside assistance, internet sites, or other materials and shall be completed in 45 minutes total.

6.5 Administration of the online test will be conducted under the supervision of either an ISFCE certified instructor or shall include a notarized statement from the candidate attesting to the student’s completing the examining in accordance with the procedure (see Exhibit A).

6.6 There shall be no disclosure of test questions or content to any online or classroom students by the instructors or other representative.

6.7 Questions for the online exam will be established and developed by the Evaluation Director and derived from the content of the CCE BootCamp.

6.8 Online test items shall be weighted as follows: Weights are based upon the following groups:

6.8.1 Modules - Module items are included based on weights determined by the ISFCE Certification Board. Initial weighting is based upon slide count for each module in the official instruction and weighted against the total number of slides in the course. These items shall total no more than 95% of the total items on the examination.

6.8.2 General Knowledge Items - These items are considered “common knowledge” and will consist of items that should be known by any person examining forensic materials. These items shall comprise no more than 5% of the total items on the examination.

6.8.3 Experimental Items - these items are considered “experimental” and shall be written using Haladyna’s (2004) recommendation of 3 distractors per question and shall be assessed with a minimum of 30 responses before being subsumed into the item base by weight. Items shall be developed in accordance with the weighting needs in I.B.1.a and I.B.1.b. See I.C for information on parameters for these items.

6.8.3.1 Experimental items which are to be included shall have both a module (or general knowledge) designation and a construct designation. Construct designations are major topic areas which are to be tested (e.g. Registry Keys). These designators are used for validation and psychometric analysis.

6.9 Online Testing Psychometrics (see Exhibit B for references) based upon literature of test validation and quality:

6.9.1 Items shall be assessed against the following criteria:

6.9.1.1 P-Value (difficulty) – Items with p-values above .9 and below .2 are deemed overly easy and difficult respectively. Items which fall above and below these values are deemed ineffective and shall be revised or removed.

6.9.1.2 Point-Biserial Correlation – This measurement correlates the outcome of the item with the overall test performance and is used to describe discrimination between students with high scores and low scores subsequently getting the question correct or incorrect. PBS values of the following are typical:

> .4 – very good items
>.3 and < .4 – good items
> .2 and < .3 – fair items
< .2 – poor items

6.9.1.3 Items which fall into categories 1 and 2 are considered effective for purposes of the CCE online exam and shall be retained. Items which fall below .3 in PBS will be revised or removed.

6.9.1.4 Cronbach’s Alpha – this is a measurement of the items inter-item correlation. Essentially this measures the amount of error in the test. As tests which measure as low as .6 alpha are used (particularly when the test is used in conjunction with other measures, viz. the practicals). The online test shall maintain Cronbach’s alpha scores of > .6.

6.10 Validation of New Items

6.10.1 New items shall be validated by assessing the p-value and the PBS of the item as well as an examination of the distracters.

6.10.2 P-value of new items shall be within the .2 - .9 range

6.10.3 PBS of the new item shall be >=.3

6.10.4 Distractor analysis shall be conducted to determine the distribution of distractors such that the distractors on the new item are evenly distributed (uniform distribution).

6.10.5 New items which do not meet these criteria shall be:

6.10.5.1 Revised and retested – if the item appears salvageable.

6.10.5.2 Removed and discarded – if the item appears seriously flawed.

6.10.6 New items which meet the criteria shall be included in the overall item pool.

6.11 Periodic Review

6.11.1 A periodic audit review of all items in the online test shall be conducted by Certification Board representatives. This review shall be conducted to evaluate the following:

6.11.1.1 Existing items in the test should meet all the criteria for new items.

6.11.1.2 Cronbach’s alpha for the test should be >=.6 for the test.

6.11.1.3 Bias by instructors should be investigated (good or bad).

6.11.1.4 Bias by online vs. bootcamp should be investigated.

6.11.2 As a result of review:

6.11.2.1 Items may be revised or removed if they fail to meet criteria.

6.12 Continuous Improvement

6.12.1 New Items will be added on an ongoing basis to the experimental items pool. These items shall be written and included for testing on a monthly basis.

6.12.2 New Items shall be assessed upon having a sample size of n>=30 using the psychometric criteria.

6.12.3 Items which pass the criteria shall be included in the overall question pool.

6.13 Item Ageing

6.13.1 Items shall be dated from the time of their inclusion into the item pool.

6.13.2 No item shall remain in the total item pool for more than 5 years.
6.14 Data Management

6.14.1 Data shall be maintained online and in an offline data repository in a reasonable secure fashion.

6.14.2 Data shall be backed up from the offline repository.

6.14.3 Data shall be synchronized with the online repository on a quarterly basis.

7.0 The CCE Practical Examinations

7.1 The CCE practical examinations shall consist of:

7.1.1 At least three actual forensic examinations of three different digital media.

7.2 Content for the practical exam problems will be established and reviewed in the following manner:

7.2.1 Practical exam questions will be based upon examination scenarios typically encountered in the digital forensic industry.

7.2.2 Practical exam questions will not deliberately try to trick or confuse the candidate, but will be challenging and straightforward in nature.

7.2.3 The ISFCE may submit sample practical examinations problems for approval and circulation in the certification process.

7.2.4 This information will be reviewed and analyzed by the Evaluation Director and a decision will be made to add or delete content to the practical exam questions accordingly.

7.2.5 All practical examination problems will be reviewed for content applicability and validity at least annually.

7.2.6 A new practical examination problem will be created at least annually, allowing for the Evaluation Director the option of retiring of at least one existing practical examination problem.

7.2.7 An ad hoc committee made up of at least 2 board members and one CCE with a minimum of one recertification (selected by board) will review and comment on the new practical test exam problem.

7.2.8 The CCE reviewing the new exam problem will receive 20 hours CPEs toward recertification in exchange for the review service.

7.2.9 This ad hoc review committee will provide the Evaluation Director with feedback including time necessary to complete exam problem, difficulty level and other general comments.

7.2.10 The board will review this feedback and ultimately approve or reject the practical exam question based on this feedback and any additional feedback by the Evaluation Director.

7.3 Practical Examination Problem Scoring

7.3.1 Score sheets for all practical examination questions will be created at the same time any new practical examination questions are created.

7.3.2 Score sheets will include a comprehensive checklist and point value assigned to each component expected to be identified by the certification candidate for that practical examination question.

7.3.3 Only the Evaluation Director and the secondary evaluation experts will have access to the score sheets.

7.3.4 Only a percentage score will be reported to the CCE candidate after completing the practical examination problem.

7.3.5 In order to help protect the integrity of the practical examination component of the CCE certification testing process, absolutely no feedback other than the certification candidate’s score will be reported to the CCE candidate or any party outside of the Evaluation Director and his/her staff.

7.3.6 After a practical examination problem is retired by the Evaluation Director, that examination problem will be posted and shared with the all certified members for review. This posting will include the score sheet components.

7.3.7 Practical examination problems will be monitored by the Evaluation Director for high rates of failure as correlated with the overall score on the certification.

7.3.8 Practical examination problems that have an excessively have high rate of failure correlated with success of the overall score on the certification will be removed immediately.

7.3.9 Any new practical examination problems will be reviewed by both secondary evaluation experts and the Evaluation Director prior to release for use in the certification process.

7.4 Periodic Review

7.4.1 A periodic audit review of the practical examination problems shall be conducted by Audit Committee. This review shall be conducted to evaluate the following:

7.4.1.1 Practical examination questions should meet all the criteria as defined in the Practical Examination Context guidelines.

7.4.1.2 Scoring bias identified during any appeal process should be investigated and addressed.

7.4.2 As a result of review:

7.4.2.1 Practical examination questions may be revised or removed from the testing process if they fail to meet defined criteria.



Exhibit A
CERTIFICATION

This certification is provided by the undersigned applicant (“Applicant”) to ISFCE, LLC (“ISFCE”) in connection with the Certified Computer Examiner (CCE)® certification (“CCE Certification”) administered by ISFCE.

1. Applicant represents, warrants and covenants to ISFCE that:

(a) All work necessary to properly achieve CCE Certification was completed by Applicant without assistance from any third party;

(b) Applicant did not collaborate, work jointly, cheat or use the work of a third party to complete the CCE Certification process;

(c) Applicant has not set up, accessed or belonged to any groups, workgroups, chatrooms, listservers, or other organizations that discuss the CCE problems, grades, scores, grading system or any related issues that may assist in the completion of the CCE Certification process;

(d) Applicant has not and will not claim to hold CCE Certification prior to the date, if at all, that the ISFCE Certifying Board grants CCE Certification to Applicant; and

(e) Applicant shall at all times comply with all requirements related to the issuance and maintenance of CCE Certification, and the ISFCE Code of Ethics and Personal Responsibility.

2. Applicant acknowledges that:

(a) The falsity of any statement contained in section 1 will result in the immediate removal of Applicant from the CCE Certification process, revocation of any CCE Certification received, and the forfeiture of any fee paid;

(b) The CCE Certification process is a test, and no feedback will be given by ISFCE to Applicant on Applicant’s examination reports; and

(c) There is no grandfathering under the CCE Certification process.

[Signature page follows.]


This certification is being executed by the Applicant on the date set forth below.

APPLICANT:

Signature

Print Name

Date


STATE OF ___________ )
COUNTY OF ___________ )

I hereby certify that on this day, before me, an officer duly authorized to administer oaths and take acknowledgments, personally appeared __________________, known to me (or proved to me on the basis of satisfactory evidence) to be the person described herein and who executed the forgoing instrument and acknowledged the execution of the within instrument for the purposes therein contained.

Sworn to and subscribed before me, this ____day of ___________, 20__.
______________________________
Notary Public

My Commission Expires: _____________________