Developing instruction includes production of draft and final materials, pilot testing, evaluation of pilot testing results, and finalization of materials based on pilot test results. The most effective development effort takes into account the appropriateness and cogence of content and the method of presentation. Selecting and matching the right development tools determines the quality, and hence the building of credibility and acceptance of the final product.

A. Researching Content

In addition to training needs analysis, content research may extend to evaluating the applicability of skills matrices published by the Department of Defense. To avoid rewriting established and published instruction on the same subject, identify sources of information for specific instruction and research papers on the topic that you are working on. In the event such documents exist, evaluate them against design criteria identified in Section 3.0.

B. Selecting Presentation Methodology Based on Content

The primary source of information for selecting presentation methodology is the design portion of the plan for training. Selection of the methodology is based on the design parameters presented in Section 3. But of all those parameters, the most important driver is value if constraints in the project can be met.

For example, what presentation methodology is best in this situation? If a lot of people need to be trained quickly, the quickest and easiest method of developing slides might be selected. However, if there is value in developing a time intensive methodology involving graphics, video capture, etc. and the adequate development resources are available and the schedule allows development under these constraints, then development should go ahead.

Value is often more subjective than objective when it comes to evaluating input from the training customer. This is why collaboration with the customer throughout the training project is important in ensuring that the solution remains driven by the design specifications and not some political or esoteric reason. The customer may increase the risk by trying to develop a solution that does not match the time, resources, risk, or schedule constraints.

C. Development Tools

In a computer-based instruction development environment, the type of presentation can range from handouts to elaborate flash-oriented, video embedded, eLearning instruction modules. The electronics as well as the software can vary in cost and in use as development tools. Development tools should be selected based on all of the design specification criteria cited in Section 3.

When delivering instruction to the intended audience, development tools should be selected based on their ability to produce products that can be used by the intended audience. If the design solution is intended to be a software application that cannot be used in a configuration environment only available to the audience, the audience will not be able to take the training. Configuration software and hardware specifications are very important to the training materials designer.

1. Conducting Pilots - During materials development, potential students/users of the training should be identified. A plan for conducting the pilot training session(s) should be constructed to identify the resources needed, provide details about how the pilot will be conducted, how the effectiveness of the training program will be determined, and how the pilot training results (cognitive and psychomotor/ performance tests, surveys, and exercises) will be evaluated and used to update or change the materials.

2. Reviewing Pilot Results - the number of training participants should include at least 30 individuals to obtain a statistically significant population. Observations should include a qualitative assessment of ease of use of training materials and effectiveness from both objective (comparison of pre-test and post-test responses) and subjective (survey response analysis). At a minimum, measurements should include correlation of test item answers, t-tests between scores on the same items for comparison with training participants who may have taken the same test, and Chi-Square tests to test significance of variance between and among answers.

In addition, training requirements should be tracked to verify that the materials provide training on them. If the course is administered by a training specialist in an instructor led classroom environment, an objective evaluation of subject matter, presentation skills, and classroom management needs to be performed.

3. Making Changes to Courseware - If modifications require a 30% re-write as indicated by the

pilot results, the materials should be pilot tested again.