Skip to content

6 ways to Innovate: How to Generate Great Ideas

September 13, 2008
tags: , ,

Evaluating and Selecting Ideas

The generation of ideas is a necessary, but not sufficient, condition of innovation. Creativity involves more than producing ideas for resolving a challenge. The ideas somehow must be reduced in number, assessed, and further narrowed down so that one or more potential solutions can be implemented. This applies to individuals as well as to groups and organizations. When innovation challenges are involved, there typically is a group of judges (for competitive idea campaigns) or at least a group of stakeholder decision makers. Therefore, this discussion will assume that at least one group will be involved in evaluating and selecting ideas.

In this chapter, a conceptual distinction will be made between the processes of evaluation and selection. Then some basic guidelines will be presented for managing these processes. Finally, several evaluation and selection techniques will be described and discussed briefly.


The process of idea evaluation precedes that of idea selection. Before you can make a choice, you first must develop some basis for the choice. Then, once you have applied this basis, the selection process will fall into place, with very little difficulty involved in how final choices are made—at least, this is what should happen.

What often happens is that ideas are selected with little consideration given to the basis for the choice. People often select ideas without any conscious awareness of what criteria they used to guide their selections. Although such intuitive decision making can work out quite well for many types of challenges, it can be a severe liability when high-quality solutions are sought—that is, solutions with the greatest probability of resolving a problem. To increase the odds of choosing high-quality solutions, everyone involved in the decision-making process should be aware of the specific criteria being used.

In most group decision situations (as well as in individual situations), criteria used to guide idea selection will be implicit, explicit, or some combination of the two. The use of implicit criteria occurs when ideas are assessed for their potential to resolve a challenge without formally acknowledging the specific criteria used to assess each idea. In groups, this means each member individually applies his or her own criteria without sharing criteria with other members. Frequently, this lack of sharing occurs when members have not agreed on a need to share criteria or when individual members are not aware of the criteria they are using. When explicit criteria are used, the group members must formally acknowledge and agree on the standards they will use to evaluate each idea. It also is possible that criteria may be both implicit and explicit. For example, a group may agree to use certain criteria, but individual members may consciously or unconsciously apply their own criteria in conjunction with the agreed-upon criteria.

Because creative approaches typically are used when unique, high-quality solutions are desired, the use of explicit criteria is an absolute necessity. However, there will be instances in which criteria cannot be fully explored and developed. For example, if time is a critical factor, voting procedures—which use no formal, explicit criteria—may be required. This consideration and others will be discussed further in the next two sections. Nevertheless, explicit criteria should be used whenever possible.


As with most of the other problem-solving modules, idea evaluation selection proceeds more smoothly when basic guidelines are available to help structure the processes involved. These guidelines are especially important when high-quality solutions are needed. If careful attention is not given to idea evaluation and selection, the odds are diminished for transforming any ideas into workable problem solutions. The best ideas in the world will be of little value if they are not first screened and evaluated, so that the cream of the crop can be selected for possible implementation.

The use of guidelines also involves another advantage. By helping structure a process that often involves little structure, guidelines increase greatly the group members’ commitment to the ideas. Even though formal techniques may be used, the outcome is more likely to result in high member satisfaction with the process when a systematic process is followed.

Although not chiseled in stone, the guidelines that follow should help groups approach the evaluation and selection process with more confidence and enthusiasm. Many people view idea generation as the most interesting and stimulating aspect of the creative problem-solving process. And it may be that it is; however, idea evaluation and selection can at least be interesting if a group approaches its task in an orderly fashion, such as using the steps that follow.

1. Assess Participation Needs

There will be times when it may be better to evaluate and select ideas alone instead of involving others; in some situations, it may be better for one or more groups to participate; and in other situations, both individual and group decision making may be better. In order of importance, the major variables that need to be assessed when making such decisions are:

1. Amount of time available

2. Importance of selecting a high-quality solution

3. Need for group members to accept the solution selected

4. Need for group members to experience the evaluation and selection process for their own personal and professional development

You should select ideas alone when there is little time available because the importance of selecting a high-quality solution is relatively low, and a group does not need to accept the solution or experience the process. Conversely, a group should participate when time is available, a high-quality solution is important, successful implementation of a solution depends on group member acceptance of the solution, and group members would benefit from experiencing the process.

However, many participation decisions cannot be made in an either/or manner. In some situations, it may be necessary for you to make some selection decisions alone and allow team members to make the remaining decisions. For example, if time is in relatively short supply, but not enough to justify excluding a group, you might select a final pool of ideas, turn these ideas over to the group, and have the group develop a final solution. Such a procedure would be especially useful when it is important for the members to accept a solution.

2. Agree on a Procedure to Use

As long as others are involved, you need to agree on a procedure for evaluation and selection. You might decide to follow the guidelines presented here, use some other procedure, or go directly to the evaluation and selection techniques. If you are working with others and they decide to ignore the guidelines described here, you still should stress the need to develop explicit evaluation criteria. You also need to decide how to narrow down the pool of generated ideas to a more manageable number. The important concern when others are involved is to agree on some procedure and avoid the temptation to proceed haphazardly.

3. Preselect Ideas

If you have a fairly large number of ideas to evaluate—for example, fifty or more—you need to reduce them to a more workable number. Some of the techniques described in the next post can be used for this purpose, or other methods may be available to you.

Whatever method is used, two major variables to consider are the amount of time available and the importance of the problem. If there is plenty of time and the problem is important—that is, the consequences will be serious if not resolved—you should devote considerable effort to this activity, assuming you are motivated to do so. If there is little time, the problem is not very important, and motivation is low, some shortcuts are needed.

An initial step you can take to narrow down an idea is to examine all the ideas with the goal of developing combinations, which are commonly referred to as affinity groups. When similar ideas are joined together, the overall number is reduced. You also might consider combining dissimilar ideas. Such combinations can often lead to higher-quality solutions than can be obtained from implementing two individual ideas separately.

If a group has developed all possible combinations, you should check for understanding on the part of all members. Decide whether all group members understand the logic, meaning, and purpose of the ideas. Then poll the group members to see whether they are comfortable with the list of ideas. (Examination of an idea pool will often stimulate new ideas that might be added to the list or combined with others. If this occurs, add the new ideas to the pool.)

After you select a final list, you need to decide how to reduce the number of ideas further. If time is inadequate for a thorough evaluation of each idea, two actions can be taken, either alone or in sequence. The group can organize all the ideas into logical categories according to some criterion such as problem type or potential effect. Then it can eliminate those categories of ideas that appear to have the potential as high-quality solutions. A second action is to give group members a certain number of votes. For example, if there are one hundred ideas, members could be given ten votes (10 percent of the idea pool). This is especially useful in brainstorming retreats where ideas need to be narrowed down rather quickly. An even better procedure might be to use idea categories and votes together. This can be done by voting on categories or by voting on ideas within categories. Of course, if you are using idea management software, voting procedures are likely to be built in.

These procedures can also be used if time is not a major concern. However, if the problem is very important, the group can use a slightly different method of reducing the ideas, either by itself or in combination with categories and voting. This procedure involves establishing a minimally acceptable criterion and then eliminating every idea that fails to satisfy this criterion. For example, the group might decide to eliminate every idea that would cost more than a certain sum of money to implement. (If time permits, more than one criterion could be used.)

A decision to use this approach, however, must be made with some caution. If an inappropriate or relatively trivial criterion is selected, many potentially valuable ideas could be eliminated prematurely. Moreover, what appears to be an obstacle to implementing an idea often can be overcome with a little additional problem solving. For instance, it may appear that an idea will cost too much to implement, but minor modifications may make it more workable.

Regardless of which approach a group uses to narrow down the number of ideas, it must come up with a final list of ideas that are ready for evaluation and selection. The group can consider this list tentative, depending on the particular technique or techniques used for final selection. It may be, for example, that the group finds itself with too many ideas to screen, even after preselecting ideas from the initial pool. Or, the amount of available time may decrease suddenly—a frequent occurrence in most organizations. In such cases, the group could recycle the list of ideas through the preselection phase and produce a shorter list.

Share with the group the final list to be used for final evaluation and selection. Ask group members to provide feedback concerning clarity and understanding of the meaning and logic underlying all the ideas. When a group understands and accepts the list, it can start the next step in the process.

4. Develop and Select Evaluation Criteria

Although many of the evaluation and selection methods described in this chapter use explicit evaluation criteria, there are some that do not. As discussed earlier, the quality of ideas selected without explicit criteria may be lower than that of those selected with such criteria. As a result, a group should attempt to make explicit all the criteria it will be using, whether or not a technique requires such explicitness. For example, a voting procedure relying on implicit criteria can be used to greater advantage if the group takes some time to discuss and agree on criteria that might be used to assign votes.

In developing criteria, a group should try to generate as many criteria as possible. As with idea generation, quantity is likely to result in quality. The more criteria a group thinks of, the greater will be the odds that the group will select a high-quality solution. And just as it is important to defer judgment when generating ideas, it is important to defer judgment when generating criteria. Encourage group members to stretch their minds and think of as many criteria as possible without regard to their initial value or relevance.

After a group has generated a list of criteria, it must select the criteria to use for evaluating and selecting ideas. The actual number of criteria selected will depend on the same variables used in preselecting ideas: available time and importance of the problem. When time is available and the problem is important, the group should select as many criteria as possible to use in evaluating ideas. When little time is available and the problem is not perceived as being important, the group can select a minimal number of criteria. However, if only a few criteria are used, it is imperative that the group choose relevant, important criteria that will be most likely to yield a high-quality solution.

5. Choose Techniques

The choice of evaluation and selection techniques can be as easy or as difficult and complex as you want. As with choosing idea generation techniques, many variables can be considered when choosing evaluation and selection techniques. However, in this instance, the choice really narrows down to one among voting procedures, evaluation procedures, and some combination of each. Beyond these considerations, the only other real issue is the amount of time available. Voting obviously will consume less time than group evaluation, even if the group agrees on voting criteria prior to actual voting.

The techniques in this chapter use “pure” voting methods, a criteria-based evaluation procedure that results in the selection of an idea without the need for voting, or “pure” evaluation procedures with no built-in voting mechanism or explicit criteria. There are four major criteria that can be used when selecting from among these techniques:

1. Ability of a technique to screen a large number of ideas

2. Provision for use of explicit evaluation criteria

3. Use of weightings for each of these criteria

4. Relative time requirements for using each technique

6. Evaluate and Select Ideas

At this point in the process, a group should apply one or more techniques to help it evaluate and select the ideas. It should select one or more ideas for possible implementation based on the number of ideas being evaluated, the amount of time available, and the perceived importance of the problem.

If a group selects more than one idea, it should examine possible ways to combine the ideas to produce a higher-quality solution. If the ideas cannot be combined easily (or logically) and the ideas are judged to be equal in quality, the group should consider implementing all the ideas, either in sequence or all at the same time. If the group decides there are too many ideas to implement, it can assign implementation priorities to the ideas and implement as many as possible in the time available. Or, it can screen the ideas again by using an evaluation and selection technique, but with more stringent criteria this time, in order to reduce the idea total. Regardless of what procedure the group chooses for dealing with the remaining ideas, it should constantly be aware of its ultimate objective of selecting a high-quality solution.

The group will, of course, determine the quality of the ideas before it selects any ideas for implementation. In judging idea quality, note that rating of the ideas, as used in many of the selection techniques, will result in a quality evaluation for each idea. Thus, quality will already have been considered at this point in the process. The group will need to evaluate again any ideas that survive these ratings, however, to ensure that it has selected the idea with the greatest probability of resolving the challenge. The group may have overlooked important criteria, or new information may have become available that could affect the previously determined quality ratings of the ideas.

Thus, the last step in the evaluation and selection process is to conduct a final check on ideas that survived the screening process. To do this, each group member could rate each idea (on a 7-point scale) with respect to its likelihood of resolving the problem. The group then could average the ratings or discuss them, with the purpose of achieving consensus. If the group judges none of these ideas to be of high quality, it could re-review ideas that initially failed to survive the screening process but were rated high, and evaluate them with respect to their quality.

No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: