Capturing Valuable Undocumented Knowledge

Research project develops guidelines for dealing with this important issue.

Important lessons were learned while developing and testing guidelines that a company might consider as it goes about implementing a new program—or expanding an existing one—to capture valuable undocumented knowledge from departing or other potentially unavailable workers. The guidelines consist of a process to follow and specific methods and tools to elicit, store, retrieve, and present valuable knowledge.

While the study (see accompanying section “Background of Research Project”) was done specifically in the energy industry, its results can be applied in any industrial setting.

Some of the lessons stemmed from development and testing of a process for capturing undocumented knowledge and for developing knowledge modules that contain the valuable knowledge for use by others.

Identify experts with valuable knowledge
An initial activity is to identify key employees who may be leaving their current jobs for whatever reasons, or may have knowledge so valuable that it should be available to others when they are absent due to travel, vacation, illness, etc. Methods to identify these key employees may range from simply asking managers to identify key employees with valuable undocumented knowledge to corporate-wide workforce surveys performed periodically.

Some of the factors to consider when identifying experts from whom knowledge may be elicited include:

• Individual should be recognized by his/her or other managers and peers as being the only expert about something of high importance, or one of only a few local site experts. Such an individual’s knowledge may be even more valuable if he/she is generally recognized as being one of only a few experts about something of importance within the entire company.

• Individuals with expertise in handling rare or infrequent events (e.g., repair of a unit that fails on average once every 10 years, or handling extensive repairs necessitated by a hurricane in areas not normally experiencing hurricanes) should be given serious consideration.

• Individuals with expertise for systems, etc., that are going to be replaced with different technology involving different skills should not be identified (e.g., “old” computer system being replaced about the same time the expert on that system retires).

Determine if experts are willing to provide knowledge
Following identification of the experts, it is important to determine if these workers are willing to permit their valuable tacit knowledge to be elicited and made available to others.

Many workers are willing—and in some cases eager—to share their knowledge. There are a variety of reasons for this positive response. For example, a worker may view it as an honor to be recognized as an expert. Others may feel an obligation to share their valuable knowledge with others in the company because of the benefits received during their careers—or because “it is the right thing to do.” Others may participate because their manager has asked them and made time available. It is simply “part of the job.”

It has been found, however, that some workers are not willing to share their expertise for a variety of reasons, including:

• Knowledge is viewed as an individual’s “intellectual property,” and may be used by that person as a basis for consulting work or another job.

• Fear of layoff because of the perception that the unique knowledge provides job protection, and making it available to others may increase vulnerability.

• Alienation against the company for some real or imagined reason (e.g., a lower-than-expected salary increase or being passed over for promotion).

• Belief that he/she does not possess any valuable knowledge, even though the person has been selected as an expert.

• Expectation that elicited knowledge will go “into a file cabinet and never be seen again,” thus wasting the time of the expert (may be based on previous experience at the company).

• Current work assignments leave no time available to participate in knowledge elicitation.

• Fear of loss of status because he/she no longer will be recognized as the expert in the organization.

Use existing resources to the extent possible
Most utilities already have programs to capture and disseminate expert-worker knowledge. For example, most companies have training groups and programs, procedure groups, human resources organizations, etc., that routinely identify, collect, and disseminate important information. In addition, some companies have effective mentoring, apprentice, job rotation, and cross-training programs.

To the extent feasible, existing resources and infrastructure should be used to collect and disseminate valuable undocumented knowledge from experts. Thus, time and costs to initiate a new program may be minimized. In fact, in many organizations, a very important step will be to assign an existing department, group, or individual with the responsibility for any expanded undocumented knowledge capture efforts.

Develop plan for knowledge capture and presentation
A plan should be prepared to elicit, store, and retrieve valuable undocumented knowledge from each individual selected. The plan should identify the specific knowledge elicitation method(s) selected for each expert or group of experts with similar skills, define the methods for storage, and describe how the stored knowledge will be retrieved.

Development of this plan will require consideration of a number of factors, such as type(s) of knowledge, availability of the departing expert, and capabilities and resources of the personnel responsible for knowledge elicitation.

Prepare knowledge modules and keep current
When this plan is implemented, the elicited knowledge should be formatted and packaged in a knowledge module. A knowledge module is explicit knowledge related to a specific task, activity, job, etc., that is retrievable when needed after having been elicited from an expert; evaluated, edited, and formatted to be in a form usable by others; and stored in electronic and/or hard-copy form.

There are at least two issues to consider when preparing knowledge modules. One issue relates to the use of the expert knowledge: Is it going to be incorporated with other material used by those receiving the information, or is it going to be used in stand-alone fashion? For example, the expert knowledge could be incorporated into a training class together with other training material. Alternatively, the expert knowledge could be linked to a step in a procedure, automatically appearing when it is time to perform that step. An example of stand-alone use involves a person in the field who inserts a CD-ROM in a laptop computer to receive guidance on how to perform a task, either just before or during task performance.

A second issue relates to the characteristics of the person using the knowledge module. If that person is not expected to be familiar with some of the technical terminology used or with the location of parts or tools discussed by the expert, then additional information may be required.

The knowledge modules must be stored appropriately and in accessible locations. Their existence must somehow be made obvious to potential users at the critical time that the knowledge should be accessed, and they must be presented in a timely fashion when needed.

It is essential that the knowledge modules be updated and corrected, as appropriate. Changes will occur in equipment, processes, procedures, practices, regulations, responsibilities, etc., over time. For a knowledge module to be useful over an extended period, it must be updated as needed. Also, with use, some of the knowledge may be found to be incorrect. It is essential that the errors be eliminated and correct information provided. Knowledge modules that no longer have value should be eliminated.

Other valuable lessons were learned during testing of knowledge elicitation methods at four sites.

Knowledge elicitor should be familiar with the domain
The person(s) responsible for eliciting the expert knowledge should be somewhat familiar with the domain about which the knowledge is to be elicited. The elicitor(s) may have the required familiarity through previous experience, or he/she may be given time to be bootstrapped into the domain prior to knowledge elicitation.

There are several reasons why such domain knowledge is necessary. Most importantly, it permits the knowledge elicitor to understand specialized domain terminology, be able to ask intelligent questions, and have some recognition of the specific areas to probe further to obtain the valuable undocumented knowledge.

Knowledge elicitors need guidance

Most elicitors should be provided with some guidance regarding the valuable domain knowledge to be captured. This may not be necessary if the elicitor is extremely familiar with the domain and the valuable knowledge that needs to be captured. Without this depth of knowledge, however, he/she may need to rely on someone else for direction regarding what needs capturing.

In many cases, the expert has a wide range of expertise, some of which is unique, and some that is also known by others. The expert is likely to be most familiar with the areas of knowledge that are of greatest importance and should be captured for transfer to others. It should be noted, however, some workers who are identified by their managers as experts may say, “I don’t have any valuable knowledge; other people know what I know.” It is not uncommon for an expert not to realize he/she has valuable knowledge not known by anyone else. Some experts assume that others have the same knowledge, even though that may not be the case.

Experts are usually extremely busy because they are the ones assigned the most demanding and difficult tasks, and may be consulted by others needing access to their unique knowledge. It may be difficult for the knowledge elicitor to have much time with the expert. Therefore, that time must be used wisely to capture the knowledge that is most valuable and not available to others.

In any event, the expert’s manager or other people most familiar with the situation should be queried regarding the specific knowledge to elicit. Such people will have an understanding of the knowledge areas that are important. They will be able to identify the most valuable and needed information that should be collected and subsequently made available to others.

Knowledge elicitation usually takes place in stages
Knowledge elicitation efforts usually take place in stages, and the nature of the knowledge is a major consideration in selecting the appropriate elicitation methods. The first stage is for the elicitor to develop an understanding of the general knowledge of importance available to the expert. Methods are available to develop a high-level description or overview of the expert’s valuable knowledge, e.g., the concept mapping method. The description created by applying the method can be reviewed with the expert and his/her manager to select areas to drill down to the levels at which the most valuable undocumented knowledge is held.

Following selection of the specific areas of importance, the elicitor may drill down to a deeper level of expertise applying the same methods used to create the high-level overview of the expert’s knowledge. Alternatively, another method may be selected that is more appropriate for the nature of the knowledge. For example, if the knowledge is based in large part on significant events occurring in the past, then an interview approach using the critical incident method or critical decision method may be appropriate.

Other approaches may be more suited to knowledge that relates to operations and maintenance processes and equipment. Such knowledge may be elicited with the help of simulations and scenarios using mock-ups or actual equipment. The simulations and constructed scenarios method and the think-aloud problem-solving method involve encouraging the expert to describe what he/she is doing and also thinking about as he/she performs the simulated or actual tasks. Video or audio recordings and photographs may be taken at appropriate times during the elicitation sessions, and then after editing and indexing be made available to others when access to the expertise would prove beneficial.

It may be desirable to drill down to even a more detailed level of knowledge at certain points during the elicitation process. For example, the expert may report that he/she senses almost unconsciously that something is in alignment, and that one “thing” can be inserted into another. If this capability to perform the action more quickly and better than anyone else has high value, then an unstructured interview approach might be applied to ferret out the important cues that are present. The elicitor may ask about visual, auditory, and tactual cues that are being used, possibly at almost an unconscious level.

Knowledge storage, presentation, and use must receive attention
Previous researchers working in the field of expert systems and knowledge management have observed the existence of a knowledge acquisition bottleneck. The knowledge elicitation methods described above, applied appropriately in the context and situation, can alleviate serious knowledge acquisition bottleneck problems.

Despite such reduction of knowledge acquisition bottlenecks, however, care must be exercised to facilitate the subsequent steps of knowledge storage, presentation, and use. The very methods that can, under certain circumstances, alleviate bottlenecks in knowledge acquisition can create time and effort barriers for subsequent stages of the process. For example, methods such as structured and unstructured interviews that rely on audio recording of elicitation sessions can create a transcription bottleneck. Transcription, editing, and reviewing audio records of interview sessions are time-consuming activities. Techniques to minimize the editing required to format knowledge for use by others include careful and selective audio recording and, for certain kinds of knowledge capture, use of video recording.

Computer speech recognition might be considered as an approach for avoiding the transcription bottleneck. At this time, the technology is not yet advanced enough to make this approach feasible. Both the elicitor and expert would need to train the speech recognition system in their respective voice patterns, and technical terms not in the speech recognition lexicon would need to be entered prior to the elicitation session. Speech recognition technology is moving ahead rapidly, and it may help reduce the transcription bottleneck problem in the near future.

No right or wrong knowledge elicitation method
The process of capturing valuable undocumented knowledge hinges on the development of an effective plan. It is important to determine whether potentially valuable undocumented knowledge will be lost with unavailability of experienced personnel; evaluate whether this knowledge is worth capturing; select appropriate method(s) to use in eliciting knowledge; and store, retrieve, and present this knowledge when needed.

The importance of each of these steps does not, however, imply that there is a “right” or “wrong” knowledge elicitation method or set of methods. The choice depends on a range of considerations, some of which may not come into play until knowledge elicitation is under way. For example, the knowledge elicitor may find that an elicitation method not considered or selected during planning may be more appropriate for the type of knowledge used by the expert. In such instances, it may prove desirable to revise the plan as the knowledge elicitation moves forward. Thus, understanding and access to a range of methods, and the flexibility to alter methods being used or planned, will result in greater benefit from the knowledge capture endeavor.

This article is based on two papers presented at the IEEE 7th Conference on Human Factors and Power Plants, September 15-19, 2002, in Scottsdale, AZ. MT

Lewis F. Hanes is employed part-time by EPRI as a project manager. Since his retirement from the Westinghouse Electric Co. Science and Technology Center, he has worked as a consultant to several organizations, and for three years was a full-time EPRI employee managing the nuclear human performance program.

BACKGROUND OF RESEARCH PROJECT

The EPRI 1999-2001 Strategic Human Performance Program included a multi-faceted research project, “Capturing Undocumented Worker-Job-Knowledge,” to assess the problems related to this potential loss of tacit knowledge, to determine and assess possible approaches to deal with them, and to develop practical guidelines for use in energy industry settings.

The overall project objectives were to deliver practical guidance for identifying employees who possess valuable undocumented knowledge; evaluating whether the knowledge is worth capturing; eliciting and storing the valuable knowledge; and retrieving and presenting this knowledge to other personnel when needed.

During an industry telephone survey as part of the research project, 92 percent of the respondents reported that loss of unique valuable expertise would pose a problem within the next 5 years, but only 30 percent of the respondents indicated that a planning effort was in place to address this problem of retaining knowledge from experienced workers in a manner that would make it accessible and usable by new or replacement members of the utility workforce.

Developing and testing the guidelines, which consist of both a process to follow and knowledge elicitation methods, occurred over the three-year period. The process and methods were implemented and tested at four utility sites with 20 workers/teams representing a range of organizations and work types. The process and methods were refined based on test results, and the final guidance report developed.

The guidelines were expanded and refined in 2002 for application to nuclear power generating sites. Detailed process flow charts were created that provide guidance on how to (1) identify experts from whom valuable knowledge would be captured, (2) develop a plan to capture the expertise and make it available when needed, and (3) implement the plan to develop knowledge modules and make them available when needed. Methods and techniques were identified and described to support accomplishment of the steps in the process flow charts.

EPRI, the Electric Power Research Institute, was established in 1973 as a nonprofit center for public interest energy and environmental research.

back to article