NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Research Council (US) Chemical Sciences Roundtable. Reducing the Time from Basic Research to Innovation in the Chemical Sciences: A Workshop Report to the Chemical Sciences Roundtable. Washington (DC): National Academies Press (US); 2003.

Cover of Reducing the Time from Basic Research to Innovation in the Chemical Sciences

Reducing the Time from Basic Research to Innovation in the Chemical Sciences: A Workshop Report to the Chemical Sciences Roundtable.

Show details

2Techniques for Structured Innovation

Allen Clamen1

ExxonMobil (retired)

The highly esteemed management professor Peter Drucker once said, “Success is more likely to result from the systematic pursuit of opportunities than from a flash of genius.” I will discuss the systematic pursuit of innovation used at ExxonMobil Chemical Company to increase the yield from basic science to commercialization. Although innovation is thought of as an inherently fuzzy process, my role in the last 5 to 10 years of my 35-year career has been to add a fair amount of structure and discipline to the process of innovation. Some of our best practices will be shared.

The innovation process can be placed into a business context by separating its distinct parts, as shown in Figure 2.1, beginning with the continual flow of ideas from a variety of sources such as customers, academia, and partnerships. Once an idea is selected to continue as a project, it enters our Product Innovation Process or Capital Investment Management Process, which is a stage-gate process that begins with data collection. The project then progresses stage-wise through the complete development of the idea into a commercial product or process. The Portfolio Management Process then analyzes these data so that decisions to progress, accelerate, decelerate, suspend, or terminate can ultimately be made. Of course, these decisions must be made with a clear understanding of the business strategy on which all relevant ideas are based.

FIGURE 2.1. The innovation process can be encouraged and controlled by breaking it into distinct parts.


The innovation process can be encouraged and controlled by breaking it into distinct parts.

The New Concept Development Model (see Figure 2.2), described in The PDMA ToolBook for New Product Development2 which was written by me and several others from different companies, provides a common language and terminology necessary to optimize the front end of innovation. This model was developed under the auspices of the Industrial Research Institute with the hope that we can begin to share best practices in a more understandable way and thus begin to improve upon current state-of-the-art innovation practices. The center portion of this model, the engine, represents the leadership, business strategy, and other factors within the company's control. This engine drives the five core front-end elements where activities such as idea generation and opportunity identification take place. The circular shape and curved arrows between these elements indicate that interaction and recycling continuously occur among all five elements. Moreover, these elements are influenced by a number of factors that are largely uncontrollable: the business climate, the economic climate, the customers, and the competitors. From a process standpoint, one can enter the model at any element and ultimately obtain a more detailed understanding of the idea, which is then termed a “concept.” At that point, the concept enters the standard stage-gate process for new product or process development.

FIGURE 2.2. The New Concept Development Model provides a common language and terminology necessary to optimize the front end of innovation.


The New Concept Development Model provides a common language and terminology necessary to optimize the front end of innovation. This figure appears in The PDMA ToolBook for New Product Development and is used with the permission of the author.

As mentioned previously, an idea is analyzed, screened, and deemed worthy of further development before it enters our stage-gate process. Figure 2.3 shows the entire outline of the stage-gate process starting from the specific idea (or concept). The first screening is mild, so it does not stifle further idea generation. The preliminary assessment, or Stage 1, of these ideas consists of activities designed to answer a number of basic questions. These include whether the project idea fits with the company's business strategy, whether it can make a profit, whether the market is large enough to justify developing the idea, and whether the idea is technically feasible.

FIGURE 2.3. The stage-gate innovation process.


The stage-gate innovation process.

After completion of the activities in Stage 1, the second screen, or Gate 1, determines whether a more detailed assessment involving the customer should be undertaken in Stage 2 to gain a better understanding of the idea's value in the marketplace. The decision to develop or drop the idea is made at Gate 2. The bulk of the project costs are in Stage 3, which is the actual development process. Therefore, the Gate 2 decision involves a major commitment by the company. The decision to commercially launch a product is not made until the customer validates the need for the product and its uniqueness in the market. Once the launch decision is made at Gate 4, the production and marketing plans are finalized, and the product or process is transferred from the project team to the ongoing business. The innovation process is not complete, however, without a postlaunch review. This review takes place in two parts: one soon after the launch and the other about a year or two later. These reviews provide an important assessment of the process and can result in improvements to the process.

Our stage-gate and portfolio management processes are integrated in the stage-gate decision-making process. The first decision is made at each gate meeting by the team of gatekeepers, who are the managers responsible for the resources required to develop the idea. This first decision is based solely on the project's merit, whether or not it has met all the objectives set out at the beginning of that stage. If so, it is deemed ready for resourcing. If not, it is put on hold for further data, killed, or returned to our idea management process for further enrichment or to be combined with other similar ideas. Once an idea is ready for resourcing, the portfolio management team decides on its relative merit, or how it compares with all the other projects in review. In addition, the team considers the balance of projects being resourced (short- versus long-range, how many market segments are represented). This decision-making process must also be integrated with all related business processes such as stewardship and staff development.

By benchmarking some of the most successful companies with respect to innovation, we were able to gather a set of best practices. First, the culture and organizational structure can make or break the innovation process in any company. As a result, it is vital to demonstrate a strong corporate commitment to innovation. It is also critical to maintain a business strategy articulated in as specific terms as possible since not all ideas will survive the entire process and reach the marketplace. Along with this focus, long-term funding must be available to allow the researchers to explore all relevant ideas through the early stages of the innovation process.

Rapid innovation works best when dedicated teams are provided adequate time and responsibility not only to explore new ideas but also to pursue them to the next stage. A culture needs to be created that supports and encourages innovation and the entrepreneurial spirit. Risk taking should be encouraged, especially since errors will inevitably occur and represent an important learning opportunity.

In some companies a top executive is made responsible for growth and innovation. This makes the innovation process more binding so that ideas are retained and used. Also, the formal creation of an “idea manager” provides someone in the process to help coordinate and assist an idea through those initial formative stages. Project management should not be taken too lightly. Projects need to be run by actual project managers, not just the scientist who had the idea or the person who is available because his or her last project has just finished. Peer audits can check the quality of execution of projects. However, even with advances in information technology, it is still considered essential to have everyone involved in the innovation process be in the same place together. The innovation process should also be tracked by a limited number of simple measurable goals, such as the percent revenue from new products, number of patents, and innovation climate.

External participation in projects by academia, other companies, and especially the customer offers a method of validating ideas against current market conditions. This is a difficult part of the business. Also, having a business champion, whether he or she is in management or is a respected scientist who advocates the importance of the project to management, generally correlates with success.

There are two types of issues involved in reducing the cycle time of the innovation process: marketing issues and technical issues. These must be addressed differently. Marketing issues involve the identification of unmet needs by the customer, and technical issues are related to the capability to deliver. Techniques that reduce cycle time include the following:

  • Practice openness in partnering with the customer or supplier. The customer ultimately determines the speed of technology acceptance. Success is not the ability to deliver but the customer's ability to use a product in a way that fits the customer's needs. To do this, reliable, up-to-date customer, market, and competitive data must be obtained before the final design along with feedback from the customer. One technique for obtaining valuable feedback from the customer is rapid physical prototyping, which is the process of showing the prototype to the customer as early as possible.
  • Measure and communicate cycle time and factors contributing to cycle time reduction. The process of simply measuring cycle time will result in the cycle time being reduced. Instantaneously obtaining information such as project data and progress using web-enabled tools will also support and facilitate the process.
  • Formalize project management practices. Formalizing project management practices will reduce cycle time because it ensures strategic and operational alignment across the entire organization. Endorsement of the project by senior management puts emphasis on ensuring that resources are available. The use of cross-functional teams throughout the project is critical to guarantee that everyone is “onboard” when activities are being conducted simultaneously in technology, manufacturing, and marketing. In addition, lessons learned from postlaunch reviews must be applied to avoid making the same mistakes repeatedly. An example of this is overestimation of the size of the market or rate of market penetration.
  • Use portfolio analysis techniques to select and progress research and development projects. Research cannot be scheduled, but finances can be budgeted and milestones set. Development, on the other hand, should be done well and as rapidly as possible to maximize profit.
  • Share credit for research and development success with the business unit. Better participation and new ideas can occur by providing appropriate incentives and rewards. Methods and practices that motivate all members of the team will foster innovation.

A faster response to the customer's needs will directly impact customer satisfaction. Speed is important not only because it lowers cost and allows the product to be first to market but also because speed yields higher success rates due to the increased likelihood of hitting rapidly changing targets. Of course, less time spent on obtaining results from research and development means that less money is spent, a higher margin is gained, and a longer product life cycle ensues. Overall, reduced cycle time translates to better economics.

The following is a set of management practices that will encourage innovation in any organization. First, a set of clear, consistent, and aggressive goals is needed. Aggressive goals grab the organization's attention and challenge it. Using parallel approaches to pursue a primary objective in a certain area of opportunity will decrease the time spent reaching that objective. This allows the team to work around any problems, because one of the other approaches may have bypassed the problems and progressed beyond that stage. Each approach must be confirmed. This enables customers, competitors, and industry in general to view each approach as a major advance. Although the expense is high, the potential benefits make it worthwhile to move along all of these paths. Unexpected findings should be explored to identify new business opportunities. On the other hand, parallel approaches should not be taken with many different projects. A limited number of objectives and technologies should be selected depending on the size of the organization.

Managers should be prepared to accelerate the most promising projects. Once such a project is identified, as many resources as possible should be given to it. It is also important to keep technical people challenged to maintain the leadership position that the company has in the area of interest.

Communication is always critical to any project. Managers should ensure that their employees talk to each other, the scientists, and the customers. Specifically, the project's scientists should be communicating with the customer's scientists, since they speak the same language.

The following are some criteria found in the literature3 that generally match those used by ExxonMobil to prioritize long-range research. Of course, a long-range research plan must fit the company's strategy and the strength of the supporting science base relative to the industry. Can the company grow, maintain, and tap into the science that is evolving out of the proposed effort on which the company is about to embark? How wide is the impact of new technology? How many products across the company's slate will be impacted by this new science that is being developed? How robust is the product in the event that there are changes in the business environment? Will the project still succeed? The chances of success are greater if multiple approaches to achieve the goal exist.

Long-range planning also requires estimates of the project's business value. At the very early stages it is difficult to determine what the market share and volume of product sold will be. The company must determine probabilities of both technical and commercial success. These are calculated independently and then combined to finally estimate how successful the project will be.

Innovation is now recognized as being essential to business. While scientists and engineers in an open and sharing culture are the champions of innovation, the customers set the product targets and are the judges of success and failure. Understanding where market needs and technology ultimately intersect as well as involving customers in the early stages of the innovation process will significantly increase the odds of commercial success. In addition, the systematic pursuit of innovative opportunities via the structured processes described above will allow an organization to capture, maintain, and employ best practices to ensure long-term success in innovation.


Robert W.R. Humphreys, National Starch and Chemical Company: Allen, you talk about your innovation process, which is not too different from the one we use. When you're doing portfolio management and a lot of projects are coming through, you've got to make decisions. The quality of your portfolio management team is obviously a critical thing or there will be many mistakes that won't be discovered for a long time. How do you choose that team?

Allen Clamen: The quality of the data, the input that goes in, is determined by the effort of the portfolio planner before all the portfolio management team's meetings. These can take place once a month, once a quarter, or as often as is necessary to select projects. All of those data go into the portfolio tool through the database and are collected.

Regarding the selection of team members, it is important to remember that they have the authority to allocate the resources. They would be the managers within each of the functions for which resources are required. It would be the manufacturing manager, marketing manager, and a technology manager. These people would bring with them the resources they need to assess the significance of the data presented. The portfolio tool presents different views, but the portfolio itself is presented for the team to make the decision based on the data, which is hopefully of the right quality, insured by the work that was done up front.

Robert W.R. Humphreys: Is every one of those people market savvy and customer savvy?

Allen Clamen: No, not at all. In fact, there has to be a great deal of trust. The manufacturing manager knows only what he knows relative to what he has learned about the marketplace through his own activities. The marketing manager has an entirely different view, that of the industry at large, the marketplace at large, and many different customers. The manufacturing manager relies to a great extent on the savviness of the marketing manager and his knowledge of that process. Similarly with technology and manufacturing, each of the functions has that degree of knowledge that is not complete for each of the others.

Richard C. Alkire, University of Illinois: To make investments today, your industry and many others use sophisticated economic tools based on assessment of risk for leases of contracts, options, and the like where there's uncertainty in the future. Can those tools be used to deal with research?

Allen Clamen: Yes, the expected value of an idea can indeed be adjusted by its probability of success based on both technology and commercial risk. The technology risk associated with a particular idea can be estimated based on a number of variables that have been validated by past experience.

It is important to learn about the variables as quickly as possible in order to adjust your priorities early based on that full knowledge. One of the things we found to be helpful was a technique called “scenario planning,” which looks at many different ways that the world is going to change in the next 5 to 10 years or more. That allows you to say, “Well, if it's going to be a green world, how does that change the business climate and the environment for our products? What does that say we should begin to work on? Here's a project that doesn't take into account this green revolution. Why should we work on that if the environment is looking that way?” Scenario planning allows you to look at several different views of the future world and adjust priorities based on which scenario you believe will take place.

J. Stewart Witzeman, Eastman Chemical Company: I'm intrigued with your comments about portfolio management. How in particular does this portfolio management team function in terms of balancing long-term work versus incremental work, and how do you fit exploratory research and development into the portfolio? In other words, are the managers just managing what they have in front of them today or are they looking at what an ideal portfolio of projects should be for the enterprise?

Allen Clamen: Manufacturing and marketing managers tend to have projects that are a little bit shorter term than your technology manager. They report to a business manager, who is responsible for long-term prospects as well as the long-term success of his enterprise. The business manager must be cognizant that a company can't have 90 percent of its activity in the short term or it will, in the long term, be unsuccessful. That is viewed at each portfolio management team meeting.

We look at the percentage of projects in each time frame and require 30 percent of the projects to be long term as part of the business strategy. Thirty percent may not be enough to some companies, but it probably is enough in the case of a chemical company. It is important to recognize where you are relative to your overall portfolio. This allows you to continue to progress those projects that are longer range and have less support from the team's business people who look at the shorter term.

Michael Schrage, Massachusetts Institute of Technology: It has been my experience working for a fairly broad variety of rather large organizations that the drivers for the bulk of these issues tend to be two words that weren't mentioned at all: politics and the allocation of overhead. Could you respond as to the nature of political horse trading during your years in industry?

Allen Clamen: Tell me a little bit more about to what you're referring. The work that I have described was individual businesses looking at their portfolios from a business standpoint in order to ascertain how all the resources internally are being allocated. I'm not talking about, in this case, supporting a central corporate laboratory. That may be behind some of this.

Michael Schrage: Absolutely, but in both of those contexts the convergence of allocation of overhead is present. Given a portfolio and a particular perception, we want to charge a certain amount to overhead rather than another amount. If the portfolio is put in the incremental innovation category, we do a different overhead formulation than if it is pioneering research.

I've seen organizations blow themselves up over the political aspects of that debate. People from various parts of the organization emphasize things that are reflective of their background. I would like to see some sort of effort made to bridge a rational approach with what actually happens in organizations, since that ultimately is the theme that we're facing.

Allen Clamen: I may be naive, but I don't see that kind of infighting. Perhaps I had not seen it to the extent that you just described. I do know, however, that all costs are rolled in, so that the cost of resource development—an engineer, for example—might be significantly higher than his or her salary. I recognize that.

That's just recognition that resources cost money. We do have to house them since they do use a laboratory and supplies. That's all factored in to the overall budget. Within that budget framework, the projects are allocated regardless of the politics. Development resources cost this much, period.

Michael Schrage: And as far as you're concerned, how the overhead is allocated is a purely rational process.

Allen Clamen: For example, there is no allocation of marketing and administrative cost. We learned a long time ago that this is something we didn't want to get in to. There is a base cost of running the business that doesn't enter in to the cost of developing new products.

Kenneth A. Pickar, California Institute of Technology: One of the realities of managing portfolios is the immortal project that is almost impossible to kill. You think you've terminated it, and then you find out that it popped up in some other guise somewhere else in practice on the side. Sometimes these projects turn out to be great discoveries, but far more often they are really a terrible drain on resources. What type of special advice or ideas do you have for killing the immortal?

Allen Clamen: First, our gate decisions are taken at appropriate stages of activity or progress on a project to discern whether or not to continue. These stages are highly visible. Once the project is terminated, the people who are working on it will stop because the staff time and associated costs are dead.

Additionally, there are some companies we've benchmarked that actually celebrate a kill. It's hard to imagine. Everybody has worked on this. They spent the better part of their last year or two working on this. Now it's dead, and they're feeling like they've lost a best friend. Yet they recognize that the project is still in their knowledge base. There is now a jumping-off point into more promising things.

There are a number of ways to kill projects, but visibility is probably the best methodology.

Kenneth A. Pickar: The visibility is a function of how much money is spent. The projects can't pass the stage gate, but then they go back into the database. It is really an act of hibernation.

Allen Clamen: If you recall the New Concept Development Model, unused ideas return to an idea bank. There is an idea manager who is managing the bulk of ideas that are in there. If one of them does look like a recycled idea, that's fine.

Kenneth A. Pickar: They'll wait until you retire and then they'll propose it again.

Allen Clamen: Right.

Mary L. Mandich, Lucent Technologies: Do you think that portfolio management prevents that or that recycled ideas are just a natural human function?

Participant: I think portfolio management helps limit the amount of resources wasted on these kinds of issues.

Allen Clamen: That is part of the people issues.

Kimberly W. Thomas, Los Alamos National Laboratory: You said one of the keys to success is an environment that is very open to risk taking. How do you reward and encourage the risk taking for your individuals? I think you gave us just one idea: celebrating the death of a project.

Allen Clamen: ExxonMobil does not practice that, but I think it's a great idea. The other is that risk takers, if they've made a name for themselves, tend to be those who have had some success in the past. Recognize that one success takes a lot of failures. But this risk taker, whoever he or she is, has had a number of minisuccesses along the way. It's worth taking another chance with them. Motivation is very important. A company should not reward an employee simply because that person was part of a great innovation. An employee should be rewarded for good attempts. It is important to recognize good tries as well as successes along the way.

Additionally, companies need to say the organization encourages and supports risk taking. The industry has done the reverse for many years. We were averse to risk, which definitely discourages innovation. Just recognizing that a company must not be risk adverse to be successful is an important element.

David J. Soderberg, BP Chemicals: You mentioned a very coherent and concise set of tools that are used: portfolio management, fuzzy front end, and stage gate. Those work very well within a business unit context. They're very focused and allow you to allocate resources.

How do you get the cross-fertilization and the opportunities that are identified, in our terminology, between the different streams, namely upstream and downstream? In a broader context, how do you obtain cross-fertilization within our industry and outside our industry?

Allen Clamen: That's still a big challenge for us. As I mentioned, the portfolio management was all within the business context. There is a roll-up intended, which hasn't happened yet, among all businesses so that we can start to look across businesses. What does the whole profile look like? What does the company portfolio look like?

More importantly, we have a connection with our corporate research labs, which is a central organization where we try to share any innovation or technology that's promising or has been found to be useful in one business with the others. They become very good ombudsmen across businesses. We are fortunate to have that. In fact, it spans downstream and upstream because corporate has all those aspects.

Venkat Venkatasubramanian, Purdue University: The issues you raised today reminded me quite a bit about the portfolio management issues in the pharmaceutical industry. We work closely with Lilly and some others back at Purdue. One of the key items that continues to come up in the early stages of the drug development process is this notion of failing fast. They know that most of the ideas they pursue will fail. They believe that only one or two will actually make it all the way through the pipeline, which right now takes about 8 to 12 years. The company wants to identify those ideas that will succeed and those that will fail very quickly, and they call that “failing fast.” They do celebrate killing those projects despite emotional ties. Is this notion strongly pursued in the nonpharmaceutical parts of product discovery?

Allen Clamen: Yes. I mentioned earlier that Stage 3 development activities take the bulk of the cost, time, and resource utilization. If a project can fail prior to that full development stage, you will be much more successful than if you had to waste resources to get to that point.

We give a great deal of effort to learning in the early stages. I called it up-front planning about the manufacturability of the products, the marketability of the products, and the home that these products would ultimately have in terms of our competitive advantage. This knowledge is important upfront rather than as you go through the process. The very early stages of basic research are a large investment. Recognizing that and making the decision early are absolutely critical. Failing fast, failing early—those are key concepts to making the whole enterprise more successful.

Michael Schrage: The term “failure” needs to be better defined. If the criteria for the project change in the course of research and development, the initiative hasn't failed; the hurdle rate has changed. It fails relative to malleable criteria, when in fact the underlying science may be quite valid and quite useful. Failure here is ironically too all encompassing, rather than focused. We need to see how our definitions of failure change.

The other thing is in terms of criteria. When we talk about risk, risk cannot be divorced from cost. When you have simulation tools, the cost of doing a test with computational chemistry is two orders of magnitude lower in the year 2002 than it was even 10 years ago. The idea that we talk about risk as some sort of fixed point is also dangerous. Additionally, the notion of our cost structures is also changing. Attention needs to be drawn to that explicitly.

Allen Clamen: What I would call a failure is the failure to recognize information that we had earlier that would have prevented the development that was wasted.

Mary L. Mandich: Could you comment on the human money resource versus the actual computers, workstations?

Allen Clamen: The ability we have in this day and age to use all the tools allows our people to do a lot more with less. That is critical.



Allen Clamen (now retired) was senior advisor for marketing/technology value creation at ExxonMobil Chemical Company in Houston, Texas. He was responsible for developing effective and efficient processes for idea management, portfolio management, and stage gating of new product development projects. For the past 4 years he has led teams to create value via improved marketing and technology processes.


P. Belliveau, A. Griffin, and S.M. Somermeyer, eds., 2002. The PDMA ToolBook for New Product Development, Hoboken, NJ: John Wiley & Sons.


G. Tritle, E. Scriven, and A. Fusfeld. 2000. Resolving uncertainty in R&D portfolios. Research Technology Management 43:47–50.

Copyright © 2003, National Academy of Sciences.
Bookshelf ID: NBK36336
PubReader format: click here to try


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (5.5M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...