NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Committee on Capitalizing on Science, Technology, and Innovation: An Assessment of the Small Business Innovation Research Program—Phase II; Board on Science, Technology, and Economic Policy; Policy and Global Affairs; National Academies of Sciences, Engineering, and Medicine. SBIR/STTR at the National Institutes of Health. Washington (DC): National Academies Press (US); 2015 Nov 12.

Cover of SBIR/STTR at the National Institutes of Health

SBIR/STTR at the National Institutes of Health.

Show details

5Quantitative Outcomes

This chapter analyzes outcomes related to the efforts by NIH to address the congressional mandate to increase commercialization of federally funded research and to stimulate technological innovation under its SBIR and STTR programs.1

This chapter reviews quantitative data provided through the 2014 Survey of Phase II SBIR/STTR recipients at NIH. It focuses primarily on the commercialization outcomes and knowledge effects from SBIR/STTR awards, as well as the longer term impact on the companies themselves. The chapter is followed by an Annex that includes more detailed descriptions and analysis.

There is evidence that NIH has achieved some success in tracking outcomes of SBIR/STTR awards, but more work needs to be done. NIH is currently building electronic links to the Small Business Administration (SBA) outcomes database now under development and working to develop its own outcomes tracking system (see Chapter 3); however, data from these sources are not yet available. Thus, the analysis of outcomes in this report is based primarily on the 2014 Survey by the Academies, which tried to survey all SBIR and STTR Phase II awardees for the period FY2001-2010 inclusive.2

A detailed description of the methodology underlying the 2014 survey is provided in Appendix A of this report (see Box 5-1). The full text of the survey is provided in Appendix C. Overall, the survey of NIH Phase II SBIR/STTR award recipients generated 726 responses. In cases where company information, as opposed to individual project information, was collected, multiple responses from the same company were aggregated and then averaged to provide a better view of company-level activities.

Box Icon

BOX 5-1

Workshop on Improving University-SBIR/STTR Linkages. A workshop convened on February 5, 2014, by the committee considered a range of issues concerning universities and the SBIR/STTR programs. Participants at this workshop addressed a range of topics including: (more...)

A more detailed presentation of the data collected via the survey, including response rates, is included at the end of this chapter in Annex 5-A.

COMMERICIALIZATION

As with our other reports on the SBIR program, we have adopted a broad view of commercialization, taking it to include additional investments from outside the SBIR/STTR programs as well as sales and licensing revenues. In addition, given the long time to market required for many life sciences technologies, we have been careful to include a range of benchmarks and metrics, having determined that no single metric can appropriately capture such a broad concept.

That said, we focus first on different ways of measuring sales and other types of commercial revenue as well as further investment. In line with previous studies by the Academies and consistent practice at all agencies, investment beyond Phase II is recognized as acknowledgement by third parties that the project has developed technologies of marketable value. For many projects, further investment is required before commercial sales can begin. An extended discussion of approaches to measuring commercialization is contained in the Annex to this chapter.

NIH is also in some ways a special case for commercialization. Because NIH is not an agency where the SBIR/STTR programs are designed to generate technologies for use by the agency itself, markets have to be found outside the agency. The path to such markets is particularly difficult for the large percentage of projects that require U.S. Food and Drug Administration (FDA) approval; such approval was required for 45 percent of the surveyed projects. Regulatory approval is expensive—sometimes extremely expensive—and time consuming. These are two formidable challenges for small companies. It is therefore especially important to capture milestones along the way to commercialization as well as commercial sales and related revenues.

Sales and Revenues

Perhaps the single most used metric for assessing SBIR-type programs is revenue or licensing fees. As recommended in a previous Academies' report3 overreliance on this particular metric may lead to incorrect conclusions about the program, although they are important considerations.

Reaching the Market

The first question in this section concerns reaching the market: Did the project generate any sales, and if not, are sales expected (a necessary question given the long cycle time of some projects)? Responses are summarized in Figure 5-1.

FIGURE 5-1. NIH SBIR/STTR sales and licensing revenues (percentage of respondents).

FIGURE 5-1

NIH SBIR/STTR sales and licensing revenues (percentage of respondents). NOTE: N=602. See Table 5-7 in the chapter annex for details. SOURCE: 2014 Survey, Question 32.

Overall, just less than one-half of projects reported some sales or licensing revenues, and a further 25 percent expected sales in the future. These data are similar to those generated by the previous survey of NIH SBIR-only awardees by the National Research Council (NRC)4 in 2005.5

Amount of Sales and Licensing Revenues

Simply identifying the percentage of projects reaching the market is an important metric, but it is not sufficient; it is also necessary to understand the scale and distribution of sales. The 2014 survey asked those who reported some sales of the technology developed for the surveyed project to report the amount of sales, grouped into ranges. These data are summarized in Figure 5-2.

FIGURE 5-2. Distribution of total sales and licensing revenues by range (percentage of respondents reporting sales).

FIGURE 5-2

Distribution of total sales and licensing revenues by range (percentage of respondents reporting sales). NOTE: N=263 (projects reporting sales). See Table 5-8 in the chapter annex for details. SOURCE: 2014 Survey, Question 34.

Most reported sales at the lower end of the scale: 62 percent were less than $500,000 and more than onehalf of those sales (39 percent of the total) were under $100,000. Four percent reported revenues of at least $20 million, while 8 percent reported sales of between $5 million and $20 million.

Markets by Sector

The 2014 Survey also asked respondents about the market sectors in which sales were made. Overall, 58 percent identified the private sector, followed by export markets (17 percent). Sales to federal agencies comprised 6 percent of sales (see Figure 5-3).

FIGURE 5-3. Markets for NIH SBIR/STTR products and services: Percentage of total sales (mean of all responses/category).

FIGURE 5-3

Markets for NIH SBIR/STTR products and services: Percentage of total sales (mean of all responses/category). NOTE: N=265 (projects reporting sales). See Table 5-9 in the chapter annex. For this question, each respondent reports a percentage distribution. (more...)

Further Investment

The ability of SBIR/STTR projects and companies to attract further investment has traditionally been an important measure of SBIR/STTR commercialization outcomes.6 There has also been interest in the sources of additional funding for high-tech innovation. Although the United States has historically been a leader in venture capital and angel investment, these are not the only or even the primary sources of additional investment funding for NIH SBIR/STTR projects.

Overall, more than 80 percent of respondents indicated that their project received additional investment in the technology related to the surveyed project.7 As with prior surveys, the data show small amounts of additional funding are most likely. Table 5-1 shows the amount of funding received. About 76 percent of all projects received less than $1 million in additional investment with the median amount of additional funding equaling $300,000. Four respondents (less than 1 percent) reported receiving $50 million or more in additional funding, while 9 percent reported receiving $5 million or more. These data highlight funding challenges for these companies because the cost of Phase III clinical trials has recently been estimated at $26,000 per patient.8 Phase III trials can require the enrollment of more than 1,000 patients.9

TABLE 5-1. Additional Funding Received by Funding Mechanism and Amount.

TABLE 5-1

Additional Funding Received by Funding Mechanism and Amount.

Of the 470 respondents that reported additional funding, 44 percent was from private-sector sources, 9 percent was from venture capital sources, and 14 percent was from angel and other private equity investors. Twenty-one percent reported strategic investments from partners, which is especially important in the context of life sciences where large pharmaceutical and medical device companies are a critical part of the path to market (see Table 5-2).

TABLE 5-2. Distribution of Responses Related to Additional Investment Funding by Source of Funds.

TABLE 5-2

Distribution of Responses Related to Additional Investment Funding by Source of Funds.

Twenty-five percent of respondents reported funding from non-SBIR/STTR federal sources, which would include such potential funders as BARDA, the Centers for Medicare & Medicaid Services, and the Veterans Administration, or the Department of Defense.

As shown in Table 5-13 in the chapter annex, 24 percent of respondents reported funding from other external sources, including 16 percent from state and local governments and 9 percent from research institutions. Overall, the most utilized funding source was the company itself (58 percent) and in many cases personal funds (26 percent).

TABLE 5-13. Distribution of All Reported Additional Investment Funding by Source of Funds.

TABLE 5-13

Distribution of All Reported Additional Investment Funding by Source of Funds.

SBIR/STTR and Clinical Trials

Many NIH-funded projects face the challenge of proceeding through clinical trials before they can seek success in the marketplace. Survey responses indicate that this was true for 46 percent of projects.10 For these companies, the road to successful conclusion of clinical trials is very challenging (Table 5-3). More than one-third of projects had abandoned the process, and about another one-third were preparing for entry into clinical trials. Nine percent of projects had completed clinical trials (21 percent of Phase IIB projects).

TABLE 5-3. Outcomes for Projects Requiring FDA Approval.

TABLE 5-3

Outcomes for Projects Requiring FDA Approval.

These figures are difficult to interpret because the survey covered a 10-year period. Newer projects are more likely to be preparing for clinical trials or in earlier stages than are older projects. Still, it is fair to conclude that a majority of those requiring approval will not in the end receive it given that the probability of a project being abandoned increases with its age.

These data can be compared with the general set of outcomes for all efforts to receive FDA approval. Table 5-4 shows that 8 percent of those projects entering the clinical trials process eventually received approval, while 30 percent of projects were approved for Phase 1 clinical trials. Survey responses match these estimates quite closely, which is noteworthy given that Table 5-4 includes data from larger and much better funded companies.

TABLE 5-4. Outcomes for FDA Applications at Different Phases.

TABLE 5-4

Outcomes for FDA Applications at Different Phases.

SBIR/STTR companies utilize a wide range of funding sources to meet the demands of clinical trials. Figure 5-4 shows that the largest single source was internal company funding (51 percent) followed by SBIR/STTR itself (32 percent) and Phase IIB (22 percent). Angel funding, venture funding, and strategic funding from other companies were each mentioned by 15 percent of respondents.

FIGURE 5-4. Sources of funding reported for FDA clinical trials (numbers of responses).

FIGURE 5-4

Sources of funding reported for FDA clinical trials (numbers of responses). NOTE: N=226 (projects requiring FDA approval). Numbers do not sum to 226 because respondents could select more than one answer. BARDA is the Biomedical Advanced Research and Development (more...)

For those receiving SBIR/STTR support of some kind for clinical trials, 48 percent thought that the funding was extremely or very useful, while 33 percent said it was not at all useful.11 Among those who received Phase IIB funding, positive responses were higher—more than three quarters of respondents said that it made a “tremendous difference,” while 24 percent said that it made little or no difference (N=21, so responses should be treated with caution).12

However, while the additional funding was regarded as helpful, it was not seen as sufficient. Figure 5-5 shows that of the 21 Phase IIB responses, one-third thought the funding was not sufficient even to complete preparation for clinical trials, and a further 29 percent thought it was sufficient only for completion of those preparations. Five percent (one respondent) thought it was sufficient for Phase 3 trials.

FIGURE 5-5. Sufficiency of Phase IIB funding for different phases of clinical trials (percentage of respondents).

FIGURE 5-5

Sufficiency of Phase IIB funding for different phases of clinical trials (percentage of respondents). NOTE: N=21. SOURCE: 2014 Survey, Question 47.

KNOWLEDGE EFFECTS

One of the four congressionally mandated objectives for the SBIR/STTR programs is to “stimulate technological innovation.” Although patents and peer-reviewed papers are not the only useful way to assess the development and transmission of knowledge by small high-tech companies, they offer a useful starting point.

Figure 5-6 shows the number of patents related to all SBIR/STTR awards reported by companies participating in the 2014 Survey. About two-thirds of respondent companies received at least one such patent, and about 13 percent received 10 or more.

FIGURE 5-6. Number of patents reported related to all company SBIR/STTR awards (percentage of company-weighted responses).

FIGURE 5-6

Number of patents reported related to all company SBIR/STTR awards (percentage of company-weighted responses). NOTE: N=409. SOURCE: 2014 Survey, Question 12.

The survey also asked about patents related to the specific project being surveyed. About 53 percent of respondents reported at least one patent related to the project, and 4 percent reported at least 10 (see Table 5-5).13

TABLE 5-5. Number of Patents Received Related to Surveyed Project.

TABLE 5-5

Number of Patents Received Related to Surveyed Project.

In addition to patents, the survey asked about articles in peer-reviewed journals. Meetings with company executives indicated that, for many companies, even though technical knowledge and trade secrets are very important, the company strongly supported peer-reviewed publication. In part, companies saw this as marketing among peers, both for eventual products and a means of attracting talent. Eighty percent of survey respondents reported publishing at least one peer-reviewed article related to the surveyed project. Forty-two percent reported publishing at least three articles, as shown in Figure 5-7. As shown in Table 5-19 in the annex section of this chapter, two-thirds of Phase IIB responses reported publishing at least 3 articles and 21 percent reported publishing at least 10.

FIGURE 5-7. Number of peer-reviewed articles relating to surveyed project (percentage of respondents).

FIGURE 5-7

Number of peer-reviewed articles relating to surveyed project (percentage of respondents). NOTE: N=508. SOURCE: 2014 Survey, Question 38.

TABLE 5-19. Peer-reviewed Scientific Publications Related to the Surveyed Project.

TABLE 5-19

Peer-reviewed Scientific Publications Related to the Surveyed Project.

Another mechanism for knowledge transfer is the development of links between SBIR/STTR companies and their projects with research institutions (RIs). Sixty-five percent of SBIR respondents reported some connection to an RI. All STTR respondents are required to partner with an RI. Many reported that RI faculty worked on the surveyed project (39 percent used faculty as consultants), while a smaller number reported that the technology was originally developed at and/or was licensed from the RI (see Table 5-6).

TABLE 5-6. Connections to Research Institutions (RIs) and SBIR/STTR Awards.

TABLE 5-6

Connections to Research Institutions (RIs) and SBIR/STTR Awards.

COUNTERFACTUALS

Because there is no available matched set of companies that did not receive SBIR/STTR Phase II funding at precisely the point in time that surveyed companies did receive funding, it is not possible to develop an appropriate control group against which to measure impacts (see discussion of the Academies efforts to do so in Appendix A). However, it is at least possible to ask—as previous surveys by the Academies and the Government Accountability Office (GAO) have done—what the company itself believed might have happened had SBIR/STTR funding not been available. While this is of course subjective, the company is best suited to provide these answers.

Because alternative funding especially for long-cycle projects and those requiring FDA approval is difficult to acquire, it is not surprising that 7 percent of respondents believed that the project would definitely or possibly have proceeded without funding. Conversely, almost three-quarters of respondents said that it would likely or definitely not have proceeded.14

These data have interesting wider implications for debates about early-stage funding; notably, they suggest poor support for the “crowding out” hypothesis (that public funding displaces private investment). Awardees in our survey—presumably those with the closest knowledge of funding prospects for the project—overwhelmingly believed it to be unlikely that alternative private funding would be found. These results also underscore the importance of SBIR/STTR funding for these small companies.

COMPANY IMPACTS

Although the effect of SBIR/STTR funding on the company is not directly included in the congressional objectives for the program, helping small companies to become self-sufficient (and in some cases to grow rapidly) does have implications for program impacts and are therefore included in our analysis.

Small high-tech companies are often fluid in structure, and the 2014 Survey found that many participating companies changed structurally in recent years. Thirty-five percent established strategic partnerships with major players, while 21 percent spun off at least one company and 16 percent were acquired by or merged with another firm.15

Ideally, companies that receive SBIR/STTR funding become more stable and develop contracts that are not SBIR-related over time. This appears to be the case for NIH SBIR/STTR companies as dependence on SBIR/STTR funding is limited. Overall, 42 percent of respondents indicated that the SBIR/STTR programs were currently funding 10 percent or less of the company's total research and development (R&D) effort, while about 34 percent indicated that they were funding more than one-half.16 This picture is reinforced by data on sources of company revenues. Thirty-four percent of responding companies reported zero SBIR/STTR revenues, while about 27 percent reported receiving more than one-half of the company's revenues from SBIR/STTR.17

The survey also asked about the overall impact of SBIR/STTR on the company. As Figure 5-8 shows, 62 percent saw a highly positive or transformative effect, and another 35 percent reported a positive impact. Two respondents reported a negative or highly negative impact.

FIGURE 5-8. Long-term impact of SBIR/STTR on companies (percentage of respondents).

FIGURE 5-8

Long-term impact of SBIR/STTR on companies (percentage of respondents). NOTE: N=580. SOURCE: 2014 Survey, Question 57.

ANNEX 5-A: QUANTITATIVE OUTCOMES

This annex elaborates the results of the 2014 survey of quantitative outcomes of the NIH SBIR/STTR programs, summarized in Chapter 5. As noted earlier, this quantitative assessment of outcomes from the NIH SBIR/STTR programs focuses primarily on two of the four congressionally mandated objectives for the program: commercialization of federally funded research and stimulation of technological innovation. Data on the participation of women and minorities is included in Chapter 6. Data on program management is included in Chapter 2.

FOCUS ON COMMERICALIZATION OUTCOMES

Although there are four statutory goals for the SBIR/STTR programs, subsequent legislation passed by Congress, as well as administrative policies pursued by NIH and the other major SBIR/STTR agencies focus primarily on the commercialization of SBIR/STTR technologies.18 Moreover, given that commercialization is among the more measurable outcomes of the SBIR/STTR programs, it has become a primary benchmark for program performance. The focus on commercialization, however, should not eclipse the requirement that the program meet all four congressionally mandated objectives. This Annex provides additional details of the commercial outcomes of the NIH SBIR/STTR programs, as well as quantitative outcome measures related to stimulating technological innovation.

SOURCES OF DATA

Although NIH was an early adopter of survey-driven outcomes research, the agency has not thereafter led the way on tracking outcomes. Other agencies have moved more rapidly to meet the need for data:

  • The Department of Defense (DoD) maintains the Company Commercialization Record, which requires all companies applying for DoD SBIR/STTR funding to update outcomes for all prior awards.
  • The National Science Foundation (NSF) utilizes a consultant to undertake phone interviews with recipients at set times several years after the end of the award.
  • The Department of Energy (DoE) internally tracks award outcomes for several years using its own metrics and methodologies.
  • The National Aeronautics and Space Administration (NASA) developed a tracking module as part of its Electronic Handbook, which has been used in recent years collect outcomes data.

NIH managers are aware of this situation and have recently moved to correct this deficiency. According to the NIH SBIR/STTR Program Office, NIH has undertaken a twin-track approach. It has built an electronic bridge between the new SBA outcomes database (which is not yet online as of this writing). In addition, it is investing in a new module for its own internal Performance Outcomes Data System (PODS) database, which will address outcomes. These issues are discussed in more detail in Chapter 2 (Program Management).

Given the lack of available data from NIH, the quantitative data presented in this chapter are derived from the 2014 Survey of award recipients. We stress, however, that these data are descriptive only and should be regarded as providing insights into outcomes rather than definitive conclusions.19

The 2014 Survey is based primarily on the 2005 survey, with some additions and modifications. This 2014 survey was sent to two distinct populations: (1) all principal investigators (PIs) who received an NIH SBIR or STTR Phase II award between FY2001 and FY2010 inclusive and (2) in cases where PIs could not be reached, alternate company contacts at the targeted companies. Results from this survey provide quantitative insights that permit the analysis provided in this chapter. They are, where appropriate, compared to the results from the 2005 Survey.20

Appendix A provides a detailed discussion of the survey methodology, including response rates and potential survey bias. Below, a series of tables summarize Phase II responses for SBIR, STTR, and Phase IIB recipients.21 The 2014 Survey is reproduced in Appendix C. References to Phase II refers to SBIR and STTR awards, not including Phase IIB.

OUTCOMES

Commercialization

At NIH, the priority for SBIR/STTR is to support the development and commercialization of technologies that will improve the nation's health. In contrast to DoD and NASA, it is not expected that SBIR/STTR technologies will be used by the agency itself. Sales are primarily made into the domestic private sector, although sales to other health-related government agencies are also quite substantial.

NIH SBIR/STTR companies also face a particular challenge, aside from the difficulties faced by all small companies as they seek to gain market traction for their products. Survey responses indicate that nearly one-half of respondent companies anticipated that they would need to successfully complete FDA clinical trials before their product could be marketed. Academic papers indicate that about 8 percent of products that file for clinical trials make it to the end of Phase 3, so these challenges are real and pervasive for these companies.

That said, SBIR/STTR program participants at NIH are—as at other agencies—small for-profit companies, and they must proceed in ways that do in the end provide a sustainable path forward for the business.

Defining “Commercialization”

Several important conceptual challenges emerge when seeking to define “commercialization” for the purposes of the SBIR/STTR programs. Like many apparently simple concepts, commercialization becomes progressively more difficult and complex as it is subjected to further scrutiny. For example:

  • Should commercialization include just sales or other kinds of revenue, such as licensing fees and funding for further development?
  • Should commercialization include only certain kinds of sales—excluding for example sales to government agencies?
  • What is the appropriate benchmark for sales? Is it any sales whatsoever, sufficient sales to cover the costs of awards, sales that lead to breaking even on a project, or sales that reflect a commercial level of success and viability? The last at least would likely be different for each project in each company.
  • Should commercialization include sales by licensees, which may be many multiples of the revenues provided to, but are largely reported by, the licensors?

For the purposes of this study, we deployed a broad net to capture a range of potentially useful data. Once acquired, these data can be analyzed in a variety of ways to provide multiple insights into this complex topic.22

Sales and Revenues

Perhaps the single most used metric for assessing SBIR-type programs is revenue or licensing fees. Although we have already cautioned against overuse of this metric—warnings that are applicable to the wide range of metrics adopted for use in the current assessment—sales and revenues are still important considerations.23

Reaching the market. The first survey question in this area concerns reaching the market: Did the project generate any sales, and if not, are sales expected (a necessary question given the long cycle time of some projects)? Responses are summarized in Table 5-7. About one-half of respondents reported some sales or licensing revenues, and a further one-quarter expected sales in the future. The percentage reporting sales to date was lower than for the 2005 Survey (57 percent). Those expecting sales in the future increased from 19 percent for the 2005 Survey to 25 percent (SBIR-only) for the 2014 Survey.

TABLE 5-7. NIH SBIR/STTR Sales Outcomes.

TABLE 5-7

NIH SBIR/STTR Sales Outcomes.

The 2005 Survey of SBIR companies found that 24 percent of respondent companies had no sales and expected none, 19 percent had no sales but expected sales in the future, and 57 percent had already generated sales from the surveyed project.24

Amount of sales and licensing revenues. Simply identifying the percentage of projects reaching the market is an important metric, but it is not sufficient. In addition, it is important to understand the distribution of sales. The survey also asked those who reported some sales of the technology developed for the surveyed project to report the amount of sales, segregated by tiers. These data are summarized in Table 5-8. Overall, 26 percent of SBIR respondents reported sales of at least $1 million.

TABLE 5-8. Distribution of Total Sales Dollars, by Range.

TABLE 5-8

Distribution of Total Sales Dollars, by Range.

Markets by sector. Because NIH is not itself a significant market, it is not surprising that most sales are made to the domestic private sector. Furthermore, because health care products are increasingly a global business, it is also not surprising that export markets accounted for 17 percent of sales (see Table 5-9). These figures are very similar to those for the 2005 Survey.25

TABLE 5-9. Markets by Sector.

TABLE 5-9

Markets by Sector.

Employment

As with prior surveys, the 2014 Survey asked respondents both about the size of the company at the time of the award and the current size, in terms of number of employees. At the time of the award, 60 percent of responding companies had fewer than 10 employees. The median was 7 employees (see Table 5-10).

TABLE 5-10. Number of Employees at Time of Award.

TABLE 5-10

Number of Employees at Time of Award.

The survey also asked about current employment (as of 2014). Respondents reported that the median size of companies was still 7 employees, but that the mean size had grown significantly, from 19 to 88. Among the 7 percent of firms now reporting at least 100 employees, some had grown substantially (see Table 5-11).26

TABLE 5-11. Employment at Time of Survey.

TABLE 5-11

Employment at Time of Survey.

Further Investment

The ability of SBIR/STTR projects and companies to attract further investment has traditionally been a defining metric for SBIR/STTR outcomes.27 There has also been interest in the sources of additional funding for high-tech innovation. The United States has historically been a leader in venture capital and angel investment.

Overall, about 80 percent of respondents indicated that their company had received additional investment in the technology related to the surveyed project.28 As with prior surveys, most respondents received small amounts of additional funding. Table 5-12 shows responses related to additional funding. Three-quarters of all NIH respondents reported receipt of less than $1 million. One percent reported receiving $50 million or more in additional funding; 9 percent reported receiving $5 million or more. Given that the cost of Phase 3 clinical trials has recently been estimated at $26,000 per patient29 and that can require the enrollment of more than 1,000 patients,30 the funding challenge for SBIR/STTR companies is immediately apparent.

TABLE 5-12. Additional Funding by Phase and Amount.

TABLE 5-12

Additional Funding by Phase and Amount.

Of those projects that received additional funding, 44 percent reported funding from U.S. private-sector sources, 25 percent from non-SBIR/STTR federal sources, and 24 percent from other external sources. Seventy percent reported additional funding from their own company, including 26 percent who reported personal investments. Overall, 10 percent and 14 percent of those who received additional funding identified U.S. venture capital and U.S. angel and other private equity investors as the sources, respectively. Twenty-one percent and 4 percent reported strategic investments from U.S. partners and foreign partners, respectively (see Table 5-13).

Company-Level Commercialization Through Mergers and Acquisitions

SBIR/STTR firms often commercialize their technology through mergers or other company-level activities. Greater than 43 percent of responding companies indicated that they had not been acquired, had not implemented or planned an initial public offering (IPO), and had not established a spin-off (see Table 5-14). Conversely, greater than one-third had entered into a strategic partnership with a major industry player, 21 percent had established one or more spin-off companies, and 16 percent had been acquired by or merged with another firm.

TABLE 5-14. Company-Level Changes.

TABLE 5-14

Company-Level Changes.

Commercialization Training and Marketing

NIH has provided commercialization training for SBIR/STTR awardees for 10 years, primarily through an arrangement with a third-party provider (see Chapter 3, Program Initiatives). Thirty-nine percent of SBIR respondents and 24 percent of STTR respondents engaged in commercialization training at NIH.31 More than 38 percent considered the training to be valuable or extremely valuable (see Table 5-15). Conversely, about one-quarter of participants thought it was not very valuable or not at all valuable. Phase IIB respondents tended to see the training as less valuable, perhaps because they were already farther down the commercialization path. STTR recipients saw the training as slightly more valuable than did SBIR recipients, perhaps reflecting that they had less information about commercial strategy to begin with given their strong academic roots.

TABLE 5-15. Value of Commercialization Training.

TABLE 5-15

Value of Commercialization Training.

However, about 27 percent of respondents indicated that they were likely to use the existing agency commercialization program in the future. Forty-nine percent expressed a preference to use the funding for their own marketing efforts, as permitted under the reauthorization legislation for SBIR/STTR (see Table 5-16). This is money that the grantees must use for paying for the training.

TABLE 5-16. Use of Different Commercialization Support.

TABLE 5-16

Use of Different Commercialization Support.

One question added to the 2014 Survey asked whether the company has at least one full-time staff person for marketing. This question provides another metric of the extent to which the company has focused on marketing. Forty-three percent of respondant companies reported that their company had at least one full-time marketing staff.32

Conclusions: Commercialization at the Company Level

Evidence from the 2014 Survey provides useful insight into the commercialization record of SBIR/STTR companies at NIH, on a number of dimensions. The data confirm that a substantial percentage of projects do indeed commercialize through sales of products or services and/or through the receipt of additional development funding.

Forty-nine percent of respondents indicated that their company had already recorded sales of products or services derived from the awarded project. A further one-quarter of respondents were expecting sales in the future. Given the relatively short time between the award date for some of these awards and the survey date, and the long time-to-market cycle for products with regulatory requirements to meet, these expectations are not unreasonable. NIH does provide independent data against which the validity of the survey responses can be cross-checked.

Overall, the scale of commercialization is limited. About three-quarters of respondents with project-related revenues indicated that these revenues totaled $1 million or less.33 About 10 projects reported sales greater than $50 million and another 13 or 14 reported sales between $10 million and $50 million.

Additional investment is another important metric for commercialization. Many Phase II projects are not yet ready for the marketplace at the end of the award period, especially given the need for regulatory compliance.34 Three-quarters of all respondents reported receiving less than $1 million in additional funding. One percent reported receiving $50 million or more in additional funding; 9 percent reported receiving $5 million or more. The source of additional funding varied. About one-quarter of respondents mentioned non-SBIR/STTR federal funding, while 44 percent mentioned U.S. private-sector funding (including 10 percent for venture capital and 14 percent for angel investments). U.S. strategic investors were also important (21 percent).

In conclusion, these data support our view that SBIR/STTR funding is associated with outcomes that meet congressional mandates for commercialization. In the future, better data, especially collected by the agency, would allow for a more definitive conclusion and also more detailed understanding of the links between agency programs and initiatives and outcomes.

Knowledge Effects

One of the four congressionally mandated objectives for the SBIR/STTR programs is to “stimulate technological innovation,” which is often equated with patenting activity. However, in the context of small business, this standard metric of innovation does not capture the entire story: patenting is important, but it is also expensive, and SBIR/STTR funds cannot legally be used for this purpose. During case study discussions (for this and previous reports by the Academies on SBIR), company executives explained that patents have their limitations and are expensive. As a result, they prefer to keep their technology secret or to rely on first-mover advantages and other market-based leverage to defend their technologies.

However, standard metrics provide at least a starting point for quantitative analysis. Consequently, the survey addressed several intellectual property (IP)-related metrics: patents, trademarks, copyrights, and peer-reviewed papers.35

Patents

Patents are to some degree the life blood of high-tech firms. Overall, about two-thirds of companies (and more than 80 percent of Phase IIB recipients) reported the award at least one patent related to any SBIR/STTR-funded technology; 13 percent reported at least 10 related patents (see Table 5-17).

TABLE 5-17. Number of Patents Related to All Company SBIR/STTR Awards.

TABLE 5-17

Number of Patents Related to All Company SBIR/STTR Awards.

The survey also asked questions about IP related to the surveyed award (Table 5-18). Greater than one-half of respondents reported receiving at least one patent related to the surveyed technology. Nine percent reported receiving 5 or more related patents, and 4 percent reported 10 or more. STTR respondents reported similar numbers. The share of respondents reporting at least one patent is slightly down from the 2005 survey (57 percent).36

TABLE 5-18. Patents Awarded Related to Surveyed Project.

TABLE 5-18

Patents Awarded Related to Surveyed Project.

Copyrights and Trademarks

Regarding copyrights and trademarks, about one-quarter of respondents reported receiving a trademark related to the surveyed project, while less than 10 percent reported receiving a copyright.37

Peer-Reviewed Publications

Publications in peer-reviewed journals and conference proceedings are a standard method for disseminating scientific knowledge. As with the first-round assessment by the Academies, several case study respondents noted that publication in peer-reviewed journals was an essential part of the firm's work.38

For the purposes of this assessment, peer-reviewed publications are important for two reasons:

  • They validate the quality of the research being conducted with program funds.
  • They are themselves the primary mechanism through which knowledge is transmitted within the scientific community.

The survey therefore asked about peer-reviewed publications (Table 5-19). Eighty percent of SBIR respondents and 85 percent of STTR respondents indicated that an author at the surveyed company had published at least one related scientific paper. Overall 42 percent reported publishing three or more related papers. The median number of publications for Phase IIB respondents was five.

Links to Universities

The survey asked a number of questions about the use of university staff and facilities on the surveyed project. Overall, about two-thirds of SBIR respondents and essentially all STTR respondents reported a university connection of some kind.

There were substantial differences between SBIR and STTR with regard to the kind of university linkage (Box 5-2). Seventy-eight percent of STTR respondents and 6 percent of SBIR respondents reported that the PI was a university faculty member. STTR respondents were also more likely to report that technology was licensed from the research institution (27 percent vs. 14 percent) and that the technology was originally developed at the research institution by a project team member (37 percent vs. 17 percent) (see Table 5-20).

Box Icon

BOX 5-2

Survey Response Rate and Non-Respondent Bias. As noted in the introduction to this report, and described in detail in Appendix A, the committee recognizes the limitations of the survey effort underlying the data presented in this chapter. The 2014 Survey was (more...)

TABLE 5-20. Links to Universities.

TABLE 5-20

Links to Universities.

Respondents were also asked to identify the universities with which they worked in various capacities on this project. Although the type of help varied widely, some universities were mentioned by a number of respondents. Overall, 255 different research institutions were identified from 488 projects. Those mentioned by four or more respondents are listed below in Table 5-21 (see Appendix D for the complete list of university mentions). Many of the names on this list are large state universities, a number of which have in recent years focused on technology transition as well as basic research, although two of the top three universities mentioned are private universities. Although far from a perfect metric, we believe these data provide a preliminary indication of the connections between specific universities, university systems, and the NIH SBIR/STTR programs.

TABLE 5-21. University Participants Mentioned by Four or More Respondents.

TABLE 5-21

University Participants Mentioned by Four or More Respondents.

Finally, for 85 percent of companies in the sample, at least one founder had an academic background (see Table 5-22) and for 60 percent of companies at least one founder was most recently employed at a research institution (see Table 5-23).

TABLE 5-22. Number of Academic Founders.

TABLE 5-22

Number of Academic Founders.

TABLE 5-23. Previous Employment of Founders.

TABLE 5-23

Previous Employment of Founders.

Conclusions: Knowledge Effects

What emerges from these data is a picture of companies that are dynamic centers of technological innovation, a considerable amount of which is protected through the patent system. About two-thirds of companies reported receipt of at least one patent based on their work under SBIR/STTR contracts, while 53 percent reported receipt of at least one patent related to the surveyed project only specifically.

SBIR/STTR companies participate at a high level in the standard form of technical knowledge dissemination: publishing in peer-reviewed journals. Eighty percent of respondents reported that their company published at least one article based on the SBIR-funded work, and more than 40 percent reported publication of more than three such papers.

Finally, some SBIR/STTR companies are closely connected to the universities. About 70 percent of respondents reported a university connection on the surveyed project, across a number of different kinds of linkages, and 22 universities were specifically mentioned as playing a role in at least five reported projects. This suggests that SBIR and STTR may in some cases play a potentially important role in supporting the practical implementation of university research.

SBIR/STTR AND COMPANIES

SBIR/STTR programs have a range of effects on companies, which affect their ability to operate and grow. Data about companies can help to define the technological space in which the SBIR/STTR programs operate. In addition, a review of the SBIR/STTR share of overall company activities can provide insights into the degree of dependence on SBIR/STTR for individual companies.

Impact on Company Formation

SBIR/STTR can have a profoundly catalytic impact on company formation. Seventeen percent of respondent companies were founded because of the SBIR/STTR programs, and a further 27 percent were formed in part because of the program (see Table 5-24).

TABLE 5-24. SBIR/STTR Impact on Company Formation.

TABLE 5-24

SBIR/STTR Impact on Company Formation.

SBIR/STTR Share of R&D Effort

The survey asked respondents to estimate how much of their company's total R&D effort—defined as man-hours of work for scientists and engineers—was devoted to SBIR/STTR-funded projects. Overall, 42 percent of SBIR/STTR respondents indicated that the programs funded 10 percent or less of total effort, while 34 percent indicated that they funded greater than one-half (see Table 5-25).

TABLE 5-25. Percentage of R&D Effort Funded by SBIR/STTR.

TABLE 5-25

Percentage of R&D Effort Funded by SBIR/STTR.

These data correspond fairly closely to responses from Question 9, which asked what percentage of company revenues during its current year were from SBIR/STTR awards. Thirty-four percent of companies reported zero SBIR/STTR revenues, while 27 percent reported receiving greater than one-half of revenues from SBIR/STTR. Three percent were entirely dependent on SBIR/STTR (see Table 5-26).

TABLE 5-26. Percentage of Company Revenues from SBIR/STTR.

TABLE 5-26

Percentage of Company Revenues from SBIR/STTR.

Prior Use of the SBIR/STTR Programs

Although a more linear interpretation of the innovation process would imply that ideas are tested in Phase I, prototyped in Phase II, and commercialized in Phase III, actual practice involves multiple iterations, or projects that must restart with an earlier phase, or multiple efforts needed to meet specific problems.

The survey asked respondents to indicate how many of the prior SBIR/STTR Phase I awards received from the NIH and other agencies were related to the project and technology being surveyed. Table 5-27 summarizes the responses. Less than 20 percent of projects received no other related SBIR/STTR awards. Greater than 20 percent received at least three additional related awards. These data strongly support the view that innovative products emerge from clusters of activity, rather than from simple straight line development from Phase I to Phase II to commercialization.

TABLE 5-27. Prior SBIR/STTR or STTR Phase I Awards Related to the Surveyed Project.

TABLE 5-27

Prior SBIR/STTR or STTR Phase I Awards Related to the Surveyed Project.

Turning to prior Phase II awards, about three-quarters of respondents reported at least one related Phase II award, while 11 percent reported at least three (see Table 5-28).

TABLE 5-28. Prior SBIR/STTR or STTR Phase II Awards Related to the Surveyed Project.

TABLE 5-28

Prior SBIR/STTR or STTR Phase II Awards Related to the Surveyed Project.

Long-Term Effects on the Recipient Company

Although SBIR/STTR awards have direct effects on specific projects, they can also have a longer term effect on the trajectory of company development. The survey asked respondents about the impacts directly. The results are summarized in Table 5-29.

TABLE 5-29. Long-Term Effects on Recipient Companies.

TABLE 5-29

Long-Term Effects on Recipient Companies.

These results indicate an overwhelmingly positive impact. Overall, 97 percent of SBIR/STTR respondents reported a positive effect, and 62 percent reported a transformative effect. Two respondents out of 570 reported negative effects.

Respondents were also asked to describe these effects in their own words. Key aspects of their comments are reported below, focused on the ways in which SBIR/STTR and STTR made a profound difference to the company in the long term.

Key Aspects of SBIR-Driven Transformation

It is not easy to summarize the numerous ways in which SBIR/STTR awards from NIH helped to transform recipient companies. What follows is therefore a limited list of impacts.

  • Supported company formation
  • Provided first dollars
  • Funded areas where venture capital and other funders were not interested
  • Supported development of critical company infrastructure
  • Opened doors to potential partners
  • Helped address niche markets too small for major players/funders
  • Funded technology development
  • Enabled projects with high levels of technical risk and high potential return
  • Supported adaptation of technologies to new uses, markets, and industry sectors
  • Funded development of core technology
  • Diversified expertise, allowed hiring of specialists
  • Gave companies immediate credibility
  • Funded researchers to enter business full time
  • Transformed company culture to become more market driven
  • Created new companies and kept companies in business (that would not exist without SBIR/STTR funding)
  • Helped increase the company's knowledge base applied to later projects
  • Expanded the scope and scale of R&D capabilities
  • Supported technology development that led to spinoff companies

From these responses it is clear that small innovative companies are highly sensitive to the impact of outside factors. The sudden withdrawal of a sponsor can crush a company; a single contract can provide funding for 2 or 3 years of growth. Above all, these small companies are highly path dependent: what happens to them at a given moment can dramatically affect long-term outcomes.

In the end, SBIR/STTR can be viewed in many cases as a positive outside factor: one that provides funding, validation, and often market access not otherwise available. Even though it seems tenuous to link one award to the eventual success of a large corporation, that is, in fact, how some very small companies grow into large ones. The evidence from survey respondents suggests that this positive jolt is not an uncommon effect of these awards.

Other Company-Level Information

The survey asked about other potentially significant aspects of the company. Previous analyses of SBIR/STTR did not address a potentially important intervening variable: industry sector. It is quite possible that commercialization outcomes may be affected by the average cycle time of product development in different sectors. For example, product cycle time is much shorter in software than in materials or medical devices. Overall, 92.3 percent of NIH SBIR/STTR survey recipients worked in the medical technology sector. Table 5-30 shows the distribution of responses by phase and sector.

TABLE 5-30. Distribution of Responses by Sector Phase.

TABLE 5-30

Distribution of Responses by Sector Phase.

This question was designed to provide an approximate map of activities by sector. There is considerable overlap between some categories, and respondents would have substantial leeway to define sectors differently, so these finding should be viewed as highly preliminary. A few key points emerge:

  • Almost all awards were to companies primarily working in medical technology.
  • About onethird of respondent companies were working in medical devices and biotechnology.
  • About onequarter were working on research tools.
  • A further 17 percent were working in pharmaceuticals, with the remainder spread across medical information technology and education.
  • Responses show that there are no substantial differences between SBIR and STTR respondents.

COUNTERFACTUALS

It is always difficult to tightly determine the impact of a given SBIR or STTR award. Many factors affect the success and failure of companies and projects, and it is difficult to determine whether a specific factor was a necessary condition for success. Worse still, the large number of factors and the multiple paths to success and failure mean that it is unusual to be able to state with confidence that a particular intervention—in this case an SBIR or STTR award—constitutes a sufficient condition for a project's success.

Still, it is worth considering what would have occurred absent SBIR or STTR funding from the perspective of those most likely to have detailed knowledge and understanding of their particular projects: the principal investigators. Accordingly, the 2014 Survey asked a series of questions focused on the likely effect of the absence of SBIR or STTR funding. Of course, asking recipients about the impact of funding raises possible conflicts of interest, so results should be interpreted with some caution. However, these awards are some years in the past now, and many recipients no longer apply for SBIR/STTR funding for a variety of reasons.

Project Go-Ahead Absent SBIR/STTR Funding

One approach has been to ask recipients for their own views on the impact of the program on their project or company. In particular, the survey asked Phase II recipients whether the project would have been undertaken absent SBIR/STTR funding and whether the scope and timing would have been affected. Responses are summarized in Table 5-31. About 7 percent of Phase II respondents indicated that the project probably or definitely would have proceeded without program funding. In contrast, almost 75 percent thought the project probably or definitely would not have proceeded absent program funding: 34 percent were definite and 41 percent thought it unlikely.

TABLE 5-31. Project Undertaken in the Absence of this SBIR/STTR Award.

TABLE 5-31

Project Undertaken in the Absence of this SBIR/STTR Award.

These data have interesting wider implications for debates about early-stage funding: they suggest a weakness in the “crowding out” hypothesis, because more than 70 percent of respondents (presumably those with the closest knowledge of funding prospects for the project) believed it unlikely that alternative funding would be found.

The small number of respondents (12) who believed the project might have proceeded without SBIR/STTR funding were asked additional questions about the impact on project scope, duration, and timelines. They responded as follows:

  • Project scope would have been narrower (67 percent)
  • Project would have been substantially delayed (75 percent)
  • Project would have taken longer (75 percent)
  • Project would not have hit necessary milestones (75 percent)

Overall, these views indicate that SBIR/STTR funding was important not only for the go/no-go decision but also for the eventual shape and indeed likely impact of the project. Delay in bringing projects to conclusion—and hence to the point of potential market entry—can have a disastrous effect, because the window for market entry can be a narrow one.

Footnotes

1

The participation of women and minorities is discussed in Chapter 6, and although some data are presented here on SBIR/STTR and agency mission, those are discussed further in Chapter 2.

2

See Appendix A for a detailed description of the survey methodology used in this report.

3

National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Washington, DC: The National Academies Press, 2009.

4

Effective July 1, 2015, the institution is called the National Academies of Sciences, Engineering, and Medicine. References in this report to the National Research Council or NRC are used in an historic context identifying programs prior to July 1.

5

National Research Council, An Assessment of the SBIR Program at the National Institutes of Health.

6

See National Research Council, An Assessment of the SBIR Program, Washington, DC: The National Academies Press, 2008.

7

2014 Survey, Question 30. N=572.

8

Jon Hess, “Clinical Operations: Accelerating Trials, Allocating Resources and Measuring Performance,” Cutting Edge Information, October 12, 2014.

9

Avik S. Roy, “Stifling New Cures: The True Cost of Lengthy Clinical Drug Trials,” FDP Project Report 5, Manhattan Institute, April 2012.

10

2014 Survey, Question 40. N=584.

11

2014 Survey, Question 45. N=79.

12

2014 Survey, Question 46. N=21.

13

2014 Survey, Question 39.1.2. N=186.

14

2014 Survey, Question 24. See Table 5-31 in the chapter annex.

15

2014 Survey, Question 11. N=436 (companies). See Table 5-14.

16

2014 Survey, Question 10. N=421 (companies). See Table 5-25.

17

2014 Survey, Question 9. N=409 (companies). See Table 5-26.

18

SBA Section 1.(c), SBIR/STTR Policy Directive, October 18, 2012, p. 3.

19

The committee previously sought to develop statistical comparisons with similar companies in similar sectors at similar stages of development, but these efforts were eventually abandoned as unworkable. See Appendix A for a discussion of this effort. A full description of the methodology employed for this survey and the resulting analysis is also provided in Appendix A.

20

All comparisons to the 2005 survey are based on data from tables and analysis in National Research Council, An Assessment of the Small Business Innovation Research Program at the National Institutes of Health, Washington, DC: The National Academies Press, 2009, Chapter 4 and Appendix B.

21

Phase IIB is an award made to some firms at the end of Phase II. It supports those working to complete clinical trials. A discussion of Phase IIB is included in Chapter 3 (Program Initiatives).

22

For an overview of the commercialization metrics and survey used in this study, see Appendix A.

23

Similar warnings can be found in the 2009 report on the NIH SBIR program—National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 2009, 81.

24

National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 249.

25

National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 251.

26

Of related interest, see Link and Scott, Employment Growth from Public Support of Innovation in Small Firms (W.E. Upjohn Institute for Employment Research, 2012). The authors use the 2005 data and compare actual employment growth with a prediction of what would have been expected to happen without the SBIR award. Interestingly, comparing the counterfactual gains from the SBIR program across agencies, they find that the gap between the actual employment and the counterfactual predicted employment (if the SBIR award had not been received) is greatest for the NIH awards.

27

See National Research Council, An Assessment of the SBIR Program, Washington, DC: The National Academies Press, 2008.

28

2014 Survey, Question 30. N=572.

29

Hess, “Clinical Operations.”

30

Roy, “Stifling New Cures.”

31

2014 Survey, Question 49. N=570.

32

2014 Survey, Question 13.

33

See Figure 3-9.

34

It is important to bear in mind that sales could continue to accumulate for many years to come, i.e., the problem may be in the assessment time frame rather than the actual return on investment.

35

The values of these knowledge repositories vary. Any unique item, painting, photo, or music score can be copy-written for a modest fee. Trademarks include more processing, as registered trademarks need to be unique in their field so as not to impinge on another prior trademark's domain. A patent can be valuable IP, and patents have been correlated with prosperity. Refereed journal articles as a metric are not as highly valued outside of academia as inside, although company executives state in meetings that publications help to attract and keep high-quality staff and also provide additional validation for—and publicity about—their technology.

36

National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 265.

37

2014 Survey, Question 38.2 and Question 38.3.

38

National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Appendix D.

Copyright 2015 by the National Academy of Sciences. All rights reserved.
Bookshelf ID: NBK338147

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (3.4M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...