NCBI Secure Website Tests
What is happening?
To improve security and privacy, and by Federal government mandate, NCBI is moving all of its web sites and services, including web APIs, to HTTPS only by November 9, 2016. This change will provide you with greatly increased privacy and security on the NCBI site.
Note: The vast majority of NCBI web pages used by Web browsers has already been successfully moved to HTTPS.
If you only access NCBI resources with a Web browser, this test should not affect you, and you need not read any further.
This document is intended only for users who have or maintain software that works with NCBIs Web APIs.
If you have problems with purchased software that fails during the test period, and recovers immediately after the test completes, contact the software vendor for an update.
NCBI met its original deadline to move web pages to HTTPS by September 30, 2016. To allow software vendors and other users of NCBI Web APIs more time to update their programs, NCBI has extended the deadline for non-interactive (API) requests to November 9, 2016.
To prepare for this change, NCBI will be running a series of tests, where we move most web traffic to https for a short period, to check for problems and to ensure that all resources work properly with HTTPS.
When are the tests?
The next tests are scheduled for:
- Wednesday, November 2, from 1:00 PM - 4:00 PM, Eastern Daylight Time (17:00 - 20:00 UTC).
- Friday, November 4, from 10:00AM - 2:00 PM, Eastern Daylight Time (14:00 - 18:00 UTC).
Scheduling of subsequent tests will depend on the outcome of the current test.
What is being tested?
This test is to allow users of NCBI Web APIs a chance to ensure that their automated programs, batch jobs, and non-browser desktop client software all communicate correctly with NCBI using HTTPS. Any brief failures that occur as a result of the test will alert users that they need to contact their software vendors, or upgrade code they have written themselves.
Of special note, we will be redirecting most requests from http to https for the http versions of NCBI ftp sites, specifically, these hosts:
We will also be redirecting much of the traffic to our site from programs that use the NCBI C and C++ toolkits. These are typically C and C++ programs written and distributed by NCBI or third-party software vendors.
Certain programs, including some commercial software, are excluded from these tests, since it is known that their most recent versions do not yet correctly make requests over https.
How will the tests affect me?
Hopefully, your application will continue to operate properly during the test. You may want to test your programs during the test period to confirm that they work properly.
If your program fails during the test period, you may see these common problems:
- File downloads fail. File downloads over http may fail, especially if the host is one
of our ftp servers (see above). Update your scripts and or links to use the
https://versions of download URLs.
- NCBI-provided or commercial software packages may fail. If you are using an installable software package downloaded from NCBI servers, or a package you acquired from a third-party software vendor, please check to be sure you are using the latest version of the software. Many vendors have new releases of their software that fix problems related to https.
- The program fails with "error 307" or "error 301". The program is failing because our server is telling your program to access a resource using http, and the server is returning a redirect code and a new URL. Browser routinely follow such redirects, but other programs often do not. Some programming platforms, notably Java, do not follow HTTP redirects from HTTP to HTTPS. To fix the problem, you must modify your program so that it accesses NCBI resources with HTTPS URLs instead of HTTP URLs. This also implies that your programming platform supports TLS.
- The program fails with "error 403". This error message means that the server rejected the request, probably HTTP POST. POST requests are not reliably redirected from HTTP to HTTPS, even by browsers, so NCBI simply disallows HTTP POST for http:// URLs. Again, the solution is to change all URL requests from http:// to https://.
- The program fails because XML does not validate. XML files often contain a URL that indicates where to find the DTD to validate the XML content. If your program does not currently support https, it may not be able to retrieve the DTD, and therefore document validation fails. The solution is to either update your program so that can validate using DTDs loaded from https:// URLs; or use an XML catalog to rewrite the DTD URLs to an accessible place.
- Your program fails because it doesn't support HTTPS. In this case, you must either update your platform to support HTTPS; or perhaps set up a reverse proxy to bridge traffic from http at your site, to https at NCBI.
If I encounter a problem, what should I do?
Many of the events described above are all logged as errors by our servers. Others are not logged, because we can't detect the failure of the client program. Whether to report the problem to NCBI depends on where you got the software:
- Always first check to see if you are using the most recent version available of the program that failed.
- If a download failed from an NCBI ftp server, be sure the URL you used for the download starts with
- If a problem occurs in software that you purchased, or is open-source, contact the vendor or maintainer and describe your problem.
- If a problem is caused by software you wrote, update your program to use HTTPS URLs instead of HTTP URLs. See our online guide for advice how how to do so.
- If the problem is software provided by NCBI, report it to email@example.com.
Will my scripts and programs run properly during the test?
If you have scripts or programs that access NCBI web services, such as eutilities and BLAST URL-API, your service may or may not work correctly during the test period. If you have doubts, you may want to try running your scripts or starting your program during the test period to see whether it operates correctly, or needs to be updated. (There's no need to wait for the test--we have set up test servers that you can use to run those tests with your scripts and programs at any time.) See HTTPS at NCBI: Guidance for users of NCBI Web APIs for advice on how to update programs and scripts to use HTTPS.
Commercial or open-source desktop client software that accesses PubMed, BLAST, or other NCBI resources; or commercial or open-source web tools like proxy servers and browser extensions, may fail during the test period. In this case, please contact the software author or vendor directly. You may want to send them the link to HTTPS at NCBI: Guidance for users of NCBI Web APIs.
Will my scripts and programs run properly after the permanent switch to https?
Again, it depends. For more detailed advice about upgrading your programs to work with https, see HTTPS at NCBI: Guidance for users of NCBI Web APIs.